Cognitive Robots and Systems Group

Objectives

Our primary research interests include knowledge representation and reasoning, cognitive systems, and interactive learning as applied to adaptive robots and agents. We seek to develop architectures and algorithms that enable robots and agents to collaborate with non-expert human participants, acquiring and using sensor inputs and high-level human feedback based on need and availability. Furthermore, we are interested in developing algorithms for estimation and prediction problems in non-robotics domains. We are part of the Institute of Perception, Action, and Behavior (IPAB) at the University of Edinburgh.

Lab members (humans and robots!)
Collaborators
Robotics projects
Non-robotics projects

Students interested in working with me should read this before you contact me.

Faculty Supervisor: Dr. Mohan Sridharan

Current (Graduate) Students (UoE): Current (Graduate) Students (UoB): Past Postdocs/Research Associates:
  • Dr. Martin Rudorfer, August 2020-September 2022.
  • Dr. Rocio Gomez, November 2016-August 2019.
  • Dr. Shiqi Zhang, August 2013-February 2014.
  • Tiago Mota, June 2020-February 2021.
  • Benjamin Meadows, June 2016-December 2019.
Past (Graduate) Students (UoB):
  • Laura Ferrante. PhD, December 2023. (pdf)
  • Saif Sidhik. PhD, May 2022 (pdf)
  • Michael Mathew. PhD, May 2020 (pdf)
  • Ermano Arruda. PhD, May 2020 (pdf)
  • Ian Temple. MRes (Natural Computation), May 2020 (pdf)
  • Maija Filipovica. MS, September 2019 (pdf)
Past (Graduate) Students (UoA):
  • Tiago Andrade Mota. PhD, December 2021 (pdf)
  • Heather Riley. MS (thesis), April 2019 (pdf)
Past (Graduate) Students (TTU):
Past (Undergraduate) Students:
  • Heather Riley. B.E. (Hons, UoA). Independent (+Part IV) Research Project, February-December 2017.
  • Prashanth Devarakonda. B.E. (Hons, UoA). Independent (+Part IV) Research Project, November 2014-March 2016.
  • Zenon Colaco. B.E. (Hons, UoA). Independent Research Project, November 2014-December 2015.
  • Rashmica Gupta. B.E. (Hons, UoA). Independent (+Part IV) Research Project, November 2014-September 2015.
  • Anna Mace. B.Sc. (UoA, Co-supervised with Dr. Pat Langley) Independent Research Project, February-June 2015.
  • Patricia Andrews, B.S. (Colorado College); Olatide Omojaro, B.S. (Georgia Perimeter College); Aaron Hester, B.S. (Mathematics+CS); REU Students, June-August 2013.
  • Emilie Featherston. B.S. (TTU, co-supervised with Drs. Susan and Joseph Urban), REU Student, June-August 2013.
  • Christian Washington, B.S. (Louisiana State University); Catie Meador, B.S. (Swarthmore College); REU Students, June-August 2012.
  • Sabyne Peeler, B.S. (Florida A&M University); Shiloh Huff, B.S. (TTU); Stephanie Graham, B.S. (TTU); co-supervised with Drs. Susan and Joseph Urban, REU Student, June-August 2012.
  • Austin Ray, David South, Kevin Thomas. B.S. (TTU, co-supervised with Drs. Susan and Joseph Urban), January-May 2012.
  • Jesse Kawell, B.S. (Samford University); David Kari, B.S. (California Baptist University); David Seibert, B.S. (Emory University); James Smith, B.S. (UT Austin); REU Students, June-August 2011.
  • Mary Shuman, B.S. (University of North Carolina), co-supervised with Dr. Susan Urban, REU Student, June-August 2011.
  • Matthew Sullivan. B.S. (TTU), January-June 2011.
  • Kshira Nadarajan. B.S. (Iowa State University), May-August 2010.
Some robot platforms used for experimental evaluation are shown below.

All robot platforms Laura with Boris Saif with Sawyer
Sawyer manipulating objects Franka and Sawyer Baxter robot manipulator

Some current (and recent) collaborators:

Some recent collaborators at different institutions are listed below.
  • Dr. Ales Leonardis, University of Birmingham, UK (Computer Science, Scene understanding, robotics).
  • Dr. Madhava Krishna, International Institute of Information Technology, Hyderabad, India (Computer Engineering, Robotics).
  • Dr. Michael Gelfond, Texas Tech University, USA (Computer Science, Knowledge representation)
  • Dr. Peter Stone, The University of Texas at Austin, USA (Computer Science, Robotics, multiagent collaboration).
  • Dr. Ranjini Swaminathan, University of Reading, UK (Meteorology, Climate informatics)
  • Dr. Jeremy Wyatt, The University of Birmingham, UK (Computer Science, Probabilistic reasoning)
  • Dr. Keith Jones, Texas Tech University, USA (Psychology, HRI & Human factors).
  • Dr. Partha Roop, The University of Auckland (ECSE, Machine learning applications).
  • Dr. Prasanna Gowda, US Department of Agriculture (Agricultural automation).

AI/Robotics Research Questions:

In the context of human-robot collaboration, we seek to answer the following key questions:
  1. How to enable robots to represent, reason, and act with different descriptions of incomplete domain knowledge and uncertainty, for reliable decision making?
  2. How to enable robots to interactively and cumulatively acquire previously unknown knowledge of domain dynamics from multimodal sensor cues and limited human feedback?
  3. How to enable robots to describe their decisions, beliefs, and experiences, and establish that their behavior satisfies desired properties?
Although many sophisticated algorithms have been developed for the associated knowledge representation, reasoning, and learning challenges, the integration of these challenges poses open problems even as it presents novel opportunities to address these challenges. We seek to develop integrated cognitive architectures and algorithms that exploit these inter-dependencies.

Integrated architecture Overview


Representative publications:
Mohan Sridharan. Cognitive Adequacy: Insights from Developing Robot Architectures. In the Workshop on Cognitive Aspects of Knowledge Representation (CAKR) at IJCAI, July 23, 2022. (pdf)

Mohan Sridharan. Integrated Commonsense Reasoning and Interactive Learning in Robotics. In the Workshop on Integrating Planning and Learning (WIPL) and the Workshop on Declarative and Neurosymbolic Representations in Robot Learning and Control (DNR-ROB) at Robotics: Science and Systems Conference (R:SS), July 12 and 15, 2021, Online. (pdf1, pdf2)

Mohan Sridharan. Refinement-Based Architecture for Knowledge Representation, Explainable Reasoning, and Interactive Learning in Robotics. In the Workshop on Combining Learning and Reasoning: Towards Human-Level Robot Intelligence (Learn-Reason) at the Robotics Science and Systems Conference (RSS), June 22, 2019, Freiburg, Germany. (pdf)

The individual components of the architecture have developed into the research projects described below.
  • Knowledge Representation and Reasoning: The objective is to develop architectures that represent and reason with tightly coupled transition diagrams of the domain at different resolutions, with the finer-resolution transition diagrams defined as a refinement of the coarser-resolution diagrams. The architectures support non-monotonic logical reasoning with commonsense domain knowledge, and probabilistic reasoning with the relevant parts of the transition diagram at the appropriate resolution. These architectures encode theories of intention, affordance, and observation, and provide relational descriptions of the robot's decisions, beliefs, and experiences. We evaluate these architectures with simulated agents collaborating with each other and on physical robots assisting humans in indoor domains (e.g., learned map).

    KRR architecture     KRR for robots


    Videos of reasoning and learning for ad hoc teamwork: video-1, video-2, video-3
    Video of experimental trials showing reasoning with intentional actions: video
    Videos of experimental trials showing planning and diagnosis: video-1, video-2, video-3
    Videos of some older experimental trials on different robots: video-1, video-2, video-3

    Representative publications:
    Hasra Dodampegama and Mohan Sridharan. Back to the Future: Toward a Hybrid Architecture for Ad Hoc Teamwork. In the AAAI Conference on Artificial Intelligence (AAAI), Washington DC, USA, February 2023. (pdf)

    • Rocio Gomez, Mohan Sridharan and Heather Riley. What do you really want to do? Towards a Theory of Intentions for Human-Robot Collaboration. In Annals of Mathematics and Artificial Intelligence, special issue on commonsense reasoning, 89(1): 179-208, February 2021. (pdf)

    Mohan Sridharan, Michael Gelfond, Shiqi Zhang and Jeremy Wyatt. REBA: Refinement-based Architecture for Knowledge Representation and Reasoning in Robotics. In Journal of Artificial Intelligence Research, 65:87-180, May 2019. (official version) (pdf)

    Mohan Sridharan and Ben Meadows. Towards a Theory of Explanations for Human-Robot Collaboration. In Künstliche Intelligenz Journal, 2019. (pdf)





  • Interactive Learning for Human-Robot/Multiagent Collaboration: The objective is to interactively and cumulatively: (a) acquire previously unknown domain knowledge by interacting with the domain and other agents (e.g., robots, humans); (b) learn associations between information extracted from different sensor cues; and (c) learn simple predictive models based on heuristic methods for human-agent/multiagent collaboration. Recent architectures have drawn on principles of non-monotonic logical reasoning, interactive learning (e.g., deep learning, reinforcement learning, multiple-instance learning) to: (1) incrementally discover axioms governing domain dynamics and learn action capabilities; (2) reason with commonsense knowledge to guide deep learning for scene understanding problems; and (4) learn associations between visual and verbal cues for object recognition and human-robot collaboration. These architectures have been evaluated in single and multiagent (simulated) domains.

    Explainable reasoning and deep learning      Ad hoc teamwork


    Representative publication:
    Hasra Dodampegama and Mohan Sridharan. Back to the Future: Toward a Hybrid Architecture for Ad Hoc Teamwork. In the AAAI Conference on Artificial Intelligence (AAAI), Washington DC, USA, February 2023. (pdf)

    • Mohan Sridharan and Tiago Mota. Towards Combining Commonsense Reasoning and Knowledge Acquisition to Guide Deep Learning. In Journal of Autonomous Agents and Multi-Agent Systems, 37(4), 2023. (official version), (pdf)

    Tiago Mota, Mohan Sridharan, and Ales Leonardis. Integrated Commonsense Reasoning and Deep Learning for Transparent Decision Making in Robotics. In Springer Nature Computer Science, 2(242), 2021. (official version) (pdf)

    Mohan Sridharan and Ben Meadows. Knowledge Representation and Interactive Learning of Domain Knowledge for Human-Robot Interaction. In Advances in Cognitive Systems Journal, 7:77-96, December 2018. (official version) (pdf)





  • Dexterous Manipulation and Control: Inspired by results in human/animal motor control, we develop architectures for dexterous grasping and manipulation. These architectures: (a) use learned (predictive) forward models and a hybrid force-motion controller to achieve variable impedance control in challenging continuous contact tasks; (b) use parametric and non-parametric models for grasp synthesis; and (c) enable joint human-robot collaborative control of prosthetic devices.

    Manipulation architecture      Changing Contact Manipulation
    Manipulation architecture      Variable Impedance Manipulation


    Representative publication:
    Mark Robson and Mohan Sridharan. A Keypoint-based Object Representation for Generating Task-specific Grasps. In the IEEE International Conference on Automation Science and Engineering (CASE), Mexico City, Mexico, August 20-24, 2022. (pdf)

    Saif Sidhik, Mohan Sridharan, and Dirk Ruiken. Towards a Framework for Changing-Contact Robot Manipulation. In the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic, September 27-October 1, 2021. (pdf)

    Saif Sidhik, Mohan Sridharan, and Dirk Ruiken. Learning Hybrid Models for Variable Impedance Control of Changing-Contact Manipulation Tasks. In the Annual Conference on Advances in Cognitive Systems (ACS), Palo Alto, USA, August 10-12, 2020. (pdf)

    Michael Mathew, Saif Sidhik, Mohan Sridharan, Morteza Azad, Akinobu Hayashi, and Jeremy Wyatt. Online Learning of Feed-Forward Models for Task-Space Variable Impedance Control. In the International Conference on Humanoid Robots (Humanoids), Toronto, Canada, October 15-17, 2019. (pdf)



Non-Robotics Projects:

We also address estimation and inference problems in other (non-robotics) application domains.
  • Intelligent Transportation: Traffic congestion results in significant monetary losses in countries around the world, and a significant amount of effort has been put into reducing congestion in cities. In many cities, it Intelligent transportation systems are being used to make best use of the available resources, because it is impractical to build new roads or to expand existing roads. A key components of such systems is short-term traffic prediction, which helps make decisions based on predictions of traffic in the near-future. Despite decades of research, there are still many open problems in short-term traffic prediction, e.g., building responsive algorithms that are able to predict non-recurring congestion, determining the optimum data-resolution, and identifying the important spatio-temporal dependencies in traffic data. In collaboration with colleagues at The University of Auckland (NZ), and the NZ Transportation Authority, we adapt sophisticated machine learning algorithms to address these problems, providing reliable efficient predictions of traffic flow on motorways and arterial roads.

    Representative publication:
    Rivindu Weerasekera, Mohan Sridharan, and Prakash Ranjitkar. Implications of Spatio-temporal Data Aggregation on Short-term Traffic Prediction using Machine Learning Algorithms. In Journal of Advanced Transportation, Vol. 2020, Article ID: 7057519, 21 pages, 2020. (pdf)



  • Climate Informatics: Climate change and climate forecasts influence policies and planning in fields such as agriculture, ecological preservation and resource management. Although sophisticated global climate models can predict large scale weather patterns in grids of approx. 100km x 100km, they cannot make accurate regional weather predictions since they do not account for local geographic variations (within the grids) such as mountains and lakes. Obtaining high-resolution regional predictions under a changing climate by downscaling global models presents formidable challenges: (a) processes contributing to global models are highly non-linear; (b) global models and their relationships with regional observations are non-stationary; (c) sensitivity to initial conditions; and (d) use of Petabyte-scale historical data to learn models that can make predictions. In collaboration with the Climate Science Center at TTU, we are developing architectures and algorithms to learn the relationships between global models and regional observations to make accurate regional predictions. We are also developing computational frameworks for modeling extreme climate events.

    Representative publication:
    Ranjini Swaminathan, Mohan Sridharan and Katharine Hayhoe. A Computational Framework for Modelling and Analyzing Ice Storms. Technical report on arXiv. May 2018. (arXiv link)

    Ranjini Swaminathan, Mohan Sridharan, Gill Dobbie and Katharine Hayhoe. Modeling Ice Storm Climatology. In the 28th Australasian Joint Conference on Artificial Intelligence (AusAI 2015), Canberra, Australia, November 30-December 4, 2015. (pdf)

    Ranjini Swaminathan, Mohan Sridharan and Katharine Hayhoe. Convolutional Neural Networks for Climate Downscaling. In the Climate Informatics Workshop (CI 2012), Boulder, USA, September 20-21, 2012. (pdf) (poster)


  • Agricultural irrigation management and yield mapping: Agricultural irrigation management poses tough challenges in arid and semi-arid regions, where crop water demand exceeds rainfall. Since daily grass or alfalfa reference ET values are widely used to estimate crop water demand, inaccurate reference ET estimates can impact irrigation costs and the demands on U.S. freshwater resources. ET networks calculate reference ET using accurate measurements of local meteorological data. With gaps in spatial coverage of existing agriculture-based ET networks (e.g., TXHPET) and lack of funding, there is an immediate need for alternative sources capable of filling data gaps without high maintenance and support costs. In collaboration with USDA-ARS and Texas A&M AgriLife Research, we adapt sophisticated machine learning algorithms that use weather observations from non-ET stations to accurately predict the reference ET values.

    Representative publications:
    Daniel Holman, Mohan Sridharan, Prasanna Gowda, Dana Porter, Thomas Marek, Terry Howell and Jerry Moorhead. Gaussian Process Models for Reference ET Estimation from Alternative Meteorological Data Sources. In the Journal of Hydrology, 517: 28-35, September 2014. (official version)

    Daniel Holman, Mohan Sridharan, Prasanna Gowda, Dana Porter, Thomas Marek, Terry Howell and Jerry Moorhead. Estimating Reference Evapotranspiration for Irrigation Scheduling in the Texas High Plains. In the International Joint Conference on Artificial Intelligence (IJCAI 2013), Beijing, China, August 3-9, 2013. (pdf)


JMP


Home

Research, Research Group

Teaching

Robot Outreach

CV, Publications