PILLAR-Robots aims at developing a new generation of robots endowed with a higher level of autonomy, that are able to determine their own goals and establish their own strategies, creatively building on the experience acquired during their lifetime to fulfil the desires of their human designers/users in real-life application use-cases brought to TRL5. To this end, the project will operationalize the concept of Purpose, drawn from the cognitive sciences, to increase the autonomy and domain independence of robots during autonomous learning and, at the same time, to lead them to acquire knowledge and skills that are actually relevant for operating in target real applications. In particular, the project will develop algorithms for the acquisition of purpose by the robot, ways to bias the perceptual, motivational and decision systems of the robots’ cognitive architectures towards purposes, and strategies for learning representations, skills and models that allow the execution of purpose-related deliberative and reactive decision processes. Given the aim of reaching TRL5, PILLAR-Robots will implement and validate demonstrators of purposeful lifelong open-ended autonomy using the resulting Purposeful Intrinsically Motivated Cognitive Architecture within three different application fields characterized by different types and levels of variability: Agri-food, Edutainment, and unstructured Industrial/retail. PILLAR-Robots will perform a complete evaluation of the possibilities and impacts of purposeful lifelong open-ended autonomy in these realms from an operational perspective, but also from a market-oriented (with significant productivity gains) and societal (socio-economic, ethical and regulatory) perspective. Engagement of industry and SME players is also expected in order to prepare the ground for further large-scale demonstration.

Imagine a scenario where multiple robots have been deployed to provide services such as object handling/transportation, or pickup and delivery operations. In such a context, different robots with varying capabilities must be coordinated in order to achieve various multi-tasking procedures. Thus, the effective supervision and coordination of the overall heterogeneous system mandates a decentralized framework that integrates high-level task-planning, low-level motion control and robust, real-time sensing of the robots’ dynamic environment. Current practice is at a great deal based on offline, centralized planning and related tasks are usually fulfilled in a predefined manner: this does not utilize the capabilities of the system to operate efficiently in a dynamic environment. In most cases, sudden changes in the environment, the type of tasks, and the need for coordination, would cause the system to halt, ask for human intervention and restart. Despite the fact that public facilities are in some degree prestructured, the need for a framework for decentralized, real-time, automated task (re)-planning is evident in a twofold manner: (i) it will pave the way to an improved use of resources and a faster accomplishment of tasks inside public facilities and workspaces with high social activity; (ii) it will make an important contribution towards the vision of more flexible multirobot applications in both professional or domestic environments, also in view of the “Industry 4.0” vision and the general need to deploy such systems in everyday life scenarios. Within Co4Robots our goal is to build a systematic methodology to accomplish complex specifications given to a team of potentially heterogeneous robots; control schemes appropriate for the mobility and manipulation capabilities of the considered robots; perceptual capabilities that enable robots to localize themselves and estimate the state of the dynamic environment; and their systematic integration approach.

The next generation of personal robotic systems needs to reach a level of cognition and motor intelligence that provides autonomy in any environment, effective interaction with humans, and adaptation of their actions to a broad range of open, dynamic situations. Robots are expected to be able to predict perceptual and functional changes that result from human actions and replicate human activities taking into consideration their own capabilities and limitations. The required human-like physical performance and reasoning cannot be achieved with the mainstream AI and robotics paradigms, because they are missing the required co-design of body (robot) and mind (AI) and are based on inefficient computing and sensing resources that cannot be scaled up to the required level. To go beyond what is currently possible, PRIMI will synergistically combine research and development in neurophysiology, psychology, machine intelligence, cognitive mechatronics, neuromorphic engineering, and humanoid robotics to build developmental models of higher-cognition abilities – mental imagery, abstract reasoning, and theory of mind – boosted by energy-efficient event-driven computing and sensing. It will produce a new unifying concept for the next generation of autonomous interaction technologies, capable of more autonomous, faster, safer, and precise interaction with real-time learning and adaptation, thanks to the integration of the capabilities to mentally represent themselves, the physical and social worlds, resemble experiences and simulate actions. PRIMI’s ambition is to induce a paradigm shift in AI and robotics to create truly autonomous socially interactive robots, which will offer new technological perspectives for transforming personal robotic services. As a proof-of-principle of the technological advancement in a relevant scenario, prototypes of neuromorphic humanoid robots will be validated in clinical pilot studies of robot-led physical rehabilitation of stroke survivors.

Around 3.3 million years ago our ancestors made the first tool. They imagined a new utensil and then knapped a stone until it became an efficient tool for cutting. Tool creation was an outstanding technological milestone for humanity providing us with unprecedented control over our environment. This ability required cognitive capabilities, such as prediction, metacognition, abstraction, and creativity—all of which are associated in humans with awareness. Current artificial intelligence systems and robots largely lack these capabilities and cannot even monitor and evaluate the consequence of their actions let alone develop new tools to address environmental challenges. METATOOL aims to provide a computational model of synthetic awareness to enhance adaptation and achieve tool invention. This will enable a robot to monitor and self-evaluate its performance, ground and reuse this information for adapting to new circumstances, and finally unlock the possibility of creating new tools. Under the predictive account of awareness, and based on both neuroscientific and archeological evidence, we will: 1) develop a novel computational model of metacognition based on predictive processing (metaprediction) and 2) validate its utility in real robots in two use case scenarios: conditional sequential tasks and tool creation. METATOOL will provide a blueprint for the next generation of artificial systems and robots that can perform adaptive, and anticipative, control with and without tools (improved technology), self-evaluation (novel explainable AI), and invent new tools (disruptive innovation). Tool-making and tool-invention are outstanding technological milestones in human history. A similar breakthrough can now be envisioned in engineering. We already have algorithms to enable machines to use tools and now it is time to develop robots that create tools.

Cognitive robots are augmenting their autonomy, enabling them to deployments in increasingly open-ended environments. This offers enormous possibilities for improvements in human economy and wellbeing. However, it also poses strong risks that are difficult to assess and control by humans. The trend towards increased autonomy conveys augmented problems concerning reliability, resilience, and trust for autonomous robots in open worlds. The essence of the problem can be traced to robots suffering from a lack of understanding of what is going on and a lack of awareness of their role in it. This is a problem that artificial intelligence approaches based on machine learning are not addressing well. Autonomous robots do not fully understand their open environments, their complex missions, their intricate realizations, and the unexpected events that affect their performance. An improvement in the capability to understand of autonomous robots is needed. This project tries to provide a solution to this need in the form of 1) a theory of understanding, 2) a theory of awareness, 3) reusable software assets to apply these theories in real robots, and 4) three demonstrations of its capability to a) augment resilience of drone teams, b) augment flexibility of manufacturing robots, and c) augment human alignment of social robots. In summary, we will develop a cognitive architecture for autonomous robots based on a formal concept of understanding, supporting value-oriented situation understanding and self-awareness to improve robot flexibility, resilience and explainability.