Robots in a rush: time-aware AI aids human-machine interaction

Time is an illusion, breakfast time doubly so. In future, human workers might not be alone in experiencing frenetic mornings thanks to EU-funded researchers who have equipped robots with human-like time perception capabilities – and tested them in the kitchen.
    
However, the results of the Future and Emerging Technologies TIMESTORM project funded by the EIC Pathfinder go far beyond food preparation. In creating robotic systems with the capacity to perceive and manage time much as humans do, the researchers laid the foundations for future generations of robots able to interact and collaborate in synchronisation with people. The innovation is driving ongoing research initiatives and supporting collaboration with robotics companies that could lead to novel commercial applications.

‘Over the last decade, there has been growing demand for service robots that can operate autonomously and interact seamlessly with humans, such as robotic housekeepers, care-givers, bartenders and museum guides to name just a few. However, although these robots exhibit rich behavioural repertoires, they either act asynchronously to human needs or assume the humans decide for them ‘which and when’ actions should be triggered,’ says TIMESTORM coordinator Panos Trahanias at the Institute of Computer Science - FORTH in Greece.

While today’s robots can count and track time very accurately using their internal clocks, the TIMESTORM team set out to imbue their prototype systems with a ‘sense of time’. This temporal perception is of the utmost importance in all human-to-human social interactions and collaborative activities.

The researchers began by exploring how humans perceive time through an extensive series of behavioural, developmental and neuroimaging experiments to define and model temporal perception. The results were used to produce a unified theoretical framework of how time perception works in humans, enabling the researchers to build computational models that could be applied to robotic cognitive systems.

Adjusting breakfast expectations
‘For example, when humans are in hurry, they want tasks to be done immediately even if the results are not ideal, but when they are more relaxed they have greater expectations in terms of quality. TIMESTORM’s technology takes into account human psychological states and behavioural performance to make predictions about human expectations and accordingly adjust robot activities to develop smooth and effective human-machine synergies,’ Trahanias explains.

This was tested in a realistic breakfast-preparation scenario in which two robots coordinate their actions, deliver fruit and collaborate with a human, synchronising and completing their tasks on time. The tests suggest that humans appreciate and consider much more natural and productive short and long-term collaboration with time-aware robots.

‘We’ve been able to demonstrate time-aware multi-agent robotic systems that are perfectly synchronised to the temporal goals of end-users and the constraints of the environment,’ Trahanias says. ‘More broadly, we have clearly demonstrated the proof of concept and the necessity of time perception for the execution of complex, collaborative tasks by humans and robots working together.’

Towards human-robot symbiosis
Longer term, time-perception capabilities will enable robots to plan actions by considering the temporal properties of tasks, predict and plan for future events, take decisions under time pressure, adapt actions based on past experiences, or even be aware of and exploit their spare time. These are all common human abilities that social robots deployed unconstrained in the real world would need to exhibit to blend into human societies.

The TIMESTORM consortium, funded by the EU’s Future and Emerging Technologies (FET) programme, has transformed the notion of time perception in artificial intelligence from an immature, poorly defined subject into a promising new research strand, drawing on diverse expertise in psychology and neurosciences as well as robotics and cognitive systems.

Currently, TIMESTORM results are supporting various ongoing national and international research activities, including the EU-funded project sustAGE that aims to extend and enhance the productivity and well-being of elderly workers. Coordinating partner FORTH has also set up a strategic long-term partnership with the Honda Research Institute Japan on the use of TIMESTORM technology, amid ongoing discussions and negotiations with robotics companies about potential commercial applications.

‘Although there are numerous aspects of time yet to be explored and further work is definitely needed to deliver mature and robust technology, we strongly believe that TIMESTORM can serve as the basis for natural, time-aware, human-robot symbiosis,’ the project coordinator says. ‘We estimate that it will be at least one decade before robots are routinely designed with the capacity to perceive time, but the market potential of these technologies is enormous.