Skip to main content
SpringerLink
Log in

Assessment of Cognitive skills via Human-robot Interaction and Cloud Computing

  • Published:
Journal of Bionic Engineering Aims and scope Submit manuscript

Abstract

Technological advances are increasing the range of applications for artificial intelligence, especially through its embodiment within humanoid robotics platforms. This promotes the development of novel systems for automated screening of neurological conditions to assist the clinical practitioners in the detection of early signs of mild cognitive impairments. This article presents the implementation and the experimental validation of the first robotic system for cognitive assessment, based on one of the most popular platforms for social robotics, Softbank “Pepper”, which administers and records a set of multi-modal interactive tasks to engage the user cognitive abilities. The robot intelligence is programmed using the state-of-the-art IBM Watson AI Cloud services, which provide the necessary capabilities for improving the social interaction and scoring the tests. The system has been tested by healthy adults (N = 35) and we found a significant correlation between the automated scoring and one of the most widely used Paper-and-Pencil tests. We conclude that the system can be considered as a screening instrument for cognitive assessment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Feil-Seifer D, Mataric M J. Defining socially assistive robotics. Proceedigns of the 9th International Conference on Rehabilitation Robotics, Chicago, IL, USA, 2005, 465–468.

  2. Tapus A, Mataric M J, Scasselati B. Socially assistive robotics [Grand Challenges of Robotics]. IEEE Robotics & Automation Magazine, 2007, 14, 35–42.

    Article  Google Scholar 

  3. Mataric M J, Scassellati B. Socially assistive robotics. In Siciliano B, Khatib O eds., Springer Handbook of Robotics, Cham, 2016, 1973–1994.

  4. Furht B, Escalante A. Handbook of Cloud Computing, Springer, New York, NY, USA, 2010.

    Book  MATH  Google Scholar 

  5. Hu G, Tay W P, Wen Y. Cloud robotics: Architecture, challenges and applications. IEEE Network, 2012, 26, 21–28.

    Article  Google Scholar 

  6. Kehoe B, Patil S, Abbeel P, Goldberg K. A survey of research on cloud robotics and automation. IEEE Transactions on Automation Science and Engineering, 2015, 12, 398–409.

    Article  Google Scholar 

  7. Novoa J, Wuth J, Escudero J P, Fredes J, Mahu R, Yoma N B. DNN-HMM based automatic speech recognition for HRI scenarios. Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, Chicago, IL, USA, 2018, 150–159.

  8. Di Nuovo A, Broz F, Wang N, Belpaeme T, Cangelosi A, Jones R, Esposito R, Cavallo F, Dario P. The multi-modal interface of Robot-Era multi-robot services tailored for the elderly. Intelligent Service Robotics, 2018, 11, 109–126.

    Article  Google Scholar 

  9. Wang N, Di Nuovo A, Cangelosi A, Jones R. Temporal patterns in multi-modal social interaction between elderly users and service robot. Interaction Studies, 2019, 20, 1–9.

    Google Scholar 

  10. Di Nuovo A, Broz F, Cavallo F, Dario P. New frontiers of service robotics for active and healthy ageing. International Journal of Social Robotics, 2016, 8, 353–354.

    Article  Google Scholar 

  11. Cavallo F, Esposito R, Limosani R, Manzi A, Bevilacqua R, Felici E, Di Nuovo A, Cangelosi A, Lattanzio F, Dario P. Robotic services acceptance in smart environments with older adults: User satisfaction and acceptability study. Journal of Medical Internet Research, 2018, 20, 264.

    Google Scholar 

  12. Iroju O, Ojerinde O, Ikono R. State of the art: A study of human-robot interaction in healthcare. International Journal of Information Engineering and Electronic Business, 2017, 3, 43–55.

    Google Scholar 

  13. Conti D, Di Nuovo S, Trabia G, Buono S, Di Nuovo A. Use of robotics to stimulate imitation in children with autism spectrum disorder: A pilot study in a clinical setting. Proceedings of the 24th IEEE International Symposium on Robot and Human Interactive Communication, ROMAN, Kobe, Japan, 2015, 1–6.

  14. Conti D, Di Nuovo S, Buono S, Di Nuovo A. Robots in education and care of children with developmental disabilities: A study on acceptance by experienced and future professionals. International Journal of Social Robotics, 2017, 9, 51–62.

    Article  Google Scholar 

  15. Conti D, Cirasa C, Di Nuovo S, Di Nuovo A. “Robot, tell me a tale!”: A social robot as tool for teachers in kindergarten. Interaction Studies, 2019, 20, 1–16.

    Google Scholar 

  16. Rabbitt S M, Kazdin A E, Scassellati B. Integrating socially assistive robotics into mental healthcare interventions: Applications and recommendations for expanded use. Clinical Psychology Review, 2015, 35, 35–16.

    Article  Google Scholar 

  17. Boumans R, van Meulen F, Hindriks K, Neerincx M, Olde Rikkert M. Proof of concept of a cocial robot for patient reported outcome measurements in elderly persons. Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, Chicago, IL, USA, 2018, 73–74.

  18. Conti D, Trubia G, Buono S, Di Nuovo S, Di Nuovo A. Evaluation of a robot-assisted therapy for children with autism and intellectual disability. Proceedings of 19th Annual Conference on Towards Autonomous Robotic Systems, Bristol, UK, 2018, 405–415.

  19. Di Nuovo A, Conti D, Trubia G, Buono S, Di Nuovo S. Deep learning systems for estimating visual attention in robot-assisted therapy of children with autism and intellectual disability. Robotics, 2018, 7, https://doi.org/10.3390/robotics7020025.

  20. Petric F, Miklic D, Kovacic Z. Robot-assisted autism spectrum disorder diagnostics using POMDPs. Proceedings of Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction, Vienna, Austria, 2017, 369–370.

  21. Wijayasinghe I B, Ranatunga I, Balakrishnan N, Bugnariu, N, Popa D O. Human-robot gesture analysis for objective assessment of autism spectrum disorder. International Journal of Social Robotics, 2016, 8, 695–707.

    Article  Google Scholar 

  22. Kojima H, Takaeda K, Nihel M, Sadohara K, Ohnaka S, Inoue T. Acquisition and evaluation of a human-robot elderly spoken dialog corpus for developing computerized cognitive assessment systems. Journal of the Acoustical Society of America, 2016, 140, 2963–2963.

    Article  Google Scholar 

  23. Varrasi S, Di Nuovo S, Conti D, Di Nuovo A. A social robot for cognitive assessment. Proceedings of Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, Chicago, IL, USA, 2018, 269–270.

  24. Feingold Polak R, Elishay A, Shahar Y, Stein M, Edan Y, Levy-Tzedek S. Differences between young and old users when interacting with a humanoid robot: A qualitative usability study. Proceedings of Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, Chicago, IL, USA, 2018, 107–108.

  25. Scassellati B, Admoni H, Mataric M. Robots for use in autism research. Annual Review of Biomedical Engineering, 2012, 14, 275–294.

    Article  Google Scholar 

  26. Desideri L, Ottaviani C, Malavasi M, di Marzio R, Bonifacci, P. Emotional processes in human-robot interaction during brief cognitive testing. Computers in Human Behavior, 2019, 90, 331–342.

    Article  Google Scholar 

  27. Rossi S, Santangelo G, Staffa M, Varrasi S, Conti D, Di Nuovo A. Psychometric evaluation supported by a social robot: Personality factors and technology acceptance. Proceedings of the 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Nanjing, China, 2018, 802–807.

  28. Petersen R C. Mild cognitive impairment as a diagnostic entity. Journal of Internal Medicine, 2004, 256, 183–194.

    Article  Google Scholar 

  29. Luis C, Loewenstein D, Acevedo A, Barker W W, Duara R. Mild cognitive impairment: Directions for future research. Neurology, 2003, 61, 438–444.

    Article  Google Scholar 

  30. Landau S M, Harvey D, Madison C M, Reiman E M, Foster N L, Aisen P S, Petersen R C, Shaw L M, Trojanowski J Q, Jack C R, Weiner M W, Jagust W J. Comparing predictors of conversion and decline in mild cognitive impairment. Neurology, 2010, 75, 230–238.

    Article  Google Scholar 

  31. Di Nuovo S, De La Cruz V M, Conti D, Buono S, Di Nuovo A. Mental imagery: Rehabilitation through simulation. Life Span and Disability, 2014, 17, 89–118.

    Google Scholar 

  32. Caraci F, Castellano S, Salomone S, Drago F, Bosco P, Di Nuovo S. Searching for disease-modifying drugs in AD: Can we combine neuropsychological tools with biological markers? CNS & Neurological Disorders — Drug Targets, 2014, 13, 173–186.

    Article  Google Scholar 

  33. Fisekovic S, Memic A, Pasalic A. Correlation between MoCA and MMSE for the assessment of cognition in schizophrenia. Acta Informatica Medica, 2012, 20, 186–189.

    Article  Google Scholar 

  34. Moirand R, Galvao F, Lecompte M, Poulet E, Haesebaert F, Brunelin J. Usefulness of the Montreal Cognitive Assessment (MoCA) to monitor cognitive impairments in depressed patients receiving electroconvulsive therapy. Psychiatry Research, 2018, 259, 476–481.

    Article  Google Scholar 

  35. Pandey A K, Gelin R A mass-produced sociable humanoid robot: Pepper: The first machine of its kind. IEEE Robotics & Automation Magazine, 2018, 25, 40–48.

    Article  Google Scholar 

  36. Varrasi S, Di Nuovo S, Conti D, Di Nuovo A. Social robots as psychometric tools for cognitive assessment: A pilot test. Proceedings of 10th International Workshop on Human Friendly Robotics, Naples, Italy, 2019, 99–112.

  37. Varrasi S, Lucas A, Soranzo A, McNamara J, Di Nuovo A. IBM cloud services enhance automatic cognitive assessment via human-robot interaction. In Carbone G, Ceccarelli M, Pisla D eds., New Trends in Medical and Service Robotics, Cassino, Italy, 2019, 169–176.

  38. Nasreddine Z S, Phillips N A, Bédirian V, Charbonneau S, Whitehead V, Collin I, Cummings J L, Chertkow H. The montreal cognitive assessment, MoCA: A brief screening tool for mild cognitive impairment. Journal of the American Geriatrics Society, 2005, 53, 695–699.

    Article  Google Scholar 

  39. Nasreddine Z. MoCA Montreal Cognitive Assessment, [2018-10-18], https://doi.org/www.mocatest.org

  40. Olson R A, Chhanabhai T, McKenzie M. Feasibility study of the Montreal Cognitive Assessment (MoCA) in patients with brain metastases. Supportive Care in Cancer, 2008, 16, 1273–1278.

    Article  Google Scholar 

  41. Wong A, Xiong Y Y, Kwan P W L, Chan A Y Y, Lam W W M, Wang K, Chu W C W, Nyenhuis D L, Nasreddine Z, Wong L K S, Mok V C T. The validity, reliability and clinical utility of the Hong Kong Montreal Cognitive Assessment (HK-MoCA) in patients with cerebral small vessel disease. Dementia and Geriatric Cognitive Disorders, 2009, 28, 81–87.

    Article  Google Scholar 

  42. Dalrymple-Alford J C, MacAskill M R, Nakas C T, Livingston L, Graham C, Crucian G P, Melzer T R, Kirwan, J, Keenan R, Wells S, Porter R J, Watts R, Anderson T J. The MoCA Well-suited screen for cognitive impairment in Parkinson disease. Neurology, 2010, 75, 1717–1725.

    Article  Google Scholar 

  43. Videnovic A, Bernard B, Fan W, Jaglin J, Leurgans S, Shannon K M. The Montreal Cognitive Assessment as a screening tool for cognitive dysfunction in Huntington’s disease. Movement Disorders, 2010, 25, 401–404.

    Article  Google Scholar 

  44. Bertrand J-A, Génier Marchand D, Postuma R B, Gagnon J-F. Cognitive dysfunction in rapid eye movement sleep behavior disorder. Sleep and Biological Rhythms, 2013, 11, 21–26.

    Article  Google Scholar 

  45. Carson N, Leach L, Murphy KJ. A re-examination of Montreal Cognitive Assessment (MoCA) cutoff scores. International Journal of Geriatric Psychiatry, 2017, 33, 379–388.

    Article  Google Scholar 

  46. Chertkow H, Nasreddine Z S, Johns E, Phillips N A, McHenry C. The Montreal Cognitive Assessment (MoCA): Validation of alternate forms and new recommendations for education corrections. Alzheimer’s and Dementia, 2011, 7, S157.

    Article  Google Scholar 

  47. Xu T (Linger), Zhang H, Yu C. See you see me: The role of eye contact in multimodal human-robot Interaction. ACM Transactions on Interactive Intelligent Systems, 2016, 6, 1–22.

    Article  Google Scholar 

  48. Sciutti A, Rea F, Sandini G. When you are young, (robot&#c2019;s) looks matter. Developmental changes in the desired properties of a robot friend. Proceedigns of the 23rd IEEE International Symposium on Robot and Human Interactive Communication (IEEE RO-MAN), Edinburgh, Scotland, UK, 2014, 567–573.

  49. Streiner D L. Starting at the beginning: An introduction to coefficient alpha and internal consistency. Journal of Personality Assessment, 2003, 80, 99–103.

    Article  Google Scholar 

  50. Cohen J. Statistical Power Analysis for the Behavioral Sciences, Lawrence Erlbaum Associates, Hillsdale, New Jersey, USA, 1988.

    MATH  Google Scholar 

  51. Landau S, Everitt B S. A Handbook of Statistical Analyses Using SPSS, CRC Press, Boca Raton, FL, USA, 2004.

    Google Scholar 

  52. Field A. Discovering Statistics Using IBM SPSS Statistics, SAGE Publications, Los Angeles, CA, USA, 2013.

    Google Scholar 

  53. Lakens D, Scheel A M, Isager P M. Equivalence testing for psychological research: A tutorial. Advances in Methods and Practices in Psychological Science, 2018, 1, 259–269.

    Article  Google Scholar 

  54. Purpura J E. An analysis of the relationships between Ttest takers’ cognitive and metacognitive strategy use and second language test performance. Language Learning, 1997, 47, 289–325.

    Article  Google Scholar 

Download references

Acknowledgment

The authors gratefully thank all university staff and students who participated in this study. This work won the IBM Shared University Research (SUR) Award. The work of Daniela Conti and Alessandro Di Nuovo was supported by the European Union’s H2020 research and innovation program under the MSCA-Individual Fellowship grant agreement no. 703489 (CARER-AID). Alessandro Di Nuovo was also partially supported by the EPSRC through project grant EP/P030033/1 (NUMBERS).

Author information

Authors and Affiliations

  1. Sheffield Hallam University, Sheffield, S1 1WB, United Kingdom

    Alessandro Di Nuovo, Simone Varrasi, Alexandr Lucas, Daniela Conti & Alessandro Soranzo

  2. IBM Hursley Lab, Winchester, SO21 2JN, United Kingdom

    John McNamara

Authors
  1. Alessandro Di Nuovo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Simone Varrasi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexandr Lucas
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daniela Conti
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. John McNamara
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Alessandro Soranzo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alessandro Di Nuovo.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Di Nuovo, A., Varrasi, S., Lucas, A. et al. Assessment of Cognitive skills via Human-robot Interaction and Cloud Computing. J Bionic Eng 16, 526–539 (2019). https://doi.org/10.1007/s42235-019-0043-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42235-019-0043-2

Keywords

Search

Navigation