CoChat
Sign Up
Cited by 154Social Robot Interaction and HRIAutism Spectrum Disorder ResearchRobotics and Automated Systems

Deep learning-based cost-effective and responsive robot for autism treatment

Open access
Singh, Aditya, Raj, Kislay, Kumar, Teerath, Verma, Swapnil, Roy, Arunabha·2023·Drones
DOI 10.3390/drones7020081
Open this paper in CoChat — free accountOpen PDF

Abstract

Recent studies state that, for a person with autism spectrum disorder, learning and improvement is often seen in environments where technological tools are involved. A robot is an excellent tool to be used in therapy and teaching. It can transform teaching methods, not just in the classrooms but also in the in-house clinical practices. With the rapid advancement in deep learning techniques, robots became more capable of handling human behaviour. In this paper, we present a cost-efficient, socially designed robot called ‘Tinku’, developed to assist in teaching special needs children. ‘Tinku’ is low cost but is full of features and has the ability to produce human-like expressions. Its design is inspired by the widely accepted animated character ‘WALL-E’. Its capabilities include offline speech processing and computer vision—we used light object detection models, such as Yolo v3-tiny and single shot detector (SSD)—for obstacle avoidance, non-verbal communication, expressing emotions in an anthropomorphic way, etc. It uses an onboard deep learning technique to localize the objects in the scene and uses the information for semantic perception. We have developed several lessons for training using these features. A sample lesson about brushing is discussed to show the robot’s capabilities. Tinku is cute, and loaded with lots of features, and the management of all the processes is mind-blowing. It is developed in the supervision of clinical experts and its condition for application is taken care of. A small survey on the appearance is also discussed. More importantly, it is tested on small children for the acceptance of the technology and compatibility in terms of voice interaction. It helps autistic kids using state-of-the-art deep learning models. Autism Spectral disorders are being increasingly identified today’s world. The studies show that children are prone to interact with technology more comfortably than a with human instructor. To fulfil this demand, we presented a cost-effective solution in the form of a robot with some common lessons for the training of an autism-affected child.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/drones7020081

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Deep learning can be used to teach drones to detect and avoid obstacles in real-time.

    “Deep learning, for instance, may be used to teach drones how to detect and avoid obstacles in real-time [93] .”

    From Artificial intelligence, machine learning and deep learning in advanced robotics, a review (2023) · cites this paper as [93]

    Section 4.2 (Computer Vision; page unavailable) describes deep-learning object detection on a robot. Section 4.4 (Obstacle Avoidance; page unavailable) says that robot avoids obstacles using ultrasonic sensors and a basic reactive algorithm, not deep learning. The paper does not establish real-time, deep-learning-based obstacle avoidance for drones. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Deep learning can be used to enable drones to perform autonomous navigation and mapping.

    “Drones can carry out difficult tasks like autonomous navigation and mapping using deep learning.”

    From Artificial intelligence, machine learning and deep learning in advanced robotics, a review (2023) · cites this paper as [93]

    The Introduction discusses deep learning for a social robot, not drones. Section 4.4, “Obstacle Avoidance” (page unavailable), says that robot lacks a mapping sensor and uses a basic reactive algorithm. Searches for drone navigation and mapping found no passage addressing whether deep learning enables drones to perform those tasks. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 51

  1. Aleem, S.; Kumar, T.; Little, S.; Bendechache, M.; Brennan, R.; McGuinness, K. Random data augmentation based enhancement: A generalized enhancement approach for medical datasets. arXiv 2022, arXiv:2210.00824. · arXiv
  2. Kumar, T.; Park, J.; Ali, M.; Uddin, A.; Bae, S. Class Specific Autoencoders Enhance Sample Diversity. J. Broadcast Eng. 2021, 26, 844-854.
  3. Khan, W.; Raj, K.; Kumar, T.; Roy, A.; Luo, B. Introducing urdu digits dataset with demonstration of an efficient and robust noisy decoder-based pseudo example generator. Symmetry 2022, 14, 1976. [CrossRef] · DOI
  4. Ch, io, A.; Gui, G.; Kumar, T.; Ullah, I.; Ranjbarzadeh, R.; Roy, A.; Hussain, A.; Shen, Y. Precise Single-stage Detector. arXiv 2022, arXiv:2210.04252.2022.
  5. Roy, A.; Bose, R.; Bhaduri, J. A fast accurate fine-grain object detection model based on YOLOv4 deep neural network. Neural Comput. Appl. 2022, 34, 3895-3921. [CrossRef] · DOI
  6. Naude, J.; Joubert, D. The Aerial Elephant Dataset: A New Public Benchmark for Aerial Object Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA, 16-20 June 2019; pp. 48-55.
  7. Kim, Y.; Park, J.; Jang, Y.; Ali, M.; Oh, T.; Bae, S. Distilling Global and Local Logits with Densely Connected Relations. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada, 11-17 October 2021; pp. 6290-6300.
  8. Tran, L., Ali, M.; Bae, S. A Feature Fusion Based Indicator for Training-Free Neural Architecture Search. IEEE Access 2021, 9, 133914-133923. [CrossRef] · DOI
  9. Ali, M.; Iqbal, T.; Lee, K.; Muqeet, A.; Lee, S.; Kim, L.; Bae, S. ERDNN: Error-resilient deep neural networks with a new error correction layer and piece-wise rectified linear unit. IEEE Access 2020, 8, 158702-158711. [CrossRef] · DOI
  10. Khan, W.; Turab, M.; Ahmad, W.; Ahmad, S.; Kumar, K.; Luo, B. Data Dimension Reduction makes ML Algorithms efficient. arXiv 2022, arXiv:2211.09392. · arXiv
  11. Kumar, T.; Park, J.; Bae, S. Intra-Class Random Erasing (ICRE) augmentation for audio classification. In Proceedings of the Korean Society of Broadcast Engineers Conference, Las Vegas, NV, USA, 23-27 April 2022; pp. 244-247.
  12. Park, J.; Kumar, T.; Bae, S. Search for optimal data augmentation policy for environmental sound classification with deep neural networks. J. Broadcast Eng. 2020, 25, 854-860.
  13. Turab, M.; Kumar, T.; Bendechache, M.; Saber, T. Investigating multi-feature selection and ensembling for audio classification. arXiv 2022, arXiv:2206.07511. · arXiv
  14. Park, J.; Kumar, T.; Bae, S. Search of an Optimal Sound Augmentation Policy for Environmental Sound Classification with Deep Neural Networks. 2020; pp. 18-21. Available online: https://koreascience.kr/article/JAKO202001955917251.do (accessed on 16 November 2022).
  15. Sarwar, S.; Turab, M.; Channa, D.; Chandio, A.; Sohu, M.; Kumar, V. Advanced Audio Aid for Blind People. arXiv 2022, arXiv:2212.00004. · arXiv
  16. Singh, A.; Ranjbarzadeh, R.; Raj, K.; Kumar, T.; Roy, A. Understanding EEG signals for subject-wise Definition of Armoni Activities. arXiv 2023, arXiv:2301.00948. · arXiv
  17. Ullah, I.; Khan, S.; Imran, M.; Lee, Y. RweetMiner: Automatic identification and categorization of help requests on twitter during disasters. Expert Syst. Appl. 2021, 176, 114787. [CrossRef] · DOI
  18. Kowsari, K.; Meimandi, K.J.; Heidarysafa, M.; Mendu, S.; Barnes, L.; Brown, D. Text classification algorithms: A survey. Information 2019, 10, 150. [CrossRef] · DOI
  19. Jamil, S.; Abbas, M.S.; Roy, A.M. Distinguishing Malicious Drones Using Vision Transformer. AI 2022, 3, 260-273. [CrossRef] · DOI
  20. Roy, A.M.; Bhaduri, J. Real-time growth stage detection model for high degree of occultation using DenseNet-fused YOLOv4. Comput. Electron. Agric. 2022, 193, 106694. [CrossRef] · DOI
  21. Roy, A.M.; Bhaduri, J. A deep learning enabled multi-class plant disease detection model based on computer vision. AI 2021, 2, 413-428. [CrossRef] · DOI
  22. Roy, A.M. An efficient multi-scale CNN model with intrinsic feature integration for motor imagery EEG subject classification in brain-machine interfaces. Biomed. Signal Process. Control 2022, 74, 103496. [CrossRef] · DOI
  23. Roy, A.M. A multi-scale fusion CNN model based on adaptive transfer learning for multi-class MI classification in BCI system. bioRxiv 2022. [CrossRef] · DOI
  24. Roy, A.M. Adaptive transfer learning-based multiscale feature fused deep convolutional neural network for EEG MI multiclassifi- cation in brain-computer interface. Eng. Appl. Artif. Intell. 2022, 116, 105347. [CrossRef] · DOI
  25. Bose, R.; Roy, A. Accurate Deep Learning Sub-Grid Scale Models for Large Eddy Simulations; Bulletin of the American Physical Society: New York, NY, USA, 2022.
  26. Khan, W.; Kumar, T.; Cheng, Z.; Raj, K.; Roy, A.; Luo, B. SQL and NoSQL Databases Software architectures performance analysis and assessments-A Systematic Literature review. arXiv 2022, arXiv:2209.06977. · arXiv
  27. Dillmann, R. Teaching and learning of robot tasks via observation of human performance. Robot. Auton. Syst. 2004, 47, 109-116. [CrossRef] · DOI
  28. Sahin, A.; Ayar, M.; Adiguzel, T. STEM Related After-School Program Activities and Associated Outcomes on Student Learning. Educ. Sci. Theory Pract. 2014, 14, 309-322. [CrossRef] · DOI
  29. Mubin, O.; Stevens, C.; Shahid, S.; Al Mahmud, A.; Dong, J. A review of the applicability of robots in education. J. Technol. Educ. Learn. 2013, 1, 13. [CrossRef] · DOI
  30. Singh, A.; Narula, R.; Rashwan, H.; Abdel-Nasser, M.; Puig, D.; Nandi, G. Efficient deep learning-based semantic mapping approach using monocular vision for resource-limited mobile robots. Neural Comput. Appl. 2022, 34, 15617-15631. [CrossRef] · DOI
  31. Kumar, T.; Park, J.; Ali, M.; Uddin, A.; Ko, J.; Bae, S. Binary-classifiers-enabled filters for semi-supervised learning. IEEE Access 2021, 9, 167663-167673. [CrossRef] · DOI
  32. Chio, A.; Shen, Y.; Bendechache, M.; Inayat, I.; Kumar, T. AUDD: Audio Urdu digits dataset for automatic audio Urdu digit recognition. Appl. Sci. 2021, 11, 8842.
  33. Singh, A.; Pandey, P.; Nandi, G. Influence of human mindset and societal structure in the spread of technology for Service Robots. In Proceedings of the 2021 IEEE 8th Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON), Dehradun, India, 11-13 November 2021; pp. 1-6.
  34. Belpaeme, T.; Kennedy, J.; Ramachandrran, A.; Scassellati, B.; Tanaka, F. Social robots for education: A review. Sci. Robot. 2018, 3, eaa5954.
  35. Billard, A. Robota: Clever Toy and Educational Tool. Robot. Auton. Syst. 2003, 42, 259-269. Available online: http://robota.epfl.ch (accessed on 16 November 2022). [CrossRef] · DOI
  36. Ricks, D.; Colton, M. Trends and considerations in robot-assisted autism therapy. In Proceedings of the 2010 IEEE International Conference on Robotics and Automation, Anchorage, AK, USA, 3-8 May 2010; pp. 4354-4359.
  37. Breazeal, C.; Kidd, C.; Thomaz, A.; Hoffman, G.; Berlin, M. Effects of nonverbal communication on efficiency and robustness in human-robot teamwork. In Proceedings of the 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, (IROS 2005), Edmonton, AB, Canada, 2-6 August 2005.
  38. Bar-Cohen, Y.; Breazeal, C. Biologically inspired intelligent robots. In Proceedings of the Smart Structures and Materials 2003: Electroactive Polymer Actuators and Devices (EAPAD), San Diego, CA, USA, 3-6 March 2003; Volume 5051, pp. 14-20.
  39. Kidd, C.; Breazeal, C. Effect of a robot on user perceptions. In Proceedings of the 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566), Sendai, Japan, 28 September-2 October 2004; Volume 4, pp. 3559-3564.
  40. Breazeal, C. Toward sociable robots. Robot. Auton. Syst. 2003, 42, 167-175. [CrossRef] · DOI
  41. Maleki, F.; Farhoudi, Z. Making Humanoid Robots More Acceptable Based on the Study of Robot Characters in Animation. IAES Int. J. Robot. Autom. 2015, 4, 63. [CrossRef] · DOI
  42. Phadtare, M.; Choudhari, V.; Pedram, R.; Vartak, S. Comparison between YOLO and SSD Mobile Net for Object Detection in a Surveillance Drone. Int. J. Sci. Res. Eng. Man 2021, 5, 1-5.
  43. Girshick, R.; Donahue, J.; Darrell, T.; Malik, J. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 24-27 June 2014; pp. 580-587.
  44. Girshick, R. Fast r-cnn. In Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile, 7-13 December 2015; pp. 1440-1448.
  45. Adarsh, P.; Rathi, P.; Kumar, M. YOLO v3-Tiny: Object Detection and Recognition using one stage improved model. In Proceedings of the 2020 6th International Conference on Advanced Computing and Communication Systems (ICACCS), Coimbatore, India, 6-7 March 2020; pp. 687-694.
  46. Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Reed, S.; Fu, C.; Berg, A. Ssd: Single shot multibox detector. In Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands, 11-14 October 2016; pp. 21-37.
  47. Ning, C.; Zhou, H.; Song, Y.; Tang, J. Inception single shot multibox detector for object detection. In Proceedings of the 2017 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), Hong Kong, China, 10-14 July 2017; pp. 549-554.
  48. Roy, A.; Bhaduri, J.; Kumar, T.; Raj, K. WilDect-YOLO: An efficient and robust computer vision-based accurate object localization model for automated endangered wildlife detection. Ecol. Inform. 2022, 101919. [CrossRef] · DOI
  49. Redmon, J.; Farhadi, A. Yolov3: An incremental improvement. arXiv 2018, arXiv:1804.02767. · arXiv
  50. Ding, S.; Long, F.; Fan, H.; Liu, L.; Wang, Y. A novel YOLOv3-tiny network for unmanned airship obstacle detection. In Proceedings of the 2019 IEEE 8th Data Driven Control and Learning Systems Conference (DDCLS), Dali, China, 24-27 May 2019; pp. 277-281.
  51. RobotLAB Group. NAO Version Six Price. 1981. Available online: https://www.robotlab.com/store/nao-power-v6-standard- edition (16 November 2022).

Cited by

Sorted by citations

Authors
Singh, A., Raj, K., Kumar, T., Verma, S., & Roy, A.
Affiliations
  • Indian Institute of Information Technology Allahabad (IN)
  • Dublin City University (IE)
  • United Kingdom Atomic Energy Authority (GB)
  • University of Michigan (US)
Corresponding author
  • Singh, Aditya
  • Roy, Arunabha
ORCID
Venue
Drones
Publisher
MDPI AG
Published
2023-01-23
Volume
7
Issue
2
Pages
81
ISSN
2504-446X (ISSN-L)
DOI
10.3390/drones7020081
OpenAlex
W4317906764
Cited by
154
References
2
Citations by year
2026: 32 · 2025: 32 · 2024: 52 · 2023: 37 · 2022: 1
FWCI
37.84
Type
journal article
Language
en
Open access
Licence
Locations
  • Drones (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Indian Institute of Information Technology Allahabad (N/A)
Topics
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Autism Spectrum Disorder Research
  • Physical Sciences › Engineering › Control and Systems Engineering › Robotics and Automated Systems
Keywords
autism spectrum disorder, social robots, low-cost robot, robot-assisted therapy, children with special needs, object detection, YOLOv3-tiny, SSD, obstacle avoidance, nonverbal communication, emotional expression, humanoid robots, voice interaction, technology acceptance

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review