Anais Estendidos do XIII Congresso Brasileiro de Informática na Educação (CBIE 2024)
Using Artificial Intelligence to Strengthen Joint Attention in
Children with Autism
Nathalia Assis Valentim1 ,Fabiano Azevedo Dorça 1 , Valéria Peres Asnis1 Nassim Chamel Elias2
1
Universidade Federal de Uberlândia, Uberlândia, Minas Gerais, Brazil
2
Universidade Federal de São Carlos, São Carlos, São Paulo, Brazil
{nathaliavalentim,fabianodor,valeria.asnis}@ufu.br, nassim@ufscar.br
The use of assistive technologies, with an emphasis on artificial intelligence (AI), aimed
at improving the learning process of children with autism, focusing on joint attention (JA). Joint
attention is a crucial skill in child development, referring to a child’s ability to follow and direct
gaze toward objects or events of common interest. Children with Autism Spectrum Disorder
(ASD) often exhibit deficits in this skill, which directly impacts their learning process and so-
cial interaction. In this context, an Intelligent Tutoring System (ITS) was developed to interact
with autistic children aged 4 to 5, with the goal of helping them improve joint attention and,
consequently, their communication and socialization skills. The ITS proposed an adaptive and
dynamic virtual environment in which AI guided the child through a series of exercises. These
exercises were designed to become progressively more challenging, adjusting to the performance
and needs of each child.
To achieve this level of personalization and adaptation, techniques such as convolutional
neural networks and reinforcement learning algorithms were used, allowing for the creation of a
personalized learning environment that aimed to enhance the child’s interaction and engagement.
AI was employed to adjust the complexity level of the exercises, monitoring factors such as re-
sponse time and task accuracy. Additionally, the ITS included a facial emotion detection system,
using neural networks to identify emotions such as satisfaction, neutrality, and dissatisfaction
during the activities. This ability to monitor the children’s emotional state allowed for real-time
adjustments, fostering a more responsive and empathetic learning environment.
The combination of these techniques enabled the creation of an adaptive system that
tracked the student’s progress, positively reinforcing their achievements and offering support
when necessary. An important component of the proposed solution was the dashboard developed
for professionals and caregivers. This dashboard allowed detailed monitoring of each child’s
progress, facilitating pedagogical decision-making and the planning of personalized educational
interventions. Thus, the ITS not only adapted to the children, but also provided valuable infor-
mation to the adults involved in their development.
Experiments conducted with the ITS showed promising results, with a significant im-
provement in the joint attention levels of the participating children. This demonstrates that the
use of intelligent tutoring systems, combined with artificial intelligence, can be an effective so-
lution to strengthen joint attention in autistic children, promoting their cognitive and social de-
velopment. Furthermore, the developed approach can be expanded to other educational areas,
serving as a model for new assistive technologies aimed at teaching children with ASD.
Throughout the development and implementation process of the ITS, various technical
and pedagogical challenges were faced and overcome. The design of the exercises required
interdisciplinary collaboration between autism specialists, educators, and AI developers to ensure
that the activities were not only challenging but also engaging and accessible to the children.
The implementation of convolutional neural networks and reinforcement learning algorithms
DOI: 10.5753/cbie_estendido.2024.243589 29
Anais Estendidos do XIII Congresso Brasileiro de Informática na Educação (CBIE 2024)
required a robust technological infrastructure and the ability to process large amounts of data in
real time, as the system needed to continuously monitor and adjust the children’s interactions
with the virtual environment. An additional important aspect was the concern for privacy and
data security, as the ITS collected sensitive information such as facial expressions and behavioral
patterns. Strict security policies were implemented to ensure that all data was protected and used
exclusively for research and system improvement purposes.
The impact of the ITS on the social and cognitive development of the children was mea-
sured through a series of evaluations conducted before and after the system’s use. These evalua-
tions focused on indicators such as the ability to maintain joint attention, response times to visual
and auditory stimuli, and interaction with other children and adults. The results of these evalua-
tions were highly encouraging, with reports from parents and educators indicating improvements
not only in joint attention skills but also in communication and the children’s willingness to en-
gage in social environments.
As a result, it can be concluded that the use of intelligent tutoring systems supported by AI
represents a powerful tool for teaching autistic children, potentially becoming a key component
in the development of new educational methodologies. The success achieved in strengthening
joint attention with this system suggests that AI has enormous potential to transform the way
children with ASD are educated, offering more personalized, dynamic, and effective teaching.
Additionally, this technological approach can be adapted to other age groups and developmental
conditions, further expanding its impact in the field of special education.
Finally, the advancements made through this work open doors for future research and
innovation at the intersection of education and artificial intelligence. Continuing this work could
include expanding the capabilities of the ITS to address a broader range of skills and challenges
faced by children with ASD, as well as exploring new technologies such as augmented and vir-
tual reality to create an even more immersive and effective learning environment. Thus, the
combination of AI and personalized pedagogy emerges as a promising frontier for the develop-
ment of assistive technologies that can benefit not only autistic children but any student in need
of adaptive educational support [1].
Acknowledgment
This research was supported by Fundação de Amparo à Pesquisa do Estado de Minas Gerais
(FAPEMIG) - UNIVERSAL DEMAND Process: APQ-00837-21.
References
[1] Valentim, N., Dorça, F., Asnis, V., Elias, N.: The artificial intelligence as a technological resource
in the application of tasks for the development of joint attention in children with autism. In:
Anais da XII Brazilian Conference on Intelligent Systems. pp. 306–320. SBC, Porto Alegre,
RS, Brasil (2023), https://sol.sbc.org.br/index.php/bracis/article/
view/28364
30
