Sarkar ’22 develops platform to help children with autism

Nandan Sarkar ’22 designed, developed, and tested an interactive, augmented reality coaching system designed to help children with autism spectrum disorder develop good tooth brushing skills in an engaging, accessible way.
By Sierra Smith, Communications Specialist

Nandan Sarkar ’22 has spent more hours than he can count working in a Vanderbilt University lab. Since February 2019, Nandan has been working alongside Z. Kevin Zheng, a mechanical engineering graduate student, to study the role and efficacy of using augmented reality to teach children with autism spectrum disorder life skills that traditionally require expensive behavioral therapy resources. Through an independent study at University School of Nashville, everyday after school, on many weekends, and over the summers, Nandan developed and tested an augmented reality program that is meant to teach toothbrushing skills to children with autism. 

“This particular project stood out to me because I feel like we all know someone with ASD; it’s very common — one in every 54 children in the United States is diagnosed with it — so it felt like a really great way to make a positive impact on many people,” Nandan said. 

The study included several phases requiring Nandan to learn and utilize many different skill sets in computer science, mechanical and electronic sensor design, and the basics of human subject study. To begin, he designed an AR platform that uses multiple coding languages to project real-world surroundings into a virtual setting that he created, complete with an animated avatar that demonstrates proper tooth brushing techniques and encourages participants. Moreover, the AR-based interaction Nandan designed needed to be fun and engaging; the result was a gamified platform with multiple levels increasing in complexity that children earn points to progress through.

“It was fulfilling because I learned a lot of new programming languages, but it was also really fun to make something that seems so much like a game, and see that the kids seemed to genuinely enjoy playing it,” Nandan shared. 

Finally, the system needed an assessment mode to test participants’ skills after completing all the coaching levels. Nandan and his partner had to find a way to track the movements of children brushing their teeth and their facial expressions in real time. This was accomplished by designing and 3D printing an instrumented mechatronic toothbrush and using a motion sensing camera paired with a custom-designed computer vision algorithm the team developed. 

“Getting the toothbrush on screen to perfectly match the child’s motions holding the actual toothbrush was one of the hardest parts,” Nandan said. “It was kind of frustrating because getting it precise was time consuming and tedious, but at the same time, once we got it, it felt so good to have accomplished something so difficult.” 

It was important that the program be effective in teaching toothbrushing skills in a way that was engaging and low stress for children. In addition to measuring children’s brushing capabilities before and after going through the program’s coaching mode, the study measured each participant’s physiological responses while using the program since research shows that physiological responses can be a measure of stress. 

Once the product was developed, the testing phase began. With support from Vanderbilt University Medical Center’s Treatment and Research Institute for Autism Spectrum Disorders and pediatricians, Nandan conducted a feasibility study using four children with autism and four typically developing children between the ages of three and six. Each child brushed their teeth in assessment mode to create a baseline for measuring their toothbrushing skill level, went through all the levels of the program, and then brushed their teeth again in assessment mode. 

“Nandan would just sit down to the same height beside the kiddos, with a genuine smile on his face and slowly demonstrate how to play. I was deeply impressed by those moments, seeing not merely a talented programmer, but a devoted researcher who sincerely cares for this underserved population,” said Zheng. 

For both sets of children, brushing speed and distance increased from pre- to post-test, an indication of the program’s positive impact. Additionally, participants and their parents answered post-assessment questions to evaluate the system’s feasibility; based on their responses, it was concluded that most children enjoyed interacting with the program and most parents found the instruction beneficial for their child. 

Nandan presented his findings to a panel of USN faculty members in December as part of his independent study. 

“What impressed me most about Nandan’s work was the motivation for it. He commented that much of the therapy sought by parents with children who have ASD is very expensive, and that the technological advances that he and his group were pursuing had the potential to provide massive savings to those parents. I appreciated how Nandan oriented himself towards work that has the chance to provide access to quality health care for those who otherwise might not be able to receive it,” said HS Math Teacher Justin Fitzpatrick, who was on the panel.

Following the completion of the feasibility study, Nandan and Zheng submitted their findings to the academic journal ACM Transactions on Accessible Computing in hopes of publishing them. His passion for the project is evident as he shares details of what’s to come. 

“Now we’re just waiting for feedback from the journal. And, of course, even once it’s published, there’s still so much more we can keep working on. That’s the thing with research; there are always improvements to be made and additional things to test. There are always ways to get better and help others,” Nandan said. 
Back

More USN News

List of 3 news stories.

Archive

Contact

Edgehill Campus
2000 Edgehill Ave.
Nashville, TN 37212

Phone:  615-321-8000
Fax:       615-321-0889 
Email:    info@usn.org

River Campus
1200 County Hospital Road
Nashville, TN 37218

Resources

USN Mission: 
University School of Nashville models the best educational practices. In an environment that represents the cultural and ethnic composition of Metropolitan Nashville, USN fosters each student’s intellectual, artistic, and athletic potential, valuing and inspiring integrity, creative expression, a love of learning, and the pursuit of excellence.