SINGAPORE – Special needs students are learning to express their emotions and are better engaged in class after using three artificial intelligence-based apps.

The apps, which were successfully tested in a nine-week-long pilot run last year among 140 students, can also be customised by teachers to fit each student’s learning needs.

Jointly created by Nanyang Technological University (NTU), the Association for Persons with Special Needs (APSN) and Yoozoo Games, the apps will be rolled out to more students in three APSN schools next year, said Ms Michele Yip, senior manager of the infocomm department at APSN.

The schools involved are APSN Katong, Tanglin and Chaoyang schools.

The three mobile apps – EmojiCapcha, Happy Bird and Betterfly – are an avenue for students to express their emotions and make it easier for them to get past the hurdle of doing so in front of their teachers, said NTU Data Science and Artificial Intelligence Research Centre project manager Ivan Yew.

The apps, available on iOS and Android devices, were demonstrated to the media in a Zoom meeting on Tuesday (Nov 17).

EmojiCapcha asks users to match their facial expressions to the one shown on the screen, while Happy Bird uses facial expressions to control the bird’s movements on screen. Betterfly uses facial expressions to create different coloured butterflies that move differently.

NTU Data Science and Artificial Intelligence Research Centre’s Professor Ong Yew Soon, who led the project, said the team’s goal was to use AI for societal good.

“Prior to this, learning about emotional intelligence was more of a pen-and-paper exercise. Now, they can enjoy a personalised learning experience… (and pick up) social and emotional skills in a fun and engaging manner,” he said.

The apps were developed on the IntelliK AI Emotion Sensing Platform that was made in NTU, which can detect facial expressions using the device’s front-facing camera and match them to the emotions they portray.

It is through this platform that educators will be able to customise the games to their student’s level of learning, such as adjusting the difficulty level and picking different scenarios and images.

Ms Yip said: “We are constantly looking for ways to enhance the teaching of our students with special needs.

“We can easily integrate these apps into our emotion learning classes and being able to incorporate the gamification aspect brings out more engagement.”

Source: Read Full Article