AR for deaf students

Introduction

Hearing loss is a global problem that many people deal with and the percentage of people with a hearing impairment will only keep growing. It is projected that we will reach a global population of 8 billion people in 2022 [1]. Of these 8 billion people it is estimated that a little over 5% suffer from a form of hearing loss, these people are either fully deaf or have a hearing impairment of more than 20 decibels. This means that there are around 430 million people in the world with a form of hearingimpairment [2]. Out of everybody that is born in this world 2-3 per 1000 will be born with detectablelevels of hearing loss in one or both ears [3]. There are numerous ways to counter small amounts of hearing loss, there are also ways to help people with significant amounts of hearing loss with certain types of hearing aids. However not everybody in the world has access to these kinds of technologies. The global population in the world was around 7.5 billion in 2016, 6.1 billion of these people lived in developing countries, that is a percentage of more than 81% [4]. This means that a large percentage of deaf people can’t afford certain types of hearing aids.

There are many problems that people with a significant loss in hearing in developing countries face, a part of this is due to their financial status. Deaf people face significantly more mental health problems than hearing people, the reason for this is that there are almost no possibilities for deaf people to get help for their mental health problems. There are many cases in which a deaf child is not able to communicate with their parents, which will increase the likelihood of gaining mental health problems [5]. Helping deaf people is not a priority in many developing nations worldwide, therefore there are a lot of problems they face with schooling. There are almost no schools with specialized personnel for teaching deaf people. The costs for their education is much higher in general and it requires much more recourses to create and supply the deaf with the right forms of educational material [6]. There are many developing countries in which people with a disability, including deaf people, are not a priority. It took Ghana from 2002 to 2015 to draw up a simple inclusive education policy, after which the party that had power lost its power. Because of this there is still no policy statement for deaf education to serve as a framework. This is a great example that shows that governments in developing countries have an absence of willpower to implement laws and policies for deaf education. The same lack of willpower can be seen in the funding. In 2007 the government of Ghana allocated 1% of their educational budget to deaf and inclusive education, this level of funding has stayed the same since 2007 and has not been changed [7]. Deaf students need a lot of support compared to hearing students and it is quite rare for the deaf to reach university grade education. About 90% of all deaf people in the world have not attended any school in their life [8]. The deaf students that do reach higher levels of education are inadequately supported in terms of their unique learning and communicationneeds [9].

The challenges highlighted above are caused by one main problem. It is difficult for the deaf to communicate with other people, even their closest friends and family. This is the main reason deaf people automatically have a disadvantage in their mental health, education, income and support from their government [5-9].

There are multiple ICT related products that could help us reach this goal and thus mitigate the problems deaf people face. A Hearing aid is something that is instantly thought of when someone is hard of hearing. Speech to text is one of the products that’s being used a lot, the idea of speech to text is to generate text from spoken languages. Speech to sign language is something that is quite new and which is still being developed, it is similar to speech to text, but in this case the spoken language gets translated to sign language. Besides these well-known technologies, there are also unique, innovative and experimental concepts that people are designing e.g., robots for little kids that help them with learning sign language. This study sets out to answer the following question: “Can deaf students gain the same amount of information as hearing students by using smart glasses with AR speech to text?” If the results of this experiment turn out to be positive for deaf people, these technologies could be put to use in deaf education around the world and give the deaf a better chance in society.

Methodology

The experiment is done with 3 groups of 8 people, aging from 16 to 30 years old, in higher education, to simulate the target audience as good as possible. All groups are hearing people.
The groups are all given a lecture of 6 minutes, about basic information security. Every group will see the exact same, pre-recorded lecture, so data will not deviate based on the difference in given lectures or stuttering from the teacher for example. The first group is used to set a baseline, these participants follow the experiment without any handicaps or tools given.

The second group is the “subtitle group”. These participants are going to be made deaf in an artificial way, by muting their surroundings and the lecture. This group uses a phone with pre-recorded subtitles matching the lecture. The phone is laying on their desk, while the lecture is playing on a screen in front of them.

The 3rd group is also artificially deaf and is given an Aryzon AR headset (This is further explained at the end of the methodology) displaying pre-recorded, augmented reality subtitles in front of their eyesight so they can look at the lecture in front of them and have subtitles in the same line of sight.
The lectures are given individually since there is only one set of AR headset to work with. Since neither of the groups should be given an advantage in any distractions, every person in every group will also have to participate individually.

Findings

A one-way ANOVA was performed to compare the effect of the experiment condition on the test score. The answer to each of the questions from each participant was normalised in a range of [0-1] and tagged with the experimental condition (hearing, AR, or subtitle).
A one-way ANOVA revealed that there was a statistically significant difference in test score between at least two groups
Tukey’s HSD Test for multiple comparisons found that the mean value of test score was significantly different between the hearing group and subtitles group. The mean value of test score was also significantly different between the AR group and subtitles group. However, there was no statistically significant difference between the hearing group and the AR group.

Conclusion

From the above findings we can conclude that simply providing real time subtitles is not sufficient since students seemed to retain a significantly smaller portion of the lesson material. However, providing these subtitles in the line of sight using AR provided the same amount of information retention as listening directly to the lecture.

The team

This work has been performed together with students from The Hague University of Applies Sciences:
Left to right: Penza Findley, Yannick Kokshoorn, Hani Alers, Sharwan Soekhoe, Marijn Runia.

References