Google Glass for the blind - wishlist (English)

How do blind people experience Google Glass. What opportunities do they see and what would they like changed. Tips and wishlist for the hardware, the software and the apps

Introduction

Foto van Google Glass

We asked blind and partially sighted students aged between 12 and 18 to try Google Glass at Bartiméus in the Netherlands. As real ‘explorers’ they tested the innovative (im)possibilities of Glass and discussed their wishes and solutions with us. Below is our report. 

We started with a short presentation to explain the different parts of Glass, how to put it on and some example voice commands. After that, the pupils tried it out one by one. We asked them to talk aloud and share their thought with the others and with us during the complete session. The end result is large number of pictures, video’s, search actions, mail messages and many tips, wishes and ideas for Glass hardware, software and apps.

Like most people, the first reaction of the visually impaired pupils - once they had Glass on and got an answer to a Google search -was enthusiasm:

 “This is just great, what does it cost? " 

“It feels like I am playing a computergame..”

But it was not all that easy. The results of our test at the school for the blind will be used by the students of the University of Twente. This is a first impression.

32 pupils at Bartimeus tested Google Glass

In total, 32 pupils aged between 12 and 18 tried Google Glass. Half of them was blind, the rest partially sighted. All of them are pupils at the Bartiméus institute for the blind in Zeist (Netherlands). Most pupils had a smartphone (22 iPhones and 8 Android phones). One had an older mobile phone (only telephone) and one did not have a mobile phone at all. They say they use their smartphone for Whatsapp, sms, Facebook, music, internet, photos, videos and to make calls (mostly to phone home). The blind pupils all have an iPhone and use VoiceOver. Most of the partially sighted pupils use enlargement for text on their smartphone.

We divided the pupils into smaller groups of about 6. For the sessions we could use one of the classrooms of the school. The groups received a short explanation of Glass (what is it, short description of the parts and how do you put it on your nose). After that we gave them a list of possible commands. Directly after that they had the opportunity to try it out for themselves.

What did we ask them to do?

The assignment was simple. This is the first time they ever try out a head mounted display, so we just let them experience it and express themselves and we made notes.

We asked them to put on Google Glass and try it out. To get them started, we asked them to start with taking a picture and with searching something. Additionally, we gave them a list of other possible commands they could use. We also prepared two questions for discussion:

  1. Do you think Google Glass could help you participate better at school, work, home, sport, fun. If yes: why? if no: why not?
  2. What would Google need to change (hardware, software, apps) to make it work for you?

About Google Glass

Google Glass is a head mounted display. You wear it a pair of glasses. But instead of glasses, the screen is in fact a small block of glass with a projection that generates an image comparable to a 24 inch screen at a distance of 2,5 meters. Although there is only one real screen above your right eye, your brain magically places the screen in front of you.

It is only 75 grams and has a modern look. Glass doesn’t yet have the possibility to insert a simcard so you need to pair it to your smartphone or to wifi. The moment you link Glass to your smartphone it lets you take pictures, videos, read and send mail, look at maps, make notes, read documents, search the web, post to Twitter or Facebook and more.

Glass can be interfaced by tapping and swiping the touchpad near the right ear. By swiping over the touchpad you can navigate Glass applications but you can also navigate using voice commands. The blind pupils in our test indicate that the voice interface is currently limited. This means that they cannot use Glass without help.

Blind users are enthousiastic about the built-in bone-conducting headphone:

“This way I can listen to Glass and at the same time hear important sounds around me like cars, bicycles and maybe even birds when I am walking through nature”.

Using Glass with blind people

After starting Glass the screen shows the homepage with the time and <ok glass>. It goes into standby mode rather quickly, but you can easily wake it up again by tapping a finger on the touchpad or by moving your head up and down. After you say ‘ok glass’ you arrive at a menu full of voice-commands. A few choices:

'google’ (and then the subject you are searching for)

‘take a picture’

‘send a message to’

‘make a call to’

‘start a stopwatch’

‘listen to’

‘show a compass’

‘translate this’

‘find a recipe for’

‘add a calendar event’

‘take a note’ 

Google Glass voice interaction

Glass has voice input and voice output. Some of the voice input commands offer the possibility of a follow-up voice input or voice command. Others need input through the touchpad (swipe or tap).

An example of additional voice input is ‘ok glass, google’. After saying ‘ok glass’ and ‘google’, the screen shows a microphone and prompts you to say what you are searching. While you speak, Glass types your text on the screen.

An example of additional voice commands is after you take a picture. The picture is then shown to you. At the bottom of the picture it says <ok glass>. You can then share or mail the picture.

All the voice input and voice command screens are currently non-spoken. There is currently not a possibility to input through an external device like a braille line.

Glass will also read the search results to you. So if you ‘google’ for ‘the temperature in Amsterdam today’ it reads the results to you. Blind people know this from Siri on the iPhone, but were nevertheless very impressed during the tests. It doesn’t however tell you that you can tap to view the page or swipe for more search results, but as they say “Siri doesn’t do that either”.

We experienced that not all search results are read out by Glass. It is not clear why. The same search results was sometimes read out and sometimes not.

Google Glass touchpad interaction

Another form of interaction with Glass is through the touchpad on the side. Glass uses touchpad interaction for most of the follow-up actions. For instance if you search using google, you can swipe through the results using the touchpad. By tapping a search result, you open the webpage. By tapping or swiping with two fingers, you can choose specific parts of a screen, links or other interface elements like buttons on a webpage (i.e. to start an audio or video player).

Google Glass interaction sounds

Glass makes specific ‘beep’ sounds to indicate that the voice commands have been well received. That was very useful for blind people. Sometimes it sounds like there is also an error beep, but that did not seem consistent. Consistent use of sounds would offer good interaction help for all users.

Blind and partially sighted pupils and Glass

Glass currently does not provide a service like Talkback included into the interface. This means blind people currently have difficulty using Glass independently. This did however not keep them from enjoying themselves with Glass. They had great fun and most blind pupils saw the potential of Glass when it comes to participation possibilities at work, school, home, sport etc. 

The blind pupils immediately started taking  pictures of each other and doing searches but they also quickly got lost, talked to Glass while it just went into standby mode etc. Mostly due to missing audio cues. Probably the most asked question by blind people was: "Nothing is happening, can you check if it is on?" That last question also explains why I have a lot of pictures taken of myself looking straight into the lens checking what is on the screen... 

Some blind pupils asked if the button above the camera could be used to go back to the homepage like the home button on their smartphone.

Some pupils played the game app (tennis) that is provided in the Glass appstore. Most tried out google searches. They looked for ‘flowers’, ‘Ajax’, ‘iPhone’, ‘Everything radio’ (local radiochannel of one of the pupils), ‘Shell’, ‘Football’ and much more. A few reactions:

"This is great! I love it."

“If everyone uses Glass, what will happen to computers?” 

"I have no idea what is happening, is it on?"

“Facial recognition would be nice. It can recognize my friends so I can greet them before they greet me”

Google Glass wishlist for the blind

This is only a first inventory of what we heard when we asked blind and partially sighted pupils aged 12 to 18 to tryout and discuss Google Glass. This is what they said they would like to have:

  1. Zoomfunction: It would be great if you could zoom in to text or signs using Glass. Would be nice to be able to read or see things without requiring an additional ocular device on your glasses or a special magnifying glass for reading.
  2. Magnifying lenses: Add-on to Glass to make the image larger for people who have difficulties seeing it. Or maybe have a version with a larger screen.
  3. Audio hints: Make the interface talk (maybe using Talkback integration) and support easy and fast navigation with specific auditory sound-clues (because they are faster to navigate than text to speech).
  4. Homebutton: Provide a ‘homebutton’ so everybody can quickly go back to the homepage of Glass <ok glass> and start a new action.  
  5. Reader function: The possibility to read more web and other content from the web including PDF, Word documents etc. This could be included in a possible extension of the Talkback possibilities of Glass.
  6. Adjustable reaction time: Sometimes Glass is very fast. For instance if you want to send a mail, it already sends the mail after you pause in a sentence for more than 1,5 second. Also Glass switches to standby mode very fast. Would be nice if you could change that time in the settings.
  7. Left eye version: Are there plans for a left eye version of Glass for people who have better  eyesight in their left eye?
  8. Contrast: It would be great if Glass could enhance the contrast of objects while I walk. Or even better, scan for objects so I do not bump into them while walking.
  9. Language: Dutch version would be great.
  10. Input: For people who have problems using their arms or hands it would be good to have more speech input or to add motion-input. For example move your head to swipe and wink to tap.
  11. Where am I: Many would like the possibility to ask ‘ok glass’ and then ‘where am I?’. Glass could then tell you where you are (street, corner, number of house, etc.)
  12. External input: the possibility to also input information, notes, meetings, homework, voice commands etc. with an accessible external device like a keyboard. That way you do not have to talk all the time, for instance in a classroom.

Wishlist for Glass apps for the blind

During the discussions we heard many different proposals for apps that would be useful. Some of them are already possible in combination with Google+, Facebook, Hangouts and video. Like taking a picture to ask friends where you are. Also asking friends to join you by watching your live videofeed and telling you which direction to go or what to do. Dutch surgeons in Africa already use this during operations so colleagues in Amsterdam can advise them. Glass is also increasingly being used in sports so trainers can watch what their players see. pupils proposed the following apps:

  1. An app to ask your friends to tell you where you are, or to say what is on a sign etc. just by looking at a shared photo or joining you live by video.
  2. A cheat-app that opens a video feed with your friends so they can help you with the answers of your assignment without the teacher noticing…
  3. An OCR reader app. Reads text from screen, book, magazine, letter, notice board etc.
  4. Object detection app: Glass warns you when you before you walk into an object. Not just if it is on the ground, but also in the air so you are clear to walk. As a blind person, your stick only checks the ground up to your waist.
  5. Help to cross the street app: Extension of above app. Also looks at moving objects like cyclists, cars, etc. This may be more important now that (electric) cars are more silent.
  6. Recognize friends app: Glass recognizes your friends and tells you their name so you can say hello. Maybe it can even tell you if it is their birthday, what they are wearing, emotional information (looks happy, sad etc.). This way you can find and join your friends in the school cantine.
  7. Read subtitles: An app that can read aloud subtitles. In the Netherlands, all movies and documentaries are subtitled.

More research

This was just a first test with blind people to let them experience Google Glass. The results will be used by students of the University of Twente to work on accessible proof of concepts and prototypes for the hardware, software and apps for people with visual disabilities.

The Accessibility foundation plans more research on Head Mounted Displays for people with visual disabilities. The results of that research will be published here and will be announced through our Facebook and Twitter account.

For more information, please contact:
Eric Velleman


20140528

  • Share on Facebook
  • Share on Twitter
  • Share on Linkedin