SignVox
SignVox is an Android app for people who sign and want to be understood by someone who does not. You sign at the phone's front camera, and SignVox speaks the words out loud in a natural-sounding voice. Everything runs on the phone, with no internet connection.
Early access
SignVox is at an early stage. Its letter recognition has been measured on public photo collections of hands, and the app has been tested on an emulator, but not yet measured with a real person signing at a real phone. Expect mistakes, especially at first. Do not rely on SignVox where a misunderstanding could cause harm, such as in a medical emergency. See How accurate is it?
What it does
| Mode | What you do |
|---|---|
| ABC (spelling) | Fingerspell words letter by letter in the American Sign Language (ASL) alphabet. SignVox speaks each word as you finish it. |
| 123 (numbers) | Fingerspell the digits 0 to 9. |
| Signs | Teach SignVox your own signs for whole words, five samples each, then sign them to have them spoken. |
| Conversation mode | A split screen: a live text transcript of what the other person says on top, your signing below, and quick phrases. |
It also has:
- Word suggestions: after a letter or two, tap a suggested word instead of spelling the rest.
- Auto-correction: a spelled word that is one letter away from a common English word is fixed before it is spoken.
- Improve Recognition: teach the app your own hand, letter by letter, so it recognises you better.
- Vocabulary and History: manage your taught signs, share them as sign packs, and repeat earlier sentences.
Who it is for
- Deaf and hard-of-hearing people, and anyone who signs, who want to speak with people who do not know sign language.
- People learning the ASL alphabet who want instant feedback on their handshapes.
How accurate is it?
The project publishes these numbers:
- On a collection of hand photos that the model never saw during training, single models scored between 69% and 76% per letter. The app combines five models, which does better than the average single model.
- On held-out photos from the same collections it was trained on, it scored 99.3%. The project notes that this figure flatters the model.
A photo collection is not a person signing at a phone. Lighting, camera angle, hand size and signing style all change the result. To get better results for your own hand, use Improve Recognition. Whether this helps measurably for real users has not been measured yet.
Privacy
SignVox asks for only two permissions: the camera, and the microphone for Conversation mode's captions. It does not ask for internet access at all, so it cannot send anything off the phone. Hand tracking, recognition and the voice all run on the phone. When you teach signs or your hand, it stores compact measurements of hand positions, never photos or video.
Platforms
Android 7 or newer, on a phone with a front camera. The current release is 1.1.0. See Install.