FunTorch
Sign Language Interpretation
A team of volunteers from PyTorch Scholarship Challenge from Facebook hosted by Udacity creating awesome Deep Learning models.
Find the project on GitHub:
Main project Android app
A team of volunteers from PyTorch Scholarship Challenge from Facebook hosted by Udacity creating awesome Deep Learning models.
Find the project on GitHub:
Main project Android appSign Languages are languages that use the visual-manual modality to convey meaning. Sign languages are fully-fledged natural languages with their own grammar and lexicon used worldwide not only by the deaf and hard of hearing, but also by hearing individuals, such as those unable to physically speak, those who have trouble with spoken language, or those with deaf family members, such as children of deaf adults.
Although, such languages have been around for a while,there is a very small percentage of world population that knows how to interpret them. Through our project we tried to somehow bridge this communication gap by providing everyone with the opportunity to interpret the hand signs by use of an application or software interface. This sign-language interpreter is just a prototype which is far from perfect and not yet capable of being used in real world conditions, but we believe it is a step in the right direction!
Here we have started small and tried classifying the numbers 0-9 signed using American Sign Language. The most part of the data in form of images has been collected and contributed by the members of our group and other helpful volunteers, using their webcams. The model is a custom-made lightweight Convolution Neural Network with a small number of parameters. The model is currently trained on around 6000 sign language images. It achieves more than 96% testing accuracy.
Have a look at the demo video below.
One of our awesome team members went ahead and put together an Android application to serve as an easy interface for our project.
You can just type in the number as a word and see how it is expressed in the sign language.
You can do the reverse. Take a picture using your phone via the app and let it interpret and tell you what number you’re expressing.
It warns you if you’re doing something wrong. Sweetly!
Check if you have the Java Developer Kit, version 7 or greater, already installed on your computer by opening a terminal window and typing:
java -versionIf JDK is not available, or the version is lower than 7, download it from Oracle's website.
Download Android Studio. Accept the terms and conditions to start the download. Double-click the downloaded file and follow all the prompts. After finishing the installation, the Setup Wizard will download and install some additional components. Be patient, this might take some time depending on your internet speed.
At the start up of Android Studio, click Check Out project from Version Control, pick GIT and paste the GitHub URL of the project: https://github.com/forfireonly/SignLanguage
Connect your phone to your computer with a USB cable. If you are using Windows, you might need to install the appropriate USB driver for your device.
Enable Developer Options and USB Debugging:
Run the app on your phone as following:
In Android Studio, click the app module in the Project window and then select Run > Run In the Select Deployment Target window, select your phone, and click OK.
We are a team of members from all over the globe. We all speak different languages, we belong to different cultures and even different age groups. But, the Udacity PyTorch Challenge and especially this project brought us together and each one of us has contributed to the progress of it!
We thank everyone at Udacity for this amazing opportunity they gave us not only to learn things individually, but also to work as a team and to interact with people from different parts of the world. It has been a fun ride. Kudos everyone!