HandSight is a prototype glove to aid the blind. It can sense the lightness or darkness of a surface with tactile feedback from a vibration motor for each individual finger. It can also sense distance from physical objects or obstructions and indicate direction and distance with the same vibration feedback. It supports additional modes, and the possibilities are nearly endless. The glove can connect over Bluetooth to switch modes and visualize the sensor readings.
This instructable was made as part of the final project requirement in the CS graduate course "Tangible Interactive Computing" at the University of Maryland, College Park taught by Professor Jon Froehlich. The course focused on exploring the materiality of interactive computing and, in the words of Hiroshii Ishii, sought to "seamlessly couple the dual worlds of bits and atoms." Please see http://cmsc838f-f12.wikispaces.com/ for more details.See our Wiki class page for the project here (where we talk about some of our challenges and limitations):
Step 1: List of Materials- Arduino Pro Mini-328 (5v, 3.3v may work but not tested) (x1)
- IR Reflectance Sensor - QRE1113 (x4)
- Vibration Motor ROB-08449 (x4)
- Ultrasonic Range Finder (x2) http://www.amazon.com/Ultrasonic-Module-HC-SR04-Distance-Arduino/dp/B004U8TOE6/ref=sr_1_1?ie=UTF8&qid=1355861471&sr=8-1&keywords=Ultrasonic+Range+Finder
- Bluetooth Mate Silver WRL-10393 (x1)
- 68 ohm Resistor (x1)
- 9v Battery and Battery Case with Switch (x1)
- FTDI cable (for programming) (x1)
- Double Sided Sticky Tape, or other adhesive to hold the ultrasonic sensors in place
- Wire, electrical tape, and soldering materials. Rainbow ribbon cable is recommended for easy wiring of the fingertip sensors.
- Glove that you donít mind repurposing permanently
- Extra cloth that matches the glove
- Velcro, Sewing Materials
- Windows Phone to run our visualization and control app, or any device that supports the Bluetooth Serial Port Protocol (SPP) if you want to develop your own app
All electronic components available on sparkfun unless otherwise noted
Step 2: 3D Printing
We printed with MakerBot Replicator 2. Download STL files, open in a program called ReplicatorG. Put on platform/center. Generate Gcode.
layer height 0.27 mm
number of shells 1
feedrate 41 m/s
travel feedrate 56
print temp 226
plastic diameter 1.75 mm
extruder nozzle diameter 0.4mm
(in the actual gcode we changed platform temp to 060, which worked well for PLA)
Print four sets of the fingertip case, and two of the ultrasonic slot. ultra_sonic_holder.stl
Step 3: Assembling the IR Sensor Package for Each Finger
Step 4: Attaching Fingertip Assemblies to the Glove
Step 5: Attaching Ultrasonic Sensors to the Glove
Step 6: Adding a Battery Pack
Step 7: Soldering connections
- Blue: Power for the IR led (sensor pin 1)
- Green: Ground for the IR led (sensor pin 2)
- Yellow: Collector for the IR detector (sensor pin 3)
- Orange: Emitter for the IR detector (sensor pin 4)
- Red: Power for the vibration motor
- Brown: Ground for the vibration motor
Replacing these colors with the corresponding wires in your setup, separate the ends of the ribbon cable and solder together the grounds (green, orange, and brown). Solder each of these wires from each of the four packages together into a single connection, with an additional wire running out for connecting to the board. Also solder the four blue wires together in the same manner. Wrap all of the connections with shrink tubing or electrical tape to protect the connections and prevent short circuits.
Step 8: Preparing the Arduino Pro Mini
Step 9: Soldering Pins on the Arduino Pro Mini
- Blue wires (from IR LEDs) to VCC through 68 ohm resistor
- Green/Orange/Brown wires (grounds) to GND
- Yellow wires to analog pins A0, A1, A2, A3 (in order from index finger to pinky)
- Red wires digital (PWM) pins 3, 5, 6, 9 (in same order)
- VCC from two ultrasonic sensors to VCC
- Ground from two ultrasonic sensors to GND
- Trigger and Echo from left ultrasonic sensor to pins 10 and 11, and from right sensor to pins 2 and 4 (respectively).
- Red wire from battery to RAW
- Black wire from battery to GND
Step 10: Adding Bluetooth Module
Step 11: Programming the Board
Our code is available at:
Download the git repository, and upload the file handsight_arduino/handsight_arduino.ino.
Step 12: Connecting Via BluetoothWe have developed a Windows Phone 8 app that can be used to control the device and visualize the sensor readings. If you don't have access to a Windows Phone 8 device, or if you would like to develop your own control/visualization program, you can do so over the Bluetooth Serial Port Profile (SPP). Simply pair the bluetooth module with your phone or computer, then open a serial port connection on the appropriate COM port. The data from the device is in the following format:
[mode] is a number from 0-5 indicating the current mode
[+|-|=] indicates whether the reading is above the detection threshold (+), between the detection and white thresholds so black is detected (=) or below the white threshold so white is detected (-).
[reading] is a number from 0-1023 for the first four readings, indicating the voltage from the IR sensor, while it is from 0-200 for the fifth and sixth readings indicating the distance measured by the ultrasonic sensors (in cm).
[text] is only present in typing mode, and gives the characters detected since the last serial update
The following commands can be sent to the device:
0: edge detection mode
1: shape detection mode (black detection)
2: grayscale mode
3: navigation mode
4: typing mode
5: massage mode
10: calibrate the detection thresholds using the current readings
11: calibrate the black thresholds using the current readings
12: calibrate the white thresholds using the current readings
Note that the numbers are the byte values, not the ascii character codes for those numbers
The Windows Phone 8 was developed in Microsoft Visual Studio 2012 on Windows 8, and is available in our GitHub repository linked in the previous step if you would like to use it.
Note that you will need to change the name of your bluetooth module to match ours, or change the C# code that connects to it.