Oct. 14, 2016 — Checking email, tracking fitness, and listening to music, are just a few things that a smartwatch can do but what if your hands aren’t free (i.e. carrying groceries or holding a bus handle)? A Dartmouth-led team has come up with a solution by developing WristWhirl— a smartwatch prototype that uses the wrist wearing the watch as an always-available joystick to perform common touch screen gestures with one-handed continuous input.

For a video about WristWhirl, visit: https://www.youtube.com/watch?v=JTPzeZGiI8w&t=10s.

The WristWhirl project will be presented at the ACM Symposium on User Interface Software and Technology on October 19.

Image

WristWhirl. Screenshot from project video courtesy of Dartmouth.

“While other studies have explored the use of one-handed continuous gestures using smartwatches, WristWhirl is the first to explore gestural input. Technology like ours shows what smartwatches may be able to do in the future, by allowing users to interact with the device using one hand (the one that the watch is worn on) while freeing up the other hand for other tasks,” said Xing-Dong Yang, assistant professor of computer science at Dartmouth.

To develop the WristWhirl prototype, researchers investigated the biomechanical ability of the wrist by tasking a small group of participants to conduct eight joystick-like gestures while standing and walking. Participants wore the watch on their left wrist and were asked to use their wrist to make four directional marks similar to flicking a touch screen, and four free-form shapes, such as a triangle. They were asked to make these gestures with their hand-up in front of their body during which they could see the gesture being drawn on the watch’s screen, and with their hand-down alongside their body. They were able to make directional marks at an average rate of half a second and free-form shapes at an average rate of approximately 1.5 seconds. Visual feedback appeared to slow participants down, as they were slowest at performing the tasks when their hand was out in front of them and fastest when their hand was down alongside their body.

To measure the accuracy of the free-form paths drawn using Wrist-Whirl, researchers used the $1 gesture recognizer and found that it was able to recognize the marks with a 93.8 percent accuracy as compared to 85 percent accuracy by three human inspectors.

WristWhirl was built from a 2“ TFT display and a plastic watch strap augmented with 12 infrared proximity sensors and a Piezo vibration sensor placed inside the wrist strap, which is connected to an Arduino DUE board, a type of microcontroller board.

Four usage scenarios for WristWhirl using off-the-shelf games and Google Map were tested: 1) a gesture shortcuts app was created, which allowed users to access shortcuts by drawing gestures; 2) a music player app was created, which allowed users to scroll through songs through wrist-swipes and play a selected song by double tapping the thumb and index fingers; 3) a map app was implemented for which 2D maps could be panned and zoomed depending on where the watch was held in relation to one’s body; and 4) game input, which often requires continuous input was tested, for which Tetris was played using a combination of wrist swipes, wrist extension and wrist flexion.

Xing-Dong Yang is available for comment at xing-dong.yang@dartmouth.edu. Jun Gong, a graduate student in Dartmouth’s department of computer science and Pourang Irani, professor of computer science at the University of Manitoba, also served as study co-authors.