Trying to make the pen as mighty as the keyboard
The New York Times
Portable computers that let you give commands or enter text using an electronic pen on the screen could usher in a new era of information sharing.
But pen computers like those using Microsoft's Tablet PC software have had only lukewarm success, and writing notes on a palmtop requires learning a kind of shorthand.
Pointing at and clicking on on-screen icons using a digital pen can be cumbersome and counterintuitive, and writing on a screen in longhand is much slower than typing. Factor in the huge amount of variation in handwriting and shorthand and it's little wonder computers have had a hard time recognizing them.
"Lots of people have trouble recognizing their own handwriting, so it's not surprising a computer can't do it," said Brad Myers, a professor in the human-computer interaction institute at Carnegie Mellon's school of computer science.
Now researchers are devising new ways to interact with tablet PCs and other pen-based computers. The machines use a special version of Microsoft Windows and a digitizer under the screen that samples pen strokes to let users write with digital "ink," peck out messages with an on-screen keyboard and navigate their desktop and folders with a stylus instead of a mouse.
The research could make it easier to enter text quickly and accurately, and it could encourage collaboration by introducing gesture commands that let friends and co-workers share photos and files with a flick of a pen.
For the latest breaking news, visit NYTimes.com
Sign up to receive top headlines
Get Dealbook, a daily corporate finance email briefing
Search the jobs listings at NYTimes.com
Last month Zhai and his team released software called Shark, or shorthand-aided rapid keyboarding, on IBM's AlphaWorks Web site, where the company previews emerging technologies from its labs. The software replaces the standard on-screen keyboard that a tablet PC usually displays with a stylus-friendly keyboard that places common letter combinations near one another in a hexagonal grid. Users write words by tracing a stylus across the virtual keys, forming shapes that Zhai calls sokgraphs.
Each time a user traces a word, the computer shows the shape on the screen. After enough practice, "you learn so much of the shape that it comes to mind and you can gesture it based on memory," he said. Meanwhile, the keyboard remains on the screen as a mnemonic device.
A tolerance for sloppiness is built in. Algorithms recognize that some letter combinations are more likely to form words than others, so users can deviate from the exact shape of each symbol. Shark includes a lexicon of about 8,000 words, and it can analyze e-mail messages and text files to learn new ones. According to user tests, people can learn about 15 sokgraph words in each 45-minute training session. Learning 100 patterns covers about 40 percent of the words most people write, Zhai said.
Ben Bederson, director of the human-computer interaction lab at the University of Maryland, said the Shark project "is an example of where the interface brings you from novice to expert with the same interaction." He added, "You think the word 'the' and execute the gesture without a huge amount of visual feedback and concentration."
Keeping people's attention on their work and not on their computers is also a goal of Ken Hinckley, a research scientist at Microsoft. Hinckley has developed software technology called Stitching that lets tablet PC and palmtop users share files by flicking their pen toward the screen of the person they want to share with.
Prototype software called StitchMaster lets users select photos on their tablet PC by tapping or circling groups of them, then wirelessly pass those files to another tablet user sitting within arm's reach by making a gesture that spans both screens. Stitching works by measuring the timing, the angle and the exit location of pen gestures that leave the edge of a machine's screen. Since tablet PCs can broadcast their Internet address using the 802.11 wireless networking standard, if a second machine within two and a half feet picks up the continuation of a pen stroke across its edge, the software can tell which machine sent it and infer a handoff.
Microsoft's usability tests have shown that a dozen participants were able to connect their tablets this way on their first or second try. But some were uncomfortable reaching across another person's computer, or even placing their machines too close together. Hinckley's latest work, which he calls collaborative stitching, aims to address that by letting a user pass a document to several people sitting beyond arm's length by dragging the file to the top of the user's screen. Then those people in the room who want the file can pull it down from the top of their screen.
According to Andy van Dam, a computer science professor and vice president for research at Brown University, who also serves on Microsoft's technical research advisory board, tablet PCs and other pen-driven computers won't take off until pen gestures provide new ways of interacting with the machines instead of simply substituting for a mouse. Pen computers could find markets in education, architecture, graphic design and user-interface design, he said. "For these people, a pencil and a piece of paper are more natural almost than a computer keyboard on a desktop."
But pen software needs more testing to find out what users really want, he said.
"For a relatively pure gesture-driver user interface, it's all research," he said. "None of these have had a field trial with a thousand users, let alone ten thousand. Gestures are never going to be for everyone, so I don't want to impose them. But we can make them an option."
Entire contents, Copyright © 2004 The New York Times. All rights reserved.
Track this story's companies and topics
|Microsoft Corp||Create alert|
|Intl Business Machines Corp||Create alert|
|Notebooks and tablets||Create alert|
PCs fall short
October 1, 2004
XP update give tablets a boost?
August 30, 2004
take their time
June 16, 2004
Tablet PCs off to solid start
March 14, 2003
From News.com Extra
- HP Compaq TC1000 Tablet PC Reviewed from Gizmodo
- The Tablet PC Takes Its Place in the Classroom from The New York Times
- A Tablet PC-based WLAN Design Option from Wi-Fi Planet
- Apple iPad? from Gizmodo
Related quotes (delayed 20+ minutes)
|IBM||95.92||0.60||(0.63%)||International Business Machines Corp.|
Heart Hospital needs no second opinion for decision to standardize on HP
server and tablet technologies help OnSite Docs make house calls (white
the TCO of Handheld Devices (white paper)
Paper: Mobile Workforce (white paper)
Mobile Worker Productivity While Securing Windows Notebooks and Tablet PCs
With CREDANT Mobile Guardian (white
America's favorite pastime is going high tech July 8, 2004