With the success of the Apple Pencil for the iPad Pro, could handwriting recognition be in the cards for iOS?

gg freckles. Eat up Martha. Cafe Fwiblob.

There was a time when these words would have elicited instant giggles. Memes from the time before Twitter or Facebook, Apple’s Newton and its often-errant handwriting recognition offered a wellspring of humor for the minds behind Doonesbury and The Simpsons, a spot-on sendup of a technology that was so close yet so far.

The signature feature of the original MessagePad was indeed worthy of ridicule. While Newton OS’s handwriting recognition engine was impressive from an I’ve-never-seen-anything-like-this perspective, its real-world application was loaded with flaws and miscues, far more than iPhone OS’s autocorrect or Siri’s listening abilities. As the jokes piled up, the Newton became synonymous with silly errors, and it was never able to fully shake its reputation, no matter how much better it got.

By the time Steve Jobs killed the whole project in 1998, Newton OS—by then on version 2.1—had a very solid handwriting recognition system that read both print and script with remarkable accuracy. It wasn’t just useable, it was widely regarded as the best in the industry. But despite its advances (not to mention his fondness for skeuomorphism), Jobs never really explored the possibilities for handwriting recognition, save the underpublicized Ink feature that was announced with OS X Jaguar, and Apple has done even less to promote the technology on its multitouch devices.

But it might finally be ready for its closeup.

Writing process

If the termination of the Newton project didn’t kill the dream of handwriting recognition, the introduction of the iPhone certainly did. Not only did it herald a whole new way of typing that combined smart autocorrect with fast, responsive keys, its capacitive screen was very clearly built for finger operation.

Jobs may have publicly dissed the stylus back in 2007, but the landscape has changed quite a bit since then. Styli are no longer dumb sticks, and the Apple Pencil represents once of the biggest advancements for input devices since the trackpad. Artists, sketchers and doodlers alike are effusive in their praise for its pressure-sensing, lag-free responsiveness, but there’s a lot for writers to love, too. Writing in Notes on the iPad Pro feels just as comfortable as using a notebook, and the end result looks nearly identical to actual handwriting, with Apple Pencil capturing the rhythm, spacing and thickness of the letters with remarkable accuracy.

There’s no denying that it does a stellar job at handwriting emulation, but real recognition is not part of the Apple Pencil experience. iPad Pro users who write with their Pencils will more or less get a digital rendition of the pen-and-paper experience, but notes aren’t any more editable or searchable than they would be inside a Field Notes book or on a scrap of paper. With the Newton, the handwriting recognition engine was meant to be a replacement for the keyboard: a quicker, more natural way to input notes than tapping away at tiny buttons, but the iPad Pro is as tied to its keyboard as the MacBook is.

Pencil sharpener

While handwriting recognition isn’t baked into iOS like it was with Newton, the underlying technology certainly hasn’t languished over the past two decades. With the Newton, it was completely software driven, but modern digital ink relies as much on the instrument as it does on the implementation.

“Devices for decades have never lived up to the expectations of the end users,” said Gary Baum, vice president of marketing at MyScript, which offers an array of handwriting recognition apps for iOS and announced a partnership with Wacom at CES to power the ink-to-text technology in Bamboo Spark. “The stylus itself needs to be as good as a pen or pencil. If it doesn’t work well it’s probably the most frustrating thing.”

Nearly every tablet platform uses a proprietary writing instrument to implement some version of handwriting recognition—Microsoft’s Surface Pen, Samsung’s S-Pen, even HP has the Executive Tablet Pen for its HP EliteBook Revolve 810 G2—but until the iPad Pro, Apple frowned upon styli, smart or otherwise. But it made a great one with Apple Pencil, and if handwriting recognition is something that’s eventually planned for iOS, Apple hit all the right marks.

“We’re seeing a tremendous focus on the stylus, because it’s the last real man-machine interface that is natural and expressive and creative,” Baum said. “No one wants to carry paper anymore; you can’t search it, you can’t store it easily. It’s kind of like Siri—once you use it you never want to go back.”