Beyond the Paper Simulator

At Hypertext ’87, Ted Nelson memorably deplored the Macintosh computer as an elaborate paper simulator and urged researchers to build new literary machines, not simply to reproduce old ones in new media.

At Multimedia 2010, Chunyuan Liao and his colleagues at FXPAL describe a clever system in which people read paper documents while a computer (equipped with a camera) looks over their shoulder. The computer recognizes margin notes and proofreaders’ symbols and is prepared to take appropriate action:

The user can choose individual words, symbols, figures, and arbitrary regions for keyword search, copy and paste, web search, and remote sharing. FACT thus enables a computer-like user experience on paper.

Thus, we are now recreating the affordances of new media in old. We are building simulated web pages on paper!

Update: Another fascinating approach: Les Éditiones Volumiques by Bertrand Duplat and Étienne Mineur. (needs Flash)