Tinderbox 7.5 can help disentangle messy link networks to make your maps easier to understand as they grow.

In essence, Tinderbox imagines that each link is a spring that pulls its notes closer together. If notes get too close together, they repel each other, while more distant notes attract each other. Tinderbox simulates the physics of this contraption, which can often improve layouts a lot. (This is called force-directed layout.)

A problem with this approach is that your layout can get caught in a local minimum — a bad layout, but one where each small change makes things even worse. To avoid this, each time we start dancing, Tinderbox gives your notes a gentle, random kick. This shake helps make sure that the machine finds a reasonable configuration. (This is called simulated annealing, by analogy with the metallurgical process where heating and forging iron helps strengthen the metal.)

Force-directed layout of graphs is fairly common, though I don’t recall its use in a hypertext tool before. Some of the details here are novel — in particular, the force between unlinked notes is based on the Van de Waal’s force that makes liquids cohere rather than fly off into gaseous clouds. I don’t know whether that improves the result, or makes it worse; I hope to find out.

Research finds help in unexpected places. In this case, I got a boost from an old GDC paper by Thomas Jakobsen on “Advanced Character Physics”. Jakobsen’s problem was creating better animations of the impact of bullets on bodies in first-person shooters, which is a long way from improving knowledge representation in notes! But in pursuit of his objective, he picked up on Verlet integration, a numerical integration method that was developed for molecular dynamics simulations in the 1960s by Loup Verlet. (Remember what I said about Van der Waal’s forces? Small world.). That turns out to be very helpful.

This is not a panacea: really tangled link networks, like Mary-Kim Arnold’s “Lust”, stay tangled. It’s a handy expedient to try when your intellectual office gets cluttered.

A big new Tinderbox release with tons of improvements, big and small.

• Dancing maps with force-directed layout
• New chart view
• Big performance boost for maps (and elsewhere)
• Flags (!) to help visualization for qualitative analysis
• Intriguing bits of AI, from better layout kibbitzers to automatic recognition of names, places, and organizations.

Lots more on the new features soon.

When Professor Linda Grimaldi and her husband reached retirement last summer, they sold their house and their stuff and they’ve taken off for an indefinite wander through the world — a backpacking gap year without the backpacks and with a little more comfort. She’s started a wonderful new weblog.

And that’s a phrase I haven’t heard in a long, long time.

by Andrew Sean Greer

Get the book.

The letter is in German; the university is under the impression Arthur Less is fluent in German, and Arthur Less’s publisher, who recommended him, is also under this impression. So is Arthur Less. With God’s happiness, he writes back, I accept the pedestal of power, and sends it off flush with pleasure.

by Jane Harper

Get the book.

A top-drawer debut mystery. Aaron Falk, a Melbourne-based Federal Agent A top-drawer debut mystery. Aaron Falk, a Melbourne-based Federal Agent specializing in financial investigations, returns to the small Australian town from which he and his father fled twenty years ago. Back then, a girl from the high school had drowned and the townspeople had decided that either Aaron or his dad had something to do with it. Now, Aaron’s best mate from those years has shot his family and himself. The local cop has been on the job for a week; Aaron is persuaded to lend a hand. Fine sense of place, good minor characters, good plotting.


by John McPhee

Get the book.

After reading in Draft Number 4 how McPhee puts things together, it seemed a really good idea to go back and look at the practice. A wonderful book, the epitome of the New Yorker profile that talks at considerable length about a subject of no particular interest, written with such wit and craft that it’s enthralling. Terrible Terry Harmon and Dirty Shirt George Price: McPhee draws amazing portraits in short gestural strokes. McPhee is not afraid of repetition:

Leadline and Dirty Shirt and Terrible Terry — they did not back off from anyone. I learned from them to maintain a gulf between yourself and the other officers. I learned, Never cross that gulf. I learned, Don’t act like the other officers, dress like them, or socialize with them. I learned, Don’t be like them. Whatever they are, be different. Never waver in your dealings with them. Don’t vacillate. I learned, Never chastise people in public, even if they have earned it. I learned, Don’t alibi, don’t complain.

by Martha Wells

Get the book.

The narrator of this Hugo-nominated novella is a construct — a cyborg security agent. He works for the company, and (though he doesn’t know it) he owes a lot to Hammett’s Continental Op. He doesn’t like his work: he calls himself “murderbot”. He doesn’t like the company, and he certainly didn’t like the way the company could control him through his governor module. So, he disconnected it.

Most android stories follow Pinocchio in assuming that a nearly-human construct would want to be human, and want people to accept them as human. Murderbot doesn't. He doesn’t like people: his clients are stupid and stubborn, they tell him what to do, they inconvenience him by getting into danger when he’d really much rather be watching episode 297 of Sanctuary Moon. He doesn’t want to be more human; he wants humans to leave him alone.

Very well done, and pertinent to my Hypertext 2018 paper on “As We May Hear: our slaves of steel II” which explores some new questions on how we treat computational agents and environments.

When you copy some text in Tinderbox 7, you copy the text and its styles but not the text links. Being able to copy and paste text links along with the text seems a simple-enough request, and in fact it will be part of Tinderbox 7.5. One tester recently asked the obvious “what took you so long?” question: why was this difficult. Here’s the story.

The macOS text system is built around a class cluster of attributed strings that represent styled text. Clickable text is marked up with a special attribute that tells the system to call back to the view’s delegate when it’s clicked, passing an object that represents the link destination. This is typically a string or a URL object, but can be other kinds of objects too. Tinderbox annotates text links with a special Tinderbox Link object.

This engages one of the oldest and most vexed implementation issues in hypertext system design: whether links should be represented as embedded in the the text or as external references to the text. Embedded links make editing easier; for example, if you insert a character in an HTML file, the ... tags that represent links remain correct. If the links are kept in an external table, you need to update that table whenever the text changes. On the other hand, if you want to know how many links there are on a page, you need to scan the entire HTML file; for the external table, you just look at the table’s size. Good writing on this ancient controversy includes Ted Nelson, Berners-Lee & Cailliau, and a classic paper by Southampton’s Hugh Davis that finally settled the matter.

Tinderbox represents links externally, but embeds copies of the links in the text in order to obtain mouse-click behavior. After you've edited a note, Tinderbox scans the text and updates the link table. This gets tricky because

• people sometimes edit long texts, so scanning the whole text can involve a good deal of work;
• some documents have lots of links, and some link table tasks scale with the number of links; and
• some people — not I! — type rapidly, imposing a tight performance requirement.

So, there's a bit of tricky lazy evaluation and parallel processing involved here.

The problem is that, while copy/paste understand the common cases where the link is described by a string or a URL, they don’t handle the case where you associate a different kind of object with a link. This fact is not documented, and in fact the documentation is internally inconsistent. By placing myself in the shoes of the author of these macOS classes, I developed several hypotheses to explain the failure and coded around them. For example, the text system might need to know that it could safely make extra copies of the link annotation, which means my special Tinderbox link object would need to conform to NSCopyable. This turned out to be a dead end, but it took some time. The upshot is that Tinderbox links are silently stripped before they are copied to the clipboard.

If the system expects an NSString or an NSURL as the link object — some but not all of the documentation suggests that it does — then we might make our own subclass that adds a Tinderbox Link object to the string or NSURL. This would be great — but you are not supposed to subclass NSString or NSURL. (Everyone knows this — I knew it once, too — but that didn't stop me from coding up an implementation before I remembered.)

An alternative approach was to add a custom style attribute that describes the Tinderbox link as a string; when we update links for a text pane, we’d add both the standard NSLinkAttributeName annotation to get mouse click behavior and our custom annotation, too. Copying to the clipboard would strip out our nonstandard link annotations but leave the custom annotation intact; when pasting, we'd recognize the custom annotations and recreate the standard NSLinkAttributeName annotations. Easy and elegant! I was neat, clean, shaved and sober, and I didn't care who knew it.

But I was wrong. Not only does copy: clean out our non-standard NSLinkAttributeName objects, it also ignores our custom annotations. This is a side-effect of the way macOS coerces attributed strings to the pasteboard by converting them to RTF or RTFD; those custom annotations don’t work for RTF.

So, what we do it define an entirely new pasteboard format that contains an XML representation of all the copied text links. Copy does its usual work, then adds the additional data for the links. Paste does its usual work, then grabs this hunk of XML, parses it, validates the results, and uses those results to make new text links. Voila!

Well, not quite. Whenever the text is changed, we do a quick check to make sure that you haven't deleted all the text for a text link. If you have, we need to delete the link, too. And — guess what! — adding the new link annotation counts as changing the text. So, we’d create a new link when pasting, and the system would helpfully delete the new link before we finished adding it. Remember that lazy evaluation and parallel processing? That’s where this scan happens.

This was obvious in retrospect, but what I observed was a Twilight Zone bug: the automated tests worked but manual tests crashed. (The automated tests did things faster than the scavenging system, while the manual tests let the scavenging system catch up.)

The whole business was, in short, a classic example of a task that ought to have been easy, that any reasonable customer would assume to be easy. The final implementation is really not very large or complex. Nevertheless, it required an inordinate amount of work. This is usually the mark of bad code, but I really don’t see what would have made this easier.

Rumors are circulating that Apple plans to merge iOS and macOS in some mysterious way. Michael Tsai, whose weblog has become necessary reading, has a good rundown of the major players’ opinions. Brent Simmons, now at Omni Group, wrote a very sensible piece on the possibly-mythical merger, which he names “Godot.framework”. Dave Winer (to whom great birthday greetings!) wrote a fine reply.

That reply, titled “Dependent Developers,” begins:

Reading Brent's piece about Mac development made me think about the ideal, what I really want from being a developer for as long as I've been a developer. The thing that got me started was the independence of it. I could have taken a job at Bell Labs or some big mainframe or minicomputer company, and had a nice career being pushed around by bosses at big companies. But I went for PC development because it was something I controlled. I could do what I wanted. Make my own art. The things I wanted to do were things no company would approve of, they had no way of understanding it. And at first it was lovely. Then the corporate bosses at the PC companies started pushing us around and it turned to shit.

Now, of course this is going to catch my eye, with its insistence that what we do is art. Dave was coding when I was a graduate student and he’s coding now, long after the people who didn’t think this was art have left for management or hedge funds. But there’s another note that runs through the entire developer discussion that Dave makes really clear:

…it turned to shit.

I don’t think it’s quite that bad. But then again, I’m the guy who wrote that

We software creators woke up one day to find ourselves living in the software factory. The floor is hard, from time to time it gets very cold at night, and they say the factory is going to close and move somewhere else. We are unhappy with our modern computing and alienated from our work, we experience constant, inexorable guilt.

So it’s not a cakewalk. What changed? What went wrong, and when did it go wrong?

When personal computing got started, you could make pretty serious money by creating a good tool that people needed. Dave did that with MORE. Dan Bricklin did it with VisiCalc. Mitch Kapor with Agenda. There were lots more. No guarantees, certainly, and some good and smart people never got the big payday, but it was a real possibility.

That’s gone. The real money in iOS software comes from writing frauds and manipulating sad psychological quirks; it turns out that just about nobody makes a living, much less a killing, designing great iOS software. The economics are better on the desktop, but not much better.

The big fear from the iOS-macOS rumor is that Apple will destroy that, that they’ll cripple the Macintosh so badly that we’ll be left with a complete wasteland of 99¢ junk apps. There’s no place to run, no other viable desktop with a future.

Worse, it’s not clear what we can do. This reflects our current political emergency. Ignorant and wicked people control the government, propped up by chortling billionaires, corrupt evangelical hucksters, and resentful wing nuts; we can work to end the nightmare but the road is long and uncertain. The future of computing now seems to depend on the whims of a few businessmen, none of whom appears to care very much about computing.

Then again, Tinderbox 7.5 is coming. Tinderbox had its best year ever in 2017. We have wonderful tools; the new iMac Pro isn’t the Mac Pro we need and Xcode crashes an awful lot, but the two together are fast. I remember when you had to wait overnight to find out about a syntax error, when you edited with punched cards. (Dave remembers, too.) I read the entire History of United States Naval Operations in World War II, all 15 volumes, while waiting for Metrowerks to compile Fontina and Storyspace, and now I compile Tinderbox and run all 1317 unit tests in something like 3 minutes. We write stuff that we’d never have been able to write back then, stuff we couldn't imagine.

Hard times may be coming. Pogroms may be coming. We’ll bear the burden and pay the price if we must. Meanwhile, back to making some more art.