May 25, 2008
MarkBernstein.org
 
Follow me on Twitter

WikiSym Planning

The papers for WikiSym have all arrived, and now the Program Committee is reviewing more than fifty technical papers. This review process is complex and demanding, and it's not always very well understood.

The primary duty of the program committee is to ensure the integrity and the reliability of the research literature. People who attend the conference, or who consult its Proceedings in the future, must be confident that the results reported here are honest, accurate, and may be relied upon.

This differs from the duty of a commercial conference like TED, which must consider first what might best attract and entertain its customers. It differs, too, from conferences that cultivate movements, such as SEED or An Event Apart, where the program seeks to advance a professional goal, or where the intent is to train people in the techniques that have made some of their colleagues particularly successful.

Computer science conferences would very much like to have an exciting and attractive program, but it is much more important, for then, that the research reported be of the highest quality.

One clear consequence is that we don’t particularly care who wrote a paper or where they work. Another consequences is that we care very much about who is reading the papers; WikiSym has an amazing program committee. We need one, because wikis are so diverse, And of course we don't know, from year to year, what research will be ready to present: will we have lots of computational theory to explore? What adjacent disciplines will we need? But just look at some of these names:

We've got George Landow, who wrote the first paper on hypertext rhetoric, and Cathy Marshall, who started spatial hypertext. We've got Lilia Efimova, who does blog research, and Amy Bruckman, who used to live in MUDs and MOOs. We've got Adrian Miles, who put vogs/vlogs on the map and who linked hypertext to film studies, and Jeremy Ruston, whose tiny TiddlyWiki has morphed into an enterprise-scale technology.

WikiSym Planning
When all the reviews are in, part of the program committee will meet in Porto. We'll discuss every paper again. Could we have overlooked a mistake? Might one of the reviewers have been too generous, or too harsh? We won’t be perfect. But every paper gets close and sympathetic scrutiny. And the committee truly doesn't care whether the paper is written by an unknown student or a titan of industry. Photo: Lapidim

And that's just for starters. (I could drop names all day; go look for yourself.) The committee is actually too small — I wanted to invite more reviewers, but my masters were worried that there might not be enough papers to warrant such a large committee.

My point is that we've got more than thirty experts. Some are quite young, others have decades of achievement. They work for large companies, small companies, government agencies, charities, ancient universities and new ones. They speak many languages. They have all sorts of degrees, or no degrees at all. They are all, naturally, extremely busy people.

And they're all hard at work, reading a thick sheaf of papers. They're checking facts and checking references. Are the equations right? Is the methodology sound? Could the result be interpreted differently — and, if they could, how might we choose which interpretation is correct?

They don't get paid. They don't even get thanked properly, because the referee comments are all anonymous. If they find a mistake in a formula, for example, all they can hope for is a footnote: “The authors thank an anonymous referee for helpful corrections.”

Next time you look something up in a Proceedings or a Journal or a textbook and feel comfortable that it's right, thank those anonymous program committees.