Web Volatility

Kathryn Cramer wonders why her Web traffic is so volatile. What accounts for the sudden, inexplicable, surges in traffic to your weblog?

I've been wondering about the noise level in Web traffic — it's not just weblogs — for years. In fact, the puzzle was one of the research topics I proposed in my 1999 Hypertext Conference keynote, "Where (again) are the hypertexts?"

We should expect lots of fluctuation. Businesses, for example, see all kinds of fluctuation. One day is busy, another is slack. One ad works, another doesn't. People try to find reasons for everything, but sometimes there is no reason at all.

Suppose you have a sandwich place and you expect 50 people to come to lunch on a typical day. Hundreds or thousands of people could come to lunch, but most of them won't; they'll go to another place, or they packed their own lunch, or they are visiting their mother in Fargo. One day, six or seven people from Topeka happen to be walking past and one says they feel hungry and so they walk in. One day, Mrs. Zinfandel's bridge party is cancelled because the Red Sox are playing the Angels and so Mrs. Zinfandel doesn't bright her eight friends. It's a swing of 20% of your business, but it's not anything you did.

The fluctuations are too big to be random. Random fluctuations are proportionally larger at low traffic levels. Some noise, for example, arises because you can't see half a reader. That noise matters more for low traffic levels, obviously.

People surf in virtual cohorts. Eastgate gets weird traffic spikes, for example, when classes at various universities all visit more-or-less at once. Kathryn gets spokes when she's in the news, or when she's the target of a right-wing harassment campaign, or when she's discovered a unique presentation of important data. This is the modern equivalent of the old slashdot effect: gangs of surfers launched on a common trajectory over a comparatively short span of time.

Traffic begets traffic. If extra people visit Kathryn's page, she has extra chances of being mentioned in their weblogs. Those extra inbound links might lead to more traffic — and even more inbound links. The same activity can lead search engines to deliver extra traffic. This can lead to explosive feedback loops (the political blog echo chamber). Because weblog links last only a few days, the blogosphere's feedback tends to be spiky.

We really don't understand Web volatility. I think we ought to have a better quantitative explanation.