Agility and Design
Software Development Methodology is traditionally the self-help refuge of failing managers. When development goes badly (and it generally does), some managers want to call in the consultants and set the rules and impose a method that Solves The Problem, without much regard to what, specifically, the problem might be.
What makes Agile Development interesting, I think, is that it reflects a new understanding of design, one that's been percolating through the arts and engineering for the last decade but that is only slowly gaining recognition. The aesthetic revolution of the early 20th century -- the embrace of honest materials, machine aesthetic, modernity, Bauhaus, Brancusi -- at its heart believed in planning. We'd sit down, rationally analyze what we needed (a house, a city, a teaspoon, it makes no difference), we'd draw up a superb design, and then we'd manufacture it for everyone. On the one hand, this led away from superstition and tradition toward a world where everyone can enjoy things that work well. On the other hand, this also led to five year plans and zoning boards, to Stalin and Pruitt-Igo.
The Agile folks begin with a rejection of software planning and formal design. In a world where everyone was taught to code to spec (and where coding was regarded as work for coolies -- a term once in vogue but now embarrassing since so much of it is currently outsourced to Asia -- the Agile guys said: just sit down and start coding.
Partly, this recognized the growing disconnect between what good programmers did and what they were told they ought to do. Partly, this was nostalgia for a legendary past of legendary programmers, as well as for the programming of our youth, undertaken alone and for fun rather than in cubicles and on deadline. Partly, I think, it's reconciling Scandinavian-model worker socialism with software; the Software Factory wasn't going to play that well in Austin and Palo Alto, but it's really not going to work smoothly in the labor environment of Scandinavia.
The core idea that makes Agile sensible, though, is that good and complex design can be incrementally created through simple, local changes. That's a new idea, a discovery of the last generation. When birds flock, they fly in complex patterns and formations -- but it turns out the birds don't need a complicated playbook, just some simple rules about following your neighbors. Cell differentiation, it turns out, leverages simple rules as well; there isn't a big master plan for building a kitten or a kidney, just lots of local rules. People didn't know this in 1960, just like people didn't know about evolution in 1860. They know it now.
Except for the currrent President of the United States, of course, who isn't sure about evolution.
The surprise in Agile Methods -- and, I think, the still-untested belief -- was its faith that code need not deteriorate over time.
- New programs are well-designed. Their abstractions are clear and intact. They are relatively small. They are comparatively clear. But new programs also, by definition, don't work: they don't solve the whole problem, they haven't been tested in the field, they can't address all the customer's concerns because those concerns are still inchoate.
- As programs are used, they begin to work. New features are added. Other features are improved. The software gains functionality. But each change blurs the abstractions and increases complexity. The passage of time and the work of many hands, not all of them equally capable, leads to tangled logic and gradual technical backwardness.
- Old software thus inevitably becomes an entropic tangle of old technologies and old ideas, patched together in an incomprehensible mess that nobody dares to change. But the old software works -- it took years but it plays a vital role in the enterprise -- and so it's impossibly valuable.
The Agile people -- especially the refactoring folk -- think they can reverse this trend. Martin Fowler showed (in Refactoring