Super-star programmers

ARE geniuses just born with their brains wired differently? Or do their early experiences fashion a richer set of neuronal interconnections that let them view the world through a sharper lens? The literature is replete with accounts of people who went on to accomplish great things—in the arts, sciences, philosophy or even politics—after exhibiting little promise in their youth. It would be encouraging to think that, if nurturing does indeed play a crucial part, there could yet be hope for the rest of us.

An outfit in San Francisco called “tenXer” has begun testing a service that aims to help people boost their mental accomplishments by up to tenfold—hence its name. That has made your correspondent wonder what distinguishes the truly talented from the journeymen of any trade. And what, if anything, the rest can do to improve their more menial lot.

Several years ago, your correspondent wrote a column about innovative operating systems (see “Heading for the clouds”, June 17th 2010). The software he admired the most was PC/GEOS, from a tiny firm in Berkeley, California, which crammed a full multitasking operating system and a whole suite of applications with scalable fonts and a stunning graphical interface into a couple of megabytes. The Microsoft equivalent of the day needed nearly ten times more memory and came with half the tools and none of the applications. In the early 1990s, GeoWorks Ensemble (as the program later became known) was the hare to the Mac and Windows tortoises.

So how come PC/GEOS failed to take the personal-computer world by storm? As the company founder subsequently explained, Microsoft got wind of what the upstart operating system could do and hastily tied up all the computer-makers who had licences to pre-install MS-DOS with exclusive two-year contracts to use only Windows. “They simply froze us out of the market.”

The point of the anecdote is not simply to show that big generally trumps best, but that a small team of talented folk will invariably outperform an army of sloggers. The whole PC/GEOS suite was written by a handful of programmers—a feat that took Microsoft hundreds of man-years to match. The former were all ten-xers, while the latter comprised hordes of one- and two-x programmers.

In his delightful guide to recruiting top talent (“Smart and Gets Things Done”), Joel Spolsky, a company founder and ten-x programmer himself, as well as a former paratrooper in the Israeli army, wrote in 2007 that the trouble with using a lot of mediocre programmers instead of a couple of good ones is that no matter how long they strive, they will still produce something mediocre. “Five Antonio Salieris won’t produce Mozart’s Requiem. Ever. Not if they work for 100 years.”

The idea that top-tier programmers are typically an order of magnitude more effective at writing computer code than their run-of-the-mill cohorts is not exactly news. In study after study since the 1960s, researchers have noticed that the productivity of individual programmers with similar levels of experience varies by a factor of at least ten-to-one.

More interesting still, in “Peopleware”, published in 1987, Tom DeMarco and Timothy Lister found that not only did the best programmers generally outperform the worst by a factor of ten, but that there was at least a tenfold difference in productivity among software organisations. Within individual firms, the difference in performance was only 20% or so. Clearly, the brightest programmers tended to congregate in places that had a reputation for attracting talented people; where the challenges were enticing, and the conditions conducive to good work. In many cases, that meant leaving large software companies to join smaller ones or to start their own.

But what exactly do ten-xers do to be so productive? It is not as though they write more lines of software per day. What little evidence there is suggests that, over any given period, all experienced programmers—good, bad or indifferent—tend to produce much the same amount of code.

The big difference is that the best coders keep more of what they have produced, while the worst constantly have to rework whole sections. The high-achievers also make each line of code achieve more. And they know how to avoid writing unnecessary code—by editing routines they have written in the past to accomplish similar things.

Above all, elite programmers understand how to manage complexity. The complexity of software rises exponentially with size. A program comprising 100,000 lines is thousands of times more complicated than one with 10,000 lines, which, in turn, is hundreds of times more complicated than one with a mere 1,000 lines. To give some idea of the problem, a typical operating system today contains upwards of 50m lines of code. Programmers who can hold in their heads all the essential elements of such a piece of software are truly in a league of their own.

Can talent on such a scale be fostered? Probably not. Apart from needing to be endowed with more than adequate mental horsepower, such people tend to be autotelic by nature—ie, internally driven, like missionaries. As such, they are not attracted by the material rewards that motivate ordinary folk. Their work alone provides them with all the satisfactions they need. But there is much that can be done to nurture lesser minds. And that is true not just for computer programmers.

An essay written by Venkatesh Rao, author, IT consultant and programmer, caused a stir in the blogosphere last year by suggesting that as many as one in four people could potentially transform themselves into ten-xers. Mr Rao divided daily tasks into short, low-value chores (answering e-mail, paying bills, going to the post office, etc) and lengthier, high-value activities (eg, coding software, composing music, writing books) that required a person’s full creative juices to flow. The former he likened to aerodynamic drag; the latter to engine thrust. According to Mr Rao, boosting performance was simply a matter of increasing thrust and reducing drag. The greater the difference, the faster the mental processes would fly.

All easier said than done. While managing the dreary drag work is something that can be mastered (or farmed out) without difficulty, thrust work is much harder to speed up. Creative tasks simply take longer to get started. But once underway, their momentum builds exponentially, until tiredness finally sets in and productivity falls. The creative momentum a person builds during a four-hour session is at least four times the peak momentum achieved in a two-hour session. Best, then, to schedule the longest blocks of uninterrupted time possible strictly for thrust work.

Where Mr Rao’s aeronautical metaphor stalls is when it comes to defining the engine for generating thrust. His tentative conclusion is that it has two components. One is a person’s natural talent (for logic arguments, perfect pitch, or juxtaposition of words)—which may well be acquired in the womb. Lucky the few who are born with it.

The other component is a person’s capacity for metacognition. This psychological term describes the tricks people develop to control their own thinking processes. Figuring out how to do a particular task, and then making sure it is done correctly, involves cognitive processes of planning, evaluating and monitoring various activities—all of which people seem to get better at with practice.

So, for people with the mental tools, tuning their engines and reducing their drag becomes simply a matter of clearing their calendars of chores and getting their heads down for some lengthy, uninterrupted sessions of creative thinking. Mr Rao admits it is not easy getting from low-x to ten-x performance. “You have to get through an initial starter-motor phase, hit an ignition point, and then keep the engine running until it wears out.”

For those wishing to have a go, tenXer in San Francisco is currently offering a beta version of its tracking software to help people analyse their own performance, set goals for themselves and improve their lot. At present, the tracking software works only for programmers. But down the road, the same techniques could just as readily be applied to other occupations. Your dilatory correspondent would love to be able to cram a day’s worth of thrust work into a couple of hours. He would then have more time to lounge in the garden, tinker in the garage, watch the playoffs, and do all the other enjoyable things some call a drag.