Leading a Software Project Like Directing a Movie?

A couple of weeks ago a co-worker passed me an article that really got me thinking about what it takes to lead a significant software development project. In Successful software management style: Steering and balance* Walker Royce, Vice President Rational Brand Services, makes an interesting argument that running a major software project is a creative act, more like directing a movie than what most of us think of when we think of project management.

I must admit his argument is seductive. I think that as an architect, I consider myself more of a “creative type” who makes something out of nothing. Often I don’t like being held to a plan I had no part in making. And when I help make a plan, I budget some time for some tinkering.

Maybe Walker Royce has a point. Here are some quotes:

“Managing software projects successfully has proven to be very failure prone when using the traditional engineering management discipline. Comparing the challenge of software management to that of producing a major motion picture exposes some interesting perspectives. Both management problems are concerned with developing a complex piece of integrated intellectual property with constraints that are predominantly economic. This article introduces some comparisons between managing a software production and managing a movie production, then elaborates four software management practices observed from successful projects. The overall recommendation is to use a steering leadership style rather than the detailed plan-and-track leadership style encouraged by conventional wisdom.”

""Heresy!" some may shout. "Software projects need more disciplined engineering management, not less." Before you dismiss my claim as an insult to the profession, consider these observations:

• Most software professionals have no laws of physics, or properties of materials, to constrain their problems or solutions. They are bound only by human imagination, economic constraints, and platform performance once they get something executable. Some developers of embedded software are the obvious exception.
• In a software project, you can change almost anything at any time: plans, people, funding, milestones, requirements, designs, tests. Requirements -- probably the most misused word in our industry -- rarely describe anything that is truly required. Nearly everything is negotiable. "

"Metrics and measures for software products have no atomic units. Economic performance more typical in service industries (value as observed by the users vs. cost of production) has proven to be the best measure of success. Most aspects of quality are very subjective, such as maintainability, reliability, and usability."

"Process rigor should be much like the force of gravity: the closer you are to a product release, the stronger the influence of process, tools, and instrumentation on the day-to-day activities of the workforce. The farther you are from a release date, the weaker the influence. This axiom seems to be completely missing, or at least grossly underemphasized, by the process evangelists and literature, but it is usually very observable in successful software projects."

"Most unsuccessful projects exhibit one of these characteristics:

• Over-engineering on the early phases (creative aspects) of the life cycle. You need maneuverable processes that easily adapt to discovery and accommodate a degree of uncertainty to attack a few major risk items, prototype solutions, and build early and coarse artifacts. What creative discipline can you think of in which more process rigor is considered beneficial in helping humans think?
• Under-engineering on the later phases (production aspects) of the life cycle. Extensive change-managed baselines of detailed and elaborate artifacts need engineering processes with insightful instrumentation and attention to detailed consistency and completeness to converge on a quality product."

On the flip side, there is the problem of “Can my customer culturally handle the kind of “give and take” in a creative software development process? Do they understand the idea of “discovery” of requirements or do they think they've identified them all already? Can customers really admit how little they know about what they want or how poorly they sometimes articulate their “requirements”. Can they accept trying out an idea, deciding it was all that good, and throwing the work away?

Another problem area is that for most of us is that we have to live within a budget which often must be cast in stone very, very early in the process before those requirements (negotiable as they may be) are really well understood. How many customers really work in an organization that allows them to “go back to the well” and ask for more money without commiting some kind of career suicide?

Then there is the recurring problem of technical orgainzations expecting to have their budget estimates cut so they pad their numbers. Business users then come to expect to routinely receive padded numbers and promptly cut the estimate by an even larger percentage. For a humorous look at this situation see Project Approval Games: Three Fantasies mentioned in a previous post Feudal Line Management and Shared Resources.

My gut tells me to accept a lot of Walker Royce’s ideas and bake them into a project plan that appears more traditional by adding a lot of tasks with formal sounding names that really mean “verify we really understand what they want”. My experience is that customers often articulate pretty well the “happy path” of what they want when everything works like they hope. They are not so good, however, at telling you how many things can go wrong and telling us what the software should do when an exception occurs and we are no longer on the “happy path”. It may be wise to add a few formal sounding task names that really mean “contingency for exception handling we don’t know about yet goes here”.

Copyright © 2006 by Philip Hartman - All Rights Reserved

The postings on this site are my own and don't necessarily represent IBM's positions, strategies, or opinions.