The Macrosite for News, Analysis and Opinion about the Future of the Internet
Brian Margolies

Exposing Agile Project Management for What It Is

Written by Brian Margolies
7/11/2012 22 comments
no ratings
1 saves
DISCUSS     Email This

Having spent a significant portion of my early career as a business analyst, I have always been keenly interested in potential means of delivering functionality faster to my users. As a long-standing practitioner of the traditional "waterfall" software development/project management methodology, I have integrated early prototyping and iterative functionality over the years.

Recently, I've become aware of a growing trend toward the adoption of agile software development and project management methodologies. I began investigating agile methods to understand if this trend offered a real breakthrough in delivery speed -- the elusive holy grail to IT organizations (not to mention our business partners).

The agile methodology was designed to speed up the delivery of developed capabilities (systems, applications, and functions). Time to market is important to all aspects of business systems development, but it is especially critical to customer-facing e-commerce systems.

I have seen the agile methodology described as involving "early and regular delivery of working software, a focus on team communications, and close interaction with the users."

Seems familiar, no? Isn't that what we have always strived to achieve? Agile methodology, it seems, is founded on a philosophy that espouses people and their interaction over methodological processes, tools, and documentation. Remember people, process, and technology? To the casual observer, this methodology seems more of an evolutionary step.

In reality, it appears to have borrowed from and recombined the best-practices that have been used in software development over the past 40 years. Essentially, the agile process for a software development project includes the following steps:

  1. The overall scope of the project is assessed. The business determines the overall priorities, and the development team estimates the effort required.
  2. A fixed iteration or "sprint" length is determined. Two weeks appears to be a common length for iterations.
  3. Functions determined to be in scope are broken into sets of high-level user stories (capabilities). Before implementation within a sprint, the stories are fleshed out at a level of detail sufficient for each function to be implemented within a single sprint. (Use cases, anyone?)
  4. Sprint planning starts. Stories are gathered until a full sprint's worth of functions are identified. Naturally, high-risk/priority functions are scheduled for implementation sooner in the development sprint cycle.
  5. Each story (capability) is refined. Specific tasks are deteremined, as we noted in step No. 3.
  6. Functions are developed, tested, and rolled out into production. This would happen in any waterfall-based project. Presumably, the process cycles back to step No. 4, concurrently with this step.

Agile project management. (Source: VFS Digital Design)
Agile project management.
(Source: VFS Digital Design)

The primary differences between the waterfall and agile methods include the frequency of functionality delivery and the formalized iterative approach in the agile method. This method has even developed its own jargon -- participants are classified as "chickens" and "pigs" (in a bacon-and-eggs breakfast, chickens are involved, but pigs are committed).

Conversely, waterfall development goes through a series of distinct phases: analyses, requirements, design, implementation/integration, testing, and rollout for the entire set of functions. This has also come to be nicknamed "the big bang."

Having used an iterative approach to software delivery (aka phases) for years, I find myself at a loss to determine the agile value proposition. I can see how it provides a codified methodology for iterative development, including how late business requirements can be incorporated (no mean feat). What isn't clear is how it addresses and improves upon the two most time-consuming activities of any software development effort: analysis and design.

If there is truth in the hackneyed expression "The devil is in the details," then how does employing an agile methodology avoid the time needed to analyze and design holistically? Wouldn't deeper requirements, specifications, and design performed just before sprint implementation reveal details that could result in more expensive development or force you to undo and then redo previous functions?

One could never say these things wouldn't happen using a waterfall approach, but it seems that the agile approach tacitly invites them to occur. I think it can be a helpful methodology, but the holy grail remains elusive.

Related posts:

— Brian Margolies is the CIO of Allied Beverage Group, New Jersey's largest distributor of wine and spirits.

DISCUSS     Email This
Current display:       newest comments first       display in chronological order
Page 1 of 3   Next >
jwallace
IQ Crew
Tuesday July 31, 2012 11:53:32 PM
no ratings

I do have a quick question. When distributing beverages, who keeps tabs of loss prevention (supply/demand at retailer) and reports it back to the mfr? 

is the role of the distributor strictly poiint a to point b with no ties to data?

JayVee
Rank: Scrivener
Friday July 13, 2012 4:49:49 PM

I thouyght the whole point of agile is that it gives the intended user more of a voice in the development process. This user is invited to see something, albeit unfinished, that works and then provide feedback and thus steer the development team more precisely towards a maximally useful and well-done finished product. The quicker you can show users something that works, the better chance you have of delivering something useful. If it's not useful in some respect, you'll find out faster and therefore be able to take corrective action more quickly than you might otherwise. The design is a constant work in progress, vs. the waterfall's all-done--at-once approach. 

Hey, I'm no programmer, at least not in any professional capacity, but this seems fairly obvious to me. But perhaps I am missing something. 

RufusJones
Rank: Web master
Friday July 13, 2012 2:16:47 PM
no ratings

No problem, Brian. It helps to have comments from people who haven't drunk the Kool-Aid.

My take: Any methodology, if used properly and rigorously, can be an effective tool for managing projects. No methodlogy is invulnerable to poor execution. 

I've worked at three sites that called themselves Agile, seven that claimed to be in various stages of adoption. All seemed related, but none were especially alike. The variations weren;t as bad as the workplaces that called themselves "Six Sigma", but not as tight as the Crosby folks.

I would bet my life that Agile is a fad that will be "refudiated" in favor of something else.... but that's a standing bet about any metholology of any kind that becomes popular.  (Currently got a pool going on the date everyone abandons ITIL and Cloud.)

 

Brian Margolies
Thinkernetter
Friday July 13, 2012 1:56:53 PM
no ratings

Thank you Rufus, that's about as close to a useful answer as I've seen.

RufusJones
Rank: Web master
Friday July 13, 2012 11:18:24 AM
no ratings

Almost certainly not. It was published in 1975, and people assume that the lessons in it must be useless because the technology has changed so greatlly.

The notion that human beings and organizations-- the constants in the development process-- have not changed one iota is lost on them.

It's one of the big misconceptions people make about dealing with ideas. If an idea failed for purely technical resons (the obvious example being Babbage's "analytical engine"), there is a good chance that it might work better 10, 20 or 40 years later.  It wasn't that computers couldn't play chess well-- just that earlier computers couldn't do the number of calculations necessary to do it well.

If the idea failed for human reasons, it's fairly likely it will fail again, unless you can identify some trigger for a change in behavior.

RufusJones
Rank: Web master
Friday July 13, 2012 11:05:11 AM
no ratings

Brian, the presumption underlying Agile is that the company:

  • Has existing assets (e.g., data) that it will use (or at least have to be used).
  • Knows what those assets are and has them properly documented in a repository.
  • Already has accurate and detailed maps of the processes.
  • Agrees on the critical issues-- at least at a high level--that need to be addressed.

That activity and consensus needs to take place before the project planning begins. Obviously, if that hasn't been done, the process can go off the rails. But that's true of any project.

The desgin part is the organization of the sprints. I'd describe it as a type of process mapping. As you sort and sequence the user stories together, it becomes clear that other stories will need to be written. 

"I want to be able to add these customers on the fly" means you will need some module to validate them against existing customers. That module becomes a story-- but it gets treated as a black box for the first story's purpose.

If not knowing how the validation works doesn't sound like a good idea-- you're worried about what could happen if it isn't robust-- then obviously that story has to be done first.  But once done, that's an exixting asset.

The concept behind Agile is that the stories and modules-- rather than being consigned to binders that go unread for all eternity-- go into a repository that becomes both a process map and collection of assets.  That's becomes the heart of the design and analysis for future projects.

It's not done all at one time in one formal "analysis and design phase", but the collective wisdom and rich heritage of the many project teams that have come before you grows together over time and becomes one in a unique harmonic convergence that leads to world peace, universal love and free dope, allowing us to talk to the animals and make contact with other worlds...

Look, if you drill down on Agile it gets sappy in a hurry. But it's not any less inance than a project methodology that says "We can understand how everything this enterprise function in an 8-12 week discovery plase where we:

  • Get our information from cadging meetings with people who don't have time to talk to us and don't really want to,
  • Might or might not (most likely 'not' given turnover, downsizing and use of contractors and vendors) speak to people who have performed the tasks they're allegedly subject matter experts about.
  • Might or might get people who can remember (or be willing to explain, for time or political reasons) why it was done and what the decision-making process was.
  • Can deem anything that looks like a bag of snakes "out of scope"

That produces bad design and analysis too-- just in a form that is considered acceptable because we've dealt with it for decades and have come to assume that it can't be done better.

As a rule, people tend to focus with laser-like precision on the flaws in any new paradigm. Those flaws, because their scope and impacts are unknown, draw great concern. The flaws in the existing process rarely get that same level of scrutiny, because the mitigation strategies are already in place. They seem smaller, regardless of whether they actually are.

It really comes to an approach. One team assumes it can get the work done right-- but if it doesn't, it'll fix things really fast. One assumes it can get everything right the first time-- and if the project is complete, the work must be right.

I think I'm taking my cue here from Mae West, who reportedly said, "Given a choice between two evils, I always pick the one I never tried before." 

slfisher
Thinkernetter
Friday July 13, 2012 10:56:20 AM
no ratings

and something I'll keep in mind myself for major projects.

that said, I do wonder whether people espousing all these methods have ever read The Mythical Man Month.

RufusJones
Rank: Web master
Friday July 13, 2012 10:07:08 AM
no ratings

Duke, I happen to agree about Agile having value, but let me give you-- and anyone else reading this-- a little bit of ammuntion for those times you go up against people who quote studies.

Everyone who working with surveys and research studies should be required to read this Atlantic Monthly article on Dr. John Ioannidis. Ioannidis is a meta-reseracher-- a doctor who does noting but study the myridal volumes of medical research published every year, to determine the percentage that is bull-puckey.

Ioannidis (who was born in Greece, graduated first in his class at the University of Atehns and then did fellowshipss at Harvard and Tufts, reports that most of those studies saying "eat six ounces of liver a week and drivk white wine and you'll never get cancer" or "this drug shrinks tumors in 97.3 percent of cases" are worthless. 

80 percent of non-randomized studies produce results and recommendations that are false. 25 percent of randomized trials produce false conclusions-- even one out of 10 large randomized trials turn out to be wrong. The issues are that:

  1. The question isn't properly framed
  2. The data set was improperly chosen or arranged
  3. The researcher made basic mistakes in statistical procedure
  4. They employed subjective standards rather than objective ones
  5. They distorted or falsified either the data or the outcomes

And this is medical research-- curing cancer and stuiff like that.

It's invaluable material for anyone who's ever had to deal with Booz, Allen, Gartner or some of the snootier consultants. Become familiar with the concepts underlying the research and you can actually rip the charts to shreds yourtself. Great way to liven up a meeting, and keep your organization from walking off a cliff.

Brian Margolies
no ratings

Rufus, I'm delighted to have amused you. What you say (also amusingly) is true however I still haven't seen the answer to my fundemental question. How does the use of the Agile methodology facilitate faaster delivery of useful functionality when the two most time consuming phases of any development project, analysis and design, are really not directly addressed? 

RufusJones
Rank: Web master
Friday July 13, 2012 9:39:55 AM
no ratings

Brian, your message tickled me to death. Like you, I think Agile is fundamentally a shuck-- a method of project management that lets teams perform activities that any decent, existing methodology already allowed.

Unlike you, I do believe that it has added value in three respects-- by making things that were considered 'optional' in Waterfall (and often not done)  required. Let me pin these to your numbered task list:

2. Implementing the "80-hour" rule.  I have always refused to believe that people are capable of estimating and performing tasks that take longer than two weeks. They overestimate productivity, underestimate obstacles and do not estimate the time needed to perform tasks.

Everyone does this, not just IT people. Look in any cookbook that tells you to carmelize onions, and it wll say to cook them on low heat for "5-10 minutes" or 10-15. As several chefs have written (and anyone with a stopwatch can confirm), it takes at least 20 to bring out the sweetness and flavor without browning them, and if you really want deep flavor, it's 40. People are just allergic to counting, for some reason.

Long timelines for a task also mean there's no way to tell if a project is on track or not. The old joke "The first 10% of the work takes 90% of the time... and the rest taked the other 90%" applies.

So I don't do tasks that take longer than 80 hours. If I have one, it becoems an activity and I break it into tasks that take less than 80 hours. That way I can tell if a project is on schedule. 

Unfortunately, a lot of people do not do that. You see 3-month tasks where the update at the weekly meetings is "We're moving forward" for two months and then panic.

I used to give people heat for this. But, on waterfall, people would condescendingly explain that this was fine, and I just didn't have the experience on large projects in good companies to learn to trust the team.

In Agile, I win those arguments every time. By clamping down on the maximum time length-- two weeks on any sprint-- Agile requires projects to be more manageable. It's a value-add for the methodology.

3. Defining success criteria for every task and requiring the task to end with a tangible deliverable. On a waterfall project, "functionality" is often a vague set of words with some bullet points. Yes, you can drill down and make things very detailed and clear and a good PM will do this... but most projects don't 

Agile requires the "story" to have an ending. They force you to spell out the  tasks (get magic sword, slay dragon, collect cup, visit stream, give flowers to fairy, use cup to get water, give to enchanted pribcess) the project team is required to complete, as well as the rewards. (sword, jewels, cup of life, princess) That means everyone lives happily ever after.

5. It mandates testing and assumes  revision. Again, you can do this in a standard methodology, but it's assumed that real project teams don't need rework-- experrienced people get it right the first time. And when they don't, nobody is willing to admit it and add tasks to the timeline, so the flawed design that doesn't really work proceeds.

On an agile project, you're expected to iterate-- doing a lot of it is sort of a bragging point some places. So people give teams the fish-eye if they aren't taking secoind and third cracks at anything.

End result-- "rework", which is a badge of dishonor, is rebranded as "iteration", whice shows your agile you are.


Do I think, deep down inside, that this is silly? Yup-- and that probably comes through. The condept is nearly as idiotic as rebranding "mainframe architecture" as "cloud computing." If people managed project correctly, Agile wouldn't be necessary-- you could track it and handle it using any of the 367 project methodologies that have come and gone sine I began my career sorting punch cards to run the stats for my father's dissertation.

Do I like the impact these new buzzwords have on projects? You bet your sweet ADSO. Agile is a godsend because people don't manage projects correctly, and Agile views these mistakes as satanic, and is designed to hold these notions up to ridicule.I like that very, very much.

I didn't mention this, but the great thing about "User Stories" are that everyone knows it means programmers have to talk to users. This has its own problems, but it is orders of magnitude better than leaving everything up to propeller-heads. They generally don't have a clue about how the business works and what the end-users need and generally don't bother to learn. 

Agile buzzwords are silly, but they're a lot better than the "go 'way, boy, you bother me!" attitude that produces code where you're delighted if you get a "camel" when you asked for a "horse". Quite often I've gotten Snuffleupaguses and Tasmanian Devils, such as... maybe the package delivery system where one of the keys (a mandatory field that had to be entered in order to save the initial request) was the customer receipt number (given when the customer got the box).

When I pointed out that people would have to enter dummy numbers-- which would probably not be updated when the actual receipt was issued-- I was told "Maybe you don't understand how a waterfall works."

For killing that moronic platitude alone, I say "bully for agile", despite all its "Emperor's New Clothes" style silliness.

Page 1 of 3   Next >
The ThinkerNet does not reflect the views of TechWeb. The ThinkerNet is an informal means of communication to members and visitors of the Internet Evolution site. Individual authors are chosen by Internet Evolution to blog. Neither Internet Evolution nor TechWeb assume responsibility for comments, claims, or opinions made by authors and ThinkerNet bloggers. They are no substitute for your own research and should not be relied upon for trading or any other purpose.
previous posts from Brian Margolies
Brian Margolies
About a month ago, I had the pleasure of participating in an interview for Internet Evolution’s Web radio broadcast. Among the topics generating considerable conversation was the use of social media in a B2B context.
IETV: the thinkerNet on film
5
of
Kim Davis
Big-Data Can’t Always Sell Wine

5|21|13   |   2:23   |   3 comments


Whole Foods Global Wine Purchaser Doug Bell told me about some of the constraints on using analytics in the US wine market.
Paul J. Fleuranges
Digital Signage Keeps NYC Subway Straphangers on Track

5|6|13   |   3:51   |   No comments


New York's Metropolitan Transit Authority is conducting a pilot test of digital kiosks to guide subway users to where they want to go more efficiently and at lower cost.
Kim Davis
Fast Forward to the Future

4|23|13   |   2:29   |   20 comments


A look back at tech writing in the 90s makes us wonder where enterprise IT will be 20 years from now.
Mitch Wagner
Google Launches Its Most Depressing Service Yet

4|15|13   |   2:59   |   10 comments


Google's new Inactive Account Manager lets you control how Google disposes of your accounts when you die.
Second Shooter
Argument Over Top-Level Domains Is 'Stupid'

4|11|13   |   2:07   |   3 comments


The whole Amazon.reader debate is a double-stupid. It's stupid to think that there's any e-book buyer who doesn't know Amazon's URL, and it was stupider to let ICANN launch the whole free-form TLD initiative to start with.
Kim Davis
Ladies, Your Tablet Awaits

3|21|13   |   2:22   |   37 comments


ePad Femme is the world’s first tablet “made exclusively for women.”
Wisdom of the Big Chair
NFC Moves Into the Mainstream

3|20|13   |   2:16   |   No comments


While NFC's original goal was to enhance mobile commerce applications, it is finding its way into a number of other uses, which is creating both opportunity as well as challenges for IT departments.
Wisdom of the Big Chair
Integrating Security Into Your Cloud Contract

3|19|13   |   3:35   |   No comments


Enterprises would like to move to cloud computing but are hesitant because they are concerned about providers’ ability to secure company data. Here are some tips that help to ensure that if breaches occur, the business is not left holding the bag.
Brian Baron
How Edmunds.com Collects Customer Information

3|18|13   |   1:15   |   No comments


Edmunds separates customers into segments based on the info it collects on its site and from partners, and uses that to push out custom content, said Brian Baron, director of business analytics for Edmunds.com, at Predictive Analytics Innovation Summit.
Brian Baron
How Edmunds.com Uses Analytics to Customize Site

3|14|13   |   0:47   |   No comments


The automotive website uses propensity modeling to target ads and customer registration forms, said Brian Baron, director of business analytics for Edmunds.com, at Predictive Analytics Innovation Summit.
an IBM information resource
sponsored content
big blue blog
an IBM information resource
sponsored content
Expert Integrated Systems: Changing the Experience & Economics of IT
In this e-book, we take an in-depth look at these expert integrated systems -- what they are, how they work, and how they have the potential to help CIOs achieve dramatic savings while restoring IT's role as business innovator.

READ THIS eBOOK
your weekly update of news, analysis, and
opinion from Internet Evolution - FREE!

REGISTER HERE
Wanted! Site Moderators
Internet Evolution is looking for a handful of readers to help moderate the message boards on our site – as well as engaging in high-IQ conversation with the industry mavens on our thinkerNet blogosphere. The job comes with various perks, bags of kudos, and GIANT bragging rights. Interested?

Please email: moderators@internetevolution.com
Internet Evolution – not for thickies
Keep Critical Data With a Knowledge Management System
Taimoor Zubair
Fortune 500 companies lose at least
$31.5 billion a year by failing to share knowledge. A Knowledge Management System (KMS) can help companies significantly reduce these costs.

CLICK FOR MORE
M2M: Rise of the Machines? Not Yet
David Weldon
In the 1970 science fiction thriller
Colossus: The Forbin Project, two giant supercomputers from the United States and Soviet Union secretly join forces to take control of the collective nuclear might of the two countries. In the film, the two machines discover each other's existence, communicate back-and-forth, share their collective data, and cut their human creators out of the process. It is the ultimate example of machine-to-machine communications, or M2M.

CLICK FOR MORE
M2M: Rise of the Machines? Not Yet
David Weldon
In the 1970 science fiction thriller
Colossus: The Forbin Project, two giant supercomputers from the United States and Soviet Union secretly join forces to take control of the collective nuclear might of the two countries. In the film, the two machines discover each other's existence, communicate back-and-forth, share their collective data, and cut their human creators out of the process. It is the ultimate example of machine-to-machine communications, or M2M.

CLICK FOR MORE
M2M: Rise of the Machines? Not Yet
David Weldon
In the 1970 science fiction thriller
Colossus: The Forbin Project, two giant supercomputers from the United States and Soviet Union secretly join forces to take control of the collective nuclear might of the two countries. In the film, the two machines discover each other's existence, communicate back-and-forth, share their collective data, and cut their human creators out of the process. It is the ultimate example of machine-to-machine communications, or M2M.

CLICK FOR MORE
M2M: Rise of the Machines? Not Yet
David Weldon
In the 1970 science fiction thriller
Colossus: The Forbin Project, two giant supercomputers from the United States and Soviet Union secretly join forces to take control of the collective nuclear might of the two countries. In the film, the two machines discover each other's existence, communicate back-and-forth, share their collective data, and cut their human creators out of the process. It is the ultimate example of machine-to-machine communications, or M2M.

CLICK FOR MORE
Yahoo Needs to Break Tumblr in Order to Fix It
Joe Stanganelli
As
Mitch Wagner discussed today, Yahoo is acquiring Tumblr. The big Internet debate at the moment is whether Tumblr will be good or bad for Yahoo. Regardless of their stances on the future of Yahoo itself, many claim that Yahoo will somehow ruin Tumblr.

CLICK FOR MORE