|
Having spent a significant portion of my early career as a business analyst, I have always been keenly interested in potential means of delivering functionality faster to my users. As a long-standing practitioner of the traditional "waterfall" software development/project management methodology, I have integrated early prototyping and iterative functionality over the years.
Recently, I've become aware of a growing trend toward the adoption of agile software development and project management methodologies. I began investigating agile methods to understand if this trend offered a real breakthrough in delivery speed -- the elusive holy grail to IT organizations (not to mention our business partners).
The agile methodology was designed to speed up the delivery of developed capabilities (systems, applications, and functions). Time to market is important to all aspects of business systems development, but it is especially critical to customer-facing e-commerce systems.
I have seen the agile methodology described as involving "early and regular delivery of working software, a focus on team communications, and close interaction with the users."
Seems familiar, no? Isn't that what we have always strived to achieve? Agile methodology, it seems, is founded on a philosophy that espouses people and their interaction over methodological processes, tools, and documentation. Remember people, process, and technology? To the casual observer, this methodology seems more of an evolutionary step.
In reality, it appears to have borrowed from and recombined the best-practices that have been used in software development over the past 40 years. Essentially, the agile process for a software development project includes the following steps:
- The overall scope of the project is assessed.
The business determines the overall priorities, and the development team estimates the effort required.
- A fixed iteration or "sprint" length is determined.
Two weeks appears to be a common length for iterations.
- Functions determined to be in scope are broken into sets of high-level user stories (capabilities).
Before implementation within a sprint, the stories are fleshed out at a level of detail sufficient for each function to be implemented within a single sprint. (Use cases, anyone?)
- Sprint planning starts.
Stories are gathered until a full sprint's worth of functions are identified. Naturally, high-risk/priority functions are scheduled for implementation sooner in the development sprint cycle.
- Each story (capability) is refined.
Specific tasks are deteremined, as we noted in step No. 3.
- Functions are developed, tested, and rolled out into production.
This would happen in any waterfall-based project. Presumably, the process cycles back to step No. 4, concurrently with this step.
The primary differences between the waterfall and agile methods include the frequency of functionality delivery and the formalized iterative approach in the agile method. This method has even developed its own jargon -- participants are classified as "chickens" and "pigs" (in a bacon-and-eggs breakfast, chickens are involved, but pigs are committed).
Conversely, waterfall development goes through a series of distinct phases: analyses, requirements, design, implementation/integration, testing, and rollout for the entire set of functions. This has also come to be nicknamed "the big bang."
Having used an iterative approach to software delivery (aka phases) for years, I find myself at a loss to determine the agile value proposition. I can see how it provides a codified methodology for iterative development, including how late business requirements can be incorporated (no mean feat). What isn't clear is how it addresses and improves upon the two most time-consuming activities of any software development effort: analysis and design.
If there is truth in the hackneyed expression "The devil is in the details," then how does employing an agile methodology avoid the time needed to analyze and design holistically? Wouldn't deeper requirements, specifications, and design performed just before sprint implementation reveal details that could result in more expensive development or force you to undo and then redo previous functions?
One could never say these things wouldn't happen using a waterfall approach, but it seems that the agile approach tacitly invites them to occur. I think it can be a helpful methodology, but the holy grail remains elusive.
Related posts:
— Brian Margolies is the CIO of Allied Beverage Group, New Jersey's largest distributor of wine and spirits.
IQ Crew
Tuesday July 31, 2012 11:53:32 PM
I do have a quick question. When distributing beverages, who keeps tabs of loss prevention (supply/demand at retailer) and reports it back to the mfr?
is the role of the distributor strictly poiint a to point b with no ties to data?
Rank: Scrivener
Friday July 13, 2012 4:49:49 PM
I thouyght the whole point of agile is that it gives the intended user more of a voice in the development process. This user is invited to see something, albeit unfinished, that works and then provide feedback and thus steer the development team more precisely towards a maximally useful and well-done finished product. The quicker you can show users something that works, the better chance you have of delivering something useful. If it's not useful in some respect, you'll find out faster and therefore be able to take corrective action more quickly than you might otherwise. The design is a constant work in progress, vs. the waterfall's all-done--at-once approach.
Hey, I'm no programmer, at least not in any professional capacity, but this seems fairly obvious to me. But perhaps I am missing something.
Rank: Web master
Friday July 13, 2012 2:16:47 PM
No problem, Brian. It helps to have comments from people who haven't drunk the Kool-Aid.
My take: Any methodology, if used properly and rigorously, can be an effective tool for managing projects. No methodlogy is invulnerable to poor execution.
I've worked at three sites that called themselves Agile, seven that claimed to be in various stages of adoption. All seemed related, but none were especially alike. The variations weren;t as bad as the workplaces that called themselves "Six Sigma", but not as tight as the Crosby folks.
I would bet my life that Agile is a fad that will be "refudiated" in favor of something else.... but that's a standing bet about any metholology of any kind that becomes popular. (Currently got a pool going on the date everyone abandons ITIL and Cloud.)
Thinkernetter
Friday July 13, 2012 1:56:53 PM
Thank you Rufus, that's about as close to a useful answer as I've seen.
Rank: Web master
Friday July 13, 2012 11:18:24 AM
Almost certainly not. It was published in 1975, and people assume that the lessons in it must be useless because the technology has changed so greatlly.
The notion that human beings and organizations-- the constants in the development process-- have not changed one iota is lost on them.
It's one of the big misconceptions people make about dealing with ideas. If an idea failed for purely technical resons (the obvious example being Babbage's "analytical engine"), there is a good chance that it might work better 10, 20 or 40 years later. It wasn't that computers couldn't play chess well-- just that earlier computers couldn't do the number of calculations necessary to do it well.
If the idea failed for human reasons, it's fairly likely it will fail again, unless you can identify some trigger for a change in behavior.
Rank: Web master
Friday July 13, 2012 11:05:11 AM
Brian, the presumption underlying Agile is that the company:
- Has existing assets (e.g., data) that it will use (or at least have to be used).
- Knows what those assets are and has them properly documented in a repository.
- Already has accurate and detailed maps of the processes.
- Agrees on the critical issues-- at least at a high level--that need to be addressed.
That activity and consensus needs to take place before the project planning begins. Obviously, if that hasn't been done, the process can go off the rails. But that's true of any project.
The desgin part is the organization of the sprints. I'd describe it as a type of process mapping. As you sort and sequence the user stories together, it becomes clear that other stories will need to be written.
"I want to be able to add these customers on the fly" means you will need some module to validate them against existing customers. That module becomes a story-- but it gets treated as a black box for the first story's purpose.
If not knowing how the validation works doesn't sound like a good idea-- you're worried about what could happen if it isn't robust-- then obviously that story has to be done first. But once done, that's an exixting asset.
The concept behind Agile is that the stories and modules-- rather than being consigned to binders that go unread for all eternity-- go into a repository that becomes both a process map and collection of assets. That's becomes the heart of the design and analysis for future projects.
It's not done all at one time in one formal "analysis and design phase", but the collective wisdom and rich heritage of the many project teams that have come before you grows together over time and becomes one in a unique harmonic convergence that leads to world peace, universal love and free dope, allowing us to talk to the animals and make contact with other worlds...
Look, if you drill down on Agile it gets sappy in a hurry. But it's not any less inance than a project methodology that says "We can understand how everything this enterprise function in an 8-12 week discovery plase where we:
- Get our information from cadging meetings with people who don't have time to talk to us and don't really want to,
- Might or might not (most likely 'not' given turnover, downsizing and use of contractors and vendors) speak to people who have performed the tasks they're allegedly subject matter experts about.
- Might or might get people who can remember (or be willing to explain, for time or political reasons) why it was done and what the decision-making process was.
- Can deem anything that looks like a bag of snakes "out of scope"
That produces bad design and analysis too-- just in a form that is considered acceptable because we've dealt with it for decades and have come to assume that it can't be done better.
As a rule, people tend to focus with laser-like precision on the flaws in any new paradigm. Those flaws, because their scope and impacts are unknown, draw great concern. The flaws in the existing process rarely get that same level of scrutiny, because the mitigation strategies are already in place. They seem smaller, regardless of whether they actually are.
It really comes to an approach. One team assumes it can get the work done right-- but if it doesn't, it'll fix things really fast. One assumes it can get everything right the first time-- and if the project is complete, the work must be right.
I think I'm taking my cue here from Mae West, who reportedly said, "Given a choice between two evils, I always pick the one I never tried before."
Thinkernetter
Friday July 13, 2012 10:56:20 AM
and something I'll keep in mind myself for major projects.
that said, I do wonder whether people espousing all these methods have ever read The Mythical Man Month.
Rank: Web master
Friday July 13, 2012 10:07:08 AM
Duke, I happen to agree about Agile having value, but let me give you-- and anyone else reading this-- a little bit of ammuntion for those times you go up against people who quote studies.
Everyone who working with surveys and research studies should be required to read this Atlantic Monthly article on Dr. John Ioannidis. Ioannidis is a meta-reseracher-- a doctor who does noting but study the myridal volumes of medical research published every year, to determine the percentage that is bull-puckey.
Ioannidis (who was born in Greece, graduated first in his class at the University of Atehns and then did fellowshipss at Harvard and Tufts, reports that most of those studies saying "eat six ounces of liver a week and drivk white wine and you'll never get cancer" or "this drug shrinks tumors in 97.3 percent of cases" are worthless.
80 percent of non-randomized studies produce results and recommendations that are false. 25 percent of randomized trials produce false conclusions-- even one out of 10 large randomized trials turn out to be wrong. The issues are that:
- The question isn't properly framed
- The data set was improperly chosen or arranged
- The researcher made basic mistakes in statistical procedure
- They employed subjective standards rather than objective ones
- They distorted or falsified either the data or the outcomes
And this is medical research-- curing cancer and stuiff like that.
It's invaluable material for anyone who's ever had to deal with Booz, Allen, Gartner or some of the snootier consultants. Become familiar with the concepts underlying the research and you can actually rip the charts to shreds yourtself. Great way to liven up a meeting, and keep your organization from walking off a cliff.
Thinkernetter
Friday July 13, 2012 10:01:41 AM
Rufus, I'm delighted to have amused you. What you say (also amusingly) is true however I still haven't seen the answer to my fundemental question. How does the use of the Agile methodology facilitate faaster delivery of useful functionality when the two most time consuming phases of any development project, analysis and design, are really not directly addressed?
Rank: Web master
Friday July 13, 2012 9:39:55 AM
Brian, your message tickled me to death. Like you, I think Agile is fundamentally a shuck-- a method of project management that lets teams perform activities that any decent, existing methodology already allowed.
Unlike you, I do believe that it has added value in three respects-- by making things that were considered 'optional' in Waterfall (and often not done) required. Let me pin these to your numbered task list:
2. Implementing the "80-hour" rule. I have always refused to believe that people are capable of estimating and performing tasks that take longer than two weeks. They overestimate productivity, underestimate obstacles and do not estimate the time needed to perform tasks.
Everyone does this, not just IT people. Look in any cookbook that tells you to carmelize onions, and it wll say to cook them on low heat for "5-10 minutes" or 10-15. As several chefs have written (and anyone with a stopwatch can confirm), it takes at least 20 to bring out the sweetness and flavor without browning them, and if you really want deep flavor, it's 40. People are just allergic to counting, for some reason.
Long timelines for a task also mean there's no way to tell if a project is on track or not. The old joke "The first 10% of the work takes 90% of the time... and the rest taked the other 90%" applies.
So I don't do tasks that take longer than 80 hours. If I have one, it becoems an activity and I break it into tasks that take less than 80 hours. That way I can tell if a project is on schedule.
Unfortunately, a lot of people do not do that. You see 3-month tasks where the update at the weekly meetings is "We're moving forward" for two months and then panic.
I used to give people heat for this. But, on waterfall, people would condescendingly explain that this was fine, and I just didn't have the experience on large projects in good companies to learn to trust the team.
In Agile, I win those arguments every time. By clamping down on the maximum time length-- two weeks on any sprint-- Agile requires projects to be more manageable. It's a value-add for the methodology.
3. Defining success criteria for every task and requiring the task to end with a tangible deliverable. On a waterfall project, "functionality" is often a vague set of words with some bullet points. Yes, you can drill down and make things very detailed and clear and a good PM will do this... but most projects don't
Agile requires the "story" to have an ending. They force you to spell out the tasks (get magic sword, slay dragon, collect cup, visit stream, give flowers to fairy, use cup to get water, give to enchanted pribcess) the project team is required to complete, as well as the rewards. (sword, jewels, cup of life, princess) That means everyone lives happily ever after.
5. It mandates testing and assumes revision. Again, you can do this in a standard methodology, but it's assumed that real project teams don't need rework-- experrienced people get it right the first time. And when they don't, nobody is willing to admit it and add tasks to the timeline, so the flawed design that doesn't really work proceeds.
On an agile project, you're expected to iterate-- doing a lot of it is sort of a bragging point some places. So people give teams the fish-eye if they aren't taking secoind and third cracks at anything.
End result-- "rework", which is a badge of dishonor, is rebranded as "iteration", whice shows your agile you are.
Do I think, deep down inside, that this is silly? Yup-- and that probably comes through. The condept is nearly as idiotic as rebranding "mainframe architecture" as "cloud computing." If people managed project correctly, Agile wouldn't be necessary-- you could track it and handle it using any of the 367 project methodologies that have come and gone sine I began my career sorting punch cards to run the stats for my father's dissertation.
Do I like the impact these new buzzwords have on projects? You bet your sweet ADSO. Agile is a godsend because people don't manage projects correctly, and Agile views these mistakes as satanic, and is designed to hold these notions up to ridicule.I like that very, very much.
I didn't mention this, but the great thing about "User Stories" are that everyone knows it means programmers have to talk to users. This has its own problems, but it is orders of magnitude better than leaving everything up to propeller-heads. They generally don't have a clue about how the business works and what the end-users need and generally don't bother to learn.
Agile buzzwords are silly, but they're a lot better than the "go 'way, boy, you bother me!" attitude that produces code where you're delighted if you get a "camel" when you asked for a "horse". Quite often I've gotten Snuffleupaguses and Tasmanian Devils, such as... maybe the package delivery system where one of the keys (a mandatory field that had to be entered in order to save the initial request) was the customer receipt number (given when the customer got the box).
When I pointed out that people would have to enter dummy numbers-- which would probably not be updated when the actual receipt was issued-- I was told "Maybe you don't understand how a waterfall works."
For killing that moronic platitude alone, I say "bully for agile", despite all its "Emperor's New Clothes" style silliness.
The ThinkerNet does not reflect the views of TechWeb. The ThinkerNet is an informal means of communication to members and visitors of the Internet Evolution site. Individual authors are chosen by Internet Evolution to blog. Neither Internet Evolution nor TechWeb assume responsibility for comments, claims, or opinions made by authors and ThinkerNet bloggers. They are no substitute for your own research and should not be relied upon for trading or any other purpose. |
|
|
|
previous posts from Brian Margolies
About a month ago, I had the pleasure of participating in an interview for Internet Evolution’s Web radio broadcast. Among the topics generating considerable conversation was the use of social media in a B2B context.
IETV: the thinkerNet on film
Brian Baron How Edmunds.com Collects Customer Information 3|18|13 | 1:15 | No comments
Edmunds separates customers into segments based on the info it collects on its site and from partners, and uses that to push out custom content, said Brian Baron, director of business analytics for Edmunds.com, at Predictive Analytics Innovation Summit.
an IBM information resource
sponsored content
big blue blog
For me, it's always a good day when IBM announces one of its new studies.

an IBM information resource
sponsored content
Expert Integrated Systems: Changing the Experience & Economics of IT
In this e-book, we take an in-depth look at these expert integrated systems -- what they are, how they work, and how they have the potential to help CIOs achieve dramatic savings while restoring IT's role as business innovator.
READ THIS eBOOK
your weekly update of news, analysis, and
opinion from Internet Evolution - FREE!
REGISTER HERE
Wanted! Site Moderators
Internet Evolution is looking for a handful of readers to help moderate the message boards on our site as well as engaging in high-IQ conversation with the industry mavens on our thinkerNet blogosphere. The job comes with various perks, bags of kudos, and GIANT bragging rights. Interested?
Please email: moderators@internetevolution.com
|