Agile Estimation and Planning Rob Morris CDL Systems
Outline 1. 2. 3.
My Background Why do we estimate? How to estimate a. b.
4. 5.
Traditionally Agilely
Firm Fixed Price Estimation The need for Actuals
October 12, 2006
CDL Systems www.cdlsystems.com
Background
Pre 1995 - How do we manage software development projects? 1995 - CDL – small contractor, control station for UAVs 1997 - GD Canada (CDC) – fairly large Canadian defence contractor 1998 - Alliant Techsystems – larger American defence contractor 1999 - AAI – Airplane defence contractor – CMM level 5 2002 - Northrop Grumman – Huge defence contractor, F14, F18, Stealth Bomber CMMI level 5
Agile Manifesto: people over processes
October 12, 2006
CDL Systems www.cdlsystems.com
Why do we estimate?
To Plan – When will something be done? To Schedule – What order should we do things in? To Hire – Do we need more people to do the work? To Price – How much will it cost? To Guide Investment – Is doing something worth it?
October 12, 2006
CDL Systems www.cdlsystems.com
Definitions • • • •
Size – something that can be counted/measured. Hopefully it is representative of effort Effort – the actual hours required to write the software. Duration – the time on the calendar to get something done Cost – strongly correlated with effort, but duration also plays a role
Estimated – our educated guess Actual – measured result
October 12, 2006
CDL Systems www.cdlsystems.com
Estimation is a Probability 0.25
0.2
0.15
0.1
0.05
0 0
October 12, 2006
5
10
15
20
CDL Systems www.cdlsystems.com
25
Cumulative Distribution 1.2
1
0.8
0.6
0.4
0.2
0 0
October 12, 2006
5
10
15
20
CDL Systems www.cdlsystems.com
25
Reality of Estimates Todd Little at Landmark Graphics did a study of 100+ projects. Data coincided with results found by Tom Demarco
October 12, 2006
CDL Systems www.cdlsystems.com
Results
October 12, 2006
CDL Systems www.cdlsystems.com
Results con’t
October 12, 2006
CDL Systems www.cdlsystems.com
Parkinson’s Law
Effort expands to fill the time Time 0.25 0.2
0.15
0.1
0.05
0 0
5
10
15
20
-0.05
October 12, 2006
CDL Systems www.cdlsystems.com
25
Traditional Estimation
Pick a size metric – SLOC, function points Select the value for a variety of cost drivers Experience of requirements analyst Type of project
Estimate the size Crunch the numbers = Effort estimate
October 12, 2006
CDL Systems www.cdlsystems.com
COCOMO II
16 cost drivers Product of the influence of all 16 cost drivers is 1454. Two different projects of the same size (in SLOC)
$10,000 and $14.54 million
Changes in the cost drivers from one project to the next can have a large impact on projects of the same ‘size’.
October 12, 2006
CDL Systems www.cdlsystems.com
Actual Data at CDL Systems
Post mortem analysis within CDL Systems saw variations from 6 LOC/hr to 18 LOC/hr on very similar projects. Factor of 3 with perfect knowledge of SLOC (not a very good estimate)
October 12, 2006
CDL Systems www.cdlsystems.com
Lesson Learned
The client (or even one person) can have a huge impact on cost and it can be completely out of your control. Classify clients like hurricanes, Category 1 to 5. Tardy clients can be costly if they can’t make a decision
Agile – Don’t start developments without requirements for the iteration.
October 12, 2006
CDL Systems www.cdlsystems.com
Agile Estimation
Estimate at 3 scales Iteration Plan Estimation Release Plan Estimation Project Estimation
October 12, 2006
CDL Systems www.cdlsystems.com
Estimation Units
Ideal Time – how long a task takes if there were no interruptions Story Point – relative measurement among Stories
Requirements Feature Lists Use Case Scenario
Velocity – how many estimation units get completed by a team in a single iteration. Be on the lookout for dysfunctional behavior
October 12, 2006
CDL Systems www.cdlsystems.com
Techniques
Analogous Estimation Planning Poker
October 12, 2006
CDL Systems www.cdlsystems.com
Analogous Estimation
This Story is like another Story (maybe a little more difficult, maybe a little less) Give this Story a comparable estimated value. Estimate against multiple Stories of the same effort. Analogous estimation is successful due to our inherent ability to better measure relative size than absolute size.
October 12, 2006
CDL Systems www.cdlsystems.com
Planning Poker 1.
Each estimator is given a deck of cards, each card contains a valid estimate.
2. 3. 4. 5. 6.
Power of 2 – 1,2,4,8,16,32 Fib – 1,2,3,5,8,13,20,30
Story is read and discussed briefly Each estimator selects a card that reflects their estimate. Cards are turned over for all to see. Discussion takes place Re-estimate to try to get convergence. Similar to Wide-band Delphi Research has shown that although planning poker is no better than wide-band Delphi, participants find it fun.
October 12, 2006
CDL Systems www.cdlsystems.com
Iteration Planning
Entire team gets together at the beginning of an iteration. Can include outside stakeholders Estimate each of the items Prioritize the items Use the team velocity to stack the tasks into the iteration.
October 12, 2006
CDL Systems www.cdlsystems.com
Estimating Velocity
Use historical data Run an iteration Make a forecast
October 12, 2006
CDL Systems www.cdlsystems.com
Historical Data
Previous iteration Running average of past iterations Valid, only when little has changed from one iteration to the next
Technology Domain Product Owner Tools Working Environment Team Size and Makeup Estimators
October 12, 2006
CDL Systems www.cdlsystems.com
Run an Iteration
Essentially performs a trial Calibrates the historical data for the current environment.
October 12, 2006
CDL Systems www.cdlsystems.com
Forecast Estimate ideal hours (per developer, per iteration)
Allowances for staff meetings, status meeting, planning session Allowance for new staff, interns, etc. (60% for 6 months) Allowances for training Allowances for stat holidays, vacation, flex time
Be aware of summer projects
Allowances for doctor, dentist, cable guy, Friday lunch 25% overhead to the team lead per developer Eg – 5 person team, 1 intern, 75 hrs/iteration/person
On paper 375 hrs Actually with allowances – 217 (~40%)
Determine ideal hours per iteration Expand stories into tasks, estimate the tasks, fill the iteration.
Tasks are like Lego blocks and must completely fit within an iteration
October 12, 2006
CDL Systems www.cdlsystems.com
Release Planning
Similar to iteration planning but extend the technique to multiple iterations Tasks are generally coarser. Two basic techniques
Schedule can be shrunk somewhat with additional resources but the best that can be hoped for is about 25% improvement in schedule with a 50% increase in staff.
Given a release date, continue filling content until the date is reached. Given content, continue adding iterations until content is completed.
Maximum team size 5 (including lead)
Use forecasting and history if available. Allow time for bug fixing – perhaps allocate an entire iteration
October 12, 2006
CDL Systems www.cdlsystems.com
Firm Fixed Price Estimate
Similar to Rough order of magnitude (ROM), or Not to exceed (NTE) Inherently risky if we use 50% probability estimates Many of us have to live in this world Sole Source and Competitive Bid
October 12, 2006
CDL Systems www.cdlsystems.com
Firm Fixed Price Technique
For each Story produce 2 estimates 50% and 90% probability Subtract and square the difference Sum the squares Square root of the sum Add sum to 50% sum
October 12, 2006
CDL Systems www.cdlsystems.com
Example Story
50% (days)
90% (days)
Δ2
Plan Flight
15
25
100
Control Weapons
5
10
25
Launch Vehicle
5
7
4
Land Vehicle
20
40
400
Total
45
Total = 45 + √529 = 68
October 12, 2006
CDL Systems www.cdlsystems.com
529
Advantages
The greater the difference between the 90% estimate and the 50% estimate is an indicator of risk Provide developers with the 50% numbers to complete the tasks. 50% and 90% confidence can be tested for accuracy by looking at actuals for all tasks.
October 12, 2006
CDL Systems www.cdlsystems.com
Scope Creep and Rework
Scope Creep – new requirements that were never allocated for in the estimate Rework – Doing work more than once because client couldn’t make up their mind Agilist accept these as a fact of life, but they are still require effort
October 12, 2006
CDL Systems www.cdlsystems.com
Scope Creep and Rework
Keep track of out of scope and rework items Every item that is added to your backlog qualify it as in scope, rework or out of scope Treat them like any other backlog item Even if you can’t recapture out of scope and rework in this contract, beware of this contractor and adjust estimates in subsequent contracts with this contractor accordingly Trying to tighten loopholes in contracts doesn’t work – leads to red tape and animosity Important to keep track of to determine the accuracy of the original estimate
October 12, 2006
CDL Systems www.cdlsystems.com
Actuals
Pure Agilist – Actuals aren’t necessary
Scrum – more concerned with time remaining to create a burn down XP – weaker concept of ‘Yesterday’s weather’ which is just number of story points completed by a team in an iteration.
Use of velocity somewhat circumvents need for actuals, but it is course grained. Velocity is a team actual
October 12, 2006
CDL Systems www.cdlsystems.com
Need for Actuals
Firm Fixed Price Bidding needs Actuals Need the historical data as a frame of reference for forecasting When teams are shuffled, individual velocity can aid in estimation Team concept is great, however we don’t pay teams, we pay individuals Individual accomplishments should not be overlooked. Actuals provide to staff to improve their estimates on subsequent iterations
October 12, 2006
CDL Systems www.cdlsystems.com
Final Words
Being agile does not circumvent need for estimates Traditional ‘size to effort’ techniques have their problems (cost driver sensitivity, size estimation) Can still use agile techniques in Firm Fixed Price environments Clients are a huge cost driver – make allowances for the client Still need actuals
October 12, 2006
CDL Systems www.cdlsystems.com
References
Software Estimation – Steve McConnell Agile Estimation and Planning – Mike Cohn http://www.toddlittleweb.com/ - Todd Little
October 12, 2006
CDL Systems www.cdlsystems.com