Agile Estimation Methods

The Cone of Uncertainty is a Project Management term used to describe the level of uncertainty existing at different stages of a project. We become more certain of our estimates as we learn more about what we’re estimating.

In Agile Project Management environments, we accept that software projects are ridden with uncertainty it comes with the territory. This uncertainty decreases as decisions are made around scope and approach the more we understand, the more certain we can be about our estimates.

To mitigate the risk of incorrect effort estimations we reduce the precision of our estimates according to how much we know about what we’re estimating. This in turn helps us to be more accurate.

Agile Requirements in order of Uncertainty

• Agile Theme
• Agile Epic
• Agile User Story
• Agile Task

Agile Estimation Techniques in order of Precision

• Hours
• Story Points
• T-shirt Sizing

Estimating in Hours vs. Fibonacci
As described in earlier post Software Estimation: The more precise you are, the less accurate you will be, studies suggest that estimates taking place during the Product Definition Phase exist somewhere between being 300% less and 0.75% longer than it actually took them to build the software i.e. out by a factor of x16. The cone of uncertainty narrows as you eliminate uncertainties.

In Agile environments, we use two main estimation metrics Fibonacci and Hours.

Estimation using hours is pretty self-explanatory, however it’s worth mentioning that Agile Teams only estimate Tasks in hours. By Tasks, I mean ‘parts of a User Story’ something very small that should take a couple of hours or so definitely no longer than a single day to complete. Because larger pieces of work are generally more complex and should/could probably be broken down into smaller component parts AKA there may still be a number of outstanding decisions that could affect the level of effort required to deliver the task. The larger the task, the lower the chances that you’ll be able to estimate the exact number of minutes/hours associated with delivering it e.g. Phone interruptions, toilet breaks, delayed meetings. In other words estimate SMALL things in time units.

Estimating in Fibonacci is the second Agile estimation metric - we call this Story Point Estimation. We use Story Points to estimate larger pieces of work i.e. User Stories and Epics. They work as follows:

0,1,2,3,5,8,13,21,34,55,89

In other words, a ’5 is 5x more effort than a ’1 and an ’8 is 8x more effort than a ’1.

Story Points can measure things we can’t measure in hours e.g. complexity do you include a task for every discussion you don’t yet know you need to have, every time you take 10 minutes out to Google an answer? It is however relatively easy (when you get started) to compare the size of one task/cluster of tasks with another task/cluster of tasks. Estimating in Story Points allows you to take into consideration all sorts of intangible ‘things’ that you sense but can’t quite put your finger on.

Estimation Using T-Shirt Sizes
T-shirt Sizing is also referred as an Agile Estimation method by many enthusiasts it’s used to estimate larger requirements i.e. Epics, but maybe the odd User Story also. In short, you attribute a number of story points to a t-shirt size that is for small it's X number of points an XXL might equal ’55 points’. T-shirt sizes are great for Product Owners and/or non-technical people as they’re totally abstract and non-threatening (that’s not meant to sound patronizing you know what I mean!). They’re easy to understand.

When estimating in T-shirt sizes, it’s still important to set your scale agree in advance what constitutes a ‘Small’, Medium, ‘Large’, XLarg and ‘XX Large’.

T-shirt sizing will normally take place at the Requirements Workshop this helps the Product Owner and Product Manager get a sense of scale, which will in turn help with the prioritization process. As always, the Scrum Team are the people that assign t-shirt sizes to Epics. The Product Owner and/or Product Manager are not allowed to participate in the estimation process they can offer insight and guidance but the estimations belong to the Scrum Team.

Estimation Using Story Points

Once the Epics have been estimated and prioritized for delivery, they will be broken down into User Stories by the Product Manager and Product Owner. The component User Stories will then be introduced at a subsequent requirements workshop and estimated in Story Points at Poker Planning.

Sprint Planning: Hours or Story Points , there is a lot of debate about whether to estimate sprint requirements in hours or to leave them in Story Points.
Mike is big on breaking User Stories down into tasks, which are then estimated in hours. Mike discusses this in his post: Why I don’t use story points for sprint planning.

Jeff claims that the best teams [he works] with burn down story points. They only burn down when a story is done.

Personally, we see pros and cons of both approaches but would use Story Points over hours where/whenever possible.

Mike claims that he doesn’t use story points for sprint planning because story points are a useful long-term measure. They are not useful in the short-term. I don’t agree with this statement as a whole. Story Points are indeed a useful long-term measure, however I believe they can also be useful in the short-term. When writing user stories, you should aim for each story to comply with the ‘INVEST’ acronym Independent, Negotiable, Valuable, Estimatable, Small and Testable. If each story is Small enough to be ‘accurately’ estimated, and Testable enough for one to create Test Confirmations then there may be little benefit achieved through breaking it down into smaller composite parts or re-estimating them in hours. In fact, back in the days when we were ’Doing Agile’, one of our teams used to create task cards with the words just do it written on them In this particular situation, the team were not benefiting from the time spent re-writing task cards and re-estimating the stories each story was taking no longer than 1-2 days to complete. We can baseline this for longer , efficient & better calibrations etc.

The main concerns we had when we initially discussed the idea of burning down in hours was that we’d not benefit from the early warning signs provided by the burn down and that we would only discover a story was taking longer than expected when it was too late.

The reality of the situation is that the team soon discovered their ‘Agile Heartbeat’ - in order to stay ‘on track’, they need to burn down +/- one story per day. Also, the team members generally know when something is taking longer than expected their communication is strong and they raise any concerns at the Daily Scrum.

With that said, if the team is less established, AND/OR if the stories are much larger AND/OR if the sprint duration is +/- 1 week or less then I’d be inclined to estimate in hours as they have the potential to provide a more granular progress snapshot.

I do agree with this point, in the sense that nothing is definite- you can’t be 100% certain that you’ll finish everything you commit to. With that said if a team has a velocity of 20 story points, aren’t they likely to commit to delivering +/-20 story points worth of stories in the next sprint? AND if they are true to their velocity, then surely they’ll deliver them will they not?

On the other hand, it is still important for the team to be comfortable and confident that they can deliver their commitments if they only accept 15 points one sprint then so be it. This discussion is about whether or not you can achieve the same end result burning down on story points vs. hours I think you can.

Value Points Estimating the relative value of a User Story

Value Points can be used to measure the relative value of User Stories using a Fibonnaci Sequence. Although not appropriate or particularly necessary in all cases, Value Points can help you to prioritize a product backlog and convey the relative importance of User Stories to a Software Development Team. This approach becomes increasingly useful as teams scale to cover multiple projects combined with Business As Usual activities.
If you are a Product Owner or a Programe Manager, you will be all to familiar with Return on Investment (ROI) models, Net Present Value(NPV) and Internal Rate of Return(IRR).
In short:

ROI measures how much money you will make off the back of an investment.
NPV measures the desirability of an investment compared to the minimum expected return over a pre-defined number of years.
Internal Rate of Return measures the percentage return of an investment after tax.

The traditional ROI model approach is useful when applied at a project level, particularly when attempting to measure viability and/or secure funding, however it is time consuming, requires business analysis skills and is wrapped up in a whole host of assumptions.

In Agile Software Development, we break projects down into Themes, Epics, User Stories and Tasks. We then define a minimum feature set (Minimum Marketable Feature(s)) and attempt to schedule development in a way that maximizes ROI early on.

So, what happens when you need to measure the value of a User Story? Do you produce an ROI model? What if there’s no direct financial return associated with the story? The reality is that this would be an unrealistic expectation due to 1) the time/effort involved and 2) the level of granularity that would be required.

So, what to do?

I started using Value Points in hopes to simplify the process of measuring relative value. This scale uses the standard Fibonacci Sequence: 1,2,3,5,8,13,21,34,55,89 as a metric.

In the same way that Story Points cannot be converted to Hours, Value Points cannot be converted to Financial Return. Because when we measure value, particularly across Business As Usual, there are so many reasons for doing something not all of them will offer an immediate financial return:

Strategic : if we don’t make this investment, we will no longer be able to maintain our market share
Compliance: if we don’t make this investment, we could be in breach of the law(s)
Revenue Generation : if we don’t make this investment, we’ll miss out on a chance of making X money
Risk Management : if we don’t make this investment X could happen to us e.g. stakeholder dissatisfaction etc.
Cost Avoidance: if we don’t make this investment, we could realize additional costs in the future
etc

Value Points can offer a useful tool to help with the prioritization process e.g. you can plot effort against value to maximize delivery of those stories delivering the highest amount of value for each unit of effort.

Value Points can also empower the development team to identify the most appropriate solution for a particular card e.g. the value’s low, so I’ll not spend as much time on it as one of the higher value cards.

I’ve created a Product Backlog Template that provides some insight into the health of your product backlog the idea is to pack it with as many low-effort/high-value stories as possible.

This template uses a simple formula to help you prioritize your stories:

Value Points - Story Points = Priority Points

It also allows you to do the following:
Assign stories to sprints (release plan) : Calculate the number of releases required to deliver your backlog based on your velocity Calculates how many sprints you can afford to deliver based on your project budget Plots your user stories on a chart based on Effort vs. Value

Re-estimating stories in hours usually comes down to estimating tasks that were identified for each story and as such they become at some point incomparable. So we lose the ability to use relative estimates as tasks fall into different types of activities. This way we need to think about hours required for each task individually and maybe comparing them to other tasks of the same type (like designing UI, writing JavaScript code, etc.) may help in estimating them. I would strongly recommend such approach because this way you can notice where your estimations are more guessing than really touching the true value (if the story point is very big it means that you have to break it into smaller ones).

Who makes those initial estimates? Product owner + team together? Initial Product owner then at contract stages the team must re-visit or re-estimate everything again.

When exactly to they produce those estimates? Before the 1st sprint? Estimation is a live document that is need to be reviewed constantly whenever the changes would likely impact the project plan, schedule. This will also mitigate the risks of scope schedule / creep.

Also to add, The number of story points that a team can complete (on average) in one sprint is known as their ‘velocity’. Before you start estimating stories, make sure that everyone agrees on what a 1 point, 3, point, 5 point (etc) story ‘looks’ like. Some teams will bring these examples along to their planning poker workshops as a reminder until they get used to their scale. So, for example, all 3 point user stories should on average take the same amount of effort to deliver. With that in mind, assuming each sprint has the same amount of effort available (same number of people in the office, with the same skill set (ideally the same people!), for the same number of days) then the same number of points will generally be delivered in the pre-agreed number of days. It usually takes a few sprints before a team can get a sense of their true velocity. This velocity then acts as a starting point for negotiation more or fewer points may actually be committed to depending on other influencing factors (e.g. team members on holiday etc.)