Estimate a release in Scrum

In Agile, time and again people ask this question "How would you estimate a release for a product?". When it comes from management they want to know the following:

• Calculate the man hours which is at their disposal to get to the release date
• Risk assessment
• How many people we need, can we expedite the development if we pump in more man power?

There are many more questions. In some cases they want to use some complex prediction algorithm to come up with the release date. Oh!! My god, I think, at the end of the day who is going to implement the features? Can an algorithm do an exact implementation? Can you use the available man hours?
 
I would say no. There is danger in using man hours. We don't take into account the diversity of a team. It can have people with experience levels. A person with ten years of experience may be able to finish a task in four hours, while another person with five years may take a day and someone less experienced may take two or more days. How does an algorithm include these differences? Or the team may be comprised of a tester, developer, BA etc. Each one has different work to do and you can't consider everyone to complete the different kind of work in the same hours at their disposal. Many a time management either doesn't understand or doesn't want to come out of its traditional way of running business.
 
There is an easy solution to this. No fancy algorithm, no man hour calculation, nothing. If you are aware of Agile concepts then what I am going to say may sound familiar. You can use the velocity of the team to come up with more accurate predictions. But before we see how we can predict release, we need to understand what velocity is.
 
Velocity is the story points a team can deliver in each sprint. A sprint can be anywhere between two to four weeks. I like to have smaller sprints of two weeks. Thinking that a sprint is of two weeks in length, let's examine further how velocity is calculated. Below is a table showing the sprint and the story points achieved in each sprint.

Sprint #	Story points per sprint	Velocity
Sprint 1	1	1
Sprint 2	3	(1+3)/2 = 2
Sprint 3	3	(1+3+3)/3 = 2.3
Sprint 4	5	(1+3+3+5)/4 = 3
Sprint 5	3	(1+3+3+5+3)/5 = 3

So from the above table you can see how a team's velocity is calculated. In the first sprint (two weeks) it achieved one story point (SP). In the second it gained three SPs. So velocity is the total number of SPs achieved in all sprints divided by the number of sprints. In the third sprint the velocity was 2.3 and by the end of fifth sprint the velocity stabilized to three SPs. With that knowledge of what velocity is, let's tackle the problem of calculating a release date for a set of features.

There are two ways to do this. One is where the team has historical data to calculate velocity and second where the team is new and there is no historical data. Let's see the easy one first.

When the team has Velocity data?
This situation is applicable for teams that have been working on products for a long time and has historical SP data to calculate its velocity. The product owner gets a set of ten new features from the client and he wants to know how quickly the team can deliver these.

The solution is simple. Give the ten new requirements to the team and ask them to come up with estimate. Here is the process which needs to be followed.

1. The team (dev, QA, BA) gets into a quick release planning (not really release planning) meeting with the product owner.
2. The team and Product Owner (PO) go through the list of features and try to understand each one of them. There is no need to know the nitty gritty of the requirements as they may be pulled out in future. The team needs to get enough information so that it can estimate.
3. Once the team has enough understanding about a feature it can estimate the user stories (features). The estimate is done in terms of SPs. The team plays planning poker. Below is the table showing the SPs estimated by the team for the features.

Features	Story points
Feature 1	5
Feature 2	3
Feature 3	3
Feature 4	8
Feature 5	5
Feature 6	8
Feature 7	3
Feature 8	3
Feature 9	5
Feature 10	2

1. After all the ten features have been estimated calculate the total SPs for the ten user stories. From the above table the total SPs is 45.
2. From the first table we know the team's velocity is 3 and the total SPs estimated for 10 stories is 45. If the team can deliver three SPs a sprint then to deliver 45 SPs it will take 15 (45/3) sprints. 15 sprints = 30 (15 * 2) weeks. 30 weeks = 7.5 months. So the team will take 7.5 months to finish the ten new features. To this you need to add the regression, stabilization and any other phases required for delivering a quality product. In essence the time taken is 7.5 months + Time required for Regression + any other stage.

For fairly new team that doesn't have historical data

1. Follow the above steps from 1 to 4. Let's go with the same 45 SPs as the estimate.
2. Now we don't have historical data to calculate the release date. What can be done? The team can decide and assume a number as its velocity. Say the team has decided two SPs as its velocity. Based on the team's assumption of its velocity you can calculate the release date.
3. As the sprints end the team can start re-calculating its velocity and adjust the release dates.

From my experience a team reaches a stable velocity within four to six iterations. So in two months you can exactly predict the release date.

What happens if we don't meet the target?
During the release planning the team needs to fix dates on feature freeze and code freeze based on the overall estimate. By the time you reach the feature freeze date the team will have a good idea on whether it is going to deliver all the promised features. If the team is not able to keep the commitment -- if it has promised ten features and only eight will be completed --then the team members need not panic. The approach would be to ask the product owner to start negotiating with the client. Either the client has to accept the product with eight features or extend the time to get the whole set of features. This is quite normal and from my experience if you have a good product owner who has good negotiating skills the client will accept any of the above options.
Another advantage to Agile is that the client need not wait for years to get features. At the end of every end of the sprint the client gets a feature delivered. And I have seen except for few scenarios, client are quite accommodative. The reason is that client can see the ROI every two weeks and this actually builds confidence.

What happens if the client cuts short the estimate from x to y?
The answer is simple. You tell the client that X will be the time taken to deliver ten features and if you cut the time short then in Y time frame you can deliver only six features.

What happens if the client adds new requirements?
If the client adds more requirement and wants it in the same timeframe, then the team needs to tell the product owner that if you add new features then out of the previously committed ten features we may have to drop few to accommodate the new ones. This can happen only when prioritization comes into play.
The product owner and client need to decide the priorities for the features. Based on the priorities the team will tackle higher priority items first. Once the higher priorities are targeted it will pick up the lesser priority items. The product owner either needs to buy more time to include the new features or need to get few items off the list so that new items can be accommodated in the timeframe.

Documentation in Agile

Documentation is a big contentious issue in Agile. There is a reason for this. When you start your presentation on Agile you start by going through the 4 statements of agile manifesto. As soon as you go to the second statement, traditional waterfall guys start frowning their eyes. The second statement in the manifesto says this.

Working software over comprehensive documentation

Seeing the statement people assume that documentation is not part of the agile process. It completely ignores documentation. Here comes questions. Without documenting the process flow how do you know what is the process we need to follow? Without document how do you know what is the requirement? Without document how do you know what has been implemented? Without document how is a new dev/qa going to understand what is coded/system all about? And if it is someone from CMMi background, then god can only help you to convince him/her.
The key thing people forget is that processes are there to help people get their work done faster. Deliver software faster to the end user. Faster delivery will help you capture market compared to your competitors. In software development you are not delivering document to the end user rather a working software. The above statement never said documentation is not needed, just that the emphasis is on delivering value to the end user.
Now let’s try to address some of the questions raised above.

Process flow documentation
Agile is well documented. If you search the internet you will get not hundreds but thousands of documentation. Just put a few links to the website and that forms your document. I would say there is no need to re-create something which is freely available on the net. If you really want to have off-line document then use the save feature of your browser and save the pages. Make sure you have the permission to do the same. Or copy paste the contents and put it in a document with a link to the original article.
One of the best way of explaining your process is to have a pictorial representation of the same. If you search agile process in Google you will find lot of high quality diagrams. Click here to see few. Take appropriate permission and paste the same all over the office. I am of the opinion that if you want to implement some process don’t file it in a word, excel or powerpoint document. Rather display it to the whole world by pasting them in prominent places across the office. Make use of information radiator concept from Agile. Information radiator according to Alistair Cockburn is:

An Information radiator is a display posted in a place where people can see it as they work or walk by. It shows readers information they care about without having to ask anyone a question. This means more communication with fewer interruptions.

Requirements document
Yes you need to have requirements but not the traditional way of gathering requirement. In traditional way you write pages after pages of requirement, spend too much time up front (before the project even starts) gathering requirement. Agile says don’t spend too much time up front on gathering requirement. The reason, tomorrow the client’s requirement would change, so gather as much as you can but don’t over do it. Requirements are written in the form of User Stories. Typically user stories are written on a piece of paper or sticky notes. The template is as given below

As a <user type>, I want <some ability or action>  so that <some result>.

If the team is using JIRA, TFS (Team Foundation Server), Rally etc to manage the agile process, then the requirements can be captured in them. Each application has features to capture the nitty gritties of a user story. For e.g. JIRA has description column where one can elaborate on what the story is all about. In agile importance is given to discussions and understanding of the requirement within the team. Also user stories are not elaborated until they are worked upon. Again the reason is product backlog (where user stories are captured) gets re-prioritised every now and then. So one user story having high priority today can have a low priority tomorrow. To save everyone’s effort from getting wasted the user stories are elaborated or discussed in detail during implementation in a sprint. Typically in a sprint planning meeting.
In short the way requirements are captured have changed. You use different tools and techniques to capture requirement. JIRA, TFS, Rally etc have taken the place of traditional document. Oh I forgot to mention sticky notes.

Without document how can one know what the team has implemented?
The modern agile teams have moved away from the traditional way of maintaining records in document format. Agile teams use sprint review meeting to show what has been implemented in the current sprint. Sprint review meeting is a casual demo of what features have been developed in a sprint. If you want to know what the implementation looks like attend the sprint review meeting. Advantages are that one need not spend time reading pages after pages of boring document. Rather you get to see a demo of a working feature. If you are an end user you can visualise how it will fit in your scheme of things.
Ok, you are not satisfied, still want a document evidence? No problem, record the demo and keep it as a documented evidence. Again another advantage, you are not spending extra time in writing document. Optimisation at it’s best. Recorded video is acting as a document.

Without documents how a new developer/QA will get up to speed with the product? 
I have been a developer and in my career of nearly 12 years I have never understood a system by reading documents. Developers understand only one language and that is the language in which they code. They understand C#, Java, JavaScript etc. The only thing they need is cleanly written code or in other words clean code. Developers understand a system by reading the code. They understand by pairing up with other developers. They understand by making bug fixes. Any new developer is thrown into the water and he learns to swim. Most of the projects, developers are asked to start with bug fixes and as and when they are comfortable they are given features to develop.
If you are following Test Driven Development (TDD), then tests are the place to start. Tests help a developer to understand a feature/application very well. By looking at tests he/she will understand what the piece of code is doing. Having tests will give confidence to a developer to fix code and get early feedback whether the fix has broken anything.
As highlighted for a developer documentation is clean code and tests.

Let go of your traditional understanding of documentation
From my experience what I feel is that people need to come out of their traditional view of having document for each and everything. Document need not be in the form of word, excel or powerpoint. The world has moved on. Document can be in the form of videos, can be in the form record entries in tools like JIRA, TFS, Rally etc. Can be in the form of display charts, cards,Kanban board, sticky notes, mobile pictures of a white board discussion, audio recording, webex recording, goto meeting recording, any link to an article to a website etc. Did I miss anything?
The key is not to burden development team in writing documents which are not adding value to the customer. For me if it doesn’t add value to the customer then there is no point in spending time on documentation. One needs to leave the technical documentation part to the team to decide. I understand when someone leaves a company he takes along with him good amount of knowledge. The way to tackle this is to have videos, good tests and clean code.
The agile manifesto doesn’t say no documentation. The way to look at document has changed. In the bad olden days of CMMi/waterfall, development team use to spend too much of time in creating documents for requirement, design (HLD/LLD) etc. Even from developer’s perspective, hardly 1% of them use to read these. People understood that and they wanted the development team to focus more on what they do best i.e. development and testing. So the onus is on delivering value to a customer by giving a working software every sprint/iteration than writing documents.
This is my personal opinion. In the not so distant history, documents were used as a proof so that nothing went wrong. I feel the trust quotient was not there. People wanted to have this and that and what not kind of document to hold people responsible for their action. With agile that has change. It is not about individual, but about team. It is about empowering team and trusting them. Documents have also taken new forms like sticky notes pasted on a kanban board, videos, casual demo etc. With trust being an integral part I feel document has taken lower precedence.
In essence the definition of document has taken new forms. They have moved from the traditional written document format to include more visual forms.