Documentation in Agile

Documentation is a big contentious issue in Agile. There is a reason for this. When you start your presentation on Agile you start by going through the 4 statements of agile manifesto. As soon as you go to the second statement, traditional waterfall guys start frowning their eyes. The second statement in the manifesto says this.

Working software over comprehensive documentation

Seeing the statement people assume that documentation is not part of the agile process. It completely ignores documentation. Here comes questions. Without documenting the process flow how do you know what is the process we need to follow? Without document how do you know what is the requirement? Without document how do you know what has been implemented? Without document how is a new dev/qa going to understand what is coded/system all about? And if it is someone from CMMi background, then god can only help you to convince him/her.
The key thing people forget is that processes are there to help people get their work done faster. Deliver software faster to the end user. Faster delivery will help you capture market compared to your competitors. In software development you are not delivering document to the end user rather a working software. The above statement never said documentation is not needed, just that the emphasis is on delivering value to the end user.
Now let’s try to address some of the questions raised above.

Process flow documentation
Agile is well documented. If you search the internet you will get not hundreds but thousands of documentation. Just put a few links to the website and that forms your document. I would say there is no need to re-create something which is freely available on the net. If you really want to have off-line document then use the save feature of your browser and save the pages. Make sure you have the permission to do the same. Or copy paste the contents and put it in a document with a link to the original article.
One of the best way of explaining your process is to have a pictorial representation of the same. If you search agile process in Google you will find lot of high quality diagrams. Click here to see few. Take appropriate permission and paste the same all over the office. I am of the opinion that if you want to implement some process don’t file it in a word, excel or powerpoint document. Rather display it to the whole world by pasting them in prominent places across the office. Make use of information radiator concept from Agile. Information radiator according to Alistair Cockburn is:

An Information radiator is a display posted in a place where people can see it as they work or walk by. It shows readers information they care about without having to ask anyone a question. This means more communication with fewer interruptions.

Requirements document
Yes you need to have requirements but not the traditional way of gathering requirement. In traditional way you write pages after pages of requirement, spend too much time up front (before the project even starts) gathering requirement. Agile says don’t spend too much time up front on gathering requirement. The reason, tomorrow the client’s requirement would change, so gather as much as you can but don’t over do it. Requirements are written in the form of User Stories. Typically user stories are written on a piece of paper or sticky notes. The template is as given below

As a <user type>, I want <some ability or action>  so that <some result>.

If the team is using JIRA, TFS (Team Foundation Server), Rally etc to manage the agile process, then the requirements can be captured in them. Each application has features to capture the nitty gritties of a user story. For e.g. JIRA has description column where one can elaborate on what the story is all about. In agile importance is given to discussions and understanding of the requirement within the team. Also user stories are not elaborated until they are worked upon. Again the reason is product backlog (where user stories are captured) gets re-prioritised every now and then. So one user story having high priority today can have a low priority tomorrow. To save everyone’s effort from getting wasted the user stories are elaborated or discussed in detail during implementation in a sprint. Typically in a sprint planning meeting.
In short the way requirements are captured have changed. You use different tools and techniques to capture requirement. JIRA, TFS, Rally etc have taken the place of traditional document. Oh I forgot to mention sticky notes.

Without document how can one know what the team has implemented?
The modern agile teams have moved away from the traditional way of maintaining records in document format. Agile teams use sprint review meeting to show what has been implemented in the current sprint. Sprint review meeting is a casual demo of what features have been developed in a sprint. If you want to know what the implementation looks like attend the sprint review meeting. Advantages are that one need not spend time reading pages after pages of boring document. Rather you get to see a demo of a working feature. If you are an end user you can visualise how it will fit in your scheme of things.
Ok, you are not satisfied, still want a document evidence? No problem, record the demo and keep it as a documented evidence. Again another advantage, you are not spending extra time in writing document. Optimisation at it’s best. Recorded video is acting as a document.

Without documents how a new developer/QA will get up to speed with the product? 
I have been a developer and in my career of nearly 12 years I have never understood a system by reading documents. Developers understand only one language and that is the language in which they code. They understand C#, Java, JavaScript etc. The only thing they need is cleanly written code or in other words clean code. Developers understand a system by reading the code. They understand by pairing up with other developers. They understand by making bug fixes. Any new developer is thrown into the water and he learns to swim. Most of the projects, developers are asked to start with bug fixes and as and when they are comfortable they are given features to develop.
If you are following Test Driven Development (TDD), then tests are the place to start. Tests help a developer to understand a feature/application very well. By looking at tests he/she will understand what the piece of code is doing. Having tests will give confidence to a developer to fix code and get early feedback whether the fix has broken anything.
As highlighted for a developer documentation is clean code and tests.

Let go of your traditional understanding of documentation
From my experience what I feel is that people need to come out of their traditional view of having document for each and everything. Document need not be in the form of word, excel or powerpoint. The world has moved on. Document can be in the form of videos, can be in the form record entries in tools like JIRA, TFS, Rally etc. Can be in the form of display charts, cards,Kanban board, sticky notes, mobile pictures of a white board discussion, audio recording, webex recording, goto meeting recording, any link to an article to a website etc. Did I miss anything?
The key is not to burden development team in writing documents which are not adding value to the customer. For me if it doesn’t add value to the customer then there is no point in spending time on documentation. One needs to leave the technical documentation part to the team to decide. I understand when someone leaves a company he takes along with him good amount of knowledge. The way to tackle this is to have videos, good tests and clean code.
The agile manifesto doesn’t say no documentation. The way to look at document has changed. In the bad olden days of CMMi/waterfall, development team use to spend too much of time in creating documents for requirement, design (HLD/LLD) etc. Even from developer’s perspective, hardly 1% of them use to read these. People understood that and they wanted the development team to focus more on what they do best i.e. development and testing. So the onus is on delivering value to a customer by giving a working software every sprint/iteration than writing documents.
This is my personal opinion. In the not so distant history, documents were used as a proof so that nothing went wrong. I feel the trust quotient was not there. People wanted to have this and that and what not kind of document to hold people responsible for their action. With agile that has change. It is not about individual, but about team. It is about empowering team and trusting them. Documents have also taken new forms like sticky notes pasted on a kanban board, videos, casual demo etc. With trust being an integral part I feel document has taken lower precedence.
In essence the definition of document has taken new forms. They have moved from the traditional written document format to include more visual forms.