In a recent meeting one of my colleagues asked me why we don’t have a test strategy document. I asked why we need one but I didn’t get a straight forward answer.
I personally do not like to have these kind of documents. I am sure these documents may help in other teams / companies but for the last 14 years these documents were more of a burden than a help or guidance.
First of all, what is a Test Strategy? It is basically defining how we are going to test the application. What sections does it contain? In high level,
- Document review and approval process or people
- Test approach or different levels of testing and by whom
- Environments the application will be tested
- Tools be used
- Release process
- Other process like defect management, etc
Almost all the test strategy documents that I have seen in my previous experiences and the templates online shows a section for definitions of each test type and who own each section. For example, Unit tests by Developers, Feature tests by Developers, Acceptance test by Testers, Performance test by Developers etc along with what is a unit test, what is a feature test etc.
Some can say, this is assigning responsibility. For me it’s encouraging a division in responsibilities. A division between development and testing teams. Becomes a waterfall method and push the responsibilities over the wall. You may disagree with me here. However, my thoughts and my behaviours are a reflection of my own experience.
As modern teams, we expect development and testing functionalities to be working together. Pair when possible. Open discussions and collaboration on how the product quality will be assessed. There for I do not want to introduce these definition documents and encourage the division of responsibilities. Sometimes people find comfort on these definitions and role association. This can be harmful if you are expecting the teams to be working together.
If you are working in a free thinking, dynamic team who know that they should be flexible around definitions and roles and responsibilities, then you can have a test strategy as a guide document. But if your teams are not there yet, there can be confusions of who do what and when and these type of documents can limit their thinking. If not having these type of documents will help the team to be independent and flexible, then definitely that’s where I would go for.
Sometimes teams can get hung with the document and as well as quote the document out of context. For example, in your test strategy document you have defined that test environment should not have live data. If there is a situation, that you have to have live data in on lower environment, someone can say, it is defined in the document and we can’t so we can’t. Deal with the issue in a different way. The different way can cost time and money and the data that is coming from live have no sensitive data. But yet because the document say so, it is not possible.
You may think this is an extreme scenario. It is actually not. I have experienced this in several occasions. As agile teams we are fast paced teams. Sometimes we have to be creative in dealing situations and the whole team should come together. The document can isolate test teams and let the rest of the team go “It’s not our responsibility. We have done what we are asked for”. I have being there, many times. What we say is no finger point but the first question anyone will ask is, how come this slipped testing? Not, how come we missed this as a team? May that question should be documented in the Test Strategy?
If you have test strategy documents, how frequently are you keeping them updated? Or have you created them so generic so the document is valid for all purposes? Once we had a 60+ pages power point document that no one in the team read, or used. But it ticked a box. The management wanted a test strategy and there is a test strategy. Nothing beyond that point.
The document sometime can limit the communication and creativity of the team. Instead of have ad-hoc discussion about how we test something, teams can refer the document and make their own assumptions and interpretations. This also can discourage people to try different tools and techniques. I had being questioned trying out a new tool just because it is not mentioned in the test strategy document. When I update the document, it had to go through the long list of approval process which I had gave up trying the tool.
Does no test strategy document mean, the team doesn’t have a strategy? Team doesn’t know what to test, how to test, what tools to be used and what environments to be used?
I think testing strategy restricts teams and their capabilities. It can depend on what type of a project / product you are working for. If you are working in a critical application you may need one to define all the complexities. Others should learn to collaborate. What’s the word? Be agile about it…
So who should have a test strategy document?
- Teams that are new to Agile practices
- Teams who have less experiences team members who need guidance over best practices and process
- Teams that works in different geographical locations
Above mentioned situations do need a strategy to guide the team and make sure the team is evolving / practicing a defined process. However, these documents should be used for short time until the teams understands the process and value. Then it should be open for discussion and changing according to the team / project / product needs.
My final thoughts to summarise above is,
Testing strategy is Testers should collaborate with Business Analysts on requirements gathering, estimations and story writing. Then collaborate with developers on pair automation while executing their exploratory testing skills. Occasionally use a written document or guideline to support the teams who need direction but later encourage them to start discussions and create team level work methods.
So what about the test plan? I have listed my thoughts about detailed test plans in here.