The need for testing strategy document

In a recent meeting one of my colleagues asked me why we don’t have a test strategy document. I asked why we need one but I didn’t get a straight forward answer.

I personally do not like to have these kind of documents. I am sure these documents may help in other teams / companies but for the last 14 years these documents were more of a burden than a help or guidance.

First of all, what is a Test Strategy? It is basically defining how we are going to test the application. What sections does it contain? In high level,

  • Document review and approval process or people
  • Scope
  • Test approach or different levels of testing and by whom
  • Environments the application will be tested
  • Tools be used
  • Release process
  • Risks
  • Other process like defect management, etc

Almost all the test strategy documents that I have seen in my previous experiences and the templates online shows a section for definitions of each test type and who own each section. For example, Unit tests by Developers, Feature tests by Developers, Acceptance test by Testers, Performance test by Developers etc along with what is a unit test, what is a feature test etc.

Some can say, this is assigning responsibility. For me it’s encouraging a division in responsibilities. A division between development and testing teams. Becomes a waterfall method and push the responsibilities over the wall. You may disagree with me here. However, my thoughts and my behaviours are a reflection of my own experience.

As modern teams, we expect development and testing functionalities to be working together. Pair when possible. Open discussions and collaboration on how the product quality will be assessed. There for I do not want to introduce these definition documents and encourage the division of responsibilities. Sometimes people find comfort on these definitions and role association. This can be harmful if you are expecting the teams to be working together.

If you are working in a free thinking, dynamic team who know that they should be flexible around definitions and roles and responsibilities, then you can have a test strategy as a guide document. But if your teams are not there yet, there can be confusions of who do what and when and these type of documents can limit their thinking. If not having these type of documents will help the team to be independent and flexible, then definitely that’s where I would go for.

Sometimes teams can get hung with the document and as well as quote the document out of context. For example, in your test strategy document you have defined that test environment should not have live data. If there is a situation, that you have to have live data in on lower environment, someone can say, it is defined in the document and we can’t so we can’t. Deal with the issue in a different way. The different way can cost time and money and the data that is coming from live have no sensitive data. But yet because the document say so, it is not possible.

You may think this is an extreme scenario. It is actually not. I have experienced this in several occasions. As agile teams we are fast paced teams. Sometimes we have to be creative in dealing situations and the whole team should come together. The document can isolate test teams and let the rest of the team go “It’s not our responsibility. We have done what we are asked for”. I have being there, many times. What we say is no finger point but the first question anyone will ask is, how come this slipped testing? Not, how come we missed this as a team? May that question should be documented in the Test Strategy?

If you have test strategy documents, how frequently are you keeping them updated? Or have you created them so generic so the document is valid for all purposes? Once we had a 60+ pages power point document that no one in the team read, or used. But it ticked a box. The management wanted a test strategy and there is a test strategy. Nothing beyond that point.

The document sometime can limit the communication and creativity of the team. Instead of have ad-hoc discussion about how we test something, teams can refer the document and make their own assumptions and interpretations. This also can discourage people to try different tools and techniques. I had being questioned trying out a new tool just because it is not mentioned in the test strategy document. When I update the document, it had to go through the long list of approval process which I had gave up trying the tool.

Does no test strategy document mean, the team doesn’t have a strategy? Team doesn’t know what to test, how to test, what tools to be used and what environments to be used?

I think testing strategy restricts teams and their capabilities. It can depend on what type of a project / product you are working for. If you are working in a critical application you may need one to define all the complexities. Others should learn to collaborate. What’s the word? Be agile about it…

So who should have a test strategy document?

  1. Teams that are new to Agile practices
  2. Teams who have less experiences team members who need guidance over best practices and process
  3. Teams that works in different geographical locations

Above mentioned situations do need a strategy to guide the team and make sure the team is evolving / practicing a defined process. However, these documents should be used for short time until the teams understands the process and value. Then it should be open for discussion and changing according to the team / project / product needs.

My final thoughts to summarise above is,

Testing strategy is Testers should collaborate with Business Analysts on requirements gathering, estimations and story writing. Then collaborate with developers on pair automation while executing their exploratory testing skills. Occasionally use a written document or guideline to support the teams who need direction but later encourage them to start discussions and create team level work methods.

So what about the test plan? I have listed my thoughts about detailed test plans in here.

Think differently when working with distributed teams

Distributed teams. We all working with distributed teams. Whether it’s in different countries, different cities in the same country, different floors in the same building, and different teams in the same floor or working from home. All count as distributed.

What issues do we face? There is a lot! However, when we start to discuss the issues, we always see the negatives and we talk about issues caused by the other side.

What we can do to correct this? We go hands on the air and say they should do this and that. Yet, we never think what might they think? What we can do to help? What behavioural, thinking and communication changes we can adopt?

These discussed in a Testbash workshop where Steve (@sjwatsonuk) and I attended. We recreated the workshop in our company as a pilot. Due to its success, we did it again to a different set of people and it got all positive feedback. Positive feedback for the workshop is brilliant. How positive the people feel at the end of the workshop matters. However, how much they practice the learnings is a question. Which we hardly get any feedback.

I am fortunate that most of my team was in the pilot program. From the next day onwards, they started to discuss how they could do things different. Now they are thinking differently. Take actions to include the nearshore team more in team discussions. They see why this is important. The nearshore team ran few retrospectives from their end and as well as conducted few stand-up sessions. Progress…

I believe if we think open-mindedly, we can collaborate successfully. Distributed teams created to deliver more work with a high quality while maintaining the costs or reduce cost. Everyone is part of  the project success. Build trust not pointing fingers, build communication openness on both sides so no one feel left out, build environments where everyone can ask questions and give answers to make the product better. Give constructive feedback and encourage it to come from every team member disregarding the location. Learn about their culture and try to help them to connect with you. Specially, praise each other when do well.

Have you ever worked from home? Was it easy to connect with your team virtually even you know them? Can you find people when you need an answer? Have you deliberately picked up work that you can do without the need of anyone else when you planning to work from home? If you are getting frustrated with your team when you work from home for one day, how your other teams might feel when they have to go through it everyday?

I am working with Abby (@a_bangser) who is one of the original presenters of the workshop, to deliver the same workshop in Helsinki for European Testing Conference. An exciting opportunity to help more people build empathy towards each other.

Remember, we all distributed to each other and small change of thinking can make a big difference.

Sometimes its good to ignore the rules 

Yes. Sometimes it is okay to ignore rules, think from a different point of view. This may cause you issues and criticism and temporary failures but as long as you have a plan, it should be all right.

This is my journey on implementing a regression test suite in current project, how I solved different hurdles. I have broken many rules but now I believe it is a success.

When I join, the project there was acceptance tests embedded to the code. So as the unit tests, feature tests and integration tests. Those acceptance tests not cover user journey but more of a single function. All different tests are based on one code base. Acceptance tests were running in TeamCity as well.

I spoke to the team and then start adding tests to the acceptance tests because I felt it is not enough. However, it was making the code build too slow and maintenance get out of control. Why? It was one code base. When acceptance tests fail, the tests were either ignored or made to pass. Tracking them was difficult.

Therefore, it was a failure. I was discouraged and almost given up!

Then I asked the team how they feel about me creating a separate test suite. Seniors objected because it will make a duplicate code, unnecessary maintenance etc. I felt disappointed and discouraged again. I maintained an Excel sheet as a check list for different functionalities but wasn’t happy. Testing had become a long process when it come to build pick and deployments.

Few months later, I started creating the test suite without telling anyone. I copied some of the functions from the main code base that I can use and build up on it. It came out as exactly I planned. I started using it for checking the releases. So far, all happening in my local machine. Then I showed it to the team saying this is what I have done. I even used the test suite against live environments, which had not happened before.

Only few impressed. Same arguments came. I mentioned this is only maintained by me so it should not be any overhead to rest of the team. It will be in the common repository but I do not want the development team to update it because I would like to have a control of what goes in and out!  Did they agree? No. They were telling about what happen if they change the modules that I have copied. My tests are independent from their code base. I just ignored all their concerns for their displease.

Some helped me setup the build in TeamCity and the team started to see results. Build picking and deployment times reduced. Issues caught up front in areas that was not thought of… Stage 1 succeeded.

I wanted to discuss the tests and its coverage with the business but the tests are very technical. Therefore, I wanted to do BDD format. However, all the documents I read and all the courses/workshops I went to told that BDD is a way of communication and it should start from the business and the teams talking to each other. I agree yes.

However, our team was doing most of these things. We are having open discussions with the Product owner. The business analysts are writing the stories in a Given, Then, When fashion .Doing changes from top to bottom will be troublesome. Many people do not want to change how they work unless there is a perfect valid reason.

Therefore, with a help of a contract tester we changed the existing test suite to use SpecFlow. Introduced BDD to the Regression tests and converted all tests. Detailed SpecFlow reporting and tests running in multi-thread mode the reduce overall time impressed the team. They now backing us with having a separate test suit.

When mobile tests needs to be done, it was easy for us. We expanded the tests to different devices to different break points with ease.

Then I showed the test suite to the product owner. Taken him through few scenarios.He was clearly impressed on what we have achieved. Now I am working with some editors to understand what scenarios I might be missing from their point of view. Involving the business with testing was great.

Now I am working with the business analysts to see how we can work together with the BDD scenarios. Whether I need to change the language that I used in the feature files or they need to change theirs or whether we can meet half way. We are going to try it on a small functionality change so there will not be less impact.

There is long way to go but I am happy the way I have made an impact without playing by the common rules.

Therefore, I think it is good to break rules now and then. Well it worked for me so far.

LeanCoffee style online meetup : Experience report

My first LeanCofee experience was with TestBash in 2015. It was before the conference start and I was very excited about the concept because it gives everyone a voice.

I wanted to introduce this to my company but was not able to do it for a while because I was looking for ways to do it online as most teams we have are distributed.

Leancoffeetable tool was introduced to me by the weekend testing group. It is not the most advance tool but it does the job. I trailed it by myself several times before introducing it to wider group.

We have a QA chapter in the company and I wanted to introduce LeanCoffee to them just because only few people raise their ideas. The rest are not actively participate because either they are not interested or they do not know how to contribute. However, I believe everyone have their unique experience which worth sharing.

The introduction went well and people wanted to try it out. So we dedicated a one QA chapter session for LeanCofee.

First, we thought we will do the leancoffee with four small groups and then feedback to the chapter but we had issues with number of people who wanted to join the chapter itself. So finally a one big LeanCofee session online, facilitated by me.

We got participants from differently businesses across company and from different geographical locations. In a normal day in the chapter, someone will do a demo or a presentation about a new tool or practice so this time it was time to share. We heard voices we have not heard before. Current issues the testers face in their own projects and want to know how others will react. What to do? Seeking information that we thought everyone is aware but not.

Meanwhile the discussions I have created a mind map for each point as well. This shared later as the meeting notes / summary.

Topics discussed,

  • Can developers help with testing in scrum?
  • How to get developers to do more unit testing?
  • How to plan testing when there is no documentation?
  • How to encourage testing community to engage more in RBI?
  • What value QA chapter add?

1.5hrs passed quickly. There were number of issues we have not addressed due to timing constraints. Nevertheless, we achieved a lot.

Feedback was fantastic. For me, it was a great experience. People share their ideas and experience to others who need help is what a chapter should be. That is what we achieved.

Most of all they want to do the chapter in a LeanCofee style from now and that I call as success…!

Recreate Building Quality in With Distributed Teams

After attending an exciting and educational workshop in Brighton (read more), Steve (@sjwatsonuk) and I decided that, we should recreate the workshop for our colleagues in Reed Business. I am very grateful to Lisa (@lisacrispin) and Abby (@a_bangser) who not only helped us with sharing all materials but with plenty of encouragement as well.

Since we are a big organisation with many different projects, we thought to stick to the projects that we are in as the pilot. We also included members from a team who deals with internal clients as well. The presentation tweaked but the rest of the workshop was as it was originally.

The teams were well mixed up and distributed over different floors (we were not able to book the rooms in the same floor). We took the liberty to mix and match people and that they work with majority of people that they do not know than the number of people they know.

We got two excellent and exciting helpers as well. Saima Poorghobad and Michael Lopez (@lopezma) was continue to support us by showing thing that we have not noticed. Fantastic!

All started well on the workshop. Steve and I had a plan but not rehearsed a thing. Therefore, we thought we would improvise as things go. We got people to talk. They came up with the issues that they face with working with distributed teams. Everyone was facing the same issues such as communication, culture, language etc. The examples we gave related to each business. We also mentioned the internal teams that support different projects that have a different level of distributed team challenge, which normally overlooked.

Then two teams sent away to different rooms with their tools. Straight away communication issues bubbled up. Lync was not working properly. Sound quality not there, video not working.

In the team I looked after as the Product Owner (PO), rushed to get in to work without even looking at the schedule. They wanted to get things done. Therefore, the inception and planning sessions were almost wasted. Many assumptions made. They did not ask me the right questions in the beginning until I clearly said I am not happy with the work they done. Offshore was asking what my favourite colour was than asking the right questions. Then they all started asking what my requirement is for each individual piece of work than one person dedicated for that – because of not planning the session and not assigning roles. Offshore team tried to gather the requirement with the work that they assigned but it was harder. Therefore, Lots of frustration! Time pressure! People running up and down! It was chaos!

In the retroactive it was highlighted that product owner did not have any bandwidth to answer all questions. I hinted saying I would be happy if one person talk to me than all. It was fascinating how teams forget the basic roles under pressure.

Another thing I observed was that the offshore team were trying their best to get the attention from the onsite team but they all busy either talking to me or colouring away. The offshore contact had to talk very loud several times and to shout to get the attention. Also later, I have to know that they were there with less or no work for most of the second sprint.

Everyone came back together and discussed the issues that each team faced. Offshore teams were frustrated and was very vocal about the whole thing. Then we decided to give additional 5 minutes to the teams to decide how they want to do the next sprint from the challenges they have/had. That was something Steve and I came up at the time so the people can learn the importance of planning and communication.

There were team members who wanted to talk to me and gather requirements while the planning, share information and retrospective sessions. This shows how people want to get on with their work than making sure they work as a team and achieve a common objective. Alternatively, let the others do the communication while these individuals busy with the ball rolling. Either way it was not helping the team.

The third sprint was a success. Steve and I took 2 minutes holiday and done a change of mind on a requirement. Teams did not seems to mind it at all. The requirements gathered for most of the work so they were caring out the work. How great is that!

Later we talked about the culture differences a bit more. I talked with my personal experience. As I speak about how other side of the table might be thinking I saw some eyebrow raises, nodes, note taking, and look of surprise. I might have invoked some thinking and awareness.

At the end, there was real light bulb moments in most people on how to do things differently. We asked what they would do differently and things like make sure communication issues (technical, non-technical) resolve sooner than later, involve product owner with offshore discussions, gather / clarify requirements early, make sure the offshore see the bigger picture and be more understanding are some of the things came out.

Well, all workshops we have people who think it is a waste of time, as they know the subject better. We had couple of them and they had given us negative feedback. One response was that, that person already knew these challenges and nothing new. Well in my point of view, everyone knows the issues but how they react to solve the problems is the key.

Two negative feedback against 21 from 23 participants. I take that as success!


Next, we have to collate the feedback and present to a wider group and who knows we might be able to take another group of people the same journey and help them to build quality with their distributed teams…

We learned a lot how this could improve. Such as dedicate someone else to coordinate the timing, bit more descriptive in the inception, bit more organised with the rules and do some adjustments to the schedule.

You still reading? Great… Few more lines…

Finally would like to thank Steve for encouraging me to do this workshop and made me believe that I can talk in front of many people without making myself a fool. Great co-host to work with. In addition, Saima and Michael was so helpful and given us fantastic feedback. Could not ask for more enthusiastic helpers than them. We were very lucky.

In summary, it was a great experience for us and as for the participants. Looking forward to recreate the workshop and of course looking forward to learn more.

If you like to read Steve’s experience you can find it here.

Onshore , offshore and me

After the Distributed team workshop I thought I should share my personal experience as an offshore resource and an onshore resource just to add some more value to the thinking around the subject.

I left Sri Lanka in 2010 and things might have changed now. Still I believe some of the experiences are still valid. I must tell that I have not experienced everything that is mentioned here as an offshore employee. However, I did see and I did listen to others.

Following are some characteristics I have noted in many companies,

  • Offshore team is normally at the third level from the customer. Example company XYZ hire company A for their development work and company A hire company B mostly offshore company to do all or part of the work
  • The leads of the project mostly communicate with the customer. As of above example it can be the Company ABC or Company A. If the team have questions that needs to go through the team leads
  • Estimations done by the project management and/or team leads and there is a high possibility that the numbers are crunched to satisfy the customer
  • Offshore team mostly work on tight deadlines where late nights in the office are common. No extra pay but food and travel can claim back.
  • Innovation, change of practices needs to approve by managers who mostly say “No” mostly due time constrains. More weight is given to the time that can be invoiced than time that can be used for innovation
  • Share knowledge with care due to job insecurities
  • Most testers trained on the job. No community and less inspiration
  • Blame culture – “Who tested this and why you did not find this issue?”
  • Communication within teams are less efficient where confusion is common due to many levels of information filtering and language barrier
  • Lack of praise on a job well done.

I was discussing this subject with one of my friend, she pointed out how testers relay on Business analysts, architects, and tech leads for information. From a cultural perspective, we tend not to ask questions and let the customer speak. I think that is due to,

  • Lack of self confidence
  • Fear of asking questions
  • Fear of being judged by others
  • Lack of confidence in speaking in English
  • Do not know how to ask questions
    • Being very technical
    • Not being specific
  • Shy by nature

After migrating to England, I sat in the other side of the table as the onsite resource. Different experience with a cultural shock!

  • Work life balance. During the first month in the office, I was not sure the time I should leave. I was trained stay late. Whether there are work or not sometimes. I was amazed to see that people leave on time. Still I felt guilty to leave at 5pm and every day I stayed until 5.30pm until my manager notice this and chase me away.
  • Encouraged to ask questions. As by nature, I ask many questions. Therefore, this is fantastic. I learnt how to ask questions without going around the bush.
  • My opinions were valued and not judged. It really boosted my confidence.
  • Information availability. The team did not depend on one person. Everyone share what they know in detail
  • I was allow to go and speak to the product owner if I need to. I was encouraged to bypass all the leads if required.
  • Estimates done by the people who actually will develop the product and they are very vocal on how much they can do and not.
  • More focused on how to do things right than finish the implementation
  • Innovation is encouraged.
  • Praise on a job well done. There were few incidents where I was overwhelm by the positive feedback and tears roll out. I used to defend myself from negativity or on things, I have not done. I had difficult managers. However, when it comes to performance review I always got a balanced feedback, which encouraged me to do well.
  • Everyone was very friendly with me even though I was from a different country and having troubles with speaking English properly.

I had the opportunity to liaise with offshore assignments; we contacted the people who develop/test directly. Some of the common characteristics I saw from onshore point of view were,

  • Business analysts take lot of time to explain the requirements to the offshore team
  • The onsite development team not happy with the offshore coding standards
  • I had to explain issues in detail before the offshore team accept them
  • Offshore team not asking questions and say yes to most of the things. Have to double check whether they understood the instructions/requirements
  • Join retrospectives but not vocal

With my own experience is being a member of the onsite team is more enjoyable. I understand the product and I have direct discussions with the customers to understand the requirements and possibilities. I communicate better. Mostly I have my work life balance and I am motivated to learn more about testing.

To be fair, my onsite experience is limited to one company. I am a person who used not to expect much from the company but give all I got so I get a sense of satisfaction. Now even the small difference makes me compare my experience now and before and appreciate things ever before. Again, this is my experience and learnings and it is unique to me.

However, we are living in an era where a one team can be located in several geographical locations. There is a part to play from both onsite and offshore teams to make these assignments a success. Onsite teams needs to think about the offshore team country culture. Simple language. Describe the bigger picture. Make sure they know what their part is in the big picture than developing/testing a small jigsaw part and not forgetting the offshore. Remembering them as a part of the team. I think most onsite teams are aware and try their best. Still it is best to be mindful about it. Offshore team needs to voice up. Ask questions and be involve with the onsite team. Try to avoid information filtering by involving the team in discussions.

I am more bias to onsite teams now. I think/believe/hope offshore team experiences are far better from what I remember.

Again, this is my view on both worlds. Yours might be different…

Greener grass on the other side?

We all have seen people leave companies / projects. Some you like and will missed and some you are glad that left. In addition, there is you and I who move from one company to other. Everyone hoping for a greener grass on the other side.

Why do we move? Some of the reasons commonly known to me are,

  • Career progression
  • Higher salary & benefits / Not paid enough
  • Change career / retire
  • Moving away from new responsibilities
  • Dissatisfaction with work assigned
  • Colleges
    • Strong characters who over power you
    • Cultural differences
    • Not feel like you are part of the team
  • House move
  • Travel issues
  • Kids
  • Technology that is used is not interested
  • Not appreciated
  • Migration to another country

In addition, some people give a complete lie and cover up the actual reason to leave.

I recently read some articles on why people leave the company and some were suggesting people leave their bosses not the companies. Some suggest otherwise.

One thing I have seen are most people do not speak about their dissatisfaction and they wait thinking the problem that they have will resolve by itself. This will only cause more stress and disappointment and ultimately, the whole team get to know your problems except your manager. What can your team do? Advice you to quit mostly. I was victim of this situation and I think you may as well.

Most of the time managers surprised when some people give the resignation. Just because they are unaware of an issue. So why people really leave? Mostly due to reasons, they do not want to share with managers. Sometimes the manager itself is the problem. Once I left a company within 3 months because my manager use to dictate what I should every day and not given me the opportunity to do what they hired me to do.

Following are some of the characters I have seen in managers are…

  • Do not care whether the employees leave or stay
  • So strong where employees do not open up to
  • Make people leave because they create a unfriendly environment
  • Senior and powerful that intimidate people to open up to them
  • Want to help and coach
  • Empathise others
  • Like to have favourites

Another common character I had seen is people had become a manager to fill a gap of a leaver. They are not happy about it and not speak about how they truly feel. Even though they get the necessary training, they are not happy of being a manager. I had seen people/teams under such managers suffer from coaching/mentoring and inspiration. They mostly continue the work but can see they are stressed and not energetic of new responsibilities.

I guess the conclusion “People leave their manager not companies” somewhat right. Nevertheless, if a company not revising the pay, not revising the benefits, not revising policies to make the employees life better, then of course people leave the company regardless of how supportive their manager is.

So where does these people go to? How companies attract these leavers? What does people look before they choose a company. Some common facts are,

  • Salary & benefits
  • Learning opportunities
  • Location
  • New challenges
  • Feeling that the people who interview you believe in you and what you are capable of

When people apply, do they go for companies that they had not known of? Well yes and no.

Yes mostly if you are desperate to leave the current company and desperate to find a job. In addition, if this company gives you the above mentioned.

No because you do not trust this company, give you what you want. Why they are not popular? Is it because they are not a big company? Is it because they are not mastered the field that you are interested on? Is it because the company is a start up? Is it because majority of the employees are not in your age range so you are not aware of the company? Is it because the company had not advertised in media that you are interested in? The list goes…

My biggest worry when I change a company whether it is a popular company or not, is what kind of manager I will get (Sometimes it is not the immediate manager who conduct the interview). What kind of a team I will work with. Will they welcome me? Will I like them?

Sometimes Interviews and actual working environment is very different. The members in the interview panel matters. They can show a different picture of the company to what it is because they want the candidate to accept the offer. I have seen people disappointed after a month or so in a company because that is not what they have told at the beginning. There were time at the interview I have told that I would have growth opportunities, learning opportunities and recondition. In actual environment, I was doing a monotonous job of testing a third party application customisation and support, not allowed to move to a different project because I had the knowledge. They offered me everything they rejected when I tell them that I am leaving.

In addition, the best candidate who see in the interview can be the worst team member you had ever worked with as well.

Some companies go through job agencies because they want to be anonymous and/or because it is easy to do so. However, they can be the reason people not apply as well.

Once an agent offered me a job, where you need to travel by train and then by bus and then walk (I did not had my driving licence at that time). Of course, I refused and the agent told me that if I were “picky” like that I would never find a job. Well I did found a job quicker than he thought.

I personally like if the companies contact me directly than job agencies annoy me every now and then. In house recruiters have this polite way of addressing you and explain what the job is where most of the job agents who contacted me just want me to apply to the job even I do not want to.

Are you familiar with statements like “work for a leading financial services company based in Surrey”? Well how do I know whether I am applying to the same company that I am working right now? How do you think people will apply to a company just taking the word from a job agency? Well I do not. Normally these agencies do not tell what the company name or where it located until they get hold of your CV. So then you run in to the risk that they call you every 6 months with job offers whether you like it or not.

Of course, companies know these issues. They invested/investing so much to retain employees and hire the best. Managers sent on training courses and teams are encourage on giving 360 feedback. Companies does try to improve the benefits and work life balance.

In addition, I have seen companies advertise the best things the company does, the things that company most proud of, spread out to meetups/conferences by being a sponsor, use their best employees to talk to potentials, be active in social media and target the age range and expertise groups.

Nevertheless, if you are an employee and if you have an issue, you have to talk to your manager or if possible manager’s manager or HR. I am sure they will give you a solution or a work around. Mostly if you like the company at least give them a chance to work out what they can do before deciding to leave because the grass in the other side is not always green. That I know for a fact…

Testbash workshop: Building Quality in With Distributed Teams

Lisa Crispin (@lisacrispin) and Abby Bangser(@a_bangser) done this awesome workshop about understanding how to succeed in distributed teams. I went to the workshop thinking that Lisa and Abby will share their experience and give us tips on best practices. How wrong I was.  The workshop was exceeded expectations and just blew my mind.

Session started with identifying the issues we have when working with a distributed team. Distributed team can be a offshore team or within the same country in a different location. List goes on as,

  • Culture
  • Time zones
  • Common understanding of work
  • Networks
  • Accents
  • Different testing practices
  • Being forgotten
  • Not as wanted to the team
  • Missing non-verble communication
  • Travel cost
  • Laws
  • Community availability etc

Then the hands on session began. Format,

  • 6 members onsite
  • 6 members offsite
  • Communicate via Skype and Email
  • Product owner is onsite and not willing to travel
  • Onsite should share work with offsite
  • Onsite and offsite did not have the same tools
  • Every task had a time to adhere to

We got pictures of two type. One set have colour codes to follow and some did not. We had to decide what work we need to sent to the other team who were physically far from us.  We thought colour coded work should be sent to the other team as they can work independently. We did not assess how difficult they are or the tools they have to complete the task (Mistake).  We had to give an estimate on how many colouring we can deliver at the end of 3 sprints. We did not consult the offshore team with the work we are assigning to them but we assume they would be fine(Mistake).

Sprint 1 started. I sat with Lisa who was our product owner and went through the non colour coded pictures asking her what colours she want them to be – role of a business analyst where others are coloring away. We received a sample from the offshore team mid way. We just kept it and did not look at it to see or sent them a reply (Mistake). Well we all were rushing away colouring and negotiating with Lisa with her colour requirements. At the end of the sprint 1 we present all the work to the product owner. All work from offshore team got rejected because they were messy and not according to the standards Lisa wanted. Mini retrospective  done after the sprint 1 and we decided to include onsite testing before pass work to product owner (Lesson)

Sprint 2 started. We brief the onsite team what the output of the retrospective but did not tell them anything about what we do(Mistake). We were in a hurry to start/finish work. When samples came from the offshore we checked them and sent some away. Sprint 2 went almost same way as the Sprint 1.(Mistake)

Then all the offshore teams summoned back to the workshop room. Discussion began. Offshore team was not happy – not happy at all…(Lesson). Outcome as below.

  • Use the words us and them even it’s a one team
  • Different tools
  • Different expectations
  • Different work
  • Quality vs Speed
  • Unclear communication
  • Have to trust the onsite team on passing information
  • Do not see the big picture etc

Then offshore team flew back to their location. This time we kept Skype open all the time (Lesson). We let Lisa talk to the offshore team and explain what she is expecting(Lesson). We asked some work that they are doing even we have different tools (Lesson) and realised the work we sent to them was actually hard and need more time(Lesson). In mid way the product owners decided they do not want any shade of blue in the new pictures and replace it with purple. We were able to send the message right away to the offshore team. Also the offshore team was able to show work to the product owner and get the feedback before the work being sent off.

Feedback after the 3rd sprint was as below.

  • Onshore negotiation with product owner for offshore
  • Asking for support
  • More frequent Skype calls
  • More wider direct communication
  • “Knowing what we were doing”
  • Questioning/justifying assumptions
  • Internal checks etc


3rd sprint was a success because of the open communication and knowing what issues offshore team is going through.

Finally the lessons learn were discussed.

  • Find out strengths and weakness and how to use them
  • Get similar resources as possible
  • Share work
  • Celebrate success
  • Minimise dependencies
  • Offer help

CeAdGhwWIAExbPe.jpg large


In summary, the workshop was a success. We learn so much and me and my colleague is planning to remake the workshop in our office. I am currently planning it and Lisa and Abby is helping me to shape it.

Thank you Lisa, Abby for the great experience and Rosi Sherry(@rosiesherry) for organising this!




SWTC:What is testing?

Updated with a link to Richard’s post.

Mark Winteringham (@2bittester) and Dan Ashby (@DanAshby04) had come to this wonderful idea of sharing their experience and knowledge with testers who are new and experienced. This is also a good playground for the experienced to gain mentoring skills.

I went in as a student as I think I still need to learn a lot and I was right.

Mark, Dan and Tony Bruce (@tonybruce77) discussed about what is testing. Everyone’s input went on to the board.


If the list is not readable the items as follow,
•    Asserting expectations
•    Ensuring quality
•    Assuring quality
•    Understanding objectives
•    Making sure it works
•    Finding / preventing bugs
•    Discovering software behavior
•    Test scripts
•    Avoiding blame
•    Analyse risks
•    Making sure product meets objectives
•    Reporting
•    Meeting requirements
•    Gatekeeping

We discussed few items from the board such as,
•    Assuring quality
Question was how we can assure quality. It is a vague task depend on the person, environment and various other facts. Quality is a responsibility of the whole team. If someone gives list of things that needs to be attain to assure the quality, still there be issues as things change.

•    Gatekeeping
Can tester be Gatekeepers? If we say No, the product release be stopped or the management decide to go ahead? If so, why they need a gatekeeper? To fingerprint if something goes wrong. All testers can do is to point the risks and let the management to take the decision.

•    Test scripts
Test scripts / Test cases can be manual or automated. Testing done only using the test scripts/cases. No. Why do we need test scripts? Mostly for the repeating tasks that can followed systematically sounds like automation. Test scripts changes with the functionality changes manual or automate. Maintenance is necessary and it is costly. If you need someone to learn the system test scripts not the right way to follow. Encourage them to do exploratory testing.

Then Mark explained how he would explain testing. He explained how testing is always stay between Expectations and Actual product whereas testers we should try to keep the gap minimal.
Unfortunately, I could not take a picture of his diagram. Sorry Mark.
Dan explained testing with the diagram below.


Information is in focus. Information can be about product design, product idea, requirements, processors etc. Information is the centre of all of it.

Testing is uncovering more information via exploration or investigation. Uncovering information be done by asking questions, explore requirements, explore designs, and join in with discussions. The information that are uncover will inform new test ideas to explore.

Information enable many checking activities and regression checks one of them. When new features introduce to ensure the existing is not broken.  Checks done in a script format that can followed either by a person or by an automated process. The output from checks inform the broken tests as well as it can inform what not tested as well. If we have passing checks that can prove that the information are behaving as it required.

Tony explained how information be used for decision making.


Information gathering will lead to experiments, learning and explorations. Results of that can be collated. This collated information helps the software intelligent. With the software intelligent the business intelligence can be supported which lead to decision making.

This can be going in loop. When business require information to make a decision same process happens again from the beginning.

So the original list of “what is testing” updated.

•    Checking
•    Learning
•    Business decision
•    Communication
•    Uncovering
•    Value to a person
•    Communicate about what we have not done!

Practical session taken place afterwards to go through

Richard Bradshaw (@Friendlytester) was kindly agreed to mentor 3 of us.
This site we have not seen before and had no knowledge. Therefore, we spent about 5 minutes exploring the site to understand its functionality. We came to following conclusions.

  • It’s a game
  • Works in mobile / different browsers
  • Have an app in app store
  • It is interactive
  • Navigation
  • Possible bug
  • Etc (Shame I didn’t take a photo of that list!!!!)

Richard was advising us to,

Richard had written a Mentor Experience Report and you can access the post from here.

I am sure I have missed some, as I was not fast enough to note them all.
At the end, all teams came together and discussed the learning. Following are some.

  • Don’t come to a conclusion without knowing enough information
  • Build a practice / discipline as a tester and do not change because of the project.
  • Note taking/ mind mapping
  • Go through social media to gather testing ideas
  • If you are not happy using the product so will your customers
  • Explore test ideas

So that’s a wrap! Until next time…

Test plan : Does it need to be a detailed document?

Recently one asked me does the test plan needs to be a lengthy document.

My immediate answer was “No”.

Because of the puzzled look, I continues.

Lengthy documents used mainly in projects that used the waterfall method. In Agile world, the documentation is minimal. Yes, Test planning is a very important and essential activity that everyone should do. However, it should not be a lengthy document where no one reads or no one have the time to maintain it or yet do not have time to create it in the first place. It should capture the essential functionality that effected. We can use mind maps or a simple Excel table.

Then I saw a smile…

I am a big fan of test plans. In my current gig there is one big product, there are feature implementations, and extensions that happen every year to cater new business potentials and to retain customers. We are in continuous delivery mode where releases happen every week. Yes, short turn around and very happy customer base. When the new features prioritised and up for discussion (requirement gathering or design or estimate) there is a high possibility that these features will not implement in next few weeks. Some could be in months because suddenly work priority changed. (This project structure and agile concepts usage may not agree with you but it works… It works for the project at least. So we will skip all the questions around it) .How do I keep my thoughts over the design/requirements and possible break points?

I do the test planning right after the discussions. It will cover various things such as the local the impact be, what areas will be effected from the implementation, what areas I think will effect with the changes, high level workflow  or user journey, rules that being discussed, what customer base should be effected and not etc. This will be a mind map or a simple table.

When the functionality is up for development, then these test plans help me to gather my thoughts. What my thinking was. I was practicing this for many feature implementations and it always helpful. Because these notes are simple and to the point they are useful to everyone who are involved with that piece of work. I even used few of these artifacts to look back at any time if someone come to me and ask “Hey do you know what we have done in x feature by any chance?”

I may not use the conventional test plans. However, the planning I do works very well for my current project and me. This not only help me to collect my thoughts. I try to learn from others and implement the best practices they recommend. If it works, I continue to use it. I may do adjustments to cater to the project need. Otherwise lookout for more.
It is a continuous learning process and I hope you learn something new today or I have given a success story of a method you wanted to try.


Feature name
Tool ABC
Region affected UK, US  Initial development will be focused only for US
Browsers Default list IE7 to IE11, Latest Firefox, Latest Chrome, Latest Safari
Changes to Landing Page Template Change Feature flagged until the MVP is implemented / Navigation template ABC
Initially the content will be only enabled for US
Landing page category show as Modules instead of Categories
Sections Landing Page Logged out view About ABC box hidden
Logged in : unentitled  view About ABC box shows. (Text will be managed via content management tool)
Logged in : entitled  view About ABC box shows. (Text will be managed via EMT)
New and updated Logged out view 10 most recent articles of all articles
Logged in : unentitled  view 10 most recent articles of all articles
Logged in  : entitled  view 10 most recent articles of all articles
Content Modules 3 content module placeholders as default for all Tools
Icon New icon should displayed along side with a new colour
Search ABC tool articles should not be searchable via Main search
Local search box should be available
Search results Should return ABC articles on results
HR Learning Center refine filter should be available
Breadcrumb Breadcrumb should display on ABC tool from landing page onward to categories and articles.
Footer ABC tool should not display under Tools section
Tools Page ABC tool should not list under a defined business category
 Menu ABC tool should not list on main menu
EMT User should be able to add categories to ABC via EMT
Adding article source ABC is not MVP
Article page Only subscribers will be able to view the articles
Download feature should not be available.
S3 resources If S3 resources being mapped in to the ABC articles
Article styles The default functionality should be applied
Print article The default functionality should be applied
Subscribe form Not mentioned on initial requirements.
Request a demo form Not applicable as the functionality is for US
Feedback form Not applicable as the functionality is for US
Entitlements ABC will be an individual product
Product name : ABC_product_name
Tags Standard analytics tags should be included
Article Tags No special article tags
Google site map Should not index on Google. Sitemap should have No index, No follow to the tool