SWTC: Oracles and Heuristics

This is my take on Oracles and Heuristics that was discussed on the 3rd SWTC session.

What is a heuristic? It can be a,

  • Pattern ( That you identified from your experience)
  • Different view
  • Example
  • Way of thinking
  • Trigger
  • Past lesson guidance
  • Lab experiment
  • Method of solving a problem – fallible
  • Mental mode

 

How to test data using heuristics?

  • CRUD (Create, Read, Update, Delete)
  • Migration
  • Goldilocks
    • Too big
    • Too small
    • Just right
  • 0, 1 or many
  • STRIDE (Spoofing, Tampering, Repudiation, Information disclosure, Denial of service, Elevation of privilege)
  • ETL ( Extract, Transform, Load)
  • Follow data

Mnemonics – read http://www.satisfice.com/articles/sfdpo.shtml

So what is Oracles?

Its all about “how do you know what you know”

People use the “Six honest men practice”, FEWHICCUPPS , Rule of thumb, Educational guess (mostly in estimates) and also fresh eyes to explain Oracles. 

F for functionality -> from experience / people

E for Explainable

W for world -> Environment, context of the project / products.

H for History -> history of the product or about the legacy application

I for Image -. Image of the company, image of the product and it represent

C for claims -> product specifications / artifacts, sales or marketing materials, help text, client claims such as bugs

C for comparable -> products can be compared with other similar products in the market, with the legacy application

U for Users -> the people who matter and their feedback

P for Product -> functionality, styling.

P for Purpose -> Answer to the question Why?

S for Standards -> guidelines, heuristics

 

Then it was the game time. We were supposed to determine/invent the game using oracles and heuristics. We came close to the original but it was fun.

  One thing I understood is I was using most of these techniques from my experience and lessons learned but never had a name.

Advertisements

SWTC: Testing requirements 

The last session of software testing clinic focused on how to test requirements. Session conducted by Mark (@2bittester), Dan (@DanAshby04) and Alan (@alan_parkinson).

We discussed how ambiguous requirements could be and the importance of requirements be detailed and transparent. Requirements can be interpreted differently by each individuals experience and views which can result the wrong outcome. Alan hosted a game to illustrate this.

He asked everyone to pair up and then decide who the developer is and who the product owner is. Then all who volunteer as developer sent out where they cannot hear or see the rest. Alan gave the product owners a picture, which more like modern art and a blank paper. He asked us to describe the picture for the developer where they can draw it. Yes, some tried to draw the picture as instructions or prototype but it not allowed. We had given 5 minutes to use words to describe the picture.  It was hard. The picture I got seems to be easy to describe compare to some pictures others got!

Then we were sent out with the picture and leaving the notes. Developers came in. They had 5 minutes to draw what described. Well we who were outside could see how confused they are. Then we went in and started comparing the pictures. There were laughter, complains, questions… It was loud. Perfect example to explain why your explanations to be detailed and not let any room for ambiguity or assumptions.

You can see below the original picture, detail description and then the output. Does not matter how much I have described the picture it had not thought from the other person’s point of view resulting confusion and wrong output.

 

Behaviour driven development (BDD) encourage everyone to reduce the ambiguity as it encourage communication. It is all about collaboration and conversation. There are many ways of understanding the requirements such as,

  • Asking questions
  • Drawing the solution / user journey on paper /whiteboard
  • Asking the value of the requirement
  • By generating mind maps , feature files in BDD and also acceptance criteria
  • Gather the 3 amigos for formal/informal discussions

Sometimes we do three amigo discussions without realising or enforcing. For an example when a developer have a question and they go to the business analyst to clarify. Test analyst hear the discussion and join in. Sometimes it then taken to the product owner. Alternatively, have a discussion at the watercooler. We all practice it knowingly or unknowingly.

However, always have to remember to,

  • Have the conversation
  • Capture the conversation
  • Implement the conversation

Is the acceptance criteria same as Given/When/Then? It is not! Acceptance criteria is business rules. Business rules could written in,

  • High level points
  • Given/When/Then format to explain with examples

What we must try is to understand the business rules so the implementation is valid.

Dan asked us to test the following requirement where Mark act as the product owner answering all the questions. He actually did a good job making us feel frustrated with his answers to some questions. So the requirement as follow.

req

All the questions everyone asked written in mind map format. The groups were What? Where? When? Who? Why? and How?

Then the practical session began. Ioana (@ioanasaysgo) as the mentor she had to come up with a requirement where we have to ask questions to clarify the requirement.  Questions were asked What? Where? When? Who? Why? and How? method. The discussion result generating visuals and understanding the requirement is complicated.

We also discussed about clarify assumptions, review requirements before develop and story mapping.

Session ended with Tony’s (@tonybruce77) motto.

Apart from my train got cancelled half way and resulting me go home late in the night, it was a fantastic day. Until next time…

SWTC:What is testing?

Updated with a link to Richard’s post.

Mark Winteringham (@2bittester) and Dan Ashby (@DanAshby04) had come to this wonderful idea of sharing their experience and knowledge with testers who are new and experienced. This is also a good playground for the experienced to gain mentoring skills.

I went in as a student as I think I still need to learn a lot and I was right.

Mark, Dan and Tony Bruce (@tonybruce77) discussed about what is testing. Everyone’s input went on to the board.

1

If the list is not readable the items as follow,
•    Asserting expectations
•    Ensuring quality
•    Assuring quality
•    Understanding objectives
•    Making sure it works
•    Finding / preventing bugs
•    Discovering software behavior
•    Test scripts
•    Avoiding blame
•    Analyse risks
•    Making sure product meets objectives
•    Reporting
•    Meeting requirements
•    Gatekeeping

We discussed few items from the board such as,
•    Assuring quality
Question was how we can assure quality. It is a vague task depend on the person, environment and various other facts. Quality is a responsibility of the whole team. If someone gives list of things that needs to be attain to assure the quality, still there be issues as things change.

•    Gatekeeping
Can tester be Gatekeepers? If we say No, the product release be stopped or the management decide to go ahead? If so, why they need a gatekeeper? To fingerprint if something goes wrong. All testers can do is to point the risks and let the management to take the decision.

•    Test scripts
Test scripts / Test cases can be manual or automated. Testing done only using the test scripts/cases. No. Why do we need test scripts? Mostly for the repeating tasks that can followed systematically sounds like automation. Test scripts changes with the functionality changes manual or automate. Maintenance is necessary and it is costly. If you need someone to learn the system test scripts not the right way to follow. Encourage them to do exploratory testing.

Then Mark explained how he would explain testing. He explained how testing is always stay between Expectations and Actual product whereas testers we should try to keep the gap minimal.
Unfortunately, I could not take a picture of his diagram. Sorry Mark.
Dan explained testing with the diagram below.

2

Information is in focus. Information can be about product design, product idea, requirements, processors etc. Information is the centre of all of it.

Testing is uncovering more information via exploration or investigation. Uncovering information be done by asking questions, explore requirements, explore designs, and join in with discussions. The information that are uncover will inform new test ideas to explore.

Information enable many checking activities and regression checks one of them. When new features introduce to ensure the existing is not broken.  Checks done in a script format that can followed either by a person or by an automated process. The output from checks inform the broken tests as well as it can inform what not tested as well. If we have passing checks that can prove that the information are behaving as it required.

Tony explained how information be used for decision making.

3

Information gathering will lead to experiments, learning and explorations. Results of that can be collated. This collated information helps the software intelligent. With the software intelligent the business intelligence can be supported which lead to decision making.

This can be going in loop. When business require information to make a decision same process happens again from the beginning.

So the original list of “what is testing” updated.

•    Checking
•    Learning
•    Business decision
•    Communication
•    Uncovering
•    Value to a person
•    Communicate about what we have not done!
4

Practical session taken place afterwards to go through http://www.drawastickman.com.

Richard Bradshaw (@Friendlytester) was kindly agreed to mentor 3 of us.
This site we have not seen before and had no knowledge. Therefore, we spent about 5 minutes exploring the site to understand its functionality. We came to following conclusions.

  • It’s a game
  • Works in mobile / different browsers
  • Have an app in app store
  • It is interactive
  • Navigation
  • Possible bug
  • Etc (Shame I didn’t take a photo of that list!!!!)

Richard was advising us to,

Richard had written a Mentor Experience Report and you can access the post from here.

I am sure I have missed some, as I was not fast enough to note them all.
At the end, all teams came together and discussed the learning. Following are some.

  • Don’t come to a conclusion without knowing enough information
  • Build a practice / discipline as a tester and do not change because of the project.
  • Note taking/ mind mapping
  • Go through social media to gather testing ideas
  • If you are not happy using the product so will your customers
  • Explore test ideas

So that’s a wrap! Until next time…