Visualization of testing is one of the new and exciting areas of testing. It is being born out of the need to see results and coverage of testing quickly. Remember the old days of writing test strategy documents at the beginning of a project, then rewriting them a week later, and then a week later until you get to the end of the project and the document looks nothing like it did at the beginning of the project. Oh and did I mention that the document never gets completed?!
Now you have stakeholders just assuming you did the right thing, documents that will just pass an ISO audit and a slight resentment to your project manager for allowing scope creep! Then came along strategy documents in a state diagram, like Graphical Test Strategy I discussed before, that allows you to get your document done in a day and sent out to stakeholders. All of a sudden you a hero to your "customers" as they become more involved in the quality of the end product.
You have drawn a couple pictures to show people what you are going to do but now lets have a look at drawing pictures to actually do the testing. If you carry on breaking down the state diagram you are given a beautiful test case diagram. Translating that into a automated test can be done with a tool called CubicTest. CubicTest is an Eclipse plugin that allows you create abstractions for test cases and then gives you Selenium or Watir Tests using a state diagram. Its a good tool if you are constantly using Eclipse but I never use Eclipse
Visualization also offers a lot of value to code metrics. Unit tests are only of value if they test different scenarios and if you are testing different scenarios you will inheritently get good code coverage. Code Coverage is a good way to see that you are testing properly. Well, that you are testing in the right areas. My tool of choice is NCover but thats because I work in a .NET shop.
All of this can be placed into a CI server and all it can then be reported on easily. I use CruiseControl.NET to do my CI and I love that it shows all the passes,ignores and fails of tests from the web front end. I can check up on the developers without having to get their code out of the source repository. The manual testing guys make their scripts update a database. Thanks to simple charting APIs( Google Charts ) I can see how projects are going from one page. So without too much cost and fuss we can see how testing is going and the stakeholders can feel attached to QA of the product from Development to Delivery! As always tools to do what I have talked have open source alternatives.
Unfortunately none of these tools make the lives of testers easier. It works is showing that we are doing well, or not if thats the case, but how do we know that we have a decent amount of code coverage on core libraries? The Microsoft XBoX team analyse their logs and apply Heat Maps so that they can see where their testers have been doing. But what if you took your normal code coverage report and put that through a heat map but put a weighting on the function. The weighting relates to how bad a bug would be if found in this area and if we are to work it how well its tested by the code coverage and the number of tests we can actually get a feel for what we are developing and testing.
Finally the thing that is desperately needed is a Testers Heads Up Display (T.H.U.D). James Whittaker discussed this in his series on the Future of Testing and in his Keynote at GTAC 2008. It takes the idea from games where the H.U.D. allows gamers to be really good at the game. The T.H.U.D will give you little notes when you are working saying that there are potentially bugs in this area and telling you about previous bugs in that area. My idea of the T.H.U.D would give you the information and then record where you have been so that you get a heat map over your application when you want to check weather you are finished in this area.
The faster the iterations get the more testers will need something like this. The average ratio of developers to testers seem to sit around 3:1 and it takes just one developer not to unit test their code to slow a tester down since they have to look for smaller bugs as well as those hard to find that we testers look forward to finding.
This leads me onto my final post in the series: Innovation
Wednesday, 11 February 2009
Testing through the Credit Crunch - Part 5 - Visualization
Wednesday, 13 February 2008
Graphical Test Planning
In December I went to conference in London and one of the lectures that I went to listen to was about Graphical Test Planning by Hardeep Sharma of Citrix Systems.
Graphical Test Planning is a way of creating a test plan without having to write entire documents. I am sure that most of the readers
that will be reading this will be practise full agile development and are probably
thinking that I am mad.
Maybe I am mad but I really think that this is a great way to work out what needs to be testing and
is a good way to track progress of testing.
The entry below is more my thoughts having tried this technique on web projects
and have used a web examples. I hope you enjoy reading it!
Graphical Test Planning is done by creating a structural relationship diagrams.
These
allow product managers to know what is going to be tested from the day that they decide there is going to be a new version of the product.
What a Graphical Test Plan is or isn't
What a Graphical Test Plan is:
- A structured relationship diagram
- A list of behavioural areas that need to be tested
- A method of getting greating feedback about what needs to be tested
- A method of tracking progress of design and execution of testing
What a Graphical Test Plan isn't:
- A flow diagram
- A tester managers thoughts or mind map
- A feature list
- A hand-holding exercise for a test engineer on how to use the application
How To Create A Structured Relationship Diagram
A structured relationship diagram will show what behavioural areas need to be
tested on different platforms and different compilers. Below is an example of
what a SRD looks like for Selenium.
Its not 100% correct for selenium but I wanted to show something
with no coverage.
From this you can see what areas need to be tested and what doesn't. If we carry this theme on we can then
start drilling the behavioural areas down into more detailed SRDs and if we were to
carry on drilling down we will start to create another form of these diagrams
called a Test Case Diagram.
A Test case diagram looks a lot like a SRD except at the end of the digram instead of showing the behavioural areas it will show the expected results.
The great thing about this is that you can create the SRD in about 10 minutes and then do the test case diagrams in a few hours and this means that you
can have all this test information done before any code is done.
you would have saved yourself somewhere in the region of 1 man month and you are likely to have fixed a number
of flaws in the design before any code is near completion.
Hardeep Sharma for showing off this technique at the SIGIST Conference in
December.
