GTAC 2008 - Day 2

Day 2 of GTAC 2008 has been quite interesting. The talks that have interested me were the talk on Context Driven Testing.

This talk, by Pete Schneider of F5, dealt with what context that tests were being created. It dealt with a common issue that testers and deverlopers have and most of them ignore. It was really interesting in seeing that they were 11 different applications that testers had created and were maintaining but there was a lot of overlap. They found that creating tests in the right context and with the right owners of tests has dramatically made a difference in getting their application tested.

The main thing that I took away from it is; When creating a testing framework ask yourself the following questions

• Who is going to write the framework
• Who is going to build and maintain it
• How are you going to use it
• How long will the tests live

The next talk that was really interesting was on Automated Model-based testing by Atif Memon and Oluwaseun Akinmade of University of Maryland. It was a really good talk on creating models for test cases, similar to my Graphical Test Strategy post from the beginning of the year. The difference between my post and this talk was the model is created on the fly by the test code that then builds a WebDriver test case. There is one draw back that I noticed and I know I wasn't the only one. This modelling works well on websites that don't have AJAX. Its a big draw back and they did mention that they are working to get it sorted.

Its value for Desktop applications is really good and I would definitely recommend it. It is able to recreate states for the test and make sure that works well and has good coverage.


The next talk was on the value of unit tests by Christopher Semturs. He was advocating the use of Mocks in your unit tests so that you can get better granularity of your tests. This is something that really interests me because I like making my tests fast because if a test is fast a developer is more likely to run the tests. He was advocating the idea of small,medium and large tests. Its an idea from some Google guys where you do unit tests in isolation and then do pair-wise integration tests. This means that you drop your large end to end tests number dramatically.


GTAC 2008 has been really cool with a lot of clever people talking about different aspects of testing and describing what is needed from testing in the future. Its definitely been worth the jetlag!

GTAC 2008 - Day 1

Day one of GTAC has been really cool. There have been a number of really good talks. I will put links to the YouTube videos when I know them.

James Whittaker's opening talk was a really good start to the day. One of the main things he talked about was the visualization of testing for the future. He discussed the way that people can visualise when there are new code differences. This can be very useful in making sure that testers can see what is new and what should be concentrated on. I am always keen on using visual ways to test and make sure that the quality is high.

The next talk that caught my attention was "Advances in Automated Software Technologies". This talk discussed the idea of Autonomous Computing in software testing and the idea that you can auto generate test cases against APIs. It is an interesting idea but the way that it was put across was that you can only use this on items where the requirements are rock solid. This is not really something that can be applied to Agile developments.

The talk on Groovy was really good. I have started playing with Groovy when trying to automate SOA testing using SoapUI. Its a nice language that can be strong typed or weak typed at the same time and really useful for scripting. Since I haven't played with it a lot I did learn a lot like you can hook into all languages that use the JVM.

The next talk that really interested me was the last talk of the day. It was on the testability of code. It was looking to see how easy is it to test and making tests simpler. This is important because it means that your tests become manageable and supportable. Vishal Chowdhary was advocating SOCK:

• Simplicity of your code
• Observe how things work and interact
  
• Control of your tests
• Knowledge of what it should be done
  

He was also saying that we need to make sure that we don't overdesign software because this can add extra complexity when there doesn't need to be!

The talks have been very good and have stimulated some ideas that I want to take back to work! I am now off to the Google Seattle office for a tour the lightning talks.

.Net Gem - How to Unit Test Internal Methods

I have recently started redoing unit tests for bit of code at work and came across a internal methods and classes that needed to be tested.

This posed a question of: how do I access the internal methods and classes to test them properly from an external assembly?

I found this little gem that a lot of the developers where I work had not heard of, this doesn't mean that its not common knowledge but thought I would share it a little more.

I have put a scenario below of how it might work.

Being a good TDD developer you decide that you want to write your unit tests for an internal method. You create your .Net Class Library structure so you know what your tests need to call to do your asserts. It may looks something like the image below.

So now you want to start writing your tests and notice that your intellisense is not bringing up your internal class. So what do you do next?

Open the AssemblyInfo.cs file that is in your Properties folder. Add the line [assembly:InternalsVisibleTo("Unit.Tests")] somewhere near the top.

Now your Internal Classes and methods are only visible to Unit.Tests assembly.

Your start writing your unit tests and intellisense should be playing Mr. Nice Guy and we should see something like below.

I hope that it has been useful!

Are regression packs still worthwhile?

Recently I was on forum and noticed the thread "Why do we do Regression testing?". In it the person was complaining about their company's regression pack and asking if it was worth the effort of fixing it?

My answer was a little quick off the mark. I replied saying "Definitely! Its the backbone for each and every build!". But then this got me thinking, How many testers out there DO NOT practice Agile development? How many testers DO NOT work in an environment where automation of tests is 2nd nature?

I do not ask this question in a malicious way, I genuinely would like to know. When I go to SIGiST there are a number of questions in the presentations/workshops by people who have not worked in an Agile environment and, to be honest, seem scared by it. These are the same people when you mention something like "Continuous Integration" and they get that deer-in-headlights look on their face.

This leads me to my next question; Is there a "Rich get richer, Poor get poorer" divide happening in testing? People in conferences mention that the lines between development and testing is blurring. It's called grey-box testing. But your average manual tester in a large non-software institution (Banks, Insurance Companies, etc) have got their "break" in testing by being seconded to the test team for a project and never wanted to leave their secondment. They may not have been given opportunities to learn how to program or may not even have heard about concepts like Agile, TDD and all the other new ways testers do their jobs.

To these people; the answer to my original question is still the same: "Yes! Definitely! Otherwise things that used to work may not work anymore. I would also suggest trying to automate the regression pack as much as possible so that you can concentrate on making testing more efficient. Try automate as much as possible so that you can concentrate on doing testing on those hard to test areas. Quality of software is still the main objective of testing."

Google Test Automation Conference 2008

I have just got back to work from my summer holidays to find a nice little gift waiting for me in my Inbox!

I have been given an invite to the Google Test Automation Conference(GTAC) in Seattle, USA! GTAC for those who don't know is the annual conference that Google hold. Its an invite only conference where people can just let their geekyness flow!

This year its being held in Seattle, USA. Last year it was held in New York and the year before that it was in London.

Below is a list of the presentations at the conference

Scheduled Presentations

* Atom Publishing Protocol, Testing a Server Implementation by David Calavera
* JInjector: a Coverage and End-To-End Testing Framework for J2ME and RIM by Julian Harty, Olivier Gaillard, and Michael Sama
* Advances in Automated Software Testing Technologies by Elfriede Dustin
* Taming the Beast: How to test an AJAX Application by Markus Clermont and John Thomas
* Automated Model-Based Testing of Web Applications by Oluwaseun Akinmade and Prof. Atif M Memon
* Boosting Your Testing Productivity with Groovy by Andres Almiray
* Practicing Testability in the Real World by Vishal Chowdhary
* The New Genomics: Software Development at Petabyte Scale by Matt Wood
* The Value of Small Tests by Christopher Semturs
* Deployment and Test Automation: Extending the Notion of 'Done' to 'System Tested on a Production Deployment' by Marc-Elian Begin

I am going to try my best to update my blog while I am there so people know how its going. If anyone reading my blog is going conference drop me an email so we can meet up and talk shop!

Tester 2.0 - Definition of an Agile Tester

I have been to a number of SIGiST Conferences over the last year and have always been impressed by the calibre of the speakers there. There has been a theme lately talking about Agile Development and Agile Testing but the thing that always seems to missing is someone giving their interpretation on Agile Testing. To actually describe what an Agile Tester is! And what they should be doing.

Below I am going give my opinion of what an Agile Tester by describing skill set needed.We are after all only as good as our skill set! I am going to rate them from 1 to 10 with 10 being needed and 1 being desirable.

Skill	Score	Reason
Creativity	10	A good tester needs to be extremely creative when trying to test applications. Developers like to think that they are clever and get all the bugs before its released. Unfortunately this is not true when a tester starts being creative in the way they test the application and breaks it very quickly!
Innovation	10	A good tester needs to be able to innovative with the tools that they have. An example would be in SaaS(Software as a Service) companies who constantly striving to be the next big thing and competing against the likes of Google, Microsoft, Salesforce, etc. This drives innovation in development which means that it drives innovation in testing!
Test Automation	9	Every Agile tester needs to be able to do some form of automated testing. I am not saying that the tester must know how to use every tool that is out there but they need to understand how to create the automated tests.
Exploratory Testing	7	Agile Testing tends to mean Exploratory testing. I know that some people will disagree with this but because testing tends to happen on the fly and appears ad hoc but it isn't. Exploratory testing does have a set of rules that need to be adhered and will mostly bring a number of bugs to the surface just as well as normal scripted testing.
Communication	7	One of the main pillars of Agile Development is communication. This is very important in my opinion because a good tester must be able to communicate bugs to developers and requirements to end users. A good tester will also be able to challenge technical architects on their ideas. I know the last sentence can be very hard to achieve in large organisations but it doesn't mean that you can't send them an email to make suggestions.
Development	5	An Agile Tester should not be afraid to look at a developers code and if need be, hopefully in extreme cases, go in and correct it. I am not suggesting for one minute that they must be able to code entire applications but must have the confidence to look at a bug, spot the error in the code and either write a unit test to break it or point out the line where they think the issue is. An example of this is where doing Paired Programming with one developer we shaved of nearly 70% off the time it took to run a unit test test suite just by doing a one line change to their tests which got them to make one other small change.
Ability to work in pairs	3	I am a fan of being able to knock ideas off each other. I wouldn't do paired programming all the time but I like the concept and how it seems to get things done properly with little fuss.

This is not a comprehensive list but these for me are the most important things. Tester 2.0 is something that a number of testers need to strive for if they do not want to get left behind. There is a definite blurring of lines between development and testing. Gone are the days of "I have finished my part, over the wall it goes!". There is also a move to more "Grey Box" testing because of this blurring.

Interpretations of the V-model

I have recently been doing interviews and one question that I like to ask interviewees is "Where do you feel testers should sit within the V-Model?"

The V-Model (as pictured below) is a representation of how most "development" tasks have an equal "testing" task. I like to think of it as testings equivalent to Newton 3rd law.

This should be a good thing because it shows that there are different aspects of testing that need to be done during the software development lifecycle.

All the interviewee's seemed to give a different answer to the question above. Being the interviewer I was hoping for one of 2 answers.

The first answer would be one that made me rethink my thoughts on the V-Model. To be honest as an interviewer I am always looking forward to someone making me rethink a topic.

The other answer is that testing should encompass the entire V-Model. From the beginning of the Business Requirements stage to the end of the Acceptance Testing stage. The reason why I think this is because Quality Assurance should be in every aspect of the software development lifecycle.

I know that this is something from the Waterfall methodology but its a good way to see what needs to be done in the lifecycle and can easily be transformed into an Agile development.