By now you will have hopefully realised that running a Test Department doesn't have to be expensive. We have seen that by making the entire company responsible for the quality of the product we take the strain off us doing the developers jobs and lets us find those show stopper bugs.
In this post I am going to discuss why you should value open source tools to automate finding those showstoppers.
Lets start at the small thing to be tools we need at the beginning:
Continuous Integration
How many time in the past have you released something and then suddenly have to roll back the release because not everything works together. Continuous Integration Servers get code out of the source repository. I am a big fan of Subversion as a source reprository, which just happens to be open source!
The top CI server application out there are CruiseControl, Cruise Control.NET, Hudson and TeamCity. 3 of those 4 are Open Source. The one that is not open source, TeamCity, requires a you pay for a licence if you want all the bells and whistles. You should be able to manage on the free version of TeamCity if that the one you decide on.
One of the most important features of all these tools is the ability to automatically run the unit tests that have been created by the developers.
Unit Testing
The de facto testing tools for Unit Testing are NUnit, JUnit and TestNG for .NET and Java. There are loads more out there, like unittest for python, that are used all the time. These tools are great for integrating into Continuous Integration Servers.
In the 2nd post of the series I mentioned that having unit tests allows us to have a safety net to work against. The CI servers will run every developers unit test. This allows developers to do a check-in/commit, once their tests pass, and then get the rest of the tests run. This means that if you make a change you will find out if you have by accident broke something. You will learn to appreciate this safety net but make sure that you don't become lazy and use that as your only testing.
Functional Testing
Web applications have the largest amount of Automation tools for functional testing. Since I also work in this industry I will be very biased towards the following applications.
- Selenium
Selenium is slowly becoming the test tool of choice for Rich Internet Applications. Selenium allows you to write functional tests against AJAX applications. Selenium is used by the top Internet application companies and Agile Consultants.
Its developed using JavaScript and can work in all Browsers on any OS. As you would have noticed I am a big fan! - Watir
Developed in Ruby its has a very good following. Test Scripts need to be developed in Ruby and you just import Gems for the Browser that you want to test.
I found that it needed a lot of work to test multiple browsers compared to Selenium. I know some people would say its easier to use that Selenium. I guess its down to the user.
More and more companies are starting to use acceptance testing tools to capture their requirements. Acceptance test frameworks can all integrated into Continuous Integration Servers. They take a plain text requirement and turn it into a Test Fixture.
The main tool for acceptance in my opinion is a manual tool commonly known as the customer. Remember that customers are both internal and external to the company! End users are the best acceptance testers as they are the subject matter experts and spot bugs quite quickly.
In the next post of the series I will discuss Visualization of Testing
2 comments:
I love the "testing through the credit crunch" series - thanks for putting this together. And of course, thanks for your support of Selenium. Hopefully the recent releases of 1.0 beta 2 are making it even better for you!
One thing you didn't cover in this post is load testing. For open source, there is JMeter, OpenSTA, Apache ab, etc. Might be worth a discussion.
Also of note, my company, BrowserMob, provides a load testing service that is low cost and uses Selenium to drive real browsers.
It's not free, but neither is open source (there are hidden costs, like labor and hardware). I built BrowserMob with the down economy in mind: I felt it was the perfect time to offer a high quality product at a low price point.
You can also check Robot Framework - an open source tool for acceptance testing. It's available at http://robotframework.googlecode.com
Post a Comment