Contracts have been signed

Today I officially received my contract to write a book on Selenium. I am going to be writing a beginners guide showing people how they are start test automation working using Selenium IDE then moving on to Selenium RC and Selenium Grid. There will also be chapters on Selenium 2.

The outline I went for was to help people who want to learn Selenium so they can maintain existing tests but also start writing tests using Selenium 2 and migrating their Selenium 1 tests over to Selenium 2.

So look out for it later this year!

Process.Start() issues with Mono on Mac OS X: My Story

Over the last month or so I have been spending my time writing .NET code for the new Selenium 2 Bindings. I have spent most of my time working on the Google Chrome Driver. But below is the story of 1 Selenium bug and 1 Mono bug and my exploration of how processes work in .NET on a Mac.

I was quite confident about the .NET bindings since it was extremely easy to get them working on my laptop. I admit I got nervous about how it would work on Non-Windows platforms since it was my first time writing .NET code on a platform that isn't windows. I shouldn't have been nervous. It was fairly easy to install Mono and MonoDevelop on my Ubuntu VM and ran the NUnit tests that we have. Google Chrome started and we had tests passing. Brillant because there was little code changes, I just updated the code where to find the Google Chrome application.

I was confident until I started working Mac OS X. I have very little experience on a Mac. Other than using it to run Selenium Grid or using it with a Browser I know very little about how the OS works.

On the Mac I installed Mono and MonoDevelop and fired them up. I went in and updated the code so that if the .NET code was running on a Mac it knew where to find the Google Chrome application and would then start up the process using the standard Process.Start() method from System.Diagnostics library. Feeling extremely confident I fired up NUnit and clicked Run. They all failed. Why? Well the browser wasn't loading. Selenium without a browser, excluding when used with HTMLUnit, isn't going to work.

I started debugging my code to see why the browser wasn't loading. The first issue that I found was the Selenium 2 .NET code wasn't recognising the Mac OS platform. We have logic that checks which platform you are on and we hadn't put a conditional for Mac OS. Updated my code and ran the tests again and they went red again. Why? No browser again.

Debugging again I came across a rather large and very annoying bug in Mono. Mono when running on Mac OS platform reports that it is running on Unix/Linux. This isn't too ridiculous since Mac OS is based on BSD however it means now that when ever the code needs to launch the browser it needs to up to 3 hits to the disk to see if it is where I think its. It also means the fix I added in Selenium earlier doesn't really matter. Anyway the code now has the ability to launch the browser...or does it? It doesn't as my tests are still failing.

I started debugging again. The error I was getting was that I was trying to open the application and open wouldn't accept the arguments I was passing in because they are not valid for Open. Since I have never developed on a Mac I had to do some research and found out that the defaults for Process.Start() want to Open the file rather than execute it. This is perfect if you want to open text files but not applications. So I tried to see about creating a bash script to start up Google Chrome. One obvious downside if it did start up was that I wouldn't have an easy way to kill it. I was starting to think that I was going to be doing Yak Shaving to get this to work.

I next had a look at using Monobjc. It is a library that allows you to write Cocoa code in .NET and there it uses a bridge to get your code working. It is something that one of the Watin guys had done to get Watin working on a Mac. After a lot of pulling my hair out for a while I gave up and asked a question on StackOverflow.

The 5th answer I was given, and the one I accepted, was that I had forgot to set a property called UseShellExecute. My code for starting up went from

chromeProcess = Process.Start(GetChromeFile(), String.Concat(Arguments, serverUrl));

to

chromeProcess = Process.Start(new ProcessStartInfo(GetChromeFile(), String.Concat(Arguments, serverUrl)) { UseShellExecute = false });

The subtle change of explicitly creating a new instance of ProcessStartInfo and updating the property { UseShellExecute = false } meant that my code went from opening an application to executing a application. The difference tells Process not to open a shell according to MSDN but on a Mac means it does what I want. I hope this is helpful for anyone else in the future.

Why should I move to Selenium 2?

As most of you know I am a really big fan of Selenium for testing of Web UI and as of Tuesday I joined the development team working on the next release of the Selenium. I am helping Jim Evans on the .NET bindings.

Selenium 2 is the merging of Selenium and WebDriver to make the whats going to be the best Web UI test framework out there.

Selenium 2 will help both WebDriver and Selenium get over their respective weaknesses. So, for those who have a decent coverage and your tests have been running for a while why should you think about moving your tests to Selenium 2?

Well I will explain why I have started moving my tests. Firstly, I love to dogfood my own code so writing my tests was a natural progression. Secondly, the ability to write really verbose tests without having to try really hard.

I have started moving my Internet Explorer Selenium Tests to use the WebDriver API to test the Editor component of our site. This is one of the hardest areas to automate with Selenium 1 because its using contentEditable on a div. I managed to test this using the components JavaScript API to inject HTML and then manipulate it and then use the API to get the HTML out again and check that it is correct. Running 1 test in Internet Explorer for this would take up to 90 seconds to complete. I would like to add a disclaimer that the test had been optimized as much as it could with no XPath, etc so it was running as fast as possible

Now to start porting the tests over to the new API. After working out the little issues in how to get to certain elements I was able to port over a number of tests. I was able to port over 4 tests in a little bit of work and then thought to run the tests. Any guesses to how long it took to run all 4 tests?

360 seconds?
270 seconds?
120 seconds?
60 seconds?
30 seconds?

The answer is 60 seconds. Well it was 57 seconds but I needed to pick from the list I gave. 4 tests in 2/3 of the time to run the original test to check for the exact same thing. The new test have very little JavaScript pokng in the test because I use the native keystrokes to type into the div where I could not with Selenium 1. This means that the tests do not have to handle the JavaScript scope chain to do basic things and makes the tests lightning fast.

The next test will be to port the WebDriverBackedSelenium code from Java to .NET and then run the first tests again to see how much faster those tests run. So if you want really fast tests then start thinking about moving to Selenium 2.

p.s. If you haven't filled in my questionaire on Selenium 2 presentations and demo's please do so here

Selenium 2 .NET Bindings And How We Can Use Them

Selenium 2 is one of the most anticipated updates to a testing framework in a very long time. The next version of Selenium will see the Selenium and WebDriver code bases merged. It will allow developers to be able to switch between Selenium and WebDriver without having to change your tests.

"So what" you might be thinking? Well, Selenium and WebDriver have their strengths and in a number of places they are in opposite places. For Example, Selenium works on every browser that supports javascript where WebDriver requires a driver for each browser.

The merge, destined for use by Selenium Remote Control and WebDriver users, will allow developers to write developer centric tests tests in their favourite language. So how does it do this at the moment? The Selenium and WebDriver developers have created an uber-jar that houses both the WebDriver and Selenium API's. Developers create their tests as normal and call the same server for both Selenium and WebDriver.

I am sure that you are still going "So what?".


Ever had a situation where you tried to use the down arrow in your Selenium test to only start pulling your hair out! This means if you use the WebDriver backed Selenium, you will be able to get around any issues what Selenium Remote Control may get stuck on. E.g. The most common issue is when trying to do keypresses, Selenium will synthesize the keypress using JavaScript where WebDriver will send through keypresses at the OS level.
We will also be able to create more intuitive tests since WebDriver allows people to create objects in their tests for each object on the page by using WebElements like this : - WebElement elementOnPage = driver.FindElement(By.Name("idOnPage")) and then use it like with typing elementOnPage.SendKeys("Text")

The .NET Bindings have a lot of potential and Jim Evans and the rest of the Selenium-WebDriver Developers are doing a brilliant job! Keep up the work guys!

So to get some people going with the .NET bindings I have created some tutorials:

• C#/NUnit Tutorial
• IronPython Tutorial
• IronRuby Tutorial

Once I get around a few Remote WebDriver issues with the .NET Bindings I hope to put up tutorials on how to use them.

Every time a child says, 'I don't believe in fairies,' there is a fairy somewhere that falls down dead.

Peter Pan is one of my favourite stories and the title of this post is one of the most iconic phrases to come out of the book.

It's at the beginning of the book when Wendy and Peter are speaking. Peter explains that when the first baby laughed, its laughter fell and broke into a thousand pieces. This big breakup of laughter means that there "ought to be one fairy for every boy and girl". Wendy notices that Peter said should and questions him.
"Ought to be? Isn't there?"
"Every time a child says, 'I don't believe in fairies,' there is a fairy somewhere that falls down dead." Peter replies.

The same can almost be said about testers when a developers don't check their code works. I don't mean a tester will die every time but a small part of their resolve dies everytime. If its not a testers resolve that dies, it may be real people that die. This may sound quite extreme but its not really.

Since the begiing of the Agile movement there has been a great movement to get the quality of software a lot higher. Aside from all the other major benefits of using scrum and XP, people have noticed the quality of software being delivered go up. They have done it by employing things that make the process more effective. Not all of these processes are efficient but they are extremely effective.

Testers resolve only dies when they feel that something could easily have been caught by the developers being effective at their job. Being efficient can lead to bugs because we are trying to develop a feature within a set timeframe which is normally slightly shorter than it should be.  This is not all doom and gloom, there are bugs that testers love to find. Remember the rule for testing that if a test case is good it will find a bug but remember that testers are not there to make sure that you are doing your job properly. They are there to make sure that its fit for purpose!

When developing software always remember that you are killing your testers really slowly  when you are not doing your job properly!

"To die will be an awfully big adventure." Peter Pan 

This is something that every tester, from load tester to functional tester to security tester, loves to do. Make an application fall to it knees because it we managed to find some big hole. Making an application die is an adventure in many many ways. From going through the process of breaking it, to working out the cause of the breakage.


I have just watched Peter Pan for the umpteenth time and it got me thinking about my profession. Treat your testers like you would Peter Pan and they will be a lot more effective in finding those elusive bugs!