As most of you know I am a really big fan of Selenium for testing of Web UI and as of Tuesday I joined the development team working on the next release of the Selenium. I am helping Jim Evans on the .NET bindings.
Selenium 2 is the merging of Selenium and WebDriver to make the whats going to be the best Web UI test framework out there.
Selenium 2 will help both WebDriver and Selenium get over their respective weaknesses. So, for those who have a decent coverage and your tests have been running for a while why should you think about moving your tests to Selenium 2?
Well I will explain why I have started moving my tests. Firstly, I love to dogfood my own code so writing my tests was a natural progression. Secondly, the ability to write really verbose tests without having to try really hard.
I have started moving my Internet Explorer Selenium Tests to use the WebDriver API to test the Editor component of our site. This is one of the hardest areas to automate with Selenium 1 because its using contentEditable on a div. I managed to test this using the components JavaScript API to inject HTML and then manipulate it and then use the API to get the HTML out again and check that it is correct. Running 1 test in Internet Explorer for this would take up to 90 seconds to complete. I would like to add a disclaimer that the test had been optimized as much as it could with no XPath, etc so it was running as fast as possible
Now to start porting the tests over to the new API. After working out the little issues in how to get to certain elements I was able to port over a number of tests. I was able to port over 4 tests in a little bit of work and then thought to run the tests. Any guesses to how long it took to run all 4 tests?
360 seconds?
270 seconds?
120 seconds?
60 seconds?
30 seconds?
The answer is 60 seconds. Well it was 57 seconds but I needed to pick from the list I gave. 4 tests in 2/3 of the time to run the original test to check for the exact same thing. The new test have very little JavaScript pokng in the test because I use the native keystrokes to type into the div where I could not with Selenium 1. This means that the tests do not have to handle the JavaScript scope chain to do basic things and makes the tests lightning fast.
The next test will be to port the WebDriverBackedSelenium code from Java to .NET and then run the first tests again to see how much faster those tests run. So if you want really fast tests then start thinking about moving to Selenium 2.
p.s. If you haven't filled in my questionaire on Selenium 2 presentations and demo's please do so here
Thursday 4 February 2010
Why should I move to Selenium 2?
Subscribe to:
Post Comments (Atom)
12 comments:
Hi David, I have a question about Selenium 2. Does it have support for CSS locators yet? we use CSS to locate elements on the page which is much faster than XPaths and as per my knowledge Selenium 2 does not support CSS locator yet. But if it does then we will have no hesitation to port our tests to Selenium 2.
At the moment, CSS Selectors are only supported if they are native in the Browser and unfortunately IE doesnt support CSS selectors for IE6 and has gaps in later versions.
I know Simon Stewart is looking for a CSS Selector library for browsers that don't support native CSS.
Haved used selenium ide for a while and now have struggled for 2 weeks to get selenium RC working via Junit - was almost going to drop RC because of all the issues which arose because of incompatibilities between RC and different versions of Firefox ; but finally got it all working ( RC 1.01 with FF 3.5.7 and hopefully tonight I'll find out the configuration works with IE 7 and GChrome 4.0). So now you say Selenium 2! I'm now pulling out the rest of my hair as I read the documentation. I'm anticipating having to do a complete rewrite of our test as the Webdriver and Selenium API "sort of get along". So my 2 concerns are API compatibility and the implied deprecated use of Selenium IDE (which is a great Agile development tool). Should I be concerned?
There is a really handy interface that we are adding that allows everyones Selenium 1 code to work on Selenium 2. There is a class WebDriverBackedSelenium that you use instead of the Default Selenium instantiation and nothing else needs to change.
Regarding the IDE, we have no plans to drop it, in fact there is a mini project to see what we need for Selenium IDE 2 which Frank Cohen is leading. I agree that the idea is great and I wouldn't want to lose it as its a good way to build your tests quickly and then port them over so you can make then really robust.
The Selenium 2 project is trying to make an even better Selenium for everyone to use!
Thanks David. So I tried an experiment with the WebDriverBackedSelenium approach. The question I had was could I take my 20 Java Junit Selenium classes and run them with no changes to the test code - any necessary changes to import statements, package namespace and inheritance was acceptable. After struggling for about 16 hours I gave up. A consistent error I got was: [junit] W12:45:12:781 .\IEThread.cpp(686) -2147467262 [No such interface supported]: Cannot obtain script engine.
The other problem, certificates, are a known bug. I tried many solutions, but couldn't get any to work. So full automation with IE is not yet possible.
So now I'm writing a wrapper class that uses webdriver API to implement all the interfaces of the Selenium class; like the WebDriverBackedSelenium approach, no changes to the test code is necessary. The few interfaces that I have implemented work fine but this is a lot of work and a fragile solution - so I don't think I should pursue it. What do you think of a wrapper class idea?
@Grizzly,
I have not worked with the Java Implementation of this. I have seen it working in the past. If you are getting an error, it might be worth sending a message to the Selenium User group on Google Groups so people can try help you work the issue out.
Your wrapper class idea sounds exactly like WebDriverBackedSelenium because your test only needs to update how it creates the selenium object. So I would avoid that route and just raise bugs against the WebDriverBackedSelenium so we can help people not have these issues.
If I understand this correctly - then selenium 2 is not going to support html test cases. If so - it will be a step back in popularity - and if it does have good IDE/recorder then hardly anyone will use it.
HTML test cases will still work but will only work as Selenium 1. This means that the users who want to stay with Selenium 1 but will be missing out on all the new features of Selenium 2.
Ideally people should be writing their tests in a proper programming language either from scratch or getting the body of a test together in the IDE and then Exporting it.
I know I would prefer people to get used to Selenium in the IDE and then port the tests to a proper browser so they can take advantage of DataDriven Tests or using Testing Frameworks to do the hard work like ignoring test if need be.
The html version of the tests are not very good in Continuous Integration servers because they dont offer the data you want in the format you want where if you did proper programming language you would.
Its a fine line trying to support non-technical people and technical people alike that we tread carefully to support them both
Selenium 2 should probably be called something else - if it does not support html - so that
people can fork selenium (like struts 2). There are many people who use it as a recorder.
Why cant selenium 2 export test in JSP(not xhtml) or ASP ? (ie most code in HTML)
Exporting it to JSP and ASP assumes that you are going to have an environment to process it. That rules out people from a couple languages.
"Record and Replay" is where we went wrong. The name should be "Record, Tweak and Replay, Export to a language". As I said in a previous reply, people not using a programming language to drive Selenium are doing themselves a huge disservice because there are things that are in RC that can't be in any of the other Selenium flavours.
Adam Goucher blogged about this here http://adam.goucher.ca/?p=1198
Hi David. I've started running some selenium tests in Python. I've experienced some cross browser issues esp. in regard to keypresses with an ajax autocomplete widget. I've gotten past the issue, mostly because of my cross-browser experience, and knowing what was going on. Will Selenium2/webdriver address any of these issues, or is that just part of ui testing at this point?
Hi David, I am looking for possibility of upgrading my selenium 1 to selenium 2. Some questions though:
1) You mentioned in one of your comments that "Ideally people should be writing their tests in a proper programming language either from scratch...". Does it mean if I've actually coded my tests (instead of using a recorder), I should be able to migrate my tests to selenium 2 without drastic changes to my codes, using WebDriverBackedSelenium?
2) I've had much trouble with the selenium Click() and WaitForPageToLoad() methods, which are known problematic. Would you happen to know if selenium 2 provides some improvement in that area?
Post a Comment