I’ve come some projects where repeated components seemed to be the trait that stood out. This was by far every automation engineer’s dream; to write some code that could be reusable across lots of different test cases. This however has proven that it could be every automation engineer’s nightmare too.
To be able to have a chunk of code with a size fits all character; it has to be quite configurable. In order for this code to be configurable, there are going to be parameters, flags, counters, and quite a bit of control words. For anyone who studied some logic design, this code could be used nicely as a multiplexer of a sort, with different control words giving different results. The problem with this is that getting lost in all the configurations could turn into a nightmare, especially if it is used across different test cases; one test case could require adding 5 items to a grid for example, another would require removing these items, a third may not even relate to the grid in the first place. These could be controlled by 2 control words for example: items added, items removed, if both are nulls then we can just leave that grid alone. But what happens if there are other control words related? What happens if this is not the only reusable chunk of code? MAYHEM!
The best way to handle this is through proper design and figuring out exactly what is needed to be done in this module, it would be very hard to iterate with new and different requirements, so a bigger picture of the whole module needs to be figured out from the start. The features, however, need not be scripted at the start as soon as they’re all catered for so that later on no messes take place.
The other trap that may come up with the reusable code issue is simply GREED! Once you have some code that can be used with lots of scenarios, you will want to run each and every scenario this code is capable of. So let’s say you have 5 control words for one chunk of code, assume they’re all Boolean values to make things easier. In order to get all the possible scenarios – do not tell me it is not tempting to think about exhaustive testing – you will need to have 32 automated test cases. You will end up with more test cases that you actually need, which is weird because us testers are always pressed for time and the more time you give us the more tests we’ll run, but what good does it do in acceptance or regression tests to cover the testing exhaustively. It will take just as much time to analyze and maintain. In addition, having isolated parameters will not really help all that much with finding new bugs, so it would be a waste of time.
Luckily, I came across this really cute, smart, and effective idea about handling testing in a non-exhaustive manner while still maintaining a nice % of coverage; Pairwise testing! For those of you who haven’t heard of pairwise and y’all call yourselves testers, shame on you and (for you who know Arabic, thakalatkom ommokom) go to http://www.pairwise.org/ for more info on the matter. The whole idea of pairwise is that instead of having an exhaustive set of tests (the 32 we talked about before), any 2 parameters can be used to generate a failure. So you need to cover all the values of parameters in pairs instead of exhaustively. We’re engineers so we won’t talk about how the algorithm works. How does this relate to reuse in automation? That’s a secret :P
Okay not a secret, what if we use pairwise to generate the different sets of control words, that way, we can have a smaller number of test cases with a decent degree of coverage, and with larger sets of control words, this would be quite the charmer. There are a lot of tools that could generate pairwise combinations of certain parameters, thus giving us the amount of test cases we want without wasting time in test runs and analysis. Pretty cool huh ;)
Code reuse in automation can be a double edged sowrd that needs to be used wisely. Having less lines of code could seriously compromise its complexity which will cause a disaster in script maintenance that could ruin the entire test suite with a tiny bug (yes automated code has bugs… it’s CODE!) so I am stressing – AGAIN – on design and objective before going into automating scripts. And God help us all!
No comments:
Post a Comment