Pages

Friday, February 28, 2014

Why not to use keyword driven framework "Always"?


There is no other reason to use automation testing but to reduce the regression effort, but many people have misconception that automation testing is used to find new bugs, automate all manual test cases and it will remove manual testing. 

I am not agree if you are also thinking the same. Working on live automation testing projects I encountered this issues many times that neither automation framework can fit to automate everything, or catch new bugs or remove manual testing. But it can just use to reduce regression efforts. In fact the thumb rule is we should always keep some scope of manual testing.

We have many test framework and the most popular are QTP, VSTS, Rational Robot, AutoIT, TOSCA and many others. They are record and play or keyword driven. So as my title says why not to use these type of framework "Always"? Lets see some advantages and disadvantages which will give clear idea why not use use "Always".  

Advantages: 


  • Provide easy way to record and play the UI actions
  • Usually comes with test management tool and bug tracking
  • Non programmer user can also record and play the actions
  • Easy to re-record user actions
  • User friendly UI and wizards to configure and execute tests

Disadvantages


  • Hard-coded values which must needs to change if anything at all changes in our product. 
  • Cost of the framework itself is too high and you need support from manufacturer which will add another cost
  • The costs associated with maintaining such framework's scripts are astronomical, and some time unacceptable. Because they use their own defined scripting language.
  • These scripts are not reliable, even if the application has not changed, it often fails on replay (pop-up windows, messages, and other things can happen that did not happen when the test was recorded).
  • Need to follow the associated programming language and tools and must follow the pre-defined rules. Some times testers needs training to understand their language and data types. Example QTP, they claims it uses VBScript but in reality it is the modified version of vbscript.
  • Can not use single framework to automate all kind of apps or validate.
  • We must need to use hybrid approach, which is a combination of different automation framework and tools to fulfill the need of a complete project
  • If the application changes, the test must be re-recorded. But cannot be record if application is not corrected or build is released.
  • This can be only applicable for standard UI application, if UI components are complex or custom made it is impossible to use without huge modification in tool/recorded script
  • Difficult to share the recorded UI objects with multiple projects
  • Cost of maintenance is comparatively higher in long run and need lot of licences
  • Difficult to debug the recorded scripts
  • Record/Play can not record complex UI combination or custom controls.

If you look at the disadvantages then it is more than the advantages. So it is not always advisable to use Keyword Driver or record Play framework. 

Then What to use which can fulfill the complete requirement? I love test library architecture framework or script based framework. This is the most appropriate way of modern application automation.

Please drop your feedback or questions.


No comments:

Post a Comment