Why not to use keyword driven framework "Always"?

There is no other reason to use automation testing but to reduce the regression effort, but many people have misconception that automation testing is used to find new bugs, automate all manual test cases and it will remove manual testing. 

I am not agree if you are also thinking the same. Working on live automation testing projects I encountered this issues many times that neither automation framework can fit to automate everything, or catch new bugs or remove manual testing. But it can just use to reduce regression efforts. In fact the thumb rule is we should always keep some scope of manual testing.

We have many test framework and the most popular are QTP, VSTS, Rational Robot, AutoIT, TOSCA and many others. They are record and play or keyword driven. So as my title says why not to use these type of framework "Always"? Lets see some advantages and disadvantages which will give clear idea why not use use "Always".  

Advantages: 

• Provide easy way to record and play the UI actions
• Usually comes with test management tool and bug tracking
• Non programmer user can also record and play the actions
• Easy to re-record user actions
• User friendly UI and wizards to configure and execute tests

Disadvantages

• Hard-coded values which must needs to change if anything at all changes in our product. 
• Cost of the framework itself is too high and you need support from manufacturer which will add another cost
• The costs associated with maintaining such framework's scripts are astronomical, and some time unacceptable. Because they use their own defined scripting language.
• These scripts are not reliable, even if the application has not changed, it often fails on replay (pop-up windows, messages, and other things can happen that did not happen when the test was recorded).
• Need to follow the associated programming language and tools and must follow the pre-defined rules. Some times testers needs training to understand their language and data types. Example QTP, they claims it uses VBScript but in reality it is the modified version of vbscript.
• Can not use single framework to automate all kind of apps or validate.
• We must need to use hybrid approach, which is a combination of different automation framework and tools to fulfill the need of a complete project
• If the application changes, the test must be re-recorded. But cannot be record if application is not corrected or build is released.
• This can be only applicable for standard UI application, if UI components are complex or custom made it is impossible to use without huge modification in tool/recorded script
• Difficult to share the recorded UI objects with multiple projects
• Cost of maintenance is comparatively higher in long run and need lot of licences
• Difficult to debug the recorded scripts
• Record/Play can not record complex UI combination or custom controls.

If you look at the disadvantages then it is more than the advantages. So it is not always advisable to use Keyword Driver or record Play framework. 

Then What to use which can fulfill the complete requirement? I love test library architecture framework or script based framework. This is the most appropriate way of modern application automation.

Please drop your feedback or questions.

Use Web Element property to identify element

In my earlier post I demonstrated how to get the web element property using the inbuilt tools of browsers. Here I will explain how to use those properties to make a unique combination to identify web element. 

Web page or html page is consists of many types of web elements, like div, input, span, table, anchor, button, ui, list etc. and can be custom tags as well but this is very advance I will cover this in some other post. The web automation can be start using these tags and properties. see below screenshot of Google.com where search box is consists of multiple tags. 

you can see the tags and attributes, the attributes defines the behavior, look and feel of the element like style, input type, events, event scripts, link image  or external scripts embedded etc. 

Here is another example embedded scripts and embedded url with the web element

How to use these properties to identify your web element

id: If any element has "id" property then it is most appropriate to use as element identifier. "id" is mostly unique but in some situation it can be duplicate also. In that case we can combine with other property to identify uniquely (We'll see later how to combine)

class: If any element has this property then it can also be used, but remember same class name can be used by multiple tags or elements. So choose carefully.

tag-name: Tag name is always used with the combination of other property, there are very very limited chances that any web page would have only one type of tag.

name: This attribute is most unique after "id" property and limited chances are there which can duplicate with other elements attributes. 

style: Mostly found common in many elements, so can not be used alone. It can be used with other attributes as combination to identify.

Now our challenge is to use them with example. For trying out practical example, please refer this post Automate Web Based Application using shdocvw.dll & mshtml.dll and C# 

In the first screenshot we can use the id="gbqfq" which is unique to identify textbox, but there are multiple input tags (move your mouse to other tag name in same section) pointing to the same textbox. So which one is the correct one? If a tag has visibility: hidden; or style: hidden or type="hidden" Do not use these elements, as it will be invisible until some events are fired. or may be you can use after required event is fired. See the example below

HTMLDocument doc = new HTMLDocument();
InternetExplorer ie = new InternetExplorer();
ie.Visible = True;
ie.Navigate("http://www.google.com");

// Store HTML Document object 
doc = ie.Document

HTMLInputElement searchTxtBox = doc.getElementsByID("gbqfq");

What happen if you have no "id" in web element?

We can try finding the similar elements and then identify the one which you need.. let me give an example. 

HTMLInputElementCollection textBoxes = doc.getElementsByTagName("input")
foreach(HTMLInputElement textBox in textBosex)
{
        if (textbox.className == "gbqfif")  // or if (textbox.getAttribute("class") == "gbqfif")
       {
               textBox.Value = "Automation Testing";
               break;
        }

// or you can even try this or combine other attributes as well.
        if (textbox.className == "gbqfif" 
                && (textbox.getAttribute("style").contains("visibility: hidden;") == false)
                && (textbox.getAttribute("type") != "hidden"))
        {
               textBox.Value = "Automation Testing";
               break;
         }
}

Searching element from web page is an exercise of combining the elements to identify element uniquely. However you may find it bit laborious or complex, this will build your confidence to use any automation framework without prior training. 

How Automation Framework like, QTP, AutoIT, VSTS, Rational Robot?

Most of the modern automation framework use the same technique, but instead of finding single attribute from element they capture the entire property and show in GUI format. See the below element snippet for google text box

<input id="gbqfq" class="gbqfif" name="q" type="text" autocomplete="off" value="" dir="ltr" spellcheck="false" style="border: none; padding: 0px; margin: 0px; height: auto; width: 100%; background-image: url(data:image/gif;base64,R0lGODlhAQABAID/AMDAwAAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw%3D%3D); background-color: transparent; position: absolute; z-index: 6; left: 0px; outline: none; background-position: initial initial; background-repeat: initial initial;">

And it will be presented on automation framework something like this. Here you can see it is just showing the GUI of Gmail logo, but it has captured the entire attribute of logo element and it will make it invisible. 

This is amazing, isn't it? so why I am posting the internal part when manual tester have very good tools? well there are many reasons. By explaining at macro level you can understand the basics of automation framework, many time record play doesn't work specially when your control is very dynamic and you need to re-record your test always, you can avoid the cost of automation framework if you have very very limited automation requirement.

Keep practicing this way to identify your elements and use my other post for your practical 

Automate Web Based Application using shdocvw.dll & mshtml.dll and C# 

Post your questions or comments or suggestions. Happy automation

Automate Web Based Application using shdocvw.dll & mshtml.dll and C#

I covered few things in automation testing and framework in my earlier posts. Its time to do some practical, I will demonstrate one simple step to automate web application using Internet Explore and C#. If you remember in my earlier posts I said I won't use any ready made framework and reveal some of hidden treasure inside the window. This article is revealing one of the hidden treasure. 

shdocvw.dll and microsoft.mshtml.dll are one of the hidden treasure. These two dlls are the very important part of Internet Explorer installed on your computer of any version. So we'll see how to use them to automate web applications. But let me give some technical background about these two dlls.

Shdocvw.dll:  It supplies the functionality associated with navigation, in-place linking, favorites and history management, and PICS support. In short this dll is used inside the Internet Explore to provide most of the functionality in IE browser, like Navigation, Page History, favorite etc. So whatever you see on the Internet Explorer like, Address bar, buttons to go back, go forward, book mark etc they all are using this dll. More detail you can see here 

Microsoft.mshtml.dll:  It is an Active Document that is capable of rendering HTML and laying out contained ActiveX Controls. In this document, it refers to an instance of the MSHTML COM object that implements IOleObject, IOleDocument, IOleDocumentView, and many other interfaces. In Short it is responsible for the document area in Internet Explorer, which renders Html content and display your requested URL. You can see more details here

Hope we got enough information about these two dlls, now I will show you how to automate a web page using them. But you might be thinking why not to use selenium or any other tools. Don't worry I will cover them later so that you can understand the difference among different automation tools and framework.

Step 1: Create a simple Console Application in visual studio. Here is the details how to create Console application

Step 2: Add reference of these two dlls in your console application. Here you can find the files and how to add as reference

• Shdocvw.dll from C:\Windows\System32

• Microsoft.mshtml.dll from C:\Program Files (x86)\Microsoft.NET\Primary Interop Assemblies or C:\Program Files\Microsoft.NET\Primary Interop Assemblies.

• How to add reference in C# Console application. Click here

Step 3: Add these two directives. 

           using SHDocVw;

           using mshtml;

Step 4: Copy and paste the code below.. 

Hey hey wait.. are you trying to copy the code? That is just an image. Copy the code from here

Step 5: Press F5 and wait, if steps executed very fast then add the Thread.Sleep(3000) in between of steps.

Note: You might have question, how did I find the Ids of the web elements. You can see my earlier post Searching Web Element property using inbuilt browser tools

This is just a sample which shows how to work on the core level of automation, the QTP or other keyword driven framework does the same thing but it hides the code complexity and you only get one GUI control instead of code. 

I know it is still difficult for some manual testers, but don't afraid of the code. Just try to play with it and use other website to automate you will love it. And Visual Studio is intelligent enough to give you more details of supported method and property. I am trying the same article for Java Users, but not very sure when I can post so forgive if it take long time.

Have fun and send comment if code is not working.

Searching Web element property using inbuilt browser tools

In this post I am going to demonstrate how to detect the web element for automation scripts. Many of the script based framework or just script needs to know about the property of web element to identify. From IE7+, Chrome, Firefox, Safari has inbuilt element detection features.

IE: In internet explorer 7 and above

• Launch website and press F12 Button
• Developer Tool will be pop up, now click on mouse pointer on developer tool and move to the element you want to detect.
• You can see the elements and its property will be displayed in developer tool
• In new version of IE (I think IE10 and above) are having same feature to inspect as Chrome.

Chrome: 

In chrome it is pretty straight forward, just right click on any of the element and choose. Inspect Element

Mozilla Fire Fox: 

Click Here

Safari:
Click Here

There are many tools and add-ins are there to detect the elements from any browser, and these are inbuilt tools which you can use as per your need. Now the next question is how to use them to identify your element for automation. 

Please check the next article to use these properties for your web automation

Use Web Element property to identify element

Test Automation Framework

If you search on Google about automation framework it will show you lots of definitions and I found many of them useless in respect of manual testers. Let's discuss about the automation framework with definitions and some examples.

Here are few examples from Google. 

http://www.ibm.com/developerworks/rational/library/591.html
 " A test automation framework is a set of assumptions, concepts, and practices that provide support for automated software testing. "

http://en.wikipedia.org/wiki/Test_automation
" A test automation framework is an integrated system that sets the rules of automation of a specific product. This system integrates the function libraries, test data sources, object details and various reusable modules. These components act as small building blocks which need to be assembled to represent a business process. The framework provides the basis of test automation and simplifies the automation effort. "

http://www.automatedtestinginstitute.com/home/index.php?option=com_content&id=1096:faq-framework&Itemid=25
" An automated test framework may be loosely defined as a set of abstract concepts, processes, procedures and environment in which automated tests will be designed, created and implemented. In addition, it includes the physical structures used for test creation and implementation, as well as the logical interactions among those components"

OK, so we have some definitions did you understand any of them? The definitions looks bit technical and all of them are giving almost same meaning, or if you search more you will find similar which may leads to loosing your interest end of the day. I will give some practical examples which we usually see everyday and will try to relate with automation framework definitions Or you can understand them by yourself. 

Garage "Where motor vehicles gets fixed"

In a garage motor vehicle gets fixed but how? Is there any process involved? Yes. may be you don't see the process. But I will tell you how, if you noticed a garage you will find the set of tools, equipment, parts all are kept with proper arrangement and when any motor vehicle comes it makes it easy to find. Mechanics check the problem and pick the right tools to fix it. But he follows some procedure, like how to open Tyre, How to open front hood of car, how to fix the problem and when done he re-arrange the equipment and closes the open part of the vehicle. So his garage is his framework which is supporting him to fix the car.. got it? it is not a rocket science a very simple and straight forward concept.

But how it is related to automation framework? Yes it is related, the definition says "automation framework is a set of assumptions, concepts, and practices" very technical but with the below explanation you can relate yourself 

An automation framework which provide tools, like start/close Browser, Write Logs, Navigate URL, Capture screenshot, put some delay between steps, check internet connections, check internet speed, get UI elements etc. An Automation Framework doesn't know which one you will need so, it assume you may need these in order to you automate your requirement but depending on the type of automation framework it can have different set of tools.

In your framework you got the right tools or the building blocks now your next job is to automate test cases. So you need some rules, like how do I start writing test cases, which language should I use, what are the procedure to write test flow, how to initialize things (like browser, or some data) how you will organize your tests, what is your application and its flow. So accordingly you will need create your tests scripts. 

You got the tools, and rules to automate your tests. Finally you need something to execute those tests and get the results. (In QTP or other coded UI tests you get all of these together and you need not to worry about tools, rules and how to execute. But here we are not using any framework, so how we'll design our own. This is pretty difficult at the moment but we'll see in other posts how it is easy.) There is a component in automation testing called Test Harness. I will cover in later part. Either I can use any available test harness or I can write my own (what! writing test harness, is it a joke?) Yes you heard it right. This is simple but depending on your need it may be complicated. So why don't we use which is available. I will also demonstrate to write your own harness. 

Painter Board

Imagine a painter board, it has lots of things together like, a big board, a clip to hold the paper, a stand to keep the brushes, a color holder, a height adjustable knob etc. This whole is a framework for a painter because he is getting everything together to make multiple paintings. The same can be related with automation framework or any kind of framework in software industry.

Summary: In simple sentence a test automation framework is combination of tools, rules, and test harness. which helps you to automate your test cases. You can check my next post for practical examples of framework. Don't complicate this with Google's definition keep it simple.

Test Harness

This is something new, some people know some people don't know. Lets say I have created many .exe files or .dlls which has some test cases. If i want to execute them I need someone to execute them one by one and collect the results. But it is annoying or better to use manual testing, isn't it? What I mean to say if I have something which executes these exe, dll or jar files and collect the results then it will be my test harness. 

For example in QTP when you record a step on browser and play, then from the play your test harness comes into picture. It executes them and collect the results. But it is collectively a part of your test automation framework only. Similarly in VSTS or other framework.

Hope I explained it more.. but if you have any questions you can comment I will try to clarify. Or if you feel this article could have been better I will love to hear your suggestions. 

Is Automation testing really that difficult? Part-2

So here we go with second part for automation. Hope you liked the first part

In previous part I demonstrated a very small part of automation. I also said I will use c# for demonstration you can use java and I will try to cover other languages as well. I will post one more topic to use c# as your automation language and will tag the link here

As we see in last post automation testing is not sodiffuclt but just a matter of bit understanding and integration of windows and its APIs (tough, but will not be). Anyone can be automation engineer without using any framework. So I am going to continue the same example here and will add few more lines of code to make your own framework. 

I will add few logging and some validation to my previous example

HTMLDocument doc = new HTMLDocument();

InternetExplorer ie = new InternetExplorer();
ie.Navigate("http://www.google.com");Log.WriteLine("google.com launched");

// Store HTML Document object 
doc = ie.Document


// Find Text Box and input text
HTMLInputElement searchTxtBox = doc.GetElementsByID("gbqfq");searchTxtBox.Value = "Automation Testing";Log.WriteLine("Automation Testing is entered into searchbox");

// Find Search Button and click
HTMLInputElement searchButton= doc.GetElementByID("gbqfba");
searchButton.Click();
Log.WriteLine("Clicked on search button");

// validation, I am expecting here 15 search result at least

HTMLInputElementCollection numberOfLinks = doc.GetElementByID("ires")).GetHTMLInputElementsByTag("li");

if (numberOfLinks.Length >= 15)

    Log.WriteLine("PASS");

else

    Log.WriteLine("FAIL");

This is done! Isn't it? Yes I know this is not workable becaues I haven't provided any other steps, like how to start, what is Log and from where I referenced InternetExplore etc etc. But don't worry the automation is just it. Nothing special in concept wise, either I can make this a exe file or a dll or a jar file and I am ready with one test case automated. 

I know still you would have many questions like we get QTP a record button and Visual Studio a record button, in fact many framework provide the same. So how can we find out the buttons, text boxes, links etc. Believe me there are many hidden treasures in Windows and Browsers which usually ignored by many people. I will reveal those treasures so that you no more needed to depended on framework, but you can make your own. 

I would request a few things before we continue our test automaiton journey. I would preffer to install Visual Studio or C# from this link. I will also post in java but on windows I preffer in c# more appropriate. 

Here are some samples of writting Console Application in c#, you can also learn VB.NET or any other language which supports Visual Studio. But my recomendation is C# it is hotcake for windows automation. http://www.dotnetperls.com/console

I will wait for the feedback. 

Automation testing is really that difficult? Part-1

I am writting my first ever blog on automation testing. Hope I will justify with it :)

I interacted with many manual testers who says please make me learn automation testing or give some material about automation testing like samples, tutorials, sample projects etc. 

So the question is "Is automation testing really that difficult, what some manual testers assume?" I don't think so. One fear is that automation testing required lot of programming skills (not to shy.. if you are one of them). But I would say it is not correct unless you did not bunk your acadmic programming classes in college. In other words you will need some basic programming skills and once you become automation tester you can make it advance.

One misconception about the automation framework is, some testers think only automation framework is capable of writting autoamtion scripts; specially the UI automation scripts. Lets assume you have google.com and want to search something, now if you don't have any framework then is it possible to write UI automation test cases? It is not? Seems difficult? or too many questions? Don't have any framework how to start Internet Explorer etc..

HTMLDocument doc = new HTMLDocument();
InternetExplorer ie = new InternetExplorer();
ie.Visible = True;
ie.Navigate("http://www.google.com");

// Store HTML Document object 
doc = ie.Document

// Find Text Box and input text
HTMLInputElement searchTxtBox = doc.GetElementsByID("gbqfq");
searchTxtBox.Value = "Automation Testing";

// Find Search Button and click
HTMLInputElement searchButton= doc.GetElementByID("gbqfba");
searchButton.Click();

(I will use C# or Java for the demonstration. For special need you can ask me, I will try to provide in other languages of your choice.)

OK now take deep breath because this is very simple, isn't it? May be you are thinking what are those keywords and what is IDs and how this is launching Internet Explore. Well this is not so simple in first impression. But this is very simple once you know about it. But if you are new then you will know it bit slow but will know for sure :)

You can assume the above scenario with login page where tester writes 10-15 cases for validation and in each of the release he/she may need to execute it. But this becomes very difficult when build is getting rolled out every day. So any manual tester can just write a loop and pass different crendentials to test various scenarios. 

Hey wait... But this is not automated testing but just a script. Umm Yes, it is true but what does automation framework do? this is just a glimps of that. But test framework hides the extra complexity and you may get the same cript in GUI format, something like below one

SearchTextBox:value("Automation Testing")
SearchGoogleButton:Action.click()

But still there is a question, what?

Test Framework gives Pass and Fail result as well but this script is not giving. ummm yes gotcha, writting script doesn't means it fulfilledthe need of a framework or solved the automation problems. Still automation testing looks complicated? will cover in next updates...Part-2

Leave your feedback, this is my first ever blog and written by interacting with some manual testers.

and keep tune in to my blog for further updates.