If you dream about a mature automation framework, then this should be your first step. There are millions of article floating around the web related to exception handling, exception class etc etc. Here, I want automation developers to understand the best practices, pros and cons of exception handling in automation framework perspective only.
We have to understand that developing a java app is different from developing a java automation framework. Think about it!
What is expected out of a automation framework:
It very straight forward, our automation framework should fail wherever there is a problem like, web element is not found, or assert fail, time out exception and other java exceptions.
We even need the stack trace(exception log) to locate the area of failure and analyse the root cause. We don’t want to suppress it.
And lastly automation framework should execute the next test irrespective of what happened for the previous test.
Below you can see that all the above points can be achieved without exception handling.
Let’s understand, what happens if we don’t do exceptional handling:
- Test will fail abruptly,
- it will print the stack trace in the console and
- further steps are not executed. Note: However in case of batch run(test suite execution in case of TestNg and runner file execution of jUnit) the next test will be executed irrespective of whatsoever happened to the previous test.
- We cant execute any special steps when exception occurs. For example: writing some log, capturing screenshot, adjusting the driver default wait time etc.
To summarize (before we jump into exceptional handling), in normal scenario we are not able to execute any customized code snippet on occurrence of exception.
We will see below how different scenario can be handled when exception occurs.
Continue reading “Exception handling in automation framework:”
Another blog about my experience, analysis and concluding browser ‘TAB’ navigation in automation.Hope this will help you to design an stable tab navigation framework:
Basics about tabs before jumping into code:
Browser window and tab are different! Each window have separate unique window handle where tabs are array of a same handle. Hence by switching between the handle we will achieve window navigation (comparatively simple!), not tab navigation. Just to elaborate with example: internally when we open new browser window, both the window will have separate handle like CDwindow-4f2d1560-16bf-40a3-81be-dd91e13d6390 (for browser window-1) and CDwindow-4fc2d1560-16bf-40a3-81be-dd91e13e6390 (for browser window-2), whereas two tabs are considered as array of same handle like CDwindow-4f2d1560-16bf-40a3-81be-dd91e13d6390 (for tab-1) and CDwindow-4f2d1560-16bf-40a3-81be-dd91e13d6390 (for tab-2).
Common tab operation:
- Right click and open a link in new tabs.
- Pointing the driver to different tabs.
- Closing the tabs.
Continue reading “Tab navigation using Selenium – a stable way:”
Brief about this discussion:
Initially, let us understand WHAT we(automation developers) update? and what is the tricky part there.
To have a basic understanding of automation framework-click here.
Mostly, AUT(Application under test) evolves on regular basis to gain edge on business competitors and to achieve business goals. Hence the framework associated with the application requires up-gradation.
Every framework is having their own coding style, naming convention, utility module, business logic, object repository and basic framework configuration. As automation developer, we should understand that anything that exists in our framework may changes. However mostly, the WebElements, business logic and Utility methods undergoes changes.
The tricky part is, the framework is used by many stakeholders (developers, QE, Automation developers, Tech lead, BA, SME, etc) on daily basis. Re-think about the effect of your change on the framework. It is important to keep your changes seamless, with minimum impact on the working part of the framework.
Before touching the framework, one should have clear idea about the
framework structure and functional workflow of AUT.
- Updating common methods: Golden rule is to keep your code short, simple and with high coverage (so that, it(your updates) can cover most of the scenario). The below will mostly help you to modify business logic and utility files.
For example: Let application be XYZ.com with customer module. Now automate job history of a employee: understand the scenario, an end user may have no job history OR he may have 10 job history. Now we assume that we have an existing method to automate the job history (as mentioned below). But the constrains is that, it can enter only one job history.
Please find the Java code below: Continue reading “Updating an existing automation framework: [Best-Practices]”
Basic understanding of a framework:
Before developing a framework we should step back and understand whether we need a framework or not? A framework makes our life easy. However, on the other hand to create a mature framework, time and cost is involved.
Here we are discussing about automation testing and it’s related frameworks. So to understand WHAT a basic automation framework does, we should understand the basic of automation testing.
Basic understanding of automation:
To be honest, we can achieve many things using automation. I will try to highlight almost everything to start automation.
In simple words, we automate the manual test cases using some tool. These automated test cases are written in such a way that it can be executed anytime without manual intervention. As time goes on, the automation scripts become complex and unorganized because of update, delete and insert in AUT(Application Under Test). Hence the need of framework comes into play.
Prerequisite: Features of automation framework.
Creation of first automation framework:
Here we will develop a automation framework with all the basic feature discussed above.
Prerequisite: basic knowledge of Selenium WebDriver, Java, TestNG
Let’s start developing:
Continue reading “Automation Framework for Beginner”
A mature framework is a fruitful result of years of technical hard work. Below are certain features that can be used as a guidelines by automation framework developers.
Note: There are countless feature that can be integrated with a framework,
below are the most important one.
- Object management: Every application can be treated as a collection of objects(text box, button, menu, title, etc.) and behavior. We should be able to insert, delete or update objects easily without affecting the test-scripts or business logic.
- Test script management: Test scripts should be readable and understood by manual testers, BAs, SMEs and other important stakeholders. Test script files should be replica of manual test cases. Change in business logic should not heavily effect test script.
- Utility management: Utility is the heart of the framework. It contains the core technical configurations. These are reusable codes which are used across the framework. For example: how to launch chrome, click, verify, integration of test data (text file, excel or database etc.), environment variable setup, pre condition, post conditions, etc…
- Test data management: Mostly test data are volatile and changes for every execution. Hence never hard-code test data in the test-scripts or in the business logic. They should be fed from an external file like text file, excel file, properties file etc…
- Report management: This is the final product, everybody is interested on. Creating a colorful and graphical summary report of the test run and saving it automatically is very important.
- Version and Source control: For allowing a team to work efficiently on the same framework, version and source control system is must. Tools like TFS, SVN, GIT are to be integrated with the framework.
- Code Merge and Review tool: These are some tools that can be integrated easily for code merging and Review. For example: git merge tool (for code merging) and Gerrit (for code review).
- CI and CD: Continuous Integration and Continuous Deployment is another feature that may be required to integrate the changes every now and then, building the code and deploying the code base to different environments. For example: Jenkins.
- Cross-Browser testing: We can configure a config file to set the environment variable for the framework to run. Suppose we want to test the application with IE and Firefox, then we can configure IE and Mozilla and alter the framework variable to achieve the same. Tool that can be integrated for cross-browser testing is Sauce lab (cloud for cross browser testing).
- Code-Coverage: To find the code coverage we can integrate tools like: sonarqube.
Every project has its own landscape and constraints. These constrains tweaks the framework design. Some Advance feature may be treated as basic feature for certain project. Take a step back before integrating a tool or feature, analyze the pros and cons thoroughly w.r.t automation and then go ahead.
Please comment for any suggestions or thoughts so that I can improve my knowledge base.
It’s simple, but complex:
It’s really as simple as – feature files describes feature of an application without duplication, and each related user stories are tagged with the corresponding feature files.
Refer to "Terminology" section for better understanding, end of this page.
However, if we deep dive, there are certain challenges:
- Challenge #1: Dividing application into feature: HOW do you divide a web-application into features? Well its not easy, for example, user management (search, add, update, deactivate an user) is a feature in facebook. Now a general question: WHAT are features? User management is a feature OR searching an user is a feature OR searching a user by city/town is a feature…! Complex right.
- Challenge #2: Identifying correct feature file for user stories: Before discussing this problem, we have to understand that teams are continuously changing (change in vendor, change in team member, ramping up and ramp down etc). Mainly due to this, it becomes difficult for the entire team to know all the features of an application. Even though they know the features of the application, they are not aware of feature file related to the features and how the application is divided into feature (challenge #1). Most of the BDD(Behavior Driven Development) projects, do not document this properly.
- Challenge #3: User Stories, impacting multiple user stories: Let’s start with an example, user story says implement auto-complete in search user page and update user page in application-X. Now, this stories impacts both search and update user feature files. Hence, identifying all the impacted feature files due to an user stories is an complex job.
Common mistakes, due to these challenges:
Mistake#1: Associating each a user stories with a new feature file:
- User stories are planning tool. They exist until they are implemented and rolled out.
- Feature are communication tool. They depict how system works.
Example:Suppose user stories is: Search a customer.Scenario: User should be able to search a customer based on Name, Address and A/C number.
Now with this type design, I have to create three feature files. Search_by_name.feature, Search_by_address.feature & Search_by_ac_number.feature
This approach can get your team going but soon it will run into problems:
- Problem-1:Feature file will increase in number, which will obviously make the framework complex. The feature folder will be like history log of development. 😊
- Problem-2: Linking. Now suppose, customer want to enhance the search feature, then we need to update all the feature file related to it.
Point to be Noted: the feature files were linked with previous user stories which have been completed. Now, there will be no relation between new user stories and feature file. Either you have to define new feature file and deleting the older one OR search for the feature file manually which covers current enhancement (user stories).
Mistake#2: Associating each user stories with an Epic:
This should be the ideal way as it is describing the product. But again “challenge# 1” will block your way.
- Problems-1: this will create issue in delivery, as one epic can’t be developed in a sprint or two.
- Problem-2: one feature file will be too long, and can create complexity. This will decrease the modularity.
Solution and best practices:
- Creating a feature map: it is very important to overcome “challenge #1”. Note: that feature map should include the feature file names for each sub-feature in the feature map. Below is an good example of feature map [referred from google]:
- Divide the product into logical sections each with a feature file. These files should be easily understood by business as one of the feature of the product.
- Point to be noted: as product undergoes changes/update /enhancements similarly feature file and step definitions should also under go updates as required. (add/update/delete).
- Even if there are One Epic, and three user stories it might have two feature.file. In other words, there should be no relation between Epic/user stories and feature files.
- In the above example, all the search scenario should go in to one feature file. Similarly, all scenarios related to customer maintenance go into the another feature file.
- If the feature files are changing from due to change in business requirements then there should be version. Version control is important.
- Benefits – These versions are also helpful in tracking the changes of requirement, from where we started and where we are now.
- Benefits – Efforts can be tracked and everything is well documented.
- Scenarios in the feature file should be independent and focused with one combination of data. Scenario should be cover unhappy (negative) path as well.
- Test Execution perspective: We should also keep in mind that we can execute each scenario separately by using scenario level tag. Tagging of scenarios and definition of runner file is to be taken care properly.
- Defining feature file: we should use the terminology and common domain language to define the feature file, so that it can be easy for business to relate.
The user story is absorbed into our features and becomes invisible, leaving the feature as a live document of what the software does now. Not as a record of how it was constructed.
Feature file should be independent of each other. Feature files should be categorised into scenario based on set of test data.
In this article, we will try to understand, WHY we need a framework for any automation. It’s not a new or unknown topic! I just wanted to draft in my own style, the way I understood.
We are NOT discussing about the selection of framework design pattern.
Continue reading “Basic about Automation Framework:”