Test automation frameworks keeps all test automation artefacts in single format and easy to maintain, in addition brings faster and better ROI to organisations.
This article was written assuming the fact that you are aware of the necessity of automation testing and how to decide on which automation tool to be considered for test automation.
Lets see what are the list of most important factors to determine if a test automation framework is best or not. You should also understand that not every framework can acheive all of these factors:
- Less maintenance
- Reusable
- Easy to debug if any issues are encountered
- Shorter execution time
- Easy to understand the scripts
- Script parametrization
- Minimal manual intervention during certifications or testing cycles.
Understanding these factors, you need to decide which automation framework would be suitable for a given test automation project or implementation, these are mostly driven by the application under test, future plan of the organisations as what they want to do with these test assets and their business plan.
Lets deep dive in to each one after the other:
Less maintenance:
Any automation script or asset created should be easily maintainable, which means even if there is any functional change or UI object change or UI Layout changes or any synchronization changes, automation resource should be able to make necessary changes in a quick turn around time and keep the automation scripts and assets intact and workable.
Reusable: plays a major role in achieving less maintenance
Developing reusable assets will help in faster development of automation scripts and assets and there by increase the coverage of testing through automation, which would in turn reduce the testing costs and increase the accuracy of testing.
It also helps in maintaining the scripts intact even if there is any change in functionality, updating the scripts assets at one place would reflect the same in all the areas where it was reused.
Easy to debug during executions:
Should not involve major effort in performing the executions of automated test scripts, automation resource executing the scripts should be able to easily identify as where exactly the automation script has failed and if the cause of script execution failure is due to a functional issue or an issue in automation script. Automation resource is not supposed to spend major time in the identification it self.
Shorter execution time:
Need to find best possible ways to reduce the automation execution time and there by reduce the testing cycle time.
Easy to understand:
Automation resources should be able to understand the automation scripts and assets in a small turn around time by looking at them irrespective of having functional knowledge, this is majorly needed when a new person enters in to team or if there is a swap of resources across different functional areas of testing or if a person a leaves and a resource needs to take up new functional area apart from the existing ones.
Script parametrization:
Many a times the customers or clients would like to run the automated scenarios with different sets of test data, the framework should have the flexibility to run multiple iterations with same test data and with different sets of test data.
Minimal manual intervention:
The process of identifying the test case scenarios and the external factors involved to develop or execute the automation scripts should either have zero or least possible manual intervention.
The automation scripts should be highly stable and have either zero or least possible manual intervention.
Also in an organisation or team where people with different skills and capabilities working in single project may have lot of differences of opinions which needs to be addressed.
In order to address above said factors automation frameworks have come up and evolved along the time. Evolved primarily due to various business development models, type of applications, features, resource skill sets, capabilities, requirements for automation and none the least the management needs.
Note: understand that the factors are to be well balanced based on the requirements to design a better framework.
I have given my views and thoughts, this might need lot of evolution, appreciate if you can add more to this through comments and discussions.
Automation framework should also give extension for good test data management.
Please do share your view points on the same, it would help others to understand and get to know.
This is a very good article guide us what is an automation framework and what characteristic it should have.
But i think automation testing sometimes seems to be clumsy due to UI and function changes. The whole testing script or even the testing scenario will fail if one object can not be found. Is there good ways to make auto scripts more stable to suit UI change? Such as package the common scripts and how to make the object recognition more smart?
Hi wennie,
thank you so much for your comments and regarding your query, when there is a change in UI object the script fail.
These issues can be solved based on the framework that is implemented at your end. In case the framework is robust enough to find out all the occurences of the object in the entire script assets you can find a mechanim to change all of them at once. This way your scripts are in sync to the the application changes.
These changes should be able to do in single shot, something like a database query or so.
please do revert to me incase you have more queries.
Hello Srinivas,
Thanks for the nice article.
I would like to know like how to start a framework from scratch. What would be our first steps while designing a framework?
Should we work on creating a generic framework or proceed according to requirement?
Thanks,
Sumer
Thanks for giving a read.
I would say it depends, check out these use cases.
Case 1: In case you are working for multiple projects and they are on same application. This generally happens when the projects are domain specific i.e. like insurance, oracle app, sap, health, etc…
Also when you know that there are chances of getting more projects in the same domain:
You can think of developing a generic one which will address the points specified in the article.
Case 2:
If projects are very small and non domain specific and it is highly unlikely to get the same application from another client or project. You can have a basic framework which can let you go through the project completion in less time.
And finally if one could develop one single generic framework for any project that might come, it is well and good to create one generic. Saves some time which indirectly saves time to client as well.
But when trying to create a generic one,few of aspects might need to be eleminated to make it generic across projects.
Some times the framework might need to have a major modification based on the type of application.
For example applications are web based, oracle forms based, .net based,window based, telnet based etc..
Now even here automation tool plays important role as we never know how each automarion tool cater to the automation of each type of application.
Hope i answered your query if not please do not hesitate to ask about it. Also feel free to ask more queries.
Thanks for the answer.
But I need information like what actual steps should we follow while designing, for example feasibility analysis, requirement gathering, test planning etc.
If you can make a flow chart for all the in between activities and explain them, that would be quite useful in my current situation.
Thanks
Sumer
Regarding feasibility analysis, it is to evaluate if the application can be automated and analyse the percentage coverage through automation and realize the benefit of automation testing for client / project.
For more details you can refer to feasibility analysis in software test automation.
Regarding requirement gathering, in brief it is to gather requirements from client and understand the application functionality clearly and document them. These are supposed to stay constant across the project duration but in the real world they keep changing.
If any requirement change is accepted by a developer or if a new requirement is added, testing team are to be notified about it, as it will effect the creation of test cases, executions and testing strategy.
Test planning is all about the plan to create test cases, deciding the scope to execute them based on the release or build plan. For more details check out software test strategy.
Hi,
On the following Environment of ours, we are seeing that oracle forms within the application does not load when we open application after launching OATS
JRE Version: JRE 7 Update: 65
OATS Version: 12.4.0.2_129 (Build)
Application Built on : Oracle EBS/Forms & Applets
Operating System: Win 7 32-Bit Professional
Browser: Internet Explorer 8
Open Test Application without launching OATS Open Script – Works Fine :
1. Login to application by opening URL using IE8 browser.
2. Once login is successful we need to click on a button, which will open the Oracle forms.
Launch OATS Open Script and open Test Application – Issue Observed :
1. Launch OATS Open Script
2. Login to application by opening URL using IE8 browser. (Login was successful)
3. Click on the button and check if Oracle forms are loading (Here is the issue – Oracle forms are not loading).
Can someone please advise if there is any work around for this? Or Is there any configuration changes that we have to do so that the application runs when OATS open script is launched.
Any help on this is much appreciated.
Thanks in Advance.
Medha
Did you “Enable the EBS/Forms automation”?
This would be present in “Tools” Main menu.
What are the list of JRE versions installed in your system?
Good article.I also faced same like medha when I was working .I just enabled the EBS/FORMS in tools menu.
Srinivas, hello! The article is interesting to read. My thoughts: In a fast-paced working environment, sometimes automation is slow and lengthy process; and pressure or frustration could be experienced. With the challenge of the effort, one can value the positive results and eventually handover the automation scripts to involved person. In between times of P-D-C-A (plan-do-check-act), what are the usual problems in doing automation? Really knowing the problems could help us to suggest actions that would lead to find solution and develop the practice – the common knowledge. 🙂
Hi Kailem,
Thanks for sharing your feedback, would come up with an article to list out the most common issues that are faced during automation and how to over come them.
Keep sharing your views and also share with your friends about the same.
please find below the details of the object identification issue, we are seeing with OATS.
Environment:
JRE Version : JRE 7 Update: 65
OATS Version : 12.4.0.2_129 (Build)
Application Built on : Oracle EBS/Forms & Applets(11g Forms)
Operating System: Win 7 32-Bit Professional
Browser: Internet Explorer 9
Object Identification Issue:
1.Launch OATS Open Script
2.Login to Test application by opening URL using IE9 browser.
(Login was successful)
3.Click on the button and check if forms are loading
(Oracle forms are loading).
4.Try spying or recording actions on any objects in the forms
(Issue – Objects are not getting identified by OATS)
Appreciate any help on this. Thanks in advance.
Applet Recording Issue:
1. Create new blank Oracle EBS/Forms script
2. Start recording
3. Goto to Script Menu > Configure Recorders…
(Config Recorder Dialog opens)
4. From the Config Recorder Dialog, check the status of Forms Event Recorder
(Status of Forms Event Recorder is ‘Running’. Pause button is enabled)
5. From the Config Recorder Dialog, check the status of Applet Recorder
(Here comes the Issue —> Status of Applet Recorder is ‘Starting’. Pause button is disabled)
How to make Applet Recorder state from ‘Starting’ to ‘Running’
Thanks for Help in advance.
If you have no issues sharing a screenshot of the application, please upload an image and share the link here, it might help people to check and give their inputs based on their experiences.
To my knowledge, when we discussed in offline, I feel they are resembling Applets inside the Oracle Forms and I doubt if Openscript supports the automation of the same.
Thanks
Srinivas P.
Hey Srinivas ,
The article which you published is good one , but information which you have shared is not in detail manner, hence needed to add some points such that a new person can understand this tool better.
Could you also share that can we integrate Selenium with OATS.
Regards,
Akhilesh Mehra
Thank you so much for your response Akhilesh.
And you are true, i had just briefed out things in a single article, i would be writing more articles with more details covering each topic. By the way is this comment related to this post or is it for https://testingtools.co/oats ?
And regarding integrate Selenium with OATS, could you please elaborate as what is that you wanna achieve by integrating these ?
Thanks
Srinivas P.
Hi Srinivas
I would like to know if OATS can be integrated with Build management tools like Maven, CI tools like Jenkins or Teamcity and with repository like SVN?
Regards,
Amrita
Hi Amrita,
Thank you so much for reaching out to me to understand about capabilities of Oracle Application Testing Suite ( a.k.a OATS ), there is no integration provided as of now by OATS dev team from Oracle, but there are platforms available in market which have gone a step ahead and have done these kind of integrations, like for example CloudTestr which have built many additional features apart from what OATS is providing.
They have this CI capability with Jenkins / Hudson, they also have web services which can be used to integrate with other tools as well.
With respect to build management tool, I don’t think there is anything available now in the market too, but this query looks very interesting, may I know for which kind of applications are you looking to automate? and are those being maintained by Code Versioning tools like SVN. The reason for asking is, there is nothing like we cannot do things, there will be different ways of achieving this, so if you can express what is your exact requirement I can give you more details. I can even look for building such a solution.
Appreciate if you can respond with more details 🙂 And I know the product manager of Cloudtestr, if your company is looking for any such platforms, I know that is have lot of capabilities that augment to OATS and can increase the success of Automation Project Implementation.
Thanks once again to reaching us testingtools.co
Hi Srinivas,
Thank you for your response! I am currently using Selenium (ROBOT framework)and we check our tests regularly in to SVN. We use Teamcity as our CI tool and Maven as build tool.
I am exploring OATS and its capabilities as my future project would be on OATS (automating Oracle EBS). I am trying to understand if OATS scripts would be auto triggered when there is a new build or how is the entire process handled otherwise.
This site has been quite helpful for me in getting an overview of OATS, I’d not been able to find these many topics on OATS anywhere else. It would be a great help if you could provide more details around this. I would be posting more questions I have in your other blogs too. Again thank you for putting in so much effort to help new OATS users.
Regards,
Amrita
Hi Amrita,
Thank you so much for giving the details requested 🙂 thank you so much for appreciating me on sharing knowledge on Oracle Application Testing Suite, Yes I did see that there was lack of knowledge sharing over internet especially on Oracle Application Testing Suite ( a.k.a OATS ).
Well I got your point, just to get further understanding may I know which Build or CI tools are used by Oracle EBS application development teams, that will give some insights and you are right with selenium ( Robot Framework ) tests can be regularly pushed in SVN and other CI tools to trigger whenever build is pushed to QA instances.
Straight forward there is no such feature available with OATS to either check in scripts to SVN or trigger scripts from CI tools on new builds ( i.e. DevOps Integrations or Continuous Testing or CI ) , but as mentioned earlier, there are few companies who have build platforms on top of Tools like OATS, like Robot Framework which is an extension to pure Selenium.
Let me know if we can discuss offline @ testingtools.co@gmail.com to understand if you want to use any other platforms for your upcoming requirement, which can help you provide all these which are you are looking for. there are even platforms which will work with OATS, Selenium simultaneously.
I would be more than happy for you to post more and more comments so that others can also learn 🙂