Minutes: Test automation and test data; Synergy!
Controling test data to control test automation
We started the afternoon in Utrecht and were welcomed by DATPROF. First speaker was Menno van den Berg with a presentation about the challenges of test automation. Afterwards Marco den Haan gave a presentation about the advantages of data subsetting for test automation.
Challenges of test automation
Menno van den Berg started off with asking the group what they expect or are expecting when they think about test automation. The answers to this question were pretty diverse:
- Saving time with testing;
- Execute more tests;
- Improving test coverage;
- Improving software quality;
- Exclude human failure;
- Track and traceability.
What is the main driver for implementing test automation? In most cases it is improving the time-to-market. The business nowadays demands smaller and more frequent software releases. But many organizations overlook the adaptability and support of a test automation tool. The initial implementation is challenging, the adaption and support of the tool is what makes the difference in your Return On Investment (ROI) in your test automation.
Only 12% of all defects are found through regression, a larger number of test defects are found through progression testing. Although the regression defects found are really important, using test automation only for regression would be missing a number of opportunities.
Aspects that influence the ROI of test automation
There are always some issues that influence the ROI of projects such as:
- The maturity of testing of your organization.
The more mature your test organization is the better the results of your test automation. More tests can be automated and therefore your ROI will improve. The maturity is influenced by the repeatability of test cases, for example unit or regression tests.
- The position of test automation in an organization.
It is important that test automation is centralized within an organization. This will create opportunities to use the automation tools for more projects as well as being able to apply the tool to more testing types, from unit to end-to-end testing. That way more tests can be executed. As more tests are automated, a bigger or even a faster ROI is achieved.
- Control your test data
Test data can frustrate your automation efforts. Normally it takes a lot of time to prepare your test data, sometimes more than 2 or 3 days. When test data is properly managed, the preparation of test data can be significantly reduced.
Getting started with test automation can be challenging, but the results are rewarding. As one of the members mentioned it saved them up to one million euros.
Test data subsets and test automation
When Marco den Haan started testing, in one of his first assignments he came in touch with automation. Not exactly the best first impression he got from automation. During this test automation project he identified major issues with test data.
Nowadays the world has changed, we all talk about agile, scrum and DevOps programs. And test automation is mentioned in many of these software development processes. As he discovered in his first assignment not every automation project is an instant success, but Marco still thinks that the concept of automation is brilliant. If you want to execute test automation properly you need to realize there are three aspects to testing:
- The system being tested
- The test environment in which the (automated) test is executed
- The test data used
When you want to assess the system under test, you need to be in control of your test environment as well as your test data. If you can’t regulate the environment or test data you can’t be sure of your test results. Test data is an issue in many cases. For example, test data can be polluted due to tests executed earlier. So controlling the test data can help you to rely on your automated test results, thus you don’t need to spend valuable time to validate your results or to prepare your test data for the automated test. Because while doing so, the added value (test faster) of test automation is lost.
So controlling your test data is improving the chance of success of your test automation. That means that as a test automation specialist you want to control your test data. And that test data management can be a major improvement for your test automation. With DATPROF Subset you’re able to influence your test data, by filtering data out of a source database.
So why should you be using Subset for automating test. With DATPROF Subset you:
- Are able to control the test data used in your automated test. With the option to reserve and preserve tests suited to accommodate your test automation.
- Are able to use a smaller sets of data. Smaller data sets deliver faster results as queries take less time.
- Are able to fill a complex data model (over 1000+ tables) with useful, relevant, test data.