Minutes of meeting: Breakfast session Testdata Management
The power of test data management in automating software testing
March 19, 2015 | Maarten Urbach
Attendees of the breakfast session Test data management, masking and subsetting were welcomed by Mark Rumpf (Business Line Manager Test Tooling of Sogeti). The event was organized to help clients with automating the test process, this time highlighting test data as a crucial element.
Why Test data Management by Dirkjan Kaper
The session was kicked off by Dirkjan Kaper presenting “Why Test data Management”. According to Dirkjan there are several reasons why you could or should start with test data management. Important reasons are the protection of your customers’ data or competitively sensitive data (eg: revenue data). As test environments are not as protected as production environments masking will help in protecting the sensitive data.
“The best test data is suitable to your needs and your environments and isn’t terabyte of data large, storage is cheaper and for example saves on server costs.”
– Dirkjan Kaper
In order to provide a test environment, it can take up to 2 weeks to create and 6 weeks to request a test environment. Therefore, a test manager who eventually gets an environment, won’t let go of it, as it costs time and resources to create a new one. As a consequence many testers today still work with test data which is more than 3 years old. Although with ‘fresher’ test data you are able to test faster, better and more consistent.
So, then what is test data management? Dirkjan defines test data management as the process to deliver the right test data, at the right moment, to a test process.
In the creation of test data the test requirements are directive. Test data requirements are a derivative of these test requirements. For example for a performance test the need for test data requirement is different. In creating test data you’ll probably answer questions like:
- How old can the test data be?
- Do we need to mask test data?
- Can we generate test data?
- How much test data do we need?
- How is test data distributed?
After test data management is initially implemented, the lifecycle of test data is to become important. The need for a system to register and support the needs will soon be seen. Who needs test data, who used test data and when is the test data distributed.
Rabobank International, Co Meerveld
Co Meerveld started his presentation with the notion that during his career he has always been busy with test data management. He attended this session to share his knowledge, but even more to improve his current level of knowledge. Today Co wants to share the Rabobank International experiences.
The challenges of Rabobank International:
- Rabobank International has quite a ‘new’ IT environment. It has one core system, this core system has several dependencies with other systems and databases;
- Many developers work in the Netherlands but there are also colleagues in -for example- Poland;
- It is a complete online bank and in total there are more than one million clients worldwide.
Why did we start with data masking or anonymization?
- A one on one copy of production is always used, but at some point this can’t be done anymore
- The general opinion about data leakages is getting more and more negative. And Rabobank wanted to act before anything happened.
- And an IT-audit indicated that a copy of production wasn’t allowed anymore.
So Rabobank International started with a project data masking. The organization was especially anxious about a consistent test set after it is masked. We get the demand of compliancy, but how usable is a testset?
We started the discussion about the provisioning of test data. Are we going to create test data manually or with a tool? We decided to buy a tool, because generating or manually creating is at first rather complex and intensive. And secondly support will become an issue in the future. So one tool to cater for all databases of Rabobank International should be the best and most logical decision.
Based on costs and how the tool works, DATPROF is a perfect fit to Rabobank International – Co Meerveld
With DATPROF a Proof of Concept was started for Rabobank International at Germany. The Proof of Concept was concluded successfully within a week. And more importantly, it gave enough confidence for the next step.
- It is more complex than we thought at first, embedding test data takes time;
- Make more than one data scrambling template;
- Constantly think what kinds of data needs to be masked;
- Be clear towards your test organization about the masking of the test databases;
- The availability of environments, test data requirements, the availability of environments and the availability of environments, for implementing a tool you’ll need time;
- Knowledge of data models
ABN Amro, Hendrik Jan Bolte en Marta Borrat Frigola
Hendrik Jan Bolte and Marta Borrat Frigola are charged with a legal role towards Privacy within the ABN Amro Bank. They are both legal counselors and are able to advice their colleagues and today also the attendees!
The current privacy law in the Netherlands does not apply to fully anonymised data. Data is anonymised when the individual to whom it relates is no longer identifiable.. The (test) data that has been fully anonimysed is no longer subject to this law. The important question is: when is an individual no longer identifiable? The law does not explicate what anonymizing is, we can only derive from it that privacy law no longer applies where the data subject is no longer identifiable..
“The Dutch Data Protection Authority (the College Bescherming Persoonsgegevens) developed guidelines on securing personal data in February 2013 (Richtsnoeren beveiliging persoonsgegevens).”
– Marta Borrat Frigola
Organisations are obliged to adopt adequate technical and organisational measures in order to secure personal data against loss or against any form of unlawful processing. These measures
have to guaranteed an appropriate level of security, taking into account the state of the art and the costs of implementation, and having regard to the risks associated with the processing and thenature of the data to be protected. These measures shall also aim at preventing unnecessary collection and further processing of personal data (article 13, wbp). The big question is what is an appropriate measure? Each organisation needs to internally assess, depending on all facts and circumstances of the processing of data and the nature of such data, what appropriate measures should be in place in order to ensure such “adequate level”: what is appropriate? Masking?Fully anonimysing? Pseudonimising?
“Privacy and technology are no opposites cases, but a mean to safeguard the future of our privacy.”
– Hendrik Jan Bolte
DATPROF, Bert Nienhuis
Bert Nienhuis is our productmanager and blogger of DATPROF. Bert mainly demonstrated DATPROF Privacy and all its abilities. In his presentation he talked about the complexity we, as DATPROF, normally encounter.
“We see clients with 40 tables but also with more than thousands of tables.”
– Bert Nienhuis.
Bert also mentioned that an analysis is conducted before starting to mask data. We need to know in which tables we encounter personal data. And you can start these analyses today.
Subscribe to our newsletter
Recieve free updates on new blogs, webinars and tutorials
Let us know how to reach you. We keep you updated on the latest developments concerning test data, test data management, subsetting and masking. You can unsubscribe at any time.