Automated test file generation – creating messages for comprehensive testing in an instant

When systems have to be tested, banks and other providers of message-based transactions face the need for extensive test data. Commonly created manually or using custom-made scripts, conventional test message batches can only ever simulate a limited subset of almost endless valid and erroneous test cases. A versatile message management platform provides a solution for the problem: creating test files for entire business rule stacks becomes a matter of seconds.

Centralized, cloud-based XML management solutions are widely recognized for their benefits: improved documentation, simplified versioning, and more efficient development, to name a few. Yet, significant additional value lies in closely integrating such platform with testing and validation workflows.

Advanced tools aid banks and their customers by native integration with testing suites. As schemas and business rules are defined, they are instantly available in validation and simulation engines that banks can utilize internally or provide as customer service to corporates and vendors. Immediate resource savings can be observed when the messaging management platform delivers test files reflecting the entire test case spectrum, in a fraction of the time it used to take.

Manual testing is always limited

Banks and payment processors need to ensure maximum compatibility with every new version of standards and rules, having to thoroughly test their systems. When production systems receive malformed message files, predefined response messages have to be returned. Files that cannot be processed automatically cause extra work, as they require manual resolving or in a worst case scenario might even lock up entire processes.

This establishes the requirement for exhaustive testing routines based on precisely crafted test messages simulating real-life scenarios. Testing systems against specification changes can be laborious, as updated rules need to be identified and test cases set up to create test files that contain both data permitted according to the new rules and message files where these rules are violatetad.

If SEPA rules for example limit the “Debtor name” field (Dbtr/Nm) to 70 characters, test files have to cover a minimum of four baseline cases: field missing, field empty, 1-70 characters, and 71 characters. In addition, error scenarios to be tested include strings containing invalid characters, or otherwise invalid content, etc. Even an instance as simple as a message field may trigger dozens of variants. Possible duplicates, wrong checksums and the like further increase the amount of files.

Given the average set of business rules applicable to an XML-based transaction, testing the smooth handling of valid messages – along with the outcome of every possible error in incoming messages – quickly leads to tens of thousands of test transactions; a single payment commonly follows around 100 business rules, requiring a batch of 300-400 test files. A mostly manual process of creating these is not only costly, but inevitably limits the amount of testable scenarios. And once the specification changes, every test file created has to be redone.

Automated test file generation to the rescue

This is where a message management platform displays its superiority: since the system is being fed with exact definitions of what is allowed and what is not it is an easy task for the user to outline required test cases for rules as they are created. Already during its setup, every rule can hence be stored along with instructions for required test cases (the XMLdation myXML service refers to these as “actions”); using an interactive user interface, the user comfortably defines scenarios and instructions for later generating a comprehensive set of test files.

“creating large batches of message files is available with a single button press” Common standard formats, such as ISO20022, CGI-MP, EPC, or country-level standards, are available by default as reusable public libraries, bringing along baseline test scenario setups for immediate use. The template file used for generating the erroneous message variants can also be changed by the developer on the fly, increasing the amount of easily available test cases exponentially—all at the click of a few buttons in an intuitive browser-based user interface.

Once the system has been taught the business rules and related test data requirements, creating large batches of message files and downloading them as a single archive is literally available with a single button press. By default, test messages are retrieved from the XML management service as a file archive whose contents can then be fed into the bank’s system.

Whether testing procedures call for a set of all possible combinations of valid message files or for an archive filled with files that cover every possible error condition: the generated test files consistently contain all test cases previously defined. Lifecycle management gives access to not only the latest, but also previous versions, enabling the creation of specific test data for a variety of situations.

And if any rules are updated or new test cases set up in the system, the entire package of test files can be recreated in an instant. Since the database integrates the new specifications with any pre-existing rules, adding new actions extends the coverage of simulated test scenarios without the risk of long standing test scenarios getting lost, as easily happens in manual processes facing an increase in complexity.

Test files for XML and other formats

Amidst the ongoing transformation from legacy formats to the standardized use of XML in financial (and other) transactions, the importance of the ISO 20022 standard is only going to increase the need for an holistic message management solution like myXML. Nonetheless, as a format-agnostic SaaS platform, myXML is not limited to XML: the current debate in payments most prominently features JSON-based APIs, popularized by mobile applications.

As PSD2 and XS2A are going to further increase the amount of interfaces to be provided, a future-proof message management system has to be able to provide the same features to test and fail-proof mobile transactions. With XMLdation’s service, payment service providers can already today utilize JSON test messages based on a similar feature set as with XML—one platform can handle all message-related management and testing needs, including JSON/APIs.

Simulating real life

Easy access to a complete set of message files with all potential errors or any possible valid combination of data has the potential to save not just man hours but man years. However, this is not limited to the most obvious—the resources freed up from manually creating test data. During development, the ability to test-run every scenario possible helps to achieve better quality faster, as the breadth and depth of a service like myXML cannot be achieved otherwise.

Yet, covering all potential issues is not the only concern. Another common need is to test systems under real-life conditions: no longer are the requirements limited to covering all possible issues, but to evaluating how a system behaves in a scenario with a certain share of incorrect messages. When XML management is combined with a versatile test file generation engine, as in the case of XMLdation’s myXML, these scenarios can be simulated with similar ease. Creating such data sets manually would already require significant investments, with unpredictable quality.

Another benefit of using a message management platform over conventional test file creation is the ability to update the content of the files according to certain specifications; for instance, changing dates on the fly or injecting user-defined test content. This allows to use actual account numbers or specific payment messages as the system is going to handle in production later.

A real-life example could be the need to randomly mix 10% of erroneous files into a million transactions in each file, with all kinds of variation in content and with up-to-date timestamps and real-life bank account numbers. The system creates both the valid and the erroneous messages based on the information stored in the database, while ensuring that the output corresponds with what a production system would receive on a daily basis. This allows banks and other message transaction providers to run complex load tests and evaluate the performance of their systems.

myXML – more than just batch processing

While the mere ability to generate scenario-based batches of test files already improves the coverage of system testing far beyond what is possible in a manual workflow, the most advanced generation of XML management services still has more to offer. The versatility of the solution increases even further as the secure retrieval of files for testing is integrated with a bank’s testing procedures.

Systems can be set up for tests with the latest formats on a regular basis with no human intervention in two ways: via the cloud API or by locally integrating a custom Java archive (JAR).

Using the API, test files can be retrieved directly from the cloud-based message management system. Even scripted integration is possible, reducing the effort for accessing millions of test files to the integration of the API communication.

For scenarios where the test data itself is sensitive, it is possible to deploy our test file generation application on premises. This locally deployed application (a Java archive or JAR) is generated based on the information stored in the XMLdation cloud platform and can then be integrated with systems not connected to the internet. This way data such as account numbers, reference codes, and debtor/creditor names can be processed entirely within the bank’s systems.

The automated creation of test files is just one of many benefits from a streamlined workflow facilitated by XMLdation’s centralized message management service myXML. Along with reduced complexity in the handling and documentation of schema definitions, even in contexts with hundreds of different applications, myXML significantly reduces time and in-house resources required for testing.