Home Blog Top Three Variability Modeling Tools for Test Generation

Top Three Variability Modeling Tools for Test Generation

QAC Automotive and Robotics (QAaR) Quality Assurance

This monthly newsletter will focus on QAC’s activities regarding Automotive and Robotics Quality Assurance Services.

FOCUSED ON THE FUTURE

Leading the way
 
Welcome to the seventh edition of the QAaR newsletter. In this edition, we’ll focus on sharing more details about two of a set of tools that have been developed or incorporated into XIVT’s toolchain. This toolchain covers variability modelling , test generation and test optimization of variant products and consists over 20 tools (either developed or currently under development) from XIVT partners.
Welcome to our 7th edition of our Automotive and Robotics Quality Assurance Newsletter

Keeping you informed

 

Our automotive and robotics quality assurance workstreams

Research and Grant Projects
Grant budgets allow for research and development of new technologies that position QAC to become a world leader in quality assurance services.
AQS (Automotive Quality Services)
Testing and Quality Assurance services exclusively developed for Automotive and Autonomous Road vehicles.
RQS (Robotic Quality Services)
Testing and Quality Assurance services exclusively developed for Robotics and Autonomous small vehicles.
Safety and Cybersecurity
Focus on developing services that adhere to ISO’s compliance verification testing automation Cybersecurity, and Connectivity standards.

XIVT program

What is XIVT?

XIVT is a platform for efficient and effective variant testing knowledge-based requirements analysis and selection, automated test case generation and variability abstraction, applied for industrial domains. The XIVT project is a consortium that consists of 23 partners from Canada, Germany, Portugal, Sweden, and Turkey. Together, this group has built an integration platform that consists of a toolchain of techniques and methodologies for testing variant products in different domains.

This toolchain covers variability modelling, test generation and test optimization of variant products and consists over 20 tools (either developed or currently under development) from XIVT partners and provides a platform for efficient and effective variant testing of highly configurable, variant-rich embedded systems in the automotive, rail, industrial 4.0 and telecom domains. This newsletter will focus on two tools, a model editor named TESTONA and another called BeVR that are categorized in variability modelling.

Variability modeling tools for test generation

TESTONA

The TESTONA tool is a model editor based on the classification tree for systematic test design in black-box-tests. The classification tree’s purpose is to separate input data characteristics of the system under test into different classes that reflect the relevant test scenarios (classifications). It supports international test standards such as the ISTQB Certified Tester and ISO 29119 and is available online. It provides complete and comprehensible test specifications that have different modes for different levels of combinatorial coverage which enhance the generation of the test suite. It provides a connection between requirements and classification of tree elements, test cases, and test results. Therefore, any change in the specification is automatically recognized by test cases and verification is enhanced by enabling a graphical visualization of the requirements coverage.

It receives functional requirements through interfacing with AUTOSAR, IBM Rational DOORS, MATLAB Simulink, HP ALM, Atlassian Jira, Trello, Redmine, and others. Next, it exports generic code as XML files, which interfaces to MATLAB, Excel, Word, etc. In fact, it allows template-based export of executable test scripts to different formats such as Shell, XML, Text, JUnit, SQL which enhance automatic test implementation. This tool provides automatic test case generation based on several test methods considering desired test intensity, (such as minimal coverage) which can lead to cost reduction in the testing process. However, it should be mentioned that the first step in the tool usage needs to be done manually. Test cases are generated by combining classes of different classifications. These test cases cover both positive and negative test cases based on definable rules. Also, various test oracles (non-automated, implicit, derived, specified) exist to determine the expected test results.

This tool has been developed by Expleo which is an engineering, quality services, and management consulting company. Expleo is one of the XIVT industrial partners from Germany.

It will be used for variability management and fall in the variability modelling, sampling, and test modelling categorizes of XIVT project. The application is independent of industry or domain and supports all test phases. In other words, it is utilized in the development of safety-critical systems in various industries such as automotive, finance, industrial automation, aerospace, medical engineering, software/IT, and telecommunications.

BeVR

The second tool is called BeVR. It is based on a previously developed tool called BVR, which stands for Base Variability Resolution. BVR is a language for modelling the software product lines (SPL) which utilize advanced concepts in feature modelling, reuse, and realization of components in SPL. Contrast to TESTONA, BVR suffers from existing bugs which prevents it from being production-ready. In fact, BeVR is intended to be “version two” of the original BVR toolset.

BeVR is another variability modelling tool in the XIVT project that enables automatic generation of abstract test cases and efficient variant selection for test optimization. It is utilized at the beginning of the test case generation workflow.

BeVR uses a modelling tool called Papyrus which is built on top of the Eclipse IDE. This tool has a multitude of features such as added UML stereotypes, customized UI, convenient install packaging, and other optimizations for product line model (PLM) generation. With this tool, project documents are manually provided as the input by the user through a GUI. Next, the tool generates a PLM in the form of a collection of UML diagrams which is encoded as XML documents.

The architecture diagram below reflects the variability modelling tool in its usage context. Utilizing the project documents as the input, a PLM is generated manually with the help of a human user in for form of UML diagrams. The user should have expert knowledge about the domain to provide information about the structure of product line, supported features, constraints between them, and the information about individual features in the form of a UML diagram. Generated PLM enables the automatic formation of abstract test cases and efficient variant selection for test optimization. It is later passed to the tool for variant selection to instantiate models for individual products and corresponding abstract test case suites in UML Testing Profile (UTP) notation.

This tool is currently under development by QAC. Similarly to TESTONA, it will be used for variability management and fall in the variability modelling, sampling and test modelling categorizes of XIVT project. Its application is independent of industry or domain and supports all test phases.

STAY TUNED

Coming next month
In our next edition, we will present more tools from XIVT project and research that the QAaR team has completed that highlights the exposures and knowledge sharing of Connected and Autonomous vehicle cybersecurity threats and opportunities to use advanced QA techniques.

Our partners:

 
 

 

Recent thought leadership

Tags: