Insights / Blog

Variant Testing Tools and Model-Driven Developments

By

on

QAC Automotive and Robotics (QAaR) Quality Assurance

This monthly newsletter will focus on QAC’s activities regarding Automotive and Robotics Quality Assurance Services.

FOCUSED ON THE FUTURE

Leading the way

 
Welcome to the second edition of QAaR newsletter. In this edition, we’ll focus on the XIVT project plenary meetings that happened in Istanbul, Turkey, during November. The issue also focus on our QAaR L&L (lunch and learn) events, what was discussed during our first meeting in December, and ideas for the next internal L&L sessions.

Welcome to our December edition of Automotive and Robotics Quality Assurance Newsletter!

Keeping you informed

 

Our automotive and robotics quality assurance workstreams

Research and Grant Projects

Grant budgets allow for research and development of new technologies that position QAC to become a world leader in quality assurance services.

AQS (Automotive Quality Services)

Testing and Quality Assurance services exclusively developed for Automotive and Autonomous Road vehicles.

RQS (Robotic Quality Services)

Testing and Quality Assurance services exclusively developed for Robotics and Autonomous small vehicles.

Safety and Cybersecurity

Focus on developing services that adhere to ISO’s compliance verification testing automation Cybersecurity, and Connectivity standards.

Within the XIVT project, a method and toolchain will be defined for testing highly configurable, variant-rich embedded systems in the automotive, rail, telecommunication, and industrial production domains. This method is founded on a knowledge-based analysis of requirements formulated in natural language, and a model-based test generation at product-line level.

The second XIVT Plenary meeting happened from November 5 – 7th, 2019 in Istanbul, Turkey, The meeting was hosted by one of XIVT’s Turkish partners Arçelik, company part of Koç Holding group. For three days, partners discussed the progress of each Work Package tasks and deliverables and addressed issues and risks. The team also planned to get prepared for the ITEA3 review meeting, happening in January 2020.

XIVT report

After a few months of hard work, the XIVT team a state-of-the-art report on requirements-based variability modelling and abstract test case generation. This is a public report that can be download here: PDF

Variant testing tools demo

As part of the XIVT project, tools and techniques were demonstrated for the group to get a deeper understanding. IFAK presented MBT Creator tool which supports the steps of model synthesis to test generation and uses common modelling notations from UML as well as components for model synthesis and test generation. Expleo presented TESTONA, a test case generation tool based on the classification tree method, with variability management, and MERAN, an integration tool for requirement management that also supports variant management. RISE presented VARA (Variability Aware Reuse Analysis), a variability-aware requirements reuse analysis method which aims to automate the requirements reuse analysis and thus helps teams achieve quick and quality delivery of software systems.

Network Opportunity

Peter Watkins, QAC Chief Operator Officer and Toni Jardini, XIVT Program Manager also had the opportunity to meet with the Canadian Trade Commissioner in Turkey. Together, they discussed the potential technology partnerships between Turkey and Canada, with a focus on CV/AV (Connected Vehicle/Autonomous Vehicle)

December 2019 QAC L&L (Lunch and Learn)

On December 11th and 12th, 2019, we had two QAC L&L sessions focused on the XIVT project and the technologies involved regarding this 3-year program with 5 different countries (Canada, Germany, Portugal, Sweden and Turkey). The purpose of those sessions was to introduce main concepts of Product Lines and the challenge of testing and quality assurance of a highly configurable, variant-rich systems. The group discussed how model-driven development, automated test generation and test case optimization can be applied in product lines testing.

Challenges

There are two main challenges discussed during L&L:

  • The difficulty in defining system requirements for a product line due to the complexity of system under analysis
  • How to optimize those requirements from a testing perspective

Sharing knowledge


QAC leadership has been developing new knowledge-transferring and training programs for our teams which involve Machine Learning, Artificial Intelligence, High-Variability and Cybersecurity. In addition to L&L, QAC is now implementing a regular training and education program for state-of-the-art developments in software testing.

Some concepts discussed during the L&L

Model-driven development

Models can be used to manage the increased complexity of product lines, helping clarify the state of the project and drive development. A feature model can be used to describe all the functionalities provided by products across the whole line, how they are distributed and whether there are dependencies between them. It describes a product line’s requirements, but to specify how those are fulfilled, other representations are required. Functional models (i.e., UML) can be used to describe system structure and behavior at various abstraction levels, down to concrete components. However, extensions are required to keep track of which functional components realize which features, and to extract individual product models out of the whole product line model. A delta model is composed of a baseline product model (the core) and a collection of deltas, which describe how individual features can be added to the core by adding, removing or replacing specific fragments in the core’s functional model.

 

Test cases generation

As the size of a product line increases, it becomes impractical to maintain test cases for all products, even as ensuring proper test coverage becomes ever more critical. The solution is automation — automatically generating test cases from requirements and functional models. There are a few ways that automatic test case generation can be performed: Model simulation, Model transformation, Machine learning (based on data collected from existing test sessions) and Natural Language Processing (based on textual product descriptions).

Test cases optimization

Test cases optimization can be described as a set of processes in order to have a better selection of what to test, when to test and why to test. Test Case Prioritization prioritizes tests based on a performance criteria set. Test Case Minimization or Reduction removes test cases that contains redundant components for the target test, and Test Case Selection determines which test cases should be included in the test suite. Two most important criteria in test prioritization and selection of the variant-intensive products are safety and security which reflect safety and security requirements of the system.

STAY TUNED

Coming next month

In our next editions, we will outline some of that technologies that have been developed by QAC regarding Model-Based Testing and Automotive System Integration Testing, plus automated framework for robotics testing in simulated environments. We also plan to demo some of those in upcoming L&L sessions or interactive events.

Our partners:

 

 

 

Recent thought leadership

[qac-carousel id=”20158″]