Insights / Blog

QA Consultants’ automotive testing journey

By

on

QA Consultants is part of a new era in vehicle technology identifying automotive industry gaps and quality issues. For twenty-plus years, QA Consultants has been a leader in quality and is poised to address automotive quality issues. By partnering with global experts and organizations, the QA Consultants team is ready to tackle automotive testing challenges.

QA Consultants has completed projects for eight major OEMs from North America, Europe, Korea, and Japan.

Such projects include:

  • Testing ECU’s​
  • Testing Head Units​
  • Connected Vehicle Testing Multiple Components
  • ​Mobility Device Compatibility
  • ​Automation Systems Testing Multiple OEM’s
  • ​AUTOSAR​
  • Motor Control units
  • ​Electrical Vehicles​
  • ADAS​
  • Blackberry QNX OS, Automotive Grade Linux (AGL)
  • ​Environmental Testing Multiple OEM’s​
  • Supplier Component Testing Multiple OEM’s​

Our comprehensive experience and testing methodologies ensure that drivers have the ultimate user experience when in their vehicles, buying cars, getting them serviced and enjoying their interactions with the vehicle dealer and manufacturer.

QA Consultants’ Global Expert Partnerships

To conduct the above-stated projects, QA Consultants required systems and components, testing labs, and the right partners. From Toronto to Oshawa, and all the way to Berlin, QA Consultants partnered with various automotive industry experts known for their testing technology. Paired with external experts, QA Consultants’ looked to the future of passenger cars, autonomous vehicles, and various personal transportation units. Each of these organizations offers world-class facilities and in-depth expertise.
 

ACE: Automotive Centre of Excellence
ACE is the first centre of its kind in Canada. Part of Ontario Tech University, ACE is one of the world’s largest and most sophisticated wind tunnels. This performance testing facility has been used for vehicle manufacturers and various companies wishing to test their mettle.

Fraunhofer Fokus
Located in Berlin, German, Fraunhofer Fokus is an internationally renowned research institute in the area of information and communication technology applied to facilitate digital transformation.

 

OTU: Ontario Tech University
Nestled in Oshawa, Ontario, OTU quickly earned a reputation for Computer Science, ranking among the top 10 programs in Canada and 4th in Ontario.

NRC: National Research Council
The National Research Council (NRC) is the Government of Canada’s largest research organization supporting industrial innovation, the advancement of knowledge and technology development, and fulfilling government mandates.

 

OCE: Ontario Centres of Excellence / AVIN (Autonomous Vehicle Innovation Network)
A not-for-profit organization, OCE is funded by the Government of Ontario and works in partnership with government, industry and academia to commercialize innovation and build a strong provincial economy.

AVIN:
The Autonomous Vehicle Innovation Network (AVIN) is an $80-million program designed to reinforce Ontario’s position as a North American leader in transportation technology and infrastructure systems.

 

Automotive industry gaps and quality issues

As a whole, automotive suppliers are segmented, there’s not one overall company who provides certification services or parts. Instead, suppliers are fragmented and that makes quality assurance difficult. The automotive supplier industry has many holes that result in quality issues.

These days, consumers seem unphased by automotive recalls since they happen too frequently. Mainly, recalls are due to poor testing practices and time-starved schedules. When this perfect storm cumulates, vehicle software is released before being adequately tested and integration errors occur. Since OEMs are not software developers the focus is on the product and this practice tends to result in software systems with frail security.

Cars are complex machines, made up of many different digital systems, no single company is ultimately responsible for the security and connectivity of all components. Once built, updating vehicle hardware and software is difficult and problems/weaknesses with cybersecurity can persist for years. According to Forbes (Feb. 2019), “A majority said their organizations don’t have the skills, budgets or resources to make their products secure before they go to market.”

Each new car requires its own separate combination of hardware and software necessitating the support of hundreds of different versions. This alone makes the job of testing incredibly difficult. There is no formal process to ensure the vehicle integrated technology behaviour when external connections and handheld devices that consumers are used to like Smartphones, tablets, GPS, etc.

When it comes to automation and system integration testing, the lack of technology and process to ensure infotainment, industry cluster, heads-up displays, telematics, and third-party capabilities (AUTOSTAR) are properly integrated is staggering.

According to an article from McKinsey, “Volume OEMs seem to struggle more with limiting the magnitude of recalls. In comparison with premium OEMs, they have recalled twice as many vehicles per vehicle sold between 2015 and 2017. This may indicate that their recalls are less confined to specific model years, variants, or extras. These more complex quality issues are not going away anytime soon. They result from the increasing complexity of vehicles, driven by growing numbers of new features and electronic or software components needed for autonomous-driving assistance, geolocation services, and other smart functionalities. Quality and safety requirements for autonomous vehicles will be even higher and require a zero-defect mindset and, consequently, a step change toward a no-recalls mindset.” Mckinsey, February 2019

Where do we go next?

QA Consultants is working towards addressing automotive quality issues with the help of professional organizations. To achieve this goal, QAC is investing $3.35 million (CAD). Such an investment will aid in developing a world-leading capability in software testing for connected and autonomous vehicles. In addition, this effort will create direct client engagements in connected vehicle and systems integration testing.

CAD $1.2M

From OCE for 2 years project

Connected and Autonomous Vehicle Testing

2018-2020

CAD $2.15 M of EURO 13 M

From ITEA3 / NRC for 3 years project

XIVT – Excellence in Variant Testing

2019-2021

Current Research Projects

At this time, QA Consultants is working on four different research projects OCE29920, Autonomous single passenger vehicle, XIVT (pronounced SHIFT), and ISO 26262 Certification.

OCE29920

This project focuses on developing a testing suite to resolve public safety risks created by inadequate software testing in connected and autonomous vehicles.

The initial approach:

  • Control of specific Operating Systems (AGL and QNX)
  • Testing focused on Infotainment Systems and its integrations
  • High dependency on OS architecture and versions
  • Control devices through AGL (fan)

 

The new approach

  • No dependency on OS architecture and versions (Independency of AGL, QNX or any other OS)
  • Framework purely focused on System Integration Testing across all 5 architecture pillars (Infotainment, Instrument Cluster, Heads-up Display, Telematics, and AUTOSAR)
  • Ensure all devices and components part of the automotive ecosystem work properly integrated
  • Future: Framework ISO26262 compliant

Autonomous single-passenger vehicle

QA Consultants is testing autonomous single-passenger vehicles that will focus on transporting users in airports and hospitals.

XIVT (pronounced as Shift): Excellence in Variant Testing

Out of all the possible partners in Canada, QA Consultants was chosen as the Canadian Coordinator for the XIVT program. The XIVT research project is a collaborative project between universities, research institutes, companies within industry and technology suppliers from five different countries. The project is financed by the companies involved in the project and research funding agencies.

Within the XIVT project, a method and toolchain will be defined for testing highly configurable, variant-rich embedded systems in the automotive, rail, telecommunication and industrial production domains. This will enable a highly effective, cost-efficient quality assurance, allowing the shift to autonomous, flexible and adaptive applications. The method is founded on a knowledge-based analysis of requirements formulated in natural language, and a model-based test generation at the product-line level. It is expected that XIVT methods will result in higher test coverage, more flexible processes of higher quality and better products.

The business goal is to develop design methods (and tools) of test suites. This results in the selection of test cases for highly configurable, variant-rich embedded systems in the automotive, rail, industrial production and telecommunication domains.

QAC is focused on

  • Pedestrian detection system (PDS) Raw map data in autonomous vehicles
  • Testing of Android devices with OEM specific variants

Autonomous vehicle testing: XIVT Testing Tool

1. Input data

a) Lidars

b) Radars

c) Cameras

d) Sensors

e) Sonar

e) Infra-red

2. Pattern generation

3. Patterns interpretation

4. Patterns validation

5. Actions based on patterns

ISO 26262 Certification

ISO 26262 defines functional safety for automotive equipment applicable throughout the lifecycle of all automotive electronic and electrical safety-related systems. For this project, QA Consultants built an automation test framework for vehicles using highly configurable and variant Model-Based Testing following ISO26262.

As North America’s largest independent quality assurance testers, QA Consultants has provided automotive services for more than 10 years and will continue to do so for many years to come. Through proprietary testing methods, QA Consultants’ expert staff have exceeded clients’ expectations. The future of automotive testing is bright and fraught with opportunities. QA Consultants plans to be on the forefront ensuring safety to drivers in Ontario and beyond.

Find out what other services QA Consultants provides. Contact QA Consultants today to begin a conversation.