Testing Challenges of Artificial Intelligence (AI) Systems
Discussion Summary
Below, is a summarized interview article based on the video interview.
Your topic is Explicability, Trustworthy, Bias, and Ethics, which sound more like terms used in philosophy and not in software systems testing. How does one move from the esoteric to the precision of definition needed for testing AI software systems?
I think we as engineers need to understand philosophy and ethics. The field of ethics concerns the study of what constitutes ideal conduct in various situations. The main goal is to try to examine the aspects of conduct and determine actions that are deemed morally acceptable. The question of what is morally acceptable requires continuous evaluation. The study of the ethics in AI needs to happen with sufficient precision. As engineers, we need to make things more precise, but we also need to understand ethics from a philosophical perspective. We need to have an engineering approach on how to measure nondiscrimination, fairness, transparency, governance, accountability, and especially the most widespread ethical principle in AI, which is explicability and transparency.
What does it mean to state that a system is trustworthy?
Trustworthiness is defined by how much trust we can have in a system that we don’t understand. I think in many cases, as a user, you need to put that into context. Trustworthiness has a relationship to many other aspects like responsibility and it’s very important when you are trying to assess AI that runs on safety-critical systems. Responsible AI is something that is more meaningful for us engineers. It’s not an option to ignore our responsibility. AI systems are artifacts that are designed and implemented by people. We are responsible to try again, fail, we observe, and denounce when we see things going wrong. Then we build better systems. So, I think it’s more important to have a responsible AI than a trustworthy one.
How does one determine what ethical standards are required for software systems? And why are the ethical aspects of AI so important?
Cultural differences pose a serious challenge to the ethics and governance of artificial intelligence. It is important to have cross-cultural cooperation for the success of these ethical initiatives. Groups from different cultures and nations working together to ensure that AI is developed, deployed, and governed in some way that society will benefit from. It is quite common to talk about the safety aspects of a software failures that can lead to economic damage and in some cases the loss of human lives. Many talks and dilemmas show the importance of ethics and testing of AI. I think, it is a rather false problem, especially for us engineers. It is not about avoiding unethical actions or harmful consequences, but more about developing and using AI to bring about good consequences. Then you need software engineering solutions that should handle actual ethical and social considerations. The main goal would be to examine the moral aspects of conduct and determine the actions that are morally acceptable.
What is bias in software systems and what is the role of a human in the loop to train AI systems against bias and establish the ethical boundaries that bias might provoke?
The idea of complementing AI with humans in the loop intelligence is not new. The idea is to help increase the interactive exploitation of AI systems. You have the role of the human, and you have some mechanism of interactive AI that must rely on simple yet powerful, visualizations or explanations. When testing AI systems, we need to look at all sources of bias. That includes skewed data and bias within the data acquisition process. The most important part of that is the trainer. How was the AI trained? We might want to consider models trained by humans instead of data scientists versus those trained by an automated AI system. This way we can try to account for some of these biases.
Who’s responsible for the behaviour of the AI system? Can you address four areas who is responsible for the diversity in the AI system or unfair bias or unethical decisions? And, who trains the trainer?
That’s a great question. It is not an option to ignore our responsibility as humans. We are responsible to be informed and to inform, to rebuild and improve. Responsible AI is more than ticking off some ethical boxes or the development of some add-on feature in an AI system. We need to think about ethics in design. The aim is to ensure that AI system development teams are mindful of the potential consequences for individuals and societies. You need to reflect upon the problems being solved by engaging all stakeholders. We realize that the principles of transparency, responsibility and accountability are the core of the design of AI systems. We need to focus not on ethics in design, but ethics by design. You need to have responsible AI by introducing ethics for designers. This refers to codes of conduct. You have regulatory requirements and standards and certification processes to ensure the integrity of all key players that design, construct, employ, and manage these artificial intelligence systems. There are many regulations that already exist today on AI responsibility and trustworthiness like the European Union initiative, but there are other companies that work in this area of responsibility.
What are the risks as we build up and evolve legacy AI systems in corporations and industry, as the technology improves with various releases of different types of AI systems that ultimately become legacy AI systems?
Yes, we are faced with the challenge of having AI systems that will become obsolete after some time. The risk is that we are stuck with software made during the infancy of AI ethics. We would need to tune the software components that contain AI. I remember seeing research papers on methods that enable dealing with errors without the need to retrain the AI systems. Meaning we can take these legacy AI systems and without retraining them and making them more useful.
What do you see as the future for A.I systems and what needs to happen to progress it?
We need to ground future research in ethical AI Explicability, trustworthiness, and bias in our engineering practice in human cognition. We need to move from ethical dilemmas to real engineering problems related to ethical AI A self-driving car may face a decision between different actions, both with harmful consequences. It is an interesting problem but, I think it’s interesting for philosophers, not for engineers, simply because this problem is unsolvable by construction. The problem is that for the question of whom to kill, all the answers are all bad or wrong. We need to look at engineering problems more than philosophical problems.
Eduard is a researcher and lecturer at Mälardalen University in Västerås, Sweden, primarily affiliated with the Software Testing Laboratory and the Formal Modelling and Analysis groups at the Department of Networked and Embedded Systems. Eduard is a native of Bucharest. He earned his engineering degree from Polytech University in Bucharest in 2009 and a Ph.D. from Mälardalen University in 2016. His research interests are in software engineering and empirical research, specializing in testing and quality assurance of industrial software systems. He teaches automated testing and model-based testing at the master’s and PhD levels as well as to industrial practitioners. He is currently doing research on a diverse array of topics, including the ethical and human aspects of AI automation in software engineering and testing.
Peter Watkins is the Chief Operating Officer at QA Consultants and is the current President of the World Network of Productivity Organizations. Peter previously worked as Executive Vice President and Chief Technology Officer for the McGraw-Hill companies, Executive Vice President and Chief Information Officer for the Canadian Imperial Bank of Canada, Global Leader for Financial Services for EDS, and National Practice Director for Information Technology for E&Y.
