Is Your AI Actually Trustworthy? What the EU Ethics Guidelines Mean for Design Teams

In a recent article we explored why AI systems can fail because the experience of using them has been neglected. We looked at how the ISO framework for Robotic, Intelligent and Autonomous Systems(this will open in a new window) provides a foundation for designing AI that people can understand, trust and control.

That piece focused on the usability and interaction design challenges. This one turns to the ethical and regulatory dimension, which is less well understood by many organisations building and deploying AI today.

The central theme is simple: ethical AI cannot be delivered by policy alone. Someone has to design it. And that design work is more demanding than most organisations appreciate.

From Good Intentions to Design Requirements

The EU High-Level Expert Group on AI published its Ethics Guidelines for Trustworthy AI(this will open in a new window) in 2019. These guidelines have shaped regulatory thinking across Europe and beyond, listing seven requirements that any trustworthy AI system should satisfy. The four requirements with the most direct implications for UX and design are:

1. Human agency and oversight. AI systems should support people in making informed decisions and not undermine their autonomy or ability to intervene.
2. Transparency. The behaviour and decisions of AI systems should be understandable to all users.
3. Diversity, non-discrimination and fairness. AI systems must avoid biased outputs and should be accessible to the widest possible range of users.
4. Accountability. There must be ways to ensure responsibility for AI systems and their outputs, including when things go wrong.

The remaining three requirements - technical robustness and safety, privacy and data governance, and societal and environmental well-being - are equally important, but are more commonly within the scope of technical and legal teams.

Each of the four design-relevant requirements describes something that has to be actually built, not just declared. A policy document that commits an organisation to transparency does nothing for the user receiving an unexplained AI decision. A values statement promising fairness does not correct a system that has never been tested with underrepresented users. Accountability is not simply having a governance framework. It results from interaction design that preserves a clear record of what the system did, why, and who can be held responsible.

Ethical AI is not a values statement. It is a design specification.

What Bias in AI Actually Looks Like

The fairness requirement deserves particular attention, since there is much troubling evidence on AI bias(this will open in a new window) that organisations deploying AI systems need to understand.

Research shows that large language models (LLMs) can produce discriminatory outputs across dimensions of race, gender, age, disability and cultural background. Studies simulating AI-assisted recruitment have found that equivalent candidates are evaluated differently based on name alone.  In other contexts, AI systems have amplified the historical biases present in the data they were trained on.

This risk is not confined to organisations building their own models. It applies wherever AI outputs influence consequential decisions.

Human oversight is both a usability requirement and a fairness safeguard. An interface that presents AI outputs without any signal of uncertainty, and with no mechanism for human review, does not just limit usability. In high-stakes contexts such as recruitment, benefits eligibility, credit assessment, or access to services, it risks causing real and demonstrable harm to the people those decisions affect.

Transparency Is Harder Than It Looks

Transparency is a core ethical requirement for AI, and probably the most poorly implemented.

For most organisations, transparency means communications: publish an explanation of how the AI works, include it in terms and conditions, perhaps add a tooltip. This is documentation rather than meaningful transparency.

Transparency in AI means designing interactions that make the system's behaviour clear to the user when it matters. In practice, this requires:

• Confidence signalling: does the interface communicate when the system is more or less certain about an output?
• Basis for decisions: can the user appropriately understand why the system produced a particular result?
• Scope boundaries: does the interface clearly show when the system is being asked to operate beyond what it was designed and tested for?
• Change over time: if the system learns and adapts, are users aware that its behaviour may differ from previous interactions?

These are not default features of most AI products. They require deliberate design decisions, specialist knowledge, and in most cases, user research to establish what level of transparency helps users rather than overwhelming them.

The transparency also must be accessible. Transparency messages that work well for digitally confident users may be confusing or inaccessible for others. A confidence percentage means little to someone who does not understand probability.  Designing for transparency means designing for the actual range of people who will use the system, not the idealised user the development team had in mind.

Accountability: The Design Question Nobody Is Asking

Among the EU ethics requirements, accountability is often treated as a governance matter rather than a design matter.

Organisations may establish AI oversight committees, appoint ethics leads and commission external audits. But accountability depends on whether, when something goes wrong, it is possible to establish what the system did, what information it was working from, what the user understood at the time, and whether there was a meaningful opportunity to intervene.

Most of these aspects are based on design decisions made long before the governance team became involved. What was logged and what was not? Was the user shown the AI's reasoning or only its conclusion? Was there a clear way for the user to challenge or override the output? Was the system designed to support human judgement or to minimise friction at the cost of oversight? These are design decisions, not legal or governance decisions, and they need to be made with accountability in mind from the outset.

Although pre-dating modern AI systems, the Horizon system in the UK Post Office is a well-known case. Sub-postmasters were held liable for discrepancies generated by software they had no means to interrogate or challenge. The interface offered no transparency, no accessible record of decisions, and no meaningful mechanism for users to raise concerns that could be taken seriously. The consequences were devastating and long-lasting.

As AI systems take on impactful roles in public services, healthcare, financial decisions and employment, designing for accountability is not optional. It is a safeguard, and in many contexts a legal requirement.

A Practical Tool: The ALTAI Checklist

For those wanting to apply these principles, the EU's Assessment List for Trustworthy AI(this will open in a new window), known as the ALTAI, provides a self-assessment tool that converts the seven ethics requirements into concrete questions an organisation can apply to review its own AI systems.

The ALTAI covers questions such as: can users contest automated decisions? Has the system been tested for bias across different demographic groups? Are users informed when they are interacting with an AI? Is there a process for monitoring and correcting unintended harms after deployment?

Running an AI product through the ALTAI is a revealing exercise. It surfaces gaps that usually sit at the intersection of design, communication, and user experience. Used well, the ALTAI is not a compliance exercise, it is a key part of design brief and review process. 

What This Means for Product Teams

There is a consistent set of principles throughout the EU ethics framework, the ISO RIA Systems framework and the ALTAI checklist. Organisations risk making a fundamental mistake if they treat AI ethics and AI usability as separate concerns, assigning one to governance and the other to design. 

They should be treated as a single design challenge with two connected lenses: one for usability, and the other for trust.

Some practical starting points:

• Map your AI system against the ALTAI checklist before your next development cycle, not after launch. The gaps it reveals are design problems that are considerably cheaper to fix early.
• Address fairness and trust within your UX research. Standard usability testing will not surface bias or how users respond to AI outputs.  It won’t show when they defer, when they question, and when they disengage. This requires research with a representative range of users, including those who may be disadvantaged by the system.
• Design transparency for the least expert user, not the most. If your confidence signalling only makes sense to technically literate users, they are decoration rather than transparency.
• Build accountability into the interaction from the start. Decide what needs to be logged, what users need to be shown, and how challenges and corrections can be made before the system is built, not as a retrofit.
• Treat ethics reviews as design reviews. Bring UX expertise into your ethics governance processes, and bring ethics questions into your design reviews. The two should not be running on separate tracks.

The Future Belongs to Trusted AI

Some organisations will discover too late that their AI systems fail the people using them. Users stop trusting them. Regulators start questioning them. Competitors who invested in the human experience pull ahead.

Others will have built AI that people trust, that works for everyone, and that holds up under scrutiny. The difference will not come down to which model they chose. It will come down to how well they designed the experience around it.

UX professionals have a central role to play in the second version of that future. The question is whether the organisations building AI recognise that early enough to involve them properly.

Key Takeaways

• The EU Ethics Guidelines for Trustworthy AI translate directly into design requirements. Human agency, transparency, fairness and accountability are outcomes to be designed, not simple values to be stated. 
• AI bias exists, including in major commercial models. Human oversight in the interface is both a usability feature and a fairness safeguard."
• Accountability is enabled by design, not just governance. The important design decisions are made long before the ethics committee meets.
• The ALTAI checklist is a practical starting point. It helps reveal gaps that are design problems, not technical ones.

Want to Go Further?

User Vision is developing a professional training course on UX and AI later this year, exploring the frameworks, methods and practical skills needed to design AI systems that are usable, trustworthy and ethical. The course draws on the work of the International UX Qualification Board (UXQB)(this will open in a new window), of which Chris Rourke serves as a UK National Expert and member of the curriculum working group. 

If you wish to be notified when the course becomes available, to discuss an AI UX audit for your own products, or simply to explore how these principles apply to your organisation, please  get in touch. 

This article is the second in a series. Read the first article: AI Has a UX Problem: What the ISO RIA Systems Framework Means for Trustworthy AI