Key Topics Covered

AI Is Already Supporting Assessments

What New Risks Does AI Introduce?

Using AI Responsibly

Every assessment is, at its heart, a promise. The candidate promises to show what they know, and the assessment organization promises to measure it fairly. Between those two promises sits trust, which is invisible when it holds and very difficult to rebuild once it breaks.

Artificial intelligence has now entered that space. It is no longer a distant prospect but a working tool. It helps to draft and refine items, flags unusual response patterns, and lightens the load of teams running high-stakes programs.

From a legal perspective, however, AI also raises important questions. An assessment result can affect someone’s career, education, or future opportunities. That is why fairness, accountability, and the ability to understand how a decision was made have always been important. Adding AI to the process does not remove these responsibilities. It may simply change how they need to be addressed.

In this article I first look at where AI already supports assessments today. Second, I examine the new risks it introduces. Third, I offer practical guidance for adopting it responsibly. The guiding idea is simple: innovation and trust are not opposing forces, because trust is what allows innovation to last.

AI Is Already Supporting Assessments

AI is already finding its way into different stages of the assessment process. It can help to draft questions, suggest assessment scenarios, support automated scoring, and assist with administering assessments. These are no longer just possible uses of AI. They are becoming part of how assessments are developed and delivered.

One of the clearest examples is item development. An AI tool can create a first draft in seconds, allowing subject-matter experts to focus on whether the question is accurate, relevant and fair. The technology may make the process faster, but it does not take responsibility for the final product. If a question is incorrect or unfair, that responsibility remains with the assessment organization.

The same principle applies when AI is used for test security. It can identify unusual response times, similar answer patterns, or other behaviour that may need further review. This can help protect the fairness and credibility of an assessment. At the same time, an AI flag is only a signal. It should prompt further review, not automatically lead to a finding of misconduct.

This is where the legal perspective becomes important. As AI becomes more involved in assessment processes, the question is not simply what the technology can do, but how its use affects the people being assessed. When an assessment decision can influence someone's education, career or professional opportunities, fairness, transparency and accountability remain essential. Human rights considerations are also important, particularly where an AI system may produce different results for different groups.

What New Risks Does AI Introduce?

Every powerful tool comes with risk, and AI is no exception. The more AI becomes involved in assessment, the more important it becomes to understand what can go wrong.

The first concern is data. Assessments can involve sensitive information, including identities, accommodations, performance histories, and in some cases, information about minors. When AI processes this data, organizations remain responsible for protecting it. This also raises practical questions: where does the data go, who can access it, and is it being used to train an AI model?

Even when the data is properly protected, there is another problem: AI can be wrong. Technology can produce information that sounds convincing but is wrong. In an assessment, this could mean an incorrect question, flawed answer key or misleading content. The fact that AI produced it does not remove the organization's responsibility.

Bias is another concern. AI learns from existing data, which can contain biases. This may affect how questions are generated, how answers are scored, or how unusual behaviour is identified. Where an assessment affects access to education or employment, these issues can also raise human rights concerns.

These issues become even harder to address when the system is difficult to understand. Transparency matters because candidates may need to know what role AI played in an assessment decision and, where appropriate, have a way to question the outcome. Organizations also need to know where assessment data goes, who can access it and whether it is used to train an AI model, particularly when data is processed across borders.

Finally, there is the risk of overreliance. The more useful and efficient an AI system becomes, the easier it is to stop questioning its output. That can turn a tool designed to support assessment professionals into a decision-maker that receives too little scrutiny. Human review therefore remains important, particularly when assessment results can affect someone's education or career. 

None of these risks means that AI should be avoided. They show why its use needs to be considered as carefully as the technology itself.

Using AI Responsibly

Trust is built by using AI carefully. Organizations should consider what they are introducing and what could go wrong. That means considering who could be affected, what data the system will process and what safeguards are needed. It also means knowing where the data is stored, who can access it and whether it may be used to train an AI model. If these questions cannot be answered clearly, the system may not be ready to use.

Once the system is in place, human oversight becomes just as important. AI can support a decision, but people should remain responsible for important assessment decisions. As an example, human involvement is an important safeguard. It provides an opportunity to catch errors, question unusual results and consider circumstances that a system may not understand. This is particularly important in high-stakes assessments.

That responsibility also extends to the technology providers behind the system. Choosing a vendor carefully matters because outsourcing the technology does not outsource accountability. Contracts should address issues such as security, data location, access and the right to audit. Organizations should know what happens to assessment data throughout the entire process, not just while it is inside their own systems.

Finally, responsible use also means being open about AI. Candidates and clients should be told when AI is being used and, where relevant, how it affects the assessment process. Clear communication helps people understand what is happening with their data and what role AI plays in their assessment.

Used this way, AI does not have to compete with trust. The goal is not to slow innovation down, but to make sure that the technology supporting an assessment is backed by the same fairness, accountability, and care expected from the people running it.

Conclusion

AI is already part of how assessments are built, delivered, and protected. The question is no longer whether it belongs in the ecosystem, but under what conditions.

Those conditions matter because AI brings risks that the law takes seriously. It may process highly sensitive personal data, produce confident but inaccurate content, reflect hidden bias, or reach conclusions that no one can explain. Questions about where data is stored and whether it trains someone else's model add further weight. And the more dependable a tool seems, the greater the temptation to stop checking it. In each case, the organization remains responsible for the outcome, whatever the technology did along the way.

The answer is discipline, not retreat. Assess the risks before deployment. Know what data the tool touches and where it goes. Keep a person in charge of decisions that matter. Choose vendors with care, and say plainly when AI is in use, particularly now that regimes such as the EU AI Act have made transparency a global expectation.

Innovation and trust are not rivals. Trust is what allows innovation to last, and an assessment that is fair, explainable, and accountable will always be worth more than one that is merely fast.

About the Author

Jana Begun is an EU-based legal professional specializing in data protection and privacy, with a focus on regulatory compliance. She holds an LL.M. from Stockholm University and is a Certified Information Privacy Professional/Europe (CIPP/E). At Vretta, she supports GDPR compliance and integrates privacy and security principles into the company’s day-to-day operations and digital learning platforms.

Her work centers on making complex legal requirements practical. She enjoys transforming privacy and security into actionable frameworks, helping build a culture where these topics are not just policies, but integral parts of everyday decision-making.

If you are interested in discussing data protection developments and explore how to strengthen security practices, please feel free to get in touch with Jana Begun at: dpo@vretta.com | LinkedIn

References

1. Policy Horizons Canada, Foresight on AI: Policy considerations, Insight 14: AI in Assessments and Evaluations (2025), available on: https://horizons.service.canada.ca/en/2025/02/10/ai-policy-consideration/pdf/insight14-aiassessments-proof4-en.pdf. 

2. Act respecting the protection of personal information in the private sector, Legis Quebec, P-39.1, s 12.1, available on: https://www.legisquebec.gouv.qc.ca/fr/document/lc/p-39.1?langCont=en. 

3. Bill 194, Strengthening Cyber Security and Building Trust in the Public Sector Act, 2024, c 24, available on: https://www.ola.org/en/legislative-business/bills/parliament-43/session-1/bill-194.

4. Ontario Human Rights Commission and Law Commission of Ontario, Human Rights AI Impact Assessment (2024), available on: https://www.ohrc.on.ca/en/human-rights-ai-impact-assessment.

5. McCarthy Tetrault, Moffatt v. Air Canada: A Misrepresentation by an AI Chatbot (2024), available on: https://www.mccarthy.ca/en/insights/blogs/techlex/moffatt-v-air-canada-misrepresentation-ai-chatbot. 

6. Supra note 4.

7. Supra note 2.

8. Supra note 2.

9. Government of Canada, Directive on Automated Decision-Making (2025), available on: https://www.tbs-sct.canada.ca/pol/doc-eng.aspx?id=32592.