Why Allego has historically resisted AI-only grading, what the regulation actually says, and how our feature design addresses the risk.
The EU AI Act is the world's first comprehensive AI regulation framework. It classifies AI systems into risk tiers (unacceptable, high-risk, limited, minimal) and imposes obligations proportional to the risk level. High-risk AI systems face mandatory requirements for risk assessments, technical documentation, bias testing, human oversight, transparency, and continuous monitoring.
The Act applies to any company deploying AI systems that affect people in the EU, regardless of where the company is headquartered.
Annex III of the Act lists specific domains where AI is automatically classified as high-risk. Two categories are relevant to our feature:
AI systems that evaluate learning outcomes, especially when those evaluations steer the learning process or determine access to education. Examples: university admissions, certification exams, final grades.
AI systems used for decisions affecting work relationships: hiring, promotion, termination, task allocation, and monitoring and evaluating the performance and behavior of workers. This is the category that creates risk for AI-Graded Exercise.
An AI-Graded Exercise is designed as a practice and learning tool. On its face, that sounds like education (Category 3), and not all educational AI is high-risk. But the Act cares about how the output is used, not just the tool's stated purpose.
Here's how an AI exercise score can escalate from "practice tool" to "employment decision input":
At the end of this chain, the AI score has become an input to an employment decision — exactly what Annex III Category 4 covers. The Act requires human oversight, explainability, bias testing, and the ability to override AI decisions in this category.
There is no way to guarantee that a client won't use AI exercise scores for consequential employment decisions. Even if the exercise is "just for practice," the data flows into systems where it can influence real decisions about people's jobs.
| Act requirement | Risk for AI-Graded Exercise | Our safeguard |
|---|---|---|
| Human oversight of high-risk AI | AI-only scoring has no human grader in the loop | Learner reviews AI score before hand-in (partial); company config gate blocks feature where required |
| Transparency & explainability | AI scoring criteria must be understandable | AI Practice Coach uses the exercise's scorecard template — criteria are defined by the exercise author |
| Risk assessment & documentation | Deployer (client company) must assess risk | Legal acknowledgment at exercise level; help link to regulatory guidance |
| Ability to override AI decisions | No human grader to override AI score | Learner can re-record to get a new AI score; human graders can still create scorecards (not blocked) |
| Bias testing & monitoring | AI scoring could exhibit bias | Not directly addressed in V1 — relies on AI Practice Coach's existing model governance |
The feature uses a two-layer gate:
The risk isn't that the feature exists — it's that a client uses it in a way the Act prohibits. The company config gate lets Allego control exposure. The legal acknowledgment and help documentation make the client aware of restrictions. Allego's terms make clear that using AI scores for employment decisions is the client's compliance responsibility.
The EU AI Act is the most comprehensive, but similar legislation is emerging:
The company config toggle gives Allego the ability to respond to new regulations by gating the feature per-company as the landscape evolves.