On April 22, the Superior Court of Quebec annulled the August 2025 arbitration Association des Ressources Intermédiaires D’Hébergement du Québec (ARIHQ) v. Santé Québec, as the ruling was based on nonexistent, likely artificial intelligence (AI)-generated legal doctrine and jurisprudence. Each citation of a legal precedent linked in the award led to completely different judgments; the precedents did not actually exist.
Quebec Judge Martin F. Sheehan ruled that arbitrator Michel A. Jeanniot could not delegate his decision-making authority to a third party, much less to an AI large language model (LLM). Since nonexistent references constituted the entirety of Jeanniot’s legal argument, this undermined confidence in his ruling, as an accurate, human-led review on Jeanniot’s part could have prompted an alternative sentence. Sheehan established that adjudicators who outsource their reasoning to an LLM are abandoning the deliberative function they are appointed to perform and are therefore liable for the model’s mistakes.
The ruling marks an important precedent for the role of AI in law, highlighting both the danger of granting decisional power to generative AI, and the pernicious integration of AI into legal systems and research. Legal professionals are increasingly using LLMs for the purposes of summarizing texts, scraping big data for relevant documents, cross-referencing precedents, and conducting legal research. AI is also being embedded into legal platforms, such as search engines, browsers, and databases, raising concerns about the supposed objectivity of legal tech, and creating a crisis of accountability for errors when humans are not deliberately prompting models.
Growing adoption of AI systems, which frequently hallucinate and possess ingrained biases against racialized groups, both negatively impacts litigants, and violates the Canadian justice system’s tenets of fairness and impartiality. Thus, AI safety—the outlining of AI risks and formation of computational and governance frameworks to address these risks—is a legal necessity.
Researchers at Mila, a Quebec AI institute, have found that transformers, the neural network architecture for LLMs, often misstate facts, generate invented content, and incorrectly fill in gaps in reasoning, all of which can interfere with the validity of legal work. AI systems are also unable to fully grasp the complexity of context-dependent human values and fail to handle shifting circumstances. This issue, commonly known as value-lock, makes LLMs ill-equipped to handle complex normative questions, which can lead them to reinforce outdated, exclusive social norms. For example, media outlets tend to ignore peaceful protest, instead covering extreme or violent events to attract viewership. LLMs trained on this data may then reinforce this view of social movements and, in doing so, maintain current regimes of power.
Further, the data used in transformer pre-training, which is gathered through web scraping, is often unrepresentative, holding a Western hegemonic bias and perpetuating discriminatory views and social stereotypes. This means that marginalized communities are at a higher risk of being victimized in legal decisions. For example, AI predictive policing systems, which use historical crime data to decide where to deploy officers and to identify individuals “most likely” to commit violent crimes, can absorb biases from past criminal data and replicate them in their recommendations for punitive outcomes. Since Black communities have historically faced overpolicing and overincarceration in Canada, such systems risk further entrapping Black Canadians in the country’s criminal justice system.
While AI tools can offer meaningful research potential, these nontransparent, inaccuracy-prone, and inherently biased models must be heavily regulated. At the baseline level, AI should be banned from direct legal decision-making, prohibited from generating text that could directly influence legal outcomes, and thoroughly audited for any purposes ranging from legal drafting to research. In increasingly complex and covert circumstances, the deep integration of LLMs into legal systems will necessarily demand both self-regulation from within legal bodies, and external public governance over which AI systems are allowed to be deployed in the legal field, and for which cases.