AI now dominates legal tech discussions, but what I find most interesting in our research and our engagement in many of these discussions is a lack of understanding and, at times, a delusional belief in the power of AI. This is from both younger and older generations in the legal profession, and it is somewhat surprising given the analytical abilities and intellect required to become an attorney. Though an overgeneralization, I have found that younger attorneys embrace technology almost without question, having grown up with digital devices in their hands. And the older generation is somewhat amazed by the power of technology and therefore unquestioning. This worries me deeply, as AI, incredible though it can be, is not intelligent at all, and, in the wrong hands, can be destructive. We talk a lot in the industry about trustworthy AI, and trust itself is core to the legal profession. It’s here that the problems start to arise.
For example, look at a recent legal scandal in the U.K., that of the British Post Office and the prosecution of 736 sub-postmasters for fraud and theft. The reason such a huge net was cast and pursued to court was that a computer system called HORIZON, built at a cost of $1 billion by Fujitsu, uncovered their fraudulent activity. Except it didn’t, because the software was full of bugs, errors, and defects, and the sub-postmasters were innocent. Some had been imprisoned, all had their worlds upended, and some took their own lives. The convictions were overturned, compensation paid, and government ministers eventually apologized for the tragic fiasco. In this example, decisions made by the computer were accepted by people in positions of trust who could not fathom how a computer system could possible do wrong.
Or, we can look at the use of COMPAS, an AI-based IT application widely used in the U.S. by courts to determine sentencing. Tests have shown it to have an accuracy rate of 65%, while random individual and group accuracy is 67%. In short, while the system is used to determine whether an individual ever sees the light of day again, it is no more accurate than asking somebody off the street. Despite this, many U.S. jurisdictions continue to use COMPAS.