A National Police car in front of a police station in a file image. E
P Ethics gradually opens space between technology. A new wind that questions the mantra that technique is without errors, that it is almost incontrovertible. By itself it has no good or bad. But its use, preparation and learning, always dependent on the human being, are closely related to such moral attributes. And this review has been subjected to an artificial intelligence system such as VeriPol, an algorithm designed to detect false complaints and deployed in all Spanish police stations for just over two years .
Presented as the only one in the world, it analyzes and detects the combinations of most common words when lying to a police officer. Among the words that predominate in false accusations are the “pulling” from “behind” or from the “back” of the “backpack” or “purse”, carried out by an attacker wearing a “helmet” or dressed in “black”. Descriptions do not normally focus on the facts, but rather on the stolen items and their value. According to the data collected by its promoter, the inspector Miguel Camacho, the system has a 91% success rate. Like all artificial intelligence, its training requires data. In this case, they come from the analysis of 1,122 complaints from 2015 —534 true and 588 false —.
This is where the ethical doubts begin. Although the police did not want to answer any questions, the information collected to launch VeriPol neglects the social diversity of Spain, according to some privacy experts. Most of the data comes from Andalusia and therefore, as Ricardo Baeza-Yates, Director of Data Science at Northeastern University and Professor of Computer Science at Pompeu Fabra University argues, the coverage of the entire sample space of language is not reflected in the system . “The problem is no longer the quantity, which a thousand is not a lot, but the variety of words used in Spain. For me it is a mistake to extrapolate the use of language in Andalusia to other autonomous regions such as Galicia and the Basque Country. Not to mention immigrants, who are not even included. ”
One of the algorithm's programmers, Lara Quijano, professor and researcher at the Autonomous University of Madrid, affirms that the training she underwent with artificial intelligence started from little more than a thousand anonymous data, chosen randomly. A police officer worked for a whole year eliminating any reference that would allow geolocation of the complaint. Neither names nor streets nor neighborhoods. “This way we avoided falling into unnecessary biases. We made sure that the algorithm was unable to process any reference to a specific location or geographic area. This is reflected in the research article that we publish. ”
There is no valid magic figure, but Baeza-Yates establishes as truly representative one per 10,000 inhabitants of each Community. Lorena Jaume-Palasi, a member of the Government's advisory council for artificial intelligence, understands that a tool like VeriPol requires dynamic data, capable of serving all types of sociolinguistic cultures . He gives the example of a Muslim woman who goes to report, who has her own expressions and may even feel intimidated by being surrounded by many men and altering the way she speaks. "Body language also matters in reporting and does not appear here. This system creates ideal types. It does not describe reality, but artificially establishes a mechanized description of reality. The reality is more dynamic than just a few words ”, says Jaume-Palasi.
The biases are increased a little more by the way in which VeriPol is used. In principle, it should process the natural language of the complainants, although, in reality, it deciphers what the police say in the complaints . The algorithm does not directly interpret the words and expressions of those who come to a police station. No one doubts the professionalism of the National Police Corps, but rather the training that artificial intelligence receives in such a sensitive matter, which carries administrative and even criminal sanctions. “That 9% are wrong implies that the system wrongly accuses nine out of 100 people. And this is a very serious ethical conflict, ”says Baeza-Yates.
Rebekah Overdorf, a computer engineer and researcher at the Polytechnic University of Lausanne, recalls that, like any other machine learning model, this system learns patterns from the data provided. In this case, the police reports establish the border between truth and lies. In other words, if the complaint of a victim collected by a police officer, for whatever reason, uses language similar to that of false ones, it would be defined by the system as misleading —even when he has been robbed. “This tool does not show if a testimony is false. It only concludes how much it resembles the complaints that have already been previously classified as true or false during the training, ”he reasons.
Greater specific regulations
The police recalled in the presentation of the tool that in no case VeriPol prosecutes the complainants, especially because it would be illegal because it is forbidden to sanction a person only after an automation process. The final decision is up to the agents. The legal debate, then, shifts to how to regulate the use of artificial intelligence . Natalia Martos, founder of Legal Army, advocates developing specific regulations in Spain that address the impact of automation in areas as sensitive as social profiling. “We could be defenseless against certain decisions that we leave in the hands of the machines. We are lagging behind in regulation. Finland, Singapore and Japan would be good examples of how to face this reality. ”
Apart from discussing whether it is feasible to homogenize the concept of truth and falsehood , data science has shown that polygraphs such as VeriPol are not conclusive, they lack validity to use them as evidence. For Jaume-Palasi, it is even more delicate to point out the alleged victims, whose testimony is directly challenged by the police authorities – according to police figures, in 2019 they used the algorithm in 35% of the complaints. “Conceptually it is very problematic. We are changing the legal bases. We no longer speak of in dubio pro reo (in case of doubt favor the accused). We directly question the victims. ”
There are also doubts about the dilemma between false positives and false negatives that always accompanies artificial intelligence . The relevance here is that we are talking as much about people who get away with it and about others who are wrongly accused – "and mistakes are always going to happen with technology," Overdorf says. Society has to consider where it sets the bar for tolerance. “Should we accept more false reports in order not to accuse an innocent victim? How many innocents are we willing to accuse to catch the bad guys? Who determines this threshold? A programmer? The legislator? Is it arbitrary? ”, Overdorf raises.
One way to tackle such dilemmas, at least that is how Baeza-Yates slides it, would be to create a regulation for an external ethics committee, without conflicts of interest, to approve the algorithms used by institutions and companies that affect people. “There is a trend that technology can do more than it really can. There is the technological phrenology. Science exists, but we apply it falsely, such as when identifying criminals by their facial features or to determine if someone is lying because of how they express themselves. ”
Different experts demand that VeriPol sit on the couch of ethics. Jaume-Palasi is categorical when defining it as a victimization system, which is based on biased concepts due to how he extracts and evaluates the data. The machine exaggerates a certain angle of the complaints. The more it is used, the more it broadens the vision and way of seeing reality with which it has been trained . “In artificial intelligence, interdisciplinary work is needed. We cannot ask that an engineer be a lawyer, a sociologist and a data scientist. There is no perfect model or 100% valid statistics ”, he concludes.
You can follow EL PAÍS TECNOLOGÍA at Facebook , Twitter , Instagram or subscribe here to our Newsletter .