Guest Blog Author! Ethics and AI with Dr. Graham Culbertson (part 2 of 3)
This is a three part series on Ethics and AI. Dr. Graham Culbertson is our guide in this exploration of what ethics is, how we apply ethical thinking in different situations, and ultimately has a little fun with what ethical thinking might look like when applied to AI. Feel free to jump in here, or go back and read Part I if you prefer (recommended).
Dr. Culbertson is a professor at the University of North Carolina at Chapel Hill, and hosts the podcast Plumbing Game Studies, in which he explores game studies through the lens of philosophy and vice versa. He has been teaching at the high school and college level since 2006.
Ethics and AI, Part II
Welcome back to this introduction to AI and ethics blog post series. After a brief overview of Part I, this post will cover the two big modern schools of morality: deontology (aka rules-based) morality and consequentialist (aka utilitarian) morality, and then explain how those two fields can be used to resolve ethical dilemmas.
First, the recap! In the first post, I focused on Rushworth Kidder’s concept of ethics as a choice between “right vs. right” decisions, whereas what I’m calling morality is a choice of right vs. wrong. So murder, which is killing someone for bad reasons (such as personal gain) is morally wrong, whereas killing in self-defense is a difficult choice in which our values call on us to weigh both sides. Kidder’s recent book is a clear explanation of these concepts with a focus on contemporary everyday life, but the concept can be illustrated more forcefully in an ethical dilemma described by Jean-Paul Sartre in his famous work Existentialism and Human Emotion.
One might think that Sartre, a member of the French resistance to the Nazis during World War II, would find there to be no moral dilemma in the question of whether or not one should fight the Nazis. Isn’t it obvious that joining the Nazis is bad and fighting them is good? Isn’t this a simple right vs. wrong question?
In a vacuum, it certainly is - there is no dilemma between joining the Nazis and fighting the Nazis, except for people who are willing to make an immoral choice and justify it to themselves somehow. But Sarte says that this misses the ethical question faced by one of his students during the war, a young man who hated the Nazis but whose mother was sick and needed care: “The boy was faced with the choice of leaving for England and joining the Free French Forces - that is, leaving his mother behind - or remaining with his mother and helping her to carry on.”
There is, according to Sartre, no obvious way to solve this problem. Both of them get at the boy’s core values; he can’t easily choose between the part of himself which wishes to liberate France by fighting the Nazis and the part which wishes to protect his mother by staying and caring for her. There is no right answer, and according to Sartre, going to look for a right answer is simply an attempt to sidestep the ethical responsibility that every human being bears; as Sartre puts it (using “man” as a synonym for “humanity”): “man, with no support and no aid, is condemned every moment to invent man.” Even if someone does adhere to a strict moral code, they are always free to choose a different moral code to adhere to; for Sartre, there is no escape from ethics into morality.
But if you do choose to embrace a particular moral code, it can provide clear answers to otherwise difficult choices. So let’s take a look at those moral systems I mentioned earlier, deontology and utilitarianism, to show how they can resolve ethical dilemmas through the application of strict rules. Deontological ethics, meaning rules or obligation based ethics, and consequentialist ethics, meaning ethics based on the consequences of decisions, are two larger categories of ethics. But in practice, however, people often use deontological ethics as a synonym for the ethics of Immanuel Kant and his followers and consequentialist ethics as a synonym for the utilitarianism of Jeremy Bentham and his follower John Stuart Mill. In this essay, I will be using the terms deontology/Kantian/rules-based ethics as synonyms; the same goes for utilitarian/consequentialist ethics.
In the broadest context, deontological and utilitarian ethics are more similar than different. Both schools of thought are liberal, rationalist responses to the work of David Hume coming from 18th century European enlightenment thinkers. By “rationalist” I mean that they think that reason and logic, not emotions or beliefs, should be used to resolve ethical dilemmas, and by “liberal” I mean that they try to value the freedom of each individual equally. So for them, the question of whether to join the Nazis or fight the Nazis is an easily resolvable one. The only distinction arises when there are liberal, rational values on either side of an ethical dilemma - as in Sartre’s example of fighting the Nazis or staying home to care for one’s mother.
But now let’s leave the Nazis out of it and turn our attention to an AI-inflected version of the trolley problem. The trolley problem is the most taught ethical dilemma in English-language philosophy, and it’s pretty simple: imagine there are 5 people on a trolley track, and a trolley is barrelling toward them. It’s certain to kill them! You, and only you, can reach a lever which will divert the trolley to a different track, saving those 5 people. But the track you divert it to has one person on it, who will certainly be killed if you divert the trolley. Should you divert the trolley, or not?
At this point, students generally want to know more (are the five people on the track good people?) or dispute the premise (why don’t I just yell at them to get off the track?). But of course, the whole point of this dilemma is that none of these things matter - as a liberal, rational thinker, you are supposed to realize that it is right to divert a trolley from killing five people, and right to not divert a trolley to kill one person. For a philosopher like the anti-rationalist Friedrich Nietzsche, who argued that some people are simply more worthwhile than others and can prove their worth by dominating others, this might not be much of a dilemma. But for liberal rationalists, there is a clear ethical dilemma here. If you are faced with this situation, you must consider your values and make your choice, inventing the kind of person you wish to be and the world you wish to live in, in a Sartrean manner.
Unless, of course, you are a self-driving car! Self-driving cars can categorically not make the kind of humanist decision that Sartre wants people to make, because self-driving cars aren’t people. Self-driving cars cannot cultivate a sense of self, or practice Aristotelian virtues. Self-driving cars require a clear mathematical rule in such a circumstance, an algorithm which clearly tells them what to do. And although any self-driving car will be programmed to be a liberal, rational decision maker (ie, don’t run over people simply in order to go a little faster; value all people the same amount; follow a set of mathematically, logically consistent rules), the instruction to rationally treat all people equally won’t help if the self-driving car needs to choose between running over five people in it’s way or swerving to avoid them but hitting a sixth person on another road. Bentham/Mill and Kant, at this point, finally disagree.
In a moment, I’ll give you some quotes and explanations from Mill and Kant to explain how they arrived at their decisions. But before I get into the details of their arguments, let me briefly explain them in broad strokes. As rational liberals, both philosophers want to count each human equally in a decision-making algorithm. This is what makes these two schools of thought so appealing to contemporary thinkers in AI: they are mathematical expressions of how to treat humans equally. Where they differ is when they measure the success of their rules. Consequentialism means concerned with consequences; the utilitarians want to know what the math looks like at the end. If the self-driving car kills five and saves one, that’s -4, not a good consequence. So the car should be programmed to swerve, saving 4 lives, and thus getting +4 people’s worth of happiness out of the situation.
Kant, on the other hand, is concerned with being logically precise in the formation of the rule, before the outcome is known. The mathematical perfection of the rule is the only thing that can be known; the outcome is too messy and unknowable to count for much. Deontology means the study of obligations or duty; Kant wants to study what we have to do, not what might happen after we act. So Kant applies his evaluative criteria to the rules itself, and invalidates any action that follows a contradictory rule. In the self-driving car example, Kant says that the rule goes something like this: “Self-driving cars should value life, and thus swerve to kill someone in order to avoid killing.” That’s not valid, according to Kant, because it’s contradictory - killing someone because you hate killing is the equivalent of saying that 1 equals -1. By contrast, the rule “Self-driving cars should value life, so they should not swerve to kill anyone” is perfectly logical and consistent.
Here’s one of Kant’s main examples. What if someone wishes to obtain a loan but they don’t intend to pay the money back? If we are looking at it from a utilitarian standpoint, we might see it as a good thing, if they are going to spend the money on something good. Should someone defraud a bank in order to feed their starving children? Probably so, says a utilitarian!
Not so fast, says Kant. “Act only according to that maxim by which you can at the same time will that it should become a universal law.” To put it another way: all human actions should only be done when they are perfectly logically consistent with themselves and our values, so you should be able to imagine a rule that will let you do what you want but breaks no other rules and also doesn’t contradict itself. Lying to a banker to feed starving children might be compatible with our rules about valuing other people. But it’s incompatible with itself.
In other words, the act of promising to pay a loan requires that everyone believe that promises are valid. You can’t make a rule that says “I give you my word to pay you back but I don’t have to keep my word.” That’s internally inconsistent! What’s more, no loans would ever be given out because everyone knows that lying is ok. Kant thinks that the utilitarian position has no foundation. It sounds nice - sometimes lies are good because they help people, and we want to live in a world in which helping people is the best outcome! Kant says: a world in which everyone could be lying at any time is not a world that’s going to help anyone.
Let’s return to the self-driving car. Imagine you are thinking about purchasing a self-driving car, and you can choose from 3 models:
The Nietzschean Will to Power Roadster
The Benthamite Best For All Buggy
The Kantian Karavan
The salesperson explains to you that the Nietzschean Roadster cares only about you and you alone, and it will happily swerve and kill any bystanders to get you to your destination a little faster! This is an obvious moral failing, and I’m confident you wouldn’t buy this car. So you consider the Benthamite Buggy - guaranteed to always kill the fewest possible people. That sounds good, until you ask the salesperson a question - what if there are two people on a bridge, and the car can only save them by swerving off and plunging you into a ravine?
“Well then,” the salesperson responds, “it will most assuredly swerve into the ravine and murder you!” Slightly perturbed by the ease with which the salesperson responded, you finally consider the Kantian Karavan, guaranteed to never kill anyone on purpose. After all, can a car truly be said to value life if it’s programmed to deliberately take life? And is the Benthamite Buggy sure that it won’t cause a forest fire with its ravine crash and kill millions? The safe thing to do, the right thing, the pure thing, is to make cars that try not to kill people, not cars that try to kill the right people.
Of course, Bentham wouldn’t be convinced by this at all. The ravine fire problem is just a matter of research and programming; program the car not to go into ravines when fire conditions are bad! Otherwise one should simply follow the rule of what’s best for the most people. Or, as John Stuart Mill, Bentham’s famous student, puts it: “Though it is only in a very imperfect state of the world's arrangements that any one can best serve the happiness of others by the absolute sacrifice of his own life, yet so long as the world is in that imperfect state, I fully acknowledge that the readiness to make such a sacrifice is the highest virtue which can be found in man.” Doing what is right doesn’t mean following the most logically consistent rules; it means doing whatever is best for everyone, even if that means sacrificing oneself. On this reasoning, engineers should be working round the clock to make sure the Benthamite Buggy doesn’t cause any unintended consequences, and otherwise we should all ride in it, secure that it probably won’t kill us, but if it does kill us, our sacrifice is “the highest virtue.”
Sartre, perhaps, might say that the fact that we have to accept a mathematical model of morality every time we ride in a self-driving car means that it is unethical and anti-human to ride in a self-driving car. This is, I think, precisely the position that Aristotle would take, whose virtue ethics was the most important form of ethical thinking in the western world up until enlightenment thinkers like Kant and Bentham. For the final blog post, we’ll take a look at how easily machine learning systems make terrible mistakes, how science fiction has imagined those mistakes being catastrophic when made by powerful machines, and how Aristotle’s virtue ethics might offer a solution.
References:
Kant, Immauel. Foundations of the Metaphysics of morals. Trans. Lewis White Beck. Prentice Hall, 1997. 2nd edition.
Mill, John Stuart. Utilitarianism 7th edition. Accessed from Project Gutenberg, https://www.gutenberg.org/cache/epub/11224/pg11224-images.html
Sartre, Jean-Paul. Existentialism and human emotions. New York, Philosophical Library, 1957.
