On Infinite Ethics
A huge problem that seems to threaten basically everything in moral philosophy
A few months ago, I wrote a piece about moral uncertainty, and its implications for Longtermism. In the piece, I quoted the philosopher Will MacCaskill, who says:
“Supposing we really do want to take moral uncertainty under account [in our decision making], how should we do that? In particular, it seems like given the obvious analogy with decision making under empirical uncertainty, we should do something like expected value reasoning where we look at a probability that we assign to all sorts of different moral views, and then we look at how good or bad would this action be under all of those different moral views. Then, we take the best compromise among them, which seem to be given by the expected value under those different moral views.”
As I said then, this seems pretty good at first! There are a ton of things left to work out, but applying the logic of expected value to moral uncertainty makes a lot of sense at first blush.
But as I wrote in a footnote:
Questions of infinite ethics really mess up this meta-ethical expected value calculation, as do questions of incomparability. MacCaskill goes into all of that in his interview, and I’ll do another post about it soon. But for now, we’ll leave it aside.
This is that post.
What is infinite ethics, and why does it break things?
To explain the basics of infinite ethics, I’ll give MacCaskill (and his interviewer, Rob Wiblin), the floor again. He says:
MacCaskill: [This meta-ethical expected value calculation leads to] a fanaticism problem, which is the worry that perhaps under moral uncertainty, what you ought to do is determined by really small credences in infinite amounts of value. Where perhaps absolutist views say that it’s absolutely wrong to tell a lie, no matter how great the consequences. Perhaps the way you’d want to represent that is by saying that telling a lie is of infinite wrongness, whereas saving lives or something isn’t. Then, you’ve got this decision, “I can tell a lie, but I’ll save 100 lives by doing so.” Let’s suppose you have a 1 in a million credence that the absolutist view is correct. Well, one in a million multiplied by infinite wrongness is still infinitely wrong, so telling the lie would still have lower expected choice worthiness than not telling the lie. That just seems crazy. That seems like what we ought to do is just dominated by these fringe fanatical seeming views.
Wiblin: I guess it could get even worse than that because you could have one view that says something is absolutely prohibited, and another one that says that the same thing is absolutely mandatory. Then you’ve got a completely undefined value for it.
MacAskill: Absolutely. That’s the correct way of thinking about this. The problem of having infinite amounts of wrongness or infinite amounts of rightness doesn’t act as an argument in favor of one moral view over another. It just breaks expected utility theory. Because [if you have] some probability of infinitely positive value, and some probability of infinitely negative value, you try to take the sum product over that you end up just with undefined expected value.
Now, we have a couple of paths we can take from here, none of which are particularly satisfactory.
Maybe infinite ethical values are impossible
One potential option is that we could just assume, as a matter of ontology, that it is impossible for a moral theory to consider something infinitely wrong, or infinitely right. Maybe, on some sort of metaphysical level, it is simply the case that moral theories can only ascribe finite ethical value (whether negative or positive) to an action.
This is a tempting response, because it would solve many of the philosophical problems at hand here.1 Unfortunately, it comes with two problems. For one thing, it’s really hard to come up with a compelling logical reason why it would be true. Lots of moral theories philosophers find at least plausible – strong Deontology, for one – have absolute prohibitions against actions. For me at least, it’s hard to see what the conceptual difference is between an absolute prohibition and an assigned value of negative infinity.
But the bigger problem is this: as long as you have a positive, finite, non-zero credence that it is possible for a moral theory to ascribe an infinite value to an action, then you’re left right back where you started. Think about it like this:
If you think there’s a .001% chance that it’s possible for an ethical theory to assign an infinite value to an action, and:
You assign a .001% chance to “infinitely strong deontology” being the correct ethical theory, then:
Your meta-ethical calculation on the moral value of, say, telling a lie – which is absolutely prohibited under strong deontology – will include a term that is equivalent to (.00001) x (.00001) x (negative infinity). This term will, of course, still equal negative infinity.
This means that your meta-ethical expected value calculation will still be, to use a a piece of philosophical jargon, completely fucked. If you only have one moral theory with infinite values, you’ll end up defaulting to all of its preferences, leading to the fanaticism problem MacCaskill describes. If you have multiple, you’ll likely end up with an undefined value for every action (the result of summing different probabilities of negative and positive infinities).This is highly unfortunate, because it means that sometimes (probably all the time) you won’t be able to use expected value to guide your decision making framework, because all actions have undefined expected values due to the conflicting infinities in our theoretical evaluations.
So unless you are literally certain that moral theories can’t include infinite values, all you’re doing by assigning a low probability to the possibility of infinite value is pushing the problem back one layer deeper. And I cannot, for the life of me, see why we would be justified in assigning a probability of 1 to infinite ethical values being impossible.
So what other options do we have? One potential answer is to decide that the meta-ethical expected value framework is simply not the right way to approach ethics. Instead, perhaps, we should simply pick a single theory, and support it to its logical conclusions.
How do we pick a theory? Well, on some level, who knows. But maybe we pick based on some combination of what seems conceptually elegant and intuitive.
This choice will obviously vary from person to person. In my case, the theory I find most conceptually appealing is, roughly speaking, aggregative consequentialism – and within the bounds of aggregative consequentialism, I’m (mostly) a total utilitarian.
But infinite ethics absolutely ruins total utilitarianism too, for a whole host of reasons. If you want to see why, I strongly recommend Joe Carlsmith’s essay on the subject (or work in a similar vein, by Nick Bostrom and Amanda Askell). Carlsmith’s essay is too long, complicated, and interesting for me to be able to summarize it well, but suffice it to say, collectively, Carlsmith, Askell, and Bostrom’s work convincingly demonstrates that infinite ethics crushes the dreams of truly bullet-biting total utilitarianism. This is pretty bad news for total utilitarian-leaning people like myself, and many of my Effective Altruist friends.
My understanding is that infinite ethics also causes problems for fanatic support of tons of other ethical theories. But personally, even if I found a theory that was perfectly compatible with infinite ethics on its own, I would have a hard time getting behind it fanatically anyway. That’s partly out of a belief that fanaticism is a bad way to approach moral philosophy, given all the considerations I mentioned in my piece on moral uncertainty. It’s also partly just out of a gut feeling that any philosophical framework that doesn’t include at least some sort of aggregative consequentialist element doesn’t make a whole lot of sense. I will freely admit that this second point is a lot less intellectually coherent than the first, but it resonates with me nevertheless.
There are other proposals in philosophy about what to do here (see this footnote for further discussion2), but none of them sit very well with me.
This is all interesting, but who cares?
As a general matter, most people don’t spend a lot of time thinking about moral philosophy. While I wish more people (especially affluent people, or those with lots of influence) would spend more time thinking about ethics, for the most part, this is fine. Arguing about philosophy isn’t for everyone, and lots of people have much more pressing concerns.
But even within moral philosophy, infinite ethics is not a well studied topic. Neither, for that matter, is moral uncertainty. This strikes me as a huge problem, because, well, there are literally infinite amounts of value potentially at stake in this conversation. Humans are scope insensitive to merely large, finite numbers, so there’s simply no way we can properly weigh the infinite. But we should probably try to! And even a tiny bit of scope sensitivity around infinity is enough to make you realize this is a massively important problem.
So what do we do?
The most honest answer to “so what do we do about this” is “I have essentially no ideas, and the fact that I have no ideas causes me a fair amount of distress on a semi-regular basis.” But here’s the little that I have come up with, after a lot of thought:
First, while solving infinite ethics seems completely intractable at the moment, it’s at least plausible (or possible) that that’s because few people have tried to solve it. (Or maybe, as an EA friend of mine once joked, it’s because we haven’t (yet) been able to turn a superintelligent AI loose on the problem.)
Let’s say you’re still set on choosing actions that maximize expected value under the meta-ethical framework MacCaskill laid out above. If you think that more resources dedicated to this philosophical problem might increase our odds of figuring it out by a finite, non-zero amount, and you think that there’s a finite, non-zero chance that figuring it out would be worth infinite value, than the most important thing to do is to increase the amount of resources (in the form of brain power) allocated to this branch of philosophy.
To be clear, that would not mean we should all quit our jobs and become moral philosophers. Funnily enough, I actually think the implications are basically just… Longtermism. It seems to me that the best way to maximize the amount of brain power allocated to infinite ethics is to ensure human civilization is flourishing and stable in the long run. One of the arguments Toby Ord makes for Longtermism in The Precipice is that evading our existential risks this century will enable us to undertake a “long reflection,” where, essentially, we’ll be able to better learn what the Good is.
Second, I think that the massive problems with infinite ethics perhaps point to this kind of moral philosophy being sort of farcical. Who are we to think we can figure out the Good with just logic, reason, and a bit of math? Maybe what we need is to think less about these issues with our head, and more about them with our guts, or with our hearts.
On that front, I do have strong opinions. As I wrote in “The Emotional Case for Longtermism,” – my first piece for Out of the Ordinary – my single most core belief is that if conscious life disappears from the universe, then nothing of value remains. I don’t feel like I need to know anything about how to resolve infinite ethics to know that. And again, the implications of this gut feeling are – in my view – simply Longtermism once more.
Centering myself on this gut feeling is generally enough to dispel the sense of existential paralysis I get every time I think about how infinite ethics destroys the most appealing approaches to moral philosophy.
But I sure wish someone would figure it out.
Thanks to Conor Downey for feedback on an earlier version of this piece.
Though even if infinite values are impossible, finite values still present fanaticism concerns to many – see this paper by Hayden Wilkinson for a discussion of the topic, and an argument in favor of fanaticism in non-infinite contexts.
One option is a social choice method, such as what’s usually referred to in the literature as "My Favorite Option," in which you essentially give all of your theories votes weighted by your credence, and then have the theories vote for the option they recommend in a particular decision context. This has the drawback of not allocating votes to theories proportionally to how intensely they feel about an action, which some people, including me, view as unintuitive.
Another option is to “maximize expected choice-worthiness” across moral theories, where the ‘choice-worthiness’ of an option under an individual theory represents the strength of reasons one has to choose an option. But as Will MacCaskill, Toby Ord, and Owen Cotton-Barratt explain in their invitingly titled paper “Statistical normalization methods in interpersonal and intertheoretic comparisons,” this leads to the difficulty of intertheoretic comparisons, which many philosophers have suggested are impossible. In the paper, the authors offer a few proposals for how to make these intertheoretic comparisons workable; I’ll confess that I don’t think I quite understand these proposals – and especially, I don’t see how they would solve the preference intensity questions raised by infinite ethics.
Did you read The Dawn of Everything? Seems the thesis is that we can re-think how [we] approach all notions*, including political and economic systems and that ideally would lead to a global justice movements. Strongly recommend if you have not.
“ Second, I think that the massive problems with infinite ethics perhaps point to this kind of moral philosophy being sort of farcical. Who are we to think we can figure out the Good with just logic, reason, and a bit of math? Maybe what we need is to think less about these issues with our head, and more about them with our guts, or with our hearts. ”
After going through 4 years of a philosophy degree and spending a lot of time reading/lurking in EA and rationalist spheres online, this is where I’ve landed. I think taking a hyper-logical, highly quantitative approach to ethical philosophy is misguided. Ethics is more like an art than a science, and so may never be boiled down to elegant, clear, general principles, or rid of tensions and fuzziness (certainly not any time soon). Quantitative reasoning can be instrumentally useful in some cases of applied ethics, such as when assessing the consequences of a policy, but that’s as far as it goes.