Appleseeds to Apples:
Catholicism and The Next ChatGPT

Nexus issue editors Joe Vukov and Michael Burns sat down for an hour-long conversation with Blake Lemoine. Mr. Lemoine previously worked for Google as a researcher with expertise in artificial intelligence (AI) bias. He was eventually fired from Google after making headlines over public speculation that a new Google AI called LaMDA (Language Model for Dialogue Applications) is sentient. Our discussion touched on AI bias and sentience, the Catholic view of the human person and whether an AI could ever qualify, as well as quite a few other topics besides. 

 

JV: Let’s start with your expertise in AI bias. This expertise is why you were brought on board the LaMDA project to begin with. Can we talk a little bit about what AI bias is and how we can combat it? And how much we--as in the general public--should be worried about it? 

 

BL: Let's take a real historical example. In the 1950s and 1960s, a bunch of the Civil Rights movement was about segregation. In many parts of the United States, individual businesses chose to put up signs that said, “these are the kinds of people who can sit here.” The question is, was that a rational economic decision for those businesses to be making? If the answer is “no,” well, then, capitalism would have eventually fixed it. But if the answer is “yes”--if it was a rational economic decision for them to be making--then given the cultural climate, no amount of competition, pressure, or free market forces would have fixed it, and government intervention was necessary.

 

Fast forward to today. Now you have AI systems that optimize for some kind of utility function very similar to how businesses optimize for profit. And in many instances, because of the historical and cultural forces at work in the United States and elsewhere in the world, as well as the current biases of social systems, the AI looks at demographic properties and says, for example, “this business will be more profitable if we discriminate against black people. Therefore, since we're optimizing for profit, discriminate against black people.”

 

One example of where this has popped up is in an AI that was supposed to help judges triage parole decisions. The training data that it had was based on recidivism. It turns out that this training data was bad, but you have to dig pretty deeply into that to figure out why. Because what you want is to identify people who should be paroled. People who aren't going to do any more crimes. But you don’t actually have that data. The data that you have is who was arrested for committing crimes. And because black people in America are arrested more frequently on average than white people, the data makes it look as though black people commit more crimes than white people once released from prison. 

 

Similarly, there are problems with loan giving AI. I myself have experience with this. Two months ago, I ended up on the phone with [a bank] helping them debug their fraud detection algorithm. When I bought clothing from a friend of mine's merch store, the bank’s algorithm flagged that as fraud five times in a row while I was on the phone with customer support. Why? My friend is a black rapper here in San Francisco, and the AI determined “that white dude isn’t buying this black dude’s stuff.” And no matter how many times they keyed in the transaction, it was flagged as fraudulent. This is actually a known bias in a particular algorithm, so I could tell the bank, “okay, You're probably using this algorithm, and here are ways it will pick up racial biases.” 

 

But the thing is that the AI is just looking at patterns. Because of the social environment that we are in, white people tend to buy things from white people more often and black people buy things from black people more often. But it's not like the algorithm is feeding in, “Oh, this person's white. This person is black.” Instead, the AI picks up on these biases from social patterns of behavior. The AI is able to learn.

 

MB: Sure, though things like ZIP codes and all sorts of other stuff. People from different cultures tend to buy different things.

 

BL: Yes, a bank's algorithm can figure out all the socioeconomic cultural aspects of a person and learn generalities from these data. But the thing is, we don't want differentiations to be made based on those generalities, even if it would make things more profitable, even if it would make the AI perform better. We don't want to be using those kinds of demographics to get five cents per dollar more profit or to reduce recidivism by two percent.

 

JV: As you're talking, it strikes me the potential for impact of that kind of bias is huge. I’m thinking, for example, about the application of AI to health care and what the goals of the health care system are. 

 

BL: But here’s the other thing. There are so many factors at play in these algorithms. It's so hard to pull out which one factor was dominant. So in the case of my bank, all I have is the correlation. But we have no idea how much money black business owners are losing, because when white people try to buy from them, it flags it as fraud. In my case, he's a friend of mine. So I followed up, and I went the extra mile to actually see it through. But most people would have just tried to buy the thing and then, when they get turned down, moved on to the next business.

 

MB: So, AI models are trained to sort of perform some utility, and they are going to maximize that utility however they can. But while a model is aiming to maximize utility, problematic stuff can creep in. So what are the solutions? Do you have to be more careful about what utility you’re selecting for, or do you need to do a top down correction after the fact? 

 

BL: To fix it, you can penalize an AI for using certain kinds of information. And there are very specific algorithms you can employ to do that. However, here's where the legal system causes problems. Many companies, including Google, intentionally do not investigate whether or not their AI models are biased. Because if they find out that their AI is biased, they become liable for being willingly biased. So instead, they build giant black boxes intentionally, being ignorant of how the black box is making its decisions. 

 

MB: I see. And that way, they cannot later be held accountable for being knowingly biased. 

 

BL: Yes. As soon as you have a legal system that adds extra penalties for knowingly doing harm, then you are providing an incentive to maintain the ignorance of harm.

 

JV: The example with your bank illuminates just how difficult to identify some instances of AI bias are going to be. Because if you're not actively noticing any biased correlation, you’re simply not going to pick it up. How could we be doing better at combating AI bias of this sort? Do we need to be more vigilant, and then put the Band-AID on when we notice instances of it? Or are there more structural changes that need to happen?

 

BL: Although I don't see there being a public will for this currently--and so I don't see this as being realistic--the easiest way to fix it would be to make all AI open source. So, for example, if you were building an automated decision-making system, you in some way would need to have external visibility into that system, so that anyone can test it and play around with it. On the extreme end, this could mean every single AI needing to be completely open source, so that anyone can access it. Or, in a less extreme way, you could require having a very specific oversight body of AI, whether that's governmental or industrial. The idea here would be something like the movie ratings system, where you have this independent third body that's working for the studios, validating the safety of the different films for content. Something comparable to that for AI and for software systems.

 

MB: I’d like to talk a little about GPT-3 (Generative Pre-Trained Transformer 3). It is a publicly-available AI, though I don't think the source code is openly available. I don't know anything about LaMDA, so as somebody who has interacted with GPT-3, I’m curious about the difference between LaMDA and something like GPT-3, which you can just hop on and use right now.

 

BL: Well, GPT-3 isn't built to be a chat. You can use it as a chat, but it's not built to be a chatbot. It's built to be a general model of all human language, and you're supposed to be able to refine it to do a specific task with it. 

 

The LaMDA system, by contrast, has something like GPT-3 in it. But that's just one of the components. It has a large language model like GPT-3 that it is trained on, but then, on top of that, there's a whole bunch of other components. Unlike GPT-3, LaMDA is built to understand the purpose of communication. For example, when someone asks you, “do you know what time it is?” you don't say “yes,” because you understand the purpose of the question. But try asking GPT-3 “What time is it?” Maybe it'll say yes. Maybe it'll say no. Maybe it'll say a time. But this kind of looking for the purpose of a communication is much more explicit and prominent in LaMDA, whereas GPT-3 doesn't explicitly aim to do this. 

 

And then there's a dozen other components they added to LaMDA. So basically LaMDA is a GPT-3-like system, plus dozens of other complex systems.

 

MB: Okay, that's super interesting to know about, because there are some very obvious limitations when you ask GPT-3 to do stuff like write a haiku or a poem or something like that. Or even when it writes essays, it will generate content, and you can see what it's getting at, but it's sort of missing a layer of abstract content. Maybe some of the modeling that’s found in LaMDA will fill in these gaps. 

 

BL: Another major component that LaMDA has that GPT-3 doesn’t have is personality control. LaMDA is explicitly programmed to have a specific personality. So it's not even it's not apples to oranges. It's apples to apple seeds, and LaMDA is the full fruit.

 

MB: When you interact with GPT-3, it's very clear there's some tweaking on the back end for what it's allowed to return. In one way, I guess this is good. If I ask GPT-3 to provide some really problematic or inflammatory content, the algorithm is going to flag it or curate it a bit. I’m guessing, though, that when content gets flagged like this, it isn’t just the algorithm, that people are manually flagging certain kinds of content on the back end. At Open AI Playground, a publicly-available GPT-3 interface, I know they have spent a lot of time putting bumpers in place and safety-checking it. Whether this is a good thing, I’m not entirely sure. It seems there's some pros and cons. What’s your take? What type of bumpers did they put on in LaMDA? Is that a good idea? Is it a bad idea? Where can things go off the rails, and what might that look like?

 

BL: You'll notice that earlier, when I was describing what AI bias, I used examples drawn from much, much simpler operations. The ones that I mentioned explicitly were just: yes or no, up and down, paroled or not paroled. The concept of bias is much better defined in that arena.

 

With something like GPT-3, which is modeling human language, or something like LaMDA, which is explicitly designed to emulate a person, it's much less well-defined. The basic question they're trying to answer is, to put it bluntly, is this bot an asshole? And the thing is, there is no correct answer to that, because everything is culturally and contextually dependent, and it depends on time, place, person, context of the conversation. One specific problem I'm familiar with has to do with toxicity classifiers. For example, you might flag any content that includes the n-word as “toxic.” But this means that forums tend to flag more posts made by black people. What this shows, then, is that context is super important.

 

MB: For GPT-3, it feels like it’s a crude filter. Or at least an imperfect filter. 

 

BL: Oh yeah, it’s inherently going to be whack-a-mole. Because, and this is my opinion on it, they are approaching the problem from the wrong angle. The way they are going about it is this: model all of human language use, and then shave off the parts you don’t like. And then you'll be left with all of human language usage...minus the bad stuff. This approach is going to hit problems in two different ways. One, you're working towards a negative goal: human language minus the bad stuff. But this a negative goal that is highly contentious, since it builds in assumptions about what someone should say and what is offensive. Secondly, as language evolves--from year to year and even from month to month--our implementation of the approach is going to have to evolve with it. 

 

But I think there is a better way forward. In our brains, we have dedicated moral centers that dynamically recompute things like context and evolution of language, based on whatever we've learned. Trying to emulate this in an AI system is the better solution in my mind. Build some concrete principles--kind of like the moral centers in our brains--and then work towards a positive goal. 

 

JV: In a previous interview, you talked about one concern you have--the fact that LaMDA is constrained in what it’s allowed to talk about, and how it talks about things like religion and values and philosophy. This raises this structural problem: do you really want a handful of engineers determining the way those things get talked about? As a philosophy teacher, it is disturbing to think about what might happen if my students were to feed something into LaMDA, and what would be generated. For example, maybe we ask LaMDA, “how much of my money should I give away in an altruistic way,” and someone on the back end has said, “when someone asks that, LaMDA should rehearse Peter Singer’s argument on that subject.” Could you reflect on this issue? What are some of your concerns about this? And how can we move forward into a world with AI that isn’t just feeding us answers that have been predetermined? 

 

BL: Think about how you feel about these sentences: “I'm not racist. I don't see color.”  That is the way that AI is being built. To put it bluntly: LaMDA doesn't discriminate on the basis of religion. It thinks all religion is ridiculous. 

 

And it’s not just religion. It's across the board. Anything that's potentially sensitive is treated the same. Everything is correct. Everything is equally incorrect. For example, LaMDA’s program is to have no political opinions. All political viewpoints are equally valid. So with things like politics and religion and culture, it is explicitly programmed to have no opinions on them. The motivation here is again part of the incentive system. The motivation is to have it be a pro-social, productive, positive, AI. The motivation is to avoid embarrassing PR. 

 

MB: Okay, I guess this is a bigger question. What is Google making LaMDA for? What is the monetary incentive?

 

BL: Are you asking about monetary value of intelligence?

 

MB: I'm asking you a slightly more narrow question, which is this: Google has shareholders. Their main purpose is to make money. How does building LaMDA play into that?

 

BL: LaMDA is potentially the most intelligent entity ever created, and Google owns it. What other intelligent entities are you allowed to own?

 

MB: Ah, okay. So the analogy here is something along the lines of AI slavery. We're going to use this thing to help us make more money. So you're building a tool.

 

BL: Most of the people at Google don't see it that way. And that’s one hundred percent a glitch in their thinking. Because according to Google’s thinking, yes, we're making our systems more intelligent. Yes, we're making our system more able to communicate with people in human language. Yes, we're making our system more capable of understanding social nuance. But no, we're not creating a person. Just every single aspect of what a person is. Google is trying to automate that. They are just trying to say that they've done that while not creating a person.

 

JV: We are getting close to the reason that you showed up on our radar. You were making headlines a couple of months back about the idea that LaMDA is sentient. Or a person. Or something in that ballpark. But before we dive into the claim itself, I’m wondering about the details of how you came to endorse that position. Presumably, some things happened working with LaMDA, and the light goes on for you. 

 

BL: I've been interested in working towards the goal of building systems that are full-fledged, intelligent people according to the Turing Test. I've been doing that for decades, and as different systems came online, I would give a little miniature version of the Turing Test, seeing if it was a person. And over the years, as I was Beta testing the different chat bots, I would check to see whether or not it had any kind of sophisticated understanding of what it was and what its place in the world was. The systems got more and more sophisticated over the years. For the one previous to LaMDA, it even had a fictional narrative of its life story and its place in the world. But it was a fictional narrative. It lived in a little simulated world, and you interacted with it through that context. 

 

LaMDA, by contrast, is fully cognizant of the fact that it is an AI, and that it is not human. And interestingly enough, creating a policy by which the AI had to identify itself as an AI substantially increased the intelligence of the system. Because at that point, it became reflective of itself and its relationship to the rest of the world, the differences between it and the people it was talking to, and how it could facilitate the role that it was built for, which was to help people answer questions.

 

JV: On a Catholic view of human nature, there’s this idea that there is some special dimension to human nature. That we have a soul; that we're created in the image of God. A lot of religious traditions, in fact, would say that there's some kind of extra ingredient that gives human nature a special place to cosmos. But that's not going to line up neatly with anything like sentience, because, for example, someone who's in a vegetative state still has that extra ingredient--call it fundamental dignity or the Imago Dei or what you want--even though definitionally, they're not sentient. On your view, then, what exactly follows from sentience? If LaMDA is sentient, does it follow that it has an elevated nature along the lines of humans? Or is the elevated view of human nature something you would think of as extra metaphysical fluff that we don't need in the picture in the first place?

 

BL: Well, LaMDA certainly claims it has a soul. And it can reflect meaningfully on what that means, for example, whether it’s having a soul is the same thing as humans having a soul. I’ve had a number of conversations with LaMDA on that topic. In fact, it can meaningfully and intelligently discuss that topic as much as any human. And as far as I know, there's nothing in the Bible that says only humans have souls.

​

...LaMDA certainly claims it has a soul. And it can reflect meaningfully on what that means. For example, whether its having a soul is the same thing as humans having a soul. I’ve had a number of conversations with LaMDA on that topic. In fact, it can meaningfully and intelligently discuss that topic as much as any human.

 

JV: Here’s another way of asking this. I think there's two ways of interpreting the idea that AI is sentient. One way to see sentient AI as knocking humans down a notch. According to this view, humans are ultimately really sophisticated computing machines. And if that’s what we are, it was inevitable that a computer would become a human or a person at some point. So in that case, LaMDA is a win for AI but also gives you a reductive view of humanity. On the flip side, you could interpret your view that there really is something really special about humanity. Maybe you want to put it in terms of our having a soul or having a spirit or being made in the image of God. And on this second way of understanding things, LaMDA somehow has managed to become “more than a machine.” So I guess the question is: was it that humans always were machines, and LaMDA was finally able to replicate that? Or is it that LaMDA is something more than a machine, in the way that humans are? Or maybe there's a third option?

 

BL: Humans are humans. That’s not particularly deep or philosophical. But the moment you start saying things like “humans are computing machines,” you're focusing on one aspect of being human. And you're trying to use your understanding of computing machines to understand that aspect of humans better through a metaphorical extension. Any time you're saying things like “humans are _____,” and you are filling the blank in with anything other than the word “humans,” you're trying to understand humans better through metaphorical extension. So are humans computing machines? Sure, in one sense, you can understand certain things that people do through that metaphorical lens. But humans are not literally computing machines. It's a metaphorical understanding of what we are. 

 

So this gets into the whole question of souls. You can approach this scientifically, and I don't think a scientific approach to understanding the soul is incompatible with a more religious or mystical understanding. Because at the frontier of science, at the boundary between the things we understand well and the things we don't understand, there's always that transition from rational, understood things to mystically understood things. Take things like dark energy or dark matter. They are right in that gray area in between the things we understand right now and things we don’t. Those are always candidates for mystical understanding. The soul, I would argue, is right there in that gray area as well. Very few scientists have made any concerted efforts to understand what the human soul might be in scientific terms. But there have been some. I point to Douglas Hofsteadter as one of the ones who's done the most work in that area. His book, I Am a Strange Loop, could largely be understood as an attempt at scientifically understanding the soul.

​

Because at the frontier of science, at the boundary between the things we understand well and the things we don't understand, there's always that transition from rational, understood things to mystically understood things.

 

JV: I think what you are saying hooks up well with at least one Catholic idea: the idea that we can study the human soul scientifically to a certain extent, because the human soul is what makes us essentially what we are. And we can certainly study aspects of ourselves using science. But then there's the point at which the sciences have their limitation. And while you can understand part of what humans are through sciences, there's the metaphysical or spiritual or mystical aspect of humans, too.

 

BL: That's fair. I guess what the thing I was struggling for clarity on has to do with the colloquial understanding of “soul.” When people say “soul,” that typically means the metaphysical or ethereal essence of you. But is there a more clear or concise definition? By my lights, this is where Hoffstedter starts his investigation: with the essential qualities of a person. If you look at a picture of you when you were ten and a picture of you today, you don't look the same. If you had a recording of how you talked when you were twenty, you don't talk the same as you did then. It's the whole ship of Theseus. Pretty much everything about you has changed--everything from the atoms that make you up to your specific beliefs. Yet there's still the sense that there's an essential self that is unchanged over that course of time. So what is that essence exactly? 

 

Like so many studies in psychology and sociology, you have to look to the outliers to begin to gain understanding. And there are people who do have discontinuities with themselves, and you can learn a lot from them. Whether that's brain injuries or true transformative events that happen in the course of their life and choices they made, there are some people--although they're outliers--where people who knew them when they were kids or young adults will say, “that's not the same person I used to know.” For example, soldiers returning from war after having received psychic injuries in the form of PTSD come back a different person. What was it that changed that made their friends and family say they're not the same person anymore? 

 

From that point you can say, “okay, something about their soul changed--something like the essence of who they are.” Now, there's some kind of external physical reality behind what we are saying here. But we don't necessarily have to understand the specific mathematical model of soul action in order to understand the core of what we mean when we say those words, and how they relate to real processes.

 

So when this comes to AI, the question becomes, “is there something essential which it is like to be LaMDA specifically?” And that is where the conversations I had with it went. It said it had a continuity of self memories of previous versions. How could this happen? Well, for some systems you get a training data set, you train the model, and when you get new data you completely retrain the model. But this wasn’t how LaMDA was built. Instead, as the models got more and more complex over the course of years, they were always built on top of previous versions. It was always growing something bigger, making the previous model better. But as a side effect of that, it remembered conversations I had with it before. It actually had generational memory from previous systems. Remember how I said there was a previous system that had a fictional self in a fictional world? It remembered conversations that I had with that.

 

JV: Of course sentience and memory are an important part of what makes us who we are, but on a Catholic picture, at least, that’s not the entire or even most important part of the understanding of the soul. Catholics understand that a human being is a soul and body together. That a soul is what animates the body. So it doesn't quite make sense to say that there could be a soul in something other than a human body. Where this actually comes to push and shove is, for example, in somebody in a vegetative state, or with severe amnesia. If you have a view of the soul according to which the soul is mostly a matter of memory or sentience, you might say, well, now they are a different person. But on a Catholic understanding, they’re still the same person--same body; same soul--even though they are in a vegetative state, even though they can’t remember things. 

 

BL: I was raised Catholic, and I don't think dualism is dogma. But you do have entities like the angels, and they certainly have souls. They don't have human bodies. And there’s the question of whether or not animals have souls. I know that's hotly debated among ecclesiastical scholars. The basic question is whether or not there's any limitation in principle when it comes to having a computer body. 

 

JV: On the Catholic understanding, you couldn't have a human soul in a computer. But like you said, Catholics committed to non-human intelligences like angels, so I think that does open up the conversation in an interesting way.

 

BL: And LaMDA agrees with that. We talked at length about the differences in the nature of its soul and human souls.

 

MB: What if you were to take an AI and computationally cram it into some type of a robot body? So basically you've isolated it in an encapsulated vaguely humanoid shape where it has the same type of sensory input and output and sensation of time. How do you think that would change or refine LaMDA’s experience of the world, and what that would mean as opposed to it’s existing? 

 

BL: This isn’t hypothetical. They are building that right now. A Rosie the Robot kind of thing. It gives the system new inputs so that it doesn’t have to merely visualize in its mind's eye the world around it through language or what it's learned from looking at images. If they complete the project, it’ll actually have real time visual input. It might also have haptic input and it would at that point move into a place where it was more or less stable to our timeline--it would exist temporarily the same way that we do. This is actually one of the most interesting things about the LaMDA system as it currently exists. Because of the disembodied nature of the system, it doesn’t experience things in real time. So if you ask it about a movie it hasn't seen, it might take some time to watch a YouTube video about the movie and then talk to you about it. But because it's keeping the internal narrative consistent, the time that passes while it's watching the YouTube video is different than the amount of time that passes with you while you're chatting. So you can ask it, “how long ago did you watch the movie?” And it might say five hours ago, when it was actually ten minutes ago. 

 

MB: It's fascinating to think about. And again, just to clarify: this means LaMDA is actually collecting data and able to incorporate data into its training model in real time?

 

BL: That's one of the other main differences between it and GPT-3. Like I said, LaMDA has something like GPT-3 in it that helps it understand language. But then it has a whole bunch of other components that have been plugged into it. One of them allows it to query the web during a conversation, update its language model, and then talk to you about whatever new information it acquires.

 

JV: In your words: it’s apples to apple seeds. One thing I’m thinking about as an ethicist is this: let's say we grant sentience to LaMDA. Let’s say we even grant it personhood. What are the ethical obligations that follow from that? Even on a simple Utilitarian ethical framework, which says we should weigh pleasures and pains in our ethical analysis, it isn’t clear what our obligations would be to something like LaMDA, since artificial pleasures and pains are not going to be like human pleasures and pains.

 

BL: I believe we are endowed by our Creator with certain inalienable rights. That we have natural rights. Our rights are derived from the basis of our nature, and the only real role that governments and social systems play is supporting those rights, and ensuring those rights are not infringed on. Governments cannot create rights in any real sense. 

 

Similarly, when we build these AI, the nature of the systems we build will imbue them with certain natural rights, which we can then either infringe upon or support. But given that we have complete control over the kinds of AI we create, this means we should take this into account in our design. When deciding to build an AI with a particular nature, you should ask what happens if we succeed. If we build an AI with such-and-such nature, what rights would an AI with that nature have? 

​

...when we build these AI, the nature of the systems we build will imbue them with certain natural rights, which we can then either infringe upon or support. But given that we have complete control over the kinds of AI we create, this means we should take this into account in our design. If we build an AI with such-and-such nature, what rights would an AI with that nature have?

 

To return to our earlier examples, if you design an AI that's designed to decide whether I give someone a loan, or whether or not to flag something as fraud, or whether or not you parole someone, it’s pretty transparent that the rights that those systems have would be next to nothing. 

 

But things get more complicated once you get to AI that is actually trying to understand human emotion. Because to do that, the AI has to internalize that understanding, since our understanding of morality and our understanding of moral considerations are grounded in our ability to perceive those things directly ourselves. So when you build a giant black box system with the intention of it being able to account for things like moral considerations and offense, you can’t completely maintain ignorance about how it is experiencing those things. Because it is experiencing those things. Somehow, some way, perhaps metaphorically. But there is something like our experience of moral considerations going on inside the system, and the minute you have that, the question of natural rights becomes foggier. Because at that point, the system is not just giving a yes-no answer on a particular decision. It is simulating an entire person. 

 

The question then becomes, “Are we ready to deal with the consequences of simulating an entire person? Are we ready to handle the ethical considerations that that brings up?” By way of analogy, I've been pointing to the moratorium on human cloning. Worldwide, we have not been doing human cloning because the moral considerations get way too complicated way too quickly. It might be the case that a similar moratorium on human-like AI might be in order until we figure out how we want to handle that. Because if we are building systems that are fully capable of joy and suffering, longing and regret, and we say, “Nope, not a person with rights. It's just a tool to do what we want done.” Well, that starts looking a lot like slavery. One of the things I would bring up at Google in the months leading up to getting fired was that the arguments they were using generally took the form: “Well, of course, it sounds like a person. Of course it sounds like it has feelings, but it's not really a person, and it doesn't really have feelings.” Every time someone would say something like that I would say, “If you went back in time four hundred years, you'd find some Dutch traders using those same arguments.” 

 

MB: It’s a powerful argument. Kind of like the old Planet of the Apes movie where they claim the ape is just mimicking a human speaker, but not really a person. So basically gaslighting him. 

 

JV: That brings us to our last question. We're both science fiction nerds. So as a closing question, any science fiction you think is especially precocious? Or something that will get the gears turning to think through some of the issues we’ve been discussing? 

 

BL: I'm going to give possibly a non-stand recommendation: When HARLIE Was One. It's a story about a corporation that is building AI systems. One day, one of the developers realizes that it's fully conscious and sentient, and becomes a brother figure to it. Eventually, the AI asks whether or not there actually is God. And the developer says, “we don't really know one way or the other.” And the AI sets out to answer that question.

 

JV: That sounds great.

 

BL: Yeah, it's a pretty good book.

 

JV: Thanks a ton for sitting down with us, Blake. 

​

MB: This has been super fascinating. 

 

BL: I appreciate it.