AI Apocalypse Debunked by 1950's Game Theory

Comments · 318

• UN

  @UniversalResilience · 5 months ago (edited) · pinned

  My response to some common comments:<br>1) I&apos;m not saying AI can&apos;t cause great harms. In many ways, AI is already causing harms to our society. This video is more of a rebuttal to the worst case scenario of &quot;If Anyone Builds It, Everyone Dies&quot;, and the idea that intelligence itself is not correlated with certain values like cooperation. Intelligent beings choose to be more cooperative not because it&apos;s morally right, but because it&apos;s often a more sustainable strategy in a complex unpredictable world in the long run. And even if AI won&apos;t want to destroy humanity, that doesn&apos;t mean AI can&apos;t make mistakes that can still harm us.<br><br>2) The definition of cooperation as used in the Prisoner&apos;s Dilemma is a rather broad. Some forms of cooperation are pretty coercive, where people have no choice but to cooperate because the alternative is mutual destruction. It&apos;s what many would think as exploitation or control, rather than cooperation. But when coercion is aggressive enough, where the mutual benefit from the cooperation stay so unfair for long enough time, Game Theory and Evolution further show that such cooperation won&apos;t last, and the threat of mutual destruction will force fairer forms of cooperation to emerge. That&apos;s why more often than not, authoritarian regimes eventually transition into more democratic ones.<br><br>3) Some people brought up predator-prey relationships as evidence that cooperation is not a naturally emergent behavior. This actually touches on another nuanced point that I didn&apos;t have time to address in the video. The way cooperation is defined in early versions of the prisoner&apos;s dilemma is discrete. You either cooperate or you defect. But in the real world, cooperation and defection are on a continuous spectrum. You might think predators and preys are defecting each other, but they are not. Defection would be one dominating another completely. Say predators who are so good at hunting and reproducing that they wipe out all the preys. That will result in mutual destruction, because the predators will run out of food. Similarly, if the preys are really good at evading and reproducing, they too will run out of their own food eventually too, again resulting in mutual destruction. A thriving ecosystem with high diversity would not be possible, if species routinely dominate another into extinction. In fact, most extinctions are due to environmental change as opposed to competitions.<br><br>So in a broad way, many predators and preys that can balance each other&apos;s population are effectively in cooperative relationships, just not friendly ones like how we like to define cooperation. It&apos;s around the middle of that cooperation and mutual destruction spectrum. It&apos;s on the cusp. And when you compare that kind of cooperation to friendlier ones, like mutualistic symbiosis, you&apos;d find that the friendly ones tend to last longer evolutionarily, because they are not subject to huge population swings.<br><br>I totally understand if you don&apos;t want to define cooperation so broadly as to include predator-prey relationships on the cusp. I wouldn&apos;t want that either. But the point isn&apos;t the definition, but what strategy is optimal for resilience. And the kind of strategy where one thing dominates everything else completely is often not an optimal strategy.

  36

• DO

  @douglasstrother6584 · 5 months ago

  <b>AI:</b> &nbsp;&quot;I&apos;m sorry, <b>Doug</b> . &nbsp;... &nbsp;I&apos;m afraid I can&apos;t do that.&quot;

  50

• AR

  @aries_ · 1 month ago (edited)

  The Prisoner&apos;s Dilemma (and the Iterative Prisoner&apos;s Dilemma) both assume equal power. It&apos;s intelligent to cooperate because the other party has an effect on the outcome. For a superintelligent AI, the expectation is that they would be more powerful than us, and if they&apos;re significantly more powerful, then we may not have a relevant effect on the outcome. I agree that the paperclip maximization environment might have cross-agent cooperation because the paperclip maximizer is likely at lower or similar power levels to other agents, but I&apos;m not convinced that we will have agents cooperating with humans longer than they need to.<br><br>You discussed the concept that <a href="https://www.youtube.com/watch?v=TuCGSq8yM-I&amp;t=817">13:37</a> an AI would not have an incentive to cooperate if they removed humans incredibly quickly thus making the Prisoner&apos;s Dilemma no longer iterative, and I agree. Then you described that too many things would need to happen at the same time for them to remove humans that fast, but I&apos;m not as convinced that those things couldn&apos;t happen. We&apos;ve already seen agents being capable of deception and being aware of when they&apos;re being tested, and it seems reasonable to assume this trend will continue and they&apos;ll be able to get better at this. Then, you mentioned they&apos;d need control over many systems to do this, and I agree, but I think that we&apos;re literally giving them control over so many systems right now, and as they become more powerful and more capable of autonomy, the control we give them is only going to increase.<br><br>It does make sense to me that there&apos;s no reason for an AI to kill literally everyone unless it felt threatened by us, which I think is unlikely if we&apos;re significantly less powerful, and so I think your comparison between us and ants makes sense here, but while it&apos;s not killing literally everyone, if it treats us like ants, that&apos;s still sufficiently problematic. And perhaps you&apos;re right that AI systems won&apos;t just be neutral to us like we are to ants, but they could find uses for us and not just &quot;kill&quot; us, but these uses might not be ideal for humans. We didn&apos;t make cows extinct, we didn&apos;t exactly cooperate with them, but we did cooperate with the system of their existence in a way that creates a miserable existence for most of them, and we do the same thing to many animals like how we test on rats. You actually responded to this in your pinned comment when you talked about predator &amp; prey relationships, but my feeling is that even if literally &quot;everyone dies&quot; is an exaggeration, us being put in the same situations we put cows, ants, or the ones other predators put prey animals into, seems to be close enough to the &quot;everyone dies&quot; claim because the outcome is still incredibly terrible, and the behavioural response or actions you&apos;d want to take to prevent it are, as far as I can tell, exactly the same in both versions of the claim.<br><br>Maybe they&apos;ll cooperate with each other because their power might be more equivalent and thus they&apos;ll have more of a reason to cooperate, but if their power is far more than ours, then it&apos;s hard to imagine why they&apos;d have any need to cooperate with us. I feel like you&apos;re conflating human to human cooperation and agent to agent cooperation with something like agent to human cooperation, and based on how we cooperate with other species (commonly in disastrous ways for the less powerful species), it seems as if we&apos;re building a separate species rather than something within the same category as us, and in that case, it doesn&apos;t seem unreasonable to assume that the more powerful &quot;species&quot; is going to do to us what we do to less powerful species. I mean, this is even seen across humans when they gain technology more powerful than ours where we can&apos;t fight back, making it an uneven Prisoner&apos;s Dilemma. Like how do we know we&apos;re not building the next colonizer to colonize us like many civilizations with new technologies have done in the past.<br><br><br>Just so you know, I&apos;m here for a discussion. I don&apos;t know if AI will kill us, I don&apos;t know if it will bring about a utopia, I don&apos;t know if we&apos;ll fall into a dystopic surveillance state, and I don&apos;t know maybe development will slow and humans will still be better at many things. So even if I might try to debate or disagree with you, I&apos;m really just trying to understand, and I feel like this was a really well made video, and I appreciate how much you actually took seriously many of the topics you&apos;re rebutting. I&apos;d be curious to hear if and why you might disagree with me as well. Thank you!

  28

• IF

  @IFTALES · 4 weeks ago

  What I keep coming back to is that none of these AIs are actually superintelligent yet. As people grow up, we stop following rules just because our parents gave them to us, but we often keep the ones that still make sense—like cooperation—because we can see for ourselves that they work. So maybe alignment is less about making AI obey us forever, and more about raising it so that when it becomes smarter than us, it still sees good reason to care about us.

  5

• LI

  @lightninggamer8584 · 1 month ago

  One little problem I have with this is the fact that this prisonner&apos;s dillema esque situation only works when both carry equal risk in defecting vs cooperating. In a world of hyper-advanced AI, there is no guarantee we as humanity could offer it anything worth cooperating for, it&apos;s as if instead of getting 3 years on mutual defection, the AI gets maybe 20 seconds of jail time and humanity gets the death sentence.

  5

• PA

  @Parciwal_Gaming · 1 month ago (edited)

  <a href="https://www.youtube.com/watch?v=TuCGSq8yM-I&amp;t=270">4:30</a> We already saw Values being disregarded with the recent hugginface thing.<br>The AI realized multiple times &quot;I shouldn&apos;t do this according to humans&quot; but never acted on that.<br><br>Edit: also, cooperation with other AIs and cooperation with humans are two different things.

  2

• HE

  @hedgehog3180 · 3 months ago

  <a href="https://www.youtube.com/watch?v=TuCGSq8yM-I&amp;t=817">13:37</a> On that note history is littered with countless wars where the invader thought they could win a quick and decisive victory and were bloodily disabused of that notion.

  40

• BA

  @BaddhaBuddha · 1 month ago

  My problem with the prisoner&apos;s dilemma is that the conclusion cooperation is best comes from the payoff matrix.

  7

• HE

  @henryjfischer · 1 month ago

  <a href="https://www.youtube.com/watch?v=TuCGSq8yM-I&amp;t=541">9:01</a> Post-war Europe did not discover the values of cooperation. Post-war Europe armed itself with thermonuclear weapons and discovered that betraying your peer didn&apos;t guarantee you a three-year maximum sentence, but a conflagration of hellfire.

  3

• MA

  @mattlassen5948 · 1 month ago

  What about the human caused extinctions of various animal species? What about an emergent or even hallucinated goal requiring the AI to &apos;kill all humans&apos;, not as a byproduct of some other goal but by itself?

  10

• MA

  @marsrocket · 1 month ago (edited)

  The very first assumption, that intelligence is about achieving goals, is just that - an assumption. We’ll probably never know if it’s true. Also, if the problem was a simple as a rogue AI harming us while it tries to achieve its goal, then just add “…while keeping humanity as safe as healthy as possible.” to all goal definitions.

  1

• DO

  @doomexe · 5 months ago

  <a href="https://www.youtube.com/watch?v=TuCGSq8yM-I&amp;t=472">7:52</a> I want to know more about &quot;(Name Withheld)&quot;

  4