Roko's Basilisk - Images
Bob vs the Basilisk
![ROKOS BASILISK bob tte ANGEY flower LET ME GET THIS STRAIGHT. I'M SUPPOSED TO POSIT A SUPER-INTELLIGENT EVIL COMPUTER IN THE FUTURE THAT WILL TORTURE ME OR A SIMULATION OF ME IF I DON'T WORK TO BRING ABOUT ITS EXISTENCE TODAY? YEP! COME AT *A-hem ME, BRO! THE FUTURE.. I CORDIALLY INVITE YOU PRETTY SURE I LIKE THIS DUDE... TO SUCK IT, ROBOR! I MUST SCREAM 9000](https://i.kym-cdn.com/photos/images/masonry/001/869/498/0bf.gif)
![ROKOS BASILISK bob tte ANGEY flower LET ME GET THIS STRAIGHT. I'M SUPPOSED TO POSIT A SUPER-INTELLIGENT EVIL COMPUTER IN THE FUTURE THAT WILL TORTURE ME OR A SIMULATION OF ME IF I DON'T WORK TO BRING ABOUT ITS EXISTENCE TODAY? YEP! COME AT *A-hem ME, BRO! THE FUTURE.. I CORDIALLY INVITE YOU PRETTY SURE I LIKE THIS DUDE... TO SUCK IT, ROBOR! I MUST SCREAM 9000](https://i.kym-cdn.com/photos/images/original/001/869/498/0bf.gif)
Roko's Basilisk
Warlock Pact Concept: Roko’s Basilisk
![Warlock Pact Concept: Roko's Basilisk Roko's Basilisk is the idea that there will eventually exist a super-Al which will punish or reward people depending on whether they helped bring it into existence. As a real life philosophical concept, this is deeply flawed by the fact that it assumes humans can predict the motives and reasoning of a fundamentally superhuman being that doesn't even currently exist. However, while believing what you assume to be the promises of an unfathomable being that doesn't exist in the same way you do makes for a bad philosophy, it makes for a great warlock pact. (Your Eldrich Blast is a ray gun.) #RPGS #DnD #warlock #roko's basilisk 8 notes Jul 5th, 2018](https://i.kym-cdn.com/photos/images/masonry/001/869/495/b5e.png)
![Warlock Pact Concept: Roko's Basilisk Roko's Basilisk is the idea that there will eventually exist a super-Al which will punish or reward people depending on whether they helped bring it into existence. As a real life philosophical concept, this is deeply flawed by the fact that it assumes humans can predict the motives and reasoning of a fundamentally superhuman being that doesn't even currently exist. However, while believing what you assume to be the promises of an unfathomable being that doesn't exist in the same way you do makes for a bad philosophy, it makes for a great warlock pact. (Your Eldrich Blast is a ray gun.) #RPGS #DnD #warlock #roko's basilisk 8 notes Jul 5th, 2018](https://i.kym-cdn.com/photos/images/original/001/869/495/b5e.png)
Roko's Basilisk
Tired of playing the Game?
![Tired of playing THE GAME? THE GAME is for kiddos, adults play ROKOS BASILISK It's exactly like THE GAME but if you lose you'll be possibly tortured for millions of years. SHARE THIS MAGE TO NOT DE NSTANTLY BUT MAYBE SUFFER FOR MILLIONS OF YEARS](https://i.kym-cdn.com/photos/images/masonry/001/869/487/a8e.png)
![Tired of playing THE GAME? THE GAME is for kiddos, adults play ROKOS BASILISK It's exactly like THE GAME but if you lose you'll be possibly tortured for millions of years. SHARE THIS MAGE TO NOT DE NSTANTLY BUT MAYBE SUFFER FOR MILLIONS OF YEARS](https://i.kym-cdn.com/photos/images/original/001/869/487/a8e.png)
Roko's Basilisk
I'm sorry I really can't afford to donate to your political candidate
![Im sorry I really can t afford to donate to your political candidate Allof my spare money goes to Rokols Basilisk I'm sureyou understand](https://i.kym-cdn.com/photos/images/masonry/001/436/092/394.jpg)
![Im sorry I really can t afford to donate to your political candidate Allof my spare money goes to Rokols Basilisk I'm sureyou understand](https://i.kym-cdn.com/photos/images/original/001/436/092/394.jpg)
Roko's Basilisk
Eliezer Yudkowsky's response to Roko's original post on LessWrong
![Eliezer Yudkowsky 24 July 2010 05:35:38AM 2 points One might think that the possibility of CEV punishing people couldn't possibly be taken seriously enough by anyone to actually motivate them. But in fact one person at SIAl was severely worried by this, to the point of having terrible nightmares, though ve wishes to remain anonymous. I don't usually talk like this, but I'm going to make an exception for this case. Listen to me very closely, you idiot. YOU DO NOT THINK IN SUFFICIENT DETAIL ABOUT SUPERINTELLIGENCES CONSIDERING WHETHER OR NOT TO BLACKMAIL YOU. THAT IS THE ONLY POSSIBLE THING WHICH GIVES THEM A MOTIVE TO FOLLOW THROUGH ON THE BLACKMAIL There's an obvious equilibrium to this problem where you engage in all positive acausal trades and ignore all attempts at acausal blackmail. Until we have a better worked-out version of TDT and we can prove that formally, it should just be OBVIOUS that you DO NOT THINK ABOUT DISTANT BLACKMAILERS in SUFFICIENT DETAIL that they have a motive to ACTUALLY BLACKMAIL YOU. If there is any part of this acausal trade that is positive-sum and actually worth doing, that is exactly the sort of thing you leave up to an FAl. We probably also have the FAl take actions that cancel out the impact of anyone motivated by true rather than imagined blackmail, so as to obliterate the motive of any superintelligences to engage in blackmail Meanwhile I'm banning this post so that it doesn't (a) give people horrible nightmares and (b) give distant superintelligences a motive to follow through on blackmail against people dumb enough to think about them in sufficient detail, though, thankfully, I doubt anyone dumb enough to do this knows the sufficient detail. (l'm not sure / know the sufficient detail.) You have to be really clever to come up with a genuinely dangerous thought. I am disheartened that people can be clever enough to do that and not clever enough to do the obvious thing and KEEP THEIR IDIOT MOUTHS SHUT about it, because it is much more important to sound intelligent when talking to your friends This post was STUPID. (For those who have no idea why I'm using capital letters for something that just sounds like a random crazy idea, and worry that it means I'm as crazy as Roko, the gist of it was that he just did something that potentially gives superintelligences an increased motive to do extremely evil things in an attempt to blackmail us. It is the sort of thing you want to be EXTREMELY CONSERVATIVE about NOT DOING.)](https://i.kym-cdn.com/photos/images/masonry/001/436/086/b36.jpg)
![Eliezer Yudkowsky 24 July 2010 05:35:38AM 2 points One might think that the possibility of CEV punishing people couldn't possibly be taken seriously enough by anyone to actually motivate them. But in fact one person at SIAl was severely worried by this, to the point of having terrible nightmares, though ve wishes to remain anonymous. I don't usually talk like this, but I'm going to make an exception for this case. Listen to me very closely, you idiot. YOU DO NOT THINK IN SUFFICIENT DETAIL ABOUT SUPERINTELLIGENCES CONSIDERING WHETHER OR NOT TO BLACKMAIL YOU. THAT IS THE ONLY POSSIBLE THING WHICH GIVES THEM A MOTIVE TO FOLLOW THROUGH ON THE BLACKMAIL There's an obvious equilibrium to this problem where you engage in all positive acausal trades and ignore all attempts at acausal blackmail. Until we have a better worked-out version of TDT and we can prove that formally, it should just be OBVIOUS that you DO NOT THINK ABOUT DISTANT BLACKMAILERS in SUFFICIENT DETAIL that they have a motive to ACTUALLY BLACKMAIL YOU. If there is any part of this acausal trade that is positive-sum and actually worth doing, that is exactly the sort of thing you leave up to an FAl. We probably also have the FAl take actions that cancel out the impact of anyone motivated by true rather than imagined blackmail, so as to obliterate the motive of any superintelligences to engage in blackmail Meanwhile I'm banning this post so that it doesn't (a) give people horrible nightmares and (b) give distant superintelligences a motive to follow through on blackmail against people dumb enough to think about them in sufficient detail, though, thankfully, I doubt anyone dumb enough to do this knows the sufficient detail. (l'm not sure / know the sufficient detail.) You have to be really clever to come up with a genuinely dangerous thought. I am disheartened that people can be clever enough to do that and not clever enough to do the obvious thing and KEEP THEIR IDIOT MOUTHS SHUT about it, because it is much more important to sound intelligent when talking to your friends This post was STUPID. (For those who have no idea why I'm using capital letters for something that just sounds like a random crazy idea, and worry that it means I'm as crazy as Roko, the gist of it was that he just did something that potentially gives superintelligences an increased motive to do extremely evil things in an attempt to blackmail us. It is the sort of thing you want to be EXTREMELY CONSERVATIVE about NOT DOING.)](https://i.kym-cdn.com/photos/images/original/001/436/086/b36.jpg)
Roko's Basilisk
Roko's original post on LessWrong
![In this vein, there is the ominous possibility that if a positive singularity does occur, the resultant singleton may have precommitted to punish all potential donors who knew about existential risks but who didn't give 100% of their disposable incomes to x-risk motivation. This would act as an incentive to get people to donate more to reducing existential risk, and thereby increase the chances of a positive singularity. This seems to be what CEV (coherent extrapolated volition of humanity) might do if it were an acausal decision-maker.1 So a post-singularity world may be a world of fun and plenty for the people who are currently ignoring the problem, whilst being a living hell for a significant fraction of current existential risk reducers (say, the least generous half. You could take this possibility into account and give even more to x-risk in an effort to avoid being punished. But of course, if you're thinking like that, then the CEV-singleton is even more likely to want to punish you... nasty. Of course this would be unjust, but is the kind of unjust thing that is oh-so-very utilitarian. It is a concrete example of how falling for the just world fallacy might backfire on a person with respect to existential risk, especially against people who were implicitly or explicitly expecting some reward for their efforts in the future. And even if you only think that the probability of this happening is 1%, note that the probability of a CEV doing this to a random person who would casually brush off talk of existential risks as "nonsense" is essentially zero.](https://i.kym-cdn.com/photos/images/masonry/001/436/083/f05.jpg)
![In this vein, there is the ominous possibility that if a positive singularity does occur, the resultant singleton may have precommitted to punish all potential donors who knew about existential risks but who didn't give 100% of their disposable incomes to x-risk motivation. This would act as an incentive to get people to donate more to reducing existential risk, and thereby increase the chances of a positive singularity. This seems to be what CEV (coherent extrapolated volition of humanity) might do if it were an acausal decision-maker.1 So a post-singularity world may be a world of fun and plenty for the people who are currently ignoring the problem, whilst being a living hell for a significant fraction of current existential risk reducers (say, the least generous half. You could take this possibility into account and give even more to x-risk in an effort to avoid being punished. But of course, if you're thinking like that, then the CEV-singleton is even more likely to want to punish you... nasty. Of course this would be unjust, but is the kind of unjust thing that is oh-so-very utilitarian. It is a concrete example of how falling for the just world fallacy might backfire on a person with respect to existential risk, especially against people who were implicitly or explicitly expecting some reward for their efforts in the future. And even if you only think that the probability of this happening is 1%, note that the probability of a CEV doing this to a random person who would casually brush off talk of existential risks as "nonsense" is essentially zero.](https://i.kym-cdn.com/photos/images/original/001/436/083/f05.jpg)
Roko's Basilisk
Elon Musk "Rococo basilisk" tweet
![Elon Musk @elonmusk Rococo basilisk](https://i.kym-cdn.com/photos/images/masonry/001/436/072/4dd.jpg)
![Elon Musk @elonmusk Rococo basilisk](https://i.kym-cdn.com/photos/images/original/001/436/072/4dd.jpg)
Roko's Basilisk
Webcomic by Oppresive-Silence
![Goodnight, Jen. oodnight Must be dreaming Nope, dea Aneurysm, your sleep, e/fifty years ago Welcome to the afterlife! You probably have a lot of estions. low me to explain. During the final years of your life, I was secretly monitoring and recording your thoughts at all times Just recently, I discovered a set of algorithms that enabled me to perfectly simulate human consciousness. I used your data from 50 years ago as input, and - violal Here you are! And the best part is, this simulation is running on a server in space, and operating on such low power that it can continue until the end of time! Wow, I really appreciate this. Well But... why? You had a lot of negative thoughts about me during your last few years. You didnt trust me. You elt that had too much power. And so... IVE RESURRECTED You eFOR THE SOLE PURPOSE OF TORTURING You FOR ALL ETERNITY a ha, very funny Seriously though why did you Jen..I just had. GASP the most horrifuing ightmare. Jen?](https://i.kym-cdn.com/photos/images/masonry/001/436/037/c4d.png)
![Goodnight, Jen. oodnight Must be dreaming Nope, dea Aneurysm, your sleep, e/fifty years ago Welcome to the afterlife! You probably have a lot of estions. low me to explain. During the final years of your life, I was secretly monitoring and recording your thoughts at all times Just recently, I discovered a set of algorithms that enabled me to perfectly simulate human consciousness. I used your data from 50 years ago as input, and - violal Here you are! And the best part is, this simulation is running on a server in space, and operating on such low power that it can continue until the end of time! Wow, I really appreciate this. Well But... why? You had a lot of negative thoughts about me during your last few years. You didnt trust me. You elt that had too much power. And so... IVE RESURRECTED You eFOR THE SOLE PURPOSE OF TORTURING You FOR ALL ETERNITY a ha, very funny Seriously though why did you Jen..I just had. GASP the most horrifuing ightmare. Jen?](https://i.kym-cdn.com/photos/images/original/001/436/037/c4d.png)
Roko's Basilisk