Namaste! You must login or signup first!

Eliezer Yudkowsky 24 July 2010 05:35:38AM 2 points One might think that the possibility of CEV punishing people couldn't possibly be taken seriously enough by anyone to actually motivate them. But in fact one person at SIAl was severely worried by this, to the point of having terrible nightmares, though ve wishes to remain anonymous. I don't usually talk like this, but I'm going to make an exception for this case. Listen to me very closely, you idiot. YOU DO NOT THINK IN SUFFICIENT DETAIL ABOUT SUPERINTELLIGENCES CONSIDERING WHETHER OR NOT TO BLACKMAIL YOU. THAT IS THE ONLY POSSIBLE THING WHICH GIVES THEM A MOTIVE TO FOLLOW THROUGH ON THE BLACKMAIL There's an obvious equilibrium to this problem where you engage in all positive acausal trades and ignore all attempts at acausal blackmail. Until we have a better worked-out version of TDT and we can prove that formally, it should just be OBVIOUS that you DO NOT THINK ABOUT DISTANT BLACKMAILERS in SUFFICIENT DETAIL that they have a motive to ACTUALLY BLACKMAIL YOU. If there is any part of this acausal trade that is positive-sum and actually worth doing, that is exactly the sort of thing you leave up to an FAl. We probably also have the FAl take actions that cancel out the impact of anyone motivated by true rather than imagined blackmail, so as to obliterate the motive of any superintelligences to engage in blackmail Meanwhile I'm banning this post so that it doesn't (a) give people horrible nightmares and (b) give distant superintelligences a motive to follow through on blackmail against people dumb enough to think about them in sufficient detail, though, thankfully, I doubt anyone dumb enough to do this knows the sufficient detail. (l'm not sure / know the sufficient detail.) You have to be really clever to come up with a genuinely dangerous thought. I am disheartened that people can be clever enough to do that and not clever enough to do the obvious thing and KEEP THEIR IDIOT MOUTHS SHUT about it, because it is much more important to sound intelligent when talking to your friends This post was STUPID. (For those who have no idea why I'm using capital letters for something that just sounds like a random crazy idea, and worry that it means I'm as crazy as Roko, the gist of it was that he just did something that potentially gives superintelligences an increased motive to do extremely evil things in an attempt to blackmail us. It is the sort of thing you want to be EXTREMELY CONSERVATIVE about NOT DOING.)

+1

Eliezer Yudkowsky's response to Roko's original post on LessWrong


Uploaded by Don Caldwell Nov 30, 2018 at 04:02PM EST

Share Pin Download More

Origin Entry: Roko's Basilisk

File type: jpg

Resolution: (1372px x 861px)

File size: 251 KB


This page is made to get you to share it.
This is Bill Bill is an advanced future Al that can travel back in time. He also has the ability to know every human who thought or imagined him Bill endlessly tortures anyone who thought of him but didn't help him come into existence. Now you know about Bill. Now you know about Bill. u/Paper-Cup
VIRGIN "THE GAME" you've just lost it LOST THE GAME "the game haha you lost i'm so quirky" completely inconsequential not even a memetic hazard CHAD ROKO'S BASILISC if you know what it is you're definitely f----- failure results in eternal suffering "have you heard about roko's basilisk? what a retarded mind game unless" you have to do it's bidding now at least 5th level memetic hazard
View All 29 Images


More:

0
+1

Roko's Basilisk - Eliezer Yudkowsky's response to Roko's original post on LessWrong

PROTIP: Press the ← and → keys to navigate the gallery, 'g' to view the gallery, or 'r' to view a random image.


◄ Previous View Gallery Random Image Next ►

Moar:


+ Add a Comment

Comments (0)

There are no comments currently available.

Display Comments

Add a Comment


+ Add an Image

Image Details

2,439 views (12 from today)

Uploaded Nov 30, 2018 at 04:02PM EST

Origin Entry

Roko's Basilisk

Source

Basilisk.neocities.org

Tags

eliezer yudkowsky, lesswrong, forum

Claim Authorship Edit History
Textile Embed