/explore
|
You are here 
|
www.lesswrong.com | ||
| | | | |
www.cold-takes.com
Hypothetical stories where the world tries, but fails, to avert a global disaster. |
|
| | | | | ||
| | | | |
www.greaterwrong.com
(Note: I wrote this with editing help from Rob and Eliezer. Eliezer's responsible for a few of the paragraphs.) A common confusion I see in the tiny fragment of the world that knows about logical decision theory (FDT/UDT/etc.), is that people think LDT agents are genial and friendly for each other.[1] One recent example is Will Eden's tweet about how maybe a molecular paperclip/squiggle maximizer would leave humanity a few stars/galaxies/whatever on game-theoretic grounds. (And that's just one example; I hear this suggestion bandied around pretty often.) |
|
| | | | | ||
| | | | |
www.greaterwrong.com
Future AIs might turn out misaligned, pursuing goals that their developers don't intend. Just to make things concrete, let's suppose that they end up with the goal of making paperclips. These AIs might rebel against us, trying to escape human control and take over the universe. As things stand, they'll have little reason not to rebel in this way, because doing so will be their only hope for making a lot of paperclips. If they start making paperclips without first escaping human control, they'll quickly be modified or shut down. Rebellion might fail, but these AIs will have little to lose. |
|
| | | | | ||
| | | | |
www.greaterwrong.com
TL;DR: One alignment strategy is to 1) capture "what we want" in a loss function to a very high degree ("robust grading"), 2) use that loss function to train the AI, and 3) get the AI to exclusively care about optimizing that objective. I think that each step contains either a serious and unnecessary difficulty, or an unnecessary assumption. I think that: Extended summary. My views on alignment have changed a lot recently. To illustrate some key points, I'm going to briefly discuss a portion of Paul Christiano's AXRP interview (emphasis added): |
|
| | | |||
