Archive for January, 2012

Entropy and unconvincing models

I’m keen to find a model of the world that is able to describe, say, an intelligence explosion. I think it would be an important step towards establishing whether the singularity is just a fantasy. Such a model may be wrong, but there is just something about a mathematical model that gives it more weight in an argument than mere words. Words can be confusing and misleading in all sorts of ways; with maths at least you know what you’re trying to say, and establishing whether it then agrees with reality is a bit more of a mechanical process.

I don’t have a model for you yet. There are people significantly smarter than me working on this kind of stuff and if they haven’t done it yet it’s unlikely I will. So the best I can do for now is to try to imagine some aspects of what such a model would look like. It’s still words but maybe the words will start to mean something a bit more.

The only totally convincing model of an intelligence explosion would be to simulate all the computational processes involved in a recursively self-improving AI. This would be a really bad idea. A simulation of a recursively self-improving AI is a recursively self-improving AI (See the AI box argument, and then also consider the possibility of technology leakage).

Continue reading ‘Entropy and unconvincing models’


Toronto LW Singularity Discussion, 2012-01-12

Warning: contains mathematics

Present: RA, SB, SD, GE, MK, JM

Minutes: GE

Host: pub

This discussion was slightly more informal due to a noisy venue and two newcomers. Welcome to Less Wrong, RA and MK! Also great to see familiar faces SB and JM taking an interest in the Singularity discussion.

GE kicks off with a discussion of Minds, Machines and Gödel. GE attempts to explain Gödel’s incompleteness theorem(wp) but I don’t get very far. (This page is a good quick introduction, but I feel it glosses over some important details – Godel’s incompleteness theorems only talk about whether statements are provable, not whether they’re true. If you get those concepts even slightly confused then you end up in a world of fail).

Continue reading ‘Toronto LW Singularity Discussion, 2012-01-12’

Toronto LW Singularity Discussion, 2011-12-06

First off, apologies to everyone for the super-late circulation of these minutes. Christmas and my day job have been getting in the way somewhat. Also as always, if I seem to have misrepresented what you said or what you meant then let me know. A technical note: rather than listing them all at the end, I’ve put wikipedia links inline with (WP) after them to let you know it isn’t linking anywhere exciting.

Present: SD (host), GE (minutes), SM

Apologies: SS

The question for this meeting was: how do we go about deciding whether the Singularity is something we should be worried about? What’s the next thing we need to do?

SD says: we need to read lots of stuff. We all generally agreed on this point. So yeah, that answered “what’s the next thing we need to do” pretty quickly. But we still had plenty to talk about.

Continue reading ‘Toronto LW Singularity Discussion, 2011-12-06’