Adactio: Links—Tagged with singularity

Why Computers Won’t Make Themselves Smarter | The New Yorker

In this piece published a year ago, Ted Chiang pours cold water on the idea of a bootstrapping singularity.

How much can you optimize for generality? To what extent can you simultaneously optimize a system for every possible situation, including situations never encountered before? Presumably, some improvement is possible, but the idea of an intelligence explosion implies that there is essentially no limit to the extent of optimization that can be achieved. This is a very strong claim. If someone is asserting that infinite optimization for generality is possible, I’d like to see some arguments besides citing examples of optimization for specialized tasks.

The Technium: Protopia

I think our destination is neither utopia nor dystopia nor status quo, but protopia. Protopia is a state that is better than today than yesterday, although it might be only a little better. Protopia is much much harder to visualize. Because a protopia contains as many new problems as new benefits, this complex interaction of working and broken is very hard to predict.

Kevin Kelly’s thoughts at the time of coining of this term seven years ago:

No one wants to move to the future today. We are avoiding it. We don’t have much desire for life one hundred years from now. Many dread it. That makes it hard to take the future seriously. So we don’t take a generational perspective. We’re stuck in the short now. We also adopt the Singularity perspective: that imagining the future in 100 years is technically impossible. So there is no protopia we are reaching for.

The Real Danger To Civilization Isn’t AI. It’s Runaway Capitalism.

Spot-on take by Ted Chiang:

I used to find it odd that these hypothetical AIs were supposed to be smart enough to solve problems that no human could, yet they were incapable of doing something most every adult has done: taking a step back and asking whether their current course of action is really a good idea. Then I realized that we are already surrounded by machines that demonstrate a complete lack of insight, we just call them corporations.

Related: if you want to see the paperclip maximiser in action, just look at the humans destroying the planet by mining bitcoin.