Last Updated on March 11, 2021
I like how Erik Brynjolfsson and Andrew McAfee put it:
After …watching one bastion of human uniqueness after another fall before the inexorable onslaught of innovation, it’s becoming harder and harder to have confidence that any given task will be indefinitely resistant to automation.
The Second Machine Age
I also love Nick Bostrom’s Superintelligence. It was simultaneously over my head & blowing my mind. A must read if you plan to exist in the future.
Far from being the smartest possible biological species, we are probably better thought of as the stupidest possible biological species capable of starting a technological civilization — a niche we filled because we got there first, not because we are in any sense optimally adapted to it.
Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an “intelligence explosion,” and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.
Is there some other way of construing the tool-AI idea so as to preserve the reassuringly passive quality of a humdrum tool?
Before the prospect of an intelligence explosion, we humans are like small children playing with a bomb. Such is the mismatch between the power of our plaything and the immaturity of our conduct. … For a child with an undetonated bomb in its hands, a sensible thing to do would be to put it down gently, quickly back out of the room, and contact the nearest adult. Yet what we have here is not one child but many, each with access to an independent trigger mechanism. The chances that we will all find the sense to put down the dangerous stuff seem almost negligible. Some little idiot is bound to press the ignite button just to see what happens.
Superintelligence