Other Terasem Journals

How We Can Manage Our Way Through the Intertwined Promise and Peril of Accelerating Change

Ray Kurzweil

page 15 of 15

Then one could say, wait a second, that sounds just like the first stage of a stealth destroyer, because the ultimate nightmare scenario of nanotechnology is not just grey goo – a nanobot that Kurzweil Quote self-replicates, because the front of destruction would move very slowly. Nanobots cannot move very quickly. Therefore, you would see it happening and you could deal with it. The real nightmare scenario is: The nanobots self-replicate, using up one in thousand trillion of the carbon atoms. It is very stealthy; nobody notices it. Then they plant themselves throughout the biosphere. At some trigger, they start self-replicating in place. Then the front of destruction does not have to move. It has already seeded the entire biosphere and it would take about 90 minutes to self-replicate and destroy the biosphere.

Here, you cannot at that time start trying to design a self-replicating immune system. One could point out that putting an immune system in place that has self-replicating capabilities looks just like the first stage of the stealth destroyer scenario. Of course, the immune system is there to protect us, not to destroy us. But how can you tell? These are all interesting questions and biology has dealt with them in a chaotic manner. Ultimately, we will need an immune system. Yet we must take steps to make sure it is friendly.

Finally, we come to the third challenge, R, which stands for robotics, which really means strong AI, AI at a human level, AI at a level that has a copy of its own design, and is able to actually go and improve its own design in a closed-loop self improvement cycle that could ultimately be very rapid. This is the sort of runaway concept of how once you get strong AI, it can rapidly improve its own capabilities.

What if you have unfriendly AI? This would be AI that does not have our values, such as the value of maintaining human civilization. That is the most daunting challenge. If we come back to the nanotechnology challenge, you can see that if you describe any level of nanotechnology, we can design a system, whether it is the broadcast architecture or some more sophisticated version of it or some sophisticated technological immune system that deals with it. Basically the solution has to be more intelligent than the challenge. If you have some level of potentially destructive nanotechnology, you need a response that is more intelligent.

The problem with artificial intelligence is if a system is more intelligent than you and if it is bent on your destruction, there is really not much you can do about it. Intelligence is the most powerful force in the universe. History has shown that the civilization with the more sophisticated technology that has harnessed its intelligence in its technology to a more effective degree generally has prevailed in history, for good or bad.

That is really the most daunting challenge, because when we have nanotechnology, it can protect us from the dangers of biotechnology. The nanobots can protect us from rogue biological viruses. Intelligent technology can protect us from rogue nanotechnology. Yet what is going to protect us from rogue intelligence that is greater than our own intelligence?

Ultimately, the solution I come up with is that we have to embody values of openness, free exchange of information, a democratic society, and all the values that that we embody in our civilization. Because this is not an alien invasion of intelligent machines coming from over the horizon. It is emerging from within our civilization. It is not going to be distinct from us, even within the room. We are going to have intelligent processes running inside our biological bodies and brains. We will have non-biological systems that are derived from the reverse engineering of biological systems. It is going to be one civilization. That civilization needs to embody the best of human values. That is the best we can do to assure that this future intelligence that we are creating, which will be us, will embody the best of human values. If you say, that does not sound foolproof; it is not, but that is really the best we can do.

For additional information, please visit www.kurzweilai.net and www.singularity.com.

Ray Kurzweil, Ph.D. is a best-selling author, inventor and entrepreneur. He is a recipient of the National Medal of Technology.

<previous 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Back to Current Issue Contents

The Terasem Pledge

We pledge allegiance to the flag of the collective consciousness of Terasem, and to the principles for which it stands, education, persistently, with diversity, unity, and joyful immortality everywhere.

TerasemJournal.com Home | Journal of Geoethical Nanotechnology | Journal of Personal Cyberconsciousness
Submission Guidelines | Editors and Sponsorship | Contact