Vulnerable World Hypothesis
- Paper by Nick Bostrom
- Bostrom discusses if a nuclear explosion could be made using glass metal and a battery. Here is a short quote:
- An alternative approach would be to eliminate all glass, metal, or sources of electrical current (save perhaps in a few highly guarded military depots). Given the ubiquity of these materials, such an undertaking would be extremely daunting. Securing political support for such measures would be
no easier than shutting down physics education. However, after mushroom clouds had risen over a few cities, the political will to make the attempt could probably be mustered.
- He is right that in principle, it is plausible that easy, massively destructive technology exists.
- I am not fully convinced by this:
- It seems if it were that easy it would probably have happened by now
- Very few people want to kill everyone, even if it is very easy, suicide bombers are pretty rare
- The technologies could get replaced - eg glass with perspex
- Humans are good at banning even very beneficial technologies Slowed technologies
- It is very easy to blow up planes, and so we have developed a huge amount of technology around stopping that - and people still fly
- This isn't a good analogy to AI - AI tools are very expensive to build and likely to be so for a while. Even then surveillance could monitor some things without monitoring everything.
- Bostrom looks at some options, dismissing the first 2:
- Restrict technological development.
- Bostrom finds this both hard and tragic.
- Seems wrong, we have narrowly restricted technologies - Slowed technologies
- Establish extremely effective preventive policing.
- Establish effective global governance.
- Ensure that there does not exist a large population of actors representing a wide and recognizably human distribution of motives.
- Prior
- we should arguably use a fairly uniform prior in log space (over several orders of magnitude) over the size of apocalyptic residual that would be required in order for civilizational devastation to occur conditional on a Type-1 vulnerability arising. In other words, conditional on some new technology being developed that makes it easy for an average individual to kill at least one
million people, it may be (roughly) as likely that the technology would enable the average individual to kill one million people, ten million people, a hundred million people, a billion people, or every human alive. - This doesn't seem right? We don't see a uniform historical prior here.
- Bostrom misses the option of making risky tech extrememly low status.