Chapter Sixteen. The Numbers Nobody Agrees On.
Reasoning Carefully Under Deep Uncertainty
The probability estimates for catastrophic artificial intelligence risk range widely, and
that range is itself important information. This chapter is about how to
think clearly when the stakes are high and the data is thin.
In 1983, a Soviet engineer named Stanislav Petrov was the duty officer
at a Soviet nuclear early warning facility outside Moscow when the
system reported that the United States had la