syllabus

This is a sample syllabus on existential risk, intended as a helpful resource for people developing courses on existential risk — in schools, universities, independent reading groups, or elsewhere. It assumes that those participating have access to The Precipice, so takes this as the central text.

The syllabus is divided into 10 topics, covering many facets of existential risk. One could remove, add, or merge topics to cater to longer or shorter course. Each topic has one or two highlighted readings (marked with an asterisk) and supplementary readings for those who wish to explore further. There are also a few key questions to guide thinking and discussion.

The syllabus is by no means comprehensive, and I’m very open to suggestions on how it could be improved or made more helpful to educators. Please send any comments to syllabus@theprecipice.com.

Topics

1. Foundations

2. Ethics of existential risk

3. Thinking about existential risk

4. Natural risks

5. Nuclear weapons

6. Climate change

7. Pandemics

8. Unaligned artificial intelligence

9. Political institutions

10. Macrostrategy

• What is an existential risk?

• How can we categorise existential risks?

• What reasons are there for treating existential risks as a separate class from other risks?

* Toby Ord (2020). The Precipice. pp. 35–42.

* Nick Bostrom (2013). ‘Existential Risk Prevention as Global Priority’.

• Why is preventing existential risk of particular moral importance?

• What are some non-consequentialist grounds for caring about existential risk?

• How does time-discounting of wellbeing affect the moral importance of existential risk?

* Toby Ord (2020). The Precipice. pp. 42–57.

* Toby Ord (2020). The Precipice. Appendix A. Discounting.

Nick Bostrom (2003). ‘Astronomical Waste: The Opportunity Cost of Delayed Technological Development’.

Annette Baier (1981). ’The Rights of Past and Future Persons’.

Derek Parfit (1983). Reasons & Persons. pp. 453–454.

Toby Ord (2020). The Precipice. Appendix B. Population ethics.

• Why are existential risks neglected?

• What are some unique methodological challenges for dealing with existential risk?

* Nick Bostrom (2013). ’Existential Risk Prevention as Global Priority’.

Toby Ord (2020). The Precipice. pp.57–64, 173–186.

Toby Ord et al. (2008). ’Probing the Improbable: Methodological Challenges for Risks with Low Probabilities and High Stakes’.

Toby Ord (2020). The Precipice. Appendix D.

• What are the major natural risks? 

• How much risk do they each pose? How can we estimate this?

• How can we use humanity’s lifespan to estimate the total level of natural extinction risk? How does this compare with the level of anthropogenic risk?

* Toby Ord (2020). The Precipice. pp. 67–87.

* Andrew Snyder-Beattie et al. (2019). ’An upper bound for the background rate of human extinction’.

Jonathan Rougier et al. (2018). ’The global magnitude–frequency relationship for large explosive volcanic eruptions’.

• What are the global effects of nuclear war that pose the greatest existential risk?

• How close have we come to the brink of accidental nuclear war? What lessons can we draw from these close calls?

Toby Ord (2020). The Precipice. pp. 90–102.

Joshua Coupe et al. (2019). ’Nuclear Winter Responses to Nuclear War Between the United States and Russia in the Whole Atmosphere Community Climate Model Version 4 and the Goddard Institute for Space Studies Model’.

Carl Sagan (1983). ’Nuclear War and Climatic Catastrophe: Some Policy Implications’.

• Is extreme climate change an existential risk?

• What are some key uncertainties in understanding the scale and impacts of climate change?

• What is geoengineering, and does it pose risks of its own?

Toby Ord (2020). The Precipice. pp. 102–113.

Myles Allen et al. (2018). ’Global Warming of 1.5 ºC — Chapter 1: Framing and Context’.

Gernot Wagner & Martin Weitzman (2015). Climate Shock: The Economic Consequences of a Hotter Planet. pp. 48-79.

GiveWell (2013). ’Extreme risks from climate change’.

• Does deliberate misuse of biological agents (biowarfare or bioterrorism) pose a serious risk to humanity?

• Are there developments in biotechnology that are particularly concerning from the perspective of existential risk?

• What risks are posed by research into potentially pandemic pathogens? Is it well-managed?

• What does COVID-19 reveal about existential risk from pandemics, and more broadly?

* Toby Ord (2020). The Precipice. pp. 124–138.

Piers Millett & Andrew Snyder-Beattie (2017). ’Existential Risk and Cost-Effective Biosecurity’.

Thomas Inglesby & David Relman (2015). ’How likely is it that biological agents will be used deliberately to cause widespread harm?’.

• How likely are we to build artificial general intelligence in the next century?

• What is the alignment problem, and why is it important?

• Are there actions we can take today to reduce existential risk from unaligned artificial intelligence?

Toby Ord (2020). The Precipice. pp. 138–153.

* Nick Bostrom (2014). Superintelligence. Ch. 7–8.

Stuart Russell (2019). Human Compatible. Ch. 6–10.

Stuart Russell, Daniel Dewey & Max Tegmark (2015). ‘Research Priorities for Robust and Beneficial Artificial Intelligence’.

Brian Christian (2020). The Alignment Problem. Part III (Ch. 7–9).

• What are the key challenges in designing political institutions to safeguard humanity?

• Why is the political representation of future generations important?

• What steps have been taken to represent future generations in our political system?

Toby Ord (2020). The Precipice. pp. 57–64, 195–205.

Natalie Jones et al. (2018). ’Representation of future generations in United Kingdom policy-making’.

William MacAskill (2020). ’Age-Weighted Voting’.

Tyler John & William MacAskill (2020). ’Longtermist institutional reform’.

• What is an information hazard?

• What is the unilateralist’s curse?

Nick Bostrom et al. (2016). ’The Unilateralist’s Curse and the Case for a Principle of Conformity’.

Nick Bostrom (2011). ’Information hazards: A typology of potential harms from knowledge’.