Rationalist problem sets

• Forecasting (esp in competitions)

  • Related, but not the same: betting on prediction markets

• Fermi estimates (esp in competitions)

• Exercises in comprehensive information gathering (a la John wentsworth)

  What is this? Why?

  [S]everal of the most durably-valuable exercises I’ve done over the years have a general theme of comprehensive information gathering.

  …

  These sort of exercises provide value in a few ways:

  • They reveal unknown unknowns - things you didn’t even realize were missing from your picture of the world.
  • You can’t make a map of a city by sitting in your room with the shades drawn; exercises like these force you to look at large slices of the world.
  • Knowledge within fields tends to have decreasing marginal returns - your first physics or CS class will teach you much more than your eighth. These exercises give a broad, brief glance at many areas where you probably haven’t reached decreasing marginal returns yet.
  • You can get a very rough big-picture sense of how much effort other people are investing in various areas - e.g. where most capital investments go or where most research effort goes - which is useful for understanding the world in general.
  • While these exercises don’t avoid biased selection of information altogether, they’re probably different biases from what you run into naturally, and they’re systematic enough that we can guess at what biases are likely to be present.
  • They’re a lot of fun, if you have a curious streak.

  Most importantly: I’ve found each of these exercises to have lasting, long-term value in exchange for a one-time investment of effort.

  Some example outputs of comprehensive info gathering:
  Some ideas for comprehensive info gathering:

  • Stolen from the original article:
    • Skim every article from Nature in the last $n$ years (stolen from the original article)
    • “If you’re in college, I strongly recommend reading your entire course catalogue, googling anything you’ve never heard of at all, and marking anything that sounds potentially interesting. This seems really obvious; it only takes a few hours, and something something a pile of value sitting on a silver platter right in front of you. (Note: I went to a small STEM school; if you’re at a big school with a bajillion courses or a school with poor STEM coverage or not at college at all, consider reading an MIT/Caltech course catalogue instead, to get a feel for what all is out there.) You never know what surprising and interesting topics might be hiding in there - microfluidics, underactuated robotics, recursive macroeconomics, systems biology, synthetic biology, origami algorithms, computational photography, evo-devo, procedural graphics, and on and on.”
    • “Read the entire CIA world factbook; you can get a paper copy for $11 on Amazon.”
    • “Go through all of the (known) functions of genes in a minimal organism.” [original link here is dead]
  • Read particular subsections (e.g. those on “business,” “risk factors,” for ) of the 10Ks for every company in the:
    • S&P 100
    • S&P 500
    • top 10 highest market cap companies for each of the last 10 years
    • top $n$ highest market cap companies in an industry (e.g. energy, tech, healthcare, etc)
  • Read every model card since GPT-2’s was released in 2019
  • Reading Cell Biology by the Numbers (h/t Joe Cavanagh)

• Shallow literature reviews

• Epistemic spot checks

• Probabilistic calibration training (see

• Exercises in agency (nmp post, niplav post) — accomplish random goals of increasing agency (e.g. from “ask a stranger for the time” to “get a free meal at a restaurant” all the way up to “acquire a human tooth”)

• Fact posts

• Learning by writing (a la Holden karnovsky)

What am I missing? Let me know: saulsmunn@gmail.com