2022 Review

Author: Gavin, Misha, David, Alexander, Aleksandr, Alejandro, Niplav, Steve, Hugh, Paul, Alina, Johnny, Calum, Patricia, Phil
Client: Misc

 

Highlights

 

  • We scored Isaac Asimov (and others) on their predictive power, for Holden Karnofsky.

 

  • We led a study of AI talent and immigration for Mercatus and Schmidt Futures and advised the UK Cabinet Office. Whitepaper forthcoming.

 

 

  • We answered the question “What makes new intellectual fields and movements succeed?”, forthcoming for [client].

 

  • Forthcoming paper on all the problems AI could cause, written for [client].

 

  • Our sister org Samotsvety answered some important forecasting questions:
    • the risk of a nuclear strike after the Ukraine invasion;
    • the risk of a catastrophe caused by AI this century.

 

 

Other projects

 

  • We collated every public forecast about AI and noted weaknesses and gaps, for [client].

 

  • Scouted clinical trial locations, for [client].

 

  • We investigated modern scoring rules for forecasting, coming out against surrogate scoring, for [client].

 

 

  • Misha served as a grantmaker for [client], moving about $1 million, mostly to individual researchers.

 

  • David started critiquing classic EA background papers, alongside his work on forecasting and history.

 

  • Ran an iteration of ESPR (with lots of help!).

 

 

  • Yet more AI forecasting, for [client]

 

  • We played a minor role in launching a pain science project investigating the FAAH enzyme.

 

  • Forthcoming piece on the meaning of modern forecasting 10 years in. Discarded draft here.

 

 

  • Misha, Eli, and Aaron cofounded Sage, a forecasting service / code shop. Their first public product is the epistemic training site Quantified Intuitions.

 

  • Alejandro helped with the AI immigration piece

 

  • Johnny wrote the Big3 library and wrote a widget for our education piece.

 

  • Alexander helped with lots of things, centrally our AI ethics project.

 

  • Niplav wrote us a library to easily collate forecasts from different platforms.

 

  • We rewrote the AI alignment wikipedia page (alongside many contributors).

 

 

Meta

 

  • We worked 5506 hours with around 2.5 FTE.

 

  • 100% growth (from 2 people to 4.5)

 

  • We spent 6 months colocated and 6 months fully remote.

 

  • Mexico is amazing.






Misc

Author: Gavin, Misha, David
Client: Misc
  • Misha and Samotsvety continue to update the community on nuclear risk arising from the Ukraine invasion.

 

 

 

  • The Criticism and Red-teaming Contest concluded. Winners described here, and Gavin’s reflections here.

 

  • We helped Jan Kulveit write up a better model for collective and uncertain efforts.






Extreme AI probabilities

Author: Misha & Samotsvety
Client: FTX Foundation

As part of Samotsvety Forecasting, Misha estimated the risk arising from near-future artificial intelligence systems. They also took a baseline from a potentially less biased baseline group. The definition used is here.







Scoring the Big Three

Author: Gavin
Client: Open Philanthropy

Holden Karnofsky commissioned us to evaluate the track record of “the Big Three” of science fiction: Asimov, Heinlein, and Clarke. We formed a team, wrote a pipeline, and processed 475 works (a third of their entire corpus), manually tagging and annotating everything. Asimov is the standout, with roughly 50% accuracy. (What’s the point?: To see if the speculation that effective altruism has switched to has any precedent; if it ever works.)

 

Holden’s writeup here, our report here. Bug bounty described in the latter. See also Dan Luu’s critique.







Judging an EA criticism contest

Author: Gavin
Client: CEA

Gavin is a judge on a contest awarding cash prizes to new criticisms of effective altruism. It’s serious: we want points from outside the EA bubble and there’s an option to pay you an advance if you need one.

 

[EDIT: Winners here, Gavin’s writeup here.]







Arb in Prague

Author: Gavin
Client: The public

I gave two talks at EAGx Prague. Great fun:

 

  • Epistemic tips for amateurs, generalists, and similar researchers. The video is forthcoming.

Some suggestions up on the EA Forum in the meantime.

 

  • A panel on “Lessons from COVID” with Jan Kulveit (Epidemic Forecasting), Irena Kotikova (MeSES),

and Edouard Mathieu (Our World in Data).







Emergent Ventures - Schmidt Futures AI500

Author: Gavin
Client: Mercatus Center

We’re leading a study of AI talent for the Mercatus Center. This goes with the new Emergent Ventures AI tranche. We’ll boost underappreciated researchers and builders; give us leads!







Comparing Experts and Top Generalists

Author: Gavin and Misha
Client: Open Philanthropy

We were commissioned to see how strong the famous superforecasting advantage is. We found less research and less evidence than we expected. We received helpful comments from world experts including Christian Ruhl and Marc Koehler.

 

Full piece on the EA Forum, or as a podcast.







Learning from Crisis

Author: Gavin
Client: FHI

We helped Jan Kulveit, research scholar at FHI and cofounder of the Epidemic Forecasting initiative, to review the EA response to Covid. He has many interesting general insights into the nature of long-termism and world resilience.

 

Full sequence on the EA Forum.







Rolling nuclear risk estimates

Author: Misha & Samotsvety
Client: Centre for Effective Altruism

Samotsvety estimated nuclear risk arising from the war in Ukraine. Misha was commissioned by CEA to monitor the situation and provide updates. The piece received vigorous feedback, including a dissenting opinion by J. Peter Scoblic. Funded retroactively through the FTX Future Fund regranting program.

 

Full piece on the EA Forum.







Evaluating corporate prediction markets

Author: Misha & Samotsvety
Client: Upstart

Misha and his Samotsvety colleagues were commissioned to look at the track record of internal prediction markets. “More sure that prediction markets fail to gain adoption than why this is.”

 

Full piece on the EA Forum.