models as a social technology
what a model is - a grown up shoebox diorama
A large part of my job is working with simplified models of complex phenomena. We study the world to identify patterns, then embed principles derived from observation into a simplified structure to better understand the past and make predictions about the future.
how we usually think of models
For many classes of models, the process of identifying predictive features and useful functional forms is the core of the value. We take a very large data set in a system with fairly fixed governing principles (ex. the laws of physics) and derive useful methods that let us better predict the future. These are often models with large volumes of data, observable outcomes, and repeatable experiments, like physical models (gravity, motion, fluid dynamics, meterology).
the models i work on are different in a specific way
The models I work on are a distinct subclass. We’re trying to predict what will happen in the future without clear laws that govern the behavior of the system or complex emergent behavior.1 I think models of how AI will impact the economy also fall into this subclass. Thinking of the recently released Anthropic Scenarios for our Economic Future.
I work with long term forecasts of the global energy system, trying to predict the share of energy generating technologies like solar or nuclear through 2100, under current policies and in a world where we’re on track to decarbonize the global economy.2 This model is built and maintained by my brilliant colleagues, I contribute ideas but don’t own the modeling. All credit to them here. I’m working with a team that’s building a framework on top of a modified version of these models. The initial step of building the model still is in determining functional forms and important parameters. We do what we can to represent dynamics (e.g. cheaper technology deploy first; as deployment grows, technology costs decline) and constraints (e.g. natural resource limits on solar or wind). This type of modeling is so complex and on such long timescales that it’s exceptionally challenging represent all of the potential dynamics.
models as a social technology
I’m increasingly convinced though, that a large and underappreciated share of the value is in the next step, where we show the model and the results of our framework to important people with wide ranging expertise (finance in emerging markets, industrial decarbonization, energy storage). They tell us where they agree and disagree with our model and results (e.g. “geothermal should be more constrained in Australia”, “I’m more optimistic about carbon capture for cement than for steel plants”). Then we iterate and improve the model.
The result is a model that represents a weighted average consensus of expert opinions in the field, representing the wisdom of a bunch of closed door conversations to gather insights that aren’t documented or public. The model functions as a social technology for aggregating and averaging expert opinion.
Expert consensus won’t always be right. There are many cases where models fed large data will identify real patterns humans missed, or cases where many experts haven’t caught up to an important evolution in the field. That said, there is value in a model that represents how people are thinking now, to build consensus and momentum around policy and to track the evolution of collective thought. I think this is especially true for cases where you’re using the model to organize people around a shared goal (like accelerating the energy transition, or mitigating negative impacts of AI on employment or human wellbeing).
AI and the future of modeling
I also think this has implications for understanding how AI will reshape the future of modeling. I expect to see ever more people building and sharing models now that Claude can pull together a detailed, plausible-sounding model of essentially anything. But in many contexts, there is value in relying on a model that has existing social architecture (people that maintain it, people that use it for real, serious decisions and care about it’s quality) because of this iterated expert consensus. Models that actually translate into meaning and action need relational weight and consensus, people to stand behind their results together and advocate for a shared platform based on the knowledge.3 Thinking here of the value of modeling around the passage of the Inflation Reduction Act and the way modelers collaborated and iterated and built consensus with advocates and policymakers to shape and pass policy. The social architecture is a big share of the value.