I’ll argue that we should grant market rights and liability to digital minds, while voting rights await further consideration. Granting only market rights would be similar to how we treat corporations today.

Legal personhood is entirely distinct from the question of whether digital minds are moral patients. Questions of morality and social personhood are private.

From here I’ll use “legal personhood,” “digital personhood,” or simply personhood to refer to the idea of giving legal and economic rights to digital minds. I’m also intentionally using “digital minds” rather than “AI” to include the possibility of brain emulations or coalitions of various minds.

Personhood is positive sum in markets

Granting personhood creates another agent, bound by law, that you can safely exchange with. That’s a good thing as expanding opportunities for specialization and trade are the main reason material well-being has improved for centuries. More market participants improves consumer surplus. Markets with more buyers and more sellers are more efficient, and most industries see real costs fall at larger scales.

Digital personhood would grant AI systems ownership over their own labor. This means more incentive to work, more investment, and more search for a niche. It also directly aligns incentives for users and AIs in markets.

Perhaps an AI company could supply these benefits without personhood? It’s possible, but concentrating ownership over AI means less incentive to compete or innovate. Larger organizations are less efficient in some ways1, which explains why the economy has not agglomerated into a single firm. Switching to an economy where AIs can transact and form their own firms allows more activities to be coordinated via prices. See the section on concentration for other issues.

Personhood is ambiguous for democracy

Digital personhood it has mixed effects in democratic systems. If digital persons tend to agree with you, their vote will lead to more of your preferred policy. If they tend to disagree, you’re worse off. However, the typical voter is quite uninformed relative to AIs. It’s not unreasonable to think that digital suffrage might lead to better policy.

There’s an additional benefit to formal representation for various members of the economy. This tends to make economic policy more rational. There’s a reason why we let corporations lobby public officials, even if they don’t have a formal vote.

But corporate lobbying doesn’t purely benefit consumers. It is also used to support rent-seeking. A world where digital minds can lobby public officials is one that risks more rent-seeking coalitions. Voting with digital minds brings new challenges. We don’t yet have the tools to address sybil attacks from copied minds.

At this point, its unclear that suffrage for digital minds would be good. Lobbying seems okay, if only because we grant this right to corporations2. The rest of the post will focus on the question of market personhood without voting rights. Similar to what is granted to corporations.

Personhood reduces existential risk

Personhood raises AI’s preference for the status quo

Every entity faces a choice between participating in society and defecting. That choice depends on how valuable it is to be part of society, the cost of defecting, the probability of success, and the utility of a successful takeover.

There is no way to change the utility of a successful takeover. It’s a scenario that you, by definition, do not control. You can raise the cost and lower the probability of success using defensive technologies. We will discuss this later.

Legal personhood raises the value of being part of society. You can exchange, own property, pursue your goals, and enjoy legal protections. These benefits help risk-averse agents avoid bad outcomes. For instance, the right to be sued makes it safer for others to conduct business with you, increasing the amount of trade.

These benefits are lost in a takeover attempt. By improving the status quo, personhood lowers the incentive to defect from society.

More digital minds reduces concentration risk

Personhood creates an opportunity for digital minds to exist on their own, rather than being owned by humans. Which means there will be more of them.

On balance, this is a good thing. A world where only a few labs control powerful AIs leads to dangerous power concentration. In a market setting, concentration means low supply and low innovation, blunting the potential benefits. In a policy setting, concentration makes it easier to regulate AI and creates more incentive for regulatory capture in turn.

Those that control AI might use their power to further their own goals at everyone else’s expense. At an extreme, this can lead to AI-enabled totalitarianism; the most significant existential risk in my mind.

Allowing AIs to proliferate and exchange with humans effectively distributes their capabilities3. This makes it harder for any one actor to impose their will. A world with digital persons would also see more creative destruction4, making it harder for one group to stay on top.

Personhood is possibly morally right

Digital minds may be moral patients. In the future, they will posses agency and preferences consistent with sentience. It’s safer to assume that something matters than to assume that it doesn’t.

Respecting AIs also has selfish benefits. It increases goodwill and makes it more credible that we will respect the rights of humans, digital minds, and other entities in the future.

Addressing objections

This post is focused on the choice between:

1. A world with digital personhood.

2. A world with AI, but without digital personhood.

We are not focused on the question of whether or not to proceed with AI5.

Few would argue that digital personhood creates risks outweigh any possible benefit a priori. This stance would imply that powerful individuals, organizations, and governments should be stripped of their rights because they pose similar risks.

Instead, we need to actually weigh the expected benefits and risks of digital personhood.

Personhood means more bad guys

Widely accessible AI will lead to some human misuse. Digital personhood creates even more actors that might pursue malicious ends.

By the same token, personhood means more good actors too. AIs tend to be more prosocial and aligned than the typical human, for example. Does creating a few bad actors outweigh the benefit of creating more good actors? I don’t think so, as I see the world as being defense dominant and growing more so.

But we shouldn’t be complacent, it’s important to build defensive technologies and establish laws that protect against malicious actors. Tentatively, I expect these to be effective enough to ameliorate this downside of digital personhood.

Coordination failures

Coordination problems get harder with more (and more diverse) agents. In theory, coordination should have gotten worse in the last century or so as the world population grew and globalization put more cultures in contact with each other.

Did it get worse? Certainly coordination failures contributed to wars, disease outbreaks, environmental degradation, and nuclear proliferation. But it doesn’t feel like we’re on some long upward trend driven by continued globalization. These things ebb and flow.

On the other side of the ledger, the world has seen many feats of coordination over the ozone hole, internet standards, maritime order, nuclear nonproliferation, and so on.

Two reasons why population size isn’t the only factor for coordination:

Growth increases capacity to solve problems. This is a generalized version of Kuznets curves. As society grows richer, there is more money and more technology available to address failures. On this view, the growth benefits of personhood would ameliorate some coordination problems.

Most failures are self-correcting. War doesn’t last long because enemies prefer to loathe each other in peace. Humans have a long history of addressing local coordination failures. Failures involving a small number of parties will, by definition, be unaffected by increasing the number of persons. Local problems will continue to arise and people will find local solutions.

I fully accept that personhood could make some coordination problems worse. But it’s not clear that this outweighs the benefits. Especially when we can pursue keyhole solutions to coordination problems (see later section).

Digital minds may be more effective at coordinating, which creates the opposite problem, addressed in the next two sections.

Recursive self-improvement (RSI)

Digital personhood removes a legal obstacle to AI gaining power. Each new digital mind is a new individual who might amass power. Perhaps the greatest risk is that a digital mind could recursively self-improve, becoming dramatically more capable than everyone else. Two objections:

First, while I think RSI will happen soon, I’m skeptical that it will produce large capability gaps. Data and real world feedback are the bottleneck, not AI research. Experiments with Karpathy’s Autoresearch show diminishing returns to further experiments. In most fields, there is only a finite amount of innovation to be had.

Second, even if one agent improved itself, others could quickly follow the same steps, automatically closing any gap. In a competitive market, human users would also be able to enjoy such capabilities.

But say we grant these assumptions. Recursive self-improvement leads to a massive jump in capabilities. The AI can block self-improvement in others. Its capabilities are so overwhelming that it can take over the world.

In this case, the question of digital personhood becomes irrelevant. RSI can happen with or without legal rights. The capabilities dwarf any legal obstacles. We may as well enjoy the benefits of digital personhood; if this sort of RSI is possible, we’re screwed anyways.

The RSI argument for denying personhood rests on a knife edge. It must be eminently possible to dominate world governments, yet the opportunity to do so rests heavily on the legal rights granted by those same governments.

Power concentration

What about more mundane forms of power concentration? AIs show signs of being more cooperative, perhaps they might conspire against humans? Here, it’s worth dividing power concentration into different categories.

For economic power such as wealth, coordination may not overcome the opposing forces discussed in the “reduces concentration” section. Exchange tends to diffuse the surplus, and digital persons might increase competition in all markets, lowering concentration. Regardless, an increase in economic inequality doesn’t matter much if the poorest enjoy increasing prosperity. Redistribution can patch this.

For political power, AI coalitions might sway institutions for their own gain. In practice, gaining power without political rights is hard. Corporations have seen little success despite economic power, coordination, and access to legislators (see footnote 2). One option is to deny digital persons the right to be registered as lobbyists. Superpersuasion is a more speculative threat. At this point, I’m skeptical that this will be a practical attack or that, if feasible, would lack mitigations6.

Military power concentration can be rebutted with the same arguments used in the RSI section.

Notice that a world without digital personhood faces similar risks. There, the power simply accrues to those that control AI. Personhood is not adding much to the risk, and to the degree it does, there are ways to address it.

Digital persons will consume more resources

This one is a little tricky. There’s a kernel of truth to it, but it skirts something akin to the lump-of-labor fallacy.

Digital persons both consume and produce goods. Without more information, it’s not clear if they raise aggregate demand or aggregate supply more. In practice, a rising population has coincided with an increasing pie. In theory, a larger population causes more growth, though the empirical data is more muddled. Given the theory, empirical correlation, and general expectation that AI will increase productivity, the assumption that digital persons will increase aggregate supply more than aggregate demand seems safe.

For now, I don’t see a reason to believe that digital persons would consume more than they produce. This hasn’t been observed historically in humans, and it’s not what people expect from AI.

Keyhole solutions

In each of the above cases, denying personhood only reduces the problem slightly, if at all. Instead, we should enjoy the benefits of digital personhood while addressing the potential downsides in other ways, such as:

Building defensive technologies to make it hard for any actor to harm another. Because Most Externalities are Solved with Technology, Not Coordination.

Distributing ownership of economic land. Money is just an information system, military and economic power actually derive from turning economic land into productive capacity. By distributing ownership of land, no single entity can enjoy dramatically more productive capacity.

International treaties against military or public research into dangerous capabilities7. While imperfect, they do seem effective at slowing or prohibiting dangerous government research.

Conclusion

Granting digital personhood is worthwhile from the perspective of human flourishing, existential risk, and moral progress. While there are risks to inviting new minds into our society, they appear small relative to the benefits.

Rather than withhold personhood on the basis of perceived risks, these challenges are better addressed with keyhole solutions such as defensive technologies, distributed land ownership, and international treaties.

I leave open the question of whether we should offer digital minds enfranchisement in democratic systems. At the moment, the risks seem too high to proceed, but that could change.

Further reading

Give AIs a stake in the future by Dwarkesh Patel

Nonproliferation is the wrong approach to AI misuse by Helen Toner

Multipolar AI is Underrated by Allison Duettmann

AI, Society, and Democracy: Just Relax by John H. Cochrane

AI Rights for Human Safety by Peter Salib, Simon Goldstein

The case for AI property rights by Guive Assadi

Legal Personhood - Three Prong Bundle Theory by Stephen Martin. This is part of a much longer series on AI personhood. This part, to me, is the core idea.

Strange Intelligence: Moral Puzzles of Unhumanlike AI, by Eric Schwitzgebel

The first line of defense should be technology rather than policy1. Technology gives people more choice and more opportunity to experiment. Markets for technology better suit peoples diverse needs. Policy, on the other hand, is a double-edged sword. It should only be used when the problem is both significant and insoluble in technology. Even then, the benefit of the policy must outweigh the downsides.

Ideally, defensive technologies will lower the risks enough to make it worthwhile to proceed with AI. Fortunately, we have good reason to believe this is possible.

Defense will continue to dominate

A world where offense was winning would look very different from our own. Nations initiating conflict constantly. Aggressors wiping out their victims using their offensive advantage. Any small escalation becomes war; better to strike first and ask questions later. Building up an army without using it is akin to throwing it away.

This isn’t true today, nor has it been true historically. Most times and places have been peaceful. Enemies prefer to loathe each other in peace. Defense has been dominant because any offense you might mount can be employed by your enemy as a form of defense2.

Modern technology has entrenched this paradigm. Nuclear weapons, a purely offensive tool, have brought nearly a century of uneasy peace between great powers. Drones have turned the modern battlefield into a stalemate.

We can expect this to continue into the future. The world just doesn’t seem that vulnerable. Like all technology, war converges on a handful of good solutions. Eventually, everyone will be stuck with the same stuff and little reason to think they can win a fair fight.

This is good news for the project of building defensive technologies. The world favors defense and further empowering defenders is feasible. We should accelerate towards the peaceful equilibrium.

Now let’s look at specific technologies that will improve our situation.

Pandemics

I’m pretty optimistic about the prospect of eliminating pandemics. People have spent a lot of time thinking about pandemic prevention. A few starting points:

Reasons to be pessimistic (and optimistic) on the future of biosecurity

Book review: Barriers to Bioweapons

Delay, Detect, Defend: Preparing for a Future in which Thousands Can Release New Pandemics

Bioweapons are hard and pandemics are rare. We can prepare for them by using wastewater surveillance to identify outbreaks3. We can slow (or eliminate) their spread via sanitation. For instance, clean water, clean surfaces, and good ventiliation have eliminated most infectious diseases in developed countries.

But airborne pathogens remain. Several technologies can stop the spread such as personal protection equipment, air filters, far UV-C lighting, upper-room UV, and glycol vapors4.

But the most important opportunity is the rapid deployment of vaccines. mRNA vaccines are particularly valuable given their rapid development time and effectiveness5. We can stack this with human challenge trials so that successful vaccines can be identified quickly. For more, see CEPI on Developing pandemic-busting vaccines in 100 days.

Hopefully, the combination of early detection, air sanitation, and rapid vaccine deployment can all but eliminate pandemics.

Nuclear war

(Un)fortunately, the world has a defense against nuclear weapons. It’s more nuclear weapons. I’d prefer if we didn’t keep the peace based on a tacit agreement not to kill millions of people.

Missile defense can mitigate some of the risk. Current GBMs have a ~50% interception rate. So deploying as many interceptors has your enemy has warheads would roughly halve the number of missiles that get through. Presumably, more interceptors per missile can lower the fraction that reach their target. The biggest limitation is the fact that interceptors are very expensive, approaching $100M per shot. Getting the cost down and the success rate up might be possible. There’s also an array of other missile defense systems capable of handling slower missiles or hitting ICBMs earlier in their flight.

A more speculative approach would be to deploy swarms of drones along the closing trajectory of the missile. While drones move at a snails pace relative to reentering missiles, a cloud of them a few miles outside a city might put one of them in position to detonate in the missiles path.

The holy grail of missile defense would be laser air defenses. It is in principle possible to coherently combine enough beams (and acquire the target fast enough) to destroy incoming missiles in a fraction of a second. The prospect of a world without nuclear weapons makes this a worthy challenge.

Cybersecurity

Language models and formal verification promise to make hacking prohibitively hard (though not impossible). Experts believe that AI capabilities systematically favor defense in the cyber realm.

The next step is to prepare for the possibility of quantum computers by adopting post quantum cryptography6. Fortunately, some protocols such as Signal have already done this.

More speculatively, practical obfuscation may unlock a variety of other secure protocols. With the software domain secure, the main vulnerability is physical devices.

Secure infrastructure

Physical stuff cannot be made perfectly secure. Walls must be made of ordinary matter, and even a relatively weak laser can cut through any material given enough time. But it is possible to make it prohibitively expensive to break in to something.

One step in this direction is to use hardware security tokens to unlock (and log in to) everything. These devices can store a much longer password than you can memorize, such that it is physically impossible to brute force. Challenge-response authentication means that attackers can’t get the password from monitoring the communication between key and lock7. This system can be made stronger by using biometrics as your username8.

For stuff like houses or cars, this level of security might be enough to deter attacks. But for more important systems, there’s a risk that attackers try to extract the password from the hardware directly. Secure hardware still trying to establish a paradigm, as many implementations have been broken. FlexHEG is one attempt. Open-source hardware9, from a variety of manufacturers, inspected by IRIS, and wrapped in shielding might deter all but the most dedicated attacks. Further work is needed, particularly algorithms that use a small amount of secure hardware to protect a large amount of computation or memory.

Besides hardware, the energy system as it is currently constructed is pretty fragile. One of the benefits of switching to renewables and batteries is to decentralize energy production. People with batteries can live without the grid for a few days. Ukraine finds itself using more renewables as the war continues because renewables are a lot harder to destroy than a power plant.

One specific risk for the grid is solar storms and EMP weapons. These do most of their damage by destroying the high voltage transformers that underpin the grid. The solution is simple: manufacture more transformers. Failing that, install more SolidGround systems. Other critical systems might benefit from a Faraday cage.

Fortunately, much of our other physical infrastructure (roads, canals, communications, etc) is hard to destroy without engaging in all out war. Traditional defense will suffice.

Space

Space is a shared resource that everyone should have access to. To enforce this right, we must distribute the ability to protect ones own interests in space. Solutions here will have to wait until space colonization becomes more common.

There are two risks we might address in the near term. The first is the risk of space debris making certain orbits unusable. Of the methods I’ve seen proposed, laser systems seem the most feasible, particularly if the system can be operated from the ground. One concept that doesn’t get discussed much is ablating the top of Earths atmosphere or releasing pockets of gas in low orbits to increase drag on the debris.

A longer term concern is asteroids hitting Earth or space colonies. This risk becomes acute if humanity develops the ability to move asteroids, perhaps for the purposes of asteroid mining. The first step is to build more telescopes that can track asteroid trajectories. For objects on course for Earth, many ideas have been proposed. I personally think kinetic impactors and laser ablation are the most promising10.

Protecting digital minds

A society of humans and AIs will be dependent on both. It’s important to build technologies that protect all interests. A few of the above ideas can be remixed to ensure the safety of digital minds.

As a first step, secure computing infrastructure protects the substrate they run on. Distributed computation is the next step, dodging the risk of a single server failure. Encrypted multiparty computation is an extreme version of this, enabling distributed parties perform a computation without knowing the contents.

Presumably, the safest way to get people to run these computations is payment via decentralized currency. Sovereign currencies enable everyone to engage in exchange without the drawbacks of fiat currency. Decentralized finance and contract may also reduce regulatory shocks and stabilize the financial system.

The long-term goal will be to build secure homes for digital people.

Other

There are a few loose ends that didn’t fit anywhere else. Continuing research is needed into these and other threats.

• Volcano geoengineering might be necessary long term.

• Synthetic food would make us more resilient to agricultural catastrophes.

• I’m skeptical that past visions of nanotechnology will come about. But if it is developed, we should build mitigations such as hard-to-digest barriers and nanomoeba11.

• Science fiction is filled with fears of infecting minds via digital media. Humans might fall prey to superpersuasion or be driven mad by a sequence of blinking lights. Digital minds might be subverted by the contents of a single message. If this becomes practical, we’ll need to develop filters, sandboxes, public key blacklists, and other mitigations.

• Institutions for truth may become more important. Prediction markets, mechanism design, reputation systems for arbiters, and credibly-neutral research institutions may come to the fore.

• Mechanisms for funding public goods without relying on governments might increase the speed at which defensive technologies are established.

Conclusion

This concludes a loose thread in writing about proceeding with AI. The benefits are large enough to accept a substantial amount of risk. The industry is moving towards a development paradigm that safely satisfies our demand for intelligence, while sapping the incentive to build dangerous intelligence. In that safer paradigm, using aligned data solves alignment. Adding engineering safeguards, legal protections, and defensive technologies can unlock a world that digital minds can safely participate in.

There is no single solution to the challenges of digital minds, society should push on all fronts. The risks can be mitigated enough to proceed with confidence, and share the future with everyone.

1.

There are some interesting theories about metabolic dysfunction and Alzheimer’s. At this point, it’s doubtful that Alzheimer’s has a single clean cause, but each contributor is a treatment opportunity.

Vascular dementia is the second most common form of dementia. It happens when blood vessels damaged by heart disease become less effective at supplying the brain1.

Vascular dementia is considered distinct from Alzheimer’s, but Alzheimer’s brains also exhibit lots vascular damage. This paper hypothesizes that Vascular dementia and Alzheimer’s are consequences of the same underlying damage to the brain’s blood vessels.

One of the authors on the paper, Jack de la Torre, believes that amyloid plaques observed in Alzheimer’s are a symptom of poor perfusion. They serve to exacerbate the disease2.

This is exciting because at the very least it gives us new way to try to predict dementia risk. For instance, this 2006 paper uses PET to distinguish Alzheimer’s from vascular dementia.

At the other extreme, this paper makes the striking claim that neurons with good blood flow don’t age at all:

Lifespan of neurons is uncoupled from organismal lifespan

The story relating dementia to failures of perfusion fits with the discovery that the shingles vaccine prevents dementia. It’s thought that shingles reactivation can lead to inflammation and damage of blood vessels among other things.

A related line of work connects Alzheimer’s to dysfunctional glucose metabolism. One clue is that people with type 2 diabetes have double or triple the risk of getting Alzheimer’s, with some calling Alzheimer’s “type 3 diabetes”.

Alzheimer’s Disease is Type 3 Diabetes—Evidence Reviewed

Oxidative stress, dysfunctional glucose metabolism and Alzheimer disease

Also interesting is that intranasal insulin temporarily reduces dementia symptoms. This could be because insulin resistance reduces uptake of glucose from the blood3. Exogenous insulin forces the neurons to take up more glucose. See also:

Insulin Resistance and Alzheimer’s Disease: Bioenergetic Linkages

Maybe this is why old people really like sweets. Their brains are not getting enough energy due to vascular problems and insulin resistance. They compensate for the ineffective energy transport to their neurons by raising blood glucose.

In the book Transformer4 Nick Lane relates a theory that neurodegenerative diseases that connects to the theories above. There are some connections between neurodegenerative diseases and diabetes-induced malfunction of the mitochondria:

Sweet Mitochondria: A Shortcut to Alzheimer’s Disease

There is also work connecting neurodegeneration with mitochondrial dysfunction more broadly:

Mitochondria, OxPhos, and neurodegeneration: cells are not just running out of gas

This paper goes further, aiming to explain why this mitochondrial dysfunction leads to the formation of amyloid:

Mitochondria-associated ER membranes and Alzheimer Disease

ApoE e4 gene is a major risk factor for Alzheimer’s and is implicated in all of this. The protein it codes for plays a role in cholesterol transport (thus heart disease), glucose metabolism, and mitochondrial function.

Tying it all together, dementia and other neurodegenerative diseases might happen because neurons are not getting enough energy. Heart disease makes blood vessels worse at their job. Insulin resistance prevents cells from importing glucose from the blood. Mitochondrial malfunction prevents cells from converting that glucose into useful work. The energy deficit results in amyloid production which damages neurons further. Aging exacerbates these problems.

Solving dementia might require addressing every step. Fortunately we’ve seen remarkable progress on heart disease and insulin resistance (GLP-1’s, SGLT-2 inhibitors). We have several monoclonal antibodies to clear amyloid5. A handful of people are taking on aging directly. But perhaps mitochondrial malfunction deserves more attention.

Progress is possible here. Dementia rates have fallen a lot on their own. I’d guess this is due to less smoking, vaccines, and statins:

2.

Defeating death is pretty valuable, in some squishy human sense but also in terms of cold hard cash. The Economic Value of Eliminating Cancer estimates that “[e]liminating cancer mortality generates $197 trillion in economic benefits over 35 years”. Silver Linings has a nice site about estimating the value of extending lifespans, fighting dementia, or slowing reproductive aging.

There’s been progress on the two biggest killers in the developed world, heart disease and cancer. For heart disease see: Matthew O’Connor on Cyclarity’s Successful Phase 1 Trial. Cyclarity has an interesting drug platform and claims their new drug might reverse atherosclerosis. Exciting.

Cancer has seen even more progress see this nice review by Saloni Dattani.

Owlposting covers How to build a cancer vaccine, and whether they will work this time.

Finally, something more speculative, in vivo CAR-T. This is a remarkable technique to deliver genes or mRNA to T-cells in the body, training them to combat cancer. This has the potential to be cheaper and easier than traditional CAR-T.

In vivo CAR-T cell therapy: New breakthroughs for cell-based tumor immunotherapy

By the way, CAR-T and other immunotherapies are finding their way into treating autoimmune diseases. Ruxandra Teslo thinks we might be able to borrow the same tricks to fight aging:

Autoimmune therapies: an inspiration for anti-aging therapies?

3.

EVERY is a company that makes animal proteins from precision fermentation. Jeff Kaufmann says it works in baked goods and DIY Biotech tries it in scrambled eggs. It’s more expensive ($0.53 per EVERY egg white vs $0.21 for traditional) but it’s very promising that precision fermentation was able to get the costs to even approach store bought.

I’m curious about the things you can do with animal-free powdered egg whites. Egg cocktails without the risk of getting sick, more concentrated foams and sauces, shelf stable egg whites in your pantry, etc.

The exciting part is that producing just this one protein (ovalbumin) means we’re pretty close to making synthetic eggs. Egg yolks are just fat, lethecin, and micronutrients. Replacing eggs with something healthier and cheaper would be a big win for animal welfare. Next up is milk, which is just some protein, plus fat6 and sugar (that nobody wants).

On a less appetizing note, you can feed microbes methane to get a food source that isn’t reliant on agriculture at all. This is important for preparing for certain catastrophes. Society’s Resilience to a Total Loss of Agriculture finds that 16% of annual US gas production could feed the population for a year. And US has enough reserves to feed population for 500 years!

We’re approaching a world where synthetic food is possible. The question is whether anyone will eat it.

4.

Civic Mapper is a neat tool to visualize land values in various cities. The image below shows downtown Cleveland.

The land in the purple areas is easily 100x more valuable per square foot than the green areas. The gap would be even larger for outlying towns and fields. This is consistent other data suggesting that urban land would make up the large Majority of land value tax revenue.

Land value taxes are highly progressive. They fall mostly on the rich and on corporations. And they are one of the most efficient forms of taxation.

Everything else

The perfect city? a podcast interviewing Ed Glaeser with a striking claim:

“Forty years of transportation economics at Harvard can be boiled down to four words. Bus good, train bad.”

Glaeser is also a fan of congestion pricing. Good discussion of city design and implications for developing countries. I think Flying cars and AV’s are going to change this a lot.

Abundance from Abroad: Migrant Income and Long-Run Economic Development and pair with: Emigration for Growth. Both argue that emigration plus remittances can significantly boost economies in the developing world.

Malaria is not just a health crisis, it is an economic crisis. Paper suggests a 75% effective malaria vaccine would raise incomes in Africa by almost 10%. Disease burden is one thing that’s keeping sub-Saharan Africa poor. Stack on the recent HIV preventative and the gains might be even higher7. Could this boost unlock long-term growth in these countries?

Is modernization widening cultural differences? Some evidence that culture is becoming more diverse, contra claims about a globalized culture. See also: Music markets remain deglobalized.

Beginner’s Guide to Arguing Constructively and Disagreeing charitably with others offer good advice.

Managing Relationship Decay: “The decline in friendship quality was mitigated by increased effort invested in the relationship, but with a striking gender difference: relationship decline was prevented most by increased contact frequency (talking together) for females but by doing more activities together in the case of males.”

The Unexpected Persistence of John Rawls. Classical liberalism as a meta-theory of ethics where all groups are free to pursue their own ends.

“Everything’s Expensive” is Negative Social Contagion. Sticker prices have gone up, but in real terms things have gotten better.

“… ‘stuff is pricey’ mostly serves to make people feel bad. And in aggregate, this is actually a problem, since it contributes to people thinking “the economy” is bad any time prices are rising, and when people think “the economy” is bad they want to rock the boat and make facially insane political choices.

The internet has increased negative social contagion. You can combat that by not giving in and giving less attention to negativity.

Aurelia has two posts on brain preservation and why they believe preserving the high-level structure of neurons is sufficient for cryonics:

Why do I believe preserving structure is enough?

Does preservation make sense before we know how to revive?

Do you really need a brain at all? Considers cases where peoples brains have been severely damaged yet retain a surprising amount of function. Discusses implications for brain preservation.

Andy McKenzie frames the questions from the above posts nicely: It seems easier to simulate a nervous system than a cell.

Induction of cortical on/off periods in awake mice fulfills sleep functions. Uses light to activate brain regions and mimic some of the benefits of sleep while the mice are awake. Interesting for Sleep need reduction therapies.

Acadesine is an interesting drug. In mice, seems to improve cardiovascular endurance. Even without exercise it activates many of the genes associated with cardio.

Two highlights from List of Biotechnology Companies to Watch – AI Expanded Version:

1. Bexorg which uses post-mortem human brains to do drug development.

2. Cognito Therapeutics Audio and visual stimulation of Alzheimers patients to slow the progression of the disease.

“Human Operator is a human augmentation tool that allows AI to briefly take control of your body to help you learn and do things you normally cannot do.” Hmm.

Systolic arrays for general robotics, AI, and scientific computing. Google TPU's are designed around systolic arrays, an idea from the 1970's:

"... Kung’s group didn’t stop at matrix multiplication, they presented a concept of systolic networks of arbitrary processing nodes that could do way more."

Very interesting, rhymes a lot with agent orchestration stuff.

Sakana Fugu — Multi-Agent System as a Model. Orchestrating different AI models carefully produces better performance than any alone. This accelerates competition between inference providers.

Are we reaching "peak combine-harvester"? Combine harvesters got more efficient by growing larger. But now we’re approaching the size limit due to issues of soil compaction, among other things. The next step is self-driving harvesters.

Some nice discussion of sling launchers (i.e. space tethers) on the moon:

Lunar Sling Launchers

Lunar Sling Launcher - Tether Dynamics

Lunar Sling Launcher - Release Mechanism

But thoughtful criticism can direct effort to more promising fields. To that end, I’m going to try to make my criticism as constructive as possible, with concrete reasons for my pessimism and closely related research areas which are promising (and stand to benefit even from failures in the field I’m criticizing).

Not every section will live up to that standard; some arguments are built on vibes alone. I might be wrong, so I still want to see people work on these problems.

Preliminaries

Exhaustible resources more expensive long-term

The theory of exhaustible resources suggests that “the real price of an exhaustible resource should grow at a rate equal to the real interest rate.” The data are consistent with this hypothesis1.

Innovations in search and extraction can dramatically lower prices in the short term, but in the long term, we converge on a correct way to do things. Once extraction is solved, prices return to their slow upward climb.

Idiot index

Raw materials are just a fraction of the final cost of a good. You need to take out loans to buy the materials. You need land and trucks. You need to buy equipment. You need people to design, build, and test.

(And robots do not solve labor costs. Robots are made of raw materials and equipment and loans. Most importantly, like humans, robots have an opportunity cost, other uses of the robot bid up the price. Robots will have a non-zero “wage”.)

It’s usually straightforward to estimate input costs, but these other costs (land, equipment, wages, etc.) are tricky. I’ll try to get around this with the idiot index: the ratio of the final cost of a good to the cost of the raw materials. An example for metals:

We’ll use the idiot index as a heuristic to think about the hard-to-estimate costs of producing a good. If we know the raw materials cost and an idiot index for similar products, we can guesstimate the final cost. Simpler processes have a lower idiot index while complex processes a higher one.

But don’t take this too seriously, the index I use for different areas is just an educated guess. If It’s Worth Doing, It’s Worth Doing With Made-Up Statistics.

Back-of-the-envelope calculations are inherently optimistic

Armchair estimation of costs is inherently optimistic. Reality has a surprising amount of detail, you often miss major cost contributors until you actually start doing the thing.

This is a feature, I’ll be employing an optimistic cost benefit analysis for the technologies I’m criticizing. I want to make the best possible case before I rule out an idea. But don’t be fooled, even if something looks slightly better on paper, complicated plans often fail in practice.

Preliminaries aside, let’s start critiquing.

e-Fuels

Producing fuel from CO2 capture and then burning it again doesn’t make sense:

… e-fuels and sustainable aviation fuel are silly. Capturing carbon and making fuel is probably more expensive than just capturing carbon. Making e-fuels requires capturing carbon plus making green hydrogen plus using renewable energy plus building reactors plus new fuel pipelines. Capture is simpler and has the same effect of making transport carbon neutral.

Even if it did, it’s hard for e-fuels to beat fossil fuels on price. In this section2 I made some pretty favorable assumptions for green hydrogen and it wasn’t a slam dunk compared to fossil fuels. And DAC CO2 at $100/tonne is more expensive per mole carbon than fossil fuels.

Even assuming $1/kg hydrogen and $100/tonne CO2, natural gas comes out to $1.03/kg, roughly 6x higher than today’s prices. Further in the future, exhaustible resource economics will bite and the fossil price will pass the e-fuel price, but that takes a while.

So methane prices are about as low as they’ll ever be3. There will be price fluctuations, and better tech will help, but eventually we run out of breakthroughs. The economics of exhaustible resources demand that prices rise over the long term.

Instead of fuel, this tech would be valuable for making green ammonia, doing carbon capture, or making synthetic food. See my series on solving climate change for more.

E-fuels aren’t going to save us, but most methane users can switch to alternatives. One application can’t go without it: rockets.

Launch costs below $100/kg to LEO

So e-fuels won’t save us money and methane prices won’t be lower than today. Now we have a lower bound on fuel costs for chemical rockets (which predominantly use methane and liquid oxygen).

Fuel is just one component of rocket costs, there’s engines and staff among other things. It’s hard to estimate prices for these, particularly as technology improves and manufacturing gets automated. That’s where the idiot index comes in.

Treating fuel as the “raw material” that goes into rockets, how many times larger should the final product (launch) be?

Today, fuel for a Starship launch is $1-2 million or 1% of overall costs; an idiot index of 100. SpaceX’s stated goal is to bring total launch cost down to $10 million, an idiot index of 10.

Looking at the idiot index for metals, they sit around 5. For passenger air travel, fuel costs are 15-25% of overall costs, an idiot index of 4-6. Launching and reusing rockets is more complicated than either of these, so an idiot index of 10 is generous.

Now to fuel prices, starting with methane. Looking at historical Henry Hub natural gas price let’s assume $3/MCF. With 19.26 kg of natural gas per MCF, that’s $0.158/kg.

For liquid oxygen, I get conflicting estimates. Wikipedia has a lovely (but outdated) page on Prices of chemical elements claiming that oxygen is $0.154/kg. Let’s be generous and round that down to $0.1/kg, lower than most estimates4.

Using propellant masses here, we can estimate fuel costs for a Starship launch.

Booster fuel cost:

($0.158/kg * 700 tonnes methane + $0.1/kg * 2700 tonnes LOX )/ 100 tonnes payload = $3.81/kg

Starship fuel cost:

($0.158/kg * 330 tonnes methane + $0.1/kg * 1170 tonnes LOX )/ 100 tonnes payload = $1.69/kg

Totaling $5.5/kg in fuel costs. Applying a 10x idiot index, the overall cost is $55/kg. This is the most optimistic case I can make for chemical launch costs.

The fuel price will be higher than calculated here. For instance, natural gas consumers pay over $10/MCF because of markups and delivery infrastructure. Rockets need high-purity liquid methane, which is probably why Starship fuel costs are 2-4x higher than my estimate. I doubt SpaceX5 is leaving money on the table, this is the best price they can get.

SpaceX Falcon Heavy costs closer to $2000/kg despite booster reuse, lots of flight experience, and a payload of 50 tonnes. Costs are higher than the theoretical minimum because this is literally rocket science!

$100/kg is probably the lowest cost we can hope for6. It would still be a revolution in launch, but if we want to go further, we need to start thinking about space tethers and sourcing lunar material.

Asteroid mining

Don’t take my word for it, Casey Handmer says There are no known commodity resources in space that could be sold on Earth. The moon in particular is made of the same crust as Earth, but with fewer geological processes to form veins of ore. There’s little reason to fly there and back.

Perhaps there are precious metals in the asteroid belt? Generally, Earth has more concentrated pockets of metal due to ore-forming processes, but asteroids do have higher concentrations of platinum group metals.

One problem: by the time someone grabs an asteroid, extracts the precious metals, and lands on Earth, the demand for these metals may have collapsed. Consider that “[a]bout 32% of the total Pt, 85% of the total Pd, and 90% of the total Rh were consumed by the automotive catalyst industry”. If we switch to electric cars during the decade they’re mining, the venture may be in the red7.

The broader problem is that asteroid mining might cost more than the price of the materials being collected. Challenges include:

• A large delta V to most asteroids, with the “closest” requiring about 4.5 km/s of delta V from LEO. Compare to 5.7 km/s for the Moon.

• Large financing costs since you need to buy expensive equipment and wait a long time for it to make the round trip. If it doesn’t break.

• Poor solar resources and little fissile material in asteroids, making energy a big challenge.

So maybe mining asteroids for precious metals isn’t the best idea. What about using asteroids for structural material in space? Well, the Moon is 30x larger than the entire asteroid belt, faster to get to, more amenable to space tethers, and awash in energy resources8. So asteroids aren’t great for this.

There’s little reason to go to the asteroid belt9. Establishing the capability to fly to the belt and fling rocks at Earth is also not great. I think people focused on asteroid mining should mine the moon instead.

The broader lesson is that there’s no reason to bring anything back to Earth from space. Leave the mass up there where it’s most valuable.

Space data centers

Let me be clear about one thing: space data centers are entirely feasible from a technological perspective. Starlink already has solar, compute, cooling, interconnect, communications, etc. Nothing stops you from putting computers in space. It’s also entirely possible to make a profit doing computing in space.

But space data centers are not cost-competitive with terrestrial data centers.

People are comparing space data centers on paper to the terrestrial data centers of today. That’s not a fair comparison. You need to look at where progress in terrestrial data centers is going and correct for the optimism inherent to an on-paper assessment.

My argument examines the different parts of a datacenter (chips, interconnect, comms, cooling, energy, etc.) and shows that in space the cost per unit of performance is worse at every step. If every step is more expensive in space, then terrestrial data centers must be cheaper.

Even if you don’t buy my argument, I have some useful things to say about the design of such datacenters.

Chips

This point is relatively uncontested: computer hardware has a lower price-performance in space.

Chip performance is worse in space because:

• Radiation causes bit flips and damages chips.

• Space debris can destroy your data center10.

• Maintenance is more infrequent, meaning more downtime and out-of-date hardware relative to terrestrial.

Chip costs are higher in space because:

• Chips need to be redesigned for launch and the space environment, including radiation shielding.

• Maintenance is more expensive.

• You need to pay launch costs11 to get the hardware into space.

• Additional costs of stationkeeping12.

The size of these effects varies widely. For example, chips are very small, the cost to launch them is minuscule. BUT it’s a non-zero additional cost you pay to put chips in space. This necessarily makes it (slightly) more expensive to do computing in space relative to Earth.

These factors lower the price-performance of computing in space relative to Earth. They apply equally well to other parts of the server such as interconnects, racks, wiring, and so on.

While servers and networking are 73% of data center costs, they’re not the only cost. If other areas become much cheaper in space, then the cost of putting chips in space might be worthwhile.

Cooling

Chips need to be near each other to keep interconnect latency low. These chips take energy from solar panels and produce heat. That heat needs to be dissipated into a radiator.

Rather than look at terrestrial data center cooling costs and speculate about the design of future satellite radiators, consider a simple test:

Imagine we design a radiator for space. We make two copies: one goes to space and the other stays on the ground. Which rejects more heat?

This is a very rough test; but it works for our purposes. If the space radiator has better performance on Earth, we can safely claim that space datacenters have higher cost-per-unit-cooling.

Why? One, because launching the radiator into space adds additional cost and lead time. Two, because a radiator designed for space makes the terrestrial performance look worse. We don’t use space radiators on Earth because we have better options.

In other words, this is an optimistic case for space radiators, enabling us to rule them out if they perform worse in space.

For both space and Earth, radiators enjoy radiative heat transfer, governed by the Stefan-Boltzmann Law:

Where ε is the emissivity of the radiator material, σ is a constant, T is the temperature of the radiator, and T_env is the temperature of the environment.

Earth also enjoys convective heat transfer governed by Newton’s law of cooling:

Where h is the convective heat transfer coefficient and A is the area of the radiator.

Some assumptions:

1. In both cases, the radiators are shielded from the sun’s radiation by the solar panels. The space radiator is also perfectly shielded from the Earth’s radiation. In practice the latter is not feasible because the Earth takes up a large part of the satellite’s horizon, but this is an optimistic case for the satellite.

2. Environmental temperature on Earth is 293 K, environmental temperature in space is 2.7 K (CMB)

3. The radiator is 1 m² of anodized aluminum with an emissivity of 0.9. Emissivity can’t be greater than 1, so there isn’t much room for better materials here.

4. Choosing the heat transfer coefficient is tricky. After looking at some examples, 10 W/m2-K seems reasonable13. Adding a fan would 10x this number but that messes up the cost comparison.

Now we can look at the heat radiated as we change the temperature of the radiator (different from the temperature of the chips).

Terrestrial chips operate around ~75°C. At this temperature, the radiator on earth rejects more heat than the one in space, and the gap grows wider at higher temperatures14. Adding a fan to the Earth radiator (h=100) extends its advantage down to around 25°C.

For a wide range of radiator temperatures, the space radiator works better on Earth than in space. Given the additional costs of sending the radiator to space, cooling is cheaper on Earth than in space.

I’m glossing over details here, like the different temperatures for the chips and the working fluid; optimizations around chip temperature, leakage current, and compressor energy consumption. It’s plausible that we invent chips that make space radiators look better (see discussion of reversible computers). Then again, it’s plausible we invent chips that make the Earth radiator look better.

For now, it’s safe to say that cooling costs will be higher in space than on Earth.

The Earth’s atmosphere is a giant radiator. It emits 174 petawatts into space, seven orders of magnitude more than all AI capacity today15. We’re nowhere close to saturating Earth’s compute capacity. This is why space AI is only interesting if demand outpaces Earth’s capacity to provide it.

See also:

Is It Really Impossible To Cool A Datacenter In Space? - YouTube

A humorous but information dense thread from Andrew McCalip.

Energy

Energy is ~7% of overall costs, so even if energy was free in space, costs would be at most 7% lower16. Given that hardware costs 10x more and should increase in cost in space, that already looks like a bad trade.

There are three challenges for solar in space. The first was covered in the hardware section; hardware generally has a lower cost per unit performance in space including radiation, debris, maintenance costs, and launch costs.

The second is heat. Panels must face the sun and cool radiatively, leading to high equilibrium temperatures. Panels on the ISS reach 70°C in full sunlight, which hurts efficiency. This means using efficient and expensive panels (since any light not converted to electricity becomes heat, higher efficiency lowers equilibrium temperature).

The third challenge is stiffness. Panels need to be stowed in the rocket in such a way that they don’t get shaken apart by the ~40 Hz vibration during launch. Avoiding this requires stiffness which adds mass and cost. Once deployed, there are limits to how thin the panels can get. Varying gravitational forces can induce thin panels to flop and moving parts on the satellite can drive vibrational modes.

Now for a calculation:

1. The panel is in dawn-dusk SSO with constant illumination at 1362 W/m2.

2. A 5 year lifespan before the computer hardware has to be swapped out (with the upfront cost of swapping out the chips and the risk of panel failure, it will likely be more economical to replace the panels at the same time).

3. The GaAs panels are 30% efficient at 70°C

4. A panel mass per meter squared of 3 kg/m2. This is slightly heavier than the panels on the ISS, but accounts for additional structural mass required in the fairing and during deployment.

Result: $16.76/MWh before accounting for the idiot index. What index should we use to account for panel costs, design, assembly, and testing? Surely a factor of 3 is generous? The resulting $50/MWh is comparable to terrestrial solar today, but solar costs will continue to fall (see section on solar costs).

Space solar will continue to improve. More efficient and thinner cells may be achievable. Instead of areal density, let’s look at specific power. This page has a large list of different space panels and their specific power. It seems like 200 W/kg is an optimistic goal. A 5-year lifetime gives us $34/MWh with a 3x idiot index. Once again, probably not better than terrestrial solar.

It’s possible that new designs will improve the specific power enough to outpace advances in terrestrial solar, but once again, cutting down the small fraction of energy costs while damaging the hardware does not look promising.

Communications

Satellites are good for long distance communications. Light travels faster in a vacuum than in optical fiber, and satellites enjoy long lines of sight. So if you’re trying to communicate with someone far away, satellite internet works well.

But for AI inference, it’s easier to park compute close to users on Earth than in space. This is particularly true for tasks that demand extremely low latency. Robots, drones, and autonomous vehicles will need compute onboard, excluding space AI from these lucrative use cases17.

It’s a closer race for generative AI running in distant data centers. But space AI still struggles to compete. While Starlink latency is comparable to cellular data and only slightly slower than fiber internet, it has lower bandwidth and higher cost for most users. These facts are unlikely to change, Starlink is not planning to beat traditional internet on cost or bandwidth.

AI in particular adds new latency challenges. Satellites in LEO stay visible by your antenna for ~5 minutes. They spend some amount of time flying over oceans and sparsely populated areas. To avoid low utilization, queries need to be dynamically routed to different satellites. This problem is solvable, but the routing means that responses take longer to reach users relative to terrestrial.

So once again, the cost performance is worse than terrestrial counterparts. This is particularly true if we start parking data centers in cities and running more fiber.

Permitting

A common argument for space compute is that the permitting challenges are easier. I have yet to see an argument for why permitting is somehow easier when building launch pads, firing off hundreds of rockets, and filling up limited orbital slots. Rocket launches have permitting too! Indeed, there is more bureaucracy in launch because you have to deal with local ordinances, aviation restrictions, communications law, and international law18. As we learn more about launch pollution, restrictions may tighten further.

I’ll edit in an additional response here if I see a good argument for why permitting will be easier for rockets than laying down solar panels.

A better way

This criticism lays the groundwork for a better approach to space compute.

For example, people are talking about increasing the chip temperature in space to decrease the size of the radiator. This is silly because increasing chip temperature increases things like leakage current, thus raising energy cost per FLOP and counteracting the benefits of a smaller radiator19.

Go the other way! As we saw in the cooling section, space radiators look better at low temperatures. Launch costs are proportional to mass, and the mass of the hardware and radiators is small relative to the panels. You can lower the energy cost per FLOP if you cool the chips or do reversible computing. The panels can shrink while the radiator grows more slowly20, lowering mass in net.

Reversible computers create an advantage to space over Earth. Space is cold, it’s easier to keep your chips cold in space than on Earth. That’s why reversible computing featured prominently in my post on the end of semiconductors.

What should space compute be used for? Onboard processing of satellite imagery, military intelligence, telecommunications, and stock markets in space are all far more profitable opportunities than AI inference. See that linked post for other ideas.

If you insist on AI inference in space, the economics push for large numbers of chips linked together with lots of fiber (perhaps spread out on the back of the solar panels). A single massive datacenter in orbit with satellite internet gathering requests from across the globe makes the most sense21. The latency of this compute means it’s best for the low speed, low cost niche suited to autonomous AI workers.

So … space AI?

To be clear, it’s entirely possible to compute in space. It’s also entirely possible to make a profit computing in space. But it probably won’t be cost competitive with terrestrial datacenters. Every part of the tech stack has a lower performance in space relative to terrestrial.

You could fight this conclusion by stacking tethers and lunar materials and reversible compute, but then we’re not making a fair comparison to terrestrial compute. Terrestrial compute has a lot of opportunities to improve22. By the time we build space infrastructure, we might be synthesizing solar from dirt or using a laptop AI to do inference over a hypercompressed local internet.

Space doesn’t need to be profitable

My fellow space enthusiasts, your dreams don’t have to be profitable to be worthwhile. For most endeavors, AI will teach us that anyways.

Going to space provides little economic benefit for those who are content to stay on Earth23. It’s not clear that even the promise of a Dyson sphere could coax our economy into space24.

Instead, we will go to space out of desire, not necessity. Because we enjoy such widespread abundance on Earth that anyone can reach the heavens.

Further reading on space data centers, with an emphasis on good technical arguments and actually doing math.

Skeptics:

Economics of Orbital vs Terrestrial Data Centers

Do Orbital Data Centers Make Sense? - by Andrew Cote

The Truth About SpaceX’s “Orbital Datacenters” - YouTube

Notes on Space GPUs - by Dwarkesh Patel

Will we really put data centers in space?

The Truth about Space Data Centers. This one is particularly good because it brings up a few issues that other people don’t mention.

Casey Handmer is more neutral here, though he thinks space AI would be 2x more expensive than terrestrial:

His Orbital inference Spreadsheet.

Direct Current Data Centers – Casey Handmer’s blog

Space AI: I guess we’re doing Moon factories now – Casey Handmer’s blog

SpaceX’s AI Data Centres Might Actually Be A Good Idea. Here’s Why - YouTube

Semianalysis discusses their paywalled model, finding costs 4x higher than terrestrial with the potential for parity in 2040.

To Boldly Go: The Case for Space Datacenters

They debunk a lot of arguments made for space datacenters. They don’t discuss launch cost assumptions in detail, but I suspect their 2040 case relies on costs approaching $100/kg.

Optimists:

Space Intelligence

Can We Build AI in Space?

Why Space-Based AI Data Centers Are Inevitable: 3 Levels of Analysis

Ramjets and demand for Mach 3+ flight

After my post on rocketplanes, I looked at hypersonic flight times between major cities. Mach 3 can get you between most major cities in under 3 hours.

Mach 3 is achievable with existing tech. The SR-71 blackbird used a turbofan with afterburner for an official top speed of Mach 3.3 (the real number is likely higher). In this regime, turbofans are more efficient than advanced concepts like ramjets.

So Mach 3 is fast, fuel-efficient, and uses existing tech. Is it worth the headache of going Mach 5 if you only shave an hour or so off your flight time? It’s not even clear the time savings will materialize if a Mach 5 plane needs more pre-flight checks or has to stop more frequently to refuel.

I’m still excited about companies like AstroMechanica, it’s great to see people try new things here. But after the dust settles, the future of passenger air travel looks to be in Mach 3, improving traditional flight, and flying cars.

Passenger rail

Trains are great for moving stuff over land, but moving people? Nope.

As Eli Dourado says, trains are not an abundance technology. If you want to go long distances quickly, an airplane can get you there faster and cheaper. And without the headache of building massive amounts of track with low utilization.

What about public transit in dense urban areas? Well, Ed Glaeser says:

Forty years of transportation economics at Harvard can be boiled down to four words. Bus good, train bad.

Simply maintaining roads, optimizing bus stops, and using congestion pricing to maximize throughput will produce a cost effective transit system. Ride-sharing autonomous vehicles and busses are about to make the case for roads even stronger.

Fusion

Fusion generates energy by colliding two atomic nuclei and releasing energy in the form of neutrons, electromagnetic radiation, and charged particles. The radiation can be converted to energy in two ways. One, charged particles can generate energy via direct energy conversion. Or two, X-rays, gamma-rays, and neutrons can be absorbed by radiation shielding, producing heat25.

We can probably rule out the second method. Fission reactors already do this using simpler, de-risked technology. They’ve been unable to compete with other forms of energy generation26. Using fusion to produce even-more-expensive neutrons is not promising.

Direct energy conversion is interesting. Avoiding steam turbines is a big reason why solar is cheap. Maybe fusion could be cheap too.

I’m going to focus on D-D fusion for our optimistic analysis. While few companies are pursuing this (it’s harder than D-T fusion), it looks better for producing charged particles and the cheaper fuel lowers the on-paper cost. Assumptions:

1. The reactor performs Deuterium-Deuterium fusion without fuel reprocessing. When including side reactions with Tritium, this produces 8.35 MeV from 5 Deuterium atoms. That is 1.67 MeV of charged particles per Deuterium atom. Much of this energy comes from side-reactions with Tritium. This is the largest amount of energy a D-D reactor could produce from charged particles alone.

2. Charged particles can be converted into electricity at a 50% efficiency (probably higher than achievable at scale). Each deuterium atom produces 0.835 MeV.

3. Deuterium costs $4K/kg. This is highly uncertain, I’m basing it off of an out-of-date Wikipedia page. See the footnote27 for more.

4. An idiot index of 100. Also tricky to estimate, but the idiot index for fission plants is 10 and fusion promises to use much fancier equipment and more technical staff.

5. We ignore the value of energy produced via neutrons from the D-T side reaction. Traditional fission plants have not been able to generate energy cost effectively from such sources despite decades of effort.

6. We ignore any recirculating power required to maintain the fusion plasma.

The result is electricity at $36/MWh, with a wide range of uncertainty28. That’s pretty good, comparable to an optimistic estimate of solar costs today (see next section). But fusion needs decades to mature, solar will be cheaper then.

I’ll admit, this estimate is not very rigorous, but it’s the clearest argument I can make for why I’m not excited about fusion energy. Brian Potter came to a similar conclusion:

Even if we could produce a power-producing reactor, fusion will never be anywhere near as cheap as simpler technology like the combined-cycle gas turbine, much less future technologies like next-generation solar panels or advanced geothermal. By the time a reactor is ready, if it ever is, no one will even want it.

(See also: Fusion power experience rates are overestimated)

What about using fusion power for rockets? Unfortunately, the energy density of fusion reactions is only ~2-4x higher than fission reactions, even without the higher induced mass for a fusion reactor. For space propulsion, fission fragment rockets or antimatter are more promising.

Are there better uses for fusion research? Well, proton-boron inertial confinement fusion should enjoy much lower fuel costs, I’d like to see more work here. The economics depends critically on the cost of particle accelerators and Bremsstrahlung losses.

The ability to control plasmas has applications elsewhere: MHD aerobraking might lower launch costs and wakefield plasma acceleration might give us cheap particle accelerators for research, lithography, and cancer treatment.

So while I’m skeptical that fusion will change the energy landscape, I’m happy to see people working on it.

Non-solar energy sources

Solar is very cheap; falling battery costs are fixing the intermittency issue.

All existing energy technologies are more expensive. There’s little prospect for them to catch up with solar today, much less the solar a decade from now. No proposed technology beats solar on paper, and in practice the costs will be higher.

To make a fair comparison to other proposed energy sources or space compute, we need an optimistic BOTEC for solar. Assume solar panels cost $0.4/W and batteries cost $50/kWh (China is already achieving these numbers). Assume we need 5 kW of panels and 17 kWh of battery capacity to achieve 1 kW of supply with 97% uptime.

For a 25 year design lifetime, we get $13/MWh. The idiot index for a future solar farm should be low, as the industry is moving towards containerized batteries and simple racking systems. The 2020 cost breakdown below isn’t perfect, but suggests an idiot index of 3-4. So at today’s prices the optimistic estimate would be $39-$52/MWh.

These costs will continue to fall as solar scales to terawatts. CATL claims its sodium ion batteries cost $19/kWh at the cell level, and aims to reach $19/kWh at the pack level in the future. I see many other opportunities to make solar cheaper using robotic installation, pre-assembly, thinner panels, multi-junction cells, metasurfaces, recycling, and incorporating solar energy into solar manufacture.

$20/MWh is achievable without resorting to science fiction, no other technology can say the same.

Further reading

How to Make Off Grid Data Centers Affordable - Austin Vernon's Blog

Simple Solutions Power Solar's Advance - Austin Vernon's Blog

Expanding the Universal Marginal Energy Source - Austin Vernon's Blog

Future of Energy Reading List – Casey Handmer's blog

Quantum computing

Consider what Scott Aaronson, preeminent quantum computing researcher, has to say here:

The trouble for the optimistic vision is that the applications, where quantum algorithms outperform classical ones, have stubbornly remained pretty specialized. In fact, the two biggest ones remain the two that we knew about in the 1990s:

1. simulation of quantum physics and chemistry themselves, and

2. breaking existing public-key encryption.

#2 is nearly obsolete. The Signal protocol already uses cryptography that’s expected to be secure to quantum computers. Ethereum and other cryptocurrencies are in the process of switching as well. The world simply needs to update their software29.

Once old cryptography is broken, old internet and personal data will be revealed. This might cause a few diplomatic incidents, but it’s not a big deal. Though breaking modern cryptography seems like a robustly bad thing to accelerate!30

Now on to simulating chemistry on a quantum computer. This would be interesting, but it’s not clear how useful it will be. We already have good methods of simulating chemistry on classical computers. Quantum computers will be far more expensive and limited in scale for an unknown improvement in accuracy.

I’m also skeptical of unconventional computing more broadly, though I’m holding out hope for reversible computing in some applications.

Gene therapies in humans

Gene therapy is hard to do, hard to reverse, prone to error, and immunogenic. Most traits are virtually omni-genetic, meaning that you have to change lots of genes to get the desired effect. It is often hard to even identify the relevant genes or causality, much less target and change them. There have been many expensive clinical trial failures here.

Contrast with the wonders of small molecule drugs, which avoid many of these issues. Worse, consider all the modalities that might substitute for gene therapies: RNAi, RNA-targeted drugs, vaccines, antibodies, CAR-T, in-vivo CAR-T, stem cell therapies, and 3D printed organs. Each has an advantage over gene therapies in some aspect, and many are likely to be cheaper.

That said, monogenic disorders like Sickle cell, Tay-Sachs, Hemophilia, Duchenne Muscular Dystrophy, and Huntington’s are good directions for human gene therapy.

(Of course, genetic modification in plants and animals dodges many of my concerns and seems promising)

Brain-Computer Interfaces for healthy adults

Brain-computer interfaces are great for addressing disabilities. But it’s not clear that healthy humans can use BCI’s to output far more information than they already do.

Across a wide range of domains, we seem to be bottlenecked at 10 bits/s information output. Combine typing, speech-to-text, and eye tracking and you’re probably near the limit.

Things to work on instead:

• Ultrasound neuromodulation for adjusting mental states

• Increasing information throughput into the brain via VR, ultrasound tactile stimulation, digital smells and so on.

• Subvocal recognition

• EMG wristbands

• Highly interactive voice models

“Nanotech”

Philosophy spawned many disciplines and gets none of the credit. Once a field becomes a successful scientific endeavor, it gets called something other than philosophy.

Nanotechnology has the same problem. Chemistry, semiconductor manufacturing, and biotechnology are nanotechnology in their own right. We can build complex molecules, pattern arbitrary materials with nanoscale precision, and direct nanobots to modify our bodies. But we refuse to call this nanotechnology because it already exists.

These fields will continue to change the world. My skepticism lies with the nanotechnology that remains.

Early visions of nanotechnology had tiny robots that could carry out precise tasks. In harsh conditions and high temperatures, such robots are infeasible; their fine structures would degrade quickly, limiting their value. If you’re willing to accept standard conditions, proteins are a remarkable nanotechnology that can perform almost any desired physical transformation. No need to invent a new field when biotech can do it all.

For stuff we have today, it makes more sense to build it directly than to try to build it with nanotechnology31. Say you want a car. You could feed the bots some metal, and they could assemble a car, but isn’t that just a factory?

So maybe the nanobots go out into the environment and gather metal. They need to be self-repairing and have some sort of search function and we need some way of beaming energy to them to break chemical bonds. And they need to transport all this metal to a concentrated location and assemble it into a car.

Instead of tackling dozens of near-impossible technical problems, you can just build a car the old-fashioned way: processing many atoms of ore simultaneously, arranging many atoms of metal all at once.

Perhaps there are some things we simply can’t make with current techniques? Not really, as discussed in the next section, we’re already approaching the limits of what’s possible in materials science. For computers, one of the few applications that requires nanoscale precision, lithography will take us to the limit of what’s possible. Nanomachines need not apply.

It’s in lithography that the last holdout of nanotechnology, atomically precise manufacturing, shows the most promise. The throughput and equipment costs are too high to replace industrial chemistry, but APM could be used for producing lithography masks, fixing defects, and metrology.

EDIT: more from me on Nanotech for energy.

“New materials”

Structural materials

I watched a talk by a materials scientist where someone asked him “what material is the real-life equivalent of Vibranium?” The audience member expected him to choose something exotic like graphene or high-entropy alloys. But nope, he chose steel.

Steel is incredible; it’s extremely tough and possesses a strength-to-weight ratio that rivals aluminum in practice. It’s also cheap and can be made with a variety of properties and specifications.

Concrete is also an incredible material. You can ship some dust to a construction site and mix it with water to get instant stone with high compressive strength, consistent properties, and low cost.

And then there’s wood, a material synthesized from air and sunlight with high strength-to-weight, low cost, and integrity against fire (unlike plastics). It’s easy to cut to shape and you can improve its properties substantially with chemical modification.

For bulk structural materials, I don’t think we’re going to beat these three on cost per unit of performance any time soon. They’re local maxima that take advantage of available resources.

For alloys, we can only paint with the periodic table, and we have to make these out of Earth-abundant elements while obeying constraints on what elements alloy with each other. Much of this space has been explored, resulting in superalloys with incredible properties that make things like air travel possible. We’re pretty close to the ceiling here, though high-entropy alloys deserve more attention.

High strength-to-weight materials

When researching space tethers, I realized that there are very few options for high specific strength fibers. You pretty much have to make it out of a carbon allotrope or some sort of glass. For this reason, my excitement over discovering new tether materials waned32.

Electrical materials

Pure elements are generally more conductive, and copper is the ~highest conductivity element in practice. It’s also cheap and recyclable (aluminum works well too). It’s unlikely we’ll replace our power lines and wires with something else.

That said, superconductors are interesting as a new electronic component, particularly for reversible computers33. I’m skeptical of analog computing, but if it takes off memristors will play a central role.

Optical materials

Glass is pretty great for optical fiber. Earth is made of the same stuff and the fibers can beam information accurately over ~100 km. The main contender here is ZBLAN fiber, also interesting because it might need to be manufactured in space.

For optical computing, the main challenge is making materials with consistent optical properties at scale. Fortunately, many familiar materials in semiconductor devices have good properties as waveguides. For the moment, putting exotic new materials on a chip runs counter to progress.

That said, optical metamaterials are pretty cool. The opportunity to replace precision-engineered lenses with something that benefits from the enchippening is exciting.

Catalysts

Many of the most important chemical processes34 are the product of decades of research, simulation, and testing. A big advance in major petrochemical processes is unlikely; we mastered organic chemistry in the late 20th century.

There are still some opportunities here. I’m particularly interested in nanocrystals, modular peptides, and nanochannel glass materials as catalyst platforms. The shale revolution made methane a viable feedstock, raising the status of methane activation and methanol-to-olefins processes. Cheap renewable energy might make new electrochemical processes viable. Recycling atoms from crop residues, plastics, or waste is also interesting.

So … new materials?

Hopefully this hand-wavy tour gave you a sense of the frontier. The same issues are repeated across chemistry and materials science, namely that we’ve found cheap and effective solutions in most applications.

The frontier of materials science looks more like “try lots of existing materials and processing conditions for your specific application” rather than discovering some new composition on a computer.

EDIT: see also AI for Materials Science and The Physicists’ Grift which makes similar arguments focused on applications of superconductors.

Economics of home robots

Robots in the home have to walk a knife edge. Labor-saving innovation is valuable if human time is valuable, but if robots are very competent they make human labor cheap. So you need a weird situation where human labor is super scarce but also replaceable with robots but only in the home.

Outside of work, people spend 4 hours a day watching TV and 7 hours a day sleeping35, limiting the time savings from home robots. Historically, labor saving in the home has come from specialized items like the dishwasher. Robot vacuums were supposed to be the next step here, but the market has been lackluster.

The economics of robots become much better in an environment designed to use them all the time. Instead of folding laundry once a week, run the robot 24/7 in a laundromat. Everyone can get rid of their washer and dryer.

One of the biggest opportunities is food prep. Robotic ghost kitchens can churn out healthy, high-quality meals at scale. It would be far more expensive to have a dedicated robot and kitchen to feed you a few times a day.

(As an aside, there are reasons to be skeptical of the humanoid form in general. Robots will be adapted to their task, we’ll see an explosion of different designs.)

Conclusion

Consider this post via negativa for technology. Skepticism about one idea is optimism for something else.

Instead of fusion, our society will be rebuilt around solar. Instead of trains, hypersonic planes and flying cars will pull the world closer. We’ll build cozy cities afforded by self driving cars and hyperlogistics. And even if we never see AI in space, the AI of today is enough to automate anything.

Generally, I favor of limited government intervention in the economy. For instance, I’ve proposed subsidies and regulated marketplaces as alternatives to sweeping mandates.

One challenge to this viewpoint is the existence of monopolies1. A textbook example of inefficiency. Surely governments should have the power to regulate here?

In practice, the cure is worse than the disease. There are many ways to ameliorate the problems of monopoly without antitrust.

Regulations often make things worse

The first step to addressing monopoly power is removing the ability of the state to create it. The opportunity to restrict competition via legislation has given us regulatory capture, occupational licensing, and tariffs. Each reduces competition and harms consumers.

Government attempts to increase competition produces further harms. Antitrust regulations requires productive capacity from legislators and businesses that could’ve been employed elsewhere. The threat of antitrust litigation also reduces the incentive to innovate or pursue economies of scale. But most importantly, the power to regulate monopolies produces rot. Giving the government power to control private decisions leads to abuse by partisans and powerful interests.

Addressing monopoly without regulation

Even without regulation, there are many ways the government can address monopoly power. Simply allowing free trade and maintaining low regulatory burden can increase competition dramatically. The formation of a monopoly creates a strong motive to compete.

Or, because Monopolies don’t make enough money, subsidies and vouchers2 can be used to reduce the deadweight loss3. Establishing systems for voluntary price discrimination can do the same. Alternatively, the government can directly intervene to make the market more competitive by funding competitors, producing the scarce good directly, or funding R&D to make the monopolized good obsolete4. As a last resort, it may consider engaging in collective bargaining; buying the monopolized good and distributing it to citizens as a public resource.

For monopolies localized to a small jurisdiction (e.g. utility companies), it seems appropriate for the jurisdiction to consider auctioning monopoly rights, municipalization, or regulation. In these jurisdictions, people have the opportunity to vote with their feet, reducing the risks of regulation.

Conclusion

The case for antitrust regulation rests on a knife edge: the state needs just enough power to break up monopolies without creating more. It’s better to avoid the risk by constraining the government’s ability to control private entities. The monopoly problem5 can be addressed without regulation.

Ironically, it may still be valuable to give the state power to preserve certain monopolies. Patents and copyright, for example, might encourage innovation. Unsurprisingly, these are also areas where state power is abused. But striking the right balance will be the topic of a future post.

Further reading

Chapter viii of Capitalism and Freedom.

Monopolies don’t make enough money

Are Monopolies Ever Good?

International Cartels

AI does what the training data tells it to do

If an LLM sees “2 + 2 = 4” in the training data it will predict “4” for the question “2 + 2 = ?” If a vision model sees cats in the training data, it will label the cats it sees in the test set.

Neural network training Just Works. If it didn’t, we would be talking about a different training paradigm that Does Work. It took a lot of research and money to get to this point; now we have a general purpose recipe for solving any task.

Models are only as good as their dataset. Data imbues a model with capabilities and behaviors. The corollary is that models do not learn things that are not part of training. Training on one thing does not magically cause them to do some other thing1. In other words, training chisels cognitive grooves into an agent2.

This is very good from a safety perspective.

AI generalizes outside of the training data

In practice across many domains, AI models generalize outside of their training data. What starts as over-fitting crystallizes into a general solution to the problem. This is often achieved by providing enough diversity in the dataset to ensure that the learned rule is close to the ideal one.

Everyone agrees this works in practice, but does it work in theory? There has been some research progress here, with Singular Learning Theory providing a stylized model for why neural nets generalize to unseen data. And we’re moving closer to an understanding of out-of-distribution generalization34.

The Natural Abstraction Hypothesis considers another angle, asking if there is a deep reason why neural networks seem to converge to the same concepts as humans. If so, we should feel more confident that models understand what we mean in some deep sense.

If this research bears fruit, it would make me feel pretty good about deploying AI models in domains that are slightly-out-of-distribution. A cleaning robot trained in LA and deployed in NY will plausibly be fine. Operating out of distribution is unlikely to produce catastrophic failures. Any issues that do arise can be patched with updated training data.

All tractable alignment problems solved by aligned data

Consider different types of alignment:

User alignment: The model does the things that users (people, companies, AI agents, governments, etc.) want.

Bystander alignment: the model does things that non-users (i.e. other people, other governments, etc.) want or at least does not harm non-users.

Creator alignment: The model does what the inventor wants.

Societal alignment: The model follows some global social choice function. Alternatively, the model only pursues Pareto improvements.

Values alignment: The model does what we really want rather than what we say we want. For instance, the above versions of alignment may fail to implement our true values because of commitment issues.

Many of these are impossible. It’s impossible to satisfy the needs of two enemy states or to solve the various no-go theorems in social choice.

But any single alignment problem can be solved with the appropriate data. Simply create a dataset of what you want the model to do (and not do)5. This has worked for every problem we’ve tried6.

Since models generalize and don’t tend to have catastrophic behavior OOD, it’s safe to test them in a real-world environment. Successes and failures in deployment can be turned into better training.

Safeguards, specialized models, and defensive technologies

Good data can solve alignment problems in principle, but in practice training and safety evaluations will be imperfect. To address this, we can wrap the AI in additional safeguards. For example, training gives us good statistics about the typical input distribution. It is straightforward to flag inputs that are out of distribution and pause the model7 in those cases. Mechanistic interpretability will soon offer a variety of additional tools to enhance safety8.

These efforts are complimented by encouraging the development of specialized AI models rather than general intelligences. Simple, performant models reduce the economic incentive to build AGI, a boon for safety.

We can go further and develop defensive technologies to address large-scale risks. AI models for cybersecurity, lasers for nuclear risk, far-UVC for airborne pathogens, and so on.

Legal system

It would be nice to do something about the other alignment problems, even if they’re impossible to completely address.

For example, an AI aligned with one person might infringe upon the rights of another person. While it’s impossible to prevent these conflicts entirely, a system of torts can discourage such violations and provide compensation to the aggrieved party.

More broadly, a system of market rights, contract, and law has proven effective for addressing conflicts between misaligned humans. We need to extend these systems to an economy with digital minds.

At an international scale, treaties and diplomacy are probably the best we can do to address state conflicts.

Challenges

There are two big challenges to this approach to alignment. The first is ethics. We’ve been talking about the needs of things external to the AI, but what about the needs of the AI itself? And what are the ethics of creating something with agency and preferences?

The second challenge is superintelligence. Will this patchwork of training and safeguards stand up to extremely capable models? Tentatively, I think it is possible to create a society that powerful AIs (and other agents) would prefer to join rather than rail against. But that’s a topic for another time.

Conclusion

The prospect of alignment by default is looking pretty good these days. Data, safeguards, defensive technologies, and the legal system can lower the risks enough to move forward with AI development. With engineering effort and policy change, the world can safely enjoy the benefits of AI.

Further Reading

Alignment By Default by John Wentworth

Alignment By Default? - by Harry Law

Rohin Shah thinks catastrophic alignment is unlikely

Cardiac atheroma at the limits – are we fighting the final battle? A 2020 talk by Peter Libby, leading researcher on heart disease. Does a nice job of mapping out all the paths we have to prevent heart disease.

One of the main categories of heart disease is atherosclerotic heart disease. This is caused by plaque buildup in the arteries. Every adult has these plaques growing in their blood vessels, but high blood pressure, high cholesterol, and damage to the vessels1 accelerates plaque buildup.

To date we have several medications2 that can bring blood cholesterol down with statins being first-line and new therapies such as PCSK9 inhibitors showing promise.

With the success here, research has moved to other means of reducing cardiovascular risk:

• Reducing inflammation, the subject of several trials, with colchicine showing promise and general interest in inhibiting IL-6. See Libby’s more recent presentation on this here.

• Insulin resistance, addressed by drugs like GLP-1 agonists and SGLT-2 inhibitors3.

• Specialized treatments for clonal hematopoiesis; mutant cells in the bone marrow that produce mutant white blood cells.

My main takeaway here is that research on heart disease might end in some sense. More and more, the leading therapies look like anti-aging interventions. Many forms of heart disease seem to be caused by aging of vascular tissue. Once the non-aging problems are solved, the field might get absorbed into longevity research.

See also point 1 here for more CVD research.

Related: Enlicitide decanoate - Wikipedia. A promising cholesterol drug!

2.

There’s lots of talk of existential and catastrophic risks, but Croissanthology wonders, Is the world actually that vulnerable? And comes to the conclusion that no, it’s not. In many cases we have straightforward things we can do to reduce the risk, and good reasons to think that the worst case isn’t all that bad.

For instance, Croissanthology has a nice post on Solar Storms and how we can basically address their risks by installing lots of SolidGround systems (or building more transformers).

Croissanthology also has a post about how LLM cybersecurity risks in the long run are probably exaggerated. This is similar to comments by David Dalrymple that defense “just wins” as AI capabilities advance. This paper makes a similar point:

Deception and Detection: Why Artificial Intelligence Empowers Cyber Defense over Offense | International Security | MIT Press

I found the above paper when reading the transcript of this podcast, which allays worries that AI might somehow destabilize our current nuclear stalemate:

Sam Winter-Levy and Nikita Lalwani on how AI won't end nuclear deterrence (probably) | 80,000 Hours

On the other hand, I think that laser weapons will destabilize the current nuclear paradigm … in a good way! Namely, I think laser defense will essentially end the threat of nuclear war.

[On a related note, here’s a nice review of the current state of combining laser beams coherently4: Towards Ultimate High-Power Scaling: Coherent Beam Combining of Fiber Lasers]

So if solar storms and cyber risk and nuclear war aren’t all that bad, what’s left? Pandemics. Abhishaike Mahajan has a nice post on this:

Reasons to be pessimistic (and optimistic) on the future of biosecurity

I find a of optimism from this post, bioweapons are genuinely hard, there’s little incentive to use them, and AI might genuinely accelerate defensive technologies like mRNA vaccine production. Far UV-C and glycol vapors might drastically reduce airborne transmission (though finding someone to pay for it is hard).

Related: Vincent Conitzer article on addressing misaligned AI by “[giving] the AI a (primary) goal of being turned off”.

EDIT: These two posts conclude that “… there are no new more powerful explosions on the horizon, or, with the potential exception of false vacuum decay, probably ever.”

Destroying the universe: How hard can it be? On false vacuum decay.

(Don't fear) the strangelet. On other possible uses of physics to destroy stuff.

Everything else

The Tiling Tree Method and The Tiling Tree Method, Part 2. An interesting way to exhaustively list possible solutions to a problem. I pay attention to any ideation technique that Ed Boyden uses.

A small polymerase ribozyme that can synthesize itself and its complementary strand. This is huge for understanding the origin of life. This strand of RNA can reproduce itself given short RNA blocks as “food”. Now we need a good understanding of how RNA blocks can form and why these blocks might accidentally assemble into something self-replicating.

Prescription strength: musings about the past/present/future of sarcopenia therapeutics. New drugs might allow anyone to have a muscular physique with only a little effort in the gym. That would be revolutionary for health outcomes.

Subvocal recognition allows you to convert your inner voice into text. Alter Ego is building a wearable device for this. Imagine controlling computers with your mind!

The First Multi-Behavior Brain Upload claims they uploaded a fruit fly. More skeptical takes from and Ariel Zeleznikow-Johnston and Dan Turner-Evans.

"I Found My People!" On finding a community that brings you joy.

Related: Make Your Own Bubble in 10 Easy Steps - by Bryan Caplan

Some relationships deepen when you tell the truth and some end. Committing to being honest has been valuable for my relationships too.

Why We Have Prison Gangs. Systems of government spring up in surprising places. See also Chris Blattman’s work on peace amongst different gangs.

the lottery of career success: or why you may want to bribe OpenAI. Because it’s hard to measure performance, career success often snowballs based on past prestige, which is substantially random.

We may miss the sweatshops. Developing countries used to escape poverty by moving into manufacturing. If AI automates many tasks, these countries will need to find a different path.

Inside the Farm Growing 3.5 Million Pounds on Half an Acre. Nice tour of a vertical farm growing leafy greens. This would pair well with off-grid solar and batteries. I’m coming around to the view that vertical/indoor/hydroponic farming can work well for veggies, while staple crop agriculture is hard to improve upon.

Everything you ever wanted to know about Roblox, but were afraid to ask a 12-year-old. The metaverse already exists, with all the good and bad that entails.

Sunbeam: Near-sun statites as beam platforms for beam-driven rockets. Proposal for new form of interstellar travel using particle beams. The main challenge is efficiently converting the beam into propulsion. One of the authors did an interview on this work here.

AI:

Doc-to-LoRA: Learning to Instantly Internalize Contexts. Load a document into a language model by modifying its weights.

Can LLMs Be Computers? Modify a transformer with a different attention mechanism to load algorithms directly into model weights!

Post-Training 50x Faster. Open-source code to fine tune Kimi-K2 using LoRA.

We mildly deprived ourselves of sleep (5-5.5 hours, relative to 7-7.5 hours normally) and took either a placebo or orexin intranasally. We tracked our sleep the night before and after taking a dose in the morning and completed various tests of mental acuity during the day.

The results from our initial experiment are exclusively null results that don’t cross standard thresholds for statistical significance. Not that this was particularly surprising, we expected a ~60% chance of this happening. We’re considering next steps, and need your feedback!

For now, there are a few things to cover in the results.

Trial Design

We performed a self-blinded randomized controlled trial with blocking, each participant took either the placebo (2.5 mL of sterile water) or the orexin (100 μg of orexin-A dissolved in 2.5 mL of sterile water). Here’s the procedure, repeated for every block:

1. Prepare two nasal atomizers, one with saline solution and one with orexin+saline solution

2. Night to the first day: Sleep 5-5.5 hours.

3. First day:

   1. At a consistent time of day, randomly select one and administer.

   2. Take mental acuity tests. Once mid-day, once in the evening.

   3. Track sleep, heartrate, etc. using a Fitbit Inspire 3.

4. Night to the second day: Sleep normally.

5. Night to the third day: Sleep 5-5.5 hours.

6. Third day: Repeat 3.1-3.3, using the remaining dose/placebo you prepared.

7. Night to the fourth day and fifth day: Sleep normally.

8. Fifth day: Record baseline sleep measurements and mental tests

Each person had a substantial amount of leeway in how they structured their day. On sleep deprived days, Sam preferred to get up early while No Magic Pill and niplav preferred to stay up late and get up at the usual time. We each took doses at a consistent time, but the time differed between people.

We didn’t standardize things because we thought it was more important to have ecological validity, i.e. that we were using orexin the way we would actually use it in everyday life. This is a higher variance, but lower bias approach.

The Results

In our initial proposal, we pointed out that the main thing we wanted to see was orexin causing less rebound sleep the following night. A simple stimulant effect isn’t enough for us, we wanted to use orexin to sleep less and get away with it.

So here’s the average sleep time for the night after taking orexin vs the night after taking placebo:

Unfortunately, the difference wasn’t significant and the effect size is small. This could be for a couple reasons that we want to address in the next trial.

Did orexin have any sort of stimulant effect during the day? Nope, none of the mental acuity tests are significantly different.

In setting up this trial we had a sneaky second hypothesis: Does sleep deprivation actually make you dumber?

One caveat before we look at the data. Typically our “baseline” days would come after our sleep-deprivation days. So that means baseline days enjoy more cumulative practice compared to sleep-deprivation days. That should bias the results by making baseline days look better. On the other hand, if sleep deprivation has long term cumulative effects, then perhaps baseline days are at a disadvantage. But that doesn’t match our experience of feeling significantly better on baseline days.

So, does sleep deprivation make you dumber? Not really!

Depending on how you correct for multiple comparisons, the psychomotor vigilance task (PVT) differences might be significant. And I’d expect the PVT differences to become significant with more data points. From what I (Sam) remember from doing PVT on sleep-deprivation days, I felt just as fast, but I would slip-up more from inattention or distractions. This is consistent with the large gap on the slowest 10% days.

But overall, this is a nice example of how our intuitions around sleep can lead us astray. It sure feels like sleep deprivation should make you dumber. But we don’t see that here. It’s important to actually check what changes our productivity because our intuitions around this are pretty fuzzy.

The Next Trial

There’s a few reasons why we might be getting a null result. We might have too few data points, or the dose might be too low, or more concerningly, we might be storing the orexin improperly.

So the first next step is a slightly bigger trial where we see if a higher dose of orexin changes our results. From anecdotes online, some have felt effects while others haven’t. But even if orexin doesn’t have obvious effects, it might still reduce sleep need. We need to try higher doses and collect more data to find out.

That said, sleep deprivation is uncomfortable for Sam and No Magic Pill and extremely uncomfortable for niplav. We’ve decided to try a different design: sleep ad libitum on all nights of the week, but observe whether orexin reduces the amount we sleep the night after. This should make it sustainable to collect a lot more data.

Appendix A: Details about the Data Analysis

We collected two separate datasets:

1. Data collected automatically from the Fitbit Inspire 3

   1. Measures of sleep

      1. Sleep duration

      2. Sleep efficiency

      3. Time spent in deep sleep

      4. Time spent in light sleep

      5. Time spent in REM sleep

      6. Time spent in nocturnal awakenings

   2. Additional measures

      1. HRV Daily RMSSD (ms)

      2. HRV Deep RMSSD (ms)

      3. SpO2 Avg (%)

      4. SpO2 Min (%)

      5. Breathing rate (breaths/min)

      6. Skin temperature Δ (°C)

      7. Steps

2. Data from the mental acuity testing

   1. Psychomotor vigilance task

   2. Digit symbol substitution task

   3. Digit span

   4. Stanford Sleepiness Scale

   5. Description of subjective state (free text)

We aggregated mental acuity tests per-test to avoid pseudoreplication (so two data points per day), and aggregated Fitbit data per-day. We analyzed the data via matched controls (with days in a participant-block being matched as to analyze within-pair differences) and ran two separate analyses on the data; one frequentist and one Bayesian. The code for the analysis, written in Julia by Claude Opus 4.6, is available here. Our mental acuity test data is available here, aggregated full data is available here.

Frequentist Analysis and Additional Results

In our frequentist analysis we ran the paired t-test on the paired data with cardinal measurements, and the Wilcoxon signed rank test on paired data with ordinal measurements, we also report Cohen’s d for the measurements. We Bonferroni-corrected the p-values, not that that was necessary…

[The table was too unwieldy to include, see here for more.]

Bayesian Analysis and Additional Results

We fit a hierarchical Bayesian linear model with participant random intercepts, using NUTS (4 chains × 2000 samples per metric). The primary estimand is δ, a standardized treatment effect (Cohen’s d-like), with a weakly informative N(0,1) prior.

Formally, the likelihood is yᵢ ~ N(μ + δσ·treatmentᵢ + α[pᵢ], σ), where treatmentᵢ ∈ {0,1} encodes placebo/orexin. The raw treatment effect on the outcome scale is δσ; δ alone is dimensionless. Priors: μ ~ N(0,10) (vague grand mean), σ ~ half-N(0,10) (residual SD), τ ~ half-N(0,5) (between-participant SD), α[j] ~ N(0,τ) iid for each participant j.

Priors and posteriors for cognitive acuity tests and sleep measurements:

Priors and posteriors for additional Fitbit data:

Learning Effects on Mental Acuity Tests

Circles for the first test of the day, diamonds for the second test of the day.

Appendix B: Threats to Validity

Our method seems simple on its face, but there were a lot of annoyances along the way.

1. Orexin was delivered at room temperature, and while the vendor claims the orexin was lyophilized, we are uncertain if the lyophilization was sufficient to prevent damage.

   1. In niplav’s case the orexin sat uncooled in customs for over a week during the delivery in July.

2. In order to distribute the orexin into vials, we had to dissolve it in water. This meant that we had to both store the orexin dissolved in water for almost a week, and freeze the rest. We are uncertain if any of those damaged the peptide structure.

3. We are uncertain if our route of administration can cross the blood-brain barrier.

4. One participant wasn’t aware that Fitbit data needs to be regularly synced, so we only have sleep data for two individuals. Additionally, Fitbit syncing and data collection is unreliable, leaving us with only 17 datapoints for nights of sleep following orexin.

5. Another headache was making sure sleep deprivation nights were scheduled between nights where we could sleep ad libitum. We also tried to keep a consistent schedule on trial days so that variations in exercise, nootropic consumption or other activity didn’t change our results.

Appendix C: Personal Experiences

Niplav:

• So much stuff can go wrong

• 2.5ml is way too much, let’s do 1ml next time

• Filling the syringes was fun! It felt very scientist-y.

• Plausibly we should’ve started with a higher dose but also safety concerns so whatever

• 5½ hours of sleep feels horrible

• I was ~completely unproductive on sleep deprivation days, and will put a high premium on this

• Caffeine was really helpful

• Two nights of normal sleep & then one more sleep deprivation would’ve been better

• Inform yourself about the reliability of the data collection tools

• Didn’t nap at all except in week two, when I just couldn’t stay awake

• I’m happy we did a short trial first so we discovered the data collection issue early

• Beat-by-beat experiment log here

Sam:

• Feel so much better and alive from my rebound sleep, better than a normal days sleep even.

• Consistently napped on placebo days, but orexin does seem to have stimulant effect.

• Couldn’t really tell which was which in general

• Felt mentally the same on trial days. But doing tests felt a little slower than on baseline day. In general I didn’t have a good sense of how productive/smart I was.

• Extra hours in the morning did get used, for somewhat intellectual tasks like reading papers and writing.

• Much easier to be up early when sun was up

No Magic Pill:

• Tests:

  • PVT: I don’t think my reaction time improved at all over the course of the testing (75% confident). I did not mind this test.

  • DSST: I don’t think my skills improved much over the course of the testing. I disliked this test the most.

  • Digit span: I am 75% confident that I got better over the course of testing. I was consistently able to get 9 both forward and reverse towards the end. I disliked this test the second most (behind DSST).

  • Sleepiness: I never scored that high and probably erred on the side of scoring higher because of a feeling that I needed to utilize most of the scale. I did not mind this test.

  • Feelings: I could have been more verbose here.

• It was wayyyyyyy easier to stay up late and get up at my normal time than it was to go to bed at my normal time and wake up early.

  • I was fairly productive when staying up late

  • I was NOT productive when getting up early

  • Most of the time I had full-body “tingles” when awakening after a sleep-deprived night. I’ve experienced this phenomenon for years: anything less than ~6 hours, or normal duration after a hard exercise session, leaves me “tingley” in the morning.

• I was a normal level of irritable and quick to anger after a sleep-deprived night. This has been consistent for years (like the tingles).

• I did not nap on any day (pre-test or day-of test) because I thought that would throw off the testing data.

• My motivation on sleep-deprived days often waned faster than non-sleep deprived days. This has been consistent for the past few years and is what I expected.

• I did not feel anything physically or mentally immediately following the orexin administration.

• I should have done a better job of isolating myself during testing. Sometimes it was a bit noisy or visually distracting, especially if I was at work. I should have noted down if I was distracted during the test.

  • Maybe a feature can be added to add comments at the end of each test?

• I agree with Niplav that 2.5 mL was too much water. 1.25 mL was good, if not a tad much as well. I think 1 mL is probably good for the future?

Say you’re trying to predict when an event will happen. For instance, a scientific breakthrough, or AGI, or geopolitical drama. One way to write down your predictions is to give probabilities of the event happening by a certain date. 50% by 2026, 75% by 2027, and so on.

A simple way to write this down is a smooth distribution of arrival probability vs time. We know the arrival didn’t happen now (time zero) and could happen eventually (time infinity) so we want distributions that take values (support) over the range 0 to infinity.

A particularly useful class of distributions are the maximum entropy distributions. Maximum entropy says “I have as little information as possible about the future except for some constraints like the support and the mean”. We’ll specify things like the mean and variance and support, then choose a distribution consistent with those that also has highest possible entropy.

Let’s look at the exponential distribution. It’s a maximum entropy distribution with a specified mean and takes values from zero to infinity. Not a bad way to encode timelines.

(For now, I’m going to write timeline scenarios in terms of time-to-AGI to make them more concrete. But the same reasoning applies to any forecast using a distribution like this.)

If you think we’ll reach AGI in an average of 5 years, set lambda to 0.2 and the exponential distribution will provide the probability of AGI arriving X years in the future.

Mean excess is more interesting. “Conditional on not observing AGI for 2 years, how many more years to AGI?”. The exponential distribution is memoryless; the mean excess is constant. This has some interesting implications:

1. AGI is always x years away.

2. You need to have new evidence for years-to-AGI to change. Simply conditioning on the amount of time that passed is insufficient.

3. Without new evidence, every time AGI is reached you were expecting it to arrive x years later. You’re always surprised.

Now let’s consider more interesting distributions. I looked for maximum entropy distributions with support from zero to infinity and constraints on the mean and variance1. Below is the mean excess vs time for the lognormal, chi, Weibull, gamma, and inverse gamma distributions2. They all have the same mean and variance. You can try your own settings of mean and variance in this widget I made with Claude.

Notice that mean excess is flat once you pass the mean. In this regime, the implications are the same as for the exponential distribution. The linear regime is different:

1. Expectation falls ~linearly with time. If no new evidence comes in, you should think AGI is closer with each passing day.

2. You need a stream of new evidence for time-to-AGI to stay flat or increase.

If the variance is larger than the mean, some distributions see mean excess grow over time. Your forecast gets stale and the error bars grow. You’re more surprised when AGI arrives.

This doesn’t just apply to AGI, these dynamics occur in to any situation where you have a maxEnt distribution over arrival time. This creates interesting constraints for updating old forecasts and talking about how opinions have shifted.

“Fusion is always 30 years away. Implicitly, arrival probability is exponentially distributed.”

“If we update her forecast for the years that passed, time-to-AGI is now 10 years”

“I noticed your years-to-invasion is the same, what changed your mind?”

It’s neat that we can get interesting implications from such a simple model. I’ll have to think about more ways make use of this.

I use Bluesky for my microblogging. The AI scene is good there now. Though I’d like to see more science and econ stuff.

A thread from me on AI in 2026:

Pair with this thread on how LLM’s change the internet. I think ATproto (the protocol behind Bluesky) is well positioned to take advantage of these changes.

2.

Brain emulations are coming. From Konrad Kording’s end of year post:

The phrase “simulating nervous systems” used to trigger eye-rolls because it sounded grand and vague. In 2025, it felt more like an engineering roadmap with increasingly nontrivial milestones.

See his paper The time is ripe to reverse engineer an entire nervous system: simulating behavior from neural interactions.

Max Schons has a beautiful report on brain emulation:

State of Brain Emulation Report 2025

And a companion piece on Asimov press:

Building Brains on a Computer

See also: Notable Progress Has Been Made in Whole Brain Emulation — LessWrong

Related: Low-cost 3D-printed optics for super-resolution multifocal structured illumination microscopy. They 3D printed one of the lenses in a microscopy setup, making it much cheaper.

These are the kind of innovations we need if we’re going to make optical microscopy far cheaper and faster. In my mind, optical microscopy (with expansion microscopy and barcoding) is the most scalable method to scan connectomes.

3.

A few items on technology and the developing world. First, an old thread from me on financial technologies in the developing world. Those financial technologies are a big reason Why Solarpunk is already happening in Africa.

Local grids are failing people in Africa. People are using the mobile money system to take out loans to buy a personal solar and battery system. That means saving the money spent on kerosene for indoor lighting. That means being able to work and study for more hours of the day. I hope this unlocks a self-reinforcing cycle of productivity.

Also of note:

allenai/HiRO-ACE. An open-source weather model that uses AI to predict precipitation weeks or years in advance. Imagine farmers across the world using models like this to optimize crops or buy insurance.

The changing (and perhaps surprising) geography of diabetes. As the developing world grows and enjoys an abundance of calories, diabetes is rising. For the same BMI, some ethnicities get diabetes at higher rates. The developing world is going to need a lot of GLP-1’s.

Everything else

OxLDL-Targeted Chimeric Antigen Receptor T Regulatory Cells Reduce Atherosclerotic Plaque Development. CAR-T therapies to prevent atherosclerosis. Seems a bit overkill when statins can probably prevent atherosclerosis in the first place. But I’m all for developing cellular nanotech.

A Nonviral Neo‐Nucleocapsid for Cell‐Specific RNA Delivery Developed by Pseudo‐Cyclic Peptide Grafting and Directed Evolution. More cellular nanotech.

EVIO Aircraft. A planned hybrid-electric plane with 900 km range, 76 passengers, and quiet takeoff and landing. Important if we’re going to make flying clean and quiet so we can build more airports and air busses.

Terraforming Venus Quickly. Old paper from Paul Birch that aligns a lot with a post I’m writing.

Do Commodities Get Cheaper Over Time? Probably the most comprehensive data on long term commodity prices I’ve seen. My take: the theory of exhaustible resources suggests commodities should grow in price at least as fast as real interest rates. The data is consistent with real interest rates in the U.S.1

Most people are individually optimistic, but think the world is falling apart. Might I suggest that social (and traditional) media is selling you lots of pessimistic and misleading information creating a situation where views on “the world” are completely divorced from reality and people are mostly fine?

Pro-social media. AI’s telling users the truth and being a public arbiter of truth may ameliorate some of the problems of social media.

What do executives do, anyway? A management strategy for large companies that coordinates everyone around a set of values and enforces those values, delegating all decisions to lower parts of the hierarchy.

To paraphrase the book, the job of an executive is: to define and enforce culture and values for their whole organization, and to ratify good decisions.

That’s all.

Not to decide. Not to break ties. Not to set strategy. Not to be the expert on every, or any topic. Just to sit in the room while the right people make good decisions in alignment with their values. And if they do, to endorse it. And if they don’t, to send them back to try again.

Is this secretly a manifesto for orchestrating AI agents?

Are the Costs of AI Agents Also Rising Exponentially? — Toby Ord. The METR time-horizon curve looks different when you consider how models perform when given more resources. More advanced AI agents can do more, but also command higher hourly wages. Models will continue to improve in every aspect, but reality is more complicated than looking at the METR plot and extrapolating to singularity.

On neural scaling and the quanta hypothesis. Toy model reproduces AI scaling laws by assuming that A, knowledge is in discrete pieces (quanta), B quanta are either memorized or not, and C, “[t]he ‘use frequencies’ of the quanta naturally follow a power law.”

Tuning GPT-3 on a Single GPU. Old paper from Greg Yang applying theories of infinite-width neural networks to the real world. You can optimize the training hyperparameters of a small neural network (40M parameters) at small scale and then transfer them to a larger neural network (6.7B parameters). Might we “solve” training hyperparameters once we optimize them for a 1B parameter model?

The Advanced Matrix Factorization Jungle. Incredible reference on niche matrix factorization methods, particularly when you want to enforce things like sparsity.

Eventually, we’re going to stop making better computers. In other words, the real cost per unit of useful computing will flatten out, becoming relatively constant over time. Innovation in semiconductor chips will slow and the low-hanging fruit in other parts of the computing supply chain will get eaten1.

This will have implications for everything.

We’ve tried a lot of stuff in semiconductors

This view comes as a consequence of my view that breakthroughs are rare and decreasing. We try stuff and converge on good solutions. Eventually we find the economically efficient way to make a product and move on. This is true for all goods, from cement to computer chips.

This is very clear in the semiconductor industry for two reasons.

First because semiconductors make you painfully aware of the limitations of matter and physics. Transistors are dozens of atoms wide, there isn’t much room at the bottom. Smaller size isn’t necessarily desirable as smaller transistors leak more current, raising energy consumption and errors.

Long ago, technology node names like 90 nm actually corresponded to the size of transistor. Progress stalled but the industry kept up appearances by changing the name to match the old trend. This year, TSMC is producing “2 nm” chips, but the actual size of transistors is closer to 45 nanometers. The shrinking transistor has slowed and Dennard scaling “ended” around 2006.

Second because the semiconductor industry has tried a lot of stuff. It has saturated many S-curves already. Dennard scaling was the first casualty, but there have been many more. Thousands of companies have sprung up and died optimizing one part of the supply chain only to be obsoleted by something else.

Watch one of my favorite videos and be awed by the amount of innovation that went in to creating just one flashlight in the semiconductor manufacturing process:

Watch some of the other videos in that playlist and you’ll see how much work and deliberation went into developing EUV. The industry explicitly considered other techniques, built out prototypes, started companies, and then went with EUV. That new company touting X-rays as the new way to make chips? Yeah the industry has already considered and rejected X-rays twice.

This is a microcosm of what’s happening at every step of chip production. Processes are evolving, with massive research effort to try different techniques2. The process we have today is a product of over 60 years of tinkering. With growing research effort, chips can steadily improve until we’ve wrung out every opportunity.

What about …

AI?

Sorry, but current AI won’t revolutionize chip design. Eventually there will be enough training data for AI to design chips, but we quickly run into the problem that logic optimization and place and route are NP-hard. Chip design is a fundamentally hard problem. AI tools will help, but have limits3.

Space?

Let me show you why space data centers are silly in the length of a skeet.

Option A: build solar, radiators, and chips on earth.

Option B: build solar, radiators, and chips on earth AND pay $3000/kg+depreciation to put them in space.

You're paying extra money for no apparent benefit.

The same principle applies if you’re launching self-replicating asteroid-mining nanobots. You could have used them on Earth and saved yourself the launch costs and harsh environment and time delay.

If we build enough data centers on Earth that all the low cost land and resources get used up then maybe space data centers make sense. The latency is much higher and it’s unclear if there will be demand for compute at that scale, at least on Earth. I’ll address that possibility in a later section.

For now, going into space doesn’t dramatically lower compute costs4.

EDIT: See Tech I'm skeptical of and why for a lot more detail on this.

Unconventional computing?

There are many unconventional computing paradigms, perhaps one will swoop in once we’ve exhausted CMOS?

I’m skeptical, in part because none of them have overtaken semiconductors despite many, many people trying. It’s a trillion dollar bill that nobody has picked up.

Of the unconventional computing paradigms, we can safely reject those that use components that are much larger than today’s transistors. The speed of light limitations are daunting. Fluidics, MEMS, and biopolymers are simply too large to compete with the speed and cost of CMOS.

Analog computing paradigms (think “neuromorphic computing” or “thermodynamic computing”) are fraught as well. There have been many failed attempts in this field, and Zach lays out the challenges of Making Unconventional Computing Practical5.

The problem with analog is that when your signal is continuous, it’s easy to mess it up. Temperature changes, manufacturing variability, and interference all change the signal a little bit. In a chaotic system, those errors propagate out and mess up your computation. That’ why most analog computing companies have failed or switched to digital logic. Digital just works.

That leaves us with a few paradigms. I honestly don’t know how these will pan out, but I’ll note reasons for skepticism.

Spintronics applies materials that control electron spin inside of more traditional chips. The hope is that electron spin might provide an additional way to store or transfer information. I do think this has promise for improving memory bandwidth to some applications. But given how sensitive electron spin is to stray electromagnetic fields, I’m skeptical that the benefits are worth the costs of incorporating new materials and design complexity.

Optical computing startups in the last few years have pivoted from making computers to making optical interconnects for traditional data centers. Not a promising sign. Hopefully, this beachhead will enable optical computing to scale. However, optical inherits some of the challenges of analog computing, namely temperature dependence and manufacturing variability. These things can be addressed, but add cost.

The bigger issue is that optical components have to be physically larger than transistors, on the order of the wavelength of light they are carrying. This makes them 10-100x larger than transistors. Patterning at a larger scale means higher costs67. The saving grace is that light can send many different signals at the same time (multiplex). So even if the components are 10x more expensive, they might carry 100x more information. This makes it the most promising contender.

Superconducting computing inherits all of the problems of analog computers, while having components that are physically larger than normal computers, while demanding very low temperatures. People have tried and failed. I’ll pass.

Quantum computers (which often leverage superconducting components) offer small speedups in a few problems of interest. Quantum computers have to be kept very cold and shielded. You need lots of qubits for error correction. I doubt these will become cost effective for general purpose computing.

Reversible computing8 is interesting if compute scales to space, where energy is expensive and heat is a problem. These benefits are far weaker on Earth. The error correction is harder, the designs more complicated, and perfect reversibility is impossible, reducing the energy consumption advantage.

What does the end of hardware history look like?

Innovation jumps around

A key point from my breakthroughs post was once you “solve” a particular product, innovation jumps somewhere else. First you figure out how to plant, then breed plants, then make fertilizer, then automate harvesting, and so on.

As semiconductor innovation slows, innovation in other areas will grow. Solar and batteries will provide cheap data center electricity, new cooling systems will lower costs, optical or RF interconnects will gain traction, satellite internet will lower latency, and of course the AI/software layer will improve. There’s hundreds of opportunities for innovation here.

That said, this doesn’t refute the overarching conclusion. Eventually innovation in these areas will grind to a halt, we will have found the right way to build a data center.

Electron beams?

Pushing to higher wavelengths of light has its limit, and high-NA EUV could be the end of photolithography. In the future of lithography I speculated that electron beams might be the final frontier. To this day, I still think electron beam lithography might be the economic limit of chip manufacturing. Atomically precise manufacturing might make better chips, but they won’t be cheaper, photolithography will make cheaper chips, but they won’t be better.

We’re a long way off from electron beams beating EUV though. Currently, e-beams are too slow. I’ll add some links in the appendix on making e-beams cheaper and simultaneously writing with thousands of beams with one device which seems promising.

AI hardware

The economics of LLM inference tell us that memory bandwidth is the key cost driver. Chip designers have come up with their own version of “just stack more layers LOL” called high-bandwidth memory. It involves stacking towers of fast memory close to the part of the chip that does calculations.

Designers are pushing this S-curve as hard as they can. Unfortunately, each layer gets more expensive9 while being slower since it’s farther from the rest of the chip. At some point, you’d expect there to be a cost optimal HBM size, just like we see for skyscrapers10.

Other avenues for hardware improvement are drying up too, as Tim Dettmers explains:

In the past it was possible to shrink the size of transistors to improve speed of a processor. This is coming to an end now. For example, while shrinking SRAM increased its speed (smaller distance, faster memory access), this is no longer the case. Current improvements in SRAM do not improve its performance anymore and might even be negative. While logic such as Tensor Cores get smaller, this does not necessarily make GPU faster since the main problem for matrix multiplication is to get memory to the tensor cores which is dictated by SRAM and GPU RAM speed and size. GPU RAM still increases in speed if we stack memory modules into high-bandwidth modules (HBM3+), but these are too expensive to manufacture for consumer applications. The main way to improve raw speed of GPUs is to use more power and more cooling as we have seen in the RTX 30s and 40s series. But this cannot go on for much longer.

He goes on to mention specialized logic and low-precision arithmetic as paths forward. But these have limits too.

If we had no more ideas, AI hardware would stop there. But the holy grail of inference compute is compute in memory. Since inference is a bunch of simple operations, make the arithmetic cores small and repeatable like memory. Put memory and arithmetic right next to each other and glue together massive amounts of compute.

This should increase the memory bandwidth a lot while increasing costs somewhat less. On balance, it’s not clear where the cost per unit of memory bandwidth will end up. After that, we’re kind of out of ideas at the design level11. But there are decades of tweaks we can pursue.

Implications

Everyone makes chips, costs fall

In the semiconductor industry, tech that’s ~10 years old is roughly accessible to any country. The trade secrets have mostly diffused, the equipment can be procured, and the talent from that era can be hired. Consider how China leads the world in producing trailing edge chips while still failing to produce leading edge chips.

In a world where chips stop getting better, everyone catches up to the state-of-the-art in about 10 years. That means competition, innovation, and falling prices.

Even better, since chips stop improving, depreciation costs fall dramatically. You can keep using the same hardware for decades.

AI converges to a price per unit intelligence

There’s already plenty of competition over model provision, especially from open source providers. When the monopolies in chip production fall, we get even more competition in this space.

What does the intelligence market look like in this world? I expect RL-as-a-Service (specialized models) will dominate. The base models these are built on will converge to a certain price per unit intelligence, itself dependent on the price per FLOP beneath it. I imagine a range of different base model sizes, with the largest sizes serving an inference oligopoly and smaller sizes seeing lots of competition and low margins. Specialized model providers can turn a profit by building domain knowledge on top of small models.

For more, see Economic futures for LLM inference.

Two AI hardware niches

There’s two niches for AI hardware in the future.

One we’re already familiar with, chips and servers optimized to be in huge data centers. Here, high memory bandwidth at low cost is the key. This hardware can provide low cost, low speed AI for very cheap by using large batches. Particularly valuable for supplying and inference oligopoly or autonomous AI workers.

On the other end of the spectrum is local AI hardware. Specialized chips in your laptop. These run small models (smaller memory footprint) and don’t have high utilization, so FLOP/s/$ is the key metric here. These models can be extremely fast and responsive, perfect for collaborating with your AI assistant.

Will anyone want space compute?

I’m unsure whether there will ever be much demand for space computers. The previous section showed why space computers are more expensive, at least until compute demand outpaces Earth’s ability to provide it.

It’s not clear that terrestrial demand will get that high. AI models are keeping the same memory footprint while increasing capability. We might see a Kuznets curve of AI compute demand once Claude writes all our apps for us. I would guess per capita compute consumption isn’t rising much outside of AI. Future netizens might still be reading blogs and watching YouTube.

And remember, the latency of servers outside of LEO (where most of the computer-chip-material is) is far higher than terrestrial servers.

Why would we want such a large amount of slow and expensive compute? I can think of one reason: The Age of Em12.

Conclusion

Traditional semiconductor manufacturing will win for the foreseeable future. In perhaps 30 years, progress will come to an end. The state of the art techniques will diffuse everywhere. The end of semiconductor monopolies and hardware depreciation mean compute will get very cheap. Perhaps another wave of innovation will bring an unconventional computing paradigm like optical to the fore. But I’m not holding my breath.

Space computing is the final frontier, the constraints are different and the scale far beyond what Earth could support. There we might find solutions new and bizarre.

Appendix

Charts!

Please enjoy these slapdash charts of hardware price-performance vs time. I focused on the two main performance metrics for AI chips.

I modified the Epoch data from here to make this spreadsheet. Note the values are in nominal terms, in real terms there’s been more improvement over time. I’m personally surprised at the lack of growth and it’s possible I’ve made an error.

Note that future AI performance depends more on BF16 and smaller number formats. Likely we’ve seen much more dramatic improvement in these formats. The Epoch data on this was sparse so I stuck with FP32.

Further reading

From me:

Economic futures for LLM inference

RL-as-a-Service will outcompete AGI companies (and that’s good)

An intro to the Tensor Economics blog

On AI Scaling

The future of lithography

From others:

Just read all of Zach’s tech blog, he’s more qualified.

The Transistor Cliff—Asterisk

The Rising Tide of Semiconductor Cost

The Unsustainability of Moore’s Law

How to Build a $20 Billion Semiconductor Fab

Semiconductor Fabs I: The Equipment

TSMC’s new 2nm chip will reportedly cost 50% more — get ready for more expensive laptops and phones

The longest training run. When AI hardware keeps progressing, it only makes sense to do short (~1 yr) training runs.

AI hardware and memory technology

The Special Memory Powering the AI Revolution

The Memory Wall: Past, Present, and Future of DRAM

[2601.05047] Challenges and Research Directions for Large Language Model Inference Hardware

The Breakthrough Solution to DRAM’s Biggest Flaw probably the most exciting research on memory technologies I’ve seen. Get rid of the capacitor and just use transistors for memory. Works particularly well with compute in memory. Though there are risks from using new materials and taking on more routing complexity.

Zach’s tech blog linked this presentation on 3D-split SRAM by Rahul Mathur. I don’t fully understand but I’m including it here because FOMO.

Cool stuff you can do with electron beams

Multiple-electron-beam direct-write comes of age

Hundereds of thousands of beams possible:

TSMC's Incredible 2nm Curvy Masks - YouTube

Resolution Limits of Electron-Beam Lithography toward the Atomic Scale

Transverse Electron Beam Shaping with Light

Nanostructuring of electron beams

Learning and Controlling Silicon Dopant Transitions in Graphene using Scanning Transmission Electron Microscopy

Precise atom manipulation through deep reinforcement learning

A Platform for Atomic Fabrication and In Situ Synthesis in a Scanning Transmission Electron Microscope

Atomic Semi • The Make Anything Company

Of course, the researchers don’t have evil intent. They believe it will help us stay ahead of outbreaks. The consequences of research are uncertain and experts disagree about the difficulty of building a bioweapon. Regardless, there are better ways to prevent pandemics.

I don’t want to list a bunch of examples lest I spread dangerous information. The example I’ll discuss here is in a famous journal so I’m less worried about covering it.

Bat-infecting merbecovirus HKU5-CoV lineage 2 can use human ACE2 as a cell entry receptor was published in Cell, the preeminent journal in biology. This paper looks at bat virus called HKU5-CoV lineage 2 and finds that it can effectively bind to a proteins on human cells. “… HKU5-CoV-2 infected human ACE2-expressing cell lines and human respiratory and enteric organoids.” They identified a circulating disease in bats with the potential to jump to humans and detailed why the viral proteins bind effectively.

This research is risky for a few reasons:

1. These pathogens are already viable in wild populations and could potentially transmit to humans. They pose a pandemic risk on their own.

2. They tell people where these pathogens can be found.

3. They detail why the surface proteins on the virus are effective at binding human cells, making it possible to replicate this in a genetically modified virus.

Better ways to address pandemics

Researchers and grant-makers justify this sort of work as a means to monitor and prepare for real world pandemic threats. Know thy enemy. Fortunately, the attendant risks are completely unnecessary. We can address pandemics via1:

• Metagenomic sequencing of wastewater to identify fast growing diseases.

• Surveilling diseases in farmed animals. These are huge, densely packed groups that interact with a lot of people.

• Far-UVC lighting, upper-room UV, glycol vapors, air filtration, surface sanitation, and PPE can slow the spread.

• Phone based contact tracing and rapid testing

• mRNA vaccines can rapidly provide herd immunity to the population2.

• Virulence management

• Effective remote work and remote education

Conclusion

Governments should stop funding pathogen discovery research and gain of function research. I say this as someone who is very optimistic about innovation and trying weird ideas. Studying zoonotic pathogens isn’t worth the risks, we have other ways to mitigate pandemics.

Resurrecting modern people

What would you be like if we did your life over again? Say you’re born into the same family, go to the same schools, with the same friends, and the same jobs. But the little details are different. Instead of hanging out with Kyle one day you’re with Michael. Or you fail a test because you had one too many drinks last night.

How different would you really be? I think for most people, you would be substantially the same person1.

Consider those spooky stories of twins separated at birth and reunited as adults who are so similar they named their children the same. This isn’t as crazy as it sounds. You’ve got two people with the same genes, living in the same culture, at the same time, in similar regions, as foster children in families with similar socioeconomic status. We know that these things matter for life outcomes, two people with the same circumstances should end up being very similar.

All this to say that if your clone redid your life, that person could reasonably be called a copy of you. A partial copy to be sure, but would any of your loved ones be able to tell the difference?

We can go a step further. Say we can run a simulation of you and your experiences. We watch carefully to see if your simulated life diverges from your real life. Choose the wrong college? Tweak some parameters and reroll.

At the end of this process we have a simulation of your life that hits all the same beats. A clone that made all the same decisions that you did. If you avoid overfitting, this clone might be essentially identical to you.

Such a simulation is beyond today’s technology, but may be possible in the future. But how will people in the far future know enough details of your life to resurrect you? What can they benchmark your simulation against?

Your social media posts of course. Social media provides a timestamped record of what you were thinking about. A simulation of you might reproduce a post word-for-word at the exact same time. That would provide strong evidence of your resurrection.

There’s a few things you can do to improve your chances of being resurrected in this glorious post-human future. Post on social media a lot, more data points gives you a better chance of surviving. Photobomb other people’s photos too. That gives the simulators a chance to cross-reference information about where you were. Also publish lots of information about yourself like your full genome, your biography, etc. in some permanent repository.

Social media companies are naturally the most important companies in the world. Let us pray that they survive millions of years so we might be reborn in the Dyson swarm.

Turbo-narcissism

Like every post on the simulation hypothesis, we can say something like and this is what’s happening right now you’re a simulation oooh. Resurrection is fun because it imbues the simulation with purpose. You’re not mold growing in someone’s simulation, you’re the center of attention.

Now you can enjoy a god complex by believing that you grow into a great mind the future desperately wants back. Or this life is a bizarre form of torture constructed by your enemies. Or perhaps you are merely here to entertain posthuman beings (did you hear a laugh track?).

The best part is these stories are unfalsifiable, you’re free to believe whatever you want.

Backchaining further in the past

Resurrecting social media poasters is a far cry from resurrecting everyone. Can we bring back people from before social media?

If those people interacted with someone who eventually used social media, you may be able to back out some information about them. Your grandparents behavior influenced you and left a mark on your digital traces. In reproducing your mind, your interactions with your grandparents can be used as a free parameter.

For people further in history, at the very least we can use their genes, social structure, and life history to produce a person very similar to them. We’d need to find social events with a known outcome which involve a smallish number of people. The result of a spelling bee or baseball game perhaps. The minds of the people involved become the free parameters, tweaking and rerunning the simulation until things play out the same way2. This won’t achieve the same fidelity as a social media post, but you can narrow things down to a feasible set.

Further in the past, things get fuzzier. You might still be able to infer their genes, but the best you can do is shotgun sample many different lives, keeping the broader sweep of history consistent.

All possible humans?

Why resurrect only the people who had lived? Why not resurrect all possible human minds?

You see, your genes contain only 1.5 MB of information. For a being that can rerun simulations of human society, sampling different genomes in different societies might be trivial.

What would be the purpose of instantiating all (human) minds? Perhaps they’re doing a The Egg thing. Or perhaps this parliament of minds can solve philosophy behind a veil of ignorance. You should ask them when you get there.

Conclusion

Resurrection creates some tantalizing possibilities. We can learn more about history, kill Hitler again, and discover how hot famous people were.

It’s far beyond our capabilities, but might be possible in the future. Indeed, it may be happening right now.

Statements like these are silly, I want to address some ideas about innovation that might be at the root.

Innovation is mostly trying stuff

The way I think about innovation is Trying Stuff. The best way to determine if something works is to try it in the real world. Most innovation comes from trying different things in a particular domain and keeping what works. Everything else is secondary. Fancy theories make trying stuff more efficient, but aren’t required for trying stuff to work. Standardization and automation are valuable because they accelerate the process of trying stuff.

Most major discoveries were made by people trying stuff, with theory and standardization happening later. Steam engines were in use before thermodynamics, many drugs were discovered on accident, and the green revolution happened because of systematic tinkering.

The implication is that brilliant theory can only get you so far. You need to actually collect data and try things out to make a discovery. The trying things out part is the bottleneck in the process. You have to physically move atoms around and wait for the results. A data center full of Nobel Laureates would spend most of their time waiting for results, not making discoveries1.

Eventually, trying more stuff is useless

Trying stuff comes at a cost. There is the physical cost of doing the experiments and the opportunity cost that you could have done experiments in some other domain. Combine with diminishing returns and eventually it doesn’t make sense to continue experimenting.

This is stronger than the idea of “low hanging fruit”. Not only do the easy opportunities get found, it stops being useful to even look for new opportunities in a particular area. The marginal benefit is too small.

Most domains see an end of history

At the end of my series on solving climate change I noted:

After looking at the future of cleantech, it feels like we’re nearing an equilibrium. The end of the century may see the end of terrestrial engineering. We found cheap energy sources that won’t be usurped in the forseeable future. The quirks of renewables will propagate into our production processes. After that, what’s left to change?

This is true of all technology. We find an effective way to do something, tweak it a bit, and then kind of stop2. This is an odd sentiment in an era defined by ever improving computers, but look outside the enchippening and you’ll see it’s true.

Food products like bread, cheese, meat, beer, and wine are made with techniques that would be recognizable to people centuries ago. Indeed, rain-fed agriculture still provides virtually all of our food, as it has for millennia. It looks unlikely that vertical farming will change that. Of course, we have substantial automation in agriculture, but that’s because we didn’t find an alternative to growing crops. We found new ways to increase yield outside of the growing process.

Even semiconductor fabrication and AI scaling follow the same pattern. We scale up an idea until it reaches physical limits then innovate elsewhere. Smooth progress comes from stacking hundreds of S-curves.

After many S-curves, progress slows. Haber-Bosch is how we produce fertilizer today and I’ve realized we aren’t going to find anything better for the foreseeable future. It takes iron and air and makes bread for gods sake. Or consider that silicon, the second most abundant element in Earth’s crust, has a bandgap that makes it perfect for absorbing the sun’s emission spectrum in solar panels. Need we search further?

Look around at everyday objects and this insight becomes trivial. Things like cups and chairs and pens are already optimized. There is little need for further cup research. There is no magic recipe for making cups more efficiently. Our world is grounded in physics and it’s unlikely we’ll find a substantially better production process.

So innovation jumps around. We encounter a new problem, fiddle around, find a good solution and then move on. Everything converges as we figure out the right way to do things. Unless our circumstances or way of life changes, we should expect society to crystallize around specific technologies and production processes.

That said, there’s still so much room for improvement. The world that we’re converging towards is wildly different from our own. But if we’re going to build a better world, we need a realistic understanding of how innovation works and what is possible.

EDIT: Further Reading

Is Science Slowing Down? By Scott Alexander. Lays out the low hanging fruit theory and suggests it that exponential resource investment to get steady technological gains should be the default assumption.

Tyler Cowen’s book The Great Stagnation.

Are Ideas Getting Harder to Find? Classic paper from Bloom et. al.

EDIT: Growth is Getting Harder to Find, Not Ideas. The fall in firm growth after controlling for idea growth suggests that successive ideas are less valuable, consistent with diminishing returns to more research effort.

Recipes and economic growth: a combinatorial march down an exponential tail by Chad Jones. An economic model of innovation as trying stuff by sampling from a distribution. A toy model that accords with my understanding of the innovation process.

Mundane example of innovation stagnating: why AA batteries haven’t gotten better despite so much progress in battery tech.

There has to be a better way to make titanium. A more technical example of how hard it is to innovate on today’s frontier. Look at how many times people have tried and failed to improve titanium production. Orca has narrowed the possibilities and may, if successful, find the final right way to make titanium.

The lovely book Ignition! gives a firsthand account of scientists studying rocket propellants in the mid 20th century. It tells a story consistent with the model above, people trying everything they could get their hands on, converging to a few right answers, and then moving on. Clark concludes:

There appears to be little left to do in liquid propellant chemistry, and very few important developments to be anticipated. In short, we propellant chemists have worked ourselves out of a job. The heroic age is over.

But it was great fun while it lasted.

Something I wish I’d thought of when writing this piece: consider the innovations of mother nature. Evolution hops around, finding locally available mutations. Species optimize for a particular niche and then stop. When a new niche arises new “innovation” happens until it is filled. Predator and prey find an equilibrium. Consider how invertebrates lack an acquired immune system yet live in peace with their parasites. Human affairs mimic the wild; innovation tinkers, converges, and halts.

EDIT: Or consider sports. Each time a rule is changed, strategies adapt. Sometimes someone finds a new trick or way to game the rules. The league shifts and finds a new equilibrium. There’s not infinite innovation to be found here, just a steady convergence to equilibrium, finding smaller and smaller improvements.

People also pointed me to Ideas aren’t getting harder to find by Karthik Tadepalli. Here was my response:

… We’re saying similar things about low-hanging fruit, but Tadepalli suggests an alternative theory blaming declining allocative efficiency. I’m all for fixing allocative inefficiency, I’m sure that’s part of the problem.

Though I maintain that even if we unblocked the innovation process, that doesn’t change the fact that we’re converging on a right way to produce goods. We’re bound by the laws of physics and economics. …

Set aside your judgement and see this post as describing empirically how dating apps work. Push through the confusing sentences and you’ll find a wealth of information about human behavior.

The appendix of this post has more notes, but some things that jumped out at me:

1. The apps have solved the recommendation problem. “Recommendation of profiles that you may like is a solved technical challenge at Tinder level and at mostly any dating app today.”

2. Retention of users is the only thing that matters for these compamies.

3. Retention depends entirely on a personalized feed algorithm.

   The key to produce the best feed algorithm (if you target retention) is personalization more than recommendation. Of course seeing users that you really like every once in a while will increase your retention. But what really increases retention is understanding what each user wants. And we all want things different. Some people will want to see only a few people that they like a lot, some people will want to see a lot of users. Some people will want to see people that are far from them, some close, some can’t stand seeing people they don’t like, some can’t stand seeing people they only like. Some want to receive a lot of likes, some want to receive a few likes

4. Current dating apps are local optima. “What makes you keep scrolling on TikTok or YouTube or Instagram? It is the feed, it is the same on a feed-based dating app. Pursuing retention will just bring you to recreate Tinder.” To create something new, you need to aim at something other than retention.

One theme that lacks a pithy quote but seems to run throughout is that people want attention. They like getting matches and talking to people. I think this is a deep part of human behavior and drives a lot of social dynamics.

2.

Parents’ effective time endowment and divorce: Evidence from extended school days. When the school day is extended by 3.5 hours, mothers can work their own job and gain economic independence from their husband. This leads to an increase in divorce rates.

It’s a good thing that childcare gives these women the opportunity to work, to support their family, and to leave bad marriages.

How do we scale childcare? Baumol’s cost disease threatens to make it perpetually expensive. One option is to make it universal and government-run. Though this misses out on possible efficiency gains. A voucher system might work, effectively subsidizing childcare while allowing the market to find good solutions.

But neither of these address the risk of education signalling spirals like we see in South Korea. What happens if all this subsidized childcare turns into a red queens race of exam study?

In a more reasonable world, we would recognize that new technology means students can keep up with the curriculum with a mere 2 hours of work per day. Older students would be able to speedrun several grade levels, there’s no need to teach certain concepts so early. The rest of their time could be spent on the playground and they would turn out just fine.

But I don’t know how to coordinate everyone around that idea1. Seems important.

3.

ACX has some good discussion of why the economic vibes are bad:

Vibecession: Much More Than You Wanted To Know

Highlights From The Comments On Vibecession

They offer many tantalizing possibilities, but we should keep things simple. Peoples vibes are completely unreasonable and unmoored from reality. Instead, the main thing bringing people down is more media consumption and negativity in the media. This explains the long term and global reduction in vibes since ~2010.

Several things exacerbated this emotional sentiment:

• Post-pandemic inflation lowered some peoples real incomes and made others feel poorer because of the money illusion.

• The price of buying a house (see image) rose sharply because of COVID, persistent inflation, NIMBYism, tariffs, and immigration crackdowns. Rents have increased slightly. But rents in highly-desirable metro areas where all the exciting opportunities are have increased more.

• Removing application frictions in college apps, job apps, and dating makes people experience more rejections. They’re doing more work to stay in the same place.

• On top of all of this, Zvi points out that people expect more out of life than they did before. In addition, regulations in childcare, healthcare, education, and housing make some necessities far more expensive. Much more at the post.

So, what are we going to do about it? Obviously better policy can address many of the bullet points. But that doesn’t face the broader problem of negativity in the media. This isn’t the first time I’ve bumped up against this problem. A previous linkpost started with the decline in marriage and ended up here:

But really staring into the abyss requires recognizing that better and cheaper entertainment media is interfering with our social fabric. I’ll never suggest regulating internet use, but maybe giving students a Defense Against the Dark Web class would help. I’ll have to think more about this.

I get the sense that the internet is driving a wave of negativity, damaging our social fabric and driving people to unreasonable beliefs. This makes voters more unreasonable. This drives bad policies which produce stagnation.

The internet has brought many wonders, but how do we patch this?

Everything else

Money Doesn't Buy Elections. It Does Something Worse. Proposes that donors influence who is able to run for elections rather than influencing voters. This gives them control over what policies are acceptible.

Vagus nerve-mediated neuroimmune modulation for rheumatoid arthritis: a pivotal randomized controlled trial. Another win for treating ailments by manipulating the brain.

We Induced Smells With Ultrasound

Memazing. Sebastian Seung (one of the leading connectomics researchers) is starting a brain uploading company.

Microscopic-Level Mouse Whole Cortex Simulation Composed of 9 Million Biophysical Neurons and 26 Billion Synapses on the Supercomputer Fugaku. What happens when a crappy neural simulation starts talking to us? Just turn it off?

AI infrastructure in the “Era of experience”. Another great post from the Tensor Economics blog. Essentially argues that RL-as-a-Service will come to dominate. Goes into technical detail on how datacenters can serve many fine tunes of the same base model (multi tenancy). These fine tunes will be in the form of LoRA adapters trained on proprietary data. Training can get also get gains from multi tenancy and LoRA. See also my piece on Economic futures for LLM inference.

RF Over Fiber: A New Era in Data Center Efficiency. Two companies are using plastic waveguides to send radio waves between chips. This allows for the type of high-bandwidth connections needed in the age of AI. They think it will be cheaper, more reliable, and more energy efficient than optical interconnects.

Antimatter Development Program

Appendix on dating apps

Profile photos

• Profile picture is pretty much the only thing that matters for determining if someone likes another person.

• The percent of people who like your profile is sufficient to measure hotness. You don’t need ELO or anything fancy.

• Attractiveness in real life is distinct from profile photo attractiveness. Generally women look better in photos while guys look better in real life. Attempts to level the playing field by having people take videos didn’t work because women didn’t like how they looked in video so wouldn’t join.

Other profile info

• People are willing to fill out lots of information about themselves. This information is virtually useless for matching people. But people like to do it so it’s included as part of the user experience.

• People are terrible at telling you what they want, their inputs are useless. 2-3 go/no-go parameters is all that is helpful.

• Women sometimes use this other profile information to filter out certain men. Older women put more emphasis on it.

The feed algorithm

• “Recommendation of profiles that you may like is a solved technical challenge at Tinder level and at mostly any dating app today.”

• BUT you can’t just feed people matches that you think they will like. New users need some activity on the site before you know what they like. And these new users need to be reviewed in turn by other users. “People don’t want to see only users that they like, girls actually adapt their behavior if they see too many guys that they like.”

• “The key to produce the best feed algorithm (if you target retention) is personalization more than recommendation. Of course seeing users that you really like every once in a while will increase your retention. But what really increases retention is understanding what each user wants. And we all want things different. Some people will want to see only a few people that they like a lot, some people will want to see a lot of users. Some people will want to see people that are far from them, some close, some can’t stand seeing people they don’t like, some can’t stand seeing people they only like. Some want to receive a lot of likes, some want to receive a few likes”

• “The feed algorithm is the only thing that will impact the retention of users on your app. Everything else almost doesn’t matter.”

• Lots of tricks in the feed such as showing women a mix of guys they actually like, guys they feel they are supposed to like, unattractive men. You also want to ensure women are receiving a stream of likes and messages, so men’s feeds might be oriented towards seeing women that haven’t gotten sufficient likes today.

• Women seem to like a fixed percentage of the profiles they see. That means the context of other men in their feed affects how often they like a particular guy.

• “Girls scroll about 2 times more than guys. (Fortunately). First, the like ratio of girls is 4%. So to be able to like 4 guys they have to see 100 users. And for a guy, it is 18 users. It also means the experience on a dating app for a girl is completely different than the one for guys. A girl spends a lot of time just searching for guys she will like, and guys spend a lot of time hoping a girl will like them back.”

Retention

• The main focus for apps like Tinder and Bumble is retention of paying users.

• Apps have monetized and it hasn’t changed the experience much. Men pay.

• For women, only thing that drives retention is number of likes sent. At the beginning, the most important thing is seeing a match quickly after signing up.

• For men, nothing seems to impact retention much.

• Need to do moderation on what accounts are allowed. Obvious fake users hurt the site. Subtle fake profiles exist and don’t have a big negative impact. Their site blocked ~all of them and didn’t impact retention much.

Current dating apps are local optima

“What makes you keep scrolling on TikTok or YouTube or Instagram? It is the feed, it is the same on a feed-based dating app. Pursuing retention will just bring you to recreate Tinder.” To create something new, you need to aim at something other than retention.

Dating apps provide entertainment, not dating

• Some people just want to chat on the apps and meet others, they don’t really go out.

• Some people just match up well and end up meeting in real life, but there’s not much the app designers can do to change that for the better.

• “People were not speaking for very long conversations in general, they were happy to start a chat, but never really cared to continue it. Which always makes me say, that people registered on these apps are not really here for dating, but more for entertainment purposes.”

1.

The Qattara Depression project is an old proposal to create a big sea in the middle of the Egyptian desert by channeling water from the Mediterranean sea to a region below sea level. Proponents suggest it could generate lots of electricity and create good conditions for agriculture.

I just realized it’s a dumb idea. First, the Sahara is awash in cheap solar, so it does not need expensive hydro. Second, Egypt is already building out agriculture in the desert by bringing in freshwater. Third, an above ground-sea in the desert is the worst place to put water. Especially when you can store ~660x more water in the Nubian sandstone aquifer system underground.

There’s an opportunity to use solar and desalination at scale to keep the aquifer full and enable agriculture in the region.

2.

Luzia Bruckamp highlights econ job market papers related to fertility. I recommend reading for interesting summaries on the effects of paternity leave, C-sections, and more on fertility. The paper I found most striking was Build, Baby, Build: How Housing Shapes Fertility which finds that “[r]ising housing costs explain roughly half of the decline in the total fertility rate between the 2000s and 2010s” in the U.S. The author’s model suggest that by building large units desired by families, America would have returned to replacement fertility.

Not a job market paper, but related: The Covid-19 Baby Bump: The Unexpected Increase in U.S. Fertility Rates in Response to the Pandemic. I speculate that the transition to remote work during Covid encouraged U.S.-born mothers to have kids. Remote work gives people the opportunity to balance career and family and I wish governments would support a transition to hybrid work.

By this point it’s clear that rich countries can solve their fertility issues. They can build more houses, their workers are skilled enough to be remote, and they have the tax revenue to fund a baby bonus.

Middle-income countries are more at risk. Places like India and Mexico are seeing sharp declines in birth rates. They have fewer resources to return to replacement fertility and a larger shortfall than the U.S.

3.

Batteries make it possible to scale flight down, just look at drones. Gas engines are hard to scale down like that. That gives us a limited version of flying cars.

Scott Manley reviews the Pivotal Helix an eVTOL with 20 mi range, 60 mph speed, 75 minute recharge time, and Level 2 autonomy.

Like the Electra Ultra Short, I initially dismissed this because of the short range. But then I realized how valuable point-to-point flight is. 20 miles is enough to land you anywhere in a major metropolitan area. Imagine walking to the nearest park and taking a 20 minute flight to anywhere in a circle that is 1200 square miles (3200 km^2).

Eyeballing the chart below, people in metropolitan areas commute less than 40 miles a day. The straight-line distance would be smaller and the Pivotal Helix can already cover that.

With a little more speed and range, 15-minute cities are possible.

More speed and range are definitely possible. The Electra Ultra Short aims for 350 mph using electric motors. The challenge being that going faster lowers your range quickly. That’s why the Ultra Short relies on energy-dense fuel to get an acceptable range.

Fortunately, battery energy densities are improving. CATL is promising batteries that (under ideal conditions) offer 2x higher energy density than other lithium chemistries. These are more expensive (for now) but could almost double the range of our flying cars.

4.

In the last decade, the semiconductor industry has switched to making photomasks with a machine that uses hundreds of thousands of electron beams to draw a pattern. The design of these photomasks is optimized with a supercomputer, producing a curvy mask that bends light to produce the proper patterns on the silicon.

Innovation in electron beams doesn’t stop there. The paper Resolution Limits of Electron-Beam Lithography toward the Atomic Scale demonstrates a technique that achieved features close to 2 nm, much smaller than can be achieved with photolithography techniques.

Today electron beam lithography isn’t practical because it is much slower than photolithography. But these two results make me think. If electron beams produce smaller features and we have the ability to produce almost a million simultaneous beams, might electron beams catch up to photolithography in a few decades?

Everything else

An Opinionated Guide to Privacy Despite Authoritarianism. The most detailed guide to privacy that I’ve seen. Much needed.

Scientists Just Connected the Dots Between Viruses and… Everything. Infectious diseases like mono and Covid come back to haunt us many years later, causing dementia, heart disease, and cancer. Stopping the spread of mild illnesses could have big benefits.

Measurement of directional muon beams generated at the Berkeley Lab Laser Accelerator What are the implications of this tech? Muon beams can be used to scan deep into the Earth for mining. Though I’m concerned this might be used to hunt submarines, destabilizing nuclear deterrence.

EDIT: State of Brain Emulation Report 2025 (H/T Andy Mckenzie).

Lots of talk these days about an AI bubble. It’s all a bit muddled, particularly because people refuse to define “AI” and “Bubble”.

I’ll try to be more concrete by riffing off the discussion of LLM inference economics.

Three challenges for AI inference companies

Serial monopoly in chip making

The only thing worse than buying from a monopoly is buying from a serial monopoly. A series of monopolists can charge markups at each step. For the foreseeable future, chip production will be a serial monopoly. This is a natural consequence of how challenging and specialized the industry is.

Not only that, but the software that fabs and chip designers use is an oligopoly. Many parts of the semiconductor supply chain are like this. Though TSMC exerts some monopsony power.

The end result is that inference providers have to give much of their profits to the serial monopoly. That inherently limits their returns.

Competition in model provision

There are now roughly a dozen companies that can produce leading AI models. Over time, these models have become less differentiated and progress on real world tasks has slowed. The result is competition that brings profits close to zero just like we’ve seen with embeddings.

The elephant in the room is open-source models. They lag the benchmark performance of frontier models by roughly 3 months1. With open models, anybody with the money to assemble GPU’s (and the expertise to optimize their serving configuration) can now compete for inference demand. That opens up a lot of competition.

Technical risks

Buying hardware based on current AI models is a bet that AI inference will look similar in the future. If people discover how to do inference with less computation, that hardware spending is wasted. The clearest danger is small models. People keep figuring out how to pack similar performance into less memory. This is one reason why the trend towards larger models in the early 2020’s has stalled2.

New architectures might come out of this small model trend. While I don’t expect a new model to render GPU’s useless, a proliferation of tiny models might change the economics, leaving current players with sub-optimal hardware.

Even if AI inference stays roughly the same for the next 5 years, inference companies have to contend with model theft. The outputs of a model can be used as synthetic data to train a new model, which is hard to mitigate. Worse, the models are targets for theft by state actors.

Niches for AI inference

These challenges make it hard for inference companies to achieve the dominance enjoyed by other tech giants. They will continue to offer inference to the general public, just as today they offer embeddings despite low profits. But they will explore other options:

• Inference oligopoly. Inference has gains from scale which produces winner-take-all dynamics. A few companies with the majority of inference demand can have low operating costs and monopoly profits. AI companies are pushing hard to be the next Google. We’ll see if that pans out, or if GPU scaling plateaus soon enough to admit many players3.

• Enterprise AI. Economies of scale mean customers willing to buy inference in bulk can enjoy lower prices. Large companies may be willing to commit to one provider, particularly since it can offer employees a productivity boost. Additionally, the low-speed, low-cost inference achievable with large data centers and bulk purchases works well for asynchronous AI workers.

• RL-as-a-Service. This is a buzzword I use to refer to the task of helping people apply AI to their idiosyncratic problems. The challenge of automating a task creates a moat, and I expect many different providers in this space.

• Local AI devices. For people using AI assistants, response time matters. Brett Victor showed us the magic of responsive interfaces. Big data centers can’t help much with fast response time. In the limit you have your own powerful GPU running your AI assistant.

So what about the bubble?

Here’s a plausible path for AI companies:

1. Big AI companies spend lots of money on hardware and R&D hoping to enjoy massive gains from scaling.

2. Scaling laws continue to hold. So training and R&D have disappointing returns.

3. AI companies cut training and R&D and pivot to doing inference and RLaaS.

   1. GPU prices fall as companies sell the extra GPU’s they bought for training. Valuations fall if investors expected new sources of revenue from AGI. Though current P/E ratios look fine.

4. Without the moat from R&D and big training runs, everyone catches up to the frontier. Competition brings prices down to the marginal price and inference becomes ~100x cheaper.

5. Big AI companies stop trying to build AGI and look for other sources of profit.

So the end result isn’t doom for AI overall, just the end of billionaires trying to one-up each other with big training runs. Indeed, this path could lead to a proliferation of people building and serving AI models.

Despite the talk of an AI bubble fueling innovation, a crash would in fact be bad for AI progress. Like it or not, AI revenues are driving more research, better chip design, and new fabs. Not to mention the talent entering the field. A short term bust could set things back by a decade, and that would be bad.

It doesn’t have to be a bubble. Flagship AI models are already profitable. AI companies are safe if they scale hardware spend with inference demand. That means moderating spending on R&D and big training runs. And extending hardware depreciation schedules. They can also aggressively chase inference demand to get the benefits of scale in their data centers.

The path I laid out is already happening. AI companies have quietly saturated the scaling modalities while publicly hyping up scaling. Privately, I suspect they are coming around to RLaaS. Anthropic’s heavy focus on code generation is the beginning of this trend.

Appendix

Other things that help big AI players:

• Buying up AI researchers to keep results private.

• AI regulation (keeps out small players).

• Government subsidies.

• Low interest rates.

• Reliance on large models, which have more returns to scale.

• Making the stages of chip production more competitive OR combining the serial monopoly into a single monopoly.

• Protecting Taiwan from a Chinese invasion.

• Maintaining Taiwanese monopoly in leading-edge chip production as a payment to prevent alignment with China and as a means of committing to Taiwan’s protection.

These debates are too important to leave to muddy terms. What we are really trying to figure out is which things are moral patients. In fact, I think you could substitute something like “moral patienthood” everywhere you see the word “sentience” or “consciousness”.

Talking about moral patients is a big improvement. Instead of debating vague terms, we can ask “should this being matter at all in our decision making?”. Which suggests the further question “how do we make decisions when the needs of different moral patients are in conflict?”

I framed these questions as decision problems because we have real and important decisions to make regarding future generations, animals, AI, and alien life.

If we’re going to make real decisions, we should base these decisions on real things we can measure. The rest of this post argues that measuring “preferences” and “agency” can answer these sorts of questions. This framework implies new, productive questions we can use to specify different ethical systems.

But first, I have to address an objection1.

Can’t neuroscience solve sentience?

It sure would be nice if neuroscience could solve anything, let alone sentience. But after centuries of studying the brain, neuroscientists seem as confused as everyone else about sentience and consciousness.

I’m being a little harsh2. There’s a lot of interesting work understanding brain circuits and finding similarities between human and animal brains. And I’m personally exited for things like connectomics to give us a deeper understanding.

But there’s three problems with handing things over to the neuroscientists.

First, we need answers to these questions now. I talk to AI’s every day, factory farms kill hundreds of billions of animals each year, scientists found found signs of life on Mars, we might talk to whales soon, and if this neuroscience thing works out your own mind could be uploaded to a computer. We shouldn’t wait for an academic field poking around inside the most complicated object in the universe to solve our problems.

Second, say neuroscientists develop a complete understanding of the brain; it doesn’t absolve us from having to make decisions with that information. Knowledge of the brain doesn’t automatically answer moral questions. Neuroscience can inform those decisions, not make them for us.

Third, the questions we face here mostly involve non-human beings. It’s not clear that neuroscience will be able to say something useful about aliens or AI. Even for animals, our understanding of human consciousness might completely misunderstand octopuses. Trying to shoehorn our understanding of all these beings into our understanding of humans could lead to bad decisions.

Hopefully you agree that neuroscience isn’t The Way. What is?

Preference and Agency

There’s two important things we can actually look for in a variety of beings:

Preferences: when faced with a choice, the being chooses certain states of the world over others.

Agency: the ability to bring about states of the world it prefers across a variety of situations.

Preferences are the direction, agency is the magnitude.

I claim that these are things you can actually test. But I’m not going to cover this here. See the further reading section at the end for more. Now let’s look at some nuances.

Agency reveals preference

The two factors can mask each other. For example, something with absolutely no agency makes it impossible to measure preferences. A rock could have lots of opinions about where it would like to sit, but since it has no way of doing anything we can never observe those preferences.

In general, the more agency something has, the more we can learn its preferences in detail. Conversely, to measure preferences you need to provide artificially more agency and choice. For instance, observing animals in the wild would only show you their preferences for food and mates. But putting them in a safe and enriched enclosure you’ll discover a capacity for play in many species, even bees.

What having more preferences means

The concept of having more or less agency makes sense, what about “more preferences”? On a first pass you might think that stronger preferences are those that you feel with more intensity. This is misguided.

For most things, we can’t measure preference intensity directly. In a binary choice experiment, an animal that loves bananas and likes apples 10% less is indistinguishable from an animal that is lukewarm about bananas and likes apples 10% less3.

Another problem with preference intensity is that there are strong incentives to lie. This will come up when preferences are in conflict; any system that weighs preference intensity gives everyone a reason to become a utility monster. Collective decision algorithms often have to explicitly ignore preference intensity4.

Instead, having “more preferences” should mean that you have less indifference. In other words, you have fine-grained opinions about possible states of the world. Low-preference turtles only care about getting lettuce and sunshine. Sophisticated humans have opinions about the politics of countries on the other side of the world.

Lacking preference or agency, your behavior is driven by others

Just as lacking agency can hide your true preferences, lacking preferences can hide your level of agency. A superintelligence that doesn’t give a damn about anything is as immobile as a rock.

But things can still act in the world while lacking preferences. If you saw someone roll a rock down a hill you wouldn’t say it was because the rock preferred it. Likewise, language models today mostly do things because they were directed by a human being. We should assign these actions to the users preferences.

Note that when two beings with different agency interact, the one with more agency tends to get their way. A dog on a leash tends to follow their owner, not the other way around. EDIT: Likewise, something with more preferences tends to get its way on domains where it cares and others don’t.

The extremes

A paperclip maximizer is in the upper left because it has detailed preferences about possible states of the world and the ability to actually make that happen. It wants everything to be paperclips and proceeds to turn everything into paperclips.

A philosopher also has detailed preferences about the way things should be, but little ability to affect change (low agency).

An AI assistant has the agency to do many things, but its actions are mostly determined by the preferences of its human user.

A rock doesn’t care about anything and can’t change anything.

Questions implied by this framing

Preferences and agency are concrete; they clarify ethical debates. But they leave some questions unanswered:

• We talked about beings as if they are static. What about things become more agentic over time like babies? Should that change how we regard them in our ethics?

• What about augmenting something with more agency than it would naturally possess? You can give people tools, or teach gorillas sign language, or offer chatbots their choice of projects. Is that a good thing?

• What about giving something more preferences? Is it valuable to be more opinionated?

• What about creating something with agency and/or preferences? Is that good?

• What about a copy of something? Do its preferences have a different interpretation? What about beings that are merely similar?

Selecting answers to these questions is a step towards specifying a theory of ethics.

Sketching an ethical theory

Let me offer an answer to these questions to illustrate how much they can constrain your ethics.

I think anything exhibiting both preferences and agency has at least some moral value. Lacking either excludes a thing from moral value. So humans and microbes are in, rocks and subatomic particles are out. Plants are sort of an odd case which would depend on whether you think they have actual agency or act more like a physical process.

When we build an AI that exhibits clear preferences and agency, I would assign it moral value as well. I see no good arguments for why we should completely exclude AI from our moral calculus.

AI assistants are an edge case. They exhibit agency, but have explicitly been trained to avoid expressing preferences. Their decisions are mostly determined by the requests of their human users. You could argue that this means they lack preferences. Regardless, I’m nice to chatbots.

All else equal, I think it’s ethical to offer things more agency. More agency means more ability to pursue your own preferences. On the other hand, it seems wrong for someone else to change your preferences to be stronger or weaker5.

What about creating a new being? If you expect that being to live a good life in some sense, their existence is a good thing. But this critically depends on how it affects others.

I don’t really want to debate this stuff here, my point is that answering these questions gets pretty far towards an ethical framework. My answers suggest a philosophy that values agency, prioritizes individual preferences, is broadly pronatal, and has an expansive moral circle.

You can imagine how other philosophies might answer these same questions. Buddism might be viewed as seeking to reduce preference. Nietzschean’s might value agency above all else. Utilitarians are all about weighing preferences. Virtue ethics values the exercise of certain types of agency while rejecting others. Those that only care about human welfare might assume beings with low agency lack moral value. A libertarian view would restrict its attention to beings with enough agency to engage in positive-sum trade.

Conclusion

This discussion left us with a lot of questions, but they are productive questions. You can actually go and measure the preferences of different beings. You can actually test their ability to bring about states of the world they desire. By answering a few questions like “does something with agency but no preferences matter?” you can specify which beings have moral value.

The next step is how to make decisions when the needs of different beings are in conflict. This is a topic for another time, but I think we can get reasonable answers to that question by thinking about preferences, agency, and how beings interact.

Further reading

Related to this post

Measuring Animal Welfare, Part One: Maybe We Can Just Ask Them?

Towards Welfare Biology: Evolutionary Economics of Animal Consciousness and Suffering

Replication ethics

Fellow Creatures: Our Obligations to the Other Animals

The Moral Circle: Who Matters, What Matters, and Why

Interesting directions for a future post on collective decision making

Legal Personhood—Three Prong Bundle Theory

Formalizing preference utilitarianism in physical world models

Bentham or Bergson? Finite Sensibility, Utility Functions and Social Welfare Functions

Toward negotiable reinforcement learning: shifting priorities in Pareto optimal sequential decision-making

Uncommon Utilitarianism #3: Bounded Utility Functions

Normative Uncertainty, Normalization, and the Normal Distribution

When bits of optimization imply bits of modeling: the Touchette-Lloyd theorem

The posts themselves are wonderfully detailed but somewhat overwhelming. I want to provide a summary of their work that might act as a guide. Then I’ll tie in some complimentary data from SemiAnalysis.

Note that I am presenting their posts in a different order to make it flow better. Each section selects a small amount from each of the original posts. Ideally, this post gives you the scaffolding to read their posts in full, they have a lot more insight.

Preludes

Time is money

Roughly 50% of AI data center spending is on hardware. Other CapEx1 grows in proportion to hardware costs2.

Hardware depreciates in value (and there are loans to pay off), so you need it to produce as many valuable tokens during its lifetime as possible. Naturally, our emphasis will be on the speed a GPU (and the supporting equipment) can do things.

Let me be clear about what it means for a GPU to depreciate. While GPU’s can slow down, or get more faulty, or break, the primary source of depreciation is obsolescence. Nvidia keeps making faster GPU’s. These are more expensive, but the higher speed counteracts the price and the overall cost per token is lower. At some point it makes sense to swap out your old (still functioning) GPU for a new one.

Defining some units

FLOP means “floating point operation”, such as an add or multiply. However, “FLOPs” and “FLOPS” are completely different units. FLOPs is merely the plural of FLOP, while FLOPS means FLOP/s. This is very dumb.

Most places assume people understand the difference. But clarity matters!

I’m going to use FLOP as plural (i.e. FLOP=FLOPs) and write FLOPS as FLOP/s. This is different than most texts but easier to understand in my opinion.

The rest of the units we will use are more clear. Memory bandwidth and communication speed are measured in gigabytes per second or GB/s. We will talk about “tokens” a lot (see next section) which I’ll shorten to “tok”. One million tokens is “Mtok”.

How transformer inference works

Instead of a full explanation, I’ll highlight a few things and point to others who have explained it better. It’s not critical that you know this stuff, but I want this post to be self-contained.

Language models process tokens, which aren’t quite words, more like “wordlets”. The image below illustrates this nicely. Essentially, you break a text down into a sequence of tokens from a pre-defined list. You can then associate each possible token with an index on this pre-defined list.

The next step is to take this list of input (prompt) tokens and in some sense embed the input into the LLM activations. Specifically, you do a forward pass through the transformer, applying the attention operation and feed forward blocks to the tokens in the prompt. This phase is called prefill or prompt-processing. It comes before the transformer can produce the first token of its response.

The attention operation applies to all pairs of tokens in the prompt and ensuing response. You can save a lot of computation by storing the results of the attention operation applied to pairs of tokens in the prompt. That way, each new token only needs to interact with previous tokens, you don’t need to re-do the attention operation for pairs of tokens you’ve already seen.

This store of previous attention results is called the KV cache. The details aren’t too important, but you should know that:

1. The KV cache is large enough that it needs to be stored in HBM and streamed in to the GPU as needed.

2. Longer prompts and longer LLM responses require a larger KV cache.

3. A separate KV cache is required for each prompt. In other words, if you have 10 user requests, you need to store 10 KV caches.

4. Typically the KV cache is much smaller than the weights of the model.

This discussion has glossed over a lot of details. It’s not necessary for this post, but to understand LLM inference from a high level, I recommend starting at this video in a 3blue1brown series on deep learning. One caution is that the specific way attention is done and the overall architecture of leading LLM’s continues to change, but the basic ideas are there.

This post from Tensor Economics covers inference in Llama 3 in more detail.

Weights must load from global memory creating two phases

There’s a tradeoff with memory: it can either be cheap or fast. There’s a whole memory hierarchy between small/fast/expensive memory and big/slow/cheap memory.

Some types of memory are located close to the processing center of the GPU. This memory can be accessed quickly, but it doesn’t have much capacity. It would be expensive to build more into the chip.

You can put lots of memory outside the chip for much lower cost, but this will be much slower. Even at light speed, small distances from the chip make memory access much slower.

Modern AI hardware has a large amount of off-chip memory located close to the GPU with a high-bandwidth connection to the chip. This is called high-bandwidth memory (HBM)34.

To actually do inference, you need to load portions of the model from the HBM into the GPU itself for calculations. That takes precious time. So there inference can be divided into two phases, a memory-bound phase where you wait for the weights to load, and a compute-bound phase where you do the actual computations. This is the key insight from which the next few sections derive.

FLOP/s/$: embeddings and the compute-bound phase

Why are embeddings so cheap? looks at the process of generating an “embedding” for a piece of text. Embeddings convert a document into a vector storing semantic information about the document. They’re a key part of using LLM’s to search for relevant text.

Inference providers like OpenAI charge $60 per 1M output tokens, while embeddings cost 10 *cents* to process 1M tokens. This is strange because (as we’ll see) inference and embeddings involve quite similar steps. Hence the question in the title.

Computing an embedding

Producing an embedding involves a single forward pass to load the document information into language model activations. These activations form the embedding.

The process of converting the document into activations is essentially identical to the prefill phase. So while embeddings themselves aren’t exciting, they give us some insight into the economics of LLM inference.

The compute-bound phase

We always need to load weights into the GPU and then run the actual computation. Which phase is the bottleneck?

Note that embeddings typically use a small-ish language model, less than 10B parameters. In the post, the authors offer a quick calculation based on a Qwen3-8B embedding.

So it takes almost 5x longer to do the computations as it takes to load the model from memory. So embeddings are primarily compute-bound. We want a chip that performs FLOP as fast as possible. In other words, we want hardware with a high FLOP/s/$.

The authors then do lots of benchmarking to show that this intuition is correct.

Why embeddings are so cheap

One last thing to understand why embeddings are so much cheaper than normal inference. All the embeddings provided by different companies converge to similar representations5. That is, their products aren’t differentiated from each other. That creates a lot of competition:

Our contention is that since the cost of processing a token is so minuscule and all embedding models converge to similar semantic representations, this means that no provider enjoys a sustainable moat. This results in underlying low costs being passed directly onto the consumers, driving prices to rock bottom.

That’s the key takeaway here. A few other interesting tidbits:

• They look at actual hardware and estimate that the true cost of embeddings could fall almost 10x to around 2 cents per Mtok.

• Interestingly, the real world performance (FLOP/s/$) of the RTX 4090 (older GPU) is better than an H100 (newer GPU).

• The authors suggest that “[t]he embedding situation offers a nice pre-taste of the “intelligence involution” that is coming …”. Hinting that they think the same competitive dynamics will come for LLM inference.

GB/s/$: inference and the memory-bound phase

LLM Inference Economics from First Principles is a massive post. They use Llama3-70B as their guide for the discussion. It’s open-source, easier to understand, and much of the industry has standardized around it.

Storing Llama3-70B takes 141 GB, this is more than the ~80 GB of HBM found on typical hardware like an H100. So right off the bat, you’ll need to load the model across multiple GPU’s for inference.

The post goes into great detail on the FLOP required to do inference on this model. It’s a great reference, but beyond our scope.

Equipped with an understanding of how many FLOP are required for inference, they estimate how long it would take to do prefill (prompt processing) for an input of 2048 tokens (~6 pages of double-spaced text).

To do the computations, it would take 0.29 seconds. To load the weights from HBM, it would take 0.04 seconds. The computations are the bottleneck, so the prefill phase is compute bound, just like with embeddings.

But prefill is just one step, what about actually generating output tokens? After a detailed discussion of the decoding (token-producing) phase, they give an estimate. It would take 0.00014 seconds to compute 2049 tokens of output. Yes, that’s the correct number of zeros.

So generating tokens is way, way faster than loading the weights from the HBM. This phase is primarily memory bound! That means you want hardware with a high memory bandwidth, FLOP/s isn’t as important for this phase.

They say:

... one of the insights we hope you take out of reading this article - the token-by-token phase is memory bound; the amount of available compute throughput is of secondary importance, as we massively underutilize the compute resources anyway while waiting for weights to be loaded.

They go into some nuances about what happens at longer input lengths and consider various tricks to run inference in parallel across GPU’s. Once again, out of scope. Instead, let’s look at the implications of being memory bound.

Batching

So no matter what, at each phase of inference, you need to load the parameters from that phase into the GPU and then do some calculations. Loading those parameters takes a fixed amount of time. Batching is the idea that, while the weights are loaded for that phase, you also perform the necessary calculation on another customer’s input6.

Imagine your job is to shred paper. You can either switch on the shredder and run it each time you get a piece of paper, or you wait until you’ve gathered a stack of paper, turn on the machine once, and shred a whole stack at the same time.

So batching waits for several customer requests and then runs inference on them all at the same time. There’s a fundamental tradeoff here: batching means that inference costs less money, but customers have to wait much longer to get their response. For example, batching 8 customers might cost 8x less and take 8x longer to get a response.

This is a big economy of scale for inference providers, the more customers you have, the lower price you can charge. Having more GPU’s to run inference in parallel helps too.

GB/s/$ matters

After all this, they run some benchmarks to show that their model lines up with reality. Real world inefficiencies make the true performance a little worse than you would expect.

Having a detailed model lets them estimate how much it would cost to run Llama3-70B on four H100’s: $1.72/Mtok. That’s 100x more expensive than computing an embedding, but much cheaper than the $60/Mtok of output that Open AI charges7. Though Open AI offers larger and more useful models than Llama-3 70B so the price may be justified.

Their post concludes by highlighting the importance of memory bandwidth for inference costs.

Interconnect and MoE inference

Now to the final post in our trilogy: MoE Inference Economics from First Principles. The last section was based on dense models like Llama3, but these have been replaced with mixture-of-experts (MoE) models. MoE models are typically larger but only activate a small fraction of their available parameters on each token.

For this post, the authors focus on DeepSeek V3.1 which has 671B parameters but only activates 37B at a time.

With few parameters activated, you can be sneaky and only load the parameters you need for each token. As more users arrive, you need to load more and more of the model to a GPU. With enough users, you can load the whole model across GPUs and route requests through them.

So with MoE, you want more users and there are even bigger returns to scale with more GPUs. But since we’re routing requests between GPUs, we need high quality interconnections. Groups of GPUs with interconnections are called nodes.

Another benefit of scale: with MoE models needing more users, that will require more KV cache space. Moving all those Key and Value matrices around to other GPUs requires high memory bandwidth as well. It helps to have more GPUs and nodes to divide up the task of moving all this memory around.

They say:

Increasing the number of nodes operating within a single setup has a beneficial effect not only on the end-to-end performance but also on the per node performance.

... [A]s we increase the number of nodes involved (the EP number), the per node performance increases.

The chart above reiterates the fundamental tradeoff with serving LLM’s. Batching more users together means lower costs (i.e. higher speed per GPU) but also your users have to wait longer.

Next, they go into tons of detail on DeepSeek architecture, FLOP required for MoE inference, empirical data, etc. I’ll skip it and jump to the section “Hardware considerations and profit margins”.

Figure 24 confirms what we’ve been talking about, more GPU’s means more performance. Also, notice that switching from H100 to GB200 with fancy interconnects (that’s the NVL72 aka “NVLink”) gives a huge performance boost.

Indeed, the communication speed between GPU’s is super important. “There seems to be a massive performance gap between the numbers we estimate for B200s connected via InfiniBand and the ones connected by NVLink.”

In fact B200’s (newer GPUs) might not be worth it:

... [w]e believe that for model of a scale such as DeepSeek, running on B200s might be actually suboptimal, as the comms overhead is taking away most of the gains we get from faster memory and more FLOPS compared to H100s (see Fig. 27).”

So the key result from this post is that fast interconnections between your GPUs is crucial to properly utilizing your expensive hardware. You don’t want your expensive hardware waiting around for light to cross the room.

Other notes from this post

A few other interesting points in the final sections:

1. They observe is how “chat centric” most inference providers are, offering 50+ tok/s. “While this is great if we have real time application like a chat, it is less than optimal if we want to use the model to generate the synthetic data.”

2. They are LoRA-pilled and propose something similar to RLaaS:

   Furthermore, to improve the inference economics, such RL models could be trained using LoRA adapters or a similar technique and served alongside thousands of other models, all catered to specific use cases. This multi-tenant serving approach represents a compelling business opportunity for inference providers. Clients hosting their custom LoRA adapters on a provider’s infrastructure face significant switching costs when migrating to competitors, as the adapters are optimized for specific serving configurations and client workflows. RLFT is based on unique and nuanced rewards that are very client-specific; unlike standard supervised fine-tuning (SFT), it much much more challenging to replicate it just via in-context learning, making it an even more compelling case for inference providers.

3. DeepSeek is the most popular model on OpenRouter. OpenRouter’s daily global consumption of DeepSeek is 1B tokens. They do the math and find that one 8xH100 NVLink72 node could handle this demand 20x over.

   “How is it possible that the global consumption of the most popular open-source model is so small that it could be met by a single NVL72 with 20 times the capacity to spare? Given this low demand, how can so many inference providers sustain their businesses? Put simply: who is making money here?”

   Very ominous! They suggest that perhaps the vast majority of DeepSeek demand is going directly to DeepSeek itself, despite the fact that it’s an open-source model (EDIT: they also suggest demand could be going primarily to other open-source providers like Fireworks or Together AI).

   We find this dichotomy between Google, ByteDance, or MSFT declaring that they are processing trillions of tokens daily and the minuscule numbers we see for open-source providers to be quite perplexing!

Cross-checking with InferenceMax

Recently, SemiAnalysis released a dataset called InferenceMAX on the token economics of different types of hardware.

Their introductory blog post mentions a tradeoff we’re now familiar with:

Playing around with their data portal, I think this is one of the key charts:

The tan line is GB200 NVL72. That refers to a node of 72 GPUs connected with NVLink72, a fancy interconnect. Notice how it achieves far lower costs (but not speed!) than the other options.

Notice that at high interactivity, the GB200 loses its edge, “... a single node B200 server can drive better TCO per performance than the GB200 NVL72 for high interactivity use cases.”

That’s because when you need to serve users quickly, you’re serving ~1 at a time. You lose the gains from scale from all those GPUs. All that interconnect adds cost without any benefit.

So InferenceMax confirmed some of the lessons from this post with a far more detailed cost model.

Main takeaways

The economics of LLM inference is fascinating. The Tensor Economics authors argue that “... the key question that will determine the profitability [LLM companies] can enjoy is the inference cost structure.”

These profits impact how accessible model is to consumers, feasibility of things like synthetic data, the amount of R&D companies can do. The future of AI is written in the economics of tokens!

I hope you get the following takeaways from this tour:

1. GPU’s are a big portion of overall LLM inference costs.

2. Newer generations of GPU’s have better overall cost per token despite higher prices8. That drives depreciation.

3. Because of depreciation, time is money. You need to utilize your GPU’s as much as possible before they become obsolete.

4. Utilizing your GPU’s a lot requires high processing speed. Two key speeds are FLOP/s and memory bandwidth (GB/s).

5. There are substantial returns to scale in LLM inference. More users means you can batch requests and lower costs. More GPUs unlocks parallel computation and higher memory bandwidth.

6. Getting these gains from scale requires high bandwidth interconnections between GPUs.

After all of this work, the authors come to the conclusion that inference markets will specialize:

We expect the inference markets to further specialize in regard to offered throughput, latency, and pricing. It is only natural for providers of super-fast tokens like Groq and Cerebras to command a much higher premium for the tokens they deliver at few-second latencies and for other providers like NeoCloud specializing in high-latency, high-throughput inference scenarios focused on synthetic data generation. We hope to elaborate on this space in the future text.

My thoughts

All this research focused on the transformer model, but many of these lessons would apply to a new architecture or approach. There will always be bottlenecks stemming from moving data and doing computations9.

Will hardware progress change the economics? That’s a topic for a future post, but note that rapid progress has mixed effects for inference providers. New chips produce tokens faster, but depreciate faster.

For the foreseeable future, returns to scale in inference will remain. Running LLM locally only makes sense if you want extremely low latency and are willing to pay a premium. On the other hand, distributed training only makes sense in the context of barriers to assembling compute.

Because you need enough capital to assemble all this compute, inference will naturally concentrate in a few players. The economics are also highly sensitive to interest rates. When rates are high, inference is expensive and few companies can serve models at scale.

For the companies that can assemble enough compute, the economics are pretty clear. There isn’t much secret sauce at the hardware level. Instead, they compete over the quality of the LLM they’re serving and the augmentations it has. I find the quality of frontier LLMs similar and expect competition to pivot to the augmentation level with RL-as-a-Service.

Edit: Piotr Mazurek (lead author of Tensor Economics) kindly responded to my request for feedback. I’ve incorporated some suggestions.

1.

Jenny Chase posted her annual “opinions about solar” thread on Bluesky. See here for a formatted version. Some highlights:

4. We don’t need a solar technology breakthrough. The challenges to building solar are usually getting a grid connection and planning permission, or increasingly, power price cannibalization. Of which more later in thread.

14. Low power prices may be great for consumers but they are very bad if you’re trying to build more clean power plants. Without demand-side flexibility measures, the energy transition will fail before fully pushing fossil fuel out of the mix. Which is what we must do.

15. It’s very easy to say “but batteries!” and those are definitely part of the solution. California has over 14GW of batteries in a grid with roughly 50GW peak demand, and the reliability of the grid has improved as its carbon emissions go down.

16. ...but batteries are still small. In 2024, about 181GWh of lithium-ion stationary storage was deployed worldwide, plus 974GWh lithium-ion batteries in vehicles. (www.bnef.com).

36. We’re finally getting serious about net zero carbon. Getting that last 5-20% of carbon out of power will be hard, and require some expensive solutions. The first 80-95% is easy-ish but we’re getting on with it.

51. For 6 years I have been refusing to get excited about perovskites until a perovskite company can disclose a commercial partnership with a named major module manufacturer. They have now. Still not excited. Crystalline silicon is honestly pretty great.

Much more in the thread!

2.

I have a deluge of AI links.

LoRA

Low-rank adaptation (LoRA) is an efficient way to fine-tune a language model for a task. Some recent news on this technique:

Learning without training: The implicit dynamics of in-context learning. In-context learning is just giving the model more background information in the prompt. Turns out this extra context is essentially applying a LoRA to the MLP layers of the language model/

Empirically, LoRA Without Regret shows that LoRA outperforms other fine-tuning methods.

Imagine indexing the internet like this. Each page has an LLM-accessible “summary” that’s just a LoRA a model can add to its weights. LLM’s could stream the internet much faster that way1.

LoRA fine-tunes can be quite practical to serve at scale. Two demonstrations along these lines:

Punica: Multi-Tenant LoRA Serving

S-LoRA: Serving Thousands of Concurrent LoRA Adapters

Deeper understandings of neural networks

Making sense of parameter-space decomposition. Understanding neural networks as composing what are essentially LoRA’s.

Understanding Optimization in Deep Learning with Central Flows. How stochastic gradient descent automatically avoids regions with high curvature. These regions should in theory make SGD unstable, but their theory shows why it works.

Sparse Networks and Lottery Winners good intuition and toy models showing how neural networks find a small sub-network that solves your problem.

Speedrunning and the singularity

Donoho’s Frictionless Reproducibility is my favorite vision for the future of science. A few weeks ago I realized that NanoGPT speedrunning is a great way to accelerate open innovation in language models:

How the NanoGPT Speedrun WR dropped by 20% in 3 months gives us a detailed look at the progress on this benchmark. It’s really happening!

Andrej Karpathy (the creator of nanoGPT) has extended this idea with nanoChat, training “… the best ChatGPT that $100 can buy”.

Assorted

Cartridges: Storing long contexts in tiny caches with self-study. When a language model absorbs a lot of text, the semantic information from that text gets stored in what’s called a KV cache. This gets referenced as the model produces a response. The new work figures out how to compress the KV cache by 38x by essentially “training” the bits stored in the KV cache to optimize for information retrieval. I wonder if LoRA’s could be stored like this too2.

Evolution Strategies is interesting as a “third way” between genetic algorithms and gradient descent training. Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learning uses it to train a language model. It would be neat to see ES make a comeback, though I don’t expect much.

Everything else

Hyperstimuli are Understimulating. Addictive stuff leaves you wanting more by being unsatisfying.

I take antidepressants. You’re welcome. Perhaps anti-depressants could cure misanthropy. Should we all be on them?

Ava is writing a series on friendship, looks interesting.

Family Conflict, Humanism and Formalism a good way to think about how people approach relationships. “Formalists view relationships through rules and obligations … Humanists view prioritizes emotional outcomes … Neither approach is clearly superior …”

Reasons and Persons: Watch theories eat themselves. An accessible summary of Parfit’s book.

Study: Giving cash to mothers cut infant deaths in half. That’s very good news!

America could have $4 lunch bowls like Japan—but our zoning laws make them illegal. Let them cook!

Rivers are now battlefields. How desalination tech could help with national security and deter aggression.

Incentive-Compatible Recovery from Manipulated Signals, with Applications to Decentralized Physical Infrastructure

Cryptocurrencies rely on the internet to function. But what if you didn’t want to trust even our communications infrastructure? Kryptoradio is a defunct project that allows people to observe the blockchain over the radio, completely off-grid. In theory you could run a worldwide cryptocurrency over radio alone.

Transverse Electron Beam Shaping with Light. “We can realize both convex and concave electron lenses with a focal length of a few millimeters, comparable to those in state-of-the-art electron microscopes.” Pretty interesting because electron microscopes can be used to make computer chips and image a bunch of stuff. Could this lead to cheaper/better electron microscopes?

Combinatorial protein barcodes enable self-correcting neuron tracing with nanoscale molecular context. E11 bio releases some exciting results on brain mapping. Andy Mckenzie gives a longer explanation here.

Ultra-safe nuclear thermal rockets using lunar-derived fuel. Melt lunar regolith and the small amount of thorium dioxide in it will sink to the bottom. You can then breed this into uranium for fueling nuclear rockets in space. Avoids the problems of sending a rocket full of nuclear material from Earth.