Links #16

Interpreting AI compute charts, take the vertical out of vertical farming, much cheaper cultivated meat, and more.

1.

This is a graph from Epoch showing the computed used to train notable AI models. These can be tricky to interpret. For one, even though the spread of the dots around the pink line looks about the same in any given year, the y-axis is on a log scale meaning that the spread is actually increasing. For example, models in 2014 ranged from about 1E14 to 1E18 flop, and in 2022 it was about 1E20 to 1E24 flop. Those span the same number of ticks on the plot, but the variance is 1 trillion times higher (1 million squared). This is important because we don’t care too much about the compute of the average model, but the compute of the highest model.

Let’s look at the expected size of the largest model in future years. Assume that we take the log of all the FLOP values and fit a trend line like Epoch did. Let’s also assume that each year, the average a_t from the trend line holds and that each point comes from a Gaussian distribution around that average. What is the expected amount of compute Z for the most powerful model in a given year?

Well if there are n companies putting out new models in a given year, then it’s:

\(E[Z] \leq a_t + \sigma \sqrt{2 \log{n}}\)

Where I’m taking that formula from the expectation of the expected maximum of a Gaussian sample.

With our logged data, a_t increases linearly each year, so the formula becomes:

\(E[Z] \leq a_0 + m \cdot t + \sigma \sqrt{2 \log{n}}\)

Where a_0 is the x-intercept and m is the slope of the best fit line. At what time t will E[Z] hit a certain threshold? The addition of the square root term moves that date closer than you might expect.

Using some plausible numbers¹, when I ignore the square root term, the date of dangerous AI is November 2038, but when I do take it into account, the date of dangerous AI is April 2035. That’s 3.5 years sooner!

So the end result is that we get models that cross a particular threshold faster than we would expect from looking at the trend line². This happens sooner if there are more companies taking “samples” from this distribution or if the variance of leading models increases.

2.

Tomas Pueyo made a great point in his pieces on vertical farming. Plants only use a small part of the solar spectrum, so solar panels and LED’s can be placed above them to deliver precisely the wavelengths they need to grow. We can produce energy and food on the same land.

The thing is, vertical farming is just too expensive to work. Instead, we should take the vertical part out entirely. Take a normal farm, put solar panels above it, and string up LED’s underneath the panels (you’ll need to rethink the irrigation system too). Panels and LED’s are getting extremely cheap so these costs should be manageable. If you want to seal off the crops from pests, add some walls to the solar roof you’ve constructed.

The key challenge will be making this compatible with farm equipment like tractors. It’s too hard to get harvesters small enough to fit under the panels and nimble enough to weave around the posts. Instead, roll the panels off parts of the land in the early morning or late evening.

If you can make the solar panel racks cheap, robust, and easy to move, this would increase the productivity of farmland a lot.

Related: Tomas Pueyo has a nice post on driving the cost of solar energy down. I don’t know what’s past the paywall but I hope he covers the fact that directly using solar DC electricity while the sun shines can eliminate the cost of batteries, inspections, permitting, inverters, etc.

Related: an update to the Pakistan solar story I mentioned here.

3.

Elliot Swartz has an excellent thread on making cell culture media out of protein-free (i.e. cheap) components instead of using albumin (paper here).

The paper makes some optimistic assumptions and determines that this cultivated chicken product would cost $21.49-$38.54 per kg of wet mass³⁴. That’s about what it costs to get higher quality beef at the grocery store which is pretty good.

However, the finished product would resemble ground chicken, so this is still a little expensive compared to real meat. And even with this innovation, the growth media is still 60-70% of costs, so further innovation in synthetic media could lower costs by up to 2-3x.

What’s next? The new media contains cyclodextrin and methylcellulose so I wonder if these could be produced in bulk from crop residues or other waste products. And to what degree can the media solution be recycled?

Another possibility is breeding better cell lines. The Swartz notes that “… the chicken fibroblast lines have an efficient metabolism without needing additional modification”. But why not continue to breed more efficient cell lines⁵ ?

The overall feed efficiency is 3x better than farmed chicken which is great⁶. It’s stuff like this that makes me think that we can greatly reduce the amount of land and resources we need for agriculture⁷. Together with moving towards making things over growing things, a lot of previously agricultural land may be used for other things.

Everything else

A team designed a changing QR code to transfer 100 kB/s between a screen and a phone camera (code, website). This is cool because changing QR codes are a good way to prove that two people are in the same location which would be useful for some dating market mechanisms.

Coordinating Decisions via Quantum Telepathy. You can use entangled bits to correlate actions over long distances better than you can with any classical source. This could have applications in arbitrage, but it’s hard to see how it would be of much use.

Bret Victor’s Dynamicland has a new site reviewing their progress from the last 10 years. They’ve got a very cool medium for thought and I’m excited to see other communities try to work with it.

Migration Financing: Filling the Missing Market with Income Sharing Agreements. Immigration is has high costs that prevent people from moving but can dramatically raise incomes. Income share agreements are a natural fit for this since they align the interest of borrowers and lenders. But unfortunately there is no legal support for them today.

How the FAA Is Keeping Flying Cars in Science Fiction. Maxwell Tabarrok points out that despite similar levels of regulation on cars versus planes, aviation has seen much less innovation. He argues that the fact that FAA regulations are applied before-the-fact makes it challenging to innovate. Cars on the other hand have regulations enforced after-the-fact with torts and lawsuits. The after-the-fact system takes the pressure off regulators to catch issues before they happen. I think this after-the-fact regulation model could be applied well elsewhere.

Refillable planet models the economics of baby bonuses. They estimate that the U.S. government should be willing to pay $290K per baby, which is also sufficient to return to above-replacement fertility. They suggest that better fertility policy research and more targeted interventions could make the overall cost smaller.

Works in progress discussed the promise of GLP-1 inhibitors months before they hit the mainstream. They later published a piece on SGLT2 inhibitors, and lo and behold, we have new evidence that they help eliminate senescent cells! Will these drugs combined with rapamycin and better medical treatments add years to our lifespan?

This researcher wants to replace your brain, little by little. A nice profile of Jean Hébert who is trying to grow fresh brain tissue and insert it into the brain, hoping that this tissue will integrate with existing tissue. Surgically this will be a challenge to implement at scale, but perhaps we could add and remove a few cells at a time using less-invasive methods?

1. a_0: -1217.37

m: log(4.1)

Threshold compute: 1E32 FLOP, or 32 since our data is logged.

n: 10 companies

Sigma: 1 (remember, the data is logged, so this corresponds to points that differ by a factor of 10 from the mean in FLOP terms)

2. And the number of companies is increasing over time, which changes things. More companies could also dilute capital or increase innovation.

3. Their headline cost estimates assume 50% of the finished product is a cheaper plant-based component to balance the structure and nutrition. This is fine to assume, but makes it harder to compare to real meat, so I’m sticking with the estimates for the “raw” product.

4. I would guess that cultured meat products contain more water by mass than real meat, so the cost per gram of protein will likely be higher than these estimates suggest.

5. What might happen is that cancer-like cells take over the cell line, as happens elsewhere in cell culture. This would increase efficiency by default which is good. I would guess that the cultured meat industry hasn’t looked into this much yet.

6. Though I think we can do orders of magnitude better.

7. See also point 3 here.