What is a dead startup's Slack worth to a model?

TL;DR

• Brokers sell defunct startups' code, Slack and email to AI buyers. The prices buyers quote vary wildly.
• Data Shapley values a training point by its average marginal contribution to model performance, averaged over every order the data could arrive in.
• It shows why the value of data depends on what the buyer already has, and why redundant data is worth less than its volume suggests.

What happened

MIT Technology Review's Matthew King reports that a market has formed for the digital remains of startups that have shut down. SimpleClosure, a wind-down service, added data sales in April through a platform called Asset Hub. The company says it has brokered more than 150 deals and expects to return over $10 million to startups by the end of 2026. Its rival Sunset values a client's assets by head count, years in operation, data volume and how the team used its tools. One founder, Keelan Miskell, says he netted roughly $25,000 for his crypto startup's code base, with about $15,000 more pending for its Google, Notion, Figma and Slack data. One buyer is Surge AI, which uses archives like these to build "reinforcement-learning gyms" where AI agents practise tasks.

Nobody has settled on a price. Surge AI's Nick Heiner says buyers quote "wildly different numbers" because valuations rest on "research theses" about how many useful tasks, say, 100,000 lines of code will yield. He also says one company's Slack history can be worth more than another company's Slack, email and calendars combined, if its conversations have variety and depth. The biggest deal is a matter of public record. In August, during Spirit Airlines' bankruptcy, Google outbid Mercor for the airline's data, which includes 100 million emails, 500 million Teams messages and 176,000 employee records. Google paid $10 million. Reuters reported that Mercor bid $7.5 million, and that Spirit said the data would be de-identified before the sale closed. That is the seller's claim, not the result of an independent audit. Attorney Greg Ewing calls this kind of data "legally radioactive." Law professor Ifeoma Ajunwa says "Employers should have express consent before selling workers' data." Apart from Spirit's court-supervised auction, the deal counts and payouts come from the companies and founders themselves.

The paper in brief

Ghorbani and Zou start with a training set, a learning algorithm and a performance metric, and ask how much of the model's performance each training point should get credit for. The obvious answer is leave-one-out: remove the point, retrain, and measure the drop. That breaks down whenever the data is redundant. The authors borrow the Shapley value from cooperative game theory instead: a point's value is its marginal contribution, averaged over subsets of the other data with a particular weighting. They show it is the only valuation that satisfies three fairness axioms. They make it computable with truncated Monte Carlo sampling, and they find that low-value points flag mislabeled or noisy data while high-value points suggest what to collect next. The paper mattered because it turned "what is a data point worth?" from a slogan into a quantity you can define and estimate.

The math

Setup

There are three ingredients. The first is a training set $D = \{1, \dots, n\}$. Each "point" can be a single example or a whole source, such as a hospital or, in principle, one startup's archive. The second is a learning algorithm $\mathcal{A}$, treated as a black box. The third is a performance score $V$. We write $V(S)$ for the test performance (say, accuracy) of the model that $\mathcal{A}$ trains on the subset $S \subseteq D$. The output is one number $\phi_i$ per point. The paper makes no distributional assumptions, not even i.i.d.

Leave-one-out, and where it fails

$$\phi_i^{\text{LOO}} = V(D) - V(D \setminus \{i\})$$

This is the performance lost when point $i$ is removed from the full set. It is a finite-difference version of a partial derivative: nudge one input and see how the output moves. The trouble is that it only looks at one context, "everyone else is already here." If a point has a near-twin, removing it costs almost nothing, so it gets valued at roughly zero even when the information it carries matters a lot.

The Shapley value

$$\phi_i = \frac{1}{n} \sum_{S \subseteq D \setminus \{i\}} \frac{V(S \cup \{i\}) - V(S)}{\binom{n-1}{|S|}}$$

The term $V(S \cup \{i\}) - V(S)$ is point $i$'s marginal contribution to the coalition $S$. There are $\binom{n-1}{k}$ coalitions of size $k$, so dividing by that count gives every size the same total weight $1/n$. In words: average $i$'s marginal contribution over all coalitions of size 0, then size 1, and so on up to size $n-1$, and then average those $n$ averages. Leave-one-out is just the size-$(n-1)$ term.

The paper's version has an arbitrary scale constant in place of $1/n$. It shows that any valuation with the following three properties must take this form:

• Null player: if adding $i$ never changes $V$, then $\phi_i = 0$.
• Symmetry: if $i$ and $j$ make identical contributions to every subset, then $\phi_i = \phi_j$.
• Additivity: $\phi_i(V + W) = \phi_i(V) + \phi_i(W)$. Test performance is a sum over test points, so a point's total value is the sum of its values for each test point.

Choosing $1/n$, the classic game-theory constant, also gives efficiency: $\sum_i \phi_i = V(D) - V(\varnothing)$. The values exactly split the improvement over an untrained model.

The permutation view

$$\phi_i = \mathbb{E}_{\pi \sim \text{Uniform}} \left[ V(P_i^{\pi} \cup \{i\}) - V(P_i^{\pi}) \right]$$

Here $\pi$ is a uniformly random ordering of the $n$ points, and $P_i^\pi$ is the set of points that come before $i$ in that ordering. Imagine the data arriving one point at a time in random order, with each point paid whatever it added when it arrived. This is the same formula as before. The probability that $i$'s predecessors are exactly $S$ is $|S|!\,(n-1-|S|)!/n!$, which equals $1/\big(n\binom{n-1}{|S|}\big)$, the same weight.

A worked example: the duplicate problem

Take three points: $a$, an exact copy $a'$, and a different point $b$. Suppose the accuracies are:

Training set	$V$
∅	0.50
{b}	0.60
{a}, {a′}, {a, a′}	0.80
{a, b}, {a′, b}, {a, a′, b}	0.90

Leave-one-out: $a$ gets $0.90 - 0.90 = 0$, and so does $a'$. $b$ gets $0.90 - 0.80 = 0.10$. So LOO calls both copies worthless, yet dropping both cuts accuracy from 0.90 to 0.60.

Shapley: $a$ comes before $a'$ in 3 of the 6 orderings. In those it adds 0.30, whether or not $b$ is already there (0.80 − 0.50 or 0.90 − 0.60). In the other 3 orderings it adds nothing. So $\phi_a = \phi_{a'} = 0.15$. $b$ adds 0.10 in every ordering, so $\phi_b = 0.10$. The total is 0.40, which is exactly $V(D) - V(\varnothing)$. The two copies split the 0.30 they provide together.

Making it computable

The exact formula sums over $2^{n-1}$ subsets, and each one needs a model retrained from scratch. With $n = 1{,}000$ that is about $10^{301}$ training runs. The permutation view points to a sample mean instead:

$$\hat{\phi}_i = \frac{1}{T} \sum_{t=1}^{T} \left[ V(P_i^{\pi_t} \cup \{i\}) - V(P_i^{\pi_t}) \right]$$

This averages over $T$ random permutations. It is unbiased, and by the central limit theorem its error shrinks like $1/\sqrt{T}$. One permutation gives a sample for every point at once, at the cost of up to $n$ retrains as you scan through it.

Truncated Monte Carlo (TMC-Shapley) adds one trick. Learning curves flatten, so once the model trained on the current prefix is within a tolerance of $V(D)$, it stops retraining and gives every later point in that permutation a marginal contribution of zero. That buys a large speedup for a small bias, a familiar bias–variance trade. For models trained by gradient descent, the paper also has a gradient-based variant. It replaces each full retrain with a single gradient step on the new point, so each permutation costs roughly one pass of training.

Where truncated Monte Carlo stops retraining

interactive

• gap to full-data accuracy
• tolerance

Hover or drag across the chart to read values.

learning-curve exponent α0.5performance tolerance0.01

retrains per permutation (of 1,000):

312

retrains skipped:

68.8%

Values can be negative. A mislabeled example or a corrupted image tends to lower test accuracy when it arrives, so its average marginal contribution falls below zero. Note that $V$ is measured on a test set someone chose, so every value is relative to that test distribution.

What the results say

The figures below come from the paper text we worked from, an earlier version titled "What is your data worth?". The ICML version's experiments may differ in detail.

• Finding bad labels: when points are inspected from lowest to highest value, Shapley surfaces mislabeled points with the fewest inspections across three models. Leave-one-out does reasonably well with logistic regression but is close to random on the other two. When 10% of the images in a dog-vs-fish set get white noise, their average value falls as the noise level rises.
• Patients (UK Biobank): logistic regression on 1,000 patients reaches 68.7% accuracy for breast cancer and 56.4% for skin cancer. Removing high-value patients hurts accuracy, and removing low-value ones improves it. Choosing new patients from a pool of 2,000 by their predicted value beats both random selection and LOO.
• Whole sources: the paper values 22 UK health centres. One, Nottingham, has a clearly negative value for colon-cancer prediction. Age, the strongest predictor overall (KS test, $p = 1.5 \times 10^{-6}$), shows no significant difference between diagnosed and healthy patients there ($p = 0.14$).
• Reweighting toward a target domain: dropping negative-value points and weighting the rest by value raises accuracy from 29.6% to 37.8% for skin lesions (trained on web images, tested on HAM10000), from 84.1% to 91.5% for gender classification on PPB, and from 87.5% to 90.1% on veterinary notes.

Limitations. The authors stress that a value only means something given a specific algorithm, metric and set of other data. They say plainly that people should not be paid exactly their Shapley value, and that privacy and personal association are outside the framework. A careful reader should add a few more points. The experiments are small: about 1,000 training points, and networks with every layer frozen except the top one, because each value needs many retrains. Truncation is a heuristic that introduces bias. The baselines are mostly LOO and random, not stronger methods. And the domain-adaptation gains rely on labeled target-domain data to compute the values.

Connecting the dots

This is a paper about the same idea, not the method behind the market. Nothing in the reporting suggests that brokers or labs compute Shapley values. Sunset's inputs (head count, years in operation, volume) measure how big an archive is, not what it adds.

Still, the framework explains why the market looks the way it does. In Data Shapley, "what is this worth?" has no answer until you fix the algorithm $\mathcal{A}$, the metric $V$, and the rest of the data. Buyers with different tasks and different existing corpora should quote "wildly different numbers." Heiner's "research theses" are informal guesses at $V$. His Slack comparison is the duplicate example at company scale: Slack, email and calendars from the same company may record the same conversations three times, so their marginal contributions overlap. The paper also values whole sources and finds that a source can be negative, as with Nottingham. By the same logic, a startup archive could make a model worse for a given task.

The news also goes beyond the paper. Shapley divides credit for a payoff that is already known, after training. A buyer has to set a price before buying, without that payoff in hand. Computing exact values at frontier-model scale is out of reach. And the paper's experiments are supervised classifiers, while Surge AI uses the data to build agent environments, which would require redefining $V$. The "legally radioactive" part, meaning consent and the personal information in employee records, lies entirely outside the framework, as the authors themselves acknowledge.

Why it matters

For practitioners, the lasting lesson is to price data at the margin. Measure candidate data against what you already have, on the task you care about, rather than by gigabytes or head count. Low-value scores also make a cheap debugging tool for spotting mislabeled or off-distribution examples. For research, the open problem is still cost: credible values for large models without thousands of retrains. For policy, the paper began with the idea of paying people for their data. Its own caveats, that value depends on context and leaves out privacy, are a good reason to keep "what a model gains" separate from "what a person is owed."

Two things to watch: whether buyers start pricing archives on measured gains, such as trial training runs, rather than on volume; and whether courts or legislators require worker consent before employee communications are sold, a question the Spirit sale has already put in front of a bankruptcy court.

Go deeper

• Data Shapley on arXiv, the paper (Ghorbani and Zou, 2019)
• Official Data Shapley code from the authors
• MIT Technology Review: AI labs are buying data from dead startups, the news source
• Reuters, via The Star: Google to buy Spirit Airlines business data for $10 million
• The Morning Paper on Data Shapley, a readable walkthrough that includes the TMC and gradient variants