There’s a firestorm raging around California’s Safe and Secure Innovation for Frontier Artificial Intelligence Models Act — aka SB1047, with all sides insisting the arguments of the other side are simple corruption. SB1047 is the product of “industry capture,” its opponents say. An effort, it is said, by the proprietary AI companies to shut down the “open source AI developers.”
I’ve expressed support for SB1047. Its protection for whistleblowers is important; its reach is relatively narrow (“covered models” are those “trained using a quantity of computing power greater than 10²⁶ integer or floating-point operations, the cost of which exceeds one hundred million dollars ($100,000,000)” or models fine-tuning covered models “using a quantity of computing power equal to or greater than three times 10²⁵ integer or floating-point operations.”); the risk of a Trump administration means it is critical we cement some of the Biden protections.
Many in the open-source community fear the law would disable open-source models. I’m not sure I know of any open-source models in America that would be affected. No doubt, Llama would be, but a license that forbids competitors from using the model is not open source (“You will not use the Llama Materials or any output or results of the Llama Materials to improve any other large language model (excluding Meta Llama 3 or derivative works thereof).”). And even if you consider Llama open source, the law expressly exempts open source developers from kill-switch responsibility for any model they don’t control (§22602(l): “Full shutdown” means the cessation of operation of any of … covered model derivatives controlled by a developer.).
But this debate is interesting to me for a different reason: it’s an elegant example of the many ways in which money might corrupt an argument. It might be useful then to get clear about the different degrees of concern one might have about the integrity of what someone says as a function of the interests that person has.
1. Interested parties
We should call someone an “interested party” if they have an economic interest in their expressed view being adopted by policymakers. Or, put differently, if policymakers agreed with them and followed their recommendation, would they be better off economically? I don’t mean better off in the way all of us would be better off — an economist recommending smart tax policy might be better off if the policy was adopted, simply because the economy was better off. That doesn’t count. I mean better off in the sense of being directly tied to the change in policy. So, e.g., banning open-source development would make proprietary companies (and their investors) better off. An advocate with an economic interest in a proprietary company advocating for the banning of open-source development would, therefore, be an “interested party” in the sense of the term I am describing.
By contrast, a non-interested party is someone with no specific economic interest in the outcome of a policy debate. “No” should not be too narrowly defined. If you run a research project, and you advocate for a policy that benefits a potential funder, at least with the intent to woo that funder, you are an interested party. But again, if your funding or economic well-being is not tied directly to the outcome of a policy choice, you are a non-interested party.
Critically, of course, whether you’re an interested party or not does not determine whether what you’re saying is true. Yann LeCun is a brilliant AI researcher working for Meta. His advocacy of policy that defends the freedom of (Meta’s version of) open-source makes him an interested party. But I’m certain much (maybe most, maybe all) of what he says he would say even if he were completely separate from Meta. His connection to Meta is thus not a reason to reject his view. But his interest — like the interest of people on the other side—gives people a legitimate reason for skepticism.
2. Rich parties
An interested party is different from a rich party. Rich people fund all sorts of things. Some of the things they fund they could benefit from economically. In those cases, they are interested parties. Some of the things they fund they could not benefit from economically. In those cases, they are not interested parties. The Koch Brothers were interested parties when they funded advocacy against climate change legislation: such legislation would have burdened them economically. By contrast, George Soros is not an interested party when advocating for the freedom of women to choose whether to terminate a pregnancy or not. Assuming he’s not running for-profit abortion clinics, he does not benefit economically from his policy being adopted.
No doubt, there are interesting intermediate cases. For many years, I’ve been helping drive the strategies to end SuperPACs. That work has been funded in part by wealthy people. On the one hand, you might say those wealthy people are not interested parties because the end of SuperPACs won’t benefit them economically. To the contrary, their funding of anti-SuperPAC work is them funding the reduction of their own political influence, which could, in some contexts, weaken their economic opportunities. Yet, on the other hand, as one funder commented to me, if SuperPACs end, many of these people will be spending much, much less in politics, so maybe we should say they would ultimately benefit economically if SuperPACs were ended. Regardless, in my view, as SuperPACs have become the most consequential and polarizing spending in America today, not to mention the most corrupt (see, e.g., Trump offering to relax oil regulations if oil execs gave $1B to his SuperPAC), I’m totally ok with wealthy people getting this benefit.
Yet there is an interesting, maybe unfair, effect that the way we fund campaigns today does have on the policy views of rich people. Specifically, it cheapens the value of the words uttered by rich people.
For example, in a world of SuperPACs, the wealthy give ungodly amounts to political campaigns (through those SuperPACs). They also have views about good policy. But whenever they utter those views, the views are cheapened by the contribution: “I gave your SuperPAC $45M; here are my views about EVs” gets read as nothing more than “accept my views because of the money” rather than “my views are true and valuable on their own, separate from the money.”
Many have come to despise the wealthy. That’s too crude. Some have developed powerful systemic accounts of the harm caused by the wealthy. Anand Giridharadas’ Winners Take All is brilliant and a must-read. Whatever your view, no doubt there are more important social policy concerns than whether the views of rich people are read for their truth or their influence. At least if you’re not a rich person.
3. Academic Signals
All this leads me back to a thought I’ve long had about academic research: We need a better way to signal independence — or specifically, that we are not an interested party. If I had another life to lead (or the funding to do it), I’d launch another version of Creative Commons, focused on academic integrity: The system would give academics a way to certify (and mark) their work as “independent,” meaning it was done as a non-interested party (in the broad sense I’ve described above). Too many in too many fields depend directly on the interests they are to be evaluating and critical of. Again, that’s not to say that what they say is false. It is only to say that we need a simpler way to say: whether you agree or not, at least know these views are not interested views.
4. As applied to the SB1047 debate
Obviously, the loudest on both sides of this debate are interested parties, though maybe differently so. VC firms, proprietary firms, open-source (or “open-source” firms): They’re all economically interested.
But I don’t think it is fair to say that everyone involved in this debate is an interested party. (Or even worse, to suggest that one side is interested and the other side just truth, justice and the scientist’s way.) Many academics critical and supportive of the bill are — I assume, though we can’t really see—non-interested parties. I, in particular, have spent the year meeting each week with senior researchers and former policymakers, reviewing and working through AI risks; none of the people I’ve spoken to in those discussions has been, in this sense, an interested party. That doesn’t make them right, any more than being an interested party makes someone wrong. But it is simply false to say that the only supporters of SB1047 are people who have an economic interest in its being passed. Yes, interested parties have been involved on both sides. Is anyone really surprised by that? But they are not the only people involved in this debate.
