The Department of War has stated they will only contract with AI companies who accede to “any lawful use” and remove safeguards in the cases mentioned above. They have threatened to remove us from their systems if we maintain these safeguards; they have also threatened to designate us a “supply chain risk”—a label reserved for US adversaries, never before applied to an American company—and to invoke the Defense Production Act to force the safeguards’ removal. These latter two threats are inherently contradictory: one labels us a security risk; the other labels Claude as essential to national security.
Regardless, these threats do not change our position: we cannot in good conscience accede to their request.
It is the Department’s prerogative to select contractors most aligned with their vision. But given the substantial value that Anthropic’s technology provides to our armed forces, we hope they reconsider. Our strong preference is to continue to serve the Department and our warfighters—with our two requested safeguards in place. Should the Department choose to offboard Anthropic, we will work to enable a smooth transition to another provider, avoiding any disruption to ongoing military planning, operations, or other critical missions. Our models will be available on the expansive terms we have proposed for as long as required.
I guess it’s good that they draw the line somewhere, but it is absolutely horrifying to me as a non-American that the moral stance is limited to:
- taking issue with fully autonomous AI weapons (purely for technical reasons according to this letter, they are working hard on making them possible)
- refusing to conduct mass surveillance of US citizens specifically (foreign nationals are fair game and the intelligence apparatus will surely only be used for good and to preserve democracy).
This is not Anthropic refusing to cooperate with the Trump administration as the title may suggest, they are in fact explicitly eager to serve the US Department of War. They are just vying for slightly better contract terms.
vying for slightly better contract terms
Do you mean that all this about principles is a smoke screen and Anthropic are just using it as a front to squeeze for more money?
No, if you want my opinion it seems too risky of a move to make all of this so public if all they want is more money. It’s possible, but I’d be surprised.
I believe them when they say that what they want is to have those two particular things, fully autonomous weapons and mass surveillance of US citizens, removed from the contract terms (for now). This could be out of genuine moral principles, or out of fear of bad PR when this would be found out. Most likely a combination of both.
My point was that from my perspective it is a very minor difference. The conclusion I kept after reading this isn’t “good guy Anthropic bravely stands against pressure from Hegseth” as some of the Hackernews comments try to paint it. It is “Anthropic mostly bends over backwards and grovels for Pentagon money, willing to massively spy on all foreign nationals and working on creating autonomous weapons - other US AI companies likely to be even worse”.
As I said, horrifying.
Crossing off mass surveillance and automated killing isn’t everything they could have taken a moral stand on. Personally I don’t think any list will be long enough for the Pentagon, and if it were, there wouldn’t be anything left that could be worked on.
But I keep hearing you say that no mass surveillance and no automated killings is so very little - almost nothing. That doesn’t seem right to me. I think those are both pretty big things. TBH I don’t know exactly how to feel about it all but I’m not horrified that their moral stance would include only that.
That’s a fair stance to take and I definitely do not mean to try to have you change your opinion. I also do not know if you are an American, and I don’t want to assume either way.
But, to better explain my own position, I need to point out:
Anthropic is not saying “no mass surveillance”, they are saying “no mass surveillance of Americans”. If you judge this stance based on effect, it literally makes no difference at all if you are not a US citizen, you are targeted either way. If you judge it based on principles, it can be argued it is even less moral than accepting mass surveillance of everyone - not only are they claiming that billions of innocent people deserve to lose their right to privacy, but they are specifically carving out an exception for themselves based on nationality.
They are also not saying “no automated killings”, but “no automated killings at this time because we haven’t ironed out the kinks yet”. This can be framed as a moral stance relating to safety concerns, so I will assume in good faith that this is their reasoning rather than fear of bad publicity. However, I would argue that it is still an insignificant difference, as the threat posed to humanity by a powerful warmongering state commanding an army of fully autonomous killing machines is already too great. Making sure the technology is ready could mean working on avoiding a Terminator scenario, but without a doubt it will also mean ensuring that the murderbots WILL obey an order to bomb striking workers or displaced refugees so long as the right Executive Order was signed first, something that a human being in the loop might have prevented.
These two red lines seem to make a world of moral difference for someone who already takes it for granted that the USA and its military are overall institutions deserving of trust and support, perhaps with the small exception of the current Secretary of War who may have jumped the gun a bit during negotiations over a new technology. At the very least, that seems to be the position of the author of this letter. But no state should ever be given that amount of trust and support. And particularly given the USA’s belligerence over the years and its current slide towards outright fascism, I am horrified that the bar is this low.
Better to be skeptical about everyone here, and there are certainly no heroes.
However it should be obvious that a country’s department of war surveilling its own citizens is a completely inappropriate overreach. They exist to protect the country from outside threats. You’re casting it as some kind of discrimination, and claiming it would be more moral to treat everyone the same, but that seems willfully obtuse to me. Calling it a “special carve out” for a country to protect its own citizens… come on. Obviously since you are not an American it does nothing for you but you are working way too hard to spin that up into a sin.
Obviously a country spying on its citizens is unacceptable overreach, I never claimed otherwise. And if my own government was conducting mass surveillance on me I would be particularly furious at the betrayal. But I would also not support it conducting surveillance on foreigners either. That is the “sin” Anthropic is guilty of, in my eyes.
Mass surveillance is simply immoral. It is targeting innocent people who have not even been accused of any crime and robbing them of their right to privacy. It is also giving states absolute leverage to harm, blackmail or manipulate anyone they want at will.
The argument that it is all done in the name of protecting its own citizens also falls flat in this case, as that is exactly the same excuse used for mass domestic surveillance - everyone loses their privacy, but the good, law-abiding citizens are protected from the criminal elements who would threaten them. “If you have nothing to hide, you have nothing to fear”.
Let’s not kid ourselves, this is not about protecting anyone. They plan to spy not only on their “enemies” but also on their closest allies, as they have in the past. This is about gaining power. And states in general already have far too much power over individuals.
Kowtowing to the Department of War and offering to sell them an AI for mass surveillance is not OK, even if it truly were to limit itself to the common, genteel use case of spying on foreign people.
I’m hardly going to defend the Pentagon, but to say a country should not even have an intelligence operation whatsoever, that this isn’t elementary to protecting its citizenry, is beyond naive and unrealistic.
How is a private company the voice of reason in this?
Anthropic was founded by former OpenAI employees who left largely due to ethical and safety concerns about how OpenAI was being run. This is just them sticking to their principles.
Anthropic’s “ethical” concerns were performative. They only fearmonger about fictional things that will make their product sound powerful (read: worth throwing money into).
They try to scare people with fictional stories of AGI, a thing that isn’t happening, while ignoring widespread CSAM and sexual harassment generation, a thing that is happening.
Are we not moving toward AGI? Because from where I stand, I only see three scenarios: either AI research is going backwards, no progress is being made whatsoever, or we’re continuing to improve our systems incrementally - inevitably moving toward AGI. Unless, ofcourse, you think we’ll never going to reach it which I view as a quite insane claim in itself.
If we’re not moving toward it, then I’d love to hear your explanation for why we’re moving backwards or not making any progress at all.
Whether we’re 5 or 500 years away from AGI is completely irrelevant to the people who worry about it. It’s not the speed of the progress - it’s the trajectory of it.
We are not “moving towards AGI” in any way with any modern technology, in the same way that we are not “moving towards FTL travel” because a car company added cylinders to an engine.
The real “AI” dangers are people like Eli Yudkowski, a man who scares vulnerable people, sexually abuses them, and has spawned at least one murderous cult.
Dario is one of the biggest AGI bullshit peddlers.
In October 2023, Amodei joined The Logan Bartlett show, saying that he “didn’t like the term AGI” because, and I shit you not, “…because we’re closer to the kinds of things that AGI is pointing at,” making it “no longer a useful term.” He said that there was a “future point” where a model could “build dyson spheres around the sun and calculate the meaning of life,” before rambling incoherently and suggesting that these things were both very close and far away at the same time. He also predicted that “no sooner than 2025, maybe 2026” that AI would “really invent new science.”
We are not “moving towards AGI” in any way with any modern technology
So that means you believe AI research is completely frozen still or moving backwards. Please explain.
Comparisons to faster-than-light travel are completely disingenuous and bad faith - that would break the laws of physics and you know it.
You can also keep your red herrings to yourself. I’m discussing ideas here - not people.
According to Dario Amodei, this is the year we are getting New Science. And apparently he believes in Dyson Spheres too. How do we feel about that?
Anthropic is not special. They’re doing the LLM thing like everybody else. The Godfather of AI, Yann LeCun himself, said LLMs were a dead end on this front. But even if he didn’t chime in, it’s your job to show they’ll lead to AGI, it’s your job to show us how, not my job to show you it won’t.
If you’re just gonna keep ignoring every single point I make and keep rambling about unrelated shit, then there’s nothing left to discuss here. If you actually had an argument, you would’ve made it by now.
This is just them sticking to their principles.
I’m not so sure about that one.
AI company Anthropic amends core safety principle amid growing competition in sector
AI safety leader says ‘world is in peril’ and quits to study poetry
I still think they deserve some credit for at least trying to do the right thing. I don’t envy the position they’re in.
Everyone’s rushing toward AGI. Trying to do it safely is meaningless if your competition - the ones who don’t care about safety - gets there first. You can slow things down if you’re in the lead, but if you’re second best, it’s just posturing. There is no second place in this race.
No AI bro company is on the path to AGI. Transformer technology will not lead to AGI.
I never claimed it will.
