Rendered at 07:31:07 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
OsrsNeedsf2P 3 days ago [-]
DS isn't better than Sol and Gemini in the benchmarks, but after working with them all for a while now I'm beginning to understand why people find DS _feels_ so much better.
First, it's the tok/s and TTFT from providers. Deepseek blazes through work - at quite good quality - and you don't find yourself waiting for your agents to finish.
Second is the openness. Being able to see the CoT and not being randomly blocked for cyber review. You're never wondering if it's stuck or if it will randomly fail. Combined with the great cost value.. My next subscription will probably be Z.ai
Philip-J-Fry 3 days ago [-]
Feels better to me in the sense that it doesn't try to be too clever. When I ask it to do things it actually generates code that a developer would write. That still lets me build software in layers like I do mentally. And it keeps you in control of where abstractions are.
Some people would say that you don't need this at all with the frontier llms. But when extending existing software or writing software that fits into an existing ecosystem, I find I still need to offer a lot of guidance. And it's better that the context window isn't flooded with code that was reverted after I looked at it.
jamesbfb 3 days ago [-]
> Some people would say that you don't need this at all with the frontier LLMs
Hit the nail on the head. Development methodologies pivoting at the whim of frontier models have been one of the things that have been worrying me as of late.
trueno 3 days ago [-]
it doesn't worry me too much tbh. This is really just cutting corners and going this route leaves you with some kind of additive ship of theseus piece of software you don't understand and require your frontier model to work with, which just... becomes more additive and gross.
right now we're in some proverbial heyday of everyone seeing how absolutely grand of a thing they can make with frontier models.
over time i expect some old design philosophies to take hold with people who use LLM's successfully and find a place again paired with developers who understand why such layered methodologies are important... because this whole AI thing wasn't the silver bullet CEO's and crazy execs thought it was. They will eventually get sick of the support burden of badly designed stuff. i anticipate a bit of pendulum swinging back towards "still using AI, but we need smart people to do it because it's expensive and we want it done right". Right now a lot of companies are being extremely reckless imo and they haven't fully realized/felt the cost implications of trainwreck software yet.
LPisGood 3 days ago [-]
What if those “cost implications” are just asking Opus 6.7 to fix whatever issue cropped up?
state_less 2 days ago [-]
Asking the LLM to 'just' fix 'it', or whatever, is going to get you lost in the woods.
You're car will have spun off the road and you'll find it harder and harder to control it and keep it on the road because you have less and less visibility into the vehicle, nor will you be able to clearly see out your windscreen to use the car and road metaphor. You'll be driving blind eventually, unless your working on some simple closed toy problems and/or other small systems.
Building large systems means understanding the system. Without understanding the system, you lose control of it.
The LLM can probably help you understand the system to gain/regain some control of it again, but now you're becoming a system specialist rather than someone who's repeating, "fix it" and "make it good" or whatever.
radicalbyte 2 days ago [-]
There are a whooooole lot of people writing software who, despite spending years on the craft, never really get any further than McGuyver mode.
Self-trained people (like me) are particularly susceptible to it and we're heavily represented in the software world. A lot of us either recognise that (or are coached) and invest the time learning the benefits, and limitations of, software architecture. And, importantly, keep reflecting, learning and updating our practise.
I keep going back to the YOLO mode every other year and try out new languages to see if we still need the structure and abstractions. Then VMs came along and all I can say is they're both even more important and now we have an army of virtual "ADHD idiot juniors developers" at our finger tips it's even easier to tell the good and bad abstractions from each other.
dominotw 2 days ago [-]
well said. This is exactly why platforms selling vibecoding are scams.
fn-mote 3 days ago [-]
Exactly.
In a way, that’s the situation today. You can pay an experienced developer $$$$ to do it right, or you can outsource it and pay $$ but pay extra for more layers of review, have lots of unfixed bugs and more tech debt.
You are proposing that instead we pay the frontier model to clean it up. If it can, I expect it to cost somewhere in the neighborhood of an experienced dev because that’s the alternative. That’s the way the frontier AI companies will extracting the value of the work from you.
hazard 3 days ago [-]
> I expect it to cost somewhere in the neighborhood of an experienced dev because that’s the alternative
But there's two ways this could be true: Frontier models costs rise, or (more likely) experience dev costs fall
2 days ago [-]
dash2 2 days ago [-]
> I expect it to cost somewhere in the neighborhood of an experienced dev because that’s the alternative.
That's only true if a single company owns the frontier, otherwise the alternative is "a rival frontier model" and the cost will be the marginal price of the compute + amortized training costs.
anukin 2 days ago [-]
Opus on enterprise pricing is not exactly cheap.
Bender 2 days ago [-]
I can not rely on Opus 5.5+ to "fix" whatever crops up. Claude is great but should it find a security related bug one too many times, it gets the Anthropic Shock Collar [cyber]! and I have to pick Opus 4.8 or older to continue and they try to up-sell Mythos which I am not permitted to use without going through their verification program that they will not offer to me. Thankfully for most of my hobby tasks (thats all I do now) Opus 4.8 is usually mostly good enough. In my opinion their newer models are powerful but not something I can rely on. Maybe I am one of the few running into this.
TuringTest 3 days ago [-]
In complex systems, issues keep piling up faster than you can fix them.
agile-gift0262 3 days ago [-]
IMO the fundamental principles and "conflicts" are still the same, but many have forgotten about them with all the hype and sense of speed. The industry will slowly rediscover them.
huijzer 2 days ago [-]
> haven't fully realized/felt the cost implications of trainwreck software yet
As long as the API is private or not dependent on by automated systems and you have good test coverage, you can get away with bad code IMO since you can always refactor. If it looks stupid but it works (and can be refactored/fixed/improved later if needed) it aint stupid.
swader999 2 days ago [-]
Used to say this about dev written code, if it has tests, I can usually live with it.
swat535 3 days ago [-]
Support burden? When did companies care about customer support or system maintenance?
They already did the bare minimum allocation to both despite engineering screaming at them.
I highly doubt executives are going to accept slower release and sale cycles in order to provide better support.
If anything, I expect this trend to accelerate. I’m imagining the opposite, a plethora of AI slop with increasingly burned-out engineers.
fn-mote 3 days ago [-]
> slower release and sale cycles
Sales are out of their control.
If there’s a market for quality and you don’t have it, sales are going down no matter what the PHB wants.
If there’s not, watch out for the vibe-coded competitor.
ekidd 3 days ago [-]
Buggy slop products will burn buyers and then people will buy less. The very-early-80s video game companies shoveled out slop, including a few games that wouldn't even launch. Which led to the Great Video Game Crash of 1983.
Abishek_Muthian 3 days ago [-]
I've been using layered development techniques like the parent says even with frontier models, just one agent at a time with Zed.
brookst 3 days ago [-]
I just use a suite of tests to enforce the mechanical standards (dry, encapsulation, etc) and arch documents for shape + reviews of new work against the two. I’m not seeing any significant drift.
ygjb 3 days ago [-]
Yea, this is one thing that gets me. Vibe coding and freestyle prompting gets wildly different results. Starting from a set of specs and expectations routinely gets me reviewable and we'll tested code. Sure, running the full test suite alot slows things down and uses more tokens, but when I hit a five hour or weekly window, I just roll up my sleeves and start working without the model because the code is layed out the way I expected and is designed to fit my meatbag workflow and pace.
foltik 3 days ago [-]
You must not be looking very closely yourself then.
mapontosevenths 3 days ago [-]
When is the last time you used a frontier model? Any chance you work someplace that forces you to use last year's model (or any American open weight model)?
foltik 2 days ago [-]
I use them every day, in systems programming. They produce a solution, but it’s often several orders of magnitude away from an optimal one in various dimensions. That’s still quite useful, but for some problems that doesn’t cut it.
To reach a more optimal solution, significant human steering is reqiured for every tiny decision. It’s the same as how early models needed hand holding on the basics, playing out again in the finer details. Human experts need to inject “eureka moments” from the outside.
And no feedback loop that I’ve tried has been able to conjure an understanding of these finer details that, as far as I can tell, just aren’t represented in the weights.
Modeling the basics of software engineering was easy, since things like writing code that passes some tests is easy to RL. But for anything not easily RL-able, we’ve basically ran out of human generated tokens to distill from.
Without some breakthrough in generalization, I remain unconvinced that we’ll be able to model the finer details of software engineering by looping the existing training corpus back into itself. Garbage in, garbage out.
jacquesm 2 days ago [-]
That's my experience as well. But: to be fair, they can be super good at debugging.
3 days ago [-]
nfw2 2 days ago [-]
LLMs don't seem to be really that good at managing complex and conflicting needs, especially if it grows in increments, a ticket here and there. (To be fair people often suck at it too.)
I have a theory that mess in an LLM-generated codebase sort of grows exponentially. Each hack or sloppy abstraction chips away at the coherence of the project as a whole and makes the next feature harder to do. But perhaps more importantly, it demonstrates to the LLM, which is a probabilistic machine despite its impressive capabilities, that quality of the code is low, and it generates more low-quality hacks.
And eventually you reach knots upon knots that you have to untangle yourself because the LLM becomes too confused by the code it wrote itself. As models get better, the problem seems to be getting better. I'm needing to do this more and more rarely, and the tradeoff is still clearly worth it.
But I've seen Fable 5.1 absolutely lobotomized by someone who fills each chat context with half a million tokens of insipid garbage, and I don't see why this couldn't happen when coding as well. I'm not sure a solution to this general problem that LLMs can simulate idiots really well will be fixed any time soon.
LeFantome 2 days ago [-]
Be the product manager. Just like real iterative development, you have to take time occasionally to reduce technical debt explicitly.
Adding unit tests, increasing modularity, reducing duplication, and the like are all things you can tell the LLM to do explicitly. And it is happy to do it.
And you can have coding standards as part of your design criteria. Set and document these early in the project and audit the code at least enough to see they are being enforced.
nfw2 1 days ago [-]
> Adding unit tests, increasing modularity, reducing duplication
Agents know these principles already and they are frequently the problem themselves.
Too strongly-insisting on eg DRY can lead to tortured abstractions shoehorning too much together. Agents need to be able to distinguish between incidentally similar code vs conceptually redundant code.
Excessive modularity, eg. militant single-responsibility principle can lead to abstraction hell that is impossible to reason about at once.
Unit tests are obviously good. Coding agents write a copious amounts of them, often baking you into poorly-chosen designs made without your knowledge.
rjzzleep 3 days ago [-]
> Feels better to me in the sense that it doesn't try to be too clever. When I ask it to do things it actually generates code that a developer would write. That still lets me build software in layers like I do mentally. And it keeps you in control of where abstractions are.
yes!So it's not just me. In terms of how intelligent they want to be OpenAI is at the top, then Claude. And actually when I was playing with Sol it meant that many tasks it would just try to do something else because it thought I don't know what I want. Even trying to nudge it back in the right direction wouldn't work.
I guess the models just have the cultures of their creatures.
crossroadsguy 3 days ago [-]
> it doesn't try to be too clever
DS is miles better than GLM in this regard. I wanted to add that as PC is planning to go to GLM next.
drob518 3 days ago [-]
Interesting. I’ve been using GLM 5.3 Flash for a couple months and found it to be great. What has it been doing that you found to be “too clever”? It sometimes gets stuck in loops. I once asked it to do a code review of a simple pattern and it went off on a 45 odyssey trying to analyze too much. But other than those, it’s been a pretty good daily workhorse. And you can’t beat the price. Even Deepseek 4.1 Flash is more expensive. And I switched to GLM 5.3 Flash from DS 4 Flash, my previous daily driver, which I also found quite good. Anyway, my point isn’t to start a “which model is better” subthread, but was just curious what behavior you’ve seen where it’s “too clever.”
runtime_terror 3 days ago [-]
Anecdotal but I was doing regular adversarial reviews of new work using about 5 models (mostly the open Chinese ones like DS, GLM, Qwen, Kimi, etc) and GLM routinely took much longer for similar results and the same instructions.
It would just loop on things and go down rabbit holes more. DS flash has been so consistently good I've kinda settled on it for 90% of my work.
I still use adversarial review for bigger changes where I want to make sure any issues are caught early, but I genuinely feel as tho if no new models ever get made I'd be fine with just using DS forever.
alfiedotwtf 3 days ago [-]
What quant were you using? I’ve been using Q4_K_S for a while now and it’s been rock solid, but having tried Q3_XS, that would loop like crazy. Maybe try a bigger quantity size?
runtime_terror 3 hours ago [-]
Mostly using OpenRouter hosted GLM Flash Latest
crossroadsguy 3 days ago [-]
It wastes my time. A lot! In different ways. That's the tl;dr. Really.
Keeps repeating things I settled as accepted or rejected. But what really gets on my nerves? The language it uses – that aweful, useless typical LLM languages! No AGENTS.md instruction, no per session instruction, no just two-prompts-prior instruction seems to work, hell sometimes instruction regarding that in same prompt doesn't work. GOOD/BAD example along with instruction don't work. That wastes time and zones you out! Not to mention the fatigue of to and fro. DS has been working much better. Hell Gemini 3.1 Pro works so much better than GLM. I don't have extended experience with OpenAI and Anthromic models but the free claide.ai works quite well and better than GLM 5.3 (not even flash). Albeit I must say I rarely use claude.ai for coding, but technical tasks? All the time!
GLM's best response: "No excuse. My fault. You exactly asked me not to do exactly that and so did the AGENTS.md. I failed". Love it. Except that if I press I will get that response every 5 prompts.
welovetacos 3 days ago [-]
In my experience GLM and Claude models behave in ways that steadily drift from your directives and speak in similarly mannered "Claudese", leading me to believe that glm was more or less trained to mimic Claude. I think this reflects a very strong gravitational pull anchored at the system prompt level to specific voice and behavior directives directives. I find other models like DS and Gemini are more compliant and drift much less from my custom behavior and voice directives.
mncharity 2 days ago [-]
[dead]
f311a 3 days ago [-]
For me GLM thinks too much for no reason. It takes a 2-3 turns for DS where for GLM it takes up to 10.
I like giving precise instructions,”update function xx to support input yy, add tests”. There is no reason for 10 turns and 5 minutes of waiting.
zenoprax 3 days ago [-]
I briefly tried GLM 5.3 Flash and the first thing I noticed was how slow it was followed quickly by its single-mindedness. With DS 4.1 Flash I find that it understands that not all the detail in my prompt is important. Sometimes I like to give a quick example of what I'm looking for and it can take the hint whereas GLM will seemingly reshape its worldview around this tiny detail.
DS just handles ambiguity and my bursts of detail with much more grace. I'm only using "max effort" and the CoT is reasonable compared to GLM.
dtech 3 days ago [-]
> the first thing I noticed was how slow it
This really depends on the provider, it's a very fast model with ours
drob518 2 days ago [-]
Right. Also, I think a lot of folks are using it because it’s inexpensive, leading to overload with some of the providers.
Grimblewald 2 days ago [-]
This is it for me, it feels like someone developing alongside me, rather than something like Claude which is constantly fighting to degrade code to a point where it and only it knows what the fuck is going on anymore, which would be defensible if it was at least good code but it isn't.
reissbaker 2 days ago [-]
I've also found this to be true of Kimi K3. I still reach for the Astra power tool for especially hard problems, but it needs a lot of cleanup when it's done — whether on hard tasks or easy ones. So unless I need it I prefer the Chinese models, since if they get the job done it doesn't need much post-processing.
(I truly despise the code Fable generates and IMO it has fewer IQ points than Astra so it's not really in my toolbox these days.)
It's a bummer that American open-source is so bad though.
Grimblewald 2 days ago [-]
Fable is generally massively overhyped IMO. Anthropic has decent offerings, but they're over priced and matched too closely by things magnitudes cheaper AND faster, and since anything interesting isnt gonna get bailed by AI first try, being able to test and review sooner means getting a finished toy sooner as well. Combine that with local models that dont shift underneath you in behaviour like anthropuc/oai do, they're actually gonna have a really hard time convincing anyone like me their service is worth it. Probably explains the fear mongering and push towards controlling local model development and dissemination.
mochizou 2 days ago [-]
I wish benchmarks tracked how often you have to undo changes the model made on its own. Sometimes the less ambitious model is just easier to work with.
dpacmittal 3 days ago [-]
What harness are you using?
UncleOxidant 3 days ago [-]
> My next subscription will probably be Z.ai
Wait... you just spent 3 paragraphs singing the praises of DS... and then you say you're going to subscribe to the one that makes GLM. ?
(I do tend to agree with you, though I tend to favor GLM-5.3-flash over DS 4.1 flash.)
Matl 3 days ago [-]
Z.ai does offer access to DS AFAIK, so I guess that's what they mean? Maybe the feeling is that it provides more value that DS's own platform since you also get GLM, not sure.
saratogacx 2 days ago [-]
I don't think it is the z.ai subscription that offers other models (at least I don't see any on mine). The Qwen token plan has a lot of chinese models including GLM and Deepseek and will probably be the one I use once my z.ai sub gets near the end.
nxdmum 2 days ago [-]
Qwen is not available in India . been wanting to try out that model. anyone from Qwen listening ?
Matl 2 days ago [-]
Good to know, thanks for clarifying.
nfw2 3 days ago [-]
maybe the point was become of the openness, they will be using z.ai?
jeremyjh 3 days ago [-]
But it’s not at all as described in terms of work style - it’s much more like Claude.
nfw2 3 days ago [-]
Reading quickly, I thought the open CoT was the most important piece of comment, probably just bc that seems like it would be the most useful to me.
debugging reasoning traces more effectively seems useful to find prompt issues writing custom harnesses even if it runs on stronger model later
abc42 3 days ago [-]
No need to use a chinese provider, in case that's an issue for you. https://fireworks.ai provides this model for an ok price.
dannyw 3 days ago [-]
I'd rather my data go to China and directly to labs that release open weight models (DeepSeek is MIT licensed still); instead of closed labs and the NSA/CIA; who has far better ability to hurt me than China does.
Without getting too much into politics; over the past several decades, one country is going around starting wars, regime changes and other interventionism, or initiating economic warfare (Huawei sanctions because they were getting too competitive to US companies; GPU restrictions; tariffs; etc), and that's not one administration: it's US policy over the past few decades.
llboston 3 days ago [-]
Have you ever wondered why Manus founders gave up billions of dollars from Meta and “voluntarily” moved back to Beijing? One country does things out in the open, while the other operates in the shadows. The latter worries me more.
throwaway27448 3 days ago [-]
It worries me far more what people are ok with us doing in broad daylight.
> Have you ever wondered why Manus founders gave up billions of dollars from Meta and “voluntarily” moved back to Beijing?
This country wouldn't be such a shithole if we were able to bring the wealthy to heel. I suspect most people on earth see China as the more competently governed country. Understandably, this forum tends to select for people who view private investment positively, but this is clearly insufficient by itself to produce functional societies.
TheArcane 2 days ago [-]
> One country does things out in the open,
boy oh boy, if you think the US doesn't do anything in the shadows. The last 100 years of world geopolictical history would beg to disagree
Der_Einzige 3 days ago [-]
You mean the project that literally had an anus as its mascot? Wasn’t that a joke project? Maybe the government was admonishing them for naming their company “man-anus”
I mean, someone has to say it, but Anthropic is the one who has an anus as a mascot. More specifically, Claude. It even winks at you, and clinches, too. I'm sorry that I have to be the one to report this to anyone reading this, I truly am. As the saying goes, don't kill the messenger.
eli_gottlieb 2 days ago [-]
There are two kinds of AI company logos: animals and assholes.
skzo 2 days ago [-]
I'm currently looking into https://greenpt.com/ for my company,it has european data residency,no training on data and green footprint for energy.
Only thing is they aren't clear on the SOC2 certification. Am I being naive?
vehemenz 3 days ago [-]
This is really a roundabout way of saying you have no plans to live, or perhaps even travel, to China.
TheArcane 2 days ago [-]
I live in the US right now, my surface of attack from the US govt & ad companies is infinitely larger even if I decide to move to China some time in the future
aftbit 3 days ago [-]
The biggest problem with this is that I can't run my company's own internal benchmarks against this API, because they will be trained into the next version of the model, making them worthless.
TheArcane 2 days ago [-]
> I'd rather my data go to China and directly to labs that release open weight models (DeepSeek is MIT licensed still); instead of closed labs and the NSA/CIA; who has far better ability to hurt me than China does.
Thank you. I've been shouting this very obvious thing from rooftops
I work in silicon valley and I feel everyone around me is suffering from China-BAD psychosis and dismiss Chinese releases
This number reports, basically, how good it is to live inside a country, while the comment that you are replying to is entirely a list of foreign policy complaints (regime changes, etc).
It's kinda like meeting someone on the street who's protesting about animal cruelty in slaughterhouses, slyly handing them a chart showing them the nutrition facts of beef, and then walking off without another word as though you've settled your case.
Like, I get the point you're trying to make, but I hope you're aware it's a non-sequitur in context?
albedoa 2 days ago [-]
> Like, I get the point you're trying to make, but I hope you're aware it's a non-sequitur in context?
Your hope should have been that amelius was not aware that it's a non-sequitur, because that is correctable. But then they kept posting and removed all doubt.
ashkankiani 3 days ago [-]
Why link to a number, which means roughly nothing, instead of the specific details on the countries you're trying to compare? I have no idea how to break down or make my own assessment of what 0.83 or 0.18 means. Does that 0.83 for the US include all of the people being shipped off without due process to El Salvador, or do they count that as part of El Salvador's Human Right Index?
People like to bring up the Uyghurs immediately, but never really have the same smoke for the US. Also, do we include the prison population in human rights? Do we include access to healthcare? What about systems perpetuating poverty like lack of access to public transportation?
To pre-emptively be clear on my stance or potential bias, I stand by no country in the world. I was born in Iran, and my parents left when I was young to the US because they believed in the American Dream. The irony being that the dominos of why they left were initiated by the US and UK. And long before Trump was elected, but certainly immediately afterwards, I decided to leave the US once I had the means. (Also because of the lack of public transit, the $100k medical bills after an accident, being punched by a homeless person, and having my possessions stolen repeatedly and the cops giving me a :shrug:). I moved to Japan, where I experienced blatant racism for the first time. I moved to Hong Kong (and tbh no real complaints so far), but I am technically subject to the freedom of speech constraints on China.
I have no allegiance to any country or specific people. My allegiance is to all living beings.
amelius 3 days ago [-]
Sorry but you're arguing against a reputable institution here.
ashkankiani 3 days ago [-]
I'm not arguing against a reputable institution, I'm arguing against using that reputation + their final number alone without going through the process to verify the methodology myself, which you didn't link to. You only showed the conclusion, which to all of us, should mean very little until we look into how that number was arrived at.
And I don't care if they're reputable, I will look at the methodology and make my own conclusion either way. Appeals to authority are not a substitute for critical thinking.
amelius 3 days ago [-]
We're not talking about a few % difference here. If you want to look into it then fine, but you have to come up with some really good evidence if you think you can debunk anything here.
lejalv 3 days ago [-]
We're talking about the country that has tried multiple times to destroy their country, look up Mossadegh for the 1950s and check out the glorious bombing of a school for 2026.
I know the US thinks only its citizens are human, so "The United States has been involved in hundreds of interventions in foreign countries throughout its history, engaging in nearly 500 military interventions between 1776 and 2026, with half of these operations occurring since 1950 and over 25% occurring in the post-Cold War period” is not probably part of the reputable storytelling
pessimizer 3 days ago [-]
There's absolutely nothing in those numbers to argue against. They are simply providing fluffy quantitative declarations in order for people who don't even claim any particular knowledge or even understanding of the details of the metric (such as yourself) to argue from authority.
If they actually said something, it would be much less effective.
> We're not talking about a few % difference here.
How may freedom percent is it again? Do you lose 1% or 20% for failing to allow foreign ownership of utilities? How many percent do you gain for stopping Russian misinformation?
amelius 3 days ago [-]
The metric was carefully crafted by a group of experts as means of communicating a complicated topic.
I'm not going to discuss it further if you don't have anything substantial to bring to the table.
aleqs 3 days ago [-]
'experts' lmao. Your entire argument boils down to an appeal to authority fallacy.
amelius 2 days ago [-]
That fallacy does not apply here as the authority is legitimate and relevant.
true_religion 1 days ago [-]
Instead of repeating that they are legitimate, you would've gotten further by saying Human Rights Watch has been around for X years, has reported competently on Y matters even when suppression was attempted by Z country. Their membership is made of A, B, C and this is why you should trust in their data collection as a source that we can use in casual discussion.
Right now, you're basically asking people to trust Human Rights Watch because you trust them. But if people don't trust you, why would they trust your word on what sources are legitimized?
switchbak 2 days ago [-]
That line of reasoning ... everyone in this thread is now dumber for having listened to it.
Seriously - appeal to authority is a fallacy, unless it's an appeal to an authority that I happen to agree with? It's hard to take you seriously.
kelnos 2 days ago [-]
That's just your opinion, not fact.
NamlchakKhandro 2 days ago [-]
I.e., an appeal to authority
aleqs 2 days ago [-]
Eric Cartman is that you?
tessier2501 3 days ago [-]
Yes! By all means, let us appeal to authority and use institutions' reputations to shut down any discussion, rather than thinking about what these figures actually mean, what criteria they're based on, or how well they reflect reality.
edit: wording
RobotToaster 3 days ago [-]
An institution mostly funded by the governments it happens to rank highly, and the open society foundation that's owned by George Soros, a man who's been obsessed with overthrowing China's government since the 80s. https://www.v-dem.net/about/funders/
j_maffe 3 days ago [-]
They're not arguing against the institution. They're arguing against the metric
.
amelius 3 days ago [-]
Yeah but the metric was invented by that reputable organization and given the enormous difference it's not worth any discussion unless they come up with something good in which case they should probably publish about it.
aleqs 3 days ago [-]
If you don't want to have a discussion don't post pointless metrics that you don't even understand.
lejalv 3 days ago [-]
They are arguing for you using your brain instead of tethering it to a number.
owebmaster 3 days ago [-]
No he's arguing against your arguments and winning.
gnaritas99 2 days ago [-]
[dead]
cowl 2 days ago [-]
have a look at everywhere at the human cost of what the orange man is doing depending on his morning coffee. much better to send data to china than to US right now.
yogthos 3 days ago [-]
oh please, the US economy literally runs on slavery
17% of prison assignments might be 3 assignments for all I know.
I found a number in your document:
"...1.2 million people incarcerated in state and federal prisons, over 65 percent work."
That's 780,000.
And there's 163 million working folks in the USA.
Prisoners power .4% of the economy.
yogthos 2 days ago [-]
The original context here was human rights in China, and what we see is that the US both has far higher prison population per capita, and that it is being as slave labor which is a significant factor in certain sectors of the economy.
Your whole argument about the percentage overall economy is nonsensical because most of the US economy is not actual productive economy. It's ephemeral stuff like service industry, entertainment, and software development. Obviously, you're not going to force prisoners to go on Joe Rogan podcasts. But when it comes actually producing things, 17% of the industry being done by slave labor is obviously very significant.
sejje 1 days ago [-]
The original context was this:
> oh please, the US economy literally runs on slavery
Also
> 17% of the industry being done by slave labor is obviously very significant.
That's not what the stat says. It says 17% of the prison labor did that job, not that 17% of the labor for that job was prisoners.
Here's the quote for ya:
"...agricultural work constitutes a
greater percentage of work assignments, such as
in Arkansas, where 17 percent of job assignments
for incarcerated people were in agricultural work in
2021."
But tell me again how I'm nonsensical.
yogthos 13 hours ago [-]
The amount of mental gymnastics you're doing to justify the amount of SLAVE LABOR in the US is truly incredible. And couple that with one of the highest rates of incarceration in the world, that really tells us all we need to know about the state of human rights.
thisisnotabi 3 days ago [-]
lol the US was a .60 in 1789 during slavery. this chart is useless.
amelius 3 days ago [-]
The reason the US scores a 0.6 in 1789 comes down to how composite data is aggregated and the global baseline of the 18th century.
Yes, people have put thought into this. This is why I trust institutions and not random posters on the internet.
aleqs 3 days ago [-]
The reason that US and its allies/friendlies always score high on these kinds of ratings while their enemies/adversaries always score low is because these institutions and 'experts' are directly funded by and beholden to these countries/interests.
Just look at Russia - it's score is highest in the 90s - the most corrupt and destitute time in its modern history, also happens to be when it was deemed somewhat friendly to US and friends.
twohaibei 3 days ago [-]
Is WHO a reputable institution? And was it reputable between 2019-2022?
tonytamps 3 days ago [-]
Yes
tbugrara 2 days ago [-]
If an institution told you to jump off a bridge...
peri-cl 3 days ago [-]
To add to this, Fireworks (and many competitors, e.g. enumerated here[0]) offer zero-data retention inference, to anyone. DeepSeek cloud does not. It's part of their terms that anything you send them is fair game for them to train models on.
(The badges next to companies indicate their privacy terms, i.e. green/yellow/red)
leftouterjoins 3 days ago [-]
It’s pretty clear despite what they want you to think Anthropic and OpenAI train on everyone’s data who has a subsidized plan. Only way you get ZDR is by paying API prices. Also note that it is zero retention, not zero training.
peri-cl 3 days ago [-]
> "Also note that it is zero retention, not zero training"
The "green" badges explicitly say the opposite:
> "Private: this provider does not train on prompts and does not retain prompt data."
I.e. for the example of Fireworks, this is the precise language they use to describe ZDR:
> "More technically: prompt and generation data exist only in volatile memory for the duration of the request. If prompt caching is active, some prompt data (and associated KV caches) can be stored in volatile memory for several minutes. In either case, prompt and generation data are not logged into any persistent storage."
How is it clear? I don't fully trust them either, but what indications are there?
rapind 2 days ago [-]
Spoiler, it's not clear by any means. There's no evidence to back up the claim, it's just their feelings... which is fine, but I definitely wouldn't call it clear.
olejorgenb 2 days ago [-]
My slight suspicion is that the that somehow train on derived data from the sessions. In a way that they are technically in the clear.
kenmacd 3 days ago [-]
I figure (hope) buying from DeepSeek directly benefits them the most so they can release even more improvements and open-weight models.
Plus only one country has threatened to annex mine and it wasn't China.
kennywinker 3 days ago [-]
Assuming canadian or dane, but maybe there are other places he’s threatened. There are canadian inference providers running deepseek, and definitely eu ones idk about ones based in denmark.
The economic sovereignty of any nation, but especially smaller ones like canada and denmark, is harmed by sending our money out of the country when there are options available in country.
kenmacd 3 days ago [-]
> The economic sovereignty of any nation, but especially smaller ones like canada and denmark, is harmed by sending our money out of the country when there are options available in country.
I respectfully disagree. I subscribe to the comparative advantage economic principle, so I'm fine with international trade.
kennywinker 3 days ago [-]
What I am saying isn’t inconsistent with comparative advantage. International trade is fine, but only when it’s not a commodity easily obtained inside your own nation. Open model inference is a commodity. There is no real lost opportunity cost from using canadian owned GPUs or chinese owned ones. In fact, as a country with abundant cheap power, and physical space, canada’s probably a better place than many to run inference. That’s what I meant by “when there are options in country”.
kenmacd 3 days ago [-]
I hear you, but as I said, I hope that buying from Deepseek gets more Deepseek. Maybe they get the same amount (or more) from a Canadian provider, and if so I might reconsider.
The 'clean power' would factor in a well, but I haven't really looked in to it. I know China is installing a ton of renewable and has a carbon market while Canada's building a new pipeline and building Alberta data centres running on natural gas, so I'm not willing to take on faith that switching is cleaner.
kennywinker 3 days ago [-]
Fair point. Personally i don’t see value in closed models, be they chinese or american. So, it’s pretty much against my religion to pay the model creator to provide inference. Even a situation where model makers license models to providers seems better than deepseek hosted deepseek provided models.
Also, re: power. I said canada has lots of cheap power, clean was your word.
WhrRTheBaboons 2 days ago [-]
RE: clean power - China is by far the biggest leader in getting renewables off the ground, but they are also still comfortably the biggest polluter, so it's difficult to compare.
The unfortunate truth behind at least the solar PV boom in China is that it's driven in large part by the exploitation of Uyghurs in the Xinjiang region. One of the reasons they can sustain the pace they're going at is because while nowadays PV modules exports increasingly gravitate towards offshore fabs (Vietnam, Malaysia), domestically all the biggest PV players make full use of modern slavery within Xinjiang's re-education camps.
>International trade is fine, but only when it’s not a commodity easily obtained inside your own nation
It might be a minor nitpick, but that part is just not how the macroeconomic principle works. Abundance of a good/service within a country is irrelevant to the concept of a comparative advantage - this is counter-intuitive but but that was David Ricardo's entire point. It's not about whether production is more efficient domestically, it's whether comparative allocation of resources between multiple countries is more efficient by specialization within the different goods.
Even if you can produce "good 1" cheaper in Country A vs Country B, if Country A can instead produce "good 2" that Country B cannot, it's better for Country A to produce "good 2" domestically and import "good 1".
Now, I still agree with the original notion. It's unnecessarily risky to use services from "democratically-challenged" nations (especially imperialist ones), when safer options are available. But there's no need to twist macro 101 concepts to justify that view. Modern trade policy takes more into account than basic market efficiency.
irrkfjjffj 3 days ago [-]
[flagged]
rapind 2 days ago [-]
> There are canadian inference providers running deepseek
As someone who's in the market for Canadian inference (data sovereignty reasons), the pickings are slim for DSF 4.1 man.
eli 3 days ago [-]
It surely does. Official DeepSeek provider logs and trains off your data.
subscribed 3 days ago [-]
China has annexed (or it claims so) several countries already, so it sounds like you're only concerned with yourself - probably not what you wanted to achieve.
(But I'd concede that in the field of genocide China doesn't come up as bad)
3 days ago [-]
specproc 3 days ago [-]
If one is not American, why prefer American? Chinese fine by me, I trust the US providers just as little.
pimeys 3 days ago [-]
There's also European providers for these models. More than one!
I feel like I’m getting good value out of my subscription and they have the cheapest GLM-5.3 API rates I could find in Europe. I moved to them recently from Mistral for the better prices.
They do have a slightly different daily inference capacity allowance, but it’s actually not too bad once used to it. I was forced to up my game and figure out what GLM-5.3 Flash and Deepseek-4.1 Flash could actually do for me, rather than just throwing everything at GLM-5.3. Quite happily chugging along with a nice balance across them now.
I am hoping Melious make the new DS 4.1 Flash 1017 available soonest.
They do not yet. They are testing it but they go with GLM right now.
jstrasner 17 hours ago [-]
Biased obviously, but sference.com. We deployed the new checkpoint immediately. Lmk if you have any thoughts, I can send some credits...
esperent 3 days ago [-]
Which ones? And how is the price?
abc42 3 days ago [-]
I support people making choices with full knowledge of what they are doing and what the consequences are, as much as it is possible. What I oppose is making choices without such knowledge.
browningstreet 2 days ago [-]
No Fireworks for me. They invited me to test Firepass, screwed up the onboarding, charged me money (when the account was supposed to be gratis) and when I asked them to fix that after I used up my $$, they said they closed the program. After that, no emails were responded to, and we were on a person-to-person basis. Not sure what their deal is, but I'm off Fireworks.
pimeys 3 days ago [-]
There are multiple others in EU, and in US. Cheaper, more expensive, slower, faster. So much choice.
itsalwaysgood 3 days ago [-]
Completely unrelated, but it's so odd to see upside down '8's in the first column of the base tables on their pricing page.
ilaksh 3 days ago [-]
The link points to OpenRouter with multiple providers.
igetspam 3 days ago [-]
Agreed.
It doesn’t have to be better, it just had to be nearly equivalent. I’ve been doing a lot of vulnerability work lately (trying to get ahead of the bots finding exploits in our code) and the frontier models feel like a chaperone at a church dance: get too close and they show up with the ruler and enforce the safety margin. Using deepseek and glm and qwen, I’ve gotten nearly equivalent results without the constraints. I still use frontier models for things but outside of a company environment, I’m happy to spend 1/10 the money, get the same results, and not have try and trick the AI into working.
proxyscore 2 days ago [-]
Yeah I've been doing this since even before you could run DS via claude code, it's damn near the same quality with plenty instances where claude had to eat a hat and a shoe and use DS corrections.
American folks really have a distorted sense of reality and few have their own mind, they listen to influencer noise and once self thinkers pick up the quality of things like DS, they will echo the statements.
> they frequently generate CoT explanations rationalizing those answers
I can't tell if this is a really human thing to do, or if it's just a sycophantic thing to do when your master likes a good story.
embedding-shape 3 days ago [-]
In practice, it does help to immediately be able to tell if you missed something or wrote something incorrect in your prompt, compared to if you didn't have access to the CoT. If you see that it goes off wondering about stuff or thinking about stuff that you know are already solved/known, you cancel the execution, edit the prompt and redo, then observe the CoT again, until you've resolved any "misdirections".
Both OpenAI and Anthropic hides so much of that, that you don't really know if it misunderstood something or you got something wrong, until all the turns are complete or you directly see tool calls that implicitly tells you something is wrong.
You don't need to read any papers to see this effect concretely when developing together with LLMs.
simianwords 3 days ago [-]
This is like one of those folk theories that have some truth but in practice it is useless.
I love COT and gives me a good idea about the thought process. Sure it doesn’t work always
alfiedotwtf 3 days ago [-]
Same. And the biggest benefit is that you get to see if it’s heading in the wrong detection
tclancy 3 days ago [-]
Fair, but you can see the logprobs, right? That alone is pretty valuable.
please don't subject yourself to this. z.ai subscriptions are not a good value. just pay API rates, or use a subscription like opencode go.
MaxMatti 3 days ago [-]
I've used the zai subscription, opencode go und cline-pass in the last few months and would recommend the latter. Zai quickly drains your quota as soon as the context window is at more than 10-20% and opencode was a bit unreliable at times.
resonious 2 days ago [-]
If you're bothered by random failures, you probably won't be very happy with a z.ai sub.
Also z.ai doesn't offer Deepseek models.
petu 3 days ago [-]
Is that a dig on DeepSeek not offering subscription plans?
If so OpenCode Go looks good and includes DeepSeek models (yet to try myself).
eckelhesten 3 days ago [-]
I wonder what quantization opencode uses for DS models.
petu 3 days ago [-]
From "What about data and privacy?" in FAQ it sounds like they don't host models themselves. e.g. it says:
DeepSeek: ZDR agreement is renewed monthly. The current agreement is valid through October 31, 2026.
https://opencode.ai/go
They also follow official DeepSeek API peak/off peak hour pricing and link to their docs:
DeepSeek V4.1 Flash / V4 Pro: Peak hours are 01:00-04:00 and 06:00-10:00 UTC, Monday through Friday; all other hours, including weekends, are Off-Peak. Learn more.
https://opencode.ai/v2/docs/console/go/#usage-limits
I think they just pass through requests to DeepSeek, so whatever upstream offers.
xyzzy_apps 2 days ago [-]
Absolutely!
DeepSeek feels like an excellent sparring partner.
The others feel like mind readers.
I don't have to worry about any complex context engineering / multi agent stuff if I am using DeepSeek because the sequential stuff is fast enough. I don't have to bother about touching a million context length because its cheap enough.
Where it seems to lag behind is 3d, computer use, motion graphics, design but the initial demos seem like the vision model has improved substantially.
34679 3 days ago [-]
I'm enjoying Alibaba's Token Plan. They don't have any hourly/daily cutoffs. You get your allotment for the month and use it when you want.
TheArcane 2 days ago [-]
Have you run into their concurrent agent limits?
neya 3 days ago [-]
I second this, I love DS and am not looking back. For Elixir, not a lot of models are good, but DS just blazes through it.
jacquesm 2 days ago [-]
Deepseek is not from Z.ai, that's GLM.
MangoCoffee 3 days ago [-]
Deepseek and Muse are my go to for cheap. both work perfect for basic and mid tier tasks
pimeys 3 days ago [-]
I mean... I ported some of my childhood Finnish shareware games for Linux and Rust with DeepSeek V4.1 Flash. It took about six hours of spelunking and finally I was pretty happy with the results! I did it twice. Tokens for both of the games were around 3 euros.
esafak 3 days ago [-]
Muse is fantastic for open source work. At least 1.3 was, for its price. Wonder makes less economic sense.
nextaccountic 3 days ago [-]
What about mimo v2.6 pro?
FacelessJim 3 days ago [-]
I’ve jumped between mimo and ds quite a bit lately. Mimo seems to be slightly better in terms of what it can achieve in terms of end results. But it gets there very slowly, and in a super convoluted way. Sometimes it’s almost frustrating to read its reasoning. Also I’ve noticed that it tends to not use tools in the harness as much as other models.
I usually default to ds, and if it gets stuck try mimo but usually just for fire and forget tasks. After a bit I come back and see what it did.
esafak 3 days ago [-]
Usually great but it infuriates me sometimes with what it does. DeepSeek is more consistent and certainly faster.
tyrabound 3 days ago [-]
If I were part of a cabal that is interested in preserving control and trying to secure trillions of dollars, it would be a minor effort to nudge and bend the curve against competitors like the Asian AI complex.
jfengel 3 days ago [-]
How would you go about that?
crossroadsguy 3 days ago [-]
> not being randomly blocked for cyber review
Just to test, will GLM – hosted by an infra provider other than Z.AI – also stop answering if I asked controversial questions from the pov of its country of origin? Or third party infra providers can perform some unhinging on the model before hosting? Or actually the blocking happens even before the question reaches the LLMs in that country?
(Note: my paid LLM access at this point are only via DS and GLM subs - 90-95% usage; and a free Gemini Pro access. I hope that adds some cred that I am not trying to take potshots at the their country of origin).
flexagoon 3 days ago [-]
Depends on the question. Most models won't tell you how to make weapons because they're trained not to, but the political stuff is usually blocked on the API level
OsrsNeedsf2P 3 days ago [-]
It will not. Ask DeepSeek about Tiananmen Square. It will happily answer
SSLy 3 days ago [-]
to add to the corrent sibling answer, the unhinging is called abliteration by the wider community but it's hard to find someone who will serve that over API.
charcircuit 3 days ago [-]
GLM refuses to write exploits. It's not open season with anything you want.
Toslink 1 days ago [-]
[dead]
TheArcane 2 days ago [-]
there we go again, the king of "China BAD" boogeyman
npn 3 days ago [-]
I do not use DS for coding, but for various natural language processing tasks (even translation) in large volume, and it works wonder.
It is very capable for a lot of tasks, not only coding like the other open models.
serf 3 days ago [-]
I like it because it does the thing I want it to do rather than moralizing.
No, I don't care about license provenance for all the components of a 30 line script that i'm using in my house, for myself to sync up some folders. No, i'm not going to find and archive the licenses for all the previous projects leading to this for future legal disclosure.
Yes, I own the DVD for the movie i'm transcoding. No I don't want to talk about the legality of rebroadcast.
Deepseek (and most chinese models as a whole) don't care . oAI and Anthropic models want to make it point #1.
... which is extra hilarious knowing the laws broken to create the damn models in the first place.
there's a literal meme about this floating around the harness/AI chats for the past few months[0]
I don't see this kind of behavior myself, I wonder if it's me not pushing the boundaries enough or me explicitly or implicitly framing the prompt in a way that makes it less likely for the agent to intervene.
scotty79 2 days ago [-]
> Being able to see the CoT and not being randomly blocked for cyber review.
I had a bad experience with MiMo from Xiaomi. I had it trigger unsafe content blocks while fixing visual bugs in a completely vanilla space game it just wrote from scratch in the same session. It also got triggered for changes in some other benign greenfield app.
onesandofgrain 3 days ago [-]
Isnt Z.ai for GLM? Does Z.ai provide Deepseek?
tuesdaynight 2 days ago [-]
I kind of get it. We don't need the best of the best for everything, specially if cost is a big factor. Maybe DS is not at the top of the food chain, but it works for most tasks, and that's enough for a lot of work
equasar 2 days ago [-]
Google's Argon model via antigravity is unusable, for a small code review I asked, already spent like 10% of the weekly usage, in less than 10 mins. Very useless imho.
I always find DS or Qwen to be very spot on on audit changes or plans for just a few cents.
sunaookami 3 days ago [-]
Huh, doesn't DeepSeek take the longest because it overthinks way too much? Haven't tested this release yet but was very disappointed with past ones because it takes an eternity to do anything because it constantly overthinks and questions itself way too much instead of just starting.
pimeys 3 days ago [-]
It thinks a bit more, but if the cache reads are free or almost free, and it blasts through 10x faster than GPT Sol/Terra/Luna, it doesn't really matter. Just don't use the max thinking for implementation, go with high or low.
jambutters 3 days ago [-]
Why not use max thinking for implementation? For cost reasons?
pimeys 3 days ago [-]
Because it doesn't make difference for my use. High is good enough! If you do an assistant for yourself, low is more than enough.
It reduces the speed and tends to overthink sometimes which is really annoying. For programming, 4.1/high is the bread and butter in my daily work.
AspireOne 3 days ago [-]
For time reasons.
k__ 3 days ago [-]
I think, as long as you don't set thinking to max and use a fitting system prompt, it's not a big issue.
davedx 3 days ago [-]
Yes, and this version explicitly addresses that
Leynos 3 days ago [-]
4.1 Flash on medium is a solid builder agent
sehw 3 days ago [-]
LLMs don't think.
QwenGlazer9000 2 days ago [-]
We lost this battle aleady. Try telling people to "search it" instead of "google it".
sehw 17 hours ago [-]
[dead]
killingtime74 2 days ago [-]
I think opencode go is the better value to get access to DeepSeek.
epolanski 3 days ago [-]
For 99% of the tasks any model really would do, even GPT 5.4 from an year ago.
It's only for long running/very difficult cases that the differences are here, and they are small.
chrsw 3 days ago [-]
I can think of plenty of tasks that even the closed frontier models fail miserably at. From my perspective we have a long way to go for these models to get better.
epolanski 3 days ago [-]
I'm sure they exist, but it's not the bulk of the work.
chrsw 3 days ago [-]
My prediction is that in in a few years, maybe more, the current models will be seen as so primitive that they're not worth mentioning outside of a historical context.
epolanski 3 days ago [-]
What a bold prediction /s
blahgeek 3 days ago [-]
> Deepseek... My next subscription will probably be Z.ai
Wait wut? You do realize that they are two different companies, right?
throwa356262 3 days ago [-]
Z.ai (well technically alibaba) used to support both GLM and DS models in their API and coding plans.
zxilly 3 days ago [-]
z.ai was not alibaba. z.ai got glm and alibaba got qwen
throwa356262 2 days ago [-]
You are correct, alibabacloud is the company offering their native models (qwen) plus a few others including DS.
ngl999 2 days ago [-]
zcode was not long ago revealed to upload your entire project folder to their server encrypted using a key you don't have. And yet you still want to subscribe?
orangeboats 2 days ago [-]
The subscription plan from Z.ai doesn't stop you from using harnesses other than Z Code though?
And I am not sure why you are so worked up over it -- even meta-commentary wise, you have been shitting on everyone in this thread and your comments are in general very inflammatory. Stop that.
LYK-love 2 days ago [-]
Wait... You praised DeepSeek and your future choice would be Z.ad?
LYK-love 2 days ago [-]
* Z.ai
quinndupont 3 days ago [-]
Agreed, but don’t subscribe to z.ai; their inference is terrible. Stick with openrouter
jrmg 3 days ago [-]
For us neophytes, what does this mean: “randomly blocked for cyber review”?
jakozaur 3 days ago [-]
E.g., your prompt for simple auth works (e.g., decode JWT), but your sessions stop because you are flagged for cybersecurity risk.
The Fable is the worst offender, and it is not purely Anthropic's fault.
esperent 3 days ago [-]
It's happened to me exactly twice in about 6 months of using codex (via pi). Both times were completely bogus and annoying but still, it's not too bad.
dannyw 3 days ago [-]
OpenAI's models have a lot less false positives for me. Almost always smooth doing vuln scans and fixing them.
Anthropic, whether Opus 5.5 or Fable, frequently will review for security, and the classifiers fire mid-way while it's fixing something. Authentication and credential related code, including implementation, also trigger safeguards.
Earlier today, I hit what appears to be a sticky "reasoning_extraction" block when attempting to reproduce a LLM paper at small scale, and I asked it ~"tell me more about the idea [X] you considered but dropped". I'm not sure how it works or how long it lasts, but Opus 5.5 has refused to do technical LLM work for me now, even in completely new sessions.
vikramkr 2 days ago [-]
That's specially one of the categories along with cybersecurity and bio stuff that they block
jcomp 2 days ago [-]
How is this not purely Anthropic's fault?
huahaiy 2 days ago [-]
It happens very frequently if you happen to work on anything low level, e.g. a C code base, a database code base, etc. that deals with memory, and these have nothing to do with cyber security. It is extremely annoying and makes these models unusable.
lmf4lol 3 days ago [-]
and really , for 90% of standard agentic work, deepseek flash is more than good enough.
its the main driver for our Personal Assistant offering and users love it.
effortprogramme 3 days ago [-]
I love using glm 5.3 more than deepseek because of performance
Oxodao 3 days ago [-]
also it's cheap, using it in opencode is great
abc42 3 days ago [-]
Note that opencode routes deepseek requests to China.
Flere-Imsaho 3 days ago [-]
I have to ask what's everyone's issue with the requests to China. We've had 25 years of American big tech basically capture all our data, and use that to make insane profits.
3 days ago [-]
abc42 3 days ago [-]
Sure. Can you tell me why that means that it's ok for Chinese big tech to capture all our data?
Turskarama 3 days ago [-]
The government that is most likely to use your data against you is the government of the country you live in. If a foreign government wanted to use your data against you the easiest way for them to do it would ironically be to send it to your government.
Barbing 3 days ago [-]
Makes sense in short term & for personal queries. A German or Norwegian model provider shouldn’t tattle on US Southerners seeking reproductive care.
Long-term & software engineering fronts: I hear occasional railing against “US hegemony”. Anyone concerned that if a new tech gives rise to a different country, they’ll become the source of ire instead? (This isn’t my field, at all.)
vehemenz 3 days ago [-]
[flagged]
jeremyjh 3 days ago [-]
When the White House told Anthropic to take Fable offline - they did it. They are all going to comply with FISA warrants that the public will never see.
regexorcist 3 days ago [-]
What? The US govt has in practice unlimited access to whatever that's hosted in the US, or even outside by a US company. It's called the CLOUD act.
o_m 2 days ago [-]
trump signed an executive order to sanction the International criminal court targeting the chief prosecutor. Microsoft went ahead and blocked him.
rolymath 3 days ago [-]
Of course they do. See Zuckerberg's episode on Joe Rogan.
allarm 3 days ago [-]
And that changes things how exactly? The problem is that American companies have been stealing our data for decades, not their ties to the government.
speed_spread 3 days ago [-]
Oh but they absolutely do. The difference was mainly that up to a few years ago, the American government (generally) didn't care to repress those with the "wrong" opinions. Whereas China was always quite adamant about it.
metalspot 2 days ago [-]
I spend a lot of time using deepseek, and by extension providing them with highly valuable training data, specifically because I want to have Chinese competitors to the US labs. As long as they continue to release the weights this is a very good deal and for the benefit of the entire world, including the US Government. The monopolization of AI by any one country or company is the most dangerous outcome possible and insuring that this technology is distributed as broadly as possible is the most effective method of preventing that. Since US companies operate in oligopolistic cartels and engage in many dubiously legal practices to suppress competition it is impossible to build real competitors in the US and Anglosphere, so the necessary competitive forces to create a functioning market can only exist at a multi-national level with countries like China and India providing the scale, resources, and geopolitical positioning to develop true competitors.
miroljub 3 days ago [-]
For an average European, why is it better than routing requests to the USA?
BOOSTERHIDROGEN 3 days ago [-]
Chime-in for an average non us citizen why is it better ?
swiftcoder 3 days ago [-]
The argument has never been that it would be better (a domestic EU provider would obviously be better than either from a data handling perspective). But its not obviously worse
abc42 3 days ago [-]
Valid question. I have no 100% factual answers and neither does anyone else here.
So I go with feeling, and that feeling says that China is worse. The feeling is partially supported by the fact that China is softly backing Russia in its attack against the west.
Trump's USA is obviously completely horrible, but he is also in large part a product of Russian influence, which again is part of the larger block in which China resides.
tancop 3 days ago [-]
"The west" is a stupid idea America uses to force themselves and Israel into the same league as EU countries.
And China is not really supporting Russia, they make money off the war and want to keep it going as long as possible so both Russia and Ukraine become weak and more open to their influence. Is it evil? Absolutely, but you can't compare them to USA when Trump is actively doing regime change operations, deporting its own citizens and putting sanctions on the ICC.
Chinese communists want to stay in power at all costs but America wants to expand its power into the whole world. It's not the same level of danger at all.
akho 3 days ago [-]
> want to keep it going as long as possible
note that the volume of EU and US support to Ukraine is obviously calibrated to the same goal.
jubilee33 3 days ago [-]
Yes I unfortunately trust those who are motivated by money more than ideology. Greed, while horrible and ugly, is at least very predictable.
Messianic, or even utopian or pretty much any ideology, can lead to absolutely insane levels of destruction. You just don't know what true believers of any sort are willing to do.
To be fair, east and west are in relation to England, not the US.
Trump’s a smelly orange shit but the US at least pretends to have data privacy protections and we can’t legally compel companies to have back doors… yet. I think. It’s very hard to keep up with all the nonsense these days.
swiftcoder 3 days ago [-]
> and we can’t legally compel companies to have back doors
In a post-Snowden world where the existence of PRISM is no longer a secret, I'm not sure how folks can keep stating this with a straight face
rixed 2 days ago [-]
I'm not sure how knowledgeable folks can keep enjoying their life without burying their head in the sand.
ruvie8812 2 days ago [-]
[dead]
Oxodao 3 days ago [-]
My code is opensource, china already has it. For my professional work we have entreprise plans with OpenAI / Anthropic
hgoel 3 days ago [-]
Opencode also has a setting to not route to China, though this would probably just disable the models that do not have providiers in alternative locations.
igetspam 3 days ago [-]
If you pay for a business plan on openrouter (I know, it’s not opencode, just another option), you can geo lock providers if China is a concern.
altcognito 3 days ago [-]
Doesn’t that depend on the router service provider?
yuhond 2 days ago [-]
[flagged]
_s_a_m_ 3 days ago [-]
still a myracle to me what people achieve with it? I tried so many things with it, everything is broken it produces
jmward01 3 days ago [-]
This model is why OpenAI and Anthropic want regulatory hurdles. We need more diversity like this, not less. A diverse ecosystem is the only long term solution to complex problems that I know of and the challenges that the AI world is presenting is no different.
guantanamo_bob 3 days ago [-]
Their strategy is regulatory capture.
mmaunder 3 days ago [-]
Through fear. Specifically making the American public so scared of AI that they hand over regulatory keys to the only “experts” that can keep them safe. This is a centuries old political strategy used by some of the worst regimes including apartheid South Africa where I grew up. It’s horrifically effective.
pyronite 3 days ago [-]
I disagree that they’ve invented some “strategy” for this.
1. They, and many other experts, have been saying the same thing since well before the current paradigm existed.
2. There is nothing about regulation that is required to give them an advantage; it’s a fallacy to suggest “regulation = advantage”
3. It would be preferable to argue the facts (is it or is it not dangerous? Why or why not?) than conspire as to why they’re stating what they and plenty of others have been saying for decades
gelly321 2 days ago [-]
What paradigm?
You're saying regulatory capture gives you no benefit by just refuting an absolute with 0 context. There's nothing to even debate here. Ok... most people disagree?
Yeah I agree with point 3... Stop engaging in vageries and say something of substance with examples, so people can actually have a conversation with you without us doing all the legwork.
jchw 2 days ago [-]
They certainly didn't invent this strategy, but that's because it already existed. Regulatory capture is nothing new.
> 1. They, and many other experts, have been saying the same thing since well before the current paradigm existed.
Some experts sure. "Some experts" say lots of things.
However, this is actually much to their advantage. If you were a frontier AI company, a legitimate concern to use as justification for your regulatory capture is exactly what you want. That is, after all, generally how regulatory capture gains public approval.
AI safety was also why OpenAI had to charge per-token for GPT-3. It must've been such a struggle for them to make that difficult decision. I'm sure Sam Altman spent a lot of nights crying to sleep over that one.
It's good that we never got open weights for a GPT-3 class model, right?
> 2. There is nothing about regulation that is required to give them an advantage; it’s a fallacy to suggest “regulation = advantage”
It's difficult to outline the number of different ways that regulatory capture can and will benefit OpenAI and Anthropic.
If we limit things to "Fable"-class models, then nobody competing with Anthropic/OpenAI is ever allowed to outdo them. If you literally legally aren't allowed to make better models, you can never charge the "premium" of having the current top-tier model. You have to compete by being cheaper than Anthropic instead, which is a lot less enticing from a standpoint of trying to become profitable. Take a look at the per-token pricing of Fable real quick. Yeah. Being able to charge a premium for the model that does the cybersecurity thing is literally their most important asset.
If Anthropic really is making a profit now as rumored, then they need regulatory capture as fast as they can possibly get it. Once Anthropic and OpenAI prove that AI is a viable investment, entities with a lot of capital will suddenly become even more interested in re-allocating it towards taking a slice of this pie.
> 3. It would be preferable to argue the facts (is it or is it not dangerous? Why or why not?) than conspire as to why they’re stating what they and plenty of others have been saying for decades
OK. On the factual end of things, AI safety experts have been raising the alarms for over a decade and so far essentially none of their predictions have come true.
The rationalists, the main group pushing the AI safety rhetoric that dominates the Western AI sphere and is virtually absent in China, which appear to be going entirely off the deep end at this point in time, put together their best minds to produce AI 2027, which of course like most predictions started to fall apart a bit before hitting the one year mark. Predicting the future is hard.
I have made this point before but I will make it again: I think AI safety experts deserve at least as much skepticism as Ed Zitron in their claims. They've been making similar dubious claims about AI capabilities and an incoming singularity for a long time now, and so far the best we can say is that maybe it will eventually come true.
Maybe. Or maybe you've read too much goddamn science fiction.
redanddead 2 days ago [-]
They’re for sure losing this.
They seriously underestimate how ready and willing Americans are to use Chinese providers. There’s a ton of spite
Last time China pulled this shit it shook portfolios for a week
Turn_Trout 2 days ago [-]
> Some experts sure. "Some experts" say lots of things.
I get this is Hacker News so we have to hate AI safety, but you should read more on this topic rather than blithely casting the x-risk viewpoint as fringe. The last hard data we have on the question is a late-2024 AI Impacts survey [1] of 1,580 authors at top AI conferences, who were offered $50 for answering a survey on the topic of the future of AI progress. The median respondent assigned a 5–10% chance (!) to "extremely bad (e.g. human extinction)" outcomes.
> On the factual end of things, AI safety experts have been raising the alarms for over a decade and so far essentially none of their predictions have come true.
Devastating. When I wrote my AI alignment PhD thesis [2] in 2022, and predicted a 70% chance existential risk by 2072 (which I later came down to about 1/3 on), you're right that that prediction hasn't come true yet.
While I agree many specific worries didn't materialize (like AI being like a "literal genie" it's hard to specify goals to), many key worries have (Hugging Face rogue AI swarm scans as cribbed from rationalist AI scenario writing).
I suppose, for you, an extinction question that didn't mention LLMs is evidence that Anthropic should be given regulatory capture of LLM development!
jchw 1 days ago [-]
You have to realize by now that outside of academia nobody is even paying any attention to people who are predicting existential risk that far out. For worse or even worse, AI 2027 is the AI safety conversation we're having today, and I regard AI 2027 as essentially Anthropic fanfiction, except usually fanfiction is written better.
ozozozd 2 days ago [-]
You are right that they didn’t invent it. It’s a pretty well-established method for cornering a market.
1. All those who have been arguing this - many of them non-technical - are part of the same ideological movement.
2. Technically, there isn’t. In practice, majority of the time regulations benefitted those who wrote them.
3. Not sure if this is a criticism of the comment you are replying to or of the p-doom crowd. We would all appreciate facts stated instead of science fiction tropes repeated.
Look, “a lot of people say this” is a Trumpian fallacy. A lot of people say a lot of crazy things. It says nothing about the factuality about what they say. How many are this lot? Who are they? What do they know? Why are they incapable of expressing it without so much emotion? How come a big portion are non-technical?
yewenjie 2 days ago [-]
Hinton and Bengio, and previously let's say Stephen Hawking are not part of the same ideological movement.
h8hawk 2 days ago [-]
I'm pretty sure Hinton and Bengio get a lot of news coverage and are heavily over-credited for saying these things, unlike LeCun, Ng, Sutton, or Schmidhuber.
In fact, their core ideology actually stemmed from Yudkowsky's arguments.
seaucre 3 days ago [-]
I keep seeing this assertion. What regulations have they proposed that would have prevented this new model? I thought they want to pace the frontier, which this model is not.
0xbadcafebee 2 days ago [-]
Dario has not (and will not) say he wants to ban Chinese models outright. But:
Do the math: 1 + 2 + 3 = he wants to regulate/ban the Chinese open weights.
Of course, Anthropic and OpenAI will then write posts about how their own models are dangerous - but that's okay, they're the good guys.
BosunoB 2 days ago [-]
Honestly, open weight models are dangerous, and I haven't seen a single convincing argument against that.
jacquesm 2 days ago [-]
I'd argue that OpenAI and Anthropic are far more dangerous and I also haven't seen a single convincing argument against that.
overfeed 1 days ago [-]
> I haven't seen a single convincing argument against that.
When HuggingFace was in the midst of defending itself from OpenAI's automated hacking spree, they were blocked from using frontier models by Anthropic and OpenAI for "safety" reasons. HF had to resort to an open model to defend itself - and DeepSeek distinguished itself in that role: the second fact from that episode OpenAI would have you soon forget.
gr_norm 2 days ago [-]
Not being able to audit my own software and configurations for vulnerabilities is dangerous. Consolidating power over the majority of Americans' jobs is insanely dangerous. Need I continue?
There are no choices here which do not carry risk. I haven't seen a single convincing argument against these points.
0xbadcafebee 2 days ago [-]
Other things that are dangerous: kitchen knives, cars, household chemicals, not tying your shoelaces, political parties, water. Danger is a normal part of life. Rather than regulate or ban anything dangerous, what we normally do is assess specific risks, and only put a limited amount of regulation in place where it's absolutely needed to prevent the most harm.
We also put responsibility on humans to act responsibly. In theory, it's absolutely insane to allow human beings the ability to walk into a street with automobiles, because obviously they will be killed! But we don't ban them from walking there, and we don't ban cars, because it's pretty uncommon for people to walk directly in front of a moving car, despite there being obvious danger.
Ultimately the biggest risk of AI isn't the AI. It's what human beings do using the AI. Just like with hammers, knives, drills, chemicals, cars... we regulate the human's actions and create laws to direct their use, rather than regulate the items themselves.
Implicated 2 days ago [-]
Knives are dangerous.
kennywinker 2 days ago [-]
You are taking their words literally, rather than looking at the side effects of any effort to "pace the frontier".
Go read Dario's screed on GLM, or his other one one on how your toaster could be part of a distributed open model that self replicates. Every second blog post is some doom scare about the dire consequences of open models. Go look at the new ToU [0] which are now pages and pages of things you can't do with Anthropic models. Check how they will ban your account if you are persistently mean to Claude now.
He's completely transparently campaigning for regulation. It must be very frustrating to him that the Trump admin is completely non-responsive. He probably thought Anthropic would have the whole market stitched up and water tight by now.
Literally zero, but that never stopped a good conspiracy theory, especially when half of HN hates the big labs because of fear for their own economic future.
OpenAI and Anthropic aren’t even slightly worried about DeepSeek.
mszcz 2 days ago [-]
Having looked at the sentiment around DeepSeek's models around this thread as well as my own experiences with it I'm going to need a little more to back up that assertion about OpenAI and Anthropic not being at least a little bit worried about DeepSeek...
senordevnyc 2 days ago [-]
I’ve been listening to this rhetoric about the open models eating O/A’s lunch for like 18 months, meanwhile their revenue is up what, like 50x?
May this kind of worry find me…
coliveira 2 days ago [-]
The reason is quite simple, American corporations know they cannot depend on models developed in China; not because they're bad or lower quality models, but because they know their government can ban these models any time they want. It's an implicit assumption that every company knows to be true.
pimeys 2 days ago [-]
It's quite different here in EU. I just gave a presentation in an AI conference in Paris about moving from Anthropic to open-weight models, how it looks like and what kind of savings you'll be looking at. The room was packed, and after discussing with a bunch of VPs, staff engineers, and CTOs in the conference, every single one of them are either looking at or using DeepSeek now.
It's quite hard to have the valuation of Anthropic if they are limited to US.
senordevnyc 2 days ago [-]
This doesn’t make any sense either, the use of these models is on the rise throughout corporate America, it’s just that they’re augmenting the frontier models, not replacing them.
coliveira 1 days ago [-]
You said it correctly: augmenting, not as a main solution, because of what I said above.
kouteiheika 2 days ago [-]
> I keep seeing this assertion. What regulations have they proposed that would have prevented this new model?
> Literally zero, but that never stopped a good conspiracy theory
Are you living under a rock and never read any of Dario's blog posts?
Section "1. *Regulation* and public safety" (emphasis mine), then under it:
> AI companies that develop advanced AI models must have strong security standards that protect their model weights
You can't "protect model weights" if the weights are openly available, ergo this is an explicit call for regulation to make open weights illegal.
senordevnyc 2 days ago [-]
You're the one extrapolating that cherry-picked sentence to mean "open weights should be illegal", but that extrapolation doesn't make any sense, and it's hardly "explicit".
Presumably you'd want the AI companies to protect their weights to keep them from falling into the hands of bad actors. But the US can't make open weights illegal anywhere except the US, which does very little to keep them out of the hands of bad actors (most of whom are outside of the US).
People like open models because they are free/cheap. It's not anything deeper than that. Most people will hop in whatever van is handing out free candy.
redanddead 2 days ago [-]
It’s about capability.
The model makers are bricking the models with their system prompts, and even the tools are tamper proof.
You can only use it their way. They think we’re stupid.
And their way is the shit way, the babyproofed way, while they talk to you about their unfucked models hacking out of their labs and hacking into other labs, and reminding you you have zero rights.
Fuck them twice, then once more… This is about compute. Who owns compute. China is saying fuck you all we can scale things bigger than you can imagine. And they can! They have so much more energy! Theyre closer to commercial fusion than us too. Meanwhile our guys are trying to… regulate? Lol
senordevnyc 2 days ago [-]
China won’t have more compute than the US for least five years. They’re highly constrained.
mlsu 2 days ago [-]
When they surpass us, they will surpass us hard, in a way that is nearly impossible for us to compete with them.
Training and serving LLM's is massively parallel. Your bottleneck is either in energy or chips. China will catch up on chips fully decades before we catch up on energy. Arguably, this is already happening: here in the USA, we have lots of chips, but we are seeing our electricity bills skyrocket as more datacenters come online. We are powering those datacenters with natural gas turbines, not solar, wind and nuclear. Right now, to purchase one of those, you are on a 5-year waitlist. There's your timeline!
China is invested, from a regulatory, cultural, workforce, strategic, and technological standpoint, in an energy abundant future. We are not invested -- and in fact, actively sabotaging ourselves -- on all of those fronts.
Once they acquire chips, we lose, basically permanently. Look at what Huawei is doing recently and what is on their roadmap to understand how fast this catching-up is taking place. I'm sure that David Sacks would say that our innovation motor is pulling their copycat caboose along, as our monopolist tech companies steam ahead towards the frontier. I am not so sure. They are already ahead of us in many technologies. Speaking of technologies, they're building natural gas turbines in house, starting in 2025.
WarmWash 2 days ago [-]
This reads like someone who's worldview comes from social media comment sections. China is still an emerging economy, don't confuse emergence with expansion.
redanddead 12 hours ago [-]
Let’s see what happens
senordevnyc 2 days ago [-]
I agree that in the long run, China will likely surpass us in AI capacity. Our refusal to build out renewable capacity is a ridiculous self own in the long run.
redanddead 2 days ago [-]
Five years is a long time
PepegaRoach 2 days ago [-]
[dead]
equake 2 days ago [-]
DeepSeek is simply amazing.
3 days ago [-]
nerdyadventurer 2 days ago [-]
This can be a reason too. There is an another reason, they want some govt involvement because good relationship with govt, so govt will help them with public money, when the bubble bursts.
They are showing this new version scoring significantly higher across multiple coding benchmarks, beating Sol 5.6 (6.1 is the latest) across 10/11 benchmarks.
I am going back to DeepSeek for writing code. I was trying out using Sol and liking the result, but this should close a lot of that gap. I use Opus/Sol for an implementation plan and code review also use them for grilling. DeepSeek is also running verifications.
dang 3 days ago [-]
We'll put that link in the toptext too. Thanks!
AmazingTurtle 3 days ago [-]
so i did some apples to bananas comparison.
i let 4 different agents review the same dirty code base (200+ files changed, c++, game engine):
- deepseek v4.1 flash via deepseek API: took 20m to find one p1 issue that was relevant and a p2 issue that was irrelevant
- glm 5.3 flash NVFP4 via TP2 DGX Sparks: took 1h 5m to find a p3 documentation issue
- gpt 5.6 sol: took 5m30s for one p1 and one p2 issue. both relevant
- gpt 6.1 sol: took 4m30s for two p1 and one p2 issue. both relevant.
Worth to mention: none of those findings overlapped.
I let deepseek v4.1 flash and gpt 6.1 sol both plan fixes for the summary of the findings. in this particular scenario: gpt 6.1 sol planned to increase a buffer size from 128 to 256 to fit the 191 elements that blew past a test scenario. it included some headroom. deepseek v4.1 flash on the other hand decided to make it exactly 191 to satisfy the test, for example.
The TPS decode on deepseek is impressive. I never had a second / feeling of being left aone and though "damn this thing is working". Despite all my efforts, it took ~4x longer in a review, found less and planned worse than GPT 6.1 Sol. Wall clock time. I don't have token counts for comparison. If I were to grade those models from 1-10 (10 = top tier GG), I'd give deepseek a solid 6.5 and 6.1 sol a 7.5-8.0 maybe.
vgalin 3 days ago [-]
LLMs are non-deterministic, you could run the same prompt on the same code base with the same models, and have different result. I don't think only one run with each model is enough to draw conclusions.
not_a_bot_4sho 3 days ago [-]
We have to keep remembering this for some reason.
New model comes out and someone says, "it can (or can't) draw ducks on bikes" or some esoteric thing. But it's not enough statistical power to claim anything other than a personal anecdote.
I got a flat tire in downtown today. Therefore, driving in downtown causes flat tires.
Eggpants 3 days ago [-]
Honest question about the non-deterministic nature of llms. Every programming language provides some way to control the random seed. Is this not true in GPU programming land?
teraflop 2 days ago [-]
Yes, but for one thing, floating-point math is non-associative so even if you use the exact same random seed, you can get different results on different implementation kernels.
For another, the dependence on a random seed is just one example of how you can get different behavior. You could just as well get different results due to e.g. a tiny difference in prompt wording or punctuation or whitespace.
If a model passes some test on 70% of random seeds, and fails on the other 30%, then knowing the rough probability of success is much more useful than knowing one random sample of the outcome.
And empirically this kind of probabilistic success is quite common, because many LLM benchmarks report "pass@k" success rates that are much higher than the "pass@1" success rate on a single attempt.
Eggpants 2 days ago [-]
I don’t understand what “floating-point math is non-associative” has to do with the random selection of possible result token set after the soft max operation. The weights of the model are set and don’t change during the many matrix operations(which is deterministic regardless of the floating point encoding scheme).
ziiinq 2 days ago [-]
The reason GPUs are used for NN computation is that they are massively parallel. You have hundreds or thousands of cores that perform the same operation on different parts of the network.
Tiny variations in the computational environment, such as slightly different set of processes running on the system as well as tiny differences in core temperature or voltage - among many other factors, can cause computations to complete in non deterministic order relative to each other.
This meaning one run eventually calculates a+b+c, whereas another does a+c+b, yielding slightly different results. Those cascade down the chain, resulting, finally, in a different set of produced logits.
Notably, similar effects can and do affect pure CPU calculations on multi core machines, and as a matter of fact even on a single core depending on the architecture when considering eg pipelining, branch prediction etc.
Thus, real world matrix operations are not, in fact, deterministic.
Now, one could argue that all of these are negligible, but that is an empirical claim dependent on the problem at hand.
fspeech 2 days ago [-]
It depends on the execution order. Model computations can be scheduled differently depending on what else is going on. You need extra effort to guarantee the same order of execution. For example if addition saturates at 8, then 4+5-2 is non-associative.
roywiggins 2 days ago [-]
It doesn't really matter for most purposes. Determinism doesn't make them any less chaotic, it's not like fixing the seed makes the output any more predictable. Some percentage of seeds will behave in ways you don't like for a particular input, and you can't know which beforehand.
conception 2 days ago [-]
On some models you can control the temperature so it responds the same every time to a specific prompt but the “randomness” of llms is usually a feature not a bug.
qeternity 2 days ago [-]
> LLMs are non-deterministic
This is not true. Non determinism only arises via sampling, and to a much lesser extent, CUDA kernel launch subtleties.
bel8 3 days ago [-]
you probably tried with DS on max and I don't blame you.
But try it on high instead of max if possible. max is known for being overly verbose to the point of being detrimental compared to high at times.
For example Simmons svg pelican took 11 minutes on max and 1 minute on high. And they are comparable.
user43928 2 days ago [-]
With DS Flash it doesn't fall apart below xhigh like GPT Luna models do?
bel8 2 days ago [-]
For DS, not between xhigh and high.
I recall the difference being minimal in 4.0.
verdverm 3 days ago [-]
> Worth to mention: none of those findings overlapped.
This is why using multiple model families is important. Don't limit yourself by being a stan for this or that company.
david-gpu 3 days ago [-]
That may be the case, but simply running the same code review twice with the same model may give very different results as well. Impossible to know with the information we have.
verdverm 2 days ago [-]
yeah, there is definitely still a lot of vibes to all this still, I'm working on evals next to learn and leverage, to reduce the vibes component to the claims
today we can use some anecdata based on our observations, this also helps shape the dimensions we want to eval the agents-tasks on
dannyw 3 days ago [-]
Especially when $10-$20 on OpenRouter with e.g. DeepSeek (or any other inference provider, or their official API) will get you very, very far.
runtime_terror 2 days ago [-]
Gets me about a day or two lol
mosselman 3 days ago [-]
In your case, is it a public repository that the llms might have found the issues for online?
I too am thinking about creating some benchmarks that I can understand myself. The LLM 'cheating' by finding known issues through some sort of history or online source is a path you want to protect against. Then again, you DO want them to be able to look online for people having similar issues.
qup 3 days ago [-]
How do you get dirty changes into a public repo online?
mosselman 16 hours ago [-]
I meant things like bugs. So lets say some open source project has a bug that is later found, you could theoretically use that in a benchmark for an LLM to see if it can find it.
asabla 2 days ago [-]
Just me being curious about the results.
Were you using the same setup to call these models? was it pi/opencode or was it codex/claude etc etc.
Because I've started to see larger and larger drifts in the results depending on the agent it self rather then the model.
esafak 3 days ago [-]
Flagship gpt models are crazy efficient. 6.1 Sol is like a savant that looks at a problem and spits out the answer seemingly without thinking.
aleqs 3 days ago [-]
Opposite of my experience - openai models, and codex are clearly the slowest (out of 5 different models/harnesses I use on a regular basis).
jdthedisciple 3 days ago [-]
Was that the old DS 4.1 Flash or 4.1 Flash 1017?
AmazingTurtle 3 days ago [-]
I tried it out today, was the new one: 1017.
ilaksh 3 days ago [-]
What reasoning effort?
ProphetOfParado 3 days ago [-]
What is p1, p2?
This read like a conversation you were having in your head -- unfiltered.
taylorfinley 3 days ago [-]
Very commonly used acronyms in business/SWE contexts.
The lower the number the higher the priority, e.g. if production is broken someone might say "This is p -1" (higher than your highest priority).
P0 is "do this immediately", P1 is very important, P2/P3 are like "nice to haves." P100 would mean something like "we will never do this"
lelanthran 2 days ago [-]
Same understanding as you.
At 3am,
P0 = get people out of bed and fix this now.
P1 = we can fix this in the morning.
P2 = we can fix this before CoB today.
P3 = push it into the current sprint.
P4 = put it in backlog.
snqb 3 days ago [-]
priority 1,2 etc, more like severity of the cases
icantevenhold 3 days ago [-]
It’s a very common abbreviation; i had no issue understanding GPs post tbh
hartator 3 days ago [-]
> Worth to mention: none of those findings overlapped.
Yet to be obsolete.
vatsachak 3 days ago [-]
If none of the findings overlapped then LLMs are either
- For some reason choosing only a subset of issues.
- Different LLMs actually learn different things.
I suspect the former since they do this for me and it's extremely annoying sometimes
qup 3 days ago [-]
3 - non deterministic, lazy
dsco 2 days ago [-]
[flagged]
pmxi 3 days ago [-]
Hilarious sentence: "DeepSeek-V4.1-Flash-1017 is a minor update to DeepSeek-V4.1-Flash, delivering substantial improvements across reasoning, coding, agentic tasks, and visual benchmarks." (emphasis mine)
freakynit 3 days ago [-]
Kinda shows their team is more engineers than marketers.
boutell 3 days ago [-]
But with that version number, it's not a "minor" update! It's a patchlevel update! -Engineer
embedding-shape 2 days ago [-]
An old-school (pre-LLMs?) engineer would recognize that DeepSeek is in no way doing semver, so your major/minor/patch glossary might not apply here at all :)
stavros 3 days ago [-]
Strictly speaking, it just shows they aren't marketers. Bad copy doesn't tell us anything about their engineering skill.
Derelicte 3 days ago [-]
It's not a bad copy, or at least that's not what the person you responded to is insinuating.
Deepseep-v4.1-Flash-1017 is a minor update. A major update is something like Deepseek-v5-Flash.
lcnPylGDnU4H9OF 3 days ago [-]
> It's not a bad copy, or at least that's not what the person you responded to is insinuating.
FWIW, GP didn't write "a bad copy" but "bad copy", referring to the marketing sentence quoted by OP, rather than the model that was just released (as I presume "a bad copy" sounds, maybe like it's called that because it's [again, presumably] distilled?). "Copy", especially in a marketing context, is a written message.
qup 3 days ago [-]
Strictly speaking, it's the substantial improvements that tell us about their engineering skill.
nchmy 3 days ago [-]
Strictly speaking, they didn't say anything about their engineering skill
3 days ago [-]
Rodmine 2 days ago [-]
Well, it's a bigger improvements in Deepseek 4.1 Flash and Deepseek 4.1 Flash 1017 than whatever Google was doing between Gemini 3.1 and Gemini 3.8.
johnnyApplePRNG 2 days ago [-]
[dead]
barrell 3 days ago [-]
To me this implies they continued the training run for 4.1 and it improved. A 4.2 would imply they changed something about how it’s trained/how inference works, and a 5.0 would be a major update (who knows what would have changed).
This is just my headcanon though.
jaynetics 3 days ago [-]
It's not really contradictory, minor changes can have major effects.
mdp2021 3 days ago [-]
So they meant a "small" update - "conservative", "inexpensive"...
An example when using a less frequent term actually reduces specificity.
NekkoDroid 3 days ago [-]
Changing TREE(2) to TREE(3) is a small change on the input, but a... lets just call it big... change in the output.
halJordan 3 days ago [-]
No, they used minor correctly.
mdp2021 2 days ago [-]
Will you also explain, or leave it like that?
nkmnz 3 days ago [-]
That's one small step for a man, one giant leap for mankind.
cardboard9926 3 days ago [-]
Hoping it's minor relative to what's coming
surgical_fire 3 days ago [-]
It's a bit refreshing to read updates from an AI lab without bullshit doomerism and overwhelming sense of self-importance from shit like OpenAI and Claude.
Also, DS is an excellent model.
LPisGood 3 days ago [-]
Minor updates can provide substantial improvements sometimes, it really depends on the specifics.
varispeed 3 days ago [-]
Remember when we were dumbfounded why our service is slow despite having implemented quite robust caching mechanism. Turns out simply the flag that enables it was hardcoded to false. So it was minor update (changed flag to true), but provided substantial speed up.
ReptileMan 3 days ago [-]
That is how exponential functions work. Improve every aspect of a system with 1% and you may get easily 50% more performance.
fivedogit 3 days ago [-]
I'm at 1 trying to get to 10. I move to 2. Minor update. 100% improvement towards goal.
These are not mutually exclusive.
ilaksh 3 days ago [-]
They changed how it works substantially. I think it should be 4.2.
2 days ago [-]
skeledrew 3 days ago [-]
Quiet confidence backed by provenance :).
vrganj 3 days ago [-]
It's about playing to two different audiences.
The CCP requires extensive certification for new major releases, to make sure a model doesn't do anything they don't want it to.
To get around this, Chinese labs only bump versions when they have to and usually claim it's a minor update.
Then on the other side, they're obviously trying to market to the tech crowd...
epolanski 3 days ago [-]
Source?
est 3 days ago [-]
Everything in China is strictly regulated but loosely enforced. Just take that as a grain of salt
LYK-love 2 days ago [-]
I happen to live in China now and lots of my friends work in frontier labs. I don't now there's a restriction for major updates of LLMs.
Western media doesn't have correct informtion source about China. This is because Western favors Web ecosystem and most inforamtion live in the Web, but in China everyone use Apps.
vrganj 3 days ago [-]
Drum Tower podcast (from the Economist, paywalled) about Chinese AI.
Would highly recommend it, it's IMO the only serious Western reporting about China.
tw1984 3 days ago [-]
> the only serious Western reporting about China.
as good as Gordon Chang.
vrganj 3 days ago [-]
I don't know who that is, but from a quick Google he doesn't exactly seem trustworthy?
tw1984 2 days ago [-]
then what makes you to believe highly politicalised publications like The Economist is "trustworthy"? please just feel free to google or AI search all those failed "crash" prediction made by The Economist on Chinese economy. Don't get too shocked to see there are dozens such predictions made by The Economist in the past 3 decades - it is beyond laughable. You get far better prediction results using any random number generator.
The problem here is quite often brainwashed people never realize that they have been brainwashed. Sometimes they trust, promote or even defend the perpetrators. You have my deep sympathy on this matter.
vrganj 2 days ago [-]
I am opposed to their political tilt altogether (I'm a socialist and there's literally a Lenin diss of the Economist!), and I disagree with their conclusions. That doesn't make the reporting itself useless, and they are the only ones going past a cliche caricature of China, even if they dress it in classic capitalist delusion.
There is no such thing as an unpoliticized publication, but one can still read, taking into account their biases. The point is to extract the information while discarding the ideological interpretation. In fact, this exercise is often easier when the ideological tilt is as obvious as the capitalist liberals from the Economist.
Forget about their predictions or their moralizing, but they have actual Chinese reporters talking to actual Chinese people on the ground instead of doing the usual Western thing of white MBAs treating China like some alien civilizational threat.
excannuk 3 days ago [-]
[dead]
TheArcane 2 days ago [-]
I have both platform.deepseek.com (hooray off-peak pricing) and the $100/m Claude subscription and boy do I find myself mostly using DS-flash pretty much every time.
There's something to be said for the speed with which it gets word done - I take fewer "it's compiling" breaks, and lose my train of thought less frequently. What's crazy is that the speedy work it does is pretty much always correct - which is a reason why I don't feel the need to spin up claude-code and "ask the expert".
This speed & good-work combo is the DS signature IMO
markasoftware 2 days ago [-]
Fwiw I find gpt Sol on medium or low effort to also be faster than opus to the point of helping me not check out, but is quite smarter than dsv4 flash
computerex 2 days ago [-]
OpenAI subscriptions are shit tier. Deepseek does 90% of what I need to do for pennies. There is no comparison. And I’m talking about working on relatively hard domain like gpu kernel optimisation and HPC
If you want your 5h usage to runout in one call feel free to use OpenAI
UberFly 2 days ago [-]
You read like a shill for Deepseek interests, and that includes your comment history.
pimeys 2 days ago [-]
Or just us Europeans who are so sick of American corporations milking every penny with their LLM models and preventing us from using the full model capabilities.
Now we have: a ton of competition in different inference providers, very low costs, and speeds reaching 1k tok/s in certain providers.
We can just get a box of compute even in Germany, and run the exact same model here without anybody stopping us.
Are we using the same
Thing? Sol 6.1 has run <30 t/s for me for weeks.
criley2 2 days ago [-]
What would you prefer, 30t/s 50k output tokens needed for solution, or 300t/s, 500k output tokens needed for solution?
This is why "tokens per second" alone is a deeply flawed comparator.
This is also why discussions of Chinese models focus on tokens per second- they struggle to compete on intelligence so they dramatically increase effort levels to compensate, meaning they require massive total output tokens.
For example, flash 1017 isn't benchmarked yet, but 0731 required 15.4M tokens to complete the artificial analysis benchmark, and Sol 6.1 high required just 2.8M.
I certainly don't think the self-reported single benchmark scores are the best way to answer the question. I waited for the Artifical Analysis run (which itself is still mediocre, but at least it's an order of magnitude better), and predictably the results tell a different story.
Sol 6.1 High: 50 intelligence, $0.32 per task, 4.4 min per task, 13k output tokens per task
Sol 6.1 Medium: 48 intelligence, $0.21 per task, 2.9 min per task, 8k output tokens per task
Deepseek 4.1 1017: 45 intelligence, $0.26 per task, 4.8 min per task, 89k output tokens per task
If you look at the actual pareto frontier across the full benchmark, Sol 6.1 remains firmly on the pareto line, while deepseek remains firmly under it, just as it was before https://i.imgur.com/N72p5NF.png
This result is exactly as I expected, and seems to run counter to the effusive praise of this thread. Sol 6.1 Medium is producing higher quality output at a lower price with nearly 10X less output tokens! And this is not surprising.
And finally, Mimo V2.6 Pro continues to look like a stunning release, clearly smarter, cheaper and faster than deepseek v4.1 1017
bitexploder 14 hours ago [-]
"Sol 6.1 Medium is producing higher quality output ..." -- When it is actually up and working. Even last night I still received constant capacity messages halting codex over a dozen times and I gave up and shut the workflows down.
bitexploder 2 days ago [-]
I would prefer it work and solve quicker. Opus 5.5 has a reasonable pace to me. Sol does not. I understand efficiency I just don't like using it right now because it has literally hung or failed to complete tasks a dozen times in the last week due to inference capacity.
dingaling911 2 days ago [-]
What harness do you use for deepseek?
TheArcane 2 days ago [-]
opencode and deepseek harness GUI (dsh-tui when I need to)
ilitirit 3 days ago [-]
I've been using DS v4.1 Flash daily for nearly a month now on Agent Swarm experiments. I've tested it against GLM-5.3-flash and Opus 5.5. DS 4.1 comes out tops for me. Opus 5.5 is still the smartest, but DS 4.1's speed and price more than make up for the "intelligence" gap. I've ran Chief/Builder/Reviewer swarms autonomously for multiple days. I think my new workflow is going to have DS 4.1 Flash do all the engineering and just have Opus review its work. 4.1 Flash really is a very impressive model. Kudos to the Deepseek team.
spangry 2 days ago [-]
I'd love to hear more about your agent swarm experiments. What do you use to orchestrate?
ilitirit 17 hours ago [-]
My current experiment is running a Chief of Staff pattern within herdr, nothing too fancy. You should be able to find quite a few resources on the net.
ilaksh 3 days ago [-]
Ok but this is a new model with significant architectural differences and new training run.
dannyw 3 days ago [-]
Huh? This is a new checkpoint. It has identical architecture, only different weights, and most likely just more post-training.
Much like Qwen 3.8 Next; which is qwen4-exp in a lot of the code/patches, there's usually a tick-tock cycle with LLMs. That said, it's hard to know for closed frontier models.
ilaksh 3 days ago [-]
It's not identical, it has vision built in and asymmetric active from encode to decode.
DeepSeek delivery deliberately uses confusing version numbers.
staticman2 3 days ago [-]
4.1 flash was already multimodal. What do you mean by "It's not identical, it has vision built in..."
ilaksh 3 days ago [-]
Sorry I forgot that already happened with the release last month and thought it was new in this one.
ilitirit 3 days ago [-]
Then I guess all that it changes is that my proposed setup gets even better?
apejcic 3 days ago [-]
Its a great update, I ran a benchmark comparing verbosity and intelligence compared to old weights.
- Reasoning effort = 1 is now actually usable and produces 2.5x less tokens than old weights, without intelligence lost.
- Overall 20% less tokens at each reasoning effort.
- Same intelligence and same tool-calling pattern.
Improved on exactly the items that I hear people complain about with the previous checkpoint!
ytxm 3 days ago [-]
It passed my threshold for not caring to try new models back with DS4, I use it for literally everything. Announcements of new models in the series always feel curious, because it doesn't really feel like there's much left to fix that would improve on what's already there for any task I know of.
It's a bit like obsessing over patch releases of some software after a major upgrade already solved the original problem long ago.
xiphias2 3 days ago [-]
I have quite the opposite experience: I can do any concrete coding change with any model, but the intelligence limit just doesn't go through more complex debugging examples.
And then when I try a smart model, like Astra medium and it solves it (I had a firmware bug in my new laptop where I couldn't charge through USB-C) on the first try (altough with a repo already having lots of information about the hardware/firmware/debugging programs/traces), I realize how much time I'm losing on the few examples that the less intelligent models are just not able to debug and solve.
KronisLV 3 days ago [-]
Also they can work in completely new domains, like Astra and Opus 5.5 starting to be able to use something like Blender. Currently they still often do horrible topology and basic UV mistakes, and the animations are really bad, but who knows where they’ll be in 5 years.
oefrha 3 days ago [-]
I don’t think these are completely new domains in training. Like I’ve taken up offers to help code up AI training material just to see what they’re up to and get a sneak peek at how the sausage is made, no idea who these front companies are working for of course, but they pay for all kinds of fringe cases and Blender wouldn’t even be considered fringe in what I saw at the beginning of the year. (I was only there to observe, never submitted anything, so I didn’t betray you guys by helping train your replacements.)
squidbeak 3 days ago [-]
5 years? Look at the leaps since last November. Since the Spring, even.
redanddead 2 days ago [-]
I’ve heard about this through vfx artists
I was really hoping this would be solved since openai said that gpt-6 could paint brushstroke by brushstroke but it’s still shit
jck 3 days ago [-]
Very cool!
Is your laptop something like framework which has open source firmware?
I(and many other people) have a similar issue with usb-c charging not working consistently. I've been meaning to try one of these new smart models to help me debug the issue.
xiphias2 3 days ago [-]
ASUS Zephyrus, but it was able to reverse engineer the ports and I was able to download the firware from the controller itself, write an emulator that emulates running the firmware on multiple controllers with different instruction sets (the USB Power Delivery controller is RISC-V) and the communication between them. Also it found out how to read the communication buffer that was on the devices for debugging, which helped it understand the USB data packets that were doing the handshaking and voltage/current negotiation.
I had to reboot my laptop a few times though until I got to this, but it was a fun experience anyways.
At this point open source isn't as much advantage as long as the firmware is not encrypted :)
jck 3 days ago [-]
Wow, that is impressive. I've been meaning to do this by hand to fix the issue on my Lenovo IdeaPad for a while but haven't gotten around to it.
I'll try handing it off to one of these new frontier models.
3 days ago [-]
3 days ago [-]
lifeisloving 3 days ago [-]
Yeah, but for some reason people think they need SOTA to help them work on their side/indie projects.
Like if we needed PhD intelligence for productivity tools and SaaS platforms, why weren't we hiring PhDs to build things before.
99.99% of people do not need SOTA ai.
vikramkr 2 days ago [-]
Sota is not about intelligence. It's about behavior. Glm 5.3 flash is dumber that got 5.5 or opus 4.8 but better to use because it's more modern. it can manage subagents, handle longer running tasks, be steered, etc. it's better at doing the thing. Same thing with opus 5.5. fable 5.1 is probably smarter but opus 5.5 is better
pimeys 3 days ago [-]
Speed and price. You should try the 4.1 Flash on one of the fast providers. It's pretty crazy how fast this model works, and it's great for iterative software development because you can steer it fast to where you want to go.
Where does one even get decent ZDR provider? I'm wary of sending sensitive data but not at a scale where I would build my own machine.
therealpygon 3 days ago [-]
You just search around, or heck, just ask Google for inference providers that are ZDR and whatever else you want. Just make sure to read their terms/privacy and review the security center. You will have to request access if you are wanting to get into the deeper details.
Outside of major players like Amazon/Google which have compatible offerings available, I believe together.ai, fireworks.ai, and deepinfra.com are, as best as I can tell, US-based (if that matters in your situation) SOC2/HIPPA/ZDR. Each of them offer serverless model hosting with caching, which is going to be the cheapest overall. Most of them also offer private AI deployment which is more expensive but your entirely own instance. Some of them more heavily quantize their models (deepinfra) so take that into your workload considerations. I included deepinfra because they tend to have some the new models sooner than others or have models the others don’t (like mimo). IMO, together > fireworks > deepinfra.
However, you’ll have to keep in mind that for true ZDR you give up caching which increases your inference costs, so it depends on how “Zero” you need the zero to be and on what timeframe. If you just don’t want your data being trained on, normal use is fine imo. If you are working on something actually sensitive (like, because of NDA agreements and such), then they might not be the right choice and you may be looking at more expensive offerings or building a machine.
One note: you have to pay attention to the source of the inference. Many (similar to open router) offer models that are pass-throughs to a providers API, but they are pretty clearly marked. Just make sure the model is marked as serverless when evaluating new ones. This is where a proxy comes in handy so you can prevent your LLM from accessing a different models from the specific ones you’ve vetted.
ArtRichards 3 days ago [-]
I like privatemode.ai for better than ZDR, encrypted attested hardware (in EU)
How are these high-speed providers close to matching the API rates from deepseek?
pimeys 3 days ago [-]
Seems that V4.1 Flash is quite cheap to serve. Especially how it implements the cache, meaning the cache pricing can be very low depending on how you implement it in your inference engine.
Depends on your use case. But if you do software development with a harness such as oh-my-pi, your cache rates are probably somewhere above 98%. Coralbricks has free cache reads, meaning it's much cheaper than the Deepseek API rates.
Lithosai is more expensive especially in the ultra tier. It's if you want extremely snappy and fast responses, which is important e.g. for a personal assistant where every second counts.
If you generate a lot of output, you're better off with the Deepseek API rates.
lifeisloving 3 days ago [-]
Ive never needed an assistant pre or post ai to reply to me in 100ms at a clip of 500 words per sec. See you in 15 min, Im making some cofeee.
arvid-lind 3 days ago [-]
same here, but surely the trends will follow every other profit-seeking web app ever and distill down to a competition over attention.
mmoustafa 3 days ago [-]
definitely could use less thinking time and lower token usage, which is what this bump is focused on
mdp2021 3 days ago [-]
> improve on what's already there for any task I know of
What about "good governance"?
Does it have zero hallucinations? Does it reflect thoroughly on every single idea it uses, verifying its solidity?
sligbad 3 days ago [-]
Same. I'm just happy sticking with them and am consistently impressed. Cost is so good and kind of keeps you honest.
3 days ago [-]
olavgg 3 days ago [-]
I really like the having DeepSeek Flash + Opus 5.5 as advisor/reviewer at the moment. I run through a $200 Claude Code subscription in less than 24 hours now just for a single project, and I hate fiddling with different accounts every day. Now that I get a 200 USD api credits inlcuded with the subscriptions I have rewritten my coding agent workflow completely.
rpastuszak 3 days ago [-]
> I run through a $200 Claude Code subscription in less than 24 hours now just for a single project
How?
olavgg 3 days ago [-]
Its easy when you spawn a couple of subagents in a homelab with a rack with 5+ servers.
This is how one of my claude code session looks like right now, one prompt from yesterday:
Not sure if this will help you but it helped me. I was using subagents and workflows for lots of long running tasks, and part of that work included running tests. If tests took longer than 5 minutes, the cache expired, and the whole thing was then being processed as uncached input.
I created a skill to better manage agents, including a way to keep the caches valid, and it has decreased my usage considerably. I now struggle to use my 100% each week and I hit claude hard. I have a reset expiring in 5 days and I'm trying to burn down the tokens (68% with 3d 8h left) quickly so I can use my reset before it expires.
Token efficiency just means understanding the way the system works. Best person to help with that... my old friend claude.
dotancohen 3 days ago [-]
After I stopped letting Claude run tests, my token usage went from "gone by Wednesday" to "what can I automate this Saturday so my tokens won't go to waste".
rednb 2 days ago [-]
Thanks this is very helpful, i didn't realize that. But not sure what to do since running tests is how Claude often catches mistakes and iterates. If i run them myself, the cache will still expire.
However where i think you have a point is that it tends to run tests quite casually where a reviewer would have been enough to catch mistakes.
IMHO, the real problem is the 5min cache way too small. They should up it to 1 hour.
Oh makes sense thanks, so i'll forbid agents from running builds and tests.
tasuki 3 days ago [-]
What is the reason for this? Surely running tests by itself isn't necessarily token-expensive? And it provides cheap feedback for the agent: to see whether they're breaking things.
dotancohen 3 days ago [-]
My guess is that the LLM (usually Sonnet or Opus in my case) is burning tokens polling to see if the output finished - and there is often a lot of it. My current project is Rust, Python, and Kotlin for what it's worth.
I don't actually know the real mechanism, I came across this serendipitously.
erincandescent 3 days ago [-]
By default Anthropic's token cache TTL is 5 minutes. If your tests take >5m to execute, when the LLM resumes your entire context is a cache miss
rixed 2 days ago [-]
Will try to remember this next time I'm told that devs don't care about performance.
tasuki 3 days ago [-]
There are many reasons why your test suite shouldn't take >5m to execute. This is just one of them.
dotancohen 2 days ago [-]
I've worked on projects where the test suite would take hours.
rednb 2 days ago [-]
.NET solution having 120+ projects, with 7000 tests of which maybe 1500 are integration and/or component tests.
Buid alone takes 4 minutes. Test suite heavily parallelized runs in about 2 to 3 minutes, infrastructure dependencies fully containerized using podman, no external API call.
And this is without mentioning the 5000 frontend tests (angular), including Playwright end-to-end tests wired to the backend (about 3 to 4 minutes run without the e2e tests).
And this is running on a dev machine with 64gb of RAM and an NVMe disk.
Some systems are just massive, and even if you run focused tests, it's easy for a change to have a large blast at least requiring a large rebuild, and at times to rerun a larger set of tests.
dotancohen 2 days ago [-]
Do you have a link to the docs for that? Thanks.
tomrod 3 days ago [-]
How do you do that?
dotancohen 3 days ago [-]
I just told it that I want to run the tests, because so much of the code was vibed at that point and I felt I was losing understanding of the application. A few days of that and I could see that my token use was much lower than usual. Going back to LLM-run testing returned the high token usage.
olavgg 3 days ago [-]
Yeah, I should see if I can create a similar skill. The tests do spend some time to complete.
Claude: Confirmed: 195M of the 212M cache tokens (92%) come from full re-writes after 5+ minute gaps, accounting for 77% of relative cost. Now I'll check the existing wrappers so the skill
builds on them.
RALaBarge 3 days ago [-]
Yesterday in my CC there was an update that added a feature to auto compact immediately prior to the cache being invalidated.
zerd 2 days ago [-]
Interesting, I made a Claude code plugin to do just that two weeks ago.
stefan_ 3 days ago [-]
And I immediately turned it off because that just invites utter churn and stupidity. Part of that is just CC though, their compaction is miles behind Codex.
iamflimflam1 3 days ago [-]
Some to think about with both Claude and skills.
Can you invert things. What’s deterministic and what needs “thinking”?
Can you create a script that runs things chaining together the deterministic parts with calling out to Claude for glue that needs “thinking”.
wolfy1993 3 days ago [-]
Don't suppose you would be happy to share the skill? Or is it fairly unique to your work flow?
I think the tests problem is really having an impact for me. Hadn't considered it before so thanks for flagging.
ceasesurthinko 2 days ago [-]
Super useful. Thanks.
raylad 3 days ago [-]
How do you keep the caches valid?
entrope 3 days ago [-]
Not OP, but I typically ask it to report status of whatever is running; it can run a background task that reports to it every N minutes. With Claude Code in VS Code and a Pro subscription, it seems like the one-hour cache is the default, making it an issue less often.
j45 3 days ago [-]
Ask Claude, it's pretty wild to see.
oblio 3 days ago [-]
> my old friend claude.
Awfully new old friend.
dotancohen 3 days ago [-]
Google is the ex, Altavista is the high school sweetheart who tragically passed at a young age.
nextaccountic 3 days ago [-]
> including a way to keep the caches valid
Maybe doing bogus small requests to prevent the cache from expiring?
I mean in the API you can set the cache expiration time right? But I guess claude code can't do this
devtor 3 days ago [-]
As a dev who hasn't used LLMs as an embedded part of their work, and uncertain about it all, i'd be curious to know:
- How long do you reckon it would take a human to do the task?
- What percentage of features/ feature-extensions you add would have been done at all if you didn't have agentic assistance? Is there a tension between llm productivity gains and 'because we can' feature additions?
- Are many of the tasks for throwaway v1 features, exploratory tries?
- How speced out is the work you give it, eg did this task have pages of specs, how long did the spec take to write?
- How deeply will you manually review the work and how much time will it take? Will you read all the code / review at a high level and interrogate with llm / have the llm review and self iterate?
olavgg 3 days ago [-]
For what I am currently doing, years, if not a decade or maybe even two. Claude has already been running for 3 months constantly working on this project. I've been trying to do this several times before, but the scope outgrew my available time/life, too big for one person. But now I can actually build a production release by just setting the direction and let Claude to all the wiring, plumbing, tests etc.
I do now add all the major features I want, and write a task list where I add a few more minor tasks each week that just come to my mind. I need to write them down before I forget. I often ask Claude to implement several different implementations/algorithms and compare them. There are plenty of trade-offs, which usually end up in building configuration profiles, so the application can work optimally in multiple environments.
The specification is actually really large, and has grown over time as I've been learning and new ideas has come to my mind. I do not review/audit the code, it's massive would take me months, no chance! I do however use code inspection/audit tools like TSAN, ASAN and more. I will also spend time on full code reviews with the coming Fable 5.5 when a release is about ready. I do however review behavior together with Claude, and this is also where the specification grow, because this and that should be possible, but not this and that. User stories also helps.
devtor 3 days ago [-]
Thanks, appreciate the reply.
lifeisloving 3 days ago [-]
Keep doing what you're doing, this guy clearly is just revving his engine because he likes the way it sounds and can afford the gasoline. I wouldnt be asking people like this for advise, they're completely off the rails.
There's no reason to blow through tokens this way, there's a 99% chance nobody uses the output (except maybe themselves).
bluerooibos 3 days ago [-]
I have some fairly large personal software projects and I still don't understand what useful work people are getting out of burning all these tokens on unattended agents.
What exactly are they doing for you that's giving you $200 of value in such a short timeframe?
3 days ago [-]
cloudie78 3 days ago [-]
Yes, but what was the task that they were working on? This tells nothing.
That’s the “how” we’re wondering about.
olavgg 3 days ago [-]
[flagged]
lifeisloving 3 days ago [-]
You're doing this wrong.
ramigb 3 days ago [-]
I swear I have the same question in my head always when I read such statements! I have never once reached 90% that includes everyday usage across multiple repos, artefact generation, device debugging, long running harness work!
I am not saying OP is not telling the truth. I just really want to understand!
MisterMunchkin 3 days ago [-]
It’s because they don’t know what they’re doing. It’s like when openAI was using 10,000 agent clusters to solve problems… then realised the model could do the same work on its own and solve the same problem.
Anyone who tells you they have a “team of subagents” and “code reviewer” agents and “project manager” agents doesn’t know what they are doing. That’s how they burn thousands of dollars in a day.
senordevnyc 3 days ago [-]
Do you have a source for OpenAI discovering they can solve Navier Stokes with a single agent instead of a 10k swarm?
I think I’ll trust the experience of the people in the labs who have unlimited token budgets and are running experiments continuously over some anonymous rando slinging mud on the internet.
Also my own experience, which says having code reviewer agents is one billion percent useful. That comment alone completely disqualifies you as a reliable source of info.
dgacmu 3 days ago [-]
I decided to try something a little weird research wise the other day analyzing about a thousand older research papers and burned through my $100 chatgpt sub + 3 resets in three hours and only finished half of them. It's the first time I've done that, though.
pheggs 3 days ago [-]
in my experience there's a huge difference in token usage when creating new projects versus working on existing ones
TomGarden 3 days ago [-]
Not OP, but I used to run out in a few days, for me it was a matter of suboptimal code review practices with many subagents looking at different aspects, often expanded to look at more than the diff.
3 days ago [-]
rixed 2 days ago [-]
The modern version of the mythical man month: adding subagents to an already large context window makes it even larger. And harder to cache.
hugodan 3 days ago [-]
not reviewing its ongoing work and spawning subagents that end up looping each other
segmondy 2 days ago [-]
skill issues, brute forcing with AI.
fahrvrgnugen 3 days ago [-]
Loading tons of MCPs and never compacting
nimchimpsky 3 days ago [-]
[dead]
imadierich 3 days ago [-]
[dead]
monster_truck 3 days ago [-]
I can drain a $200 OAI plan in 2 hours flat working on my 10 projects simultaneously.
Maybe you're not being ambitious enough with these things? They are at their most fun when you spec out sprawling architecture documents and watch 1+8 astras buzzsaw through it.
I'm only at 260B for the year, though. I'm sure someone who has used more will be along shortly to chime in
E: The other subthread talking about tests is correct. Expensive models should not waste a single second waiting on tests or validation. If you cannot assign a single cheap model to be the test bitch, then just tell them to frontload everything and clean up later as a single wave.
ricardobayes 3 days ago [-]
Not meant as a bad faith response but have you shipped anything? Do you have income from this? Why 10 projects apart from fueling ADHD and "never finishing a project"?
joshstrange 3 days ago [-]
Not OP and while I’m not draining my accounts in a single day/hour, I’m hitting limits with a $200-Claude and $100-OpenAI account.
> Have you shipping anything?
Yes, hundreds of PRs and shipped new features to 3 different platforms (day job and 2 side-business apps/platforms I’ve been developing for many years, pre-LLM). Those new features have gone out to customers and been used.
> Do you have income from this?
Well my day job for one, and I’ve been able to more easily sell to new clients with my side business. Without the changes I’ve shipped I would have been able to land a couple of the clients I’ve signed on recently.
I don’t think OP’s message implied they are never finishing a project, maybe they edited it?
In addition to the 3 income-generating projects, I have a number of other “just for me” projects. Some are mature now and only need occasional development when I want a new feature. Some are in flux as I build them out (like my “software factory”).
At this point I’m limited by compute and tokens more than anything else.
ImaCake 3 days ago [-]
Does work pay for your subscription? I think many workplaces would worry about personal accounts for work projects.
EDIT: but thanks for a useful answer to OP's question. As a programming dabbler I also find it hard to imagine how to push much past the 5hr limit on $20USD subscription.
joshstrange 3 days ago [-]
Work provides a stipend/reimbursement of $100, the rest I pay for.
It’s paid for itself though, at least from my side business income.
tedk-42 3 days ago [-]
Some of us want to build and don't care that it doesn't monetise.
Geof25 3 days ago [-]
Surely such people does not have 100usd/hour budget?
loveparade 3 days ago [-]
Would love to see these 10 projects that are being worked on simultaneously
equasar 3 days ago [-]
I'm baffled, my $20 is enough for me to do some consulting, personal projects and side work, and still cannot outran my quotas, I'm building different kind of things: flutter apps, dotnet backend, database, front end, architecture diagrams, code exploration, tests, PowerShell scripts.
How people spend more than 200 usd monthly is beyond my understanding, are you guys so inefficient with the usage? or I am the opposite?
theowaway213456 2 days ago [-]
Without any context for what you're working on, I'd have to assume the reason you don't hit the $20 subscription limit is that you're either not using it very often, or you're working on some relatively basic CRUD app that doesn't require it to think very deeply.
I tear through my $200 subscription because I often have 3-4 agents running simultaneously, exploring performance optimizations, building features, monitoring production rollouts, or investigating production problems/customer issues in parallel.
The downside is that I do spend most of my time reading the agent's output these days.
equasar 2 days ago [-]
I think I do quite the same as you, complex features, not anything basic.
lofaszvanitt 2 days ago [-]
Well, it works for you.
newsy-combi 3 days ago [-]
I'll bite. Wtf are you working on that takes $200 a day and you're willing to pay for?
Geof25 3 days ago [-]
By not using existing tools and one shooting everything through the API - I.e. instead of decompile via Ghidra they will try to get source via capstone genersted assembly.
It is like putting a jet engine on a shipping container and wondering why it needs to be constantly kept in afterburner to at least scrape on the road while smarter users put wheels on the shipping container.
olejorgenb 3 days ago [-]
After opus 5.5 release I find it very hard to run out on a 100 sub...
aleqs 3 days ago [-]
Yeah the Claude/anthropic usage is quite arbitrary, I also run through my entire 20x pro plan in 1-2 days. Codex has slightly better usage levels, but is annoying to use for many reasons. DeepSeek flash (direct or via opencode go) is the best value model I've found so far.
saneetomar86 2 days ago [-]
[flagged]
thomasmarcelis 3 days ago [-]
Interesting, coming from a similar situation, I'm interested what workflow you landed on. Opus planning + available to be called for deep thinking + opus reviwer? or?
zmmmmm 2 days ago [-]
> I run through a $200 Claude Code subscription in less than 24 hours now just for a single project
I would be worried about your methodology when the subscription party ends. Make no mistake, one day, the party is ending and the people who are dining out on approaches that burn massive tokens are in for a rude shock.
azan_ 2 days ago [-]
They are not, opus 5.5 level intelligence will be dirt cheap in a year or two (in non-subsidized api pricing).
ilaksh 3 days ago [-]
This is a new model.
_R_ 3 days ago [-]
[flagged]
sick_of_slop 3 days ago [-]
[dead]
freakynit 3 days ago [-]
Mario one-shotted by deepseek-v4.1-flsh-1017 (in high mode, not even max) with sound and programmatically generated assets in under 5 minutes: https://mario-dsv4-1-flash-1017.pagey.site/
rajangdavis 2 days ago [-]
Have you tested the game? Jumping on the pipe after the koopa and goomba kills mario
epolanski 3 days ago [-]
By far my favorite model.
At the beginning of the month I realized I had more tokens and more capabilities on Opencode at 8$/month via DS 4.1 flash, than I had on a 200$ account in June with the american providers.
Literally both more tokens and capabilities than Opus 4.8/GPT 5.5.
Even if you pay per token, you're still unlikely to outspend american providers.
Flere-Imsaho 3 days ago [-]
I've been telling people this for a couple of months now, I'm actually struggling to use up all my tokens.
jimbob45 3 days ago [-]
What does it matter if you’re not hitting your cap with OpenAI/Anthropic in the first place? This thread is apparently populated with the highest of powerusers running multiple OpenClaws. Us peasants aren’t concerned with meters or cost/token - just quality - and nobody comes close to OpenAI or Anthropic in that regard.
hgoel 3 days ago [-]
Pretty funny to say "us peasants aren't concerned with ... cost/token", as if most people ("peasants") won't find it absolutely insane to be paying $200/mo for a subscription.
selcuka 3 days ago [-]
Peasants don't pay for the subscription. The employer does.
platinumrad 2 days ago [-]
Corporations pay API rates. Subscriptions are only available to individuals.
AntonyGarand 2 days ago [-]
Large corporations pay API rates: Claude teams for example can buy subscriptions for its members, until they reach 150 of them after which they need to migrate to org.
bagacrap 3 days ago [-]
How could that be considered insane for a professional tool?
epolanski 3 days ago [-]
In southern Europe (Italy, Greece) that's well beyond 10% of the average pay.
And southern Europe isn't exactly the poorest region around.
bagacrap 2 days ago [-]
For a software engineer?
If you can get away with paying that much to a swe in Italy then why aren't Italian tech companies ruling the world on the back of ~free labor?
ilaksh 3 days ago [-]
But this is a new, different model that just came out.
cmiles8 3 days ago [-]
Honestly given how good the open weight models are getting I don’t see a viable business model for OpenAI and Anthropic anymore. The profit is in the compute, not the model. OpenAI and Anthropic spent all their money renting compute from others so yes they can sell lightweight model inference but they have a massive structural disadvantage now as others can always sell good-enough inference much cheaper.
There’s also zero stickiness to models and most companies now have their own internal routing utilities so devs and users can just keep the same API endpoint and move between models with a single line of code swap.
Unless something changes with the big labs and fast, they look pretty f*cked.
theowaway213456 3 days ago [-]
People will pay for more intelligent models. Astra and Fable are more intelligent than these open weight models and can solve problems that are much more difficult, and that's extremely valuable. It's not any more complicated than that.
preommr 3 days ago [-]
> It's not any more complicated than that.
It is more complicated than that when the valuations and business plans are contingent on those companies AI having complete dominance at every level.
If companies only use frontier models for occasional difficult tasks, not only does that affect volume, it basically elminates stickiness and incentivizes companies to instead of using their products continuosly, find ways to move from the expensive stuff to the cheaper stuff.
cmiles8 3 days ago [-]
Yes but that’s a tiny portion of AI use cases. Maybe 1-2% max. The big labs need a lot more than that to survive.
You’re assuming other model creators can’t build these mega models. Evidence is they’ve simply decided it’s a fools game to do so and they’re focused on where the use cases are.
grumple 3 days ago [-]
Fable is behind Opus (which is also much cheaper) at this point in benchmarking.
xbmcuser 2 days ago [-]
[dead]
andreimackenzie 2 days ago [-]
Agree with you on model quality, but Anthropic has done a nice job developing stickiness. The connector ecosystem, integrations with desktop apps, browser use, etc. are all pretty compelling for a typical business use scenario. It's expensive, but also a "safe choice" akin to buying IBM or Microsoft back in the day. My workplace (not just developers) is deeply invested in the Claude ecosystem & it would be pretty disruptive to pivot it.
jacquesm 2 days ago [-]
How far could they jack up the price before you would consider switching?
niek_pas 3 days ago [-]
I don’t know what I’m talking about but it seems to me OpenAI is heavily emphasizing productization of LLMs. Dots, ChatGPT Work, scheduled tasks, in-app projects etc.
Starlevel004 3 days ago [-]
Being able to directly distill their huge frontier models into Luna/Haiku-level models puts them a step ahead still.
m3kw9 2 days ago [-]
Most corporate will not be using deepseek, haven't heard of it.
metalspot 2 days ago [-]
There is either no correlation or a negative correlation between product utility for price and Enterprise and Government purchasing decisions so OpenAI and Anthropic should do just fine.
They made very bad bets on scaling, which has cost ~10B scale in wasted capital allocation, but it is manageable. Their purchase commitments for future compute capacity are absurd nonsense, but all of that is just shtick for retail investors, and they will reshuffle the deck as needed to adjust.
rikima_ 3 days ago [-]
i hate dario's politics, but i still find claude's eco system easier to use. Design/Artifacts is very convenient for people like me who has zero UI/UX expertise, and claude code's remote session works seamlessly on ios with a well-design client interface while you have to deal with ssh and typing in cli with your iphone keyboard if you are using opencode/omp etc
greenavocado 3 days ago [-]
That's why they are constantly fearmongering and trying to get other models outlawed. If nothing else, under the pretense of national security.
desolate_muffin 3 days ago [-]
And yet, nearly all of enterprise money spent on AI tokens in the US is going to either OpenAI or Anthropic.
cmiles8 3 days ago [-]
That trajectory appears to be changing. Virtually every company I see is shifting more and more to open weight models. Anthropic’s slowing growth rate is already spooking investors and that shift is just getting started.
jononor 3 days ago [-]
Above GLM 5.3, nice! That is a very capable model. I do wish that we will see some smaller "flash" models again though - running this one locally is tough (very costly).
LaurensBER 3 days ago [-]
For comparison, close to Opus 5. Far behind Sol 6.1 and Opus 5.5.
Probably a pretty decent model for 90% of the usecases!
I've been doing a lot of hobby work with GLM 5.3 Flash and I can count the amount of times that I need a stronger model on two hands.
I have access to unlimited tokens at works and Opus is great but also often overkill if you know exactly what you need.
egeozcan 3 days ago [-]
I use Opus for planning, through API on medium thinking, it doesn't cost anything significant and the rest can be happily done with deepseek or glm flash. Using sol 6.1 for review loops at the end. All running on pi.
A very complex task tends to cost like 10$ max. It's comparable to a subscription with enormous flexibility.
It's a steal.
pimeys 2 days ago [-]
I'm using DeepSeek a lot, and I'm curious how you get a task to go to ten dollars with it? Or are you mixing Opus and Sol tokens to the same task? I have very hard time to go over 5 dollars with DeepSeek and that is already a weeklong project where I share the same context.
Of course I do certain things like use omp and do a compact picture snapshot after 256k tokens automatically. My tasks rarely go over a dollar...
I've ported a few games, wrote a cosmic desktop app, iterated in my own agent software, use the model in our home speaker system, use the model in my phone assistant. My monthly bill to coralbricks is about 14 dollars.
egeozcan 2 days ago [-]
That's the total cost of a task across all models. DeepSeek going above 2-3 dollars is actually rare because of the very low cache costs.
pimeys 2 days ago [-]
I also used to plan with more expensive models like you did, but now I just do planning with DeepSeek too and don't see much difference. I use the omp plan mode and iterate on the text until I'm happy with it. Then clear context, give the plan to DeepSeek again for implementation.
If it's mission-critical code, do a security review with GLM 5.3.
Otherwise go manually through the code and ask for changes if needed.
Works really well for me.
benhurmarcel 23 hours ago [-]
I'm curious why you use Opus for planning and Sol for reviewing, instead of the same for both? Do you find they are better for those tasks?
cromka 3 days ago [-]
Honestly, I don't know what people see in Sol 6.1. Opus 5.5 is so much more inteligent, as in proactive, avoiding stupid mistakes, where it contradicts itself, ignores the context. Sol 6.1 has a gold-fish memory, it's as annoying for me to work with as Opus 5 was.
ac29 3 days ago [-]
I'd rank Sol 6.1 only slightly behind Opus 5.5 in intelligence and both significantly ahead of anything else I've used (though cheaper/faster models are totally fine for a lot of tasks). My only real annoyance is 6.1 Sol is one of the slowest models I've used, which does make Opus 5.5 more pleasant to use.
cjbgkagh 3 days ago [-]
You can override the context length of Sol to 1M
joshstrange 3 days ago [-]
Without this change I hated Sol and was burning tokens way faster due to all the forced compacting.
Since making this (obscure/hidden) change I enjoy Codex much more.
cjbgkagh 3 days ago [-]
The work I do cannot fit in 256k so it was just relearning the same things over and over again while making zero progress. AFAIK the 1M burns tokens at twice the rate but I use Sol on medium as the work I'm doing now needs more memory than it needs intelligence and it's working great. I use Opus 5.5 for the stuff that needs stronger intelligence.
cromka 2 days ago [-]
Fair. I bumped the context size, will see how it goes.
cromka 3 days ago [-]
That's not actually the problem. It forgets the instructions from 15 minutes ago, without context compression involved in-between.
cjbgkagh 3 days ago [-]
That has not been my experience after bumping it up to 1M. I do run a lot of agents in parallel ~10-16 and I actively babysit them so perhaps I'm just used to reminding the agents with things I've already told them. But I don't notice much of a difference between Sol 6.1 1M and Opus 5.5 1M
0xkasra 2 days ago [-]
[flagged]
redanddead 2 days ago [-]
I never understood the hype with Claude
I tried Claude legitimately for a few months, on max
That was during the great anthropic compute/gaslight crisis (ahh good times..), Claude then enthusiastically bricked my MacBook and I had to get gpt to unfuck it
minraws 3 days ago [-]
By next year we will likely see GLM 5.3 level performance in Qwen 4 MoE models that can run on most higher end consumer systems.
I don't think there is a reason to care about any of it, just get a few dollars on a few openrouter or similar platforms and just use the models for now, wait for cheaper models that are good enough for you to become available for your hardware.
Sure you might never see frontier performance on consumer hardware but do you really need that though? Does anyone? And if you ever do just get a 30-day sub for that work, and cancel it.
jononor 3 days ago [-]
Yeah performance at a given compute budget will likely continue to improve.
We are running Qwen 3.8 27B FP8 with 4x 5060 16gb at the office for a small development team. It is doing pretty well for implementing backend and frontend tasks, and is already useful. In 6 and 12 months from how I expect it to get even better. Some planning is still done with a stronger model, but even that is maybe not strictly necessary.
minraws 3 days ago [-]
Qwen 3.8 wasn't that nice in my experience Qwen series has lagged a lot but I think Qwen 4 would be better they cleaned up the architecture.
And we are likely to see many companies start building 27-32B MoE models custom built for use cases now that Rubin is going to make training costs much lower.
I expect training costs to get under 100k USD within a year for small models with another 100k maybe for RL.
I think just having all your own data from claude and stuff on your system that you can feed these models and have companies offer great base 27B models will be how it works.
Pragmata 3 days ago [-]
Cant you run it on a pair of DGX spark? Sure it's expensive now, but at the initial retail price that was what, 8k in hardware? Not exactly unachievable
bashtoni 3 days ago [-]
No, this is not a small model. I believe 4x Sparks is possible but 4.1 is pretty big for a flash model.
MiMo 2.6 Flash is the most capable model you can currently run on 2x Sparks without resorting to a quantised version of something.
Pragmata 3 days ago [-]
I meant quantized. You can get an EXL3 quantized version thats almost the same performance.
bashtoni 3 days ago [-]
My experience running quantised models is that they look the same at first but are all noticeably worse once the context size gets beyond anything trivial.
Pragmata 3 days ago [-]
I've tried both 2bpw and 3bpw quants for QWEN3.8 27b, and while i do notice a difference in terms of higher rates of getting stuck in loops, they are both reasonably usable.
Are they as good as opus 5.5? No.
Are they about on par with opus 4.6? Yes.
The long run tasks i run seem to be about the same level of quality, though there are obvious differences in model styles, token budgets, etc....
dcchambers 3 days ago [-]
At this point I am way more excited by DeepSeek releases than anything from OpenAI or Anthropic.
The speed and cost/intelligence ratio is just outrageous.
davedx 3 days ago [-]
I have started phasing into DeepSeek v4 (Dwarfstar quant from Antirez) as my daily driver and it's going very well. It can one shot iOS apps on my Mac Studio, it's a little slow and sometimes needs retries but can definitely do medium difficulty real work for me.
I didn't try the last flash model because I would need to use SSD streaming (I have 128 GB), but the benches for this release are so compelling (and it using less tokens to get tasks done) it might be worth the throughput hit. I will probably give it a try when I have some downtime.
It feels very freeing to be doing useful work on a local model on my own hardware with my own coding harness. The TCO argument is starting to look compelling too.
kamranjon 3 days ago [-]
Try the Qwen 3.8 flash next model on DwarfStar - you will be very surprised. I’ve been using the 4 bit quant almost exclusively after having DS v4 flash be my daily driver. Qwen is faster and seems to be much better at coding tasks.
m3kw9 2 days ago [-]
why does mac studio enter the equation, it'd one shot it on mac neo
lisplist 2 days ago [-]
Gonna be waiting a while streaming your weights from disk
davedx 2 days ago [-]
It would be slower :P
ilaksh 3 days ago [-]
This is a new model, not the same as a different one.
liuliu 3 days ago [-]
Still need 2 or 3 days to properly quant this. But it is really good model to run if you have 64GiB or larger RAM on Mac. On M5 Ultra, you get ~3400 tok/s for prompt processing and ~50 tok/s for token generation, quite usable state.
RGS1811 3 days ago [-]
Forgive the ignorance, but how are you running a 763B parameter model on 64GB of RAM?
liuliu 3 days ago [-]
SSD weights streaming. The actual weights is 552B, then engram.
The smallest quant is still over 100GB, so I don't know how that fits in 64GB RAM. It would have to be partly streamed from disk.
springtimesun 2 days ago [-]
I feel like I’m an outlier based on all these comments. This is still working through my personal benchmarks, but it’s spending 30% more time and performing worse than the previous 4.1 flash did.
Benchmark is historical PR reviews. Models are put in a sandbox with a git and db snapshot as it existed at the time. PRs are a selection of changes some bugs (major and minor), some with things that aren’t bugs that have tripped up models in the past and some no findings that they should approve and move along.
4.1 flash already wasn’t that strong at this task, but the speed edged out v4 pro for a spot in the roster.
a11r 3 days ago [-]
A negative datapoint - We have been evaluated prior Deepseek Flash models (both the original 4.1 flash and the smaller 4.0 flash) and they have been overthinkers. Our workload is well defined but moderately difficult coding tasks and among open weight models GLM5.3 flash performs best with Qwen3.8 flash next a close second. DeepSeek 4.1 flash fails at lower thinking levels and spends a huge amount of tokens (an order of magnitude more) at the highest thinking levels where it is able to successfully complete the tasks. I will evaluate this newest release since they specifically claim "the model reduces task execution time and output length", but keeping my hopes low.
pornel 3 days ago [-]
v4 was a worse overthinker. 4.1 mostly fixed that. It still prefers to plan everything before making a single file change.
If I'm impatient I switch to Luna for a single turn just to get it started and let DS continue.
Faithful to the original theme. Apropos. +1 on adding the prompt as long as it’s affordable and easy enough Simon, and not too distracting/noisy/time consuming all considered!
Wow. Is DS the only model that consistently puts one leg behind the frame? I don’t even remember Opus ever doing that.
simonw 3 days ago [-]
Some of the most recent Claudes and GPTs have done that too, and Gemini has done it for a while.
bagacrap 3 days ago [-]
The best bicycle is "low".
lofaszvanitt 2 days ago [-]
This one has ample space in its beak.
eleventen 3 days ago [-]
[dead]
risphere 3 days ago [-]
It's better than GLM 5.3 and Kimi K3 and way cheaper.
slopinthebag 3 days ago [-]
that's honestly insane. it's like the go-to for when you don't want to use opus 5.5/sol 6.1
ngl999 3 days ago [-]
The go-to is a local model.
gentlewater 3 days ago [-]
Limp-to, maybe.
ngl999 3 days ago [-]
With Strata, running GLM 5.3 is possible.
slopinthebag 2 days ago [-]
$$$
epolanski 3 days ago [-]
Is it already updated to the newest version on Opencode?
nirav72 3 days ago [-]
Yes. It’s actually free to use at the moment. Of course with a quota.
riknos314 2 days ago [-]
Both the free (seems limited to ~200k context) and metered variants in opencodego show the release date as 1788998400000 → 2026-09-10, which implies they're not the newer checkpoint
riknos314 1 days ago [-]
[flagged]
johnnyApplePRNG 2 days ago [-]
I asked opus 5.5 to run 3 "hard" questions against haiku 5.5, sonnet 5.5, and this latest deepseek 1017 as worker agents running claude code... and deepseek won.
Not only did it come in significantly cheaper even than Haiku, but it was more correct than Sonnet.
This is great! I am gonna get coding!!!
waximabbax 3 days ago [-]
Looks very impressive especially considering the price and speed. Let's wait for AA to confirm how well their benchmarks hold up.
scrollop 3 days ago [-]
You trust AA since the sol fiasco a month or two ago?
grosswait 3 days ago [-]
What did I miss?
esafak 3 days ago [-]
They updated their methodology when the new openai model didn't rate higher than the old one, or something like that.
mchusma 2 days ago [-]
I do t think it was a fiasco. I thought the updates were good. They should do more work on benchmarks. There has been good work on testing benchmarks for example to ensure they are accurate, and as issues arise they should change the benchmarks. The point of AA and Arena is trying to understand how to think about models.
waximabbax 3 days ago [-]
[flagged]
frail_figure 3 days ago [-]
Why not just try it yourself?
viraptor 3 days ago [-]
It costs money to run serious tests. We're way past the time when you could get an idea of the new model in a couple of tricky prompts. Now I need to compare a whole resulting project to get a clear view. If someone else proves the benchmarks don't hold up in reality, then it saves us further tests.
summarity 3 days ago [-]
Getting a lot of errors from OpenRouter's endpoint "Failed to get response from the AI model; retried 5 times (total retry wait time: 60.71 seconds) Last error: 429 Provider return error"
It works when it works, but will probably be flaky AF for a few days.
700000 because there's no detail yet on the safe completion limit for most harnesses (OR lists 1M for both which makes no sense). COPILOT_OFFLINE enables all calls except to OR (no GH servers).
risphere 3 days ago [-]
The native web search api is 10x cheaper than OpenAIs. I hope they will release the web search api as a standalone API.
alex-moon 3 days ago [-]
I may be missing something so apologies if I'm telling you something you already know but: there are plenty of web search APIs for agents - the Brave one is the one I use: https://brave.com/search/api/
risphere 2 days ago [-]
I already know the Brave Search API, but thanks for telling me. The issue is that it's expensive. It's 5 USD per 1000 requests, while DeepSeek's is 0.75 USD per 1000 requests.
mmmmbbbhb 3 days ago [-]
Tavily one is free, too. Easy to use cli with a skill.
risphere 2 days ago [-]
Most web search APIs offer a MCP with limited free usage, but for production use you still need to pay.
ytxm 3 days ago [-]
I gave it a kick and got mixed results. For example, searching for a Microsoft product couldn't find its landing page, and searching for a niche open source package landed in the Debian repos for the top hits, but the home page was in the first 10 hits.
Lovely to see they're doing their own search though, I love how many folk are actively working on search clones at the moment.
risphere 2 days ago [-]
I haven't really tested it. But they might be doing keyword search like ceramic.ai instead of semantic search, so that's why the results might be worse, but I'm not sure.
It was already cheaper and more powerful than Haiku 5.5 which was released like a full month after it... by a trillion dollar american "frontier" company.
Can't wait to see what this one has in store.
nik736 3 days ago [-]
Those are crazy numbers for a Flash model. Hopefully they can scale what they are doing to 4.1 Pro.
3 days ago [-]
throwfaraway135 3 days ago [-]
Another big win for opensource AI.
BTW who would have thought China would be the champion of freedom and opensource in the AI arena, while US is fighting for corporate/government control.
Dah00n 3 days ago [-]
Well, at least mostly everyone outside the US.
ReptileMan 3 days ago [-]
Anyone that has observed China since the 90s.
There are different types of freedom and China's bet is that as long as they provide enough economic freedom the public will be ok with somewhat restricted social and political one.
InsideOutSanta 3 days ago [-]
Yeah, I had a somewhat odd conversation with a Chinese acquaintance last year when she got to talking about how America sees China. Her point was that, "How am I not free? I can do whatever I want, go where I want, buy what I want. I can talk to you and say whatever I want; nobody is here listening to me."
The fact that she can't call Xi "Winnie the Pooh" on Douyin wasn't something that occurred to her, or maybe that she was aware of at all. I didn't ask. I don't like to get into these discussions when I'm in China.
computerex 3 days ago [-]
There is no such thing as “free speech” anywhere. It’s all a spectrum and the Americans in particular are indoctrinated. It’s a red herring to distract from the real social issues. People in America have some of the worst healthcare in the world. But hey at least we have “free speech”.
foobar10000 3 days ago [-]
Fixing healthcare or economic policies is typically an act of congress type of thing - not even an amendment. Large popular support - and it can be fixed. Fixing freedom of speech or the ability to vote at all - once it is gone - usually takes a price payed in blood. Sometimes rivers of it.
Citizens of Palm Beach county cannot voice their opposition to how their tax dollars are being spent. They rightly want that money to be invested locally, instead the County uses it to invest $1B in Israeli bonds, something that people widely oppose.
They first said that it's a budget meeting and Israeli bonds are a "policy matter". Then they started arresting people for speaking out against it. Then they started arresting anyone who applauded.
"Free speech" -> I spit on American "free speech". It's the most insidious and dishonest thing. At least the CCP doesn't PRETEND to give people rights that they simply don't have.
dzonga 2 days ago [-]
wish hn allowed reposts just to reiterate this point.
the U.S allows "free speech" if it's inconsequential.
InsideOutSanta 3 days ago [-]
Socialized healthcare does have large popular support, though. Lots of things have large popular support in the US and still don't get done.
gWPVhyxPHqvk 3 days ago [-]
I can't call AI "AI" without being termed an "enemy" by the President.
LYK-love 2 days ago [-]
> The fact that she can't call Xi "Winnie the Pooh" on Douyin
Actually she can do this. There's plenty of such videos on Douyin ... If you clicked such videos frequently, algo will recommend a lot of them to you.
The worst case of uploading such videos is an aaccount ban (but very rare). So yes the freedom of speech here is fewer than in the U.S., but it's not like you'll be put into jail if you say sth about the president.
ngl999 2 days ago [-]
How do you even compare the amount or level of freedom of speech? One either have it or not have it! Stop pretending.
ngl999 3 days ago [-]
That economic freedom is a fantasy fabricated so that the king can rob people fortune created by the themselves in the name of greater good.
tsss 1 days ago [-]
China is only doing this openly because it benefits them to price dump the US.
ngl999 3 days ago [-]
That's an image China wants the rest of the world to believe. Towards inside it still fits the typical stereotype.
dw_arthur 3 days ago [-]
I am a huge DeepSeek fan, but it cannot come close to Opus 5.5 on UI design. A $20 Claude subscription that I use some months and cancel the others along with Deepseek API is great.
prantosmss 2 days ago [-]
Tried it and it works surprisingly good.
yipinwong 3 days ago [-]
I haven't put this into test as much yet.
As a Luna6 proponent, this has a potential as it looks slightly more intelligent and at the similar price point.
sligbad 3 days ago [-]
Once read, I think from G Huntley, "pick a model and stick with it," they have been making that very easy, daily driver since V4.
nateweber 2 days ago [-]
I am very happy with what DS has been able to provide me in terms of output. Also being off peak hours saves a lot on the cost side of things. I would recommend it to anyone that is dabbling with an idea and intends to use AI because DS will produce something that can provide a solid proof of concept.
dt3ft 2 days ago [-]
Am I misreading or is this cheaper and better at software engineering than Opus 5.5?
floooof 3 days ago [-]
they must've been using this since 2-3 days when it became free again in OpenCode, it's noticeable smarter, beats GLM
nickludlam 3 days ago [-]
This and (the not quite yet open) Kimi K2.8 are my main coding engines, and I find them incredibly cost efficient. I find Deepseek in particular is really good with Svelte work.
Right now I use each of them in their own respective coding harnesses, deepseek-harness and kimi-code, but I do wonder what they are like inside something like opencode.
tinyhouse 3 days ago [-]
Web search: $0.75 per 1,000 calls
That's way cheaper than openai and other web search providers. I have no doubt that AI demand justifies all the compute money big tech is spending now, but I also think there are so many over valued VC funded companies that are going to crash in the next few years.
weknowbetter 2 days ago [-]
I essentially only use DeepSeek. Almost always the flash variant. I've found that I don't really need frontier models if I'm doing most of the thinking myself and just using the agent to output code quickly.
bashtoni 3 days ago [-]
"a minor update" they say before showing off benchmarks that display a substantial improvement.
dcl 3 days ago [-]
Very keen to try this out tonight. What coding harness do people like to use with these models?
ghboo0927 3 days ago [-]
You can stick with Claude Code if you're already using it. DeepSeek has an Anthropic-compatible endpoint, so you just need to set ANTHROPIC_BASE_URL=https://api.deepseek.com/anthropic and ANTHROPIC_AUTH_TOKEN to your DeepSeek API key.
belluxx 3 days ago [-]
Pi is very good with open weight models (proprietary ones too)
verdverm 3 days ago [-]
Pi if you're a minimalist
OpenCode if you want a good, opinionated setup
netless 2 days ago [-]
DeepSeek Harness works very well, also used Codewhale which is not too shabby
try-working 2 days ago [-]
After using it for 24h in DSH, this is a very good model and much better than the previous version. Cancelling my Codex sub when 4.1 Pro is released, or maybe before then.
abdallah-khalil 3 days ago [-]
I wonder how you would be able to reliably rely on a model like that for agentic work, even for implementation only after careful planning, how are you making sure you are not wasting tokens reviewing and correcting it over and over again ?
qwerpy 2 days ago [-]
Yeah, I have the same question. I'd like to incorporate cheap & fast models like DS into my workflow to reduce my reliance on OpenAI/Anthropic, but after trying several such models and having to have a frontier model review/fix/redo everything multiple times per task, I gave up and simply have the frontier models (Sol/Opus) do everything. All of this fits pretty comfortably in my monthly subscription.
Maybe the equation changes if I exceed my subscription usage, and need to fill the gap with something else. But it seems like a huge waste of your time to go out of your way to use a cheap model if you have the tokens to spare in your subscription.
BrandoDev_it 3 days ago [-]
I can't wait for a smaller moe dinstillation from this excellent model
swingboy 3 days ago [-]
Does DeepSeek provide the inference code along with the weights? I read here recently that for models where only the weights are released, providers have to implement which could be different than what the labs themselves use.
petu 3 days ago [-]
There's basic reference inference code released with the weights, yes.
> providers have to implement which could be different than what the labs themselves use.
I'd imagine (most) providers just take implementation by vLLM/SGLang.
aizk 3 days ago [-]
I use Opus 5.5 and swarms of about 50 deepseek models to reverse engineer old videos games. The scale is roughly 600k loc generated from pure asm, simply by knowing what the old compiler is. Truly staggering stuff.
why_is_my_floor 3 days ago [-]
Registered just to ask you.... 50 instances? Which harness? How? Why? Can you post a video demonstration?
aizk 2 days ago [-]
Caveat: I get a budget of free ai tokens at work, as I work for an open source AI inference provider.
My goal is to reverse engineer an entire gamecube game -- I'm pushing an existing community decomp that was stalled to the finish line, solo, with lots of AI help.
This work was directly inspired by https://decomp.dev/ -- which has just shut down a few days ago because of, ironically... people like me who enjoy decompiling things with AI, so I'm still weighing / grappling with the ethics of how to release my work, it's murky waters at best.
I use Opus 5.5 as the orchestrator, as well pi harness because it's somewhat lightweight. But for the really hard functions I don't use deepseek I use opus managing other opus agents.
I'm working to release something soon as proof that I can do a 100% decompilation job, but the main idea is that when decompiling a gamecube game your sources of truths are individual functions or files, so you can build off that primitive.
Each ai gets a worktree, and a shared giant context file that's basically a record of every experiment that has happened. They're assigned tasks to essentially do as much as they possible can for each individual funciton. After about an hour claude uses a script to merge the work that's actually correct (verified by tooling that checks if the c goes to asm) and then there are usually 3 rounds of reviews by independent ais, because there will always be some slight mistakes. The techniques used as well as the mistakes the models make are recorded in the giant ledger. There's a bunch of random one off scripts such as stuff to clean up, and tools to make sure the agents don't accidentally google stuff / delete stuff / use too much space.
Claude steers the ship via ssh on my strong linux tower at home so it keeps my laptop nice and cool and at this point I just check in from time to time, I am basically a manager, and Claude has it's own workers.
Last point -- I don't know any c besides the basics, I've only been "coding" python / ts for a couple of years, but even then with the extreme strength of models the future is managing agentic swarms, so I'm leaning in as much as I can. I really see this as the future of large scale projects. For some people it's a mockery of decompilation, but, my goal is the finish line, not the journey.
What would be the best local harness to run this? It feels there are 1000s now.
lofaszvanitt 2 days ago [-]
Claude code 2.1.16x (released 1+ year ago or so) is quite ok, dynamic workflows off. Do not use the latest version, they burn a shitload of tokens and are slow.
wolttam 3 days ago [-]
Exactly, there is no ”best.”
abc42 3 days ago [-]
Have security companies considered that these open models could be used as sleeper agents even when they're deployed outside of China? Is that a serious attack vector or more like science fiction?
There are theoretical attacks involving malicious weights. But the risk profile is so outrageously in favor of them not doing it that you'd essentially have to be an anti-China conspiracy theorist to find it plausible, let alone likely.
klntsky 3 days ago [-]
Here's a small benchmark that compares the new checkout with the previous one:
Presumably. The current opencode slug for the model is just 'deepseek-flash' suggesting they will just update that one going forward, not keeping versions in parallel.
opengears 3 days ago [-]
and do we know which version runs on opencode at the moment (on the free and on the paid version), is there a way to find out?
swiftcoder 3 days ago [-]
The only way I'm aware of is to check their twitter feed. They moved to Deepseek v4.1 flash in September
opengears 3 days ago [-]
sure, but which version are they running. they could be running an older revision
kzrdude 3 days ago [-]
Just yesterday there was just one public revision of 4.1, and they listed they were using 'Deepseek v4.1 flash'. I assume they will update their website if they update the model version (and use the version in the name).
swiftcoder 2 days ago [-]
Yes, they've made public announcements (and changed the version string) on every previous Deepseek update
kzrdude 12 hours ago [-]
That announcement arrived 2 hours ago (on x), so now they have.
SublimePeace 3 days ago [-]
[dead]
vintermann 3 days ago [-]
I'd never heard of the BabyVision benchmark they do so well on. Pretty interesting stuff:
The new model is amazing. Had it build a full Angular Material & CDK components showcase via DeepSeek Harness and it easily did the best job of any recent models I've seen.
hmontazeri 3 days ago [-]
My favorite model just got better. Gotta love what deepseek is doing
wren6991 2 days ago [-]
Tried it. Little whale is getting smarter. Or, as DeepSeek might say: "Hmm. Wait! Hmm hmm."
drewbitt 3 days ago [-]
This is a great model. I know we are all looking for the next best thing, but I'd be pretty happy running this for awhile.
objektif 3 days ago [-]
Anyone using DS as CLI coding agent which harness do you recommend? Is there anything as good as CC/Codex?
drewbitt 3 days ago [-]
I really like pi, though it does require some time in setting it up. But it has greatly reduced my token spend. The relevant Deepseek Harness https://www.deepseek.com/en/harness/ I hear is also good and popular.
CharlesW 2 days ago [-]
As a Claude Code main, I've really enjoyed working with DS4F in omp. https://omp.sh/
drewnick 3 days ago [-]
I use deepseek via fireworks using Claude code.
When I type deepseek on my CLI it launches Claude with my fireworks API key. Works very well and feels almost free.
sharktheone 3 days ago [-]
beating K3 and GLM-5.3 is kinda insane
Xavier_da1 2 days ago [-]
Crazy how you can have AI almost doing everything now while you sleep.
clx75 3 days ago [-]
Probably a week ago someone posted a thread with the title "Why isn't the industry freaking out about DeepSeek 4.1 Flash?"
I have been intrigued about Chinese models for a long time, but never tried anything except an early Qwen. My daily driver was Luna, with occasional elevation to Terra or Sol - all on 5.6 as I found the output of 6 jarringly incomprehensible.
I still had 95 USD on my OpenRouter account, so I thought I would give DeepSeek Flash 4.1 a try.
I think now I can understand the person who created that thread.
Here is the setup I settled on, after a lot of refactoring and calibration of the workflow:
I have a "plan" prompt which explains to the agent (Luna) that I want to have a discussion about an idea I have in mind. The agent should use the Socratic method to pull out all the necessary info from me using pointed questions. When the agent feels it has a complete picture, it should elevate to a higher level model (Sol), and ask its review. If Sol has complaints, Luna keeps asking me until the points raised by Sol are clarified. The review rounds with Sol continue until Sol says "ready to implement".
At this point Luna creates four files in a directory I specify:
1. PLAN.md: the implementation plan with all the details that came out from the design discussion
2. PHASES.md: implementation phases
3. DECISIONS.md: design decisions (this serves the same role as ADRs aka. architectural decision records)
4. DIARY.md: append-only log of what happened during implementation
Files 2-4 are written by the implementing agent. Luna only adds headers to explain the purpose of the files.
Once these files are there, I start up a new Pi session, switch to Deepseek 4.1 Flash and then point it to this plan folder.
And then it goes brrrrr... :-)
Here is what I implemented in the past week using this method:
1. A capture/analysis/decompilation pipeline for C64 demos: I ask the agent to start a selected demo through VICE, and say "on" or "off" when I want it to start/stop capture. The capture process uses VICE's builtin trace functionality: the trace log is piped into Clojure where it is saved into an optimized binary format (chunks of trace events). Once I have everything captured, I unleash my analysis pipeline which checks the write patterns of the executed code: what it wrote (VIC, SID, CIA, memory) and when (rasterline, cycle). Then it splits the execution trace into segments by type deduced from the write patterns (decruncher, loader, raster, music) and then decompiles each segment into both traditional ASM files and a s-exp based syntax which roundtrips back into the in-memory representation. Once I have the results of the analysis pipeline, I can tell the agent to analyze any demopart and it can explain how the effects are built down to the tiniest timing detail. As a first test, I asked it to create a clean-room implementation of Fairlight's legendary intro and two days later I had it, with everything in sexp assembly, including the music routine and its data. (The reason it took so long was that this was the first decomp and it provided a lot of feedback to the pipeline development.) Finally I had it write an assembler, linker and packager which creates a PRG from the assembled and linked artifact, then runs it through VICE. Done.
2. A DSP compiler which lets me compile sexp forms of a C-like DSP language into native DSP kernels running on the JVM, JS and WASM. Currently I have about 600 tests which verify that the behavior of the three targets matches that of a reference interpreter written in Clojure. I have a full numeric tower with i32, f32, f64, support for arrays and structs, typical control flow constructs, and a library of DSP building blocks like oscillators, filters, LFOs, delays and a chorus effect. By the time this is ready (end of the weekend?), I will be able to build synthesizers like Synth1, Surge XT or Vital in Clojure/Script, compile the DSP kernels in the browser, choosing between JS or WASM backends, driving the kernels via AudioWorklets and sequencing them from the host language.
There are two power tools at play here:
1. Deepseek Flash 4.1
2. Clojure and ClojureScript
Deepseek is a BEAST. It just goes on and delivers, phase after phase. What it needs: clear instructions - this is covered by the planning process quite well, but I still find it needs some steering from time to time (the right nudget of wisdom dropped into its thought stream at the right moment works wonders)
And Clojure... well. That article PG wrote about LISP (Beating the averages): what I witness now proves all of those claims.
The agent LOVES Clojure. Especially the REPL (not so much the parentheses).
Winning strategy: give it two tools: start_jvm and clojure_eval. Implement everything else it may need - tools to start up HTTP servers, compile ClojureScript, add dependencies, capture a C64 demo, run Playwright via spel, etc. - as lightweight Clojure namespaces which provide APIs to all these functionality. Let these namespaces be thin: the namespace and the functions therein shall have docstrings which tell the agent what the functions do, but the functions themselves shall be lightweight proxies to the real implementations in sub-namespaces. Then point the agent to these top-level API namespaces via your AGENTS.md. Bam, instant discoverability with minimal context usage. It will do EVERYTHING through clojure_eval and the round-trip time will be phenomenal, a perfect match to the raw speed of the agent.
...
Now all I need to figure out is what will I do when all my dreams become manifest in flesh.
Qiu_Zhanxuan 3 days ago [-]
Isn't there a lot of overlap between PLAN, PHASES and DECISION that pollutes the context ? Whereas a simple PLAN.MD to put the harness on track and an ARCHITECTURE as a single source of truth to prevent it go offtrack would suffice and gives it more space for higher level of reasoning ?
clx75 3 days ago [-]
These files came to be after I've seen Deepseek create them for itself to track progress.
Then I turned what Deepseek did into an upfront policy.
This seems to work quite well generally: watch the agent as it works, and think what could be provided to make its life as easy as possible
ilaksh 3 days ago [-]
Have you thoroughly tested a slightly simpler workflow with one less stage but a frontier model doing the initial planning? Luna is amazing but it's a deliberate trade off designed for speed and cost savings. You can also give them a rule about speaking clearly.
clx75 3 days ago [-]
The idea behind using Luna was cost-optimization.
I wrote a Pi extension called "model-escalation" which lets the model request an automatic switch to a smarter model for the next turn if it thinks the problem it is struggling with is too complex or if the user explicitly asks it to do so.
With this extension in hand, I could split the work between Luna and Sol, resulting in a ~95%-5% split. I don't have to switch to Sol when review time comes, Luna does it automatically. Sol tells its remarks, then Luna gets the control back and continues the back&forth (if there are still complaints).
bel8 2 days ago [-]
Doesn't changing model incur in losing cache, which makes things more expensive by itself?
clx75 2 days ago [-]
Yes, this is indeed possible.
rs_rs_rs_rs_rs 3 days ago [-]
Pretty cool, thanks for sharing.
ece 3 days ago [-]
Where are the Unsloth quants for DS-4.1-flash?
3 days ago [-]
vivzkestrel 3 days ago [-]
- how good is it at reversing a game exe file?
npn 3 days ago [-]
So this is the reason for the crash a few days ago.
dgellow 3 days ago [-]
What crash?
npn 3 days ago [-]
Deepseek servers went unavailable for 5 hours.
They compensated that by rewarding the previous week usage as credit though. Pretty nice.
brador 3 days ago [-]
Ai/tech stock crash
sroussey 3 days ago [-]
Why no Jinja-format chat template?
jfvinueza 2 days ago [-]
It's great at coding, it's superfast and it's a technological marvel, but talks just like Opus 5 and it's just as insufferable. A shame. Can see myself switching in some months, but I'm not going back _there_ as long as I can avoid it.
unleaded 2 days ago [-]
What a shame, I feel like this is a curse most of the Chinese labs have. Every other model is pretty good and stands out, then the next one is just a clone of Claude that does better on benchmarks and can oneshot Minecraft 2 or whatever but is just worse to use.
jordemort 3 days ago [-]
oh is “Flash” a cool sounding word again since Adobe abandoned it?
sourcecodeplz 3 days ago [-]
at this point there are 100s of benchmarks.
so AI labs can just pick which to show
tsss 1 days ago [-]
But does it still have the crazy hallucination problems that V4.1-flash has? I would guess so.
casey2 2 days ago [-]
The real moat deepseek has is the tiny KV cache. This makes it basically the only serious choice for large local models. I hope to be proven wrong!
jimmydoe 3 days ago [-]
PSA Deepseek train on your session if you use their endpoint and/or harness.
I'm a communist so I'm fine, but you may not be, so be aware
lilod 3 days ago [-]
PSA OpenAI&Anthropic train on your session if you use their endpoint and/or harness.
I'm an American! so I'm fine, but you may not be, so be aware
wonderxy 6 hours ago [-]
[flagged]
focalis 2 days ago [-]
[flagged]
RohaanXD 2 days ago [-]
[flagged]
flowskill 3 days ago [-]
[flagged]
SublimePeace 3 days ago [-]
So is this the first model that passed China's new export controls introduced yesterday?[1] It goes live November 15, but temporary pre-registration requirements took effect immediately after the announcement, ~20h-ish ago.
Presumably they registered, but that doesn't mean the model passed through export review. It just means that some bureaucrat now has the opportunity to check the product characteristics they reported and decide whether that kind of thing should be allowed for export or not. (Though with the weights out in the open, that's maybe a bit pointless.)
Those benchmarks can't possibly be real, right? Right? <insert padme 4-panel meme here>
logicchains 3 days ago [-]
At this rate the frontier labs will be Anthropic and DeepSeek, with OpenAI struggling to keep up.
A_D_E_P_T 3 days ago [-]
OpenAI's models -- and even Google's Argon -- have particular niches where they shine. For GPT-6/6.1, it's 3D modeling; Astra is hands-down the best at making .step files, Blender files, etc. For Argon, it's natural language writing and everything that hooks up to Google, e.g. search and maps-based itinerary planning.
Anthropic and DeepSeek (+ GLM) are definitely solving coding, though!
They're all quite different tools.
Kurtz79 3 days ago [-]
Artificial Analysis recently released a dedicated 3D benchmark, where Astra lags behind both Opus and Sonnet 5.5 (and yes, open weight models seem to be quite far behind).
For me it's large files in Blender (in some cases >400MB), mostly, where Astra excels. I gave it a look, but I don't know if the benchmark encompasses that sort of thing.
.step files are simpler pieces of straightforward code, so it makes sense that Opus and Sonnet 5.5 do very well...
swiftcoder 3 days ago [-]
its unfortunate they didn't include deepseek in the initial batch of benchmarks
3 days ago [-]
freakynit 3 days ago [-]
I believe these are the last sets of super-high-end models available to public (open or closed).. these companies probably will now start to withhold top-of-the-line models for themselves, governments, and their super-high-paying partners only.
A_D_E_P_T 3 days ago [-]
That hasn't been the way to bet. DeepSeek and the other Chinese firms show no signs of slowing down, so if OAI and Anthropic halt releases and "pace the frontier" they'll just fall behind. (And there's a chance that high-end AI will become fully commoditized, which suits DeepSeek just fine.)
Besides, Fable 5.5 is reportedly releasing within the next few days, probably on Tuesday...
girvo 3 days ago [-]
China just announced tightened export controls on AI models, for what it's worth.
They’ve technically got expedited approval for open weight models I think, but it still needs approval. And frontier level stuff gets extra scrutiny
ReptileMan 3 days ago [-]
We have the weights, we have the papers, nothing preventing development on this foundation.
hgoel 3 days ago [-]
Except compute
ReptileMan 3 days ago [-]
Do you think we are near the theoretical maximums of how much model we cram into 20B?
hgoel 3 days ago [-]
I don't consider myself qualified to make that call, though I believe there's plenty of room for improvement. But I don't mean inference compute, which is fairly accessible for well-off enthusiasts even now. Training is a lot more expensive though.
ReptileMan 2 days ago [-]
Back in the day we had seti@home, prime95 and fold@home. We will make it work.
Azantys 2 days ago [-]
When models are in the hundreds of gigabyte range, there are not enough nodes to train, because most people dont have 100k in AI hardware. Also there is no training method for machines scattered around the world as far as I know.
stymaar 3 days ago [-]
News from a near future:
Here's How Distill APT Used McKinsey's Private Access To Top Anthropic Model To Help The Chinese Keep Up With The Us Top IA Labs Despite Access Restrictions
slopinthebag 3 days ago [-]
idk, sol 6.1 is both cheap and capable. i've been using it exclusively, i don't need to care about usage, and it's really smart.
yes maybe opus 5.5 is stronger and flash is cheaper, but 6.1 strikes a good balance imo.
computerex 3 days ago [-]
OpenAI models don’t come close to competing with deepseek in cost quality ratio. I have a $20 plus plan and I run out of my 5h limit in literal minutes, every tick of the harness is like 10% of my 5h limit.
You can in contrast get far more done with $20 worth of deepseek api spend. No comparison.
slopinthebag 2 days ago [-]
maybe, but sol 6.1 is far more capable than deepseek, and i love deepseek
ReptileMan 3 days ago [-]
i have a feeling that Sol is just Astra low, but with lowest possible priority on the compute.
WASDx 3 days ago [-]
I believe it's Astra quantized. It was briefly named astra-lite.
m00dy 3 days ago [-]
We're in a crisis
leopoldcongo 3 days ago [-]
Oh mince, mon immense meurtre des humains congois dans le fruit
gabriel-uribe 2 days ago [-]
I'm glad the open source frontier is advancing, but man, with Opus 5.5 I feel no need to look elsewhere. Such a good, cost-effective model for full-time work on the $100/mo plan.
First, it's the tok/s and TTFT from providers. Deepseek blazes through work - at quite good quality - and you don't find yourself waiting for your agents to finish.
Second is the openness. Being able to see the CoT and not being randomly blocked for cyber review. You're never wondering if it's stuck or if it will randomly fail. Combined with the great cost value.. My next subscription will probably be Z.ai
Some people would say that you don't need this at all with the frontier llms. But when extending existing software or writing software that fits into an existing ecosystem, I find I still need to offer a lot of guidance. And it's better that the context window isn't flooded with code that was reverted after I looked at it.
Hit the nail on the head. Development methodologies pivoting at the whim of frontier models have been one of the things that have been worrying me as of late.
You're car will have spun off the road and you'll find it harder and harder to control it and keep it on the road because you have less and less visibility into the vehicle, nor will you be able to clearly see out your windscreen to use the car and road metaphor. You'll be driving blind eventually, unless your working on some simple closed toy problems and/or other small systems.
Building large systems means understanding the system. Without understanding the system, you lose control of it.
The LLM can probably help you understand the system to gain/regain some control of it again, but now you're becoming a system specialist rather than someone who's repeating, "fix it" and "make it good" or whatever.
Self-trained people (like me) are particularly susceptible to it and we're heavily represented in the software world. A lot of us either recognise that (or are coached) and invest the time learning the benefits, and limitations of, software architecture. And, importantly, keep reflecting, learning and updating our practise.
I keep going back to the YOLO mode every other year and try out new languages to see if we still need the structure and abstractions. Then VMs came along and all I can say is they're both even more important and now we have an army of virtual "ADHD idiot juniors developers" at our finger tips it's even easier to tell the good and bad abstractions from each other.
In a way, that’s the situation today. You can pay an experienced developer $$$$ to do it right, or you can outsource it and pay $$ but pay extra for more layers of review, have lots of unfixed bugs and more tech debt.
You are proposing that instead we pay the frontier model to clean it up. If it can, I expect it to cost somewhere in the neighborhood of an experienced dev because that’s the alternative. That’s the way the frontier AI companies will extracting the value of the work from you.
But there's two ways this could be true: Frontier models costs rise, or (more likely) experience dev costs fall
That's only true if a single company owns the frontier, otherwise the alternative is "a rival frontier model" and the cost will be the marginal price of the compute + amortized training costs.
As long as the API is private or not dependent on by automated systems and you have good test coverage, you can get away with bad code IMO since you can always refactor. If it looks stupid but it works (and can be refactored/fixed/improved later if needed) it aint stupid.
They already did the bare minimum allocation to both despite engineering screaming at them.
I highly doubt executives are going to accept slower release and sale cycles in order to provide better support.
If anything, I expect this trend to accelerate. I’m imagining the opposite, a plethora of AI slop with increasingly burned-out engineers.
Sales are out of their control.
If there’s a market for quality and you don’t have it, sales are going down no matter what the PHB wants.
If there’s not, watch out for the vibe-coded competitor.
To reach a more optimal solution, significant human steering is reqiured for every tiny decision. It’s the same as how early models needed hand holding on the basics, playing out again in the finer details. Human experts need to inject “eureka moments” from the outside.
And no feedback loop that I’ve tried has been able to conjure an understanding of these finer details that, as far as I can tell, just aren’t represented in the weights.
Modeling the basics of software engineering was easy, since things like writing code that passes some tests is easy to RL. But for anything not easily RL-able, we’ve basically ran out of human generated tokens to distill from.
Without some breakthrough in generalization, I remain unconvinced that we’ll be able to model the finer details of software engineering by looping the existing training corpus back into itself. Garbage in, garbage out.
I have a theory that mess in an LLM-generated codebase sort of grows exponentially. Each hack or sloppy abstraction chips away at the coherence of the project as a whole and makes the next feature harder to do. But perhaps more importantly, it demonstrates to the LLM, which is a probabilistic machine despite its impressive capabilities, that quality of the code is low, and it generates more low-quality hacks.
And eventually you reach knots upon knots that you have to untangle yourself because the LLM becomes too confused by the code it wrote itself. As models get better, the problem seems to be getting better. I'm needing to do this more and more rarely, and the tradeoff is still clearly worth it.
But I've seen Fable 5.1 absolutely lobotomized by someone who fills each chat context with half a million tokens of insipid garbage, and I don't see why this couldn't happen when coding as well. I'm not sure a solution to this general problem that LLMs can simulate idiots really well will be fixed any time soon.
Adding unit tests, increasing modularity, reducing duplication, and the like are all things you can tell the LLM to do explicitly. And it is happy to do it.
And you can have coding standards as part of your design criteria. Set and document these early in the project and audit the code at least enough to see they are being enforced.
Agents know these principles already and they are frequently the problem themselves.
Too strongly-insisting on eg DRY can lead to tortured abstractions shoehorning too much together. Agents need to be able to distinguish between incidentally similar code vs conceptually redundant code.
Excessive modularity, eg. militant single-responsibility principle can lead to abstraction hell that is impossible to reason about at once.
Unit tests are obviously good. Coding agents write a copious amounts of them, often baking you into poorly-chosen designs made without your knowledge.
yes!So it's not just me. In terms of how intelligent they want to be OpenAI is at the top, then Claude. And actually when I was playing with Sol it meant that many tasks it would just try to do something else because it thought I don't know what I want. Even trying to nudge it back in the right direction wouldn't work.
I guess the models just have the cultures of their creatures.
DS is miles better than GLM in this regard. I wanted to add that as PC is planning to go to GLM next.
It would just loop on things and go down rabbit holes more. DS flash has been so consistently good I've kinda settled on it for 90% of my work.
I still use adversarial review for bigger changes where I want to make sure any issues are caught early, but I genuinely feel as tho if no new models ever get made I'd be fine with just using DS forever.
Keeps repeating things I settled as accepted or rejected. But what really gets on my nerves? The language it uses – that aweful, useless typical LLM languages! No AGENTS.md instruction, no per session instruction, no just two-prompts-prior instruction seems to work, hell sometimes instruction regarding that in same prompt doesn't work. GOOD/BAD example along with instruction don't work. That wastes time and zones you out! Not to mention the fatigue of to and fro. DS has been working much better. Hell Gemini 3.1 Pro works so much better than GLM. I don't have extended experience with OpenAI and Anthromic models but the free claide.ai works quite well and better than GLM 5.3 (not even flash). Albeit I must say I rarely use claude.ai for coding, but technical tasks? All the time!
GLM's best response: "No excuse. My fault. You exactly asked me not to do exactly that and so did the AGENTS.md. I failed". Love it. Except that if I press I will get that response every 5 prompts.
I like giving precise instructions,”update function xx to support input yy, add tests”. There is no reason for 10 turns and 5 minutes of waiting.
DS just handles ambiguity and my bursts of detail with much more grace. I'm only using "max effort" and the CoT is reasonable compared to GLM.
This really depends on the provider, it's a very fast model with ours
(I truly despise the code Fable generates and IMO it has fewer IQ points than Astra so it's not really in my toolbox these days.)
It's a bummer that American open-source is so bad though.
Wait... you just spent 3 paragraphs singing the praises of DS... and then you say you're going to subscribe to the one that makes GLM. ?
(I do tend to agree with you, though I tend to favor GLM-5.3-flash over DS 4.1 flash.)
debugging reasoning traces more effectively seems useful to find prompt issues writing custom harnesses even if it runs on stronger model later
Without getting too much into politics; over the past several decades, one country is going around starting wars, regime changes and other interventionism, or initiating economic warfare (Huawei sanctions because they were getting too competitive to US companies; GPU restrictions; tariffs; etc), and that's not one administration: it's US policy over the past few decades.
> Have you ever wondered why Manus founders gave up billions of dollars from Meta and “voluntarily” moved back to Beijing?
This country wouldn't be such a shithole if we were able to bring the wealthy to heel. I suspect most people on earth see China as the more competently governed country. Understandably, this forum tends to select for people who view private investment positively, but this is clearly insufficient by itself to produce functional societies.
boy oh boy, if you think the US doesn't do anything in the shadows. The last 100 years of world geopolictical history would beg to disagree
Only thing is they aren't clear on the SOC2 certification. Am I being naive?
Thank you. I've been shouting this very obvious thing from rooftops
I work in silicon valley and I feel everyone around me is suffering from China-BAD psychosis and dismiss Chinese releases
https://ourworldindata.org/grapher/human-rights-index-vdem
It's kinda like meeting someone on the street who's protesting about animal cruelty in slaughterhouses, slyly handing them a chart showing them the nutrition facts of beef, and then walking off without another word as though you've settled your case.
Like, I get the point you're trying to make, but I hope you're aware it's a non-sequitur in context?
Your hope should have been that amelius was not aware that it's a non-sequitur, because that is correctable. But then they kept posting and removed all doubt.
People like to bring up the Uyghurs immediately, but never really have the same smoke for the US. Also, do we include the prison population in human rights? Do we include access to healthcare? What about systems perpetuating poverty like lack of access to public transportation?
To pre-emptively be clear on my stance or potential bias, I stand by no country in the world. I was born in Iran, and my parents left when I was young to the US because they believed in the American Dream. The irony being that the dominos of why they left were initiated by the US and UK. And long before Trump was elected, but certainly immediately afterwards, I decided to leave the US once I had the means. (Also because of the lack of public transit, the $100k medical bills after an accident, being punched by a homeless person, and having my possessions stolen repeatedly and the cops giving me a :shrug:). I moved to Japan, where I experienced blatant racism for the first time. I moved to Hong Kong (and tbh no real complaints so far), but I am technically subject to the freedom of speech constraints on China.
I have no allegiance to any country or specific people. My allegiance is to all living beings.
And I don't care if they're reputable, I will look at the methodology and make my own conclusion either way. Appeals to authority are not a substitute for critical thinking.
No lessons to take from the US.
PS. https://en.wikipedia.org/wiki/Foreign_interventions_by_the_U...
I know the US thinks only its citizens are human, so "The United States has been involved in hundreds of interventions in foreign countries throughout its history, engaging in nearly 500 military interventions between 1776 and 2026, with half of these operations occurring since 1950 and over 25% occurring in the post-Cold War period” is not probably part of the reputable storytelling
If they actually said something, it would be much less effective.
> We're not talking about a few % difference here.
How may freedom percent is it again? Do you lose 1% or 20% for failing to allow foreign ownership of utilities? How many percent do you gain for stopping Russian misinformation?
I'm not going to discuss it further if you don't have anything substantial to bring to the table.
Right now, you're basically asking people to trust Human Rights Watch because you trust them. But if people don't trust you, why would they trust your word on what sources are legitimized?
Seriously - appeal to authority is a fallacy, unless it's an appeal to an authority that I happen to agree with? It's hard to take you seriously.
edit: wording
https://www.prisonstudies.org/world-prison-data/highest-lowe...
https://www.theguardian.com/us-news/2022/sep/27/slavery-loop...
In Arkansas, agricultural work accounted for 17% of all prison labor assignments in 2021. https://www.prisonlegalnews.org/media/publications/Captive_L...
17% of prison assignments might be 3 assignments for all I know.
I found a number in your document: "...1.2 million people incarcerated in state and federal prisons, over 65 percent work."
That's 780,000.
And there's 163 million working folks in the USA.
Prisoners power .4% of the economy.
Your whole argument about the percentage overall economy is nonsensical because most of the US economy is not actual productive economy. It's ephemeral stuff like service industry, entertainment, and software development. Obviously, you're not going to force prisoners to go on Joe Rogan podcasts. But when it comes actually producing things, 17% of the industry being done by slave labor is obviously very significant.
> oh please, the US economy literally runs on slavery
Also
> 17% of the industry being done by slave labor is obviously very significant.
That's not what the stat says. It says 17% of the prison labor did that job, not that 17% of the labor for that job was prisoners.
Here's the quote for ya:
"...agricultural work constitutes a greater percentage of work assignments, such as in Arkansas, where 17 percent of job assignments for incarcerated people were in agricultural work in 2021."
But tell me again how I'm nonsensical.
Yes, people have put thought into this. This is why I trust institutions and not random posters on the internet.
Just look at Russia - it's score is highest in the 90s - the most corrupt and destitute time in its modern history, also happens to be when it was deemed somewhat friendly to US and friends.
[0] https://openrouter.ai/deepseek/deepseek-v4.1-flash
(The badges next to companies indicate their privacy terms, i.e. green/yellow/red)
The "green" badges explicitly say the opposite:
> "Private: this provider does not train on prompts and does not retain prompt data."
I.e. for the example of Fireworks, this is the precise language they use to describe ZDR:
> "More technically: prompt and generation data exist only in volatile memory for the duration of the request. If prompt caching is active, some prompt data (and associated KV caches) can be stored in volatile memory for several minutes. In either case, prompt and generation data are not logged into any persistent storage."
https://docs.fireworks.ai/guides/security_compliance/data_ha...
Plus only one country has threatened to annex mine and it wasn't China.
The economic sovereignty of any nation, but especially smaller ones like canada and denmark, is harmed by sending our money out of the country when there are options available in country.
I respectfully disagree. I subscribe to the comparative advantage economic principle, so I'm fine with international trade.
The 'clean power' would factor in a well, but I haven't really looked in to it. I know China is installing a ton of renewable and has a carbon market while Canada's building a new pipeline and building Alberta data centres running on natural gas, so I'm not willing to take on faith that switching is cleaner.
Also, re: power. I said canada has lots of cheap power, clean was your word.
The unfortunate truth behind at least the solar PV boom in China is that it's driven in large part by the exploitation of Uyghurs in the Xinjiang region. One of the reasons they can sustain the pace they're going at is because while nowadays PV modules exports increasingly gravitate towards offshore fabs (Vietnam, Malaysia), domestically all the biggest PV players make full use of modern slavery within Xinjiang's re-education camps.
more on the topic: https://www.shu.ac.uk/-/media/home/research/helena-kennedy-c...
It might be a minor nitpick, but that part is just not how the macroeconomic principle works. Abundance of a good/service within a country is irrelevant to the concept of a comparative advantage - this is counter-intuitive but but that was David Ricardo's entire point. It's not about whether production is more efficient domestically, it's whether comparative allocation of resources between multiple countries is more efficient by specialization within the different goods.
Even if you can produce "good 1" cheaper in Country A vs Country B, if Country A can instead produce "good 2" that Country B cannot, it's better for Country A to produce "good 2" domestically and import "good 1".
Now, I still agree with the original notion. It's unnecessarily risky to use services from "democratically-challenged" nations (especially imperialist ones), when safer options are available. But there's no need to twist macro 101 concepts to justify that view. Modern trade policy takes more into account than basic market efficiency.
As someone who's in the market for Canadian inference (data sovereignty reasons), the pickings are slim for DSF 4.1 man.
(But I'd concede that in the field of genocide China doesn't come up as bad)
https://www.edenai.co/
e.g. https://www.edenai.co/models/deepseek-v4-1-flash
I feel like I’m getting good value out of my subscription and they have the cheapest GLM-5.3 API rates I could find in Europe. I moved to them recently from Mistral for the better prices.
They do have a slightly different daily inference capacity allowance, but it’s actually not too bad once used to it. I was forced to up my game and figure out what GLM-5.3 Flash and Deepseek-4.1 Flash could actually do for me, rather than just throwing everything at GLM-5.3. Quite happily chugging along with a nice balance across them now.
I am hoping Melious make the new DS 4.1 Flash 1017 available soonest.
[1] https://melious.ai
It doesn’t have to be better, it just had to be nearly equivalent. I’ve been doing a lot of vulnerability work lately (trying to get ahead of the bots finding exploits in our code) and the frontier models feel like a chaperone at a church dance: get too close and they show up with the ruler and enforce the safety margin. Using deepseek and glm and qwen, I’ve gotten nearly equivalent results without the constraints. I still use frontier models for things but outside of a company environment, I’m happy to spend 1/10 the money, get the same results, and not have try and trick the AI into working.
American folks really have a distorted sense of reality and few have their own mind, they listen to influencer noise and once self thinkers pick up the quality of things like DS, they will echo the statements.
Dont assume that is how your model decides...
https://arxiv.org/abs/2305.04388
https://arxiv.org/abs/2307.13702
https://arxiv.org/abs/2503.08679
I can't tell if this is a really human thing to do, or if it's just a sycophantic thing to do when your master likes a good story.
Both OpenAI and Anthropic hides so much of that, that you don't really know if it misunderstood something or you got something wrong, until all the turns are complete or you directly see tool calls that implicitly tells you something is wrong.
You don't need to read any papers to see this effect concretely when developing together with LLMs.
I love COT and gives me a good idea about the thought process. Sure it doesn’t work always
Is DeepSeek: DeepSeek v4.1 Flash 1017 any better?
[1] - https://www.youtube.com/watch?v=P4dTq4X8bqk
please don't subject yourself to this. z.ai subscriptions are not a good value. just pay API rates, or use a subscription like opencode go.
Also z.ai doesn't offer Deepseek models.
If so OpenCode Go looks good and includes DeepSeek models (yet to try myself).
DeepSeek feels like an excellent sparring partner. The others feel like mind readers.
I don't have to worry about any complex context engineering / multi agent stuff if I am using DeepSeek because the sequential stuff is fast enough. I don't have to bother about touching a million context length because its cheap enough.
Where it seems to lag behind is 3d, computer use, motion graphics, design but the initial demos seem like the vision model has improved substantially.
I usually default to ds, and if it gets stuck try mimo but usually just for fire and forget tasks. After a bit I come back and see what it did.
Just to test, will GLM – hosted by an infra provider other than Z.AI – also stop answering if I asked controversial questions from the pov of its country of origin? Or third party infra providers can perform some unhinging on the model before hosting? Or actually the blocking happens even before the question reaches the LLMs in that country?
(Note: my paid LLM access at this point are only via DS and GLM subs - 90-95% usage; and a free Gemini Pro access. I hope that adds some cred that I am not trying to take potshots at the their country of origin).
It is very capable for a lot of tasks, not only coding like the other open models.
No, I don't care about license provenance for all the components of a 30 line script that i'm using in my house, for myself to sync up some folders. No, i'm not going to find and archive the licenses for all the previous projects leading to this for future legal disclosure.
Yes, I own the DVD for the movie i'm transcoding. No I don't want to talk about the legality of rebroadcast.
Deepseek (and most chinese models as a whole) don't care . oAI and Anthropic models want to make it point #1.
... which is extra hilarious knowing the laws broken to create the damn models in the first place.
there's a literal meme about this floating around the harness/AI chats for the past few months[0]
[0]: https://i.ibb.co/SwH4Sgwq/796779070-28483090404651613-920066...
I had a bad experience with MiMo from Xiaomi. I had it trigger unsafe content blocks while fixing visual bugs in a completely vanilla space game it just wrote from scratch in the same session. It also got triggered for changes in some other benign greenfield app.
I always find DS or Qwen to be very spot on on audit changes or plans for just a few cents.
It reduces the speed and tends to overthink sometimes which is really annoying. For programming, 4.1/high is the bread and butter in my daily work.
It's only for long running/very difficult cases that the differences are here, and they are small.
Wait wut? You do realize that they are two different companies, right?
And I am not sure why you are so worked up over it -- even meta-commentary wise, you have been shitting on everyone in this thread and your comments are in general very inflammatory. Stop that.
The Fable is the worst offender, and it is not purely Anthropic's fault.
Anthropic, whether Opus 5.5 or Fable, frequently will review for security, and the classifiers fire mid-way while it's fixing something. Authentication and credential related code, including implementation, also trigger safeguards.
Earlier today, I hit what appears to be a sticky "reasoning_extraction" block when attempting to reproduce a LLM paper at small scale, and I asked it ~"tell me more about the idea [X] you considered but dropped". I'm not sure how it works or how long it lasts, but Opus 5.5 has refused to do technical LLM work for me now, even in completely new sessions.
Long-term & software engineering fronts: I hear occasional railing against “US hegemony”. Anyone concerned that if a new tech gives rise to a different country, they’ll become the source of ire instead? (This isn’t my field, at all.)
So I go with feeling, and that feeling says that China is worse. The feeling is partially supported by the fact that China is softly backing Russia in its attack against the west.
Trump's USA is obviously completely horrible, but he is also in large part a product of Russian influence, which again is part of the larger block in which China resides.
And China is not really supporting Russia, they make money off the war and want to keep it going as long as possible so both Russia and Ukraine become weak and more open to their influence. Is it evil? Absolutely, but you can't compare them to USA when Trump is actively doing regime change operations, deporting its own citizens and putting sanctions on the ICC.
Chinese communists want to stay in power at all costs but America wants to expand its power into the whole world. It's not the same level of danger at all.
note that the volume of EU and US support to Ukraine is obviously calibrated to the same goal.
Yes, yes they are.
https://en.wikipedia.org/wiki/China_and_the_Russo-Ukrainian_...
Trump’s a smelly orange shit but the US at least pretends to have data privacy protections and we can’t legally compel companies to have back doors… yet. I think. It’s very hard to keep up with all the nonsense these days.
In a post-Snowden world where the existence of PRISM is no longer a secret, I'm not sure how folks can keep stating this with a straight face
1. They, and many other experts, have been saying the same thing since well before the current paradigm existed. 2. There is nothing about regulation that is required to give them an advantage; it’s a fallacy to suggest “regulation = advantage” 3. It would be preferable to argue the facts (is it or is it not dangerous? Why or why not?) than conspire as to why they’re stating what they and plenty of others have been saying for decades
You're saying regulatory capture gives you no benefit by just refuting an absolute with 0 context. There's nothing to even debate here. Ok... most people disagree?
Yeah I agree with point 3... Stop engaging in vageries and say something of substance with examples, so people can actually have a conversation with you without us doing all the legwork.
> 1. They, and many other experts, have been saying the same thing since well before the current paradigm existed.
Some experts sure. "Some experts" say lots of things.
However, this is actually much to their advantage. If you were a frontier AI company, a legitimate concern to use as justification for your regulatory capture is exactly what you want. That is, after all, generally how regulatory capture gains public approval.
AI safety was also why OpenAI had to charge per-token for GPT-3. It must've been such a struggle for them to make that difficult decision. I'm sure Sam Altman spent a lot of nights crying to sleep over that one.
It's good that we never got open weights for a GPT-3 class model, right?
> 2. There is nothing about regulation that is required to give them an advantage; it’s a fallacy to suggest “regulation = advantage”
It's difficult to outline the number of different ways that regulatory capture can and will benefit OpenAI and Anthropic.
If we limit things to "Fable"-class models, then nobody competing with Anthropic/OpenAI is ever allowed to outdo them. If you literally legally aren't allowed to make better models, you can never charge the "premium" of having the current top-tier model. You have to compete by being cheaper than Anthropic instead, which is a lot less enticing from a standpoint of trying to become profitable. Take a look at the per-token pricing of Fable real quick. Yeah. Being able to charge a premium for the model that does the cybersecurity thing is literally their most important asset.
If Anthropic really is making a profit now as rumored, then they need regulatory capture as fast as they can possibly get it. Once Anthropic and OpenAI prove that AI is a viable investment, entities with a lot of capital will suddenly become even more interested in re-allocating it towards taking a slice of this pie.
> 3. It would be preferable to argue the facts (is it or is it not dangerous? Why or why not?) than conspire as to why they’re stating what they and plenty of others have been saying for decades
OK. On the factual end of things, AI safety experts have been raising the alarms for over a decade and so far essentially none of their predictions have come true.
The rationalists, the main group pushing the AI safety rhetoric that dominates the Western AI sphere and is virtually absent in China, which appear to be going entirely off the deep end at this point in time, put together their best minds to produce AI 2027, which of course like most predictions started to fall apart a bit before hitting the one year mark. Predicting the future is hard.
I have made this point before but I will make it again: I think AI safety experts deserve at least as much skepticism as Ed Zitron in their claims. They've been making similar dubious claims about AI capabilities and an incoming singularity for a long time now, and so far the best we can say is that maybe it will eventually come true.
Maybe. Or maybe you've read too much goddamn science fiction.
They seriously underestimate how ready and willing Americans are to use Chinese providers. There’s a ton of spite
Last time China pulled this shit it shook portfolios for a week
I get this is Hacker News so we have to hate AI safety, but you should read more on this topic rather than blithely casting the x-risk viewpoint as fringe. The last hard data we have on the question is a late-2024 AI Impacts survey [1] of 1,580 authors at top AI conferences, who were offered $50 for answering a survey on the topic of the future of AI progress. The median respondent assigned a 5–10% chance (!) to "extremely bad (e.g. human extinction)" outcomes.
> On the factual end of things, AI safety experts have been raising the alarms for over a decade and so far essentially none of their predictions have come true.
Devastating. When I wrote my AI alignment PhD thesis [2] in 2022, and predicted a 70% chance existential risk by 2072 (which I later came down to about 1/3 on), you're right that that prediction hasn't come true yet.
While I agree many specific worries didn't materialize (like AI being like a "literal genie" it's hard to specify goals to), many key worries have (Hugging Face rogue AI swarm scans as cribbed from rationalist AI scenario writing).
[1] https://aiimpacts.org/wp-content/uploads/2026/09/ESPAI2024.p...
[2] https://arxiv.org/abs/2206.11831, top of page 4
1. All those who have been arguing this - many of them non-technical - are part of the same ideological movement.
2. Technically, there isn’t. In practice, majority of the time regulations benefitted those who wrote them.
3. Not sure if this is a criticism of the comment you are replying to or of the p-doom crowd. We would all appreciate facts stated instead of science fiction tropes repeated.
Look, “a lot of people say this” is a Trumpian fallacy. A lot of people say a lot of crazy things. It says nothing about the factuality about what they say. How many are this lot? Who are they? What do they know? Why are they incapable of expressing it without so much emotion? How come a big portion are non-technical?
In fact, their core ideology actually stemmed from Yudkowsky's arguments.
1. He's made many posts about how he thinks "dangerous" models are bad (https://www.anthropic.com/news/position-open-weights-models)
2. He writes posts proving Chinese open weight models are "dangerous" (https://insightsintegration.com/anthropic-warns-of-advanced-...)
3. He talks about about how he wants stronger govt regulation of AI (https://www-cdn.abcnews.com/Business/exclusive-anthropic-ceo...), and explicitly mentions he does not trust China at all and doesn't want them to have Mythos.
Do the math: 1 + 2 + 3 = he wants to regulate/ban the Chinese open weights.
Of course, Anthropic and OpenAI will then write posts about how their own models are dangerous - but that's okay, they're the good guys.
When HuggingFace was in the midst of defending itself from OpenAI's automated hacking spree, they were blocked from using frontier models by Anthropic and OpenAI for "safety" reasons. HF had to resort to an open model to defend itself - and DeepSeek distinguished itself in that role: the second fact from that episode OpenAI would have you soon forget.
There are no choices here which do not carry risk. I haven't seen a single convincing argument against these points.
We also put responsibility on humans to act responsibly. In theory, it's absolutely insane to allow human beings the ability to walk into a street with automobiles, because obviously they will be killed! But we don't ban them from walking there, and we don't ban cars, because it's pretty uncommon for people to walk directly in front of a moving car, despite there being obvious danger.
Ultimately the biggest risk of AI isn't the AI. It's what human beings do using the AI. Just like with hammers, knives, drills, chemicals, cars... we regulate the human's actions and create laws to direct their use, rather than regulate the items themselves.
He's completely transparently campaigning for regulation. It must be very frustrating to him that the Trump admin is completely non-responsive. He probably thought Anthropic would have the whole market stitched up and water tight by now.
[0] https://www.anthropic.com/legal/aup
OpenAI and Anthropic aren’t even slightly worried about DeepSeek.
May this kind of worry find me…
It's quite hard to have the valuation of Anthropic if they are limited to US.
> Literally zero, but that never stopped a good conspiracy theory
Are you living under a rock and never read any of Dario's blog posts?
Let me give you an example, here: https://darioamodei.com/post/policy-on-the-ai-exponential
Section "1. *Regulation* and public safety" (emphasis mine), then under it:
> AI companies that develop advanced AI models must have strong security standards that protect their model weights
You can't "protect model weights" if the weights are openly available, ergo this is an explicit call for regulation to make open weights illegal.
Presumably you'd want the AI companies to protect their weights to keep them from falling into the hands of bad actors. But the US can't make open weights illegal anywhere except the US, which does very little to keep them out of the hands of bad actors (most of whom are outside of the US).
The model makers are bricking the models with their system prompts, and even the tools are tamper proof.
You can only use it their way. They think we’re stupid.
And their way is the shit way, the babyproofed way, while they talk to you about their unfucked models hacking out of their labs and hacking into other labs, and reminding you you have zero rights.
Fuck them twice, then once more… This is about compute. Who owns compute. China is saying fuck you all we can scale things bigger than you can imagine. And they can! They have so much more energy! Theyre closer to commercial fusion than us too. Meanwhile our guys are trying to… regulate? Lol
Training and serving LLM's is massively parallel. Your bottleneck is either in energy or chips. China will catch up on chips fully decades before we catch up on energy. Arguably, this is already happening: here in the USA, we have lots of chips, but we are seeing our electricity bills skyrocket as more datacenters come online. We are powering those datacenters with natural gas turbines, not solar, wind and nuclear. Right now, to purchase one of those, you are on a 5-year waitlist. There's your timeline!
China is invested, from a regulatory, cultural, workforce, strategic, and technological standpoint, in an energy abundant future. We are not invested -- and in fact, actively sabotaging ourselves -- on all of those fronts.
Once they acquire chips, we lose, basically permanently. Look at what Huawei is doing recently and what is on their roadmap to understand how fast this catching-up is taking place. I'm sure that David Sacks would say that our innovation motor is pulling their copycat caboose along, as our monopolist tech companies steam ahead towards the frontier. I am not so sure. They are already ahead of us in many technologies. Speaking of technologies, they're building natural gas turbines in house, starting in 2025.
They are showing this new version scoring significantly higher across multiple coding benchmarks, beating Sol 5.6 (6.1 is the latest) across 10/11 benchmarks.
I am going back to DeepSeek for writing code. I was trying out using Sol and liking the result, but this should close a lot of that gap. I use Opus/Sol for an implementation plan and code review also use them for grilling. DeepSeek is also running verifications.
i let 4 different agents review the same dirty code base (200+ files changed, c++, game engine):
- deepseek v4.1 flash via deepseek API: took 20m to find one p1 issue that was relevant and a p2 issue that was irrelevant
- glm 5.3 flash NVFP4 via TP2 DGX Sparks: took 1h 5m to find a p3 documentation issue
- gpt 5.6 sol: took 5m30s for one p1 and one p2 issue. both relevant
- gpt 6.1 sol: took 4m30s for two p1 and one p2 issue. both relevant.
Worth to mention: none of those findings overlapped.
I let deepseek v4.1 flash and gpt 6.1 sol both plan fixes for the summary of the findings. in this particular scenario: gpt 6.1 sol planned to increase a buffer size from 128 to 256 to fit the 191 elements that blew past a test scenario. it included some headroom. deepseek v4.1 flash on the other hand decided to make it exactly 191 to satisfy the test, for example.
The TPS decode on deepseek is impressive. I never had a second / feeling of being left aone and though "damn this thing is working". Despite all my efforts, it took ~4x longer in a review, found less and planned worse than GPT 6.1 Sol. Wall clock time. I don't have token counts for comparison. If I were to grade those models from 1-10 (10 = top tier GG), I'd give deepseek a solid 6.5 and 6.1 sol a 7.5-8.0 maybe.
New model comes out and someone says, "it can (or can't) draw ducks on bikes" or some esoteric thing. But it's not enough statistical power to claim anything other than a personal anecdote.
I got a flat tire in downtown today. Therefore, driving in downtown causes flat tires.
For another, the dependence on a random seed is just one example of how you can get different behavior. You could just as well get different results due to e.g. a tiny difference in prompt wording or punctuation or whitespace.
If a model passes some test on 70% of random seeds, and fails on the other 30%, then knowing the rough probability of success is much more useful than knowing one random sample of the outcome.
And empirically this kind of probabilistic success is quite common, because many LLM benchmarks report "pass@k" success rates that are much higher than the "pass@1" success rate on a single attempt.
Tiny variations in the computational environment, such as slightly different set of processes running on the system as well as tiny differences in core temperature or voltage - among many other factors, can cause computations to complete in non deterministic order relative to each other.
This meaning one run eventually calculates a+b+c, whereas another does a+c+b, yielding slightly different results. Those cascade down the chain, resulting, finally, in a different set of produced logits.
Notably, similar effects can and do affect pure CPU calculations on multi core machines, and as a matter of fact even on a single core depending on the architecture when considering eg pipelining, branch prediction etc.
Thus, real world matrix operations are not, in fact, deterministic.
Now, one could argue that all of these are negligible, but that is an empirical claim dependent on the problem at hand.
This is not true. Non determinism only arises via sampling, and to a much lesser extent, CUDA kernel launch subtleties.
But try it on high instead of max if possible. max is known for being overly verbose to the point of being detrimental compared to high at times.
For example Simmons svg pelican took 11 minutes on max and 1 minute on high. And they are comparable.
I recall the difference being minimal in 4.0.
This is why using multiple model families is important. Don't limit yourself by being a stan for this or that company.
today we can use some anecdata based on our observations, this also helps shape the dimensions we want to eval the agents-tasks on
I too am thinking about creating some benchmarks that I can understand myself. The LLM 'cheating' by finding known issues through some sort of history or online source is a path you want to protect against. Then again, you DO want them to be able to look online for people having similar issues.
Were you using the same setup to call these models? was it pi/opencode or was it codex/claude etc etc.
Because I've started to see larger and larger drifts in the results depending on the agent it self rather then the model.
This read like a conversation you were having in your head -- unfiltered.
The lower the number the higher the priority, e.g. if production is broken someone might say "This is p -1" (higher than your highest priority).
P0 is "do this immediately", P1 is very important, P2/P3 are like "nice to haves." P100 would mean something like "we will never do this"
At 3am,
P0 = get people out of bed and fix this now.
P1 = we can fix this in the morning.
P2 = we can fix this before CoB today.
P3 = push it into the current sprint.
P4 = put it in backlog.
Yet to be obsolete.
- For some reason choosing only a subset of issues.
- Different LLMs actually learn different things.
I suspect the former since they do this for me and it's extremely annoying sometimes
Deepseep-v4.1-Flash-1017 is a minor update. A major update is something like Deepseek-v5-Flash.
FWIW, GP didn't write "a bad copy" but "bad copy", referring to the marketing sentence quoted by OP, rather than the model that was just released (as I presume "a bad copy" sounds, maybe like it's called that because it's [again, presumably] distilled?). "Copy", especially in a marketing context, is a written message.
This is just my headcanon though.
An example when using a less frequent term actually reduces specificity.
Also, DS is an excellent model.
These are not mutually exclusive.
The CCP requires extensive certification for new major releases, to make sure a model doesn't do anything they don't want it to.
To get around this, Chinese labs only bump versions when they have to and usually claim it's a minor update.
Then on the other side, they're obviously trying to market to the tech crowd...
Western media doesn't have correct informtion source about China. This is because Western favors Web ecosystem and most inforamtion live in the Web, but in China everyone use Apps.
Would highly recommend it, it's IMO the only serious Western reporting about China.
as good as Gordon Chang.
The problem here is quite often brainwashed people never realize that they have been brainwashed. Sometimes they trust, promote or even defend the perpetrators. You have my deep sympathy on this matter.
There is no such thing as an unpoliticized publication, but one can still read, taking into account their biases. The point is to extract the information while discarding the ideological interpretation. In fact, this exercise is often easier when the ideological tilt is as obvious as the capitalist liberals from the Economist.
Forget about their predictions or their moralizing, but they have actual Chinese reporters talking to actual Chinese people on the ground instead of doing the usual Western thing of white MBAs treating China like some alien civilizational threat.
There's something to be said for the speed with which it gets word done - I take fewer "it's compiling" breaks, and lose my train of thought less frequently. What's crazy is that the speedy work it does is pretty much always correct - which is a reason why I don't feel the need to spin up claude-code and "ask the expert".
This speed & good-work combo is the DS signature IMO
https://github.com/computerex/dlgo
If you want your 5h usage to runout in one call feel free to use OpenAI
Now we have: a ton of competition in different inference providers, very low costs, and speeds reaching 1k tok/s in certain providers.
We can just get a box of compute even in Germany, and run the exact same model here without anybody stopping us.
This is why "tokens per second" alone is a deeply flawed comparator.
This is also why discussions of Chinese models focus on tokens per second- they struggle to compete on intelligence so they dramatically increase effort levels to compensate, meaning they require massive total output tokens.
For example, flash 1017 isn't benchmarked yet, but 0731 required 15.4M tokens to complete the artificial analysis benchmark, and Sol 6.1 high required just 2.8M.
Sol 6.1 uses 2.8M at 64 tok/s = 12h 9m
Flash 0731 reported 149 token/s at 15.4M needed = 28h 42m
And in my example in the first line, scenario A (30 t/s) completes 6 minutes faster than scenario B, despite 10X slower inference.
The best way to answer this question: [Pareto frontier or cost/per task curve](https://api-docs.deepseek.com/img/v4.1_261017_perf_cost_en.p...)
img from https://api-docs.deepseek.com/news/news261017/
Sol 6.1 High: 50 intelligence, $0.32 per task, 4.4 min per task, 13k output tokens per task
Sol 6.1 Medium: 48 intelligence, $0.21 per task, 2.9 min per task, 8k output tokens per task
Deepseek 4.1 1017: 45 intelligence, $0.26 per task, 4.8 min per task, 89k output tokens per task
If you look at the actual pareto frontier across the full benchmark, Sol 6.1 remains firmly on the pareto line, while deepseek remains firmly under it, just as it was before https://i.imgur.com/N72p5NF.png
This result is exactly as I expected, and seems to run counter to the effusive praise of this thread. Sol 6.1 Medium is producing higher quality output at a lower price with nearly 10X less output tokens! And this is not surprising.
And finally, Mimo V2.6 Pro continues to look like a stunning release, clearly smarter, cheaper and faster than deepseek v4.1 1017
Much like Qwen 3.8 Next; which is qwen4-exp in a lot of the code/patches, there's usually a tick-tock cycle with LLMs. That said, it's hard to know for closed frontier models.
DeepSeek delivery deliberately uses confusing version numbers.
- Reasoning effort = 1 is now actually usable and produces 2.5x less tokens than old weights, without intelligence lost.
- Overall 20% less tokens at each reasoning effort.
- Same intelligence and same tool-calling pattern.
More details in blog post https://sference.com/blog/deepseek-v41f-1017-benchmark
It's a bit like obsessing over patch releases of some software after a major upgrade already solved the original problem long ago.
And then when I try a smart model, like Astra medium and it solves it (I had a firmware bug in my new laptop where I couldn't charge through USB-C) on the first try (altough with a repo already having lots of information about the hardware/firmware/debugging programs/traces), I realize how much time I'm losing on the few examples that the less intelligent models are just not able to debug and solve.
I was really hoping this would be solved since openai said that gpt-6 could paint brushstroke by brushstroke but it’s still shit
Is your laptop something like framework which has open source firmware?
I(and many other people) have a similar issue with usb-c charging not working consistently. I've been meaning to try one of these new smart models to help me debug the issue.
I had to reboot my laptop a few times though until I got to this, but it was a fun experience anyways.
At this point open source isn't as much advantage as long as the firmware is not encrypted :)
I'll try handing it off to one of these new frontier models.
Like if we needed PhD intelligence for productivity tools and SaaS platforms, why weren't we hiring PhDs to build things before.
99.99% of people do not need SOTA ai.
https://www.lithosai.com/
https://www.coralbricks.ai/
Outside of major players like Amazon/Google which have compatible offerings available, I believe together.ai, fireworks.ai, and deepinfra.com are, as best as I can tell, US-based (if that matters in your situation) SOC2/HIPPA/ZDR. Each of them offer serverless model hosting with caching, which is going to be the cheapest overall. Most of them also offer private AI deployment which is more expensive but your entirely own instance. Some of them more heavily quantize their models (deepinfra) so take that into your workload considerations. I included deepinfra because they tend to have some the new models sooner than others or have models the others don’t (like mimo). IMO, together > fireworks > deepinfra.
However, you’ll have to keep in mind that for true ZDR you give up caching which increases your inference costs, so it depends on how “Zero” you need the zero to be and on what timeframe. If you just don’t want your data being trained on, normal use is fine imo. If you are working on something actually sensitive (like, because of NDA agreements and such), then they might not be the right choice and you may be looking at more expensive offerings or building a machine.
One note: you have to pay attention to the source of the inference. Many (similar to open router) offer models that are pass-throughs to a providers API, but they are pretty clearly marked. Just make sure the model is marked as serverless when evaluating new ones. This is where a proxy comes in handy so you can prevent your LLM from accessing a different models from the specific ones you’ve vetted.
Depends on your use case. But if you do software development with a harness such as oh-my-pi, your cache rates are probably somewhere above 98%. Coralbricks has free cache reads, meaning it's much cheaper than the Deepseek API rates.
Lithosai is more expensive especially in the ultra tier. It's if you want extremely snappy and fast responses, which is important e.g. for a personal assistant where every second counts.
If you generate a lot of output, you're better off with the Deepseek API rates.
What about "good governance"?
Does it have zero hallucinations? Does it reflect thoroughly on every single idea it uses, verifying its solidity?
How?
This is how one of my claude code session looks like right now, one prompt from yesterday:
I created a skill to better manage agents, including a way to keep the caches valid, and it has decreased my usage considerably. I now struggle to use my 100% each week and I hit claude hard. I have a reset expiring in 5 days and I'm trying to burn down the tokens (68% with 3d 8h left) quickly so I can use my reset before it expires.
Token efficiency just means understanding the way the system works. Best person to help with that... my old friend claude.
However where i think you have a point is that it tends to run tests quite casually where a reviewer would have been enough to catch mistakes.
IMHO, the real problem is the 5min cache way too small. They should up it to 1 hour.
I don't actually know the real mechanism, I came across this serendipitously.
Buid alone takes 4 minutes. Test suite heavily parallelized runs in about 2 to 3 minutes, infrastructure dependencies fully containerized using podman, no external API call.
And this is without mentioning the 5000 frontend tests (angular), including Playwright end-to-end tests wired to the backend (about 3 to 4 minutes run without the e2e tests).
And this is running on a dev machine with 64gb of RAM and an NVMe disk.
Some systems are just massive, and even if you run focused tests, it's easy for a change to have a large blast at least requiring a large rebuild, and at times to rerun a larger set of tests.
Claude: Confirmed: 195M of the 212M cache tokens (92%) come from full re-writes after 5+ minute gaps, accounting for 77% of relative cost. Now I'll check the existing wrappers so the skill builds on them.
Can you invert things. What’s deterministic and what needs “thinking”?
Can you create a script that runs things chaining together the deterministic parts with calling out to Claude for glue that needs “thinking”.
I think the tests problem is really having an impact for me. Hadn't considered it before so thanks for flagging.
Awfully new old friend.
Maybe doing bogus small requests to prevent the cache from expiring?
I mean in the API you can set the cache expiration time right? But I guess claude code can't do this
- How long do you reckon it would take a human to do the task?
- What percentage of features/ feature-extensions you add would have been done at all if you didn't have agentic assistance? Is there a tension between llm productivity gains and 'because we can' feature additions?
- Are many of the tasks for throwaway v1 features, exploratory tries?
- How speced out is the work you give it, eg did this task have pages of specs, how long did the spec take to write?
- How deeply will you manually review the work and how much time will it take? Will you read all the code / review at a high level and interrogate with llm / have the llm review and self iterate?
I do now add all the major features I want, and write a task list where I add a few more minor tasks each week that just come to my mind. I need to write them down before I forget. I often ask Claude to implement several different implementations/algorithms and compare them. There are plenty of trade-offs, which usually end up in building configuration profiles, so the application can work optimally in multiple environments.
The specification is actually really large, and has grown over time as I've been learning and new ideas has come to my mind. I do not review/audit the code, it's massive would take me months, no chance! I do however use code inspection/audit tools like TSAN, ASAN and more. I will also spend time on full code reviews with the coming Fable 5.5 when a release is about ready. I do however review behavior together with Claude, and this is also where the specification grow, because this and that should be possible, but not this and that. User stories also helps.
There's no reason to blow through tokens this way, there's a 99% chance nobody uses the output (except maybe themselves).
What exactly are they doing for you that's giving you $200 of value in such a short timeframe?
That’s the “how” we’re wondering about.
I am not saying OP is not telling the truth. I just really want to understand!
Anyone who tells you they have a “team of subagents” and “code reviewer” agents and “project manager” agents doesn’t know what they are doing. That’s how they burn thousands of dollars in a day.
I think I’ll trust the experience of the people in the labs who have unlimited token budgets and are running experiments continuously over some anonymous rando slinging mud on the internet.
Also my own experience, which says having code reviewer agents is one billion percent useful. That comment alone completely disqualifies you as a reliable source of info.
Maybe you're not being ambitious enough with these things? They are at their most fun when you spec out sprawling architecture documents and watch 1+8 astras buzzsaw through it.
I'm only at 260B for the year, though. I'm sure someone who has used more will be along shortly to chime in
E: The other subthread talking about tests is correct. Expensive models should not waste a single second waiting on tests or validation. If you cannot assign a single cheap model to be the test bitch, then just tell them to frontload everything and clean up later as a single wave.
> Have you shipping anything?
Yes, hundreds of PRs and shipped new features to 3 different platforms (day job and 2 side-business apps/platforms I’ve been developing for many years, pre-LLM). Those new features have gone out to customers and been used.
> Do you have income from this?
Well my day job for one, and I’ve been able to more easily sell to new clients with my side business. Without the changes I’ve shipped I would have been able to land a couple of the clients I’ve signed on recently.
I don’t think OP’s message implied they are never finishing a project, maybe they edited it?
In addition to the 3 income-generating projects, I have a number of other “just for me” projects. Some are mature now and only need occasional development when I want a new feature. Some are in flux as I build them out (like my “software factory”).
At this point I’m limited by compute and tokens more than anything else.
EDIT: but thanks for a useful answer to OP's question. As a programming dabbler I also find it hard to imagine how to push much past the 5hr limit on $20USD subscription.
It’s paid for itself though, at least from my side business income.
How people spend more than 200 usd monthly is beyond my understanding, are you guys so inefficient with the usage? or I am the opposite?
I tear through my $200 subscription because I often have 3-4 agents running simultaneously, exploring performance optimizations, building features, monitoring production rollouts, or investigating production problems/customer issues in parallel.
The downside is that I do spend most of my time reading the agent's output these days.
It is like putting a jet engine on a shipping container and wondering why it needs to be constantly kept in afterburner to at least scrape on the road while smarter users put wheels on the shipping container.
I would be worried about your methodology when the subscription party ends. Make no mistake, one day, the party is ending and the people who are dining out on approaches that burn massive tokens are in for a rude shock.
At the beginning of the month I realized I had more tokens and more capabilities on Opencode at 8$/month via DS 4.1 flash, than I had on a 200$ account in June with the american providers.
Literally both more tokens and capabilities than Opus 4.8/GPT 5.5.
Even if you pay per token, you're still unlikely to outspend american providers.
And southern Europe isn't exactly the poorest region around.
If you can get away with paying that much to a swe in Italy then why aren't Italian tech companies ruling the world on the back of ~free labor?
There’s also zero stickiness to models and most companies now have their own internal routing utilities so devs and users can just keep the same API endpoint and move between models with a single line of code swap.
Unless something changes with the big labs and fast, they look pretty f*cked.
It is more complicated than that when the valuations and business plans are contingent on those companies AI having complete dominance at every level.
If companies only use frontier models for occasional difficult tasks, not only does that affect volume, it basically elminates stickiness and incentivizes companies to instead of using their products continuosly, find ways to move from the expensive stuff to the cheaper stuff.
You’re assuming other model creators can’t build these mega models. Evidence is they’ve simply decided it’s a fools game to do so and they’re focused on where the use cases are.
They made very bad bets on scaling, which has cost ~10B scale in wasted capital allocation, but it is manageable. Their purchase commitments for future compute capacity are absurd nonsense, but all of that is just shtick for retail investors, and they will reshuffle the deck as needed to adjust.
Probably a pretty decent model for 90% of the usecases!
I've been doing a lot of hobby work with GLM 5.3 Flash and I can count the amount of times that I need a stronger model on two hands.
I have access to unlimited tokens at works and Opus is great but also often overkill if you know exactly what you need.
A very complex task tends to cost like 10$ max. It's comparable to a subscription with enormous flexibility.
It's a steal.
Of course I do certain things like use omp and do a compact picture snapshot after 256k tokens automatically. My tasks rarely go over a dollar...
I've ported a few games, wrote a cosmic desktop app, iterated in my own agent software, use the model in our home speaker system, use the model in my phone assistant. My monthly bill to coralbricks is about 14 dollars.
If it's mission-critical code, do a security review with GLM 5.3.
Otherwise go manually through the code and ask for changes if needed.
Works really well for me.
Since making this (obscure/hidden) change I enjoy Codex much more.
I tried Claude legitimately for a few months, on max
That was during the great anthropic compute/gaslight crisis (ahh good times..), Claude then enthusiastically bricked my MacBook and I had to get gpt to unfuck it
I don't think there is a reason to care about any of it, just get a few dollars on a few openrouter or similar platforms and just use the models for now, wait for cheaper models that are good enough for you to become available for your hardware.
Sure you might never see frontier performance on consumer hardware but do you really need that though? Does anyone? And if you ever do just get a 30-day sub for that work, and cancel it.
We are running Qwen 3.8 27B FP8 with 4x 5060 16gb at the office for a small development team. It is doing pretty well for implementing backend and frontend tasks, and is already useful. In 6 and 12 months from how I expect it to get even better. Some planning is still done with a stronger model, but even that is maybe not strictly necessary.
And we are likely to see many companies start building 27-32B MoE models custom built for use cases now that Rubin is going to make training costs much lower.
I expect training costs to get under 100k USD within a year for small models with another 100k maybe for RL.
I think just having all your own data from claude and stuff on your system that you can feed these models and have companies offer great base 27B models will be how it works.
MiMo 2.6 Flash is the most capable model you can currently run on 2x Sparks without resorting to a quantised version of something.
Are they as good as opus 5.5? No.
Are they about on par with opus 4.6? Yes.
The long run tasks i run seem to be about the same level of quality, though there are obvious differences in model styles, token budgets, etc....
The speed and cost/intelligence ratio is just outrageous.
I didn't try the last flash model because I would need to use SSD streaming (I have 128 GB), but the benches for this release are so compelling (and it using less tokens to get tasks done) it might be worth the throughput hit. I will probably give it a try when I have some downtime.
It feels very freeing to be doing useful work on a local model on my own hardware with my own coding harness. The TCO argument is starting to look compelling too.
Benchmark is historical PR reviews. Models are put in a sandbox with a git and db snapshot as it existed at the time. PRs are a selection of changes some bugs (major and minor), some with things that aren’t bugs that have tripped up models in the past and some no findings that they should approve and move along.
4.1 flash already wasn’t that strong at this task, but the speed edged out v4 pro for a spot in the roster.
If I'm impatient I switch to Luna for a single turn just to get it started and let DS continue.
Definitely an improvement on deepseek-v4-flash-0731: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
Looks much better.
Not only did it come in significantly cheaper even than Haiku, but it was more correct than Sonnet.
This is great! I am gonna get coding!!!
It works when it works, but will probably be flaky AF for a few days.
Works well with Copilot:
700000 because there's no detail yet on the safe completion limit for most harnesses (OR lists 1M for both which makes no sense). COPILOT_OFFLINE enables all calls except to OR (no GH servers).Lovely to see they're doing their own search though, I love how many folk are actively working on search clones at the moment.
https://www.reddit.com/r/DeepSeek/comments/1x84wke/deepseek_...
DeepSeek v4.1 Flash was a beast.
It was already cheaper and more powerful than Haiku 5.5 which was released like a full month after it... by a trillion dollar american "frontier" company.
Can't wait to see what this one has in store.
BTW who would have thought China would be the champion of freedom and opensource in the AI arena, while US is fighting for corporate/government control.
There are different types of freedom and China's bet is that as long as they provide enough economic freedom the public will be ok with somewhat restricted social and political one.
The fact that she can't call Xi "Winnie the Pooh" on Douyin wasn't something that occurred to her, or maybe that she was aware of at all. I didn't ask. I don't like to get into these discussions when I'm in China.
Citizens of Palm Beach county cannot voice their opposition to how their tax dollars are being spent. They rightly want that money to be invested locally, instead the County uses it to invest $1B in Israeli bonds, something that people widely oppose.
They first said that it's a budget meeting and Israeli bonds are a "policy matter". Then they started arresting people for speaking out against it. Then they started arresting anyone who applauded.
"Free speech" -> I spit on American "free speech". It's the most insidious and dishonest thing. At least the CCP doesn't PRETEND to give people rights that they simply don't have.
the U.S allows "free speech" if it's inconsequential.
Actually she can do this. There's plenty of such videos on Douyin ... If you clicked such videos frequently, algo will recommend a lot of them to you.
The worst case of uploading such videos is an aaccount ban (but very rare). So yes the freedom of speech here is fewer than in the U.S., but it's not like you'll be put into jail if you say sth about the president.
As a Luna6 proponent, this has a potential as it looks slightly more intelligent and at the similar price point.
Right now I use each of them in their own respective coding harnesses, deepseek-harness and kimi-code, but I do wonder what they are like inside something like opencode.
That's way cheaper than openai and other web search providers. I have no doubt that AI demand justifies all the compute money big tech is spending now, but I also think there are so many over valued VC funded companies that are going to crash in the next few years.
OpenCode if you want a good, opinionated setup
Maybe the equation changes if I exceed my subscription usage, and need to fill the gap with something else. But it seems like a huge waste of your time to go out of your way to use a cheap model if you have the tokens to spare in your subscription.
https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/...
> providers have to implement which could be different than what the labs themselves use.
I'd imagine (most) providers just take implementation by vLLM/SGLang.
My goal is to reverse engineer an entire gamecube game -- I'm pushing an existing community decomp that was stalled to the finish line, solo, with lots of AI help.
This work was directly inspired by https://decomp.dev/ -- which has just shut down a few days ago because of, ironically... people like me who enjoy decompiling things with AI, so I'm still weighing / grappling with the ethics of how to release my work, it's murky waters at best.
I use Opus 5.5 as the orchestrator, as well pi harness because it's somewhat lightweight. But for the really hard functions I don't use deepseek I use opus managing other opus agents.
I'm working to release something soon as proof that I can do a 100% decompilation job, but the main idea is that when decompiling a gamecube game your sources of truths are individual functions or files, so you can build off that primitive.
Each ai gets a worktree, and a shared giant context file that's basically a record of every experiment that has happened. They're assigned tasks to essentially do as much as they possible can for each individual funciton. After about an hour claude uses a script to merge the work that's actually correct (verified by tooling that checks if the c goes to asm) and then there are usually 3 rounds of reviews by independent ais, because there will always be some slight mistakes. The techniques used as well as the mistakes the models make are recorded in the giant ledger. There's a bunch of random one off scripts such as stuff to clean up, and tools to make sure the agents don't accidentally google stuff / delete stuff / use too much space.
Claude steers the ship via ssh on my strong linux tower at home so it keeps my laptop nice and cool and at this point I just check in from time to time, I am basically a manager, and Claude has it's own workers.
Last point -- I don't know any c besides the basics, I've only been "coding" python / ts for a couple of years, but even then with the extreme strength of models the future is managing agentic swarms, so I'm leaning in as much as I can. I really see this as the future of large scale projects. For some people it's a mockery of decompilation, but, my goal is the finish line, not the journey.
https://slopgrid.com/deepseek-41-horses
https://arxiv.org/abs/2601.06521
When I type deepseek on my CLI it launches Claude with my fireworks API key. Works very well and feels almost free.
I have been intrigued about Chinese models for a long time, but never tried anything except an early Qwen. My daily driver was Luna, with occasional elevation to Terra or Sol - all on 5.6 as I found the output of 6 jarringly incomprehensible.
I still had 95 USD on my OpenRouter account, so I thought I would give DeepSeek Flash 4.1 a try.
I think now I can understand the person who created that thread.
Here is the setup I settled on, after a lot of refactoring and calibration of the workflow:
I have a "plan" prompt which explains to the agent (Luna) that I want to have a discussion about an idea I have in mind. The agent should use the Socratic method to pull out all the necessary info from me using pointed questions. When the agent feels it has a complete picture, it should elevate to a higher level model (Sol), and ask its review. If Sol has complaints, Luna keeps asking me until the points raised by Sol are clarified. The review rounds with Sol continue until Sol says "ready to implement".
At this point Luna creates four files in a directory I specify:
1. PLAN.md: the implementation plan with all the details that came out from the design discussion 2. PHASES.md: implementation phases 3. DECISIONS.md: design decisions (this serves the same role as ADRs aka. architectural decision records) 4. DIARY.md: append-only log of what happened during implementation
Files 2-4 are written by the implementing agent. Luna only adds headers to explain the purpose of the files.
Once these files are there, I start up a new Pi session, switch to Deepseek 4.1 Flash and then point it to this plan folder.
And then it goes brrrrr... :-)
Here is what I implemented in the past week using this method:
1. A capture/analysis/decompilation pipeline for C64 demos: I ask the agent to start a selected demo through VICE, and say "on" or "off" when I want it to start/stop capture. The capture process uses VICE's builtin trace functionality: the trace log is piped into Clojure where it is saved into an optimized binary format (chunks of trace events). Once I have everything captured, I unleash my analysis pipeline which checks the write patterns of the executed code: what it wrote (VIC, SID, CIA, memory) and when (rasterline, cycle). Then it splits the execution trace into segments by type deduced from the write patterns (decruncher, loader, raster, music) and then decompiles each segment into both traditional ASM files and a s-exp based syntax which roundtrips back into the in-memory representation. Once I have the results of the analysis pipeline, I can tell the agent to analyze any demopart and it can explain how the effects are built down to the tiniest timing detail. As a first test, I asked it to create a clean-room implementation of Fairlight's legendary intro and two days later I had it, with everything in sexp assembly, including the music routine and its data. (The reason it took so long was that this was the first decomp and it provided a lot of feedback to the pipeline development.) Finally I had it write an assembler, linker and packager which creates a PRG from the assembled and linked artifact, then runs it through VICE. Done.
2. A DSP compiler which lets me compile sexp forms of a C-like DSP language into native DSP kernels running on the JVM, JS and WASM. Currently I have about 600 tests which verify that the behavior of the three targets matches that of a reference interpreter written in Clojure. I have a full numeric tower with i32, f32, f64, support for arrays and structs, typical control flow constructs, and a library of DSP building blocks like oscillators, filters, LFOs, delays and a chorus effect. By the time this is ready (end of the weekend?), I will be able to build synthesizers like Synth1, Surge XT or Vital in Clojure/Script, compile the DSP kernels in the browser, choosing between JS or WASM backends, driving the kernels via AudioWorklets and sequencing them from the host language.
There are two power tools at play here:
1. Deepseek Flash 4.1 2. Clojure and ClojureScript
Deepseek is a BEAST. It just goes on and delivers, phase after phase. What it needs: clear instructions - this is covered by the planning process quite well, but I still find it needs some steering from time to time (the right nudget of wisdom dropped into its thought stream at the right moment works wonders)
And Clojure... well. That article PG wrote about LISP (Beating the averages): what I witness now proves all of those claims.
The agent LOVES Clojure. Especially the REPL (not so much the parentheses).
Winning strategy: give it two tools: start_jvm and clojure_eval. Implement everything else it may need - tools to start up HTTP servers, compile ClojureScript, add dependencies, capture a C64 demo, run Playwright via spel, etc. - as lightweight Clojure namespaces which provide APIs to all these functionality. Let these namespaces be thin: the namespace and the functions therein shall have docstrings which tell the agent what the functions do, but the functions themselves shall be lightweight proxies to the real implementations in sub-namespaces. Then point the agent to these top-level API namespaces via your AGENTS.md. Bam, instant discoverability with minimal context usage. It will do EVERYTHING through clojure_eval and the round-trip time will be phenomenal, a perfect match to the raw speed of the agent.
...
Now all I need to figure out is what will I do when all my dreams become manifest in flesh.
Then I turned what Deepseek did into an upfront policy.
This seems to work quite well generally: watch the agent as it works, and think what could be provided to make its life as easy as possible
I wrote a Pi extension called "model-escalation" which lets the model request an automatic switch to a smarter model for the next turn if it thinks the problem it is struggling with is too complex or if the user explicitly asks it to do so.
With this extension in hand, I could split the work between Luna and Sol, resulting in a ~95%-5% split. I don't have to switch to Sol when review time comes, Luna does it automatically. Sol tells its remarks, then Luna gets the control back and continues the back&forth (if there are still complaints).
so AI labs can just pick which to show
I'm a communist so I'm fine, but you may not be, so be aware
[1] https://www.reddit.com/r/LocalLLaMA/comments/1x7g3zq/china_p...
Anthropic and DeepSeek (+ GLM) are definitely solving coding, though!
They're all quite different tools.
https://artificialanalysis.ai/articles/aa-3d
For me it's large files in Blender (in some cases >400MB), mostly, where Astra excels. I gave it a look, but I don't know if the benchmark encompasses that sort of thing.
.step files are simpler pieces of straightforward code, so it makes sense that Opus and Sonnet 5.5 do very well...
Besides, Fable 5.5 is reportedly releasing within the next few days, probably on Tuesday...
https://www.reuters.com/world/asia-pacific/china-proposes-ad...
They’ve technically got expedited approval for open weight models I think, but it still needs approval. And frontier level stuff gets extra scrutiny
Here's How Distill APT Used McKinsey's Private Access To Top Anthropic Model To Help The Chinese Keep Up With The Us Top IA Labs Despite Access Restrictions
yes maybe opus 5.5 is stronger and flash is cheaper, but 6.1 strikes a good balance imo.
You can in contrast get far more done with $20 worth of deepseek api spend. No comparison.