this post was submitted on 26 Jul 2026
282 points (99.3% liked)

Technology

86757 readers
3123 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 3 years ago
MODERATORS
top 50 comments
sorted by: hot top controversial new old
[–] nkat2112@sh.itjust.works 100 points 5 days ago (2 children)

The article wastes no time getting to the underlying point in the very first paragraph:

Top executives at leading Western AI companies are increasingly warning about the safety and national security risks posed by Chinese open-weight frontier models. What they tend not to mention is that these models are improving rapidly and, because they are freely available, pose a serious threat to Western labs’ business models.

I found the following two paragraphs interesting:

By mid-2026, however, open-weight frontier models from Chinese labs such as Alibaba, DeepSeek, and Moonshot AI had nearly matched the leading Western models in intelligence and performance. Many companies have already begun building their AI systems on top of these free models, avoiding the high cost of closed-model APIs.

Because businesses can host open-weight models in their own private clouds, they can also avoid sending proprietary data to systems controlled by outside providers. Developers can fine-tune the models for specific needs, build applications and tools on top of them, and optimize them for their preferred infrastructure.

[–] Mosfar@sh.itjust.works 74 points 5 days ago (1 children)

Back to local computing is the way to go

[–] artyom@piefed.social 15 points 5 days ago

It always has been. And yet consistently for decades they continually turn to Big Tech anyway...

[–] Buffalox@lemmy.world 49 points 5 days ago* (last edited 5 days ago) (6 children)

they can also avoid sending proprietary data to systems controlled by outside providers.

Not only proprietary but also personal or other kinds of sensitive data.
If you are doing health research on databases of personal health data, you should be able to guarantee the safety of that data.
That means you can't use the current American systems, because they've been shown to be insecure.
This would be a major issue in EU, where such data is legally protected.

[–] plyth@feddit.org 11 points 5 days ago (1 children)

This would be a major issue in EU, where such data is legally protected.

Nothing a EU–US Data Privacy Framework can't handle.

[–] Buffalox@lemmy.world 13 points 5 days ago

I absolutely agree that that agreement is complete and utter bullshit.
Hopefully the shift there has been to achieve IT independence from USA will mean EU doesn't give in so easy next time.

load more comments (5 replies)
[–] FauxLiving@lemmy.world 49 points 5 days ago (2 children)

The Western model was doomed to failure from the start. The only barrier to entry was being able to download thousands of TBs of internet archives/books and to have a lot of compute.

The math for these models isn't proprietary and most CS students are exposed to machine learning and neural networks while in school.

The only advantage western companies had was the ability to buy up the entire hardware market, pricing out domestic competition, and to use their politicians to manipulate trade policy in order to restrict sales of critical hardware to China.

Every US tech company has dumped billions investing in an unsustainable business model with the hope of buying a global monopoly by strangling competition.

China can destroy all of that by making their models open weight. The real money is in finding and implementing custom AI solutions... not in charging for access to the models. By having freely available models, they're making the barrier of entry as low as possible.

Not to mention that the insane amount of money being poured into hardware by US tech companies has created an environment where building fabs has a much shorter ROI, which also helps China's development in that sector.

US companies are playing Monopoly while China is playing Civilization.

[–] Cheebus@lemmy.world 10 points 5 days ago

I still think this type of AI can lead to a worse state of living for most of us, but it’s better than the bullshit in the US.

Eat my ass Musk, Zuck, Altman and all you other fuck face tech bros

load more comments (1 replies)
[–] DJKJuicy@sh.itjust.works 39 points 5 days ago (2 children)

I ran DeepSeek and Llama and Mistral at home on my consumer grade gaming PC.

With a little tweaking of the system prompts and configuring web search, I was running a local LLM that felt pretty darn close to the commercial LLMs.

With this technology out in the open internet where you can download the models in a few hours I don't see how the commercial AI companies are going to last. If selling "Artificial Intelligence" subscriptions is all your company does for revenue, you're screwed.

I downloaded and ran an LLM that I could have a conversation with and feed basic coding problems to for basically zero dollars and ran it on my puny gaming machine...puny compared to enterprise-class hardware. It would be trivial for a company with a very moderate budget to buy some servers and start running their own LLMs that they can use to feed all the PII and HIPPA data they want.

[–] brucethemoose@lemmy.world 24 points 5 days ago (2 children)

And Llama and Mistral are ancient history at this point.

The cutting edge of local is lightyears better now. It's basically where ChatGPT/Anthropic were not that long ago, with a bit less world knowledge because of the size.

[–] DJKJuicy@sh.itjust.works 11 points 5 days ago (2 children)

What's the cutting edge now? Skool me...I want to try it. Can I grab one using ollama?

[–] brucethemoose@lemmy.world 16 points 5 days ago* (last edited 5 days ago) (11 children)

https://sleepingrobots.com/dreams/stop-using-ollama/

And this is just the tip of the iceberg for ollama. They're the same kind of scammy tech bros as OpenAI.

The best setup depends on your hardware. There is no "easy button" unfortunately, quantized LLMs are just too intense and finicky to run without making some informed choices.

It also depends on what you want to do with the LLM. For example, some are too slow or bad at long context for agenic use, some quantizations are great at scripts but terrible outside that, or vice versa.

But LM Studio and Qwen 3.5 35B Q4 is probably the "easiest" flat recommendation I can make.

Or... honestly, just pay $40 for basically unlimited usage for a year from an API, then roll your own frontend.

[–] LedgeDrop@lemmy.zip 6 points 4 days ago (3 children)

Or... honestly, just pay $40 for basically unlimited usage for a year from an API, then roll your own frontend.

Can you clarify, what you mean by this? Rent a VPS? Or is there a legitimately good place that'll offer "unlimited" llm api access for $40 / year (and would you have any sort of privacy with this)?

load more comments (3 replies)
load more comments (10 replies)
[–] Balinares@pawb.social 4 points 4 days ago

Depends on your RAM (main + GPU), but assuming 32GB total: Qwen 3.6 35B A3B for coding support, Gemma 4 26B for general stuff. The LM Studio app curates a list of recommended models that will run well in it and makes it easy to run them.

Mind you, what I like most about local models is their limitations, because it turns out closed models have limitations of the same nature, just with quite a bit more runway; and becoming aware of those limitations is valuable.

[–] D1re_W0lf@piefed.social 4 points 4 days ago (1 children)

Thanks for the info. I don’t follow it closely but afaik wasn’t Mistral the only western open weight model around?

[–] brucethemoose@lemmy.world 4 points 4 days ago (1 children)

Mistral is still around, but for local LLMs... they're kinda irrelevant, sadly. Their models have regressed. I think they're being choked by ambiguous EU law.

If you're looking for "western" weights, I'd look at the Laguna series first:

https://huggingface.co/poolside

And ik_llama.cpp quantizations like this: https://huggingface.co/sigargv/Laguna-M.1-GGUF/tree/main

There's a couple of other interesting startups, but TBH its hard to keep track of where they're from.

[–] D1re_W0lf@piefed.social 3 points 4 days ago

I wasn’t aware of that.

Thank you so much for the info.

[–] stankmut@lemmy.world 6 points 5 days ago (2 children)

Since you mention using Ollama, you probably aren't running actual deepseek on your pc. Ollama took a Qwen model that was finetuned using deepseek output and named it deepseek.

Those are pretty out of date models at this point. Right now, the model most people would recommend for consumer gaming hardware is Qwen 3.6 27b.

load more comments (2 replies)
[–] ZILtoid1991@lemmy.world 6 points 3 days ago

Yes, but only if either the required hardware becomes affordable (don't expect frontier models to run on 5090s, let alone on the 1050Ti you're keeping as a backup), or somehow the models become more efficient. Even if by reduced capability, which might be a good thing in the grand scheme of things. Some people use overengineered frontier models as synonym search engines.

[–] eicker@lemmy.world 21 points 5 days ago (2 children)

The awkward part is that software has a habit of racing toward free once it becomes good enough. If an open weight model delivers 95 percent of the value without recurring API costs, plenty of companies will choose that and spend the savings on integration instead. History keeps rhyming, even if investors hate the tune.

[–] MangoCats@feddit.it 4 points 4 days ago (1 children)

racing toward free once it becomes good enough.

I don't know... I was using Open(now Libre)Office over 20 years ago, it wasn't just good enough, it was better than MS Office at the time, and yet... even though LibreOffice has been free and far more than "good enough" for long over a decade, my corporate decision makers insist that we all use Office365 subscriptions, complete with their service outages and other issues.

[–] eicker@lemmy.world 6 points 4 days ago (1 children)

Without a doubt. And yet: without open source, pretty much everything digital would come to an end straight away. Sometimes what’s visible isn’t what really matters.

[–] MangoCats@feddit.it 5 points 4 days ago

I have definitely seen MS Office and Visual Studio stall at a level and stay there until after they're clearly surpassed by open source competition, then they fund another round of development and ... change, some might say improve, but often it looks like change for the sake of change, especially in Office.

[–] ExLisper@lemmy.curiana.net 6 points 4 days ago (3 children)

The awkward part is that software has a habit of racing toward free once it becomes good enough

Has it? Jira, Teams, Slack, Google Meet, Cisco WebX... There's obviously Jitsi but I don't see companies racing towards it. When it comes to SaaS most companies prefer to pay and be done with it. Few are choosing to host open source solutions. Windows and Mac are also way more popular than Linux in office settings. With LLMs hosting it will be even more complicated because companies will have to invest heavily in GPUs.

If the bubble bursts companies will lose funds to work on new models but someone will still be able to offer existing models as a service. Companies will chose the one with better price and functionality. Being free or not will have little to do with it.

[–] eicker@lemmy.world 7 points 4 days ago (2 children)

Unfortunately, based on many years of experience, I have to agree. However, I also see a light at the end of the tunnel, particularly in Europe, where there is a growing desire to break free from the US SaaS stranglehold, using open source software.

Regarding OWAI I see a fundamental difference: Slack isn’t just software, it’s a hosted service with identity, storage and network effects.

An open weight model is more like a compiler: once downloaded, nobody can revoke it. You may still pay for inference, but pricing power drops when anyone can host the same model.

load more comments (2 replies)
[–] brucethemoose@lemmy.world 3 points 4 days ago (4 children)

They don’t have to host it themselves. They could use a number of providers for the same model, and basically keep doing whatever they were doing with OpenAI/Anthropic via the exact same APIs.

load more comments (4 replies)
[–] postmateDumbass@lemmy.world 3 points 4 days ago

When you pay for something that means passing responsibility (liability) for that function.

Much harder to sue over free software.

[–] MangoCats@feddit.it 9 points 4 days ago* (last edited 4 days ago) (1 children)

I see a business model where "we're done, this one is (finally) good enough and now we'll stop bleeding cash on the training and turn up the screws on the customers we've hooked on loss leader pricing." Open weight models will never stop training for improvement, the costs for training seem to be inexorably falling, and any business model built on the idea that they can kick back and roll in the profits after their initial "hard work" is going to lose all their customers to better products.

This isn't some captive market like US automobile customers who have no choice but the limited selection of crap, crap and more crap that is put in front of them. At least not as long as the internet remains relatively open.

[–] boonhet@sopuli.xyz 1 points 3 days ago* (last edited 3 days ago) (1 children)

It's not necessarily that any of the open models are cheaper to train (for similar model size), it's more that China has deeper pockets. And MoE inference is cheaper than dense models I believe

[–] MangoCats@feddit.it 2 points 3 days ago

If China manages to scale their hardware production, this could finally be "their day in the sun" where they clearly surpass the West in the way the West has outshone them for 100 years. MoE has its place, but a MoDE where each E is itself a dense model would be more powerful still, mostly you need the silicon gates and power to drive them.

Three Gorges makes hydro-power, right? 22MW -> 100 TWh per year, just from that one structure, I bet that will run at least one AGI... https://www.fredgao.com/p/deepseeks-liang-wenfeng-breaks-his

[–] humanspiral@lemmy.ca 12 points 4 days ago

Open models are still most cheaply hosted on a cloud, with batching and 24/7 use. API rates from developer lab are generally fair. Self hosting does have some significant tangible benefits though: Fine tuning for domain specific to organization, and not letting LLM provider train from your prompts/answers, followed by competing with your organization in the future as a result of "distilling your IP".

[–] danielfm123@lemmy.zip 9 points 4 days ago (2 children)

That's the reason they are trying to embed AI into everything.

load more comments (2 replies)
[–] Greg@lemmy.ca 22 points 5 days ago* (last edited 5 days ago) (5 children)

I just wish I could buy enough memory to run one of these models locally. Specially Kimi K3

[–] Dionysus@leminal.space 14 points 5 days ago (2 children)

Same, getting ~3 trillion parameters in consumer hardware is rough.

If Nvidia has any foresight they'll see the writing on the wall and start getting higher memory Spark style SMB inference machines, few people in the long run are going to pay retail API token costs,

load more comments (2 replies)
[–] brucethemoose@lemmy.world 6 points 5 days ago* (last edited 5 days ago) (2 children)

How much RAM do you have?

I can run MiMo 2.5 at about 9 tokens/sec, on 128GB RAM, a 7800 and a 3090 in an SFF rig. That's a fantastic 310B model. I'm requantizing it right now, to see if I can speed it up with Dflash.

Still fantastic models can be run on 64GB or 32GB CPU RAM, as long as you have some GPU. We're awash in sparse models these days.

load more comments (2 replies)
load more comments (3 replies)
[–] ndupont@lemmy.blahaj.zone 8 points 4 days ago
[–] brucethemoose@lemmy.world 11 points 5 days ago

The localllama crowd has know this for years.

It happened faster than I expected, though; OpenAI/Anthropic hardly even got the chance to tighten the screws.

[–] yesman@lemmy.world 11 points 5 days ago (2 children)

is their a meaningful difference between open weight and open source?

[–] herrvogel@lemmy.world 26 points 5 days ago (2 children)

Pretty big difference. An open weight model is a model that you can run on your own machine. You just download and it's yours to host and use. You don't need to have anyone host it on their own backend for you, the entire model is available to you to do that on your own. What you don't have is any control over or access to anything related to how the model was trained. You don't know what kind of data they used to train it, and how exactly they used that dataset. If you did, that'd be an open source model.

load more comments (2 replies)
[–] melfie@lemmy.zip 5 points 5 days ago

Open weight is analogous to a compiled binary. Similar to how Windows is closed source, but still runs in your own hardware, whereas Linux is truly open source.

[–] Mwa@thelemmy.club 4 points 4 days ago (1 children)

We even got open weight models that's 27B + 1-bit (and it still has good performance)

[–] brucethemoose@lemmy.world 3 points 4 days ago* (last edited 4 days ago) (1 children)

Bonsai? Or whatever it’s called? It’s a con, so far; it’s not better than smaller models quantized to 3-4 bits.

I love, love the idea of bitnet, but it only seems to work with models trained from scratch, which no one has done at scale yet.

load more comments (1 replies)
load more comments
view more: next ›