this post was submitted on 23 Jul 2026
55 points (96.6% liked)

Europe

493 readers
1 users here now

All about Europe

founded 1 year ago
MODERATORS
 

The scientific publication: A Sovereign, Open-Source Foundation Model for German and English

Soofi S 30B-A3B is a sovereign, open-source Mixture-of-Experts (MoE) hybrid Mamba–Transformer foundation model for German and English. Its hybrid design activates only 3B of 30B parameters per token and keeps the inference cache near-constant as context grows, giving it a decisive throughput advantage over dense models for long-context, high-concurrency deployment.

Pretrained on roughly 27 trillion tokens with deliberately up-weighted German, Soofi S matches dense 14–27B models on aggregate English and German benchmarks, achieves the best code aggregates in both languages among 17 open base models, and outperforms every European sovereign baseline in our comparison — including ones far larger in active parameters. Among fully open models, it obtains the highest English and German evaluation scores, ahead of Olmo 3 32B and Apertus 70B. Soofi S was built end-to-end on the German Industrial AI Cloud, a sovereign HPC-scale AI infrastructure operated by Deutsche Telekom in Munich.

...

top 8 comments
sorted by: hot top controversial new old
[–] boonhet@sopuli.xyz 8 points 3 days ago

I was gonna complain "open weight not open source", but this one actually comes with both the training AND data set reconstruction scripts apparently, making this actually open source in a meaningful way

[–] onlinepersona@programming.dev 1 points 3 days ago

Next question :are there European huggingface alternatives? I'd like to be able to point opencode or any opensource agent to a European service hosting opensource models.

[–] monkeyFromTheLake@programming.dev 2 points 4 days ago (1 children)

Hopefully we will see more initiatives like this and also step up to frontier models.

[–] Valmond@lemmy.dbzer0.com 1 points 3 days ago (1 children)
[–] monkeyFromTheLake@programming.dev 2 points 3 days ago (1 children)

It's what high end models are called. They are at the technological frontiet.

[–] Valmond@lemmy.dbzer0.com 1 points 3 days ago (1 children)

Still predictive like the 2019 google paper though right?

[–] monkeyFromTheLake@programming.dev 2 points 3 days ago (1 children)

I don't know that paper and am not an expert in model architecture. But to my understanding they are still predictive. An important improvement beyond adding more parameters was the introduction of MoE (mixture of experts?) architecture. Basically this segregates the models into experts that are only activated when needed.

[–] Valmond@lemmy.dbzer0.com 1 points 3 days ago

Yeah that seems reasonable because of their size.

Fun fact, when this was tested when AI just was predictive networks (well, before LLM), humans got beat to how to handle these cut up smaller networks, a bigger all inclusive net always yielded better results.