🇩🇪 German Language Support — Community Tracking Issue

[leonbubova]

This issue is a gathering place for everyone interested in bringing German language support to Moonshine — researchers, developers, and data contributors alike.

German is underserved in the edge STT space. Moonshine's architecture is a great fit, but there's no German model yet.

Comment if you're:
working on a German fine-tune
know of useful German audio datasets
interested in following progress

Let's coordinate here instead of scattered issues and PRs.

[SimonSchroffner]

Interested in following the progress and helping out where I can, I'm a full-stack developer.

[fosple]

Attached a list of good German datasets for training.

Original Datasets

Dataset	License	Link	Size	Notes
Mozilla Common Voice (de)	CC0 1.0 Universal	https://datacollective.mozillafoundation.org/datasets/cmj8u3p05005xnxxbqah56m27	~370h	Crowdsourced read speech, many speakers & accents. Widely used in wav2vec2 and Whisper fine-tunes.
Multilingual LibriSpeech (German)	CC BY 4.0	https://openslr.org/94	~1100h	German portion of LibriVox audiobooks. Clean read speech. Used by many research ASR models.
Spoken Wikipedia Corpus (SWC)	CC BY-SA 4.0	https://nats.gitlab.io/swc	~386h	Volunteers reading Wikipedia articles. High-quality transcripts with precise alignment.
M-AILABS Speech Dataset (German)	BSD-3-like	https://github.com/i-celeste-aurora/m-ailabs-dataset	~237h	Audiobook speech from LibriVox / Gutenberg. Clean read speech.
VoxForge (German)	GPL v2/v3	https://www.voxforge.org/de/downloads	~30h	Community-contributed speech corpus historically used in Kaldi.
TUDA-De (TU Dortmund ASR Corpus)	CC BY 4.0	https://github.com/uhh-lt/kaldi-tuda-de	~36h	Multi-microphone recordings of German sentences. High quality lab recordings.
Telecooperation Speech Corpus	CC BY 2.0	http://www.repository.voxforge1.org/downloads/de/german-speechdata-TUDa-2015.tar.gz	~35h	Kinect-recorded speech dataset used in distant-speech research.
German Bundestag Speech Corpus	Custom	https://opendata.iisys.de/dataset/asr-bundestag/	~600h	Parliamentary speeches with aligned transcripts. Real-world political speech.
Emilia-YODAS (German subset)	CC BY 4.0	https://huggingface.co/datasets/amphion/Emilia-Dataset	~5600h	Large curated web speech dataset (podcasts, lectures, etc.). Used in modern speech model training pipelines. Attention: only Emilia-YODAS subset is licensed under CC BY 4.0. Emilia dataset is CC-BY-NC.
Thorsten-Voice Dataset 2021.02 (de)	CC0-1.0	https://datacollective.mozillafoundation.org/datasets/cmm2fwi490024nt0779vv9o0d	~23h	High-quality single-speaker German speech released under CC0; primarily TTS-focused
Thorsten-Voice Dataset 2022.10 (de)	CC0-1.0	https://datacollective.mozillafoundation.org/datasets/cmm4b8y6100nomk07w678zb9d	~11h	High-quality single-speaker German speech released under CC0; primarily TTS-focused
Google FLEURS (German)	CC BY 4.0	https://huggingface.co/datasets/google/fleurs	~12 h	German subset of the FLEURS multilingual speech corpus (FLoRes-based sentence-parallel data).
VoxPopuli (German)	CC0-1.0	https://huggingface.co/datasets/facebook/voxpopuli	~282 h	German portion of the large VoxPopuli corpus (EU Parliament plenary recordings with aligned transcripts).
Multilingual TEDx (German)	CC BY-NC-ND 4.0	https://www.openslr.org/100	~15 h	German portion of the Multilingual TEDx corpus (TEDx talks with sentence-aligned transcripts).

Collections

Nvidia Granary

Some large German speech datasets (e.g. Emilia-YODAS, VoxPopuli, YouTube Commons) are also available via Granary: https://huggingface.co/datasets/nvidia/Granary

Granary is a cleaned and standardized multilingual speech corpus (~1M hours). It aggregates and processes public datasets into a ready-to-train format.

[leonbubova]

@fosple awesome, i started with M-AILABS already but this is a crazy collection!

[leonbubova]

@SimonSchroffner I'll ping you later, thanks for the support:)

[RobFlo98]

Great to see someone working on a finetuned German ASR model. That's definitely needed!
One thing that might be worth looking into: cleaning the training data by removing subtitle metadata before fine-tuning. I haven't looked into the specific datasets used here, but I've run into the annoying Whisper behavior where it reproduces patterns like Untertitelung. BR or Untertitel im Auftrag des ZDF/Funk in transcriptions, which is obviously not what we want.
There's also a slightly bigger issue with German subtitle data in general. Unlike English, where subtitles are usually close to verbatim, German subtitles often paraphrase or condense the spoken content. Mostly because the language just takes up more space. So the model ends up learning a summarized version of what was said rather than an actual transcription, which isn't great for ASR.
If the dataset is large enough to be selective, it might be worth filtering out subtitle sources or at least running a cleanup pass to strip known boilerplate patterns. Probably not a huge effort, but could make a noticeable difference in output quality.

[leonbubova]

oh that's very interesting. I'd love to chat with you @RobFlo98

[fosple]

Maybe something as inspiration:

https://github.com/pierre-cheneau/finetune-moonshine-asr

[petewarden]

Just a note that I'm keen to see German support too. :)

[dattazigzag]

Hi there @leonbubova @RobFlo98 @fosple and @petewarden ,
Patiently waiting for DE support and model (incl. benchmark).
Meanwhile will try to give the training a go

[fosple]

I trained a first model and got the WER for German down to 13.16% (Fleurs DE) / 11.95 % (MCV DE) for moonshine-streaming-small - not yet where I would like it to be.

To get it down even further I would need access to more GPU compute resources. If you have access to some compute cluster (e.g. of H100 GPUs) feel free to reach out :)

[dattazigzag]

I trained a first model and got the WER for German down to 13.16% (Fleurs DE) / 11.95 % (MCV DE) for moonshine-streaming-small - not yet where I would like it to be.

To get it down even further I would need access to more GPU compute resources. If you have access to some compute cluster (e.g. of H100 GPUs) feel free to reach out :)

@fosple wow!
I'm stuck at WER 31.5 -- trying to get to 20-25 ..
But well now I guess I won't need to now 😏

I do not have H100 - but have 2*32GB 5090 - would this work? what's your setup like?

[leonbubova]

@fosple that's crazy good. Let me try to beat that over the weekend 🙏🏻

[dattazigzag]

I trained a first model and got the WER for German down to 13.16% (Fleurs DE) / 11.95 % (MCV DE) for moonshine-streaming-small - not yet where I would like it to be.

To get it down even further I would need access to more GPU compute resources. If you have access to some compute cluster (e.g. of H100 GPUs) feel free to reach out :)

Also, few quick questions (popped in my head during lunch):

1. Which dataset did you use to train on?
2. Can we try your model that you have currently (maybe if it is uploaded to hf) as is?

[SimonSchroffner]

I would love to help with training but dont have the hardware. But if theres enough interest maybe we could pool in a few € and use a service like runpod https://www.runpod.io/pricing.

Also let me know If I can help in any other way.

[leonbubova]

@SimonSchroffner or something like https://lambda.ai/ but yes i would be down for that as well!

I've uploaded my findings and research in the meantime as well here: https://github.com/leonbubova/moonshine-german

I haven't reached more than Proof of concept stage yet though. WER ~50%