[pauses] i think i heard demons

That's certainly unusual ...

That was... amazing.

I have a hunch they're pulling data from radio shows to give it that "high quality" vibe. Tried running it through this script and hit some weird bugs too:

    [S1] It really sounds as if they've started using NPR to source TTS models
    [S2] Yeah... yeah... it's kind of disturbing (laughs dejectedly).
    [S3] I really wish, that they would just Stop with this.

You hittin them balloons again, mate ?

Yeah, Eleven Labs must be raking it in.

You can get hours of audio out of it for free with Eleven Reader, which suggests that their inference costs aren't that high. Meanwhile, those same few hours of audio, at the exact same quality, would cost something like $100 when generated through their website or API, a lot more than any other provider out there. Their pricing (and especially API pricing) makes no sense, not unless it's just price discrimination.

Somebody with slightly deeper pockets than academics or one guy in a garage needs to start competing with them and drive costs down.

Open TTS models don't even seem to utilize audiobooks or data scraped off the internet, most are still Librivox / LJ Speech. That's like training an LLM on just Wikipedia and expecting great results. That may have worked in 2018, but even in 2020 we knew better, not to mention 2025.

TTS models never had their "Stable Diffusion moment", it's time we get one. I think all it would take is somebody doing open-weight models applying the lessons we learned from LLMs and image generation to TTS models, namely more data, more scraping, more GPUs, less qualms and less safety. Eleven Labs already did, and they're profiting from it handsomely.

Thank you for the kind words <3

This is amazing. Is it possible to build in a chosen voice, a bit like Eleven Labs does? ...This may be on the git summary, being lazy and asking anyway :=) Thanks for your work.

Realistic voice acting for audio books, realistic images for each page, realistic videos for each page, oh wait I just created a movie, maybe I can change the plot? Oh wait I just created a video game

Wouldn’t it be more desirable to hear an actual human on an audiobook? Ideally the author?

For anyone who wants to listen, it's on this page: https://yummy-fir-7a4.notion.site/dia

Thank you!! Indeed the script was inspired from a scene in the Office.

This is oddly reminiscent of the office. I wonder if tv shows were part of its training data!

This is so good. Reminds me of The Office. I love how bad the other examples are.

Yeah, that example is insane.

Is there some sort of system prompt or hint at how it should be voiced, or does it interpret it from the text?

Because it would be hilarious if it just derived it from the text and it did this sort of voice acting when you didn't want it to, like reading a matter-of-fact warning label.

I know it’s taboo to ask, but I must: where’s the dataset from? Very eager to play around with audio models myself, but I find existing datasets limiting

Amazing that you developed this over the course of three months! Can you drop any insight into how you pulled together the audio data?

Could one usecase be generating an audiobook with this from existing books? I wonder if I could fine-tune the "characters" that speak these lines since you said it's a single pass whole the whole convo. Wonder if that's a limitation for this kind of a usecase (where speed is not imperative).

Hi! This is awesome for size and quality. I want to see a book reading example or try it myself.

This is a tangent point but it would have been nicer if it wasn't a notion site. You could put the same page on github pages and it will be much lighter to open, navigate and link (like people trying to link some audio)

This is super awesome. Several questions.

1. What GPU did you use to train the model? I'd love to train a model like this, but currently, I only have a 16GB MacBook. Thinking about buying a 5090 if it's worth.

2. Is it possible to use this for real time audio generation, similar to the demo on the Sesame website?

Its really amazing cant wait to play with it some, the samples are great... but oddly all seem... really fast, like they'd be perfect but they feel like they're playing at 1.2x speed or is that just me?

This is a pretty incredible three month creation for a couple of people who had no experience with speech models.

In terms of guiding voice and expression, audio prompts are promising but I believe text instructions serve different experiences as well. Will there be support for that as well?

Easily 10 times better than recent OpenAI voice model. I don't like robotic voices.

Example voices seems like over loud, over excitement like Andrew Tate, Speed or advertisement. It's lacking calm, normal conversation or normal podcast like interaction.

Are there any examples of the audio differences between the this and the larger model?

hey, this looks (or rather, sounds) amazing! Does it work with different languages or is it English only?

Thank you for this! My desktop GPU has only 8GB VRAM, but my MacBook has plenty of unified RAM.

Thanks, works well but slowly on a Mac Air M3 with 24gb. Will have to try it again after freeing up more ram as it was doing a bit of swapping with Chrome running too.

(later). It did nicely for the default example text but just made weird sounds for a "hello all" prompt. And took longer?!

Thank you for the contribution! We'll be merging PRs and cleaning code up very soon :)

Can confirm, runs straight forward on 15.4.1@M4, THX.

Sorry for the confusion. the license is plain Apache 2.0, and we changed the wording to "intended for research and educational use." The point was, users are free to use it for their use cases, just don't do shady stuff with it.

Thanks for the feedback :)

Hmm, the "strictly forbidden" part seems more important than whatever are their stated intentions... Either way, it seems like it needs clarifying.

I have a sneaking suspicion it's because they lifted the model architecture almost directly from Parakeet: https://jordandarefsky.com/blog/2024/parakeet/

Parakeet references WhisperD which is at https://huggingface.co/jordand/whisper-d-v1a and doesn't include a full list of non-speech events that it's been trained with, except "(coughs)" and "(laughs)".

Not saying the authors didn't do anything interesting here. They put in the work to reproduce the blog post and open source it, a praiseworthy achievement in itself, and they even credit Parakeet. But they might not have the list of commands for more straightforward reasons.

Fun, I can't get to it because I can't get past the "Making sure you're not a bot!" page. It's just stuck at "calculating...". I understand the desire to slow down AI bots, but . If all the gnome apps are now behind this, they just completely shut down a small-time contributor. I love to play with Gnome apps and help out with things here and there, but I'm not going to fight with this damn thing to do so.

And another one, not open source but in AI sphere: https://www.diabrowser.com/

Thanks for the heads-up! We weren’t aware of the GNOME Dia project. Since we focus on speech AI, we’ll make sure to clarify that distinction.

I know it's a bit ridiculous to see that as some kind of conspiracy, but I have seen a very long list of AI-related projects that got the same name as a famous open-source project, as if they wanted to hijack the popularity of those projects, and Dia is yet another example. It was relatively famous a few years ago and you cannot have forgotten it if you used Linux for more than a few weeks. It's almost done on purpose.

> Also don't understand how to control the S1 and S2 speakers...

Do a clip with the speakers you want as the audio prompt, add the text of that clip (with speaker tags) of the clip at the beginning of your text prompt, and it clones the voices from your audio prompt for the output.

Thank you for the kind words! Dia wasn’t fine tuned on certain speaker, so you will get random voices every time you run it, unless you add a prompt / fix the seed.

The outputs are a bit unstable, might need to add cleaner training data and run longer training sessions. Hopefully we can do something like OAI Whisper and update with better performing checkpoints!

> Does mean it pulls the model every time on startup though which isn't ideal

Surely it just downloads to a directory that can be volume mapped?

Interesting. I haven't thought of that problem before. I'm guessing a large enough audio dataset for medical terminology does not exist publicly.

But AFAIK, even if you have just a few hours of audio containing specific terminology (and correct pronunciation), fine-tuning on that data will significantly improve performance.

NPR voice is a thing.

It started with Ira Glass voice and now the default voice is someone that sounds like they're not certain they should be saying the very banal thing they are about to say, followed by a hand-shake protocol of nervous laughter.

Thank you so much for the kind words :) We only support English at the moment, hopefully can do more languages in the future. We are planning to release a technical report on some of the details, so stay tuned for that!

Sounds awesome! I think it won't be very hard to run it using output streaming, although that might require beefier GPUs. Give us an email and we can talk more - nari.ai.contact at gmail dot com.

It's way past bedtime where I live, so will be able to get back to you after a few hours. Thanks for the interest :)

The examples on your site are impressive, but I'm having trouble getting good results on HF - it's generating a lot of near-silence (often nothing but) and when it does produce speech it bears no resemblance to the audio prompt and only produces parts of the text prompt. Would you suggest any adjustments to the default parameters to improve adherence, or might I expect better results running locally? Thanks!

Thanks for the kind words :)))

We're envisioning a platform with a social aspect, so that is the biggest difference. Also, bigger models!

We are aware of the fact that you do not need to create a venv when using pre-existing uv. Just added it for people spinning up new GPUs on cloud. But I'll update the README to make that a bit clearer. Thanks for the feedback :)

Seek back a few tens of bytes which states "Play with a larger version of Dia"

We'll try to give a high-level overview when we publish the technical report!

Hi! I'm Dia's developer. We currently only support English.

We will try to make it work, but not sure if will be an easy task. For now, you can try with https://huggingface.co/spaces/nari-labs/Dia-1.6B

Only two voices at the moment... We will need to upgrade the dataset to make that happen, and are considering that as one of the next steps.

Thank you for the kind words! We only support English at the moment.. Hope to add more languages in the future.

We will work on a quantized version of the model, so hopefully you will be able to run it soon!

We've seen Bark from Suno go from 16GB requirement -> 4GB requirement + running on CPUs. Won't be too hard, just need some time to work on it.

Thank you for the kind words! We don't have plans for that yet, but you can always open an issue or RP on Github.

We're adding guides for Zero-shot voice cloning. You can try it using the second example on Gradio: https://huggingface.co/spaces/nari-labs/Dia-1.6B

You can add an audio prompt and prepend text corresponding to it in the script. You can get a feel for it by trying the second example in the Gradio interface!

Unfortunately yes at the moment

We use descript audio codec! I’m not sure if DAC works on iOS…

https://huggingface.co/spaces/nari-labs/Dia-1.6B we fixed it!

We are in the progress of fixing it! Thanks for letting us know :)

We just clarified in the README, sorry for the confusion ;(

Note that the model was not fine-tuned on a specific voice. Hence, you will get different voices every time you run the model. You can keep speaker consistency by either adding an audio prompt (a guide coming VERY soon - try it with the second example on Gradio or HF Space for now), or fixing the seed.

Thanks for the interest! We also enjoyed using E5-F2 :) You can try it now on HF Spaces: https://huggingface.co/spaces/nari-labs/Dia-1.6B

Thanks you!! We personally used Quickpod and Runpod the most. But you can try it now on HF Spaces without spinning up GPUs yourself!

https://huggingface.co/spaces/nari-labs/Dia-1.6B

> first time I've seen such expressiveness in TTS for laughs, coughs, yelling about a fire, etc!

The old Bark TTS is noisy and often unreliable, but pretty great at coughs, throat clears, and yelling. Even dialogs... sometimes. Same Dia prompt in Bark: https://vocaroo.com/12HsMlm1NGdv

Dia sounds much more clear and reliable, wild what 2 people can do in 3 months.