Today we're releasing three new models with 80M, 40M and 14M parameters.
The largest model (80M) has the highest quality. The 14M variant reaches new SOTA in expressivity among similar sized models, despite being <25MB in size. This release is a major upgrade from the previous one and supports English text-to-speech applications in eight voices: four male and four female.
Here's a short demo: https://www.youtube.com/watch?v=ge3u5qblqZA.
Most models are quantized to int8 + fp16, and they use ONNX for runtime. Our models are designed to run anywhere eg. raspberry pi, low-end smartphones, wearables, browsers etc. No GPU required! This release aims to bridge the gap between on-device and cloud models for tts applications. Multi-lingual model release is coming soon.
On-device AI is bottlenecked by one thing: a lack of tiny models that actually perform. Our goal is to open-source more models to run production-ready voice agents and apps entirely on-device.
We would love your feedback!
I'm impressed with the quality given the size. I don't love the voices, but it's not bad. Running on an intel 9700 CPU, it's about 1.5x realtime using the 80M model. It wasn't any faster running on a 3080 GPU though.
Regarding running on the 3080 gpu, can you share more details on github issues, discord or email? it should be blazing fast on that. i'll add an example to run the model on gpu too.
Had to get the right Python version and make sure it didn't break anything with the previous Python version. A friend suggested using Docker, so I started down that path until I realized I'd probably have to set the whole thing up there myself. Eventually got it to run and I think I didn't break anything else.
I hate Python so much.
I suspect success is highly variable on macOS vs. Linux; the spacey bug is only in newer (3.14 only or later) Pythons, which Linux will have.
I should learn to give up quicker.
I did eventually do that though, and I'm pretty sure I had to mess about with installing and uninstalling torch.
I dread using anything made in python because of this. It's always annoying and never just works (if the version of python is incompatible, otherwise it's fine) .
Maybe a dumb and slightly tangential question, (I don't mean this as a criticism!) but why not release a command line executable?
Even the API looks like what you'd see in a manpage.
I get it wouldn't be too much work for a user to actually make something like that, I'm just curious what the thought process is
I didn't expect it to pronounciate 'ms' correctly, but the number sounded just like noise. Eventually I got an acceptable result for the string "Startup finished in one hundred and thirty five seconds.
The above SECDED check-bit encoding can be implemented in a similar way, but since it uses only three-bit patterns, mapping syndromes to correction masks can be done with three-input AND gates.
It sounded quite good indeed for the normal English stuff, but I guess predictably was quite bad at the domain-specific words. It misspoke "SECDED", had wrong emphasis on "syndromes", and pronounced "AND gates" like "and gates".
Could you give some example of what kind of preprocessing would help in this case? I tried some local LLMs, but they didn't do a good job (maybe my prompts sucked).
I'm not sure if you're misspelling it deliberately or not, but the word you're looking for is "pronounce" and it's verb form "pronouncing", as in "It just has issues pronouncing numbers" and "I didn't expect it to pronounce 'ms' correctly."
At 25MB you can actually bundle it with the app. Going to test whether this works in a Vercel Edge Function context -- if latency is acceptable there it opens up a lot of use cases that currently require a round-trip to a hosted API.
Perhaps his YouTube channel is worth a watch. This video from four months ago compares various STT tools: https://youtu.be/pKU9CABtnOw
Speaking of apps that would, if I had to guess, love to integrate you:
FluidVoice is incredible and developing quickly. Handy is really hot right now. Also have VoiceInk out there, solid iOS option.
[ps-not parent commenter]
On macOS, it's a markedly different experience: it's only ~700 MiB there; I'm assuming b/c no NVIDIA libs get pulled in, b/c why would they.
For anyone who might want to play around with this: I can get down to ~3 GiB (& about 1.3 GiB if you wipe your uv cache afterwards) on Linux if I add the following to the end of `pyproject.toml`:
[tool.uv.sources]
# This tells uv to use the specific index for torch, torchvision, and torchaudio
torch = [
{index = "pytorch-cpu"}
]
torchvision = [
{index = "pytorch-cpu"}
]
torchaudio = [
{index = "pytorch-cpu"}
]
[[tool.uv.index]]
name = "pytorch-cpu"
url = "https://download.pytorch.org/whl/cpu"
This is an example of what one sees under Linux:
nvidia-nvjitlink-cu12 ------------------------------ 23.83 MiB/37.44 MiB
nvidia-curand-cu12 ------------------------------ 23.79 MiB/60.67 MiB
nvidia-cuda-nvrtc-cu12 ------------------------------ 23.87 MiB/83.96 MiB
nvidia-nvshmem-cu12 ------------------------------ 23.62 MiB/132.66 MiB
triton ------------------------------ 23.82 MiB/179.55 MiB
nvidia-cufft-cu12 ------------------------------ 23.76 MiB/184.17 MiB
nvidia-cusolver-cu12 ------------------------------ 23.84 MiB/255.11 MiB
nvidia-cusparselt-cu12 ------------------------------ 23.99 MiB/273.89 MiB
nvidia-cusparse-cu12 ------------------------------ 23.96 MiB/274.86 MiB
nvidia-nccl-cu12 ------------------------------ 23.79 MiB/307.42 MiB
nvidia-cublas-cu12 ------------------------------ 23.73 MiB/566.81 MiB
nvidia-cudnn-cu12 ------------------------------ 23.56 MiB/674.02 MiB
torch ------------------------------ 23.75 MiB/873.22 MiB
It also then crashes for me, with:
File "KittenTTS/.venv/lib/python3.14/site-packages/pydantic/v1/fields.py", line 576, in _set_default_and_type
raise errors_.ConfigError(f'unable to infer type for attribute "{self.name}"')
pydantic.v1.errors.ConfigError: unable to infer type for attribute "REGEX"
-requires-python = ">=3.8"
+requires-python = ">=3.8,<3.14"
Also:
+ curated-tokenizers==0.0.9
Hilariously, removing it breaks but only on macOS machine. I think you're using it solely for the side-effect that it tweaks phonemizer to use espeakng, but you can just do that tweak yourself, & then I think that dependency can be dropped. That drops a good number of dependencies & really speeds up the installation since we're not compiling a bunch of stuff.
You need to add `phonemizer-fork` to your dependencies. (If you remove misaki, you'll find this missing.)
People from outside the UK often use British as synonymous with English, and in the context of accents, often a South East English accent or some sort of Received Pronunciation (RP) accent. Technically a "British" accent could be from anywhere in England, Scotland, or Wales, and therefore by extension might not even be the English language.
While I'm here, since it's generally confusing, the UK is Great Britain and Northern Ireland. Great Britain is England, Scotland, and Wales.
Kokoro TTS for example has a very good Norwegian voice but the rhythm and emphasizing is often so out of whack the generated speech is almost incomprehensible.
Haven't had time to check this model out yet, how does it fare here? What's needed to improve the models in this area now that the voice part is more or less solved?
If only I could have that in Norwegian my SO would be pleased.
Also I totally misremembered regarding Kokoro TTS. It's good, but not what was butchering Norwegian. Forgot which one I was thinking of, maybe it was the old VITS stuff Rhaspy uses. Points stand, the voice was good but could barely understand what was said.
Huge fan of Ava multilingual and hopefully there are many other others with similar taste, so my feedback might shape things towards a halfway decent direction at least for some.
btw, use case is most often to listen to news/articles.
No need to DM me, just post on HN or /r/LocalLLama and I'll catch wind of it.
Thanks for your work!
Either in the form of the api via pitch/speed/volume controls, for more deterministic controls.
Or in expressive tags such as [coughs], [urgently], or [laughs in melodic ascending and descending arpeggiated gibberish babbles].
the 25MB model is amazingly good for being 25MB. How does it handle expressive tags?
A stretch goal is 'arbitrary tags' from [singing] [sung to the tune of {x}] [pausing for emphasis] [slowly decreasing speed for emphasis] [emphasizing the object of this sentence] [clapping] [car crash in the distance] [laser's pew pew].
But yeah: instruction/control via [tags] is the deciding feature for me, provided prompt adherence is strong enough.
Also: a thought...
Everyone is using [] for different kinds of tags in this space: which is very simple. Maybe it makes sense to differentiate kinds of tags? I.E. [tags for modifying how text is spoken] vs {tags for creating sounds not specifically speech: not modifying anything... but instead it's own 'sound/word'}
If the author doesn't describe some detail about the data, training, or a novel architecture, etc, I only assume they just took another one, do a little finetuning, and repackage as a new product.
Also:
I couldn't locate how to run it on a GPU anywhere in the repo.
The new 15M is way better than the previous 80M model(v0.1). So we're able to predictably improve the quality which is very encouraging.
Is there any way to get those running on iPhone ? I would love to have the ability for it to read articles to me like a podcast.
The iOS version is Swift-based.
I want to be my own personal assistant...
EDIT: I can provide it a RTX 3080ti.
Qwen 3 TTS is good for voice cloning but requires GPU of some sort.
Is there any way to do a custom voice as a DIY? Or we need to go through you? If so, would you consider making a pricing page for purchasing a license/alternative voice? All but one of the voices are unusable in a business context.
This is a mind numbing task that requires workers to make hundreds of calls each day with only minor variations, sometimes navigating phone trees, half the time leaving almost the exact same message.
Anyway, I believe almost all such businesses will be automated within months. Human labour just cannot compete on cost.
(That's using the example as-is. If you switch it to the smaller model, modify the above with +57 MiB of models from HuggingFace, or =727 MiB.)
So I toyed with this a bit + the Rust library "ort", and ort is only 224M in release (non-debug) mode, and it was pretty simple to run this model with it. (I did not know ort before just now.) I didn't replicate the preprocessing the Python does before running the model, though. (You have to turn the text into an array of floats, essentially; the library is doing text -> phonemes -> tokens; the latter step is straight-forward.)
text ="""
Hello world. This is Kitten TTS.
Look, it's working!
"""
voice = 'Luna'
Phonemes: ðɪs ɪz kˈɪʔn ̩ tˌiːtˌiːˈɛs ,
edit: it really helps to use the same model facepalm. It's the 80M model, and it happens on both OS. Wildly the nano gets it better? I'm going to join the Discord lol.
Tldr: generate human-like voice based on animal sound. Anyway maybe it doesn't make sense.
Downloading https://github.com/KittenML/KittenTTS/releases/download/0.8.1/kittentts-0.8.1-py3-none-any.whl (22 kB)
Collecting num2words (from kittentts==0.8.1)
Using cached num2words-0.5.14-py3-none-any.whl.metadata (13 kB)
Collecting spacy (from kittentts==0.8.1)
Using cached spacy-3.8.11-cp314-cp314-win_amd64.whl.metadata (28 kB)
Collecting espeakng_loader (from kittentts==0.8.1)
Using cached espeakng_loader-0.2.4-py3-none-win_amd64.whl.metadata (1.3 kB)
INFO: pip is looking at multiple versions of kittentts to determine which version is compatible with other requirements. This could take a while.
ERROR: Ignored the following versions that require a different python version: 0.7.10 Requires-Python >=3.8,<3.13; 0.7.11 Requires-Python >=3.8,<3.13; 0.7.12 Requires-Python >=3.8,<3.13; 0.7.13 Requires-Python >=3.8,<3.13; 0.7.14 Requires-Python >=3.8,<3.13; 0.7.15 Requires-Python >=3.8,<3.13; 0.7.16 Requires-Python >=3.8,<3.13; 0.7.17 Requires-Python >=3.8,<3.13; 0.7.5 Requires-Python >=3.8,<3.13; 0.7.6 Requires-Python >=3.8,<3.13; 0.7.7 Requires-Python >=3.8,<3.13; 0.7.8 Requires-Python >=3.8,<3.13; 0.7.9 Requires-Python >=3.8,<3.13; 0.8.0 Requires-Python >=3.8,<3.13; 0.8.1 Requires-Python >=3.8,<3.13; 0.8.2 Requires-Python >=3.8,<3.13; 0.8.3 Requires-Python >=3.8,<3.13; 0.8.4 Requires-Python >=3.8,<3.13; 0.9.0 Requires-Python >=3.8,<3.13; 0.9.2 Requires-Python >=3.8,<3.13; 0.9.3 Requires-Python >=3.8,<3.13; 0.9.4 Requires-Python >=3.8,<3.13; 3.8.3 Requires-Python >=3.9,<3.13; 3.8.5 Requires-Python >=3.9,<3.13; 3.8.6 Requires-Python >=3.9,<3.13; 3.8.7 Requires-Python >=3.9,<3.14; 3.8.8 Requires-Python >=3.9,<3.14; 3.8.9 Requires-Python >=3.9,<3.14
ERROR: Could not find a version that satisfies the requirement misaki>=0.9.4 (from kittentts) (from versions: 0.1.0, 0.3.0, 0.3.5, 0.3.9, 0.4.0, 0.4.4, 0.4.5, 0.4.6, 0.4.7, 0.4.8, 0.4.9, 0.5.0, 0.5.1, 0.5.2, 0.5.3, 0.5.4, 0.5.5, 0.5.6, 0.5.7, 0.5.8, 0.5.9, 0.6.0, 0.6.1, 0.6.2, 0.6.3, 0.6.4, 0.6.5, 0.6.6, 0.6.7, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.7.4)
ERROR: No matching distribution found for misaki>=0.9.4
BTW, it seems that kitten (the Python package) has the following chain of dependencies: kittentts → misaki[en] → spacy-curated-transformers
So if you install it directly via uv, it will pull torch and NVIDIA CUDA packages (several GB), which are not needed to run kitten.
In case it helps anyone else, the first time I tried to run purr I got "OSError: PortAudio library not found". Installing libportaudio (apt install libportaudio2) got it running.