README.md

---
license: mit
language:
  - en
library_name: onnxruntime
tags:
  - music-generation
  - symbolic-music
  - lilylet
  - notagen
  - onnx
  - int8
pipeline_tag: text-generation

# For ModelScope
frameworks:
- onnx
tasks:
- symbolic_music
- music-generation
---

# LilyNota

A symbolic-music generation model that writes scores in **[Lilylet](https://github.com/k-l-lambda/lilylet)** — a compact, LilyPond-flavored text notation. Given a short metadata prompt (composer / genre / instrument / key / time signature …), the model autoregressively composes a multi-measure, multi-staff piece.

It powers the [**LilyScript**](https://huggingface.co/spaces/k-l-lambda/LilyScript) Space, where generated Lilylet is rendered to a sheet-music score (Verovio) and played back as MIDI.

> Note: the weights here are an early snapshot and will be refreshed as training continues.

## Model

`LilyNota` — a hierarchical, two-level NotaGen-style decoder (Llama backbone):

- **Patch-level decoder** — 10 layers, hidden 1024, 16 heads. Consumes the score as a stream of fixed-size *patches* (16 tokens each) and produces a hidden state per patch.
- **Token-level decoder** — 4 layers. Expands each patch state into its concrete Lilylet tokens.

| | value |
|---|---|
| params | ~196 M |
| base type | llama (bf16 trained) |
| patch size | 16 tokens |
| vocab | 256 |
| context | 1024 patches |

## Files

```
model_*.chkpt                 torch training checkpoint (full precision)
.state.yaml                   training / architecture config
tokenizer.json                tokenizer
onnx/
  patch_kv_int8.onnx          patch decoder, int8, with KV-cache (incremental)
  token_kv_int8.onnx          token decoder, int8, with KV-cache (incremental)
  wte.npy                     token-embedding table [vocab, hidden]
  geometry.json               patch size, special ids, per-level KV geometry
```

The `onnx/` bundle is **torch-free**: a generator needs only `onnxruntime` + `numpy`
to run it (the embedding lookup and sampling live outside the graph). int8 dynamic
quantization plus a two-level KV cache make it a fast CPU inference path.

## Usage

The reference runtime is `StreamingLilyletGenerator` in the
[LilyScript Space](https://huggingface.co/spaces/k-l-lambda/LilyScript)
(`lilyscript/generator.py`). Sketch:

```python
from lilyscript.generator import StreamingLilyletGenerator

gen = StreamingLilyletGenerator(model_dir='onnx', asset_dir='onnx')
prompt = '%Beethoven, Ludwig van\n%Classical\n%Keyboard'
for raw, pretty, done in gen.generate_stream(prompt_text=prompt, measures=8, temperature=1.0, seed=42):
    pass        # `pretty` is measure-segmented Lilylet; streams one patch at a time
print(pretty)
```

Output is Lilylet text, e.g.:

```
%Beethoven, Ludwig van
%Classical
\key g \major \time 3/4 \clef "treble" \tempo 4=54 ^\markup "Andante con moto" r2. \\
...
```

## License

MIT.