FredyRivera-dev/diffusiongemma-26B-A4B-it-HERETIC-Uncensored
Text Generation • 26B • Updated • 59 • 2
👋 Open to Work
Fredy Rivera
FredyRivera-dev
AI & ML interests
None yet
Recent Activity
liked a model 1 day ago
thinkingmachines/Inkling-Small liked a model 2 days ago
unsloth/Kimi-K3-GGUF reacted to theirpost with 🚀 4 days ago
We wrote a full technical guide on how to train a bilingual (ES/EN) LLM from scratch: TinyQwen.
Covers:
- Hybrid architecture based on Qwen3.5
- Pre-training with 15B tokens
- Cost benchmark between H200 and B200
- Post-training with SFT + LoRA
- Full code and data, open source
With ~$11 of compute on an H200 we ran an initial training run, enough to validate the full architecture and pipeline.
Blog post: https://aquiles-ai.vercel.app/blog/tinyqwen-from-scratch
Technical feedback welcome, especially from anyone looking to replicate the pipeline with more compute.Organizations
