Coder — Qwen3.6-4B Coding & Autonomous Software Engineering Model

A fine-tuned 4B-parameter variant of Qwen3.6 specialized for autonomous software engineering, code generation, debugging, repository exploration, terminal/shell agent behavior, Git operations, database/SQL, web development, DevOps, security, performance, architecture, and full multi-file feature implementation.

Verified facts: This repository contains qwen3.6--coder.safetensors (8.8 GB, BF16), a 20,000-example instruction dataset (coding-finetune-dataset/dataset.jsonl), training notes (qwen3.6-4B.txt), and dataset creation docs (coding-finetune-dataset/CREATE_INSTRUCTIONS.md).

Model Overview

Coder is intended as an autonomous software engineering assistant that can investigate repositories, identify problems, implement solutions, edit files, run tests, use Git, write SQL, and complete multi-step tasks independently. It was fine-tuned on a comprehensive 20,000-example dataset covering 20 engineering categories (A–U) across Python, JavaScript/TypeScript, Java, C/C++, Go, Rust, Bash, SQL, HTML/CSS, React/Node, DevOps, and more.

What makes it different: The dataset is designed around realistic workflow scenarios — not just code snippets — including debugging loops, repository exploration, file editing with minimal changes, multi-file feature implementation, verification steps, and repair loops.

Model Details

Property	Value (verified from repository)
Model type	Causal language model (fine-tuned)
Architecture	Qwen3.6 (Transformer, GQA)
Parameters	4B (from `qwen3.6-4B.txt`)
Layers	5 (from weight key analysis: `layers.0`–`layers.4`)
Hidden size	5120 (from weight shapes)
Attention heads	32 Q heads, 8 KV heads, head_dim=128 (from `q_proj`/`k_proj`/`v_proj` shapes)
Context length	Not specified in repository
Vocabulary size	248,320 (from `embed_tokens.weight` / `lm_head.weight`)
Data type	BF16 (`tensor type: BF16` per notes)
Quantization	Not specified
Base model	Qwen3.6 (notes reference `Qwen/Qwen3.6-27B` for the upstream draft head; this weight file is a 4B fine-tune)
Training method	Supervised fine-tuning (SFT) on 20k instruction examples
License	Apache 2.0 (`coding-finetune-dataset/README.md`)

Intended Use

Appropriate use cases based on the actual dataset and model:

Code generation (Python, JS/TS, Java, C/C++, Go, Rust, SQL, Bash, HTML/CSS, React, Node)
Bug finding and debugging workflow (reproduce, isolate, identify root cause, patch, test, verify)
Repository exploration (structure analysis, framework identification, dependency tracing)
File editing (minimal changes, preserving style, updating imports/tests)
Multi-file feature implementation (auth, payments, uploads, notifications, caching, logging, error handling)
Terminal/shell agent behavior (pwd, ls, find, grep, git, python, npm, docker, etc.)
Git operations (status, diff, log, branch, rebase, cherry-pick, bisect)
Database / SQL (SELECT, INSERT, UPDATE, JOIN, CTE, window functions, transactions, indexes)
Web development (React + TypeScript frontend; Express / FastAPI backend)
DevOps / CI/CD (Docker, Docker Compose, GitHub Actions, deployment)
Security review (SQL injection, XSS, CSRF, auth mistakes, secret leakage, path traversal)
Performance optimization (slow algorithms, memory, query optimization, caching, concurrency)
Architecture planning (monoliths vs microservices, event-driven, queues, caching)
Full autonomous coding tasks (iterative debugging, verification loops)

Not Intended For

Real-time production inference without evaluation (no benchmark results provided)
Medical, legal, or high-stakes decision-making without human review
Generating malicious code (security risks remain; model can discuss vulnerabilities but should not be used to create exploits)
Guaranteed factual accuracy (hallucinations and incorrect code may occur; always verify outputs)
Replacement for professional code review (model assists, does not replace expert review)
Use in contexts requiring guaranteed deterministic output (sampling parameters not verified)

Training

Verified from coding-finetune-dataset/README.md and CREATE_INSTRUCTIONS.md:

Base model: Qwen3.6 (4B parameter scale; upstream reference is Qwen3.6-27B for the draft-head variant documented in notes)
Dataset: coding-finetune-dataset/dataset.jsonl — 20,000 examples (full dataset at dataset_full_20k.jsonl; deduplicated at dataset_deduplicated.jsonl)
Dataset source: Synthetic generation via autonomous agent process (generate_full_dataset.py, generate_examples.py, merge.py); categories A–U with ~1,000 examples each; languages distributed across Python (20%), JS (15%), TS (15%), SQL (10%), Java (8%), C++ (7%), C (5%), Go/Rust/Bash (5% each), Other (5%)
Example format: Chat-style JSON lines with messages array (system, user, assistant) and metadata (category, language, difficulty, task_type, tools_used, requires_terminal/editing/testing, generated_at, version, example_id)
Difficulty: Easy 15%, Medium 35%, Hard 35%, Expert 15% (per CREATE_INSTRUCTIONS.md)
Training objective: Supervised fine-tuning on instruction-following and autonomous engineering workflows
Framework: Not specified in repository
Hardware / GPUs / duration / epochs / batch size / learning rate: Not specified in repository (only dataset generation scripts and statistics exist; no training_config.json or trainer_state.json found)

Not verified from repository: Training hyperparameters (learning rate, batch size, epochs, optimizer, scheduler, sequence length, GPU count, training duration, framework, exact base-model initialization weights beyond the 4B file).

Dataset

Property	Value (verified)
Name	`coding-finetune-dataset`
Source	Generated locally (scripts in `coding-finetune-dataset/scripts/`)
Size	20,000 examples (full dataset 29.6 MB; `dataset.jsonl`)
Format	JSON Lines (`.jsonl`) with chat-style messages
Categories	20 categories (A–U): Code Generation, Bug Finding, Terminal/Shell, Repository Exploration, File Editing, Multi-File Implementation, Debugging Workflow, Testing, Git, Database/SQL, Web Development, DevOps, Security, Performance, Code Review, Architecture, Requirement→Implementation, Error Message Interpretation, Agent Tool Use, Verification, Repair Loop, Full Autonomous Coding Tasks
Languages	Python, JavaScript, TypeScript, SQL, Java, C++, C, Go, Rust, Bash/Linux, HTML/CSS, React, Node, Express, FastAPI
Difficulty	Easy / Medium / Hard / Expert
Tools used (per example)	`read_file`, `edit_file`, `run_tests`, `run_terminal`, `git_*`, etc.
Deduplicated version	`dataset_deduplicated.jsonl` (29.5 MB)
Validation script	`validate_dataset.py`
Sample	`samples/sample_100.jsonl` (100 examples, all Bug Finding / JavaScript / Hard)
Batch scripts	`batch_001.jsonl` through `batch_024.jsonl` (generated in batches)

Data licensing: Apache 2.0 (from dataset README). The dataset was generated through simulated coding scenarios and does not contain real-world sensitive information.

Architecture

The weight file (qwen3.6--coder.safetensors) contains the following verified architecture components (extracted from tensor names and shapes):

Transformer decoder with 5 layers (layers.0–layers.4)
Hidden size: 5120
MLP intermediate: 17408 (Gated MLP with gate_proj, up_proj, down_proj)
Attention: Multi-head self-attention with GQA (32 Q heads, 8 KV heads, head_dim=128)
Normalization: RMSNorm / LayerNorm (norm.weight, hidden_norm.weight, input_layernorm.weight, post_attention_layernorm.weight)
Embeddings: embed_tokens.weight [248320, 5120]
LM head: lm_head.weight [248320, 5120]
Speculative-decoding head (DSpark-style): confidence_head.proj.weight [1, 5376], markov_head.markov_w1.weight [248320, 256], markov_head.markov_w2.weight [248320, 256]
Position encoding: Not specified in weights (likely RoPE or ALiBi; not verified)
Activation: Not specified (GELU / SwiGLU typical for Qwen)

Note: The file is named qwen3.6--coder.safetensors (double dash) and is 8.8 GB. The notes (qwen3.6-4B.txt) reference a 27B upstream draft head (abstract-extraordinary/Qwen3.6-27B-DSpark) and describe speculative-decoding benchmarks at 8-speculative-token configuration. The weight file here is a 4B fine-tune for coding/autonomous-agent use.

Tokenizer

Not verified from repository: No tokenizer.json, tokenizer_config.json, special_tokens_map.json, or generation_config.json is present in this directory. The dataset uses chat-format messages (system/user/assistant) consistent with Qwen3's chat template, but the exact tokenizer configuration must be loaded from the base Qwen3.6 model or provided separately.

Recommended token usage: Load tokenizer from the base Qwen/Qwen3.6-27B (or equivalent 4B tokenizer) and apply the standard Qwen3 chat template (<|im_start|>system...<|im_end|>...) when formatting messages.

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

# Replace with your actual upload path
model_id = "AnuragMahajan/coder-4b"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto"
)

messages = [
    {"role": "system", "content": "You are Coder, an autonomous software engineering assistant..."},
    {"role": "user", "content": "Find the JWT expiration issue in src/auth/middleware.js and fix it."}
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

outputs = model.generate(
    inputs,
    max_new_tokens=1024,
    do_sample=True,
    temperature=0.7,
    top_p=0.95
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Note: The repository does not include tokenizer.json. Use the base Qwen3.6 tokenizer (or download it separately) and verify chat-template compatibility before inference.

Generation Parameters

Not verified from repository. Recommended values (based on dataset notes and typical Qwen3 settings):

temperature: 0.7 (used in dataset regeneration notes)
top_p: 0.95
max_new_tokens: 256–1024 depending on task complexity
do_sample: True for diverse responses; False (greedy) for deterministic debugging
repetition_penalty: 1.05–1.1 (typical)

Hardware Requirements

Not verified exactly. Estimates based on 4B parameters + BF16:

Minimum: GPU with ~10 GB VRAM (BF16, 4B model ~8 GB weights; inference overhead requires additional memory)
Recommended: 16–24 GB VRAM GPU (NVIDIA A10, RTX 4090, L4, A100 40GB for batch inference)
CPU: Possible with torch_dtype="float32" but very slow; not recommended
RAM: 16 GB system RAM minimum; 32 GB for dataset loading / training
Disk: ~9 GB for model weights + ~30 MB for dataset + dependencies
Quantized options: Not present in repository; can try bitsandbytes 4-bit / 8-bit if needed

Quantization

No quantized versions (quantize_*.py, *.gguf, *.q4_*.safetensors, adapter_config.json) are present in the repository. The weight file is full-precision/bf16 (qwen3.6--coder.safetensors).

Performance / Benchmarks

No formal benchmark results are currently provided for this fine-tuned model.

The repository contains only dataset-generation statistics (scripts/batch_*_stats.json, dataset.jsonl statistics) and notes referencing upstream DSpark speculative-decoding benchmarks (for a separate 27B draft-head model, not this 4B fine-tune). No accuracy, perplexity, BLEU, ROUGE, HumanEval, MBPP, or agent-evaluation scores exist in this directory.

Limitations

Evaluation: No formal evaluation results provided for this fine-tuned model.
Context length: Not verified from repository.
Base model dependency: Fine-tuned from Qwen3.6; exact initialization weights not documented beyond the 4B file.
Tokenizer: Not included; must use base Qwen3.6 tokenizer.
Quantization: Not supported out of this repo.
Hallucination: Like all LLMs, output may contain incorrect code, outdated APIs, or missing error handling. Always verify generated code with tests.
Domain: Trained primarily on simulated software-engineering scenarios; performance in specialized domains (embedded, real-time, high-security) is not guaranteed.
Safety: The dataset includes security-related examples (SQL injection, XSS, auth mistakes) for educational/debugging purposes. Do not use model outputs to construct exploits.

Bias, Safety and Responsible Use

The dataset is synthetic (simulated coding scenarios) and does not contain real-world personal data. However:

The model may reproduce biases present in synthetic training data (e.g., over-representation of common frameworks, under-representation of niche languages).
Security examples describe vulnerabilities; the model should not be used to generate malicious payloads.
Generated code should always be reviewed by a human engineer and tested before deployment.
No formal safety evaluation (toxicity, fairness, adversarial robustness) has been conducted for this fine-tune.

Repository Structure

.
├── qwen3.6--coder.safetensors          # Fine-tuned weights (BF16, 8.8 GB)
├── README.md                           # Training notes, datasets, upstream references, benchmark notes

Citation

If using this model or dataset, please cite sources present in repository:

Base architecture: Qwen3.6 (Qwen Team)
Upstream draft-head reference: abstract-extraordinary/Qwen3.6-27B-DSpark
Dataset: coding-finetune-dataset/ (local generation via scripts in repo)
Licensing: Apache 2.0

Changelog

v1.0.0 (initial release — repository state 2026-09-26)

Fine-tuned weights: qwen3.6--coder.safetensors
Instruction dataset: 20,000 examples (coding-finetune-dataset/)
License: Apache 2.0
Notes: qwen3.6-4B.txt (datasets, upstream references, benchmark context)

Downloads last month: -; Downloads are not tracked for this model. How to track

Model tree for AnuragMahajan/coder-4b

Base model

Qwen/Qwen3.6-27B

Finetuned

(400)

this model