Qwen3.6-27B-Heretic2-Uncensored-Finetune-Thinking-mlx-3bit

MLX 3-bit conversion of DavidAU/Qwen3.6-27B-Heretic2-Uncensored-Finetune-Thinking for native Apple Silicon inference.

Optimized for 16GB unified memory Macs — fits comfortably within oMLX and LM Studio memory ceilings while preserving vision, thinking mode, and uncensored behavior.

Model details

Property	Value
Base model	Qwen/Qwen3.6-27B
Abliteration	Heretic2 by DavidAU (~95-99% refusal removal)
Format	MLX 3-bit (q-bits=3, group-size=64)
Parameters	~27B
Vision	✅ Preserved (multimodal)
Thinking mode	✅
Context window	262K tokens (extensible to 1M)
Languages	English, Chinese (140+)
Architecture	qwen3_5

Why 3-bit?

The 4-bit version (14GB) exceeds the effective memory ceiling on 24GB unified memory Macs when accounting for KV cache and system overhead. The 3-bit version (11GB) runs comfortably within a 16GB oMLX ceiling, leaving room for context and vision processing.

Quality difference between Q4 and Q3 on a 27B model is minimal (~4%) — imperceptible for coding, scripting, YAML, and infrastructure tasks.

Benchmark highlights (base model)

Benchmark	Score
SWE-bench Verified	77.2%
Terminal-Bench 2.0	59.3% (matches Claude 4.5 Opus)
SkillsBench	48.2%
GPQA Diamond	87.8%
Context window	262K native / 1M extended

Requirements

• Apple Silicon Mac (M1 or later)
• Unified memory: 16GB minimum, 24GB recommended
• mlx-vlm installed

pip install mlx-vlm

Usage

Text / Chat

from mlx_vlm import load, generate

model, processor = load("IITheLordII/Qwen3.6-27B-Heretic2-Uncensored-Finetune-Thinking-mlx-3bit")
output = generate(model, processor, prompt="Write a bash script to monitor Proxmox CPU usage.", max_tokens=512)
print(output)

Vision (image input)

from mlx_vlm import load, generate

model, processor = load("IITheLordII/Qwen3.6-27B-Heretic2-Uncensored-Finetune-Thinking-mlx-3bit")
output = generate(
    model,
    processor,
    prompt="Describe what you see in this image.",
    image="path/to/image.jpg",
    max_tokens=256
)
print(output)

Thinking mode

output = generate(
    model,
    processor,
    prompt="<think>\nAnalyze this infrastructure issue step by step.\n</think>",
    max_tokens=1024
)

oMLX

Load directly in oMLX pointing to the downloaded model folder. Recommended ceiling: 13-14GB.

LM Studio

Search IITheLordII/Qwen3.6-27B-Heretic2-Uncensored-Finetune-Thinking-mlx-3bit in LM Studio model browser (MLX filter enabled).

Conversion details

Converted on Apple Silicon M4 Pro (24GB) using:

python3 -m mlx_vlm convert \
  --hf-path DavidAU/Qwen3.6-27B-Heretic2-Uncensored-Finetune-Thinking \
  --mlx-path ./qwen3.6-27b-heretic2-mlx-3bit \
  -q --q-bits 3 --q-group-size 64

Note: mlx-vlm is required (not mlx-lm) to preserve the vision tower.

About Heretic2

Heretic2 is a multi-stage abliteration technique by DavidAU that removes refusal mechanisms while preserving model quality. Achieves ~95-99% refusal removal with minimal KL divergence from the original BF16 weights.

Disclaimer

This model has its refusal mechanisms removed via abliteration. It may generate sensitive or controversial content. Use responsibly and in accordance with applicable laws and the Apache 2.0 license.