Instructions to use AutomatosX/AX-Gemma-4-31B-IT-MLX-OptiQ-4bit-Assistant-MTP with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AutomatosX/AX-Gemma-4-31B-IT-MLX-OptiQ-4bit-Assistant-MTP with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("AutomatosX/AX-Gemma-4-31B-IT-MLX-OptiQ-4bit-Assistant-MTP")
config = load_config("AutomatosX/AX-Gemma-4-31B-IT-MLX-OptiQ-4bit-Assistant-MTP")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use AutomatosX/AX-Gemma-4-31B-IT-MLX-OptiQ-4bit-Assistant-MTP with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "AutomatosX/AX-Gemma-4-31B-IT-MLX-OptiQ-4bit-Assistant-MTP"

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "AutomatosX/AX-Gemma-4-31B-IT-MLX-OptiQ-4bit-Assistant-MTP"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use AutomatosX/AX-Gemma-4-31B-IT-MLX-OptiQ-4bit-Assistant-MTP with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "AutomatosX/AX-Gemma-4-31B-IT-MLX-OptiQ-4bit-Assistant-MTP"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default AutomatosX/AX-Gemma-4-31B-IT-MLX-OptiQ-4bit-Assistant-MTP

Run Hermes

hermes

OpenClaw new

How to use AutomatosX/AX-Gemma-4-31B-IT-MLX-OptiQ-4bit-Assistant-MTP with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "AutomatosX/AX-Gemma-4-31B-IT-MLX-OptiQ-4bit-Assistant-MTP"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "AutomatosX/AX-Gemma-4-31B-IT-MLX-OptiQ-4bit-Assistant-MTP" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

AX Gemma 4 31B IT MLX OptiQ 4-bit Assistant MTP

Parameter count: approximately 31.27B logical target parameters (31B class). 4-bit is the target quantization precision, not a 4B model-size claim. The separately packaged assistant is an acceleration component and is not included in the target count.

This is a self-contained MLX package for Apple Silicon. It combines the pinned upstream OptiQ target with its paired BF16 Gemma assistant for AX Engine multi-token prediction (MTP) / speculative decoding.

Try this model locally with AX Engine.

Attribution and changes

AutomatosX did not train Gemma 4, create the OptiQ quantization, or train the assistant. The target comes from mlx-community/gemma-4-31B-it-OptiQ-4bit at revision 23616162c5a8f928cac5b21d3e974d1dbc0b9877. The assistant comes from mlx-community/gemma-4-31B-it-assistant-bf16 at revision 28e92270316e89288579ec59c17939541d9ca433.

AutomatosX packaged the pinned target and assistant together, generated native manifests, applied the target tokenizer to the assistant subtree so both sides use identical token IDs, and added the exact-pairing contract. The target and assistant weights themselves are unchanged. The upstream OptiQ card is preserved as UPSTREAM_README.md.

No new training or benchmark results are claimed by AutomatosX.

Package details

Property	Value
Target format	MLX Safetensors
Target quantization	OptiQ mixed 4/8-bit, group size 64
OptiQ allocation	226 components at 4-bit; 184 at 8-bit
Achieved target BPW	5.1992
Assistant precision	BF16
Pairing	Exact
Maximum draft depth	1
Configured context	262,144 tokens
Intended hardware	Apple Silicon

Download and serve

hf download AutomatosX/AX-Gemma-4-31B-IT-MLX-OptiQ-4bit-Assistant-MTP \
  --local-dir ./AX-Gemma-4-31B-IT-MLX-OptiQ-4bit-Assistant-MTP

ax-engine doctor \
  --mlx-model-artifacts-dir ./AX-Gemma-4-31B-IT-MLX-OptiQ-4bit-Assistant-MTP

ax-engine serve ./AX-Gemma-4-31B-IT-MLX-OptiQ-4bit-Assistant-MTP --port 31418

The download is approximately 23 GB. AX Engine reads ax_gemma4_assistant_mtp.json, loads the drafter from assistant/, and uses the target model to verify every accepted token. Do not use assistant/ by itself as a general-purpose chat model.

The server exposes an OpenAI-compatible API. Consult the AX Engine repository for installation and API examples.

Validation and provenance

Validated on macOS arm64 with AX Engine 6.9.0 on 2026-07-20:

AX artifact doctor: ready, with no model issues
Safetensors headers, data bounds, and index mappings: passed
Target weight shards: byte-exact against the pinned OptiQ source
Assistant weight: byte-exact against the pinned assistant source
Pairing contract: exact

See ax_provenance.json for immutable source revisions, transformations, and artifact hashes.

Quantization can change model quality, and speculative-decoding acceptance depends on the workload. Evaluate this package on your own tasks.

License

Apache License 2.0. See LICENSE, the Gemma 4 license page, and the official Google model cards for limitations and responsible-use guidance.

Downloads last month: 329

Safetensors

Model size

31B params

Tensor type

BF16

U32

MLX

Hardware compatibility

4-bit

Model tree for AutomatosX/AX-Gemma-4-31B-IT-MLX-OptiQ-4bit-Assistant-MTP

Base model

google/gemma-4-31B

Finetuned

google/gemma-4-31B-it

Quantized

(290)

this model

Collection including AutomatosX/AX-Gemma-4-31B-IT-MLX-OptiQ-4bit-Assistant-MTP

AutomatosX MLX Model Catalog

Collection

Official AutomatosX MLX releases for Apple Silicon. OptiQ mixed 4/8-bit packages sit next to uniform 4bit/6bit siblings. • 31 items • Updated 4 days ago