Instructions to use mkurman/NeuroBLAST-V3-0.6B-SYNTH-EC-144B-TOK with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mkurman/NeuroBLAST-V3-0.6B-SYNTH-EC-144B-TOK with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="mkurman/NeuroBLAST-V3-0.6B-SYNTH-EC-144B-TOK", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("mkurman/NeuroBLAST-V3-0.6B-SYNTH-EC-144B-TOK", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use mkurman/NeuroBLAST-V3-0.6B-SYNTH-EC-144B-TOK with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mkurman/NeuroBLAST-V3-0.6B-SYNTH-EC-144B-TOK"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mkurman/NeuroBLAST-V3-0.6B-SYNTH-EC-144B-TOK",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/mkurman/NeuroBLAST-V3-0.6B-SYNTH-EC-144B-TOK

SGLang

How to use mkurman/NeuroBLAST-V3-0.6B-SYNTH-EC-144B-TOK with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "mkurman/NeuroBLAST-V3-0.6B-SYNTH-EC-144B-TOK" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mkurman/NeuroBLAST-V3-0.6B-SYNTH-EC-144B-TOK",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "mkurman/NeuroBLAST-V3-0.6B-SYNTH-EC-144B-TOK" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mkurman/NeuroBLAST-V3-0.6B-SYNTH-EC-144B-TOK",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use mkurman/NeuroBLAST-V3-0.6B-SYNTH-EC-144B-TOK with Docker Model Runner:
```
docker model run hf.co/mkurman/NeuroBLAST-V3-0.6B-SYNTH-EC-144B-TOK
```

NeuroBLAST-V3 0.6B SYNTH EC 144B TOKENS

⚠️ EXPERIMENTAL EARLY CHECKPOINT ⚠️

This is an Early Checkpoint (EC) of the NeuroBLAST V3 architecture, a novel hybrid model designed with a biologically inspired "cortical" structure.

This specific checkpoint represents the "decay" phase of training. It has been trained on longer contexts with a lower learning rate than the previous one and is intended for architectural evaluation and research purposes.

Model Details

Architecture: NeuroBLAST V3 (Custom Hybrid Architecture)
Checkpoint Step: 40,000
Parameters: 596,728,320
Num layers: 72
- Sensory layers: 24
- Associative layers: 32
- Motor layers: 16
Hidden size: 512
Vocab size: 65538
Intermediate size: 3072
Num attention heads: 16
Num kv heads: 8
Head dim: 128
Tie word embeddings: False

Architecture Highlights

NeuroBLAST differs from standard Transformers by utilizing a three-stage cortical design:

Sensory Cortex: Hybrid layers alternating between Attention and Dilated Causal 2D Convolutions.
Associative Cortex: Hybrid layers with alternating RoPE usage.
Motor Cortex: Pure Attention layers.
Deep Residual Bridges: Long-range residual connections injecting the original embeddings (and their negations) between cortical stages to improve signal propagation.

Training Details

This model is currently being trained using the Google TPU Research Cloud (TRC).

Dataset: PleIAs/SYNTH
Tokens Processed: ~144 Billion
Hardware: TPUv4-16
Training Time: ~13 Days
Effective Batch Size: 1024
Context Length: 2048 tokens (Current phase)
Learning rate: 2e-4
Weight decay: 0.01
Optimizer: AdamW
Precision: BFloat16
Current State: Decay phase

Usage

Note: You must use trust_remote_code=True as this model utilizes custom modeling code (modeling_neuroblast.py).

import torch
from transformers import AutoTokenizer, TextStreamer, AutoModelForCausalLM

model_id = "mkurman/NeuroBLAST-V3-0.6M-SYNTH-EC-144B-TOK"

# Load the tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_id)

# Load the model with custom code trust
model = AutoModelForCausalLM.from_pretrained(
    model_id, 
    torch_dtype=torch.bfloat16, 
    device_map='cuda', 
    trust_remote_code=True
).eval()

streamer = TextStreamer(
    tokenizer, skip_prompt=False, decode_kwargs={"skip_special_tokens": False}
)

# Prepare input
input_ids = tokenizer.apply_chat_template(
    [{"role": "user", "content": "what is hypertension?"}], 
    tokenize=True, 
    return_tensors="pt", 
    add_generation_prompt=True
)

print(f"Input IDs: {input_ids}")

# Generate
with torch.no_grad():
    outputs = model.generate(
        input_ids=input_ids.to(model.device),
        max_new_tokens=128,
        streamer=streamer,
        use_cache=True,
        # Important: Keep repetition_penalty at 1.0 for this early checkpoint
        repetition_penalty=1.0, 
    )