---
title: Groq Models - ChatBot
emoji: 📚
colorFrom: yellow
colorTo: blue
sdk: streamlit
sdk_version: 1.56.0
app_file: app.py
pinned: false
license: mit
short_description: Chat with Groq models
---

# Groq Models - ChatBot

A chat application built with **Streamlit + Groq API** to interact with the models available in your Groq account, including multimodal support for vision-capable models.

## Capabilities

- Automatically discovers and lists **all models** available in your Groq account (no hardcoded model filters).
- Dynamically sets `max_tokens` per model using Groq model metadata plus a **free-tier rate-limit map**.
- Applies a runtime cap to completions using the current prompt size and the model TPM budget.
- Shows dynamic **Model characteristics** from the API for the selected model (`context_window`, `max_completion_tokens`, `active`, `created`, and more).
- Shows the latest available **rate limit snapshot** (`remaining requests`, `remaining tokens`, reset times) after chat requests.
- Preserves chat history even when switching models.
- Supports both **streaming** and non-streaming response modes.
- Supports image input for vision-capable models (for example, Llama 4 Scout/Maverick), enabled dynamically per selected model.
- Supports context document uploads (`.txt`, `.md`, `.pdf`) with automatic token-budget trimming per selected model.
- Uses OpenAI `tiktoken` (`cl100k_base`) for token counting/trimming instead of character heuristics.
- Enforces one attached document per conversation and correctly resets uploader state on `Remove Document` and `Clear Chat`.
- Keeps uploaded document context hidden from visible chat bubbles (clean UX), while still sending it to the model.
- User-friendly error handling (auth, rate limit, timeout, network, invalid request) with internal logging for debugging.
- Sidebar with Groq branding (local asset) and chat/session controls.

## Hugging Face Space

Space URL: `https://huggingface.co/spaces/dromerosm/groq-chatbot`
Live app: `https://dromerosm-groq-chatbot.hf.space`

## Duplicate This Space

1. Open the Space on Hugging Face.
2. Click `Duplicate this Space`.
3. In the duplicated Space, go to `Settings -> Secrets`.
4. Add:
   - `GROQ_API_KEY` (required)
5. Start or restart the Space.

## Run Locally

### Requirements

- Python 3.10+ recommended.
- A Groq API key.

### 1) Clone the repository

```bash
git clone https://huggingface.co/spaces/dromerosm/groq-chatbot
cd groq-chatbot
```

### 2) Create a virtual environment and install dependencies

```bash
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```

### 3) Configure environment variables in `.env`

```bash
GROQ_API_KEY=your_groq_api_key
HF_TOKEN=your_optional_huggingface_token
```

### 4) Start the app

```bash
streamlit run app.py
```

Headless mode (similar to Space runtime):

```bash
streamlit run app.py --server.headless true --server.port 7860
```

## Environment Variables

- `GROQ_API_KEY`:
  - Required to load models and send prompts.
- `HF_TOKEN`:
  - Optional for local development.
  - Useful for pushing to the Space over HTTPS.
- `GROQ_MODEL_LIMITS_JSON`:
  - Optional JSON object to override built-in per-model rate limits without editing code.
  - Useful if your Groq org has limits different from the default free-tier map.

## How Context Upload Works

- Upload one file (`.txt`, `.md`, or `.pdf`) in the main area.
- The app extracts plain text, computes tokens with `tiktoken`, and trims the content to fit the selected model budget.
- The document is attached as hidden reference context for your prompts.
- The chat UI keeps showing your original question (not the full document dump).
- To attach a different file, click `Remove Document` (or `Clear Chat`).

## Deploy to Space (git push)

With `HF_TOKEN` set in `.env`, you can push without interactive login:

```bash
set -a; source .env; set +a
cat > /tmp/git-askpass-hf.sh <<'EOF2'
#!/bin/sh
case "$1" in
  *Username*) echo "__token__" ;;
  *Password*) echo "$HF_TOKEN" ;;
  *) echo "" ;;
esac
EOF2
chmod 700 /tmp/git-askpass-hf.sh
GIT_ASKPASS=/tmp/git-askpass-hf.sh GIT_TERMINAL_PROMPT=0 git push origin main
```

Current remote:

```bash
origin https://huggingface.co/spaces/dromerosm/groq-chatbot
```

## Project Structure

- `app.py`: main Streamlit application.
- `requirements.txt`: Python dependencies.
- `assets/groq-logo-from-site.svg`: sidebar logo asset.
- `tmp/evidences/`: local validation screenshots/logs (ignored by git).

## UI Validation (Playwright)

Local validation used in this project covered:

- text chat in streaming mode
- text chat in non-streaming mode
- document upload and context usage
- `Remove Document`
- `Clear Chat`
- sidebar rate-limit snapshot rendering
- guarded image uploader behavior when no vision model is available locally

Example smoke check command:

```bash
npx -y playwright@1.52.0 screenshot \
  --browser chromium \
  --wait-for-selector '[data-testid="stSidebar"] img' \
  --wait-for-timeout 3000 \
  --full-page \
  http://127.0.0.1:7860 \
  tmp/evidences/sidebar-check.png
```

Remote validation after deploy covered:

- text chat on the published Space
- document upload and removal on the published Space
- vision chat on the published Space using `meta-llama/llama-4-scout-17b-16e-instruct`

## Troubleshooting

- `Authentication failed`:
  - Check `GROQ_API_KEY` in `.env` or in Space `Settings -> Secrets`.
- `Invalid request for the selected model`:
  - Verify the selected model supports the current input type.
  - For vision, prefer a normal JPEG/PNG image and keep the base64 payload under the Groq limit.
- No models are shown:
  - Verify your API key access and check for temporary rate limiting.
- Network/timeout errors:
  - Retry, reduce request load, or switch models.
- Rate-limit-related failures:
  - Check the sidebar rate snapshot after a request.
  - Reduce `Max Tokens` or switch to a model with a higher free-tier TPM budget.
- Deploy permission errors:
  - Use a valid `HF_TOKEN` with write access to the target Space.

## Security

- Do not commit `.env` to version control.
- Keep secrets only in local environment variables or HF `Settings -> Secrets`.
- `tmp/` and local evidences are excluded from version control.

## License

MIT