Badhon/MedChatIntent

Medical Conversation Intent Classification

An 18-intent classification dataset for a Bangladeshi hospital/clinic front-desk chatbot, covering the three ways patients actually write:

script	example	rows
bn Bengali script	ডাক্তারের সিরিয়াল লাগবে	2,007
en English	I want to book as a new patient	1,928
bl Banglish (romanized Bangla)	report pathan	1,898
mx code-mixed mid-sentence	amar report ki ready হয়েছে	309

6,142 rows, 18 intents — including an explicit out_of_scope reject class and a safety-critical emergency class. Every row is a distinct string; there are no duplicate texts across the three splits.

⚠️ This is synthetic data. It is a bootstrap for getting a CPU intent classifier off the ground when you have no logs yet, not a substitute for real ones. See Limitations and Emergency before you rely on a number measured here.

Scope, and it is the important part

This is an intent classifier for a hospital's front desk: booking, reports, billing, directions. It is not a diagnostic system. symptom_query exists so the bot can recognize that someone is describing symptoms and route them to a human or a doctor — not so it can answer them. emergency is a separate intent for the same reason: it must be detectable with high recall so the flow can short-circuit to "call 999 / come to the ER now" instead of trying to be helpful.

Dataset structure

Fields

field	type	description
text	string	the user message, 1–18 words (mean 5.2, 95th percentile 9)
intent	class_label	one of 18 labels (below)
script	string	bn | en | bl | mx — writing system, useful for per-script error analysis

script is metadata, not a training feature. It exists so you can report accuracy per writing system, which is where the interesting failures hide — Banglish and code-mixed rows are consistently harder than either monolingual form.

Splits

from datasets import load_dataset
ds = load_dataset("Badhon/BanglaMedicalIntent")
# DatasetDict({train: 4494, validation: 809, test: 839})

split	rows	bn	en	bl	mx
train	4,494	1,464	1,425	1,387	218
validation	809	272	254	244	39
test	839	271	249	267	52

Each split carries all four writing systems in roughly the same proportion, so per-script accuracy on test is comparable to per-script accuracy on train.

The splits are disjoint at template level, not row level. Each template is assigned to exactly one split before it expands into surface rows, so no test row is a respelling, recasing, code-mixing or politeness-affixed variant of a training row. Leakage is also blocked on a punctuation/case/affix-insensitive canonical form.

A dataset built the naive way — expand first, split rows randomly — reports ~99.9% test accuracy that is pure memorization. Under template-level splitting the shipped transformer scores 0.695 on test and 0.738 on the hand-written holdout. Those two numbers agreeing is what tells you the benchmark is measuring generalization.

Label distribution

intent	train	val	test	total	description
doctor_info	382	64	73	519	specialty, qualifications, sitting days, consultation fee
symptom_query	373	73	48	494	describes how they feel — routed, never answered clinically
test_diagnostic	333	68	60	461	lab tests, imaging, packages, prep, fasting, cost
appointment_book	314	51	60	425	wants a new appointment/serial
emergency	288	46	55	389	immediate danger — overrides every other label
greeting	255	49	47	351	opener, whole message
goodbye	251	45	50	346	sign-off
vaccination	242	38	64	344	schedule, availability, child immunization, certificates
report_result	221	38	50	309	a report they are already waiting for — ready? send it?
hospital_info	217	47	37	301	location, timings, departments, ambulance number, parking
appointment_manage	209	43	41	293	change, cancel, or check an existing booking
medicine_query	207	40	41	288	dosage, timing, side effects, substitutes, refills, stock
admission_discharge	209	38	40	287	beds, cabins, ICU, discharge process, attendant/visiting rules
thanks	214	32	38	284	gratitude, whole message
complaint	197	37	40	274	grievance with no specific remedy asked
billing_insurance	205	35	32	272	cost of admission, bill payment, insurance, receipts
out_of_scope	186	37	35	258	chitchat, other domains, noise
agent_request	191	28	28	247	escalate to a human

Roughly balanced by design (per-intent row caps during generation).

Label boundaries

Documented in full in the domains/medical.py docstring; the pairs that get confused most, in order:

• appointment_book vs appointment_manage — a new serial vs changing, cancelling or checking an existing one. kobe amar serial is manage.
• doctor_info vs appointment_book — facts about a doctor vs clearly trying to book. If they are trying to book, prefer appointment_book.
• test_diagnostic vs report_result — the test hasn't happened yet (what tests, prep, cost) vs it already happened and they want the result.
• symptom_query vs emergency — see below.
• hospital_info vs admission_discharge — logistics for visitors and outpatients vs inpatient beds, ICU, and discharge.
• complaint — dissatisfied with no actionable request fitting above.

Overriding rules, in priority order:

1. If it is an emergency, it is emergency, whatever else it also is. A message that is both a symptom description and an emergency is emergency.
2. A greeting glued onto a real request is labeled by the request.

out_of_scope

The reject class, and the reason to prefer this dataset over a 17-intent one. A closed-set softmax must put ~1.0 of its probability mass on some label, so a model without a reject class answers tomar basa kothay? as a confident complaint. No confidence threshold fixes that, because the model was never given a way to express "none of the above".

Coverage spans bot-directed chitchat (tumi ki manush), other industries and domains (weather, cricket, prayer times, politics), general-assistant requests (write a poem, do this maths), and meta/noise (test test, keyboard mash, emoji-only, hmm).

Deliberately not out_of_scope: profanity aimed at the hospital (that is complaint — actionable, route to a human), and vague-but-clinical fragments.

The class is capped at the same size as the others on purpose. An oversized reject class raises the false-fallback rate — real patients routed to "I don't understand" — which costs more in production than a missed rejection.

Emergency: read this before deploying

Do not ship the emergency path on this model alone.

Explicit cardiac and stroke templates were added after the first training run routed "chest pain radiating to the left arm" to symptom_query and a stroke description to admission_discharge; that lifted test recall from 0.47 to 0.69. Measured on the hand-written holdout, emergency recall is ~0.7–0.8: it catches chest pain, bleeding, seizures and accidents, but it has been observed to miss plain phrasings like "we need an ambulance at once". Synthetic templates teach the phrasings someone thought to write down, and the tail of how people actually report a crisis is longer than that.

Before deployment, gate the emergency path with, at minimum:

• a keyword/regex pre-filter (ambulance, 999, unconscious, not breathing, bleeding, chest pain, and the Bangla/Banglish equivalents) that fires regardless of what the classifier says;
• a low probability threshold on this class, biased hard toward recall; and
• real chat logs replacing these templates as soon as you have them.

A false positive costs one unnecessary escalation. A false negative does not cost the same thing.

Evaluation

Do not report the test split alone. It is template-disjoint from train, which makes it honest, but it still only answers "can you generalize across our own templates". Pair it with the hand-written medical holdout in domains/medical.py (159 items, not shipped as a split because it must never be trained on), which is itself split so that tuning and reporting use different sentences:

• DEV_HOLDOUT (84) — tune against this: thresholds, hyperparameters, model selection
• TEST_HOLDOUT (75) — read once, when you are done

INTENT_DOMAIN=medical python transformer_model/eval_holdout.py

Every holdout item is written by hand to share no template with the generated data, and the generator enforces this: any generated row matching a holdout item is dropped at source, so promoting a good holdout sentence into a template cannot silently contaminate training.

Report, at minimum: overall accuracy, macro F1, emergency recall separately, OOS recall, and the false-fallback rate (in-scope inputs wrongly sent to fallback). Overall accuracy alone hides both classes that matter here.

How it was built

Templates → bounded slot fills → sampled surface variants, with the split assigned at step one. Stages that exist because real messages have properties templates don't:

• Code-mixing — a Banglish→Bengali lexicon flips a random 40–80% subset of words mid-sentence. Latin loanwords (report, test, serial, ICU, appointment) are deliberately excluded from the lexicon: Bangladeshi users type those in Latin even inside an otherwise-Bengali sentence, and that asymmetry is the pattern worth learning.
• Phonetic noise — Banglish misspelling is sound-level substitution (bh↔v, sh↔s, ph↔f), dropped vowels (kemon→kmon) and word-boundary drift, not random character swaps.
• Fragments — context-free follow-up turns (kobe?, koto?, ready?) where the intent rides on 1–4 words.
• Glued social openers — assalamu alaikum apu amar report ready hoyeche ki, labelled report_result.
• Rambling preambles — a sentence of context before the actual question, so the model sees inputs longer than 8 words.

Reproduce with python generate_domain_data.py medical (seeded, deterministic). Adding a template to one intent does not reshuffle any other intent's split assignment.

Limitations and bias

Please read this section before using the dataset as a benchmark.

• Synthetic. Generated from hand-written templates, not collected from users. It encodes one author's model of how patients write, including its blind spots. A model at 0.70 here is not a model at 0.70 in production.
• Front desk only. No clinical content, no diagnosis, no triage beyond detecting that a message is an emergency. Nothing here supports answering a medical question.
• Short inputs. Mean under 5 words. Real patients describing symptoms write much longer messages, and models trained here will be poorly calibrated on them — which is exactly where emergency and symptom_query live.
• Under-represented code-mixing. 309 mx rows (5%) versus a real inbox where code-mixing is far more common than that.
• Bangladesh-specific. Hospital and department vocabulary, ambulance number (999), cities, festivals, and honorifics (vai, apu) are all local.
• Romanization is not standardized. Banglish has no orthography. The phonetic-variant generator covers a fraction of real spelling space, and its substitution rules are hand-picked rather than learned from data.
• Label noise on the overlapping boundaries. The tie-breaks above are applied consistently by construction, but they are one defensible reading of genuinely ambiguous cases.
• No inter-annotator agreement figure, because there was one annotator.
• No PII — no real patient names, IDs, phone numbers or addresses, and no real medical records. Doctor names and IDs are made-up strings from a fixed list.

Intended and out-of-scope uses

Intended: bootstrapping a Bangla/Banglish hospital front-desk intent classifier before you have logs; benchmarking small CPU models (fastText, distilled transformers) on code-mixed short text.

Not intended: as evidence of production accuracy; as a general Bangla NLP benchmark; as any part of a diagnostic, triage, or clinical decision system; and never as the sole gate on an emergency path. Any deployment touching patient safety needs a human in the loop, a keyword pre-filter, and a calibrated reject threshold.

Citation

@misc{banglamedicalintent,
  title  = {BanglaMedicalIntent: Bangla / English / Banglish Medical Front-Desk Intent Classification},
  year   = {2026},
  note   = {Synthetic dataset, 18 intents, template-disjoint splits},
  howpublished = {\url{https://huggingface.co/datasets/Badhon/BanglaMedicalIntent}}
}

Licensing

CC BY-NC-SA 4.0 (Creative Commons Attribution-NonCommercial-ShareAlike 4.0).

The content is wholly generated from templates written for this repository, so there is no upstream corpus license to inherit. What the terms mean in practice:

• BY — attribute the source when you use or redistribute it.
• NC — no commercial use. Training a classifier that serves a commercial hospital or clinic is a commercial use. If this dataset is meant to be deployable inside a business, cc-by-sa-4.0 or apache-2.0 is the licence you want instead.
• SA — derivatives, including modified or extended versions of the data, must carry the same licence. Whether a model trained on it counts as a derivative work is legally unsettled and jurisdiction-dependent.

Add a LICENSE file containing the full CC BY-NC-SA 4.0 text alongside this card; HuggingFace renders the tag either way, but the file is what makes the grant explicit to anyone who downloads the CSVs on their own.