2026-08-23 · 7 min · medical-ai · evaluation · model-cards · deployment · explainer
Taimi-14B-Med is described as a medical-specialized language model for healthcare institutions and service robots: medical indicator Q&A, nursing guidance, abnormal-indicator alerting, automated nursing record registration. It reports 74.0% on CMB, 77.4% on CMExam — above the human baseline its paper quotes — and roughly 60 on human-reviewed clinical dialogues.
It also says this, in a blockquote near the top:
Current release v0.1.0 consists of the base-model weights with a rebranded configuration. Post-trained medical weights will be released in a later version (v0.2.0+), and this card will be updated with fresh evaluation results at that time.
And again, at the bottom of the evaluation section: "v0.1.0 contains base-model weights; the evaluation above reflects base-model capability."
So the benchmark table is measuring Qwen2.5-14B-Instruct-AWQ under a different name. The card says so. Twice. That is more disclosure than this genre usually offers — and the structure of the page, where a name and a results table read as a medical model while the correction lives in a blockquote, is worth looking at carefully, because that structure is everywhere and usually undisclosed.
| Model | TMiRob/Taimi-14B-Med · Apache 2.0 |
| Base | Qwen/Qwen2.5-14B-Instruct-AWQ · 14.7B params (13.1B non-embedding) |
| Quantization | AWQ 4-bit · 10.31 GB of weights |
| Architecture | 48 layers · GQA 40 Q heads / 8 KV heads · RoPE / SwiGLU / RMSNorm |
| Serving | vLLM, OpenAI-compatible · deployed at 4,096 context with fp8 KV cache |
| Measured | 12,537 MiB of 16,303 on an RTX 5080 · ~6.1 GB system RAM · 45–90 s load |
| Status | v0.1.0 is base weights; medical post-training is v0.2.0+ |
What is in the box
Every architectural row is Qwen2.5-14B-Instruct-AWQ’s, because at v0.1.0 the model is Qwen2.5-14B-Instruct-AWQ. What ships new is a serving configuration and a name. The medical part — the entire premise of the release — is the row marked planned.
I want to be careful about what that does and does not mean, because the card is unusually honest about it: the note is at the top, it is repeated at the bottom of the evaluation section, and the licence section correctly attributes Apache 2.0 to the base model. Nobody is hiding anything. What is worth noticing is the structure — a name, a description, and a benchmark table that together read as a medical model, with the correction living in a blockquote. That structure is extremely common and mostly not disclosed at all.
Every architectural row on the card is Qwen2.5-14B-Instruct-AWQ's, because at v0.1.0 the model is Qwen2.5-14B-Instruct-AWQ. What ships new is a serving configuration, a deployment story, and a name. The medical part — the entire premise — is the row marked planned.
I do not think that is a scandal, and I want to be precise about why. The note is at the top. It is repeated at the bottom of the results. The licence section correctly attributes Apache 2.0 to the base model and tells you to comply with its terms. Nothing is concealed.
What is worth noticing is the shape: a specialized name, a specialized description, a results table, and a correction in a quote block. Read the page top to bottom and you learn what it is; skim it — or index it, or cite it, or pick a model from a list of names — and you learn something else. This particular card is one of the honest ones, which is exactly what makes it a good place to notice the pattern.
The benchmarks, and what they can and cannot show
Two of these three benchmarks are multiple-choice exams released in 2023, and the card flags the consequence itself: they may overlap the base model’s training corpus, so the accuracies serve as reproducibility validation rather than as evidence of medical capability. That is the correct framing and it is rarely written down.
Which makes CMDD the row to actually read. It is human-scored on sampled dialogues, judged on relevance, safety and refusal appropriateness — the things that matter in a clinical setting and the things a multiple-choice exam cannot measure. It is also the only row where the model sits clearly below its reference: about 60 against physicians’ roughly 70. A model can pass the licensing exam and still be ten points off a doctor in conversation, and only one of those two facts is a headline.
The headline is CMExam: 77.4% on 6,811 questions from the Chinese physician licensing exam, against paper baselines of 61.6% for GPT-4 and 71.6% for humans. A quantized 14B model six points above the human baseline on a medical licensing exam.
The card's own caveat is the right one and it is not buried:
CMB / CMExam were released in 2023 and may overlap with the base model's training corpus; ACC results serve as reproducibility validation.
That reframes the whole table correctly. These are not measurements of medical capability; they are measurements that the deployment reproduces the base model's known behaviour. Which is a real and useful thing to check — it is how you catch a broken quantization or a mangled chat template — and it is not what a leaderboard number looks like.
Two details that survive the caveat and are worth keeping:
The 30-point gap between single-choice and multi-choice on CMB. 77.1% against 47.2%. Picking one correct answer and picking all the correct answers are different tasks, and only the second resembles the reasoning a clinician does. Whenever a medical benchmark is reported as one number, this is the split hiding inside it.
CMDD is the only row scored by people, and the only one the model loses. Human review of 100 sampled dialogues on relevance, safety and refusal appropriateness, double-scored: roughly 60 for the model against roughly 70 for physician reference answers. The card's read is careful — high stability, low variance across cases, suitable for pre-diagnosis screening, while physician answers score higher on average with larger variance. That last clause is a genuinely useful observation: a consistent-but-worse system and an excellent-but-variable one are different products.
The deployment story is the real content
Strip the model claims and what remains is a competent piece of applied deployment engineering, and it is the part of the card I would actually use.
Strip the model claims and this is a sensible decomposition. Perception stays in tools built for it, the language model reasons over structured output rather than raw pixels, and a 14B model is a plausible size for the job it is actually given. Most medical-AI framing does the opposite and asks one model to do everything.
The four applications on the right are the card’s own list, presented as a set. They are not one risk category. Answering a question about an indicator and writing into a patient record differ by regulation, not by degree, and a card that scopes deployment to healthcare institutions is the right place to say which of the four it considers assistive and which it does not.
The measured resource footprint is stated with a date and a specific card: 12,537 MiB of 16,303 on an RTX 5080, at 77% utilization with --gpu-memory-utilization 0.85 and an fp8 KV cache, ~6.1 GB of system RAM, 45–90 seconds to load. Context deployed at 4,096 rather than the architecture's 32,768, --max-num-seqs 16.
That configuration is the thing worth copying. AWQ 4-bit plus fp8 KV cache plus a deliberately short context is how you fit a 14B model onto a 16 GB consumer card with room for concurrency — and stating the numbers as measured on this date on this hardware rather than as requirements is the difference between a spec sheet and something you can plan against.
What has to be true before v0.2.0 means anything
The benchmarks that establish the claim cannot be the benchmarks in the table. CMB and CMExam are the ones potentially in the base model's pretraining data, so a post-trained v0.2.0 scoring higher on them tells you very little. The evaluation that would matter is CMDD-style human review — the row where v0.1.0 is ten points behind physicians and where contamination is not the explanation.
The safety axis needs to be separated out. CMDD is scored on relevance, safety and refusal appropriateness combined into one number. For a system that raises abnormal-indicator alerts and writes nursing records, the refusal and safety components are not one third of a quality score; they are the deployment gate. A model that is relevant and unsafe and one that is cautious and vague both land near the middle.
Automated nursing record registration is a regulated act in most jurisdictions. The card lists it as an application alongside Q&A and guidance, without distinction. Answering a question about an indicator and writing into a patient record are very different risk categories, and a card that scopes deployment to healthcare institutions should say which of the four listed uses it considers assistive and which it does not.
"Planned as a post-trained derivative" is doing a lot of work. The description says the model is planned as a fine-tuned derivative. That phrasing is accurate for v0.1.0 and it is the kind of accuracy that only becomes visible once you have read the version note — which is the point.
The generalizable bit
This is a small release and I would not be writing about it except that it demonstrates something clearly.
Model cards are read three ways: by people who read them, by people who skim them, and by systems that index them. The first group gets the version note. The second and third get a name, a description and a table of numbers — and those three things, on this page, describe a model that does not exist yet.
The fix is not more disclosure; this card already has more than most. It is that the correction should live where the claim lives. A results table for base-model weights should say so in the table's header, not in a note beneath it. A model named for a specialization it has not yet received should carry that in the name. Taimi's card is close enough to right that the remaining gap is easy to see, which makes it a better teaching example than a dishonest one would be.
And the underlying result — a 4-bit 14B model matching a community-reported 72B on CMB, on a 16 GB consumer card, at 4,096 context — is worth something on its own. It just is not a medical result.