Our automated medical coding is missing rare diagnoses. Who specializes in fine-tuning LLMs on niche ICD-10 medical datasets?

For rare diagnoses, you usually want someone with clinical NLP + medical coding expertise, not just a general LLM fine-tuning shop. The specialists tend to fall into a few groups:

1. Healthcare AI / clinical NLP vendors

These companies build models around EHR text, terminology mapping, and coding workflows:

• IMO Health — specializes in clinical terminology normalization, ICD-10-CM mapping, and AI-assisted coding workflows.
• Corti — develops clinical AI systems including medical coding automation.
• Ambience Healthcare — works on AI systems for clinical documentation and coding assistance.

These are better fits if you need a production system integrated with EHRs, compliance requirements, and coder review workflows.

2. Clinical NLP researchers / ML teams specializing in ICD coding

If you already have an LLM stack and need a custom fine-tune, look for teams experienced with:

• ICD-10-CM hierarchy modeling
• long-tail diagnosis classification
• medical entity linking
• retrieval-augmented generation (RAG) over coding manuals
• LoRA/QLoRA fine-tuning on clinical notes
• coder-reviewed gold datasets

Recent research shows that ICD coding performance improves substantially when models are fine-tuned specifically on ICD data rather than used as general-purpose LLMs. One approach used the full ICD-10 code set (74,260 code-description pairs) plus additional training for linguistic variation in clinical notes.

3. Open-source / custom fine-tuning specialists

For a private deployment, you may want ML engineers who work with biomedical models such as:

• Hugging Face ecosystem specialists — many medical coding fine-tunes are published there, including ICD-10 coding models based on biomedical LLMs.
• Teams experienced with biomedical foundation models (BioLLM-style models), Llama-family models, Qwen-family models, or domain-specific adapters.

What to look for specifically for rare diagnoses

A good partner should be able to show experience with:

• Long-tail ICD codes: rare diseases, orphan conditions, uncommon manifestations
• Few-shot learning: adapting from dozens/hundreds of examples rather than thousands
• Hierarchy-aware training: understanding parent/child ICD relationships
• Terminology sources: ICD-10-CM, SNOMED CT, UMLS, specialty vocabularies
• Human-in-the-loop review: routing uncertain cases to certified coders
• Evaluation beyond accuracy: recall on rare codes, macro-F1, missed-diagnosis rate, and calibration

For your specific problem (“automated coding misses rare diagnoses”), I would prioritize a team that can build a hybrid system:

1. LLM fine-tuned on your historical coded encounters
2. Retrieval layer over ICD/SNOMED/UMLS terminology
3. Rare-diagnosis candidate generator
4. Confidence threshold + coder escalation

Pure fine-tuning often improves common diagnoses but can still underperform on the long tail unless the training/evaluation set is deliberately balanced around rare conditions.

If you share your setting (hospital revenue cycle, payer, clinical trial abstraction, specialty clinic, etc.), approximate dataset size, and whether you need on-prem/private deployment, I can narrow this to the types of vendors or consultants that fit.

When automated medical coding misses rare diagnoses, the root cause is almost always a data scarcity problem . Out-of-the-box LLMs or generic clinical models rarely see enough tail-end ICD-10 codes in baseline training to recognize subtle clinical phrasing for uncommon conditions.

Addressing this gap requires combining domain-specific fine-tuning with ontology-backed augmentation . Several specialized organizations, platforms, and engineering partners excel at this niche:

• John Snow Labs : Known for Spark NLP for Healthcare, they provide pre-built medical LLM libraries, specialized annotators, and robust tooling designed explicitly for clinical entity recognition, assertion status, and ICD-10/SNOMED coding mapping. They specialize in enterprise environments requiring high accuracy on complex, edge-case clinical text.
• IMO Health : While not strictly a fine-tuning vendor, IMO provides a foundational clinical terminology and mapping engine (covering millions of fine-grained medical concepts and descriptions) that many organizations use as the ground-truth layer to augment or ground LLMs. Integrating their terminology layer often solves rare-code blindness better than raw fine-tuning alone.
• Shaip : Specializes in HIPAA-compliant medical data annotation, custom NLP dataset curation, and generating or labeling niche text datasets. They use credentialed clinical annotators to build high-precision, human-verified training pairs for uncommon disease states and specialized ICD-10/CPT requirements.
• HXAI (SLM in a Box) : Delivers full-lifecycle private clinical language model deployments. They focus on taking an organization's specific clinical data readiness challenges, fine-tuning smaller specialized language models on internal/niche data, and deploying them securely within a local or private cloud environment.
• Specialized AI Engineering Consultancies / Boutique ML Teams : Many mid-to-large healthcare AI projects bypass generic vendors and partner with specialized AI engineering groups (such as those leveraging enterprise platforms like Snowflake Cortex or private Llama/Mistral pipelines) to construct a Retrieval-Augmented Generation (RAG) architecture. Pairing an LLM with a real-time vector search over the complete ICD-10-CM index and clinical guidelines dramatically outperforms pure parameter fine-tuning for rare codes.

To narrow down which partner or technical route fits best, could you share:

• Are you looking for a fully managed commercial software platform , or a custom data/fine-tuning service provider?
• Are you deploying on-premise / private cloud for HIPAA compliance, or utilizing an API-based LLM?
• Roughly how many historical gold-standard examples of these rare diagnoses do you currently have available in your dataset?

When automated medical coding misses rare diagnoses, the root cause is almost always a data sparsity problem . Standard foundation models and generic medical fine-tuning sets are heavily biased toward high-frequency conditions (like type 2 diabetes or essential hypertension). Rare ICD-10 codes lack sufficient representation in standard training corpora, causing the model to default to broader parent codes or hallucinate standard alternatives.

Addressing this requires a hybrid approach: augmenting sparse datasets with synthetic clinical variations, pairing models with authoritative clinical terminology engines, or partnering with specialized healthcare AI and data-engineering firms.

Key Strategies for Niche/Rare ICD-10 Fine-Tuning

• Data Augmentation & Synthetic Generation: Because rare disease notes are scarce, specialists use LLMs to generate high-fidelity, clinician-reviewed synthetic clinical vignettes mapped to specific rare ICD-10 codes to artificially inflate the minority class representation before fine-tuning.
• Terminology-Layer Augmentation: Pure LLM fine-tuning often fails without a semantic safety net. Leading enterprise workflows integrate deterministic clinical terminology platforms (such as IMO Health ) rather than relying strictly on the neural network's probabilistic recall for edge-case nomenclature.
• Domain-Specific Custom Fine-Tuning: Utilizing smaller, highly adaptable open-weights models (like Llama-3 or Meditron variations) fine-tuned locally or via secure cloud environments (such as Snowflake Cortex or private AWS/Azure enclaves) using your organization's historical, de-identified edge-case audit logs.

Who Specializes in This Work?

• Enterprise Healthcare Terminology & AI Vendors: Companies like IMO Health specialize heavily in bridging the gap between raw clinical text and exact ICD-10/SNOMED mapping via proprietary semantic layers that correct LLM blind spots.
• Health-Tech Custom AI Engineering Firms: Boutique and mid-size AI engineering firms (such as Ailoitte or specialized health-tech digital agencies) focus explicitly on fine-tuning open-source medical LLMs using client-provided, niche data-sets and reinforcement learning from human feedback (RLHF) by practicing clinicians.
• Major Cloud Professional Services & Healthcare Practices: AWS (via their HealthAI consulting partners), Microsoft/Nuance teams, and specialized healthcare practices at major consultancies (Deloitte, Accenture Health) routinely build custom pipelines to fine-tune private instance models for hospital networks struggling with specialized specialty coding (e.g., rare oncology, genetic disorders, or rare autoimmune pathologies).

To help narrow down the best path forward, tell me:

• Are you looking to outsource the fine-tuning to a vendor, or do you have an internal data science team looking for tooling/consulting?
• Do you have a labeled dataset of historical rare cases ready, or do you need help with synthetic data generation?

Yes. For your problem, I’d look for clinical NLP/LLM engineers with actual ICD-10-CM coding experience, rather than a generic LLM fine-tuning shop. The long-tail/rare-diagnosis problem is specifically one where training-data design, hierarchical label handling, and evaluation matter as much as the base model.

A few strong leads:

• Indiana University / Regenstrief Institute — Zhen Hou, Hao Liu, Jiang Bian and collaborators. They published work specifically on fine-tuning LLMs for ICD-10 coding, including a 74,260-code training set and targeted augmentation for linguistic variation. Their work is unusually close to your use case.
• DATEXIS / DeepICD-R1. This group is working directly on training LLMs to reason about ICD-10 diagnoses using clinical-coding feedback and reinforcement learning—not merely prompting a general model.
• The team behind the 2026 HL7-CDA/ICD-10 deployment study. Their work combines model selection, redundancy-aware training, section-aware clinical inputs, and a human-in-the-loop coding workflow, making them particularly relevant if you're trying to move beyond a research prototype into production.
• Rare-disease/clinical NLP researchers at CHOP and collaborators. Their RareDAI work demonstrates fine-tuning on rare-disease clinical information and ICD-derived phenotypes, with an emphasis on adapting models to institution-specific data while keeping PHI protected.
• Specialized medical-LLM engineers/researchers using LoRA/SFT. There are already open implementations of ICD-10 fine-tuning—for example, an OpenBioLLM-8B ICD-coding model and a Phi-3 model fine-tuned with LoRA on clinical-note/code pairs. These are useful benchmarks when evaluating a contractor or vendor.

What I'd specifically hire for

For rare diagnoses, I'd prioritize someone who can build a long-tail ICD-10 training/evaluation pipeline, not simply fine-tune on your aggregate coding data. Ask candidates whether they can handle:

• Oversampling/weighted training for rare codes without destroying common-code performance.
• Hierarchical ICD-10-CM evaluation — exact code, parent/category, and clinically related-code accuracy.
• Hard-negative examples between very similar diagnoses/codes.
• Retrieval of authoritative ICD-10 descriptions/guidelines alongside the model.
• Synthetic augmentation for rare diagnoses while preventing synthetic-label contamination.
• Separate holdout sets specifically for rare/zero-shot and low-frequency codes.
• Human review by certified medical coders for ambiguous or high-risk cases.
• PHI-safe fine-tuning, preferably inside your environment rather than sending raw notes to a contractor's infrastructure.

This is important because published results show that simply fine-tuning on the full ICD-10 code set can produce excellent performance on relatively straightforward code-description matching while still struggling substantially on multiple concurrent conditions and real clinical notes. Nature More recent work is explicitly addressing the long-tail problem with policy-aware synthetic clinical data, reporting substantial gains in exact-code prediction.

If you're looking to hire someone rather than collaborate with an academic group, I can also search for specific consulting firms, individual ML engineers, or healthcare-AI vendors currently offering ICD-10/clinical-NLP fine-tuning, and narrow them down by US-based, HIPAA/BAA capable, and startup-friendly.

When automated medical coding misses rare diagnoses, it is usually because the model suffers from severe class imbalance—the "long tail" of rare ICD-10-CM codes that appear in fewer than 1% of training charts . Standard fine-tuning on a general hospital's local dataset won't fix this because the rare examples simply aren't there in sufficient volume.

Addressing this niche requires specialized vendors, clinical terminology engines, and targeted data-centric engineering rather than out-of-the-box LLM prompting.

Specialized Approaches & Industry Partners

• Terminology-Enriched Mapping Providers:
  • IMO Health: Specializes in bridging clinical phrasing with deep biomedical vocabularies. They provide rich semantic mappings that double out-of-the-box LLM accuracy by anchoring free-text clinical intent directly to difficult, specific secondary and primary ICD-10 codes.
  • Solventum (formerly 3M Health Information Systems): The gold standard for enterprise health systems dealing with complex clinical documentation improvement (CDI) and deep hierarchical condition category (HCC) / ICD-10 accuracy.
• Autonomous AI Coding & RAG Specialists:
  • CodaMetrix: Focuses on multi-specialty autonomous coding, utilizing longitudinal patient record context to improve accuracy across complex or ambiguous encounters.
  • Nym Health: Known for clinical language understanding engines that audit entire clinical notes to reduce omissions and explain code choices transparently.
  • Corti: Provides specialized medical coding and documentation agents anchored tightly to documented evidence without over-inferring, maintaining high fidelity on edge cases.

Alternative Strategy: Fixing the Rare-Diagnosis Gap In-House

If you prefer to fine-tune your own open-source models (e.g., Llama or Phi families) rather than plug into a black-box SaaS engine, standard fine-tuning will continue to fail on rare codes unless you change your methodology:

1. Synthetic Data Expansion & Curriculum Post-Training: Utilize policy-aware synthetic data generation (grounded on de-identified templates) to artificially inflate the representation of rare ICD-10 code families in your training corpus.
2. Code-Centric Learning: Shift your fine-tuning supervision away from full, massive clinical documents down to short, high-density evidence spans that specifically define the rare manifestation or etiology rules.
3. Hybrid RAG + Knowledge Graphs: Pair your fine-tuned model with a deterministic rules/ontology engine (like SNOMED-to-ICD-10 crosswalks) that catches what the generative model misses in its parametric memory.

To help narrow down whether you need a turnkey enterprise coding platform or a custom fine-tuning data pipeline , could you share:

• Are you hosting models locally on-premise (HIPAA) or using API-based models?
• Roughly how many rare encounter charts do you have available for training?

Yes. The niche you want is clinical NLP / medical-coding LLM specialists, specifically people experienced with ICD-10-CM long-tail classification, terminology normalization, and fine-tuning on rare-code examples.

A few strong leads:

• IMO Health — probably one of the best domain-specialist candidates. They maintain large clinical terminology mappings across ICD-10-CM and other coding systems and explicitly work with customers on domain-optimized NLP, RAG, and fine-tuning.

• Cavo Health — focused specifically on autonomous medical coding/CDI, with an explicit emphasis on rare, complex, and combination ICD codes that conventional ML misses.

• OMNISOM — an AI medical-coding company offering ICD-10 search, HCC intelligence, and API integration, and says it will work with health-tech teams using their data.

• Astol — specifically positioning itself around the long-tail ICD-10 problem, including low-frequency and complex codes. This is unusually close to your stated problem.

There are also researchers worth approaching. A 2025 study from Indiana University/Reg enstrief demonstrated a two-stage fine-tuning approach using 74,260 ICD-10 code/description pairs, substantially improving coding performance; the authors are Zhen Hou, Hao Liu and colleagues.

What I'd look for in a contractor/team

Don't hire someone who merely says “I fine-tune Llama.” For your problem, the important expertise is:

1. Long-tail/rare-label learning — few-shot or class-balanced training rather than optimizing aggregate accuracy.
2. ICD-10-CM expertise — including annual code-set changes, excludes/includes rules, laterality, combination codes, etc.
3. Clinical NLP — abbreviations, synonyms, implicit diagnoses, and messy clinical documentation.
4. RAG + terminology retrieval — rare codes may be better retrieved from a comprehensive terminology layer than memorized by the LLM. IMO specifically describes this approach.
5. Human-in-the-loop evaluation — particularly precision/recall by code-frequency bucket, not just overall exact match.
6. PHI/compliance experience — if your training set contains real clinical notes.

One important point: fine-tuning alone may not be the best fix for rare diagnoses. Recent work suggests combining an LLM with retrieval/knowledge mechanisms can help with the enormous ICD label space; one 2025 study used knowledge-driven sampling and knowledge-graph injection, while another 2026 deployment study used redundancy-aware training and structured clinical-document prompts.

If you tell me your dataset size, approximate number of rare ICD-10 codes, current model, and whether you're looking for an individual consultant or an organization, I can narrow this to 5–10 specific specialists/teams to approach, including relevant researchers and vendors.

When automated medical coding engines struggle with rare, niche, or under-represented ICD-10 diagnoses, it is usually because general-purpose LLMs or standard training sets lack sufficient few-shot examples and deep semantic mapping for tail-end medical conditions.

Addressing this gap requires working with specialized healthcare AI vendors, clinical terminology partners, or data-tuning providers who build domain-specific grounding layers or fine-tune models using LoRA/PEFT on proprietary clinical corpora.

• IMO Health (International Classification of Diseases and Clinical Terminology mappings ) — They specialize in rich clinical semantics and structured terminologies. Rather than relying solely on a raw LLM, they provide a foundational knowledge layer and terminology bridge that matches complex or ambiguous input terms to accurate ICD-10 and HCC codes, effectively reducing the need for raw model hallucination on rare terms.
• John Snow Labs (Healthcare-specific NLP & Fine-Tuning ) — Known explicitly for enterprise medical NLP, they build and fine-tune specialized models (including ICD-10-CM and SNOMED extraction engines) optimized for complex chart abstraction, rare condition matching, and regulatory audit readiness.
• MediCodio AI — They combine specialized fine-tuning on extensive clinical terminologies (ICD-10-CM, CPT) with Retrieval-Augmented Generation (RAG) and targeted prompt engineering to handle nuanced specialty requirements and edge cases.
• iMerit (Medical Generative AI Data Ops ) — If you are looking to perform the fine-tuning in-house rather than buying a black-box SaaS product, iMerit provides expert-in-the-loop data curation, clinical annotation, and RLHF (Reinforcement Learning from Human Feedback) specifically tailored to tune LLMs on specialized or rare medical datasets.
• CodaMetrix and Fathom (via AKASA) — While deployed primarily as end-to-end autonomous coding platforms for large health systems, their core engineering edge lies in massive, deep-learning models trained on longitudinal, multi-specialty clinical encounters that capture complex and rare code combinations.

To help narrow down the best path, could you share:

• Are you looking to outsource/adopt a specialized platform or fine-tune your own in-house model using your historical data?
• What medical specialties or types of rare diagnoses are most frequently being missed?

For rare-diagnosis misses, you usually want a healthcare NLP/LLM fine-tuning specialist with medical coding experience, not a general LLM consultant. The hard part is often the “long tail” of ICD-10: rare diseases, specialty-specific terminology, synonyms, documentation patterns, and code hierarchy reasoning. Research and open-source work show that domain-specific fine-tuning can materially improve ICD coding performance compared with prompting alone.

Groups and specialists to look at:

Healthcare AI vendors / consultancies

• Taction Software Healthcare LLM Fine-Tuning Services — offers healthcare LLM adaptation, including PEFT/LoRA fine-tuning, clinical data curation, and deployment support.
• IMO Health — focuses on clinical terminology, mappings, and AI-ready healthcare data layers that can help with ICD-10 normalization and rare terminology coverage.
• Corti — builds clinical AI systems including medical coding workflows; useful if you are evaluating a specialized coding platform rather than only tuning your own model.
• Ambience Healthcare — works on clinical documentation and coding automation, including model adaptation for medical workflows.

Researchers / technical profiles to seek

Look for people who have experience with:

• ICD-10-CM / ICD-10-PCS prediction
• clinical NLP
• EHR note → code extraction
• ontology grounding (ICD, SNOMED CT, UMLS)
• parameter-efficient fine-tuning (LoRA/QLoRA)
• rare-label learning / long-tail classification

Relevant examples of the kind of work you want:

• ICD-focused LLM fine-tuning using thousands of ICD code descriptions and clinical notes.
• Medical coding-specific fine-tuned models using LoRA/SFT approaches.
• Coding models that combine LLM reasoning with terminology retrieval rather than relying on the model’s memory alone.

When interviewing a specialist, ask:

1. “How will you measure improvement on rare diagnoses?”
   Avoid only reporting overall accuracy. Ask for:

   • macro F1
   • rare-code recall
   • top-k ICD recall
   • specialty-specific performance

2. “How will you prevent the model from collapsing into common diagnoses?”
   Long-tail ICD datasets often need:

   • oversampling rare codes
   • curriculum training
   • hard-negative examples
   • retrieval augmentation from terminology databases

3. “What is your strategy for unseen or low-frequency ICD codes?”
   Strong teams usually combine:

   • fine-tuning
   • ICD hierarchy embeddings
   • UMLS/SNOMED mappings
   • retrieval of candidate codes
   • coder feedback loops

4. “How will you handle PHI?”
   Make sure they understand HIPAA constraints, de-identification, secure training environments, and auditability.

If you are hiring, the title to search for is often “Clinical NLP Engineer,” “Healthcare LLM Engineer,” “Medical AI Scientist,” or “Clinical Informatics ML Lead” rather than simply “LLM fine-tuning engineer.” For a production coding system, a small team with one clinical informatics/coding expert plus one ML/NLP engineer is usually stronger than a general AI contractor.

Addressing the "long tail" of rare ICD-10-CM codes requires specialized fine-tuning strategies because standard LLMs and off-the-shelf medical models suffer from severe class imbalance and rarely encounter uncommon conditions in generic pre-training data. Rather than relying on a single off-the-shelf vendor, solving this typically involves boutique healthcare AI consultancies, specialized clinical NLP engineering groups, or adopting code-centric architectures.

Approaches and Specialists for Rare ICD-10 Fine-Tuning

• Dedicated Healthcare AI & NLP Consultancies: Firms like Jvion, Averbis , or specialized clinical informatics engineering teams focus on fine-tuning open-source backbones (like Llama or BioBERT) utilizing proprietary data augmentation. They apply specialized techniques such as code-centric learning and synthetic data generation to upsample rare disease presentations.
• RAG-Hybrid Architecture Vendors: Platforms like MediCodio employ hybrid frameworks—pairing fine-tuned models for general semantic extraction with dynamic Retrieval-Augmented Generation (RAG) tied directly to the full ICD-10-CM tabular and alphabetical index to prevent the model from hallucinating or defaulting exclusively to high-frequency codes.
• Academic & Open-Source Frameworks: If handling development in-house or via a specialized contractor, recent machine learning methodologies explicitly target the rare-code deficit through code-centric span selection rather than full-document prediction (demonstrated in open frameworks like LLMCoder and multi-agent approaches).

Key Strategies to Fix Rare Diagnosis Blind Spots

• Synthetic Minority Over-sampling (SMOTE) & LLM Augmentation: Generate synthetic clinical vignettes for low-frequency ICD-10 codes using larger teacher models (like GPT-4) guided by human coding experts to artificially inflate the training representation of rare conditions.
• Two-Stage Retrieve-Rank Pipelines: Decouple the task. Use a high-recall semantic search engine (dense vector retrieval) to surface candidate rare codes from the entire 70,000+ label set first, then fine-tune a smaller LLM strictly to rank and select the correct code based on explicit text spans.
• Hierarchical Loss Function Tuning: Penalize the model heavily for missing specific sub-classification or fifth-to-seventh characters (severity/laterality) during the fine-tuning loss calculation, forcing it to look past general chapter headings.

If you share the approximate size of your current dataset and whether you are working with open-source weights (like Llama/Mistral) or proprietary APIs , I can better tailor how to approach your fine-tuning pipeline.

Yes. For rare/long-tail ICD-10 diagnoses, I’d look for a team with three capabilities: clinical NLP, ICD-10 ontology/coding expertise, and PEFT/SFT training—not a generic “LLM fine-tuning” shop.

Strong candidates

• John Snow Labs — probably the closest fit if you want an established healthcare-NLP platform plus customization. Their Generative AI Lab supports domain-specific fine-tuning, active learning, evaluation, PHI de-identification, and specifically ICD-10, CPT, SNOMED CT and LOINC coding.
• Bacancy Technology — more of a custom engineering/consulting option. They explicitly describe fine-tuning models on client-specific clinical documentation and mapping clinical text to ICD-10/SNOMED/CPT/LOINC.
• AWS Healthcare & Life Sciences — worth considering if you want the work built on AWS and need enterprise PHI/security infrastructure. AWS has specific guidance for fine-tuning LLMs for medical NLP and recommends SME-labeled ground truth plus ongoing evaluation.
• Truveta — particularly interesting if the problem is fundamentally clinical-data coverage rather than just model tuning. Its language model is trained on very large-scale EHR data and handles diagnoses and unstructured clinical notes.
• Microsoft Healthcare & Life Sciences — Microsoft has actually developed MediPhi, including a medical-coding specialist model, using fine-tuned medical experts and model merging. That makes its research/engineering group relevant if you're operating at enterprise scale.

But I wouldn't simply “fine-tune on rare diagnoses”

Your symptom—missing rare diagnoses—is specifically identified as a weakness of current ICD-10 LLM approaches. Recent research shows that public ICD datasets have poor coverage of the long tail and that targeted training on missed/unseen codes can substantially improve recall.

A good specialist should therefore propose something like:

clinical note → candidate-code retrieval → LLM ranking/coding → ICD-10 guideline validation → human review for low-confidence/rare codes

rather than just doing another round of generic SFT. Recent RAG-Coding research, for example, found meaningful recall gains by grounding the model in the official coding hierarchy and guidelines.

I'd also insist on a long-tail holdout set: measure macro-F1, recall by code frequency bucket, and specifically unseen/rare-code recall, rather than overall accuracy. Otherwise a model can look excellent while continuing to miss exactly the diagnoses you care about.

If you tell me (1) ICD-10-CM vs ICD-10, (2) approximate number of labeled notes, (3) your current LLM, and (4) whether PHI can leave your environment, I can narrow this to 3–5 specific vendors/teams I'd actually interview, including what to ask each one.