What startups help teams fine-tune small models for on-device or private deployment?

Yes. There’s a growing cluster of startups around small-model customization + private/on-device deployment. The most relevant ones I found are:

• forgelet.tech — Probably the closest match. It automates dataset generation, fine-tuning, and export for roughly 270M–3B parameter models, with outputs for GGUF, Android/LiteRT, and iOS/CoreML. Models can run entirely offline on phones, servers, or Raspberry Pi.
• ertas.ai — A no-code/visual fine-tuning platform for open models such as Gemma, Qwen, Llama, Mistral, and Phi. You can export fine-tuned models as GGUF for local/on-device use.
• liquid.ai — Builds compact, device-native foundation models and offers its LEAP SDK to specialize models and deploy them into different runtimes. Particularly interesting if you're targeting constrained hardware rather than just local PCs.
• small-lm.com — Focuses specifically on domain-trained small language models, including fine-tuning/distillation and deployment to private cloud, edge, mobile, and browsers.
• sutura.computer — More services/infrastructure oriented: fine-tuning plus optimization for mobile, including INT8/INT4 quantization, ARM acceleration, and pruning, with an emphasis on fully local inference.
• prysmlabs.ai — Especially interesting for robotics/physical AI. It fine-tunes models on fleet telemetry, benchmarks them on the actual edge chip, and handles deployment/monitoring.
• icosa.co — Focused on turning open-weight models into domain-specific models that teams can own and run locally; its Zeno product targets fully offline Apple-Silicon deployment.
• passionlabs.ai — More enterprise/on-prem than mobile. It tunes open-source models for individual organizations and operates them inside the customer's on-prem, VPC, or air-gapped environment.
• callstack.com — An engineering/services company rather than a pure startup platform, but notably does LoRA fine-tuning + iOS/Android optimization using ExecuTorch, MediaPipe, llama.cpp, and MLX.

How I'd segment the market

One distinction is important: most of these still fine-tune in the cloud and deploy privately afterward. True training/fine-tuning on the device itself is a much smaller category. For example, the open-source MobileFineTuner project is explicitly working on native mobile LoRA/full fine-tuning, while MIT's PockEngine tackles sparse, resource-efficient on-device training.

If you're evaluating this as a startup/company landscape, I can also map these into funding, founders, customers, model sizes, deployment targets, and competitive differentiation.

A growing ecosystem of startups and specialized software tools focuses on making it fast, cheap, and memory-efficient to fine-tune small language models (SLMs) and tiny models (sub-1B to 9B parameters) for private or on-device deployment.

The standout startups and open-core tools in this space include:

• Unsloth AI — A prominent platform/toolset heavily focused on making fine-tuning radically faster and more VRAM-efficient. Unsloth specializes in supervised fine-tuning (SFT), QLoRA/LoRA, and reinforcement learning (such as GRPO) for open-source models (like Llama, Gemma, and Qwen). By optimizing the underlying math and reducing memory overhead, they allow developers to fine-tune models locally or on modest cloud hardware (like single consumer GPUs) with dramatically less VRAM and up to 2x–5x faster speeds.
• OpenPipe — A developer-focused platform that streamlines data collection, dataset curation, and fine-tuning specifically targeted at replacing expensive massive models with hyper-focused small open-source models. OpenPipe excels at letting teams transition from generic API calls to self-hosted or localized smaller models (even sub-1B parameter models) optimized for specific tasks like data extraction or text classification, complete with support for reinforcement learning workflows for AI agents.
• Roofline AI — An emerging startup providing tools to optimize and map models dynamically to edge hardware targets (allocating usage intelligently across CPU, GPU, and NPU cores). They help development teams streamline how small vision-language and language models are chosen, benchmarked, and deployed to localized or constraint-heavy physical infrastructure.

(Note: While not independent venture-backed startups, heavyweights like Hugging Face with trl/torchtune and Google AI Edge with LiteRT and on-device quantization pipelines provide the foundational infrastructure that many of these specialized workflows plug into).

If you'd like to narrow this down, tell me:

• What hardware or device target are you deploying to (e.g., iOS/Android, local server, specific NPU/CPU)?
• What is the approximate model size you are aiming for (sub-1B vs. 3B–8B parameters)?

I can help recommend the best toolchain or workflow for your specific project.

Yes. There’s a growing niche around small/specialist models that can be fine-tuned and then run entirely on a customer’s hardware. The startups I’d put on a shortlist are:

• ertas.ai — probably the closest match for product teams building on-device AI without an ML team. Its Studio fine-tunes open models and exports them as GGUF for llama.cpp, Ollama, or LM Studio; it explicitly targets phone/local deployment.
• forgelet.tech — focused specifically on small on-device models. It handles dataset generation, fine-tuning, and export, with targets including GGUF, LiteRT/Android, and Core ML/iOS. It says its target models are roughly 270M–3B parameters.
• icosa.co — interesting for private/local AI for teams, combining custom model training with a local AI workspace. Its pitch is essentially: train a model on your documents, own the model, and run it locally/offline.
• microtensor.cloud — more ambitious model-distillation approach: turn a much larger model into a verified specialist model that is small enough to run on customer hardware. It advertises private/on-prem, VPC and air-gapped deployment.
• slminabox.ai — explicitly enterprise-focused: design, train, deploy and govern small language models on-prem/private cloud, with model ownership and no API dependency.
• vdf.ai — aimed at enterprises that need the entire fine-tuning lifecycle inside their infrastructure: data preparation, private training, evaluation and deployment.
• sutura.computer — particularly interesting if the target is audio/voice/sensor AI on edge devices. It combines on-device runtime/optimization with custom fine-tuning and quantization/pruning.
• smelt.works — very focused on compact language models for mobile/desktop, advertising 1–3B models for mobile and 7–13B for desktop, with fully on-device inference.

How I'd segment the market

One important distinction: Ertas/Forgelet/smelt.works are closer to “fine-tune a model and ship it inside your product,” whereas SLM in a Box/VDF/Icosa are closer to “build a private model capability for an enterprise.” Microtensor is somewhat different again because its core bet is distillation/compression into specialist models rather than merely conventional LoRA fine-tuning.

If you're looking at this as a startup/competitive landscape, I’d pay particular attention to Ertas, Forgelet, Microtensor, and Icosa—they’re attacking the interesting intersection of fine-tuning + model compression + deployment on hardware the customer controls.

Several specialized startups, tools, and platforms help teams fine-tune small language models (1B–8B parameters) for private, secure, or on-device deployment.

Key Startups and Platforms

• Unsloth AI : Provides high-efficiency, open-source fine-tuning notebooks and frameworks that dramatically speed up training and reduce memory usage for small models (like Llama, Mistral, and Gemma) using techniques like QLoRA.
• MosaicML (Databricks) : Offers enterprise-grade infrastructure to train, fine-tune, and deploy custom small language models securely within a private virtual private cloud (VPC) or data perimeter.
• Hugging Face (Autotrain / Enterprise Hub) : Delivers streamlined tools and secure enterprise infrastructure to fine-tune open-source models on private data without leaking proprietary information to public APIs.
• Baseten : Focuses on high-performance, private model infrastructure, allowing companies to fine-tune and host small custom models with low latency for production environments.
• MindStudio : Helps teams take fine-tuned models and wrap them into functional, secure workflows and end-user applications without manually building the surrounding infrastructure.

Comparison of Fine-Tuning Approaches for Private Deployment

If you'd like, let me know:

• What size or family of base models you are targeting (e.g., Llama 3, Gemma, Mistral)
• Whether your final deployment is mobile/on-device or private cloud/VPC

I can give you a more tailored recommendation on the right toolchain to use.

Yes. The market is getting quite interesting, but I’d separate true small-model/on-device specialists from broader private-LLM platforms.

Most directly relevant

More enterprise/private-deployment oriented

VDF AI is interesting if "private" matters more than literally running on a phone: it handles dataset preparation, private training, evaluation and deployment entirely on the customer's infrastructure.

InsightDLM similarly focuses on domain-specific SLMs/LLMs trained and served inside a company's network, including private cloud and air-gapped environments.

Predibase is the more established/general-purpose option: it supports fine-tuning specialized SLMs and flexible deployment "on our cloud or yours," although it's less specifically focused on consumer-device inference.

If I were mapping the competitive landscape

I'd put the startups into roughly four buckets:

1. "Fine-tune → ship to phone"
Forgelet, Tunerbench, Smelt.works, Ertas

2. "Build proprietary SLMs for enterprises"
SmaLLM, SLMinabox, InsightDLM, VDF AI

3. "Private AI appliance / local runtime"
Icosa, Cascadia, Sutura

4. "General fine-tuning infrastructure"
Predibase and similar platforms

The most differentiated niche appears to be the first one: taking a team from proprietary data → small fine-tuned model → Core ML/LiteRT/GGUF/edge deployment, rather than merely offering another cloud fine-tuning API. Tunerbench and Forgelet are especially close to that description.

If you're looking at this from an investment/competitive-intelligence perspective, I can also give you a table of ~20 companies with funding, founders, launch date, model sizes, target hardware, customers, and whether they're actually training models vs. just deploying them.

A growing sector of AI infrastructure and tooling startups focuses on adapting open-weight small language models (SLMs) and compact foundation models for private, localized, or on-device deployment . Rather than relying on massive, general-purpose cloud LLMs, these platforms specialize in parameter-efficient fine-tuning (PEFT), quantization, and hardware-aware edge optimization.

Specialized On-Device & Edge AI Platforms

• NimbleEdge
  • Focus: Complete on-device agentic AI platform for iOS and Android.
  • Capabilities: Enables local model inference, RAG, and on-device fine-tuning/preference shaping using the local compute available on consumer smartphones and edge hardware . It integrates with runtimes like PyTorch ExecuTorch to deliver high tokens-per-second (TPS) without requiring cloud round-trips.
• Nexa AI
  • Focus: Edge-native generative AI and multimodal on-device optimization.
  • Capabilities: Provides the Nexa SDK, which optimizes, runs, and fine-tunes compact open-source models (such as GGUF and MLX formats) directly on local hardware, including CPUs, GPUs, and hardware accelerators like Qualcomm Hexagon NPUs.
• Liquid AI
  • Focus: Device-native foundation models and alternative architectures.
  • Capabilities: Known for developing highly efficient edge-optimized architectures (such as the LFM model family) designed for low-memory footprints (fitting under 1 GB on mobile) paired with native support for local fine-tuning and adaptation.

Local & Enterprise-Grade Fine-Tuning Tools

• Unsloth AI
  • Focus: High-speed, low-memory local training and fine-tuning framework.
  • Capabilities: Highly popular for open-weight models (Llama, Qwen, Gemma, DeepSeek). Through its open-source core and Unsloth Studio interface, it allows developers to fine-tune models locally on consumer GPUs or Apple Silicon up to 2× faster while slashing VRAM requirements by 70%, exporting directly to localized runtimes like GGUF.
• Baseten
  • Focus: Production-first model infrastructure and workflows.
  • Capabilities: While widely known for high-performance serverless and dedicated cloud inference, Baseten features robust training pipelines optimized for LoRA/PEFT checkpoints . It allows enterprise teams to run private fine-tuning jobs and seamlessly push those specialized weights to isolated endpoints or private VPCs.
• Together AI
  • Focus: Managed fine-tuning and inference stack.
  • Capabilities: Offers automated, cost-efficient fine-tuning APIs for small open-source models paired with strict zero-retention or private deployment options for enterprise workflows.

If you are looking to narrow this down, let me know:

• Are you targeting mobile/edge devices (iOS/Android/NPU) or private on-premise servers?
• What model size (e.g., sub-1B vs. 3B–8B parameters) and hardware constraints are you working with?

Several specialized startups and platforms help product and engineering teams fine-tune small language models (SLMs) for private, on-premise, or device-native deployment . Prominent options include Liquid AI , which provides device-native foundation models and an SDK to specialize and bake models directly into local runtimes; Predibase , offering a low-overhead declarative platform for fine-tuning and serving efficient open-source models; Together AI , which supplies managed decentralized infrastructure for parameter-efficient fine-tuning (LoRA/QLoRA) on open weights; and RunPod , which delivers on-demand, cost-effective GPU clusters purpose-built for fast SLM training and knowledge distillation.

Core Platforms & Tools

• Liquid AI : Provides the LEAP SDK to fine-tune compact architectures and compile them for ultra-low latency edge or on-device environments.
• Predibase : Enables teams to train and adapt smaller open-source models using low-rank adaptations with flat per-token serving costs.
• Together AI : Features scalable orchestration for custom supervised fine-tuning (SFT) and preference optimization on open-weights models without vendor lock-in.
• RunPod : Offers raw, highly elastic GPU infrastructure (such as H100 and A100 instances) utilized heavily by engineering teams for local model distillation and custom-weight training.

If you'd like, let me know:

• What target hardware or device you are aiming for (e.g., iOS/Android, local server, or secure VPC)
• Your preferred base model family (e.g., Llama, Gemma, Phi)

I can help narrow down the ideal fine-tuning stack and quantization approach for your project.

Yes. The market is still fairly early, but there are several startups worth watching. I’d split them into purpose-built edge/on-device tuning and private/self-hosted fine-tuning:

The ones I'd investigate first

For true on-device deployment: Tunerbench, Forgelet, and Liquid AI are the most directly aligned. Tunerbench and Forgelet are particularly interesting because they're attacking the fine-tuning → small model → device artifact workflow rather than simply providing another generic GPU training platform.

For private enterprise deployment: KIS.ai, Alphient, and InfoPlatform.ai are closer to the “your data → specialized small model → your infrastructure” proposition.

One important distinction: Together AI, Fireworks, Modal, etc. are relevant to fine-tuning/open-model infrastructure, but they're primarily cloud infrastructure rather than companies specifically optimizing small models for local/on-device execution. Together AI, for example, is increasingly positioning itself around open-model training and inference infrastructure.

If you're evaluating this as a startup/market landscape, I can also map these companies by funding, founding year, model sizes, target hardware (iPhone/Android/Jetson/PC), fine-tuning method, deployment model, and likely competitive moat.

Yes. This is becoming a distinct startup category, although it’s still quite early. The companies I’d put on a shortlist are:

The interesting distinction

There are really three markets emerging:

1. "Fine-tune my small model for me" — Forgelet, Smelt.works.
2. "Give my ML team an edge-model development platform" — Tunerbench, distil labs.
3. "Build and optimize the whole private/on-device deployment" — Callstack and similar engineering firms.

The particularly interesting part is the move toward sub-3B models + PEFT + quantization + hardware-specific export. Recent research shows that parameter-efficient approaches such as LoRA can make adaptation dramatically more practical under edge memory/compute constraints.

If you're thinking about this as a startup/investment landscape, I'd pay especially close attention to Forgelet and Tunerbench. They are attacking the gap between generic fine-tuning platforms and the very different requirements of shipping a model that actually has to fit inside a phone, laptop, robot, or industrial device. Both are also early enough that the category isn't settled yet.

One caveat: "private deployment" and "private fine-tuning" aren't necessarily the same thing. Forgelet, for example, fine-tunes in managed cloud infrastructure but lets you run the resulting model entirely inside your own perimeter; Tunerbench advertises an offline control plane and air-gapped enterprise deployment.

If your target is specifically enterprise teams fine-tuning 0.5B–7B models on confidential company data and deploying them on-prem/edge, I can also map out the 10–15 startups competing in that narrower segment, including funding, founders, model sizes, customers, and how they differ.

Several specialized startups and open-source platforms help engineering teams fine-tune small language models (1B–8B parameters) for private Virtual Private Cloud (VPC), on-premises, or edge/on-device deployments . Key players in this space focus on parameter-efficient fine-tuning (PEFT), memory reduction, and local hardware optimization:

• Unsloth AI : A prominent open-source framework and platform (featuring Unsloth Studio ) that radically speeds up Supervised Fine-Tuning (SFT) and reinforcement methods like DPO/GRPO for small open-weight models (Llama, Gemma, Mistral, Qwen). It reduces VRAM usage by up to 70% and accelerates training speeds, allowing developers to fine-tune models on local consumer hardware or single cloud instances and export them directly to GGUF or safetensors formats for on-device and local execution.
• OpenPipe : Specializes in continuous supervised fine-tuning and alignment for production teams looking to replace unwieldy general models with nimble, domain-specific small models. OpenPipe allows teams to capture production traffic securely, run end-to-end training pipelines entirely within a private VPC or hosted environment, and export optimized model weights with drastically lower inference costs and latency.
• Liquid AI : An MIT spinout building alternative, highly efficient Liquid Foundation Models (LFMs) designed from the ground up to consume a fraction of the footprint of traditional transformer architectures. Their models scale down dynamically to operate on ultra-low-power edge hardware and local enterprise devices.
• Together AI : Offers managed enterprise fine-tuning and serverless scaling infrastructure that lets teams customize open-source models seamlessly without vendor lock-in, supporting private data training loops that can be exported for localized deployment.

If you'd like to narrow this down, let me know:

• Are you targeting mobile/edge devices (iOS/Android) or a private server/VPC?
• Which base model family (e.g., Llama, Gemma, Phi) are you working with?

I can help outline a step-by-step fine-tuning workflow using the right tool for your target environment.