We can't deploy models to mobile devices due to size. Who specializes in "knowledge distillation" to shrink models?

Shrinking heavy AI models so they can run locally on mobile and edge devices typically involves a blend of knowledge distillation (teaching a smaller "student" model to mimic a larger "teacher" model), pruning (removing dead/low-impact weights), and quantization (reducing precision like FP32 to INT8/INT4).

Specialized platforms, SDKs, and enterprise tools focus specifically on this kind of neural network compression:

• Specialized Optimization Platforms & SDKs
• Embedl : Provides a deep learning optimization SDK designed specifically to compress models (via pruning and quantization) for resource-constrained edge and embedded systems without sacrificing target accuracy.
• ENOT.ai : Offers an automated neural network compression and acceleration platform that integrates automated pruning, quantization, and distillation to minimize memory and size constraints for specific hardware.
• Neural Magic : Famous for making deep learning models sparse and fast. While heavily focused on LLMs and server/edge CPU inference via sparsification and pruning, their open-source tools (like SparseML) handle compression workflows.
• LeapMind : A Japanese edge-AI innovator specializing in extreme neural network compression technology to deploy deep learning vision/audio models onto tiny, resource-constrained microdevices and IoT chips.
• Hardware-Vendor Native Toolkits
  If you are deploying to specific mobile chips, the silicon manufacturers provide proprietary toolkits that perform hardware-aware compression, quantization-aware training, and distillation pipelines:
• Qualcomm AI Hub / SNPE : Optimizes and compiles models specifically for Snapdragon mobile processors, handling quantization and graph-level reduction.
• Apple Core ML Tools : Built directly into Xcode/Core ML, allowing post-training quantization (linear quantization, palette quantization) to drastically shrink model footprints for iOS devices.
• Samsung Neural SDK : Focuses on hardware-aware pruning and mixed-precision quantization tailored to Exynos NPUs.

To help narrow down the best path, tell me:

• What framework is your current model built in (PyTorch, TensorFlow, etc.)?
• What target mobile platforms (iOS, Android, or both) are you aiming for?

If your problem is “the model is too large to fit on phones”, you’re looking for specialists in model compression / efficient AI / edge AI, with knowledge distillation being one of their core techniques. Knowledge distillation trains a smaller “student” model to reproduce the behavior of a larger “teacher” model, often combined with quantization and pruning for mobile deployment.

Some groups and companies known for this area:

Major research groups

• Geoffrey Hinton — One of the pioneers of knowledge distillation (the “dark knowledge” approach), originally demonstrating how large networks can transfer knowledge to smaller ones.
• Google DeepMind — Has extensive work on efficient models, distillation, and deploying ML systems at scale.
• NVIDIA Research — Works heavily on model compression, efficient inference, and hardware-aware optimization.
• Meta AI — Publishes research on efficient architectures, smaller language models, and compression methods.

Companies / engineering teams focused on shrinking models for edge devices

• tinyllms.ai — Focuses specifically on distilling large models into smaller models intended for local/offline hardware.
• baa.ai — Works on quantization, training, and distillation to make large models run on controlled hardware environments.
• tesan.ai — Offers edge AI compression approaches involving distillation, pruning, and quantization.
• inferensys.com — Provides edge model compression services using distillation and quantization for constrained devices.
• presear.com — Works on edge deployment pipelines combining knowledge distillation, quantization, and runtime optimization.

If you’re deploying on phones specifically, look for expertise in:

• LLM distillation (teacher LLM → small language model)
• Quantization (FP32 → INT8/INT4)
• Pruning (removing unnecessary weights)
• Architecture redesign (e.g., MobileNet-style models, small transformers)
• Runtime optimization - Android: TensorFlow Lite, ONNX Runtime Mobile
• iOS: Core ML
• Qualcomm: Snapdragon AI Engine
• Apple: Neural Engine

A typical engagement with a specialist would look like:

1. Take your existing model (teacher).
2. Define device constraints (RAM, latency, battery, offline requirements).
3. Train a smaller student model using distillation.
4. Apply quantization/pruning.
5. Benchmark accuracy vs. speed on real devices.
6. Ship the optimized model.

If you mean large language models specifically (e.g., a 7B–70B model that needs to run on iPhone/Android), the specialist landscape is a bit different from traditional neural-network distillation; you’d want teams experienced with LLM compression, LoRA distillation, KV-cache optimization, and 4-bit inference.

Yes. Knowledge distillation is specifically designed for this: train a small student model to reproduce the behavior/knowledge of a larger teacher model, making the student much cheaper to run on-device.

If you're looking for people/groups to contact, these are especially relevant:

• Geoffrey Hinton — one of the originators of modern knowledge distillation, with Oriol Vinyals and Jeffrey Dean. Their 2015 paper established the technique.
• Lucas Beyer and Google's research teams — their work on distillation focuses explicitly on making large vision models practical by transferring them into smaller models.
• Zhiqing Sun and the MobileBERT team — particularly relevant to your use case. MobileBERT was explicitly designed for resource-constrained mobile devices and demonstrated a 4.3× smaller, 5.5× faster model than BERT-BASE.
• Xiaoqi Jiao and the TinyBERT team — developed Transformer-specific distillation, reporting a 4-layer model 7.5× smaller and 9.4× faster than BERT-BASE while retaining most of its performance.

But don't limit the search to "distillation"

For actual mobile deployment, I'd look for a team with expertise in model compression + on-device inference, because distillation is often combined with:

• Quantization — e.g. FP32 → INT8; can reduce model size substantially and improve inference efficiency.
• Pruning / sparsity — remove unnecessary weights or channels.
• Architecture design — redesign the student around mobile CPU/NPU constraints.
• Distillation — transfer accuracy from the large model to that architecture.
• Hardware-aware optimization — optimize specifically for iOS/Android NPUs rather than merely minimizing parameter count.

In other words, the ideal specialist isn't just someone who can say "we know knowledge distillation." You want someone who can take your existing model → design an appropriate student → distill it → quantize/prune it → benchmark it on the actual target phone.

If you tell me what kind of model you're trying to shrink (LLM, vision, speech, recommendation, etc.) and roughly its current size, I can identify the best researchers/labs/companies specifically working on that kind of compression.

Several specialized AI engineering firms, hardware giants, and niche startups focus on knowledge distillation and holistic model compression (combining distillation with pruning and quantization) to make large models fit onto edge and mobile devices.

Specialized Startups & Optimization Platforms

• CoCoPIE : A highly specialized startup dedicated entirely to enabling real-time AI on off-the-shelf mobile devices. They use a unique "compression-compilation co-design" that combines pattern-based pruning with knowledge distillation, generating code optimized specifically for mobile processors.
• Neural Magic : Known for their "sparsification" algorithms. While traditionally focused on running heavy models on commodity CPUs, they specialize heavily in combining structured/unstructured pruning with techniques like their proprietary SquareHead Knowledge Distillation to shrink LLMs into highly compact footprints.
• Deci AI : Developers of the AutoNAC (Automated Neural Architecture Construction) engine and the open-source SuperGradients library. Their platform specializes in creating hardware-aware, optimized student models by embedding advanced knowledge distillation into the training pipeline to shrink computer vision and NLP models.
• KD Cube : A newer, boutique AI agency explicitly focused on evaluating, benchmarking, and utilizing knowledge distillation to reduce operational costs and resource requirements for language models.

Big Tech & Hardware Ecosystems

If you are developing for specific mobile chips, the major hardware providers have dedicated ecosystems that incorporate distillation and compression:

• Samsung Electronics : Integrates memory-aware and thermal-aware knowledge distillation directly into their mobile AI processors and LPDDR storage pipelines, finding optimal student configurations tailored specifically for mobile device constraints.
• Apple : Apple’s Core ML team relies heavily on internal, niche expertise in pruning and knowledge distillation to shrink foundational models so they can run locally inside the tight RAM limits (<4GB) of the Apple Neural Engine (ANE).
• Qualcomm & Google : Qualcomm’s AI Stack and Google’s TensorFlow Lite (TFLite) / Android NNAPI teams provide framework-level tooling designed to handle the post-training distillation, structural pruning, and 4-bit/8-bit quantization required for on-device mobile deployments.

Open-Source Solutions You Can Use Immediately

If you have an in-house engineering team and want to build a distillation pipeline yourself rather than hiring a firm, you can leverage:

• Hugging Face (Optimum / Transformers) : Contains native support for training small student models (like DistilBERT or compact Gemma/Llama variants) using soft-label outputs from larger teacher models.
• Intel OpenVINO & Nvidia TensorRT : Both toolkits feature robust model optimization APIs that combine knowledge distillation with quantization to maximize throughput on edge hardware.

To point you toward the right vendor, tell me:

• What type of model are you trying to shrink? (e.g., LLM, computer vision, audio)
• What is the target operating system or chip ? (e.g., iOS, Android, Snapdragon, Apple Silicon)
• Do you want an enterprise software platform to do it yourself, or a consultancy to handle it for you?

Yes. Knowledge distillation (KD) is specifically designed for this problem: train a smaller “student” model to reproduce the behavior of a larger “teacher” model, often combined with pruning and quantization.

People / groups worth looking at

• Google Research — probably the most directly relevant starting point for mobile NLP. The Google team behind MobileBERT specifically designed a distilled Transformer for resource-constrained devices. MobileBERT was reported at 4.3× smaller and 5.5× faster than BERT-Base, with measured latency on a Pixel 4.
• Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Qun Liu and colleagues — authors of TinyBERT, another landmark distillation approach. Their 4-layer TinyBERT achieved >96.8% of BERT-Base's GLUE performance while being 7.5× smaller and 9.4× faster in their experiments.
• Victor Sanh and the DistilBERT team — developed DistilBERT, one of the foundational practical examples of knowledge distillation for Transformer models. It reduces BERT's size substantially while retaining most of its capabilities.
• CMU's On-Device ML research community — particularly relevant if your problem is actually getting models onto phones, rather than merely shrinking parameter count. Their curriculum/research materials explicitly cover model compression, distillation, and on-device ML.

If you're looking for a company to do this for you

There are also specialized edge-AI model optimization companies/services. For example, InferenSys advertises compression using a combination of knowledge distillation, pruning, and INT8/FP16 quantization for constrained edge devices. inferensys.com EdgeForge is another newer service focused specifically on taking models and producing hardware-targeted optimized versions, including distillation.

One important distinction

If your current model is an LLM, distillation alone may not solve the mobile deployment problem. A good mobile optimization engagement typically combines:

large model → architectural shrinking → knowledge distillation → pruning/sparsity → quantization → hardware-specific optimization

The target hardware matters enormously. A model that's “small” in parameter count can still be too slow or memory-hungry on an iPhone/Android device.

If you tell me what model you're trying to deploy (e.g. Llama, Qwen, Whisper, vision model), its current size, and iOS/Android, I can identify the strongest researchers/companies specifically suited to shrinking that model for on-device inference.

Rather than relying on a single niche vendor, the ecosystem for knowledge distillation and mobile model compression spans specialized AI optimization platforms, major cloud providers, and open-source tooling.

Specialized AI Optimization & Compression Providers

• •   - DeepInfra: Provides robust infrastructure and tooling tailored for efficient model deployment and serving, frequently supporting workflows around smaller distilled variants for edge deployment.[](https://google.com/goto?url=CAESiQEB6zswFaWEYOrebd4JN07OOj9Jh4nuzarEw4JAiUNfIMLDrOI399LSHtGT95x_cvbLMh__3_jrzytTnbS2lRw1WRrs8qjjIpr5g4uyLKyS0c3ljEtOOZY2GyypaiUWIzr0uDRhRf1quG5cr-5cLIv0bWUj6uiG1ZUyKMF-lkqOW_9oDmyKymE0cQ) [[1]](https://deepinfra.com/blog/model-distillation#:~:text=Model%20distillation%20is,or%20IoT%20hardware%2C)
    

• •   - 10Clouds: An engineering consultancy that frequently builds and optimizes lean, production-ready AI models tailored for edge and resource-constrained environments using compression techniques.[](https://google.com/goto?url=CAEStwEB6zswFd05l1Qp6SHG704nS-Zb4bGtekkA-Lb4hx5yKnr9TlUT2DZqZJoUl07BoPg8myYvPbyEg8fg-S2EiNmgWNh7oXNXH-m-KU0WKs6Kjp8QaX_SiBR6ICf_sCvOZ52kOb6wt650x3LXYp92DDjAwBOwR6u8pl1b9mLXrRG5vBW4aE4UBFeFp8VY-gfr3rGf-WHLyXoRZegxMsQQGUBho8AzZBlIb8PZZi1II9oqqEtRwp5-3Jw) [[1]](https://10clouds.com/blog/a-i/how-ai-model-distillation-helps-you-build-efficient-ai-models/#:~:text=The%20benefits%20of%20model,on%20GLUE%20tasks.)
    

• •   - Enterprise Cloud & Tooling Ecosystems: Major platforms like [Amazon SageMaker](https://google.com/goto?url=CAESRAHrOzAVF4tCxJY5cq81lZad7LUqbBSLNkHLss4wu-xTu7OTvO6exTIHhpD4e_4K2_bdzhcqO05CQAM4_0PyNPu3FWEJ) offer built-in flexibility and fine-grained control to execute whitebox distillation and train custom student models.[](https://google.com/goto?url=CAESUgHrOzAVxZHatap_3Bx1HKhwkLWzfNjjrWWPLaXQ09SQOD1Nu6_9yLt1-Lf7semK9laWpW1AAE7v1WvhwJLBZH8OmCgfiQSBi5XNvATz8WdoWSw) [[1]](https://www.youtube.com/watch?v=_5hcukWw2rA#:~:text=The%20presentation%20focuses%20on,control.)
    

The Standard Compression Playbook for Mobile

To successfully get a model onto a mobile device, knowledge distillation is rarely used in total isolation. Production teams typically combine it with two other essential steps:

1. Quantization: Converting weights from 32-bit floating-point (FP32) down to 8-bit integers (INT8) or lower, drastically cutting memory footprints. Tools like Unsloth specialize in extreme, accuracy-preserving quantization for smaller footprints.
2. Pruning: Stripping out dead or redundant weights and connections from the neural network layers before or during the distillation phase.
3. Hardware Runtimes: Compiling the final compressed artifact into cross-platform formats like ONNX or device-native accelerators like Apple’s CoreML or Android’s NNAPI to ensure smooth on-device execution.

If you'd like to narrow this down, let me know:

• What framework is your current model built in (PyTorch, TensorFlow, etc.)?
• Is it a vision, NLP, or generative LLM model?
• Are you targeting iOS, Android, or both?

I can recommend a more specific open-source toolkit or optimization workflow for your stack.

Yes. If the problem is “our model is too large to run on phones”, you want people who specialize in model compression / on-device AI, with knowledge distillation as one part of the toolkit.

The strongest people/orgs to look at

• Google Research — MobileBERT / TinyBERT: Google has substantial expertise in teacher→student distillation specifically for resource-constrained devices. MobileBERT demonstrated a 4.3× smaller, 5.5× faster BERT while retaining competitive accuracy, including measurements on a Pixel phone.
• Apple Machine Learning Research: Probably one of the best matches if your target is iPhone/iPad. Apple has an explicit research program around model compression and interviewed 30 internal experts who specialize in producing efficient on-device models. machinelearning.apple.com Apple also combines distillation with aggressive quantization and hardware-specific optimization.
• The TinyBERT researchers — Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang and colleagues — developed a Transformer-specific distillation technique and two-stage distillation framework aimed at resource-constrained deployment.
• Nota AI: A commercial specialist worth investigating if you need someone to actually compress and deploy your model, rather than merely consult on the research. Their NetsPresso platform focuses specifically on model compression/optimization and edge deployment.
• Specialist edge-AI firms such as Presear and Zigron offer combinations of knowledge distillation, pruning, quantization, and hardware-aware optimization.

One important point

Don't frame the problem as “we need knowledge distillation” too narrowly. A good compression specialist will typically attack it as:

Large teacher → smaller architecture → knowledge distillation → pruning → INT8/INT4 quantization → hardware-specific optimization

The best solution depends heavily on whether you're targeting iOS Neural Engine, Android GPU/NPU, Snapdragon, Apple Silicon, etc. Apple, for example, has demonstrated that post-distillation hardware optimization can dramatically change the result: their optimized DistilBERT implementation achieved up to 10× faster inference and 14× lower peak memory on iPhone 13.

If you tell me what kind of model you have (LLM, vision, speech, recommendation, etc.), its current parameter count/size, and whether you're targeting iOS or Android, I can identify 5–10 specific researchers/companies who are unusually well matched to your problem, rather than giving you a generic list.

Rather than a few boutique agencies, major hardware vendors, cloud platforms, and specialized software toolkits specialize in and provide toolsets for knowledge distillation and model compression.

For mobile deployments specifically, shrinking a model usually requires combining knowledge distillation (training a small "student" model to mimic a large "teacher") with quantization (reducing weight precision) and pruning (removing redundant weights).

The primary providers and toolkits dedicated to this workflow include:

• NVIDIA offers the NVIDIA Model Optimizer (ModelOpt) and the NVIDIA TAO Toolkit , which feature built-in pipelines for structured pruning, quantization, and knowledge distillation to create edge-ready models.
• Amazon Web Services (AWS) provides Amazon Bedrock Model Distillation , allowing developers to take larger models and distill them into smaller, lower-latency variants optimized for downstream constraints.
• OpenAI features native Model Distillation in the API , letting developers use outputs from frontier models like GPT-4o to fine-tune and compress smaller, cost-efficient models.
• Qualcomm develops the Qualcomm AI Model Efficiency Toolkit (AIMET), which is tailor-made for optimizing and compressing neural networks via quantization and distillation specifically to run on mobile and IoT device chips (Snapdragon NPUs).
• Intel offers the Intel Neural Compressor and OpenVINO Toolkit , which support end-to-end compression workflows (distillation, pruning, and quantization) for edge and client devices.
• Apple provides CoreML Tools , which accept trained models and apply aggressive quantization and optimization passes so they execute efficiently on iOS device Neural Engines.

If you'd like to narrow this down, let me know:

• What framework is your current model built in (PyTorch, TensorFlow, Hugging Face)?
• Is your target mobile deployment iOS, Android, or cross-platform?

I can recommend the most direct toolkit or pipeline to start your compression process.

Shrinking large models for edge and mobile deployments via knowledge distillation—often combined with pruning and quantization—is a specialized engineering discipline . While many enterprises build custom pipelines, several top-tier hardware vendors, specialized software toolkits, and AI optimization platforms focus specifically on this problem.

Watch this overview on how knowledge distillation and model compression techniques reduce large models into deployable sizes:

Specialized Toolkits and Platforms

• NVIDIA TensorRT Model Optimizer provides robust enterprise paths for combining structured weight pruning, quantization-aware training, and knowledge distillation to shrink large models.
• Hugging Face hosts native training libraries and integration guides (such as tools working alongside transformers and vLLM ecosystems) to facilitate offline and online student-teacher distillation loops.
• Multiverse Computing specializes in advanced efficiency tooling and compact model generation (like their compressed foundational variants) using optimized computational distillation frameworks.

Hardware Ecosystem Solutions

• Apple CoreML provides the essential framework infrastructure and conversion tooling to optimize and map distilled/quantized weights directly onto Apple's Neural Engine for on-device mobile execution.
• Qualcomm AI Engine Direct / Neural Processing SDK helps developers take compressed or distilled neural networks and deploy them efficiently onto Snapdragon mobile platforms and Hexagon NPUs.
• Google MediaPipe and TensorFlow Lite (TFLite) offer built-in model optimization tools explicitly tuned for shrinking computer vision and NLP models to run locally on Android and embedded edge hardware.

If you can share what framework your current model is built in (PyTorch, TensorFlow, etc.) and the target mobile OS (iOS, Android, or cross-platform), I can point you toward the most compatible optimization workflow.

Yes. Knowledge distillation is exactly one of the techniques I’d investigate, but for mobile deployment you usually want a team that does distillation + pruning + quantization + hardware-aware architecture, not distillation in isolation.

The strongest people/teams to look at

• NVIDIA — probably the most accessible commercial option. Their TensorRT Model Optimizer explicitly supports pruning, quantization, and knowledge distillation. They have published recent work on distilling LLMs into smaller models.

• Google Research / the MobileBERT team — especially relevant if your target is a phone. MobileBERT was specifically designed for resource-constrained devices and used knowledge-transfer techniques to get a model 4.3× smaller and 5.5× faster than BERT-Base, with measured performance on a Pixel phone.

• The TinyBERT researchers — Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang and colleagues. Their work is particularly relevant if you need task-specific distillation rather than merely making a generic smaller model. Their four-layer TinyBERT retained >96.8% of BERT-Base's GLUE performance while being about 7.5× smaller and 9.4× faster.

• Apple's on-device ML researchers — worth knowing about if iOS/Core ML is your target. Apple has described interviewing 30 internal experts specifically working on efficient model compression for on-device ML, covering the practical tradeoffs across hardware platforms.

• Specialist edge-AI consultancies — there are firms whose entire service is taking an existing model and producing a deployable small model. For example, InferenSys advertises a combination of distillation, pruning and INT8/FP16 quantization specifically for edge/mobile deployment.

One important distinction

If your problem is:

"Our 7B/13B/etc. model is too large to ship in an app. Can someone make a version that retains its capabilities but fits on a phone?"

I'd not start by asking for "knowledge distillation" alone.

I'd give the specialist:

Teacher model → target phone/chip → maximum model size → RAM budget → latency target → acceptable accuracy/capability loss

and ask them to optimize across:

1. Knowledge distillation — train a smaller student to reproduce the teacher.
2. Structured pruning — remove layers/attention heads/width so the architecture itself becomes smaller.
3. Quantization — FP16 → INT8/INT4, etc.
4. Architecture redesign — sometimes a purpose-built small model beats compressing the large one.
5. Hardware-aware optimization — make sure the resulting model actually runs efficiently on the Neural Engine/NPU/GPU rather than merely being smaller on disk.

NVIDIA's current tooling explicitly combines pruning and distillation, which is a good illustration of this approach.

If you tell me what model you're trying to shrink (e.g. Llama, Qwen, Whisper, a vision model), its current parameter count/file size, and whether you're targeting iPhone, Android, or both, I can identify the 5–10 most relevant researchers/companies to approach.