I am looking for a model host that allows "serverless" GPUs and does not charge for idle time.

Several modern cloud platforms specialize in serverless GPU infrastructure that scale down to zero when receiving no traffic, meaning you do not pay for idle time (you are billed strictly per-second or per-inference during active execution).

The top model hosts supporting serverless scale-to-zero GPUs include:

- **How it works:** Offers "Flex Workers" which automatically scale to zero when idle and scale up instantly when a request hits your endpoint.
- **Pricing:** Billed strictly per second of active compute time. You pay nothing when there are no requests. (Note: You still pay nominal fees for persistent network/container storage volumes if you keep model weights cached).[](https://docs.runpod.io/serverless/pricing) [[1]](https://docs.runpod.io/serverless/pricing)[[2]](https://www.runpod.io/articles/guides/serverless-gpu-pricing)[[3]](https://www.spheron.network/blog/runpod-h100-pricing-2026/)

- **How it works:** A developer-first serverless platform designed for custom AI/ML model execution, inference, and batch jobs using Python decorators. It scales down to zero automatically after a configurable idle timeout (defaulting to a short window).
- **Pricing:** Billed strictly by actual compute time (CPU/GPU second used). You do not pay for idle resources once the container spins down.[](https://modal.com/pricing) [[1]](https://modal.com/pricing)[[2]](https://modal.com/docs/guide/cold-start)

- **How it works:** Similar to Modal, Beam provides serverless GPU infrastructure tailored for fast cold-starts and custom AI model deployments. It handles autoscaling based on incoming queues.
- **Pricing:** Pay-as-you-go per-second billing for serverless execution, with zero charges when the infrastructure is scaled to zero during inactive periods.[](https://www.reddit.com/r/MachineLearning/comments/1dvqygu/discussion_looking_for_true_serverless_gpu/) [[1]](https://www.reddit.com/r/MachineLearning/comments/1dvqygu/discussion_looking_for_true_serverless_gpu/)[[2]](https://www.beam.cloud/pricing)

- **How it works:** Offers native serverless GPU deployments with built-in scale-to-zero capabilities for AI and containerized applications.
- **Pricing:** Scale-to-zero infrastructure ensures that when your service has zero incoming traffic, your resource allocation drops to zero and you incur no charges.[](https://www.koyeb.com/blog/recap-scale-to-zero-serverless-gpu-price-drop-ai-model-one-click-deployments-and-new-plans) [[1]](https://www.koyeb.com/blog/recap-scale-to-zero-serverless-gpu-price-drop-ai-model-one-click-deployments-and-new-plans)

- **How it works:** Microsoft supports serverless scaling for NVIDIA T4 and A100 GPUs within container apps.
- **Pricing:** Features scale-to-zero GPU capabilities paired with per-second billing so you only pay when an active request triggers execution.[](https://learn.microsoft.com/en-us/azure/container-apps/gpu-serverless-overview) [[1]](https://learn.microsoft.com/en-us/azure/container-apps/gpu-serverless-overview)

To help narrow down the best option, could you share:

• Which specific model or framework you plan to deploy (e.g., a custom PyTorch model, Llama 3, Stable Diffusion)?
• What your expected traffic pattern looks like (e.g., completely sporadic requests vs. occasional predictable spikes)?

Yes. If by “serverless” you mean your GPU worker goes to zero when there are no requests, and you pay $0 for that idle period, there are several good options.

Best fits

• runpod.io — probably the strongest match. Workers scale to zero and billing is per second while the worker is actually running. RunPod explicitly advertises zero idle cost. It also lets you bring your own Docker container, giving you considerable control over CUDA/runtime dependencies.
• Modal — excellent if you're comfortable deploying through a Python SDK. It is genuinely serverless GPU infrastructure, with per-second billing and scale-to-zero. Its developer experience is arguably better than RunPod, but you're working within Modal's Python-native model rather than simply deploying an arbitrary container.
• Baseten — a more managed inference platform. Deployments can scale to zero, and you aren't charged for idle replicas after they have scaled down. The tradeoff is that its pricing is generally higher and billing includes deployment/scaling time.
• fal — particularly interesting for generative-media workloads. It supports serverless GPUs and per-second billing, but be careful: fal explicitly says runner idle time, including keep_alive, is billable. So it isn't quite the same “zero idle cost” model unless your runner actually terminates.

My shortlist

RunPod is especially compelling right now because its Serverless catalog includes everything from smaller GPUs through H100/H200/B200/B300, with per-second billing. Its current published Serverless rates include H100 $4.79/hr, A100 $2.72/hr, H200 $5.93/hr, and B300 $9.98/hr.

One important distinction: “no idle charge” doesn't necessarily mean “no charge between requests.” Some platforms keep a worker warm for faster subsequent requests, and that warm period may be billable. If you want literally zero GPU spend whenever there are zero requests, configure the minimum/warm worker count to 0 and check whether the provider bills startup/teardown time.

If you tell me what model you're hosting (e.g. vLLM/Llama, Whisper, Flux, SDXL), approximate requests/day, and GPU/VRAM requirement, I can narrow this down to the cheapest 2–3 options and estimate your monthly cost.

Yes. If your requirement is “GPU spins up on demand, scales to zero, and I pay nothing while it’s idle,” there are several good options.

Best fits

• runpod.io — Probably the closest match. It runs your containerized model as an autoscaling endpoint, scales workers to zero, and bills by the second only while workers are running. Current Serverless pricing starts around $0.58/GPU-hour for 16 GB-class GPUs; an H100 is listed at $4.79/hr.
• modal.com — Excellent if you're comfortable deploying Python code through its SDK. Modal explicitly says you never pay for idle resources; GPU compute is metered by the second. It handles autoscaling and scale-to-zero for you.
• replicate.com — Very convenient if the model you want is already in its public catalog: public models generally charge only for the time spent processing requests. Caveat: private custom models normally do incur idle charges while their dedicated instance is online, so it doesn't satisfy your requirement as generally as RunPod or Modal.
• inference.sh — Another serverless GPU option with per-second billing and scale-to-zero-style deployment, particularly interesting if you're building AI/agent workflows.

My recommendation

If you're hosting your own model, I'd narrow it to:

| RunPod Serverless | Modal --- | --- | --- Scale to zero | ✅ | ✅ No idle GPU billing | ✅ | ✅ Bring your own model | ✅ Docker | ✅ Python/container Docker flexibility | Excellent | Good GPU selection | Very broad | Broad Easiest deployment | Good | Excellent Best for | Production inference/API | Python-native ML apps

I'd pick RunPod Serverless if you want maximum control over the model/container and GPU, especially for something like vLLM, ComfyUI, Whisper, or a custom inference server. I'd pick Modal if you want the simplest developer experience and your application is naturally Python-based.

One subtle point: “no idle GPU billing” doesn't necessarily mean zero cost between requests. There can still be storage, networking, CPU/memory, or other ancillary charges depending on how you deploy.

If you tell me what model you want to host (e.g. Llama, Qwen, Flux, Whisper) and roughly how many requests/day, I can recommend the cheapest setup and estimate the monthly cost.

Yes, several specialized AI/ML cloud platforms offer true serverless GPU hosting where you scale to zero when there's no traffic, meaning zero cost for idle time . You pay strictly by the second (or per-inference/token) for active compute.

The top providers fitting this description include:

• Modal – Widely praised for developer experience, Modal lets you run Python code and AI models on serverless infrastructure. You pay by the second based on exact active execution time (down to the CPU/GPU cycle), and it scales down to zero instantly when idle.
• RunPod Serverless – Offers serverless endpoints that automatically scale from 0 to thousands of workers. They feature zero idle cost when endpoints aren't running and include optimization options like FlashBoot to drastically cut down cold-start times.
• Baseten – Provides high-performance model infrastructure with scale-to-zero capabilities. You do not pay for idle time on serverless setups; you are only billed when the model is actively processing predictions, scaling, or deploying.
• Beam – Another developer-centric serverless infrastructure platform tailored for deploying AI models and functions on cloud GPUs, billing strictly for active runtime execution.
• Replicate – If you are looking to host open-source models via an API where you don't manage infrastructure at all, Replicate scales your custom models to zero and bills based on the compute-seconds used during inference runs.

Note on the "Serverless Tax": While you don't pay for idle time, active per-second serverless rates are typically higher than traditional hourly dedicated GPU rentals, and spiky traffic can introduce cold starts (a brief delay while the model loads back into VRAM).

To help narrow down the best fit, tell me:

• What framework or model are you trying to deploy (e.g., a custom PyTorch model, a Llama/LLM variant, or a Stable Diffusion pipeline)?
• Do you have a target cold-start latency requirement , or is spiky, infrequent traffic your main priority?

Yes. I searched specifically for serverless GPU model hosting where the GPU scales to zero and you aren't billed while it's idle. As of August 2026, several good options fit.

My top picks

1. RunPod — probably what you're looking for

RunPod Serverless explicitly scales workers to zero when there are no requests, with billing only while the worker is running. Current pricing is per second; for example, their published rates include roughly $0.69/hr for L4-class GPUs, $1.10/hr for RTX 4090, $2.72/hr for A100, and $4.79/hr for H100.

This is especially attractive if you have a model that might receive 20 requests in an hour and then nothing for several hours—you don't pay for those empty hours.

2. Modal — nicest developer experience

Modal lets you deploy a Python function/class with a GPU and automatically starts/stops the container. It bills by container uptime, so a container that has actually scaled to zero doesn't incur GPU charges.

I'd choose Modal if you want something like:

@app.function(gpu="A100")
def generate(prompt):
    ...

rather than managing Docker images and GPU infrastructure yourself.

3. Baseten — polished production serving

Baseten bills GPU compute by the minute and says idle replicas that have scaled to zero aren't charged. It's more opinionated than RunPod and aimed heavily at production inference.

One important distinction

Make sure you're buying the provider's serverless/scale-to-zero product, not a normal GPU instance.

For example:

• RunPod Pod: GPU remains allocated → you pay while it's running.
• RunPod Serverless: workers scale to zero → $0 while idle.
• Modal function: starts when invoked and can shut down → no GPU cost at zero traffic.
• Baseten scaled-to-zero deployment: no idle GPU charge.

What I'd choose

If you tell me what model you're hosting (e.g. Llama 70B, Flux, SDXL, Whisper, a custom PyTorch model), VRAM requirement, and roughly requests/day, I can compare the current RunPod vs Modal vs Baseten vs Replicate economics and tell you which would likely be cheapest.

Here are top web results for exploring this topic:

Reddit·https://www.reddit.com [Discussion] Looking for True Serverless GPU Services – Only Pay ...On-demand and serverless are two different compute models. Most of cloud-based things are on-demand, since you request resources and you get them exclusively. Serverless services scale up or down as n Modal·https://modal.com Top 5 serverless GPU providers - Modal Serverless GPUs refer to a type of cloud computing service that allows you to run GPU-accelerated workloads that automatically scale up and down from 0 based on demand. You pay only for the compute ti Spheron·https://www.spheron.network 10 Best Modal Alternatives in 2026: Serverless GPU Without the ...Modal's serverless GPU model is compelling until cold starts, billing opacity, and lock-in become real problems. Here are 10 alternatives for every workload. DEV Community·https://dev.to I Tested 9 Serverless GPU Providers for AI Inference in 2026. Here's ...If you're shipping AI inference and tired of babysitting GPUs, serverless is the way out. You deploy the model, the platform scales it from zero to hundreds of GPUs and back, and you only pay for the buildmvpfast.com·https://www.buildmvpfast.com Scale-to-Zero Serverless GPUs : Modal vs RunPod vs Replicate Deploy fine-tuned AI models on serverless GPUs that cost $0 when idle. Compare Modal, RunPod, Replicate, Beam, and Baseten on pricing and cold starts. Runpod·https://www.runpod.io Runpod: The AI Developer Cloud AI infrastructure with on-demand GPUs and serverless compute. Run training, inference, and batch workloads on the cloud with Runpod. Fastio·https://fast.io Best Serverless GPU for AI Agents - 2026 Comparison - Fastio Serverless GPU platforms allow developers to run heavy compute tasks like model fine-tuning or high-throughput inference without managing underlying infrastructure or paying for idle time. You send a Blaxel·https://blaxel.ai Top 5 serverless GPU platforms for AI teams in 2026 | Blaxel Blog A serverless GPU platform allocates a GPU only when a request arrives, scales to zero between jobs, and bills only for active compute time. The platform handles driver setup, CUDA initialization, mode GetDeploying·https://getdeploying.com**Serverless GPUs** : 4 Cloud Providers Compared - GetDeploying What exactly is a serverless GPU? At its core, "serverless" doesn't mean there are no servers. Of course, there are - they just aren't yours to manage. Instead of renting a full virtual machine with a Northflank·https://northflank.com Best serverless GPU providers in 2026 | Blog - Northflank What are serverless GPUs? Serverless GPUs let you run GPU-powered workloads without manually provisioning infrastructure. Instead of renting full-time access to GPU machines, you submit a job or deplo

Several modern GPU cloud platforms offer "scale-to-zero" serverless infrastructure, meaning they allocate a GPU only when an API request arrives, bill you strictly by the second (or per token/request) while processing, and charge $0 when idle.

Top options for hosting custom models with zero idle cost include:

• RunPod Serverless : Allows you to deploy custom Docker containers for inference that automatically scale from 0 to thousands of workers. You pay per-second of actual compute execution time, and it scales entirely to zero (no idle charges) when no requests are in the queue.
• Modal : A code-first platform where you write Python functions decorated with your required GPU specifications. It provisions the GPU instantly on demand, bills strictly by the second of execution, and scales back down to zero when finished.
• Vast.ai Serverless : Provides a decentralized marketplace matching workloads to autoscaling serverless GPU capacity. It features per-second billing with scale-to-zero capability so that resources—and charges—are released immediately upon inactivity.
• Beam Cloud : Offers serverless GPU infrastructure tailored for deploying custom AI models and web APIs as autoscaling endpoints. It handles container orchestration and charges only for active runtime.
• Inferless : A dedicated serverless framework for model deployment that supports scale-to-zero configurations, ensuring you only pay per-second when the model is actively processing inference payloads.

If you can share what framework or model you are trying to deploy (e.g., a custom LLM, Stable Diffusion, or a specific PyTorch script) and your expected traffic pattern , I can help you pick the provider with the best cold-start latency and pricing for your stack.

If you want serverless GPU inference with no charges while idle, look for platforms that scale to zero and bill only while a worker is processing or spinning up. Good options:

My shortlist:

1. Runpod Serverless — probably the closest match to “rent a GPU only when someone calls my model.” It supports arbitrary containerized models, scale-to-zero workers, and per-second billing.
2. Modal — best if you are comfortable writing Python and want a serverless programming model rather than a traditional model endpoint.
3. Baseten — best if you are building a commercial API and need deployment tooling, monitoring, and production features.
4. Replicate — easiest if your model fits their ecosystem, but check the exact deployment billing because some private models can incur online-instance costs.

A few things to watch for:

• Cold starts: scale-to-zero means the first request may wait for the GPU/container to start.
• Model loading time: a 10–20 GB model can dominate latency unless the provider offers caching or warm pools.
• Minimum billing windows: “no idle charges” can still mean you pay during startup, shutdown, or a short keep-warm period.

If you tell me the model type (LLM, Stable Diffusion/Flux, speech, embeddings, etc.), expected requests/day, and whether you need an OpenAI-compatible API, I can narrow this down.

Yes. If by “serverless GPU” you mean the GPU worker scales to zero and you pay nothing while it is idle, there are several good options.

Best fits

Modal — probably the cleanest fit. Modal explicitly says you never pay for idle resources and bills compute by the second. It supports custom containers/models and autoscaling GPU workloads. Current listed GPU rates include H100 at about $3.95/hr, A100 80GB at $2.50/hr, and L4 at $0.80/hr.

Runpod Serverless — another very strong choice if you want more traditional container/GPU control. Serverless workers scale to zero, and billing is per second from worker start until it stops; flex workers have no charge while idle. Current rates include H100 $4.55/hr, A100 $2.72/hr, L40S $1.75/hr, and 4090 $1.10/hr.

Vast.ai — worth investigating if lowest GPU cost is the priority. Vast launched a serverless offering with pay-per-execution/no-idle-cost billing, using its GPU marketplace. I would, however, evaluate reliability/availability more carefully than Modal or Runpod before using it for a latency-sensitive production API.

One to be careful about

Replicate can look like a serverless option, but custom/private models generally aren't truly no-idle-cost: Replicate says you're charged while a dedicated instance is online, including time waiting for requests. Its fast-booting fine-tunes are an exception.

My shortlist

If you tell me what model you're hosting (e.g. Llama 8B, Qwen 32B, Flux, Whisper), expected requests/minute, and whether you need an OpenAI-compatible API, I can narrow this down to the cheapest/best option and estimate the monthly cost.

Several modern cloud platforms offer true serverless GPU hosting that scales down to zero when inactive, meaning you pay $0 during idle time and are billed strictly by the second (or per request/token) when active.

Top model hosts supporting scale-to-zero serverless GPUs include:

• Modal - A code-first, Python-native serverless platform. You write standard Python code decorated with annotations indicating GPU requirements, and it spins up instantly, bills by the second of actual execution, and scales completely to zero when idle.
• RunPod Serverless - Offers serverless GPU endpoints (using "Flex" workers) that scale up with incoming API requests and scale down to zero when there is no traffic, ensuring no idle charges.
• Google Cloud Run (GPUs) - Google's fully managed serverless platform supports NVIDIA GPUs (such as the L4) with per-second billing. It automatically scales containers from zero to handle requests and shuts them down entirely when idle.
• Beam Cloud - Designed specifically for AI and machine learning inference, Beam provisions serverless GPU infrastructure on demand, handles fast container/image loading, and stops billing the moment your function finishes executing.
• Replicate - A popular platform for running and hosting machine learning models via an API. You push your model weights or custom code via Cog, and you are only charged for the compute time utilized during inference runs.

If you want to narrow this down, let me know:

• Are you looking to deploy a pre-trained model (like Llama or Stable Diffusion) or a custom container/code base?
• What is your preferred programming language or deployment workflow (Python SDK, Docker container, or CLI)?