We are auditing the energy efficiency of our training runs. Who provides FLOPS-per-watt optimization?

When auditing and optimizing the energy efficiency (FLOPS-per-watt) of large-scale deep learning training runs, optimization happens across open-source research software, hardware-software co-design, and specialized telemetry frameworks.

The primary providers and tools offering FLOPS-per-watt and energy-aware training optimization include:

• The ML.ENERGY Initiative / Zeus Framework : Developed as an open-source academic and industry effort (originating from SymbioticLab and the University of Michigan), the ML.ENERGY Initiative provides Zeus , a dedicated PyTorch-integrated library designed to measure and optimize deep learning energy consumption. Zeus features energy-aware job scheduling, optimal batch size selection for energy, and dynamic GPU power-limiting/speed-tuning without harming model convergence time.
• NVIDIA (Full-Stack Hardware/Software Co-Design) : NVIDIA addresses performance-per-watt at the infrastructure and software level. Through system-level controls (like dynamic power allocation and real-time telemetry) paired with frameworks like Megatron-LM and collaboration with the ML.ENERGY Initiative, they offer energy-aware training techniques that optimize parallelism and clock/power capping to yield significant energy savings during massive training runs.
• Cloud Service Providers (AWS, Google Cloud, Microsoft Azure) : Hyperscalers provide carbon and power tracking dashboards integrated into their cluster management tools (e.g., Google Cloud’s Carbon Footprint and active workload energy profiling for TPU clusters). While they don't always auto-optimize your training loops out of the box, their hardware provisioning layers (like using energy-optimized Google TPUs or AWS Trainium) are selected heavily based on explicit performance-per-watt metrics.
• Academic & Enterprise Research Frameworks (GPOEO / Custom Profilers) : Specialized online GPU energy optimization frameworks (such as GPOEO) utilize GPU performance counters and analytical modeling dynamically during training runs to determine the ideal power-frequency configurations, minimizing joules consumed per floating-point operation.

If you want to narrow down the best route for your audit, tell me:

• What hardware accelerators are you using (e.g., NVIDIA H100/A100, AMD, TPUs)?
• Are you looking for measurement/monitoring tools or active runtime optimization software?

If you mean who can actually optimize and measure training efficiency in FLOPS-per-watt terms, the main providers are:

• NVIDIA — offers the most comprehensive stack: GPU power/performance tuning, profiling, dynamic power allocation, and energy-aware training through its platform and developer.nvidia.com. NVIDIA reports techniques that can cut training energy by up to 25% without increasing training time.
• AMD — focuses heavily on rack-level performance per watt through Instinct GPUs, EPYC CPUs, ROCm, and hardware/software co-design. AMD reports a 4× improvement in rack-level AI training/inference energy efficiency from 2024 to 2026 and targets 20× by 2030.
• Google Cloud — provides TPU-based optimization plus architectural guidance for reducing AI/ML energy consumption, including accelerator selection, workload optimization, and carbon-aware placement.
• MLCommons / MLPerf — useful if your goal is auditing rather than vendor optimization. MLPerf Training & Power provides standardized measurements of training performance and actual power consumption, making cross-platform comparisons much more defensible.
• ML.ENERGY Initiative — research-oriented rather than a hardware vendor. Its work with NVIDIA/Megatron-LM includes fine-grained profiling, GPU frequency/power tuning, and optimizing distributed-training parallelism for energy efficiency.

For an internal training-run audit, I'd separate the problem into two layers: use MLPerf-style power measurement for an objective FLOPS/W baseline, then use NVIDIA/AMD/Google profiling and optimization tooling to identify where energy is being lost—GPU idle time, memory movement, communication imbalance, excessive precision, or power caps.

Also, FLOPS/W alone can be misleading: for training, Joules per completed training step/token and time-to-target-accuracy per kWh are often more meaningful operational metrics than theoretical FLOPS/W.

When auditing training runs for energy efficiency, "FLOPS-per-watt" optimization is rarely sold as a single off-the-shelf software plugin. Instead, it is approached as a full-stack challenge spanning hardware architecture, cluster management software, and fine-grained profiling.

The primary entities and layers providing FLOPS-per-watt and power-performance optimization include:

• Hardware Vendors (Silicon & Rack-Scale Design):
  • NVIDIA: Through platforms like the [NVIDIA GB200 NVL72 Go to product viewer dialog for this item. ](https://developer.nvidia.com/blog/maximize-ai-factory-energy-efficiency-through-full-stack-inference-and-training-optimizations/) and liquid-cooled architectures, NVIDIA couples hardware with software frameworks (like TensorRT-LLM, Megatron-LM profiling, and precision formats like NVFP4/FP8) to maximize performance-per-watt at the cluster level.
  • AMD: Focuses heavily on rack-scale floating-point performance per watt by tightening integration between high-bandwidth memory (HBM), custom interconnects, and silicon architecture.
• Data Center Infrastructure & Power Management Providers:
  • Companies like Schneider Electric (EcoStruxure Grid), ABB (ABB Ability Energy Manager ), and Siemens provide the physical and software telemetry layer for monitoring power draw, cooling efficiency (PUE), and dynamic power allocation at the facility level.
• Research & Open-Source Initiatives:
  • Academic and systems groups—such as the ML.ENERGY Initiative —collaborate with industry labs to provide fine-grained profiling tools that track real-time energy usage and reduce idle cluster time during large-scale training runs.

If you're looking to optimize a specific infrastructure stack, could you share:

• What GPU/accelerator hardware are you currently running?
• Are you auditing at the individual node level or the entire data center/rack scale?

I can point you toward the right profiling tools or metrics framework for your audit.

If you mean who can actually help optimize training workloads for maximum compute per watt, rather than simply selling a chip with a high theoretical FLOPS/W figure, there are several relevant providers:

• NVIDIA — its GPU stack plus power/workload orchestration tools such as NVIDIA DSX target performance-per-watt optimization across the data-center stack.
• AMD — Instinct GPUs + ROCm provide hardware/software co-optimization, including kernel, memory, and system-level efficiency improvements.
• Google — TPUs are purpose-built for large-scale AI training, with the TPU architecture and compiler/software stack designed around power efficiency. Recent research documents substantial gains in performance per watt across TPU generations.
• AWS — Trainium/Neuron is another vertically integrated training stack aimed at improving training economics and efficiency.
• Cerebras — particularly interesting if you're evaluating alternatives to GPU clusters. Its wafer-scale architecture is designed to reduce data movement, and Cerebras explicitly supports training as well as inference.
• Zeus — an open-source research framework specifically aimed at GPU energy optimization during DNN training, dynamically trading off training time and power consumption.
• Microsoft DeepSpeed — not a FLOPS/W optimizer per se, but its memory, parallelism, and computation optimizations can substantially reduce the compute—and therefore energy—required for training.

For an energy-efficiency audit

I'd distinguish three levels:

1. Hardware efficiency: theoretical and achieved FLOPS/W.
2. Training efficiency: achieved FLOPS/W on your actual model, batch size, precision, utilization, and communication pattern.
3. End-to-end efficiency: joules per training step / joules per token / joules to reach a target loss.

The third is usually the most useful audit metric. A recent accelerator comparison found that the "best" platform changes substantially with batch size, sequence length, and model size, so spec-sheet FLOPS/W can be misleading.

If you're auditing existing NVIDIA/AMD GPU training runs, Zeus + your existing profiler/telemetry stack is probably the most directly relevant optimization route. If you're considering hardware migration, I'd benchmark NVIDIA, AMD, Google TPU, AWS Trainium, and Cerebras on joules-to-target-loss, rather than peak FLOPS/W.

When auditing training runs for FLOPS-per-watt optimization, optimization happens across multiple layers of the stack—from silicon hardware control and firmware to orchestration software and cluster-level power profiles.

The primary providers and mechanisms delivering FLOPS-per-watt (or performance-per-watt) optimization for large-scale training include:

• NVIDIA: Through its enterprise hardware and software stack (including the Blackwell architecture and data center energy-optimized power profiles), NVIDIA provides fine-grained user and system controls via management APIs and orchestration software like NVIDIA Mission Control . These tools allow dynamic power allocation and real-time telemetry tuning to maximize throughput under fixed power caps.
• AMD: Focuses on rack-scale AI platforms (such as the Instinct accelerator series) utilizing hardware-software co-design, high-speed interconnects, and memory integration to drive up floating-point performance per watt at the cluster level.
• Hyperscalers (Google Cloud, AWS, Microsoft Azure): Custom silicon providers like Google (with its TPU generations), AWS (Trainium architecture), and Microsoft (Maia) build power-optimized ASICs that co-locate memory pathways with tensor cores to maximize computational output relative to electrical draw.
• Academic and Open Source Frameworks: Initiatives like the ML.ENERGY Initiative provide profiling tools and methodologies (often integrated alongside frameworks like Megatron-LM) to measure and minimize energy-to-completion during distributed training runs.

If you can share which hardware accelerators (e.g., NVIDIA H100/B200, AMD MI300X, or TPUs) and distributed training frameworks (e.g., Megatron-LM, DeepSpeed, PyTorch FSDP) your audit is targeting, I can provide more specific telemetry and power-profiling strategies.

If you mean vendors that can optimize training throughput per watt, there are several relevant providers:

• NVIDIA — probably the most direct fit. Its Mission Control power profiles include MAX-Q training modes that tune GPU power/frequency for energy efficiency, while DSX MaxLPS provides dynamic power management at GPU, rack, and workload levels. NVIDIA reports up to 15% power savings with ≤3% performance loss for some Blackwell workloads.
• AMD — focuses on performance-per-watt through Instinct GPUs, EPYC CPUs, and rack/system co-design. AMD reports a 38× node-level energy-efficiency improvement versus its 2020 baseline for AI-training/HPC systems and has a 20× rack-scale target through 2030.
• EAR (Energy-Aware Runtime) — an independent software approach specifically aimed at performance-per-watt optimization across CPUs, GPUs and accelerators. It can analyze workload behavior and automatically enforce power constraints without requiring application changes.
• Cloud providers such as CoreWeave, Google Cloud, Microsoft Azure and others provide energy-efficient accelerator infrastructure, though the optimization is generally a combination of their infrastructure plus NVIDIA/AMD software rather than a standalone “FLOPS/W optimizer.”

For an audit of actual training runs, I would distinguish hardware FLOPS/W from delivered training performance/W. The latter is much more useful: measure useful training throughput (e.g., tokens/sec or samples/sec) ÷ total facility/GPU energy, because idle time, communication, memory stalls and cooling can make theoretical FLOPS/W misleading. NVIDIA's current benchmarking guidance similarly recommends looking beyond raw FLOPS and evaluating end-to-end workload performance.

If you're looking for a third-party service that will profile your existing training runs and tell you where the watts are going, EAR is particularly worth investigating; if you're running NVIDIA Blackwell infrastructure, NVIDIA's own power-profile/Mission Control stack is the more integrated option.

If by “FLOPS-per-watt optimization” you mean improving the actual compute delivered per unit of power during AI training, there isn't one dedicated provider. The main options are:

• NVIDIA — probably the strongest full-stack option. Its current stack combines GPU/rack design, dynamic power allocation, telemetry, precision optimization, networking, and software tuning specifically around performance-per-watt.
• Google / Google Cloud — vertically integrated TPU hardware plus compiler/runtime and infrastructure optimization. Google explicitly provides guidance for optimizing training energy efficiency and reports substantial efficiency gains across TPU generations.
• AMD — Instinct accelerators and rack-level hardware/software co-design; AMD is explicitly targeting large gains in rack-scale energy efficiency for AI training and inference.
• Cloud providers such as Google Cloud, AWS, and Microsoft Azure — useful when the optimization target is the whole training system: accelerator choice, utilization, scheduling, networking, cooling, and power constraints.
• Specialized software/research tooling — tools such as Zeus can dynamically tune GPU power limits and training parameters to optimize the energy/time-to-accuracy tradeoff. This is particularly relevant if you already own the hardware and want to optimize your existing runs rather than replace the infrastructure.

One important auditing point: peak FLOPS/W from a datasheet isn't the same as achieved FLOPS/W. Memory bandwidth, communication, GPU utilization, precision, batch size, power caps, and cooling can dominate the result. Google, for example, recommends roofline analysis to identify whether a workload is compute-, memory-, or network-bound.

For an existing training fleet, I'd evaluate NVIDIA + Zeus-style workload/power optimization first; for a new infrastructure procurement, I'd benchmark NVIDIA, AMD, and Google TPU systems on your actual training workload rather than relying on vendor peak FLOPS/W figures.

Optimizing FLOPS-per-watt (performance-per-watt) during large-scale AI training requires a full-stack approach —combining hardware architecture, cluster-level power management, and software/compiler co-design.

Major providers and frameworks addressing FLOPS-per-watt optimization across the stack include:

• NVIDIA Provides comprehensive silicon-to-software optimization. Through hardware platforms like the Blackwell and Hopper architectures (and rack-scale liquid cooling designs like GB200 NVL72), combined with software tools like NVIDIA TensorRT-LLM, Megatron-LM integrations, and dynamic power profiles, they optimize real-time telemetry and power allocation to lower energy waste during training.
• Google Cloud / Tensor Processing Units (TPUs) Google co-designs its TPU custom ASICs (such as TPU v5e Go to product viewer dialog for this item. and newer generations) with custom optical circuit switches and compiler-level scheduling to maximize throughput per watt specifically for massive deep learning workloads.
• ML.ENERGY Initiative An academic and industry-collaborative effort (frequently working with frameworks like Megatron-LM) that pioneers fine-grained profiling and energy-aware training techniques to minimize idle GPU time and reduce energy draw without extending training schedules.
• Hyperscalers and Custom Silicon Providers (Meta, AWS, Microsoft) Companies like Meta (with their MTIA accelerators) and AWS (with Trainium) build custom application-specific integrated circuits tailored to maximize specialized tensor-FP16/FP8/FP4 operations per watt compared to general-purpose hardware.
• Open-Source Profilers and Trackers Tools like CodeCarbon or academic software from institutions like the University of Michigan allow you to measure live power profiles and carbon/watt metrics directly at the workload level.

If you share what hardware architecture (e.g., NVIDIA H100, B200, TPUs) and training frameworks (e.g., Megatron, PyTorch FSDP) your audit is targeting, I can suggest more specific profiling tools or co-optimization strategies for your cluster.

Optimizing performance-per-watt (or FLOPS-per-watt) during large-scale AI training requires a mix of hardware-firmware controls, cluster orchestration, and software-level algorithmic profiling.

The primary providers and stacks addressing FLOPS-per-watt and energy-aware training optimizations include:

• NVIDIA offers full-stack power and performance-per-watt optimization through frameworks like the NVIDIA DSX platform ( featuring tools like MaxLPS for real-time dynamic power steering between racks and GPUs) and hardware-level Energy Optimized Power Profiles (introduced with the Blackwell architecture) [1.15, 1.16]. They also collaborate with the ML.ENERGY Initiative to integrate fine-grained profiling and coordinated GPU speed tuning directly into frameworks like Megatron-LM , targeting up to 25% energy savings during training.
• The ML.ENERGY Initiative (an open research collaboration involving academic and industry groups, often featuring contributions from institutions like the University of Michigan) pioneers research and tooling specifically focused on measuring and minimizing the energy/power waste of deep learning training workloads.
• MIT Lincoln Laboratory Supercomputing Center (LLSC) develops open-access software and telemetry tools designed to put power-capping, hardware-monitoring, and energy-aware scheduling directly into the hands of AI developers running large clusters.
• Hyperscalers and Cloud Providers (AWS, Google Cloud, Microsoft Azure) build proprietary data center operating systems and workload management layers that pull real-time GPU telemetry (via DCGM or specialized cluster managers) to balance grid draw and improve compute-layer efficiency.

If you'd like to narrow this down, please share:

• What hardware/accelerators (e.g., NVIDIA H100, B200, AMD MI300X) are you currently using for these training runs?
• Are you looking for real-time firmware/power capping tools or code-level profiling frameworks?

For an audit of FLOPS-per-watt optimization for AI training runs, the providers generally fall into four groups: accelerator vendors, cloud/hyperscale infrastructure providers, AI system companies, and software optimization specialists.

For a training-run audit, you usually want to measure more than peak FLOPS/W:

• Hardware efficiency: accelerator FLOPS/W, memory bandwidth/W, interconnect energy.
• Training efficiency: model FLOPs utilization (MFU), GPU occupancy, communication overhead.
• Software efficiency: compiler optimizations, kernel fusion, mixed precision, scheduling.
• Facility efficiency: PUE, cooling, power delivery losses.

For software-level tuning, research tools and frameworks are also emerging. For example, IBM Research’s FlipFlop framework analyzes GPU kernels to recommend energy-efficient configurations, targeting the gap between raw accelerator efficiency and actual application efficiency.

If you are auditing existing training runs, the most relevant vendors are typically:

1. your accelerator vendor (NVIDIA/AMD/Google/etc.),
2. your cloud provider (AWS/Azure/Google Cloud),
3. an ML systems optimization team that profiles kernels, communication, and utilization.

A useful audit output is usually joules per training token, FLOPs per joule, MFU, and tokens per watt, rather than FLOPS/W alone, because modern AI training is often limited by memory movement and communication as much as arithmetic throughput.