How Much Carbon Does an AI-Generated Answer Emit?

At best-measured operating points using Qwen3.5 FP8, the server produced approximately 1.36 output tokens per joule, equivalent to about 0.20 kWh per million output tokens.

Applying a facility PUE of 1.2 and 2026 regional grid-intensity values, estimated emissions per million output tokens ranged from:

• 6.9 g CO₂e — France
• 59.0 g CO₂e — Virginia, USA
• 138.0 g CO₂e — China

Pushing concurrency increases energy efficiency by spreading power consumption across more work, but too much concurrency increases wait times.

For DeepSeek-R1 FP4, moving from 4 to 128 concurrent requests cut energy per token by ~80%, but pushed 99th-percentile time-to-first-token from 571 milliseconds to 11.2 seconds.

Adding more GPUs does not automatically yield cleaner results.

For Qwen3.5 FP4 at high concurrency, a two-GPU configuration produced approximately 3,337 output tokens per second per GPU, compared with approximately 2,074 for a four-GPU configuration. The smaller setup was about 60% more productive per GPU.

GPU telemetry alone severely understates use-phase energy. Across microbenchmarks, the server components outside the GPUs (CPUs, memory, networking, storage, fans, and power conversion) accounted for 30% to over 50% of total metered wall power.

Moving from BF16 to FP8 reduced server energy per output token by approximately 55%.

Moving from FP8 to FP4 yielded a much smaller 6% reduction. Making FP8 a compelling operational middle ground for precision and headroom.

Increasing input prompts from 1,024 to 8,192 tokens increased server energy per million output tokens by approximately 84%, illustrating that context length must be managed as a scarce resource.