Nvidia Orin Appears in MLPerf

Nvidia Orin Appears in MLPerf

The newest set of MLPerf Inference results showcase the same old vendors; almost all the data-center and edge accelerators came from Nvidia and Qualcomm. Orin was the notable newcomer.
Linley Gwennap
Linley Gwennap

The newest set of MLPerf Inference results showcase the same old vendors; almost all the data-center and edge accelerators came from Nvidia and Qualcomm. The one notable newcomer was Nvidia’s Orin processor, which targets automotive and edge equipment.

Officially in the “preview” category until production shipments in 2H22, Orin achieved 6,139 images per second (IPS) on ResNet-50 v1.5 Offline, less than a low-end Ampere A10 card but using much less power. In fact, Orin scores twice Ampere’s IPS per watt on ResNet, though its lead shrinks on larger benchmarks that stress the edge processor’s diminutive cache.

Qualcomm’s Cloud AI 100 DLA trounces Orin on ResNet at 23,808 IPS. At that speed, however, it uses 75W, and the comparison is unfair since the Nvidia chip lacks host CPUs. To even things up, Qualcomm paired a Snapdragon host processor with a 20W Cloud AI chip for the MLPerf power comparison; this combination nearly triples Orin’s system-level ResNet IPS per watt while also topping Orin’s efficiency on the other three benchmarks Qualcomm submitted.

By packing 18 PCIe cards into a single 2U server, Qualcomm generated greater ResNet-50 performance than a 6U DGX-A100 system with eight Ampere cards. This comparison shows the Cloud AI 100 can deliver greater performance density, at least for image-processing models.

Free Logic Newsletter

Get the latest analysis of new developments in microprocessors and other semiconductor products.


Subscribers can view the full article in the TechInsights Platform.


You must be a subscriber to access the Microprocessor Report

If you are not a subscriber, you can purchase a subscription or you can purchase an article. Enter your email below to contact us about access.