MODULE 03 · LESSON 03

Execution providers

An execution provider runs graph operations supported by one hardware backend.

PLAIN-LANGUAGE INTRODUCTION

What is this?

An execution provider runs graph operations supported by one hardware backend.

One simple example

GPU runs MatMul and Add. CPU runs unsupported CustomNormalize. Data crosses providers once.

A graph and an ordered list of available execution providers.

A graph partition that assigns each operation to a provider.

Placement affects supported operations, transfers, memory, and speed.

Selecting a GPU provider does not guarantee that every operation uses the GPU.

WORK THROUGH THE IDEA

A provider checks which graph nodes it supports. ONNX Runtime partitions nodes using that support.
In this illustration, the GPU claims MatMul and Add.
The custom node is unsupported there. The CPU provider runs CustomNormalize.
Moving its input to CPU adds a device transfer. Logs and profiling reveal the actual placement.
Common mistake: reading the provider list alone does not prove full GPU execution.