Under the hood everything boils down to a sequence of commands put into a buffer that is sent to the GPU. With streams, every time you launch, you have to recreate that buffer, which involves both allocation and encoding of information, and then send it to the GPU. With graphs, we compile the graph first, which does the allocation and encoding of information ahead of time. That compiled information can be reused for multiple launches. So if you're launching the same work multiple times, then graphs almost always when, because they amortize the compilation overhead. A long time ago when we were first launching cuda graphs, I did some benchmarks to evaluate the overhead of streams versus graphs for single use. The benchmark evaluated the cost of launching a linear chain of N kernels. If I recall correctly, both were linear functions, but streams had a y-intercept of zero with a steeper slope (greater cost per additional kernel) while graphs had a non-zero y-intercept (fixed cost) with a shallower slope (lower cost per additional kernel). At the time (I believe ampere era), the crossover point for single use was about 20 noop kernels in the chain - before that streams would be more efficient, after that graphs would be more efficient.