compile-overview

torch.compile Pipeline Reference

Quick reference for pipeline architecture, logging, and debugging tools.

Pipeline Architecture

Python Source
    ↓
┌─────────────── DYNAMO ─────────────┐
│ Bytecode capture → FX graph (aten) │
│ Pre-grad pattern matching           │
└─────────────────────────────────────┘
    ↓
┌─────────────── AOT ────────────────┐
│ Functionalization (remove mutations)│
│ Decompositions (break down ops)    │
│ Joint graph (fwd+bwd if training)  │
│ Partitioning (separate graphs)     │
│ Post-grad pattern matching         │
└─────────────────────────────────────┘
    ↓
┌─────────────── INDUCTOR ───────────┐
│ Lowering (ATen → IR nodes)         │
│ Scheduling & fusion decisions      │
│ LoopBody generation                │
│ Codegen (Triton/C++ kernels)      │
└─────────────────────────────────────┘
    ↓
GPU/CPU Execution

Stage Summary

Stage	Input	Output	Key Operations	Skills
Dynamo	Python bytecode	FX graph (Full ATen IR)	Capture, guards, pre-grad passes	compile-trace-dynamo, pytorch-dynamo
AOT	FX graph (Full ATen)	FX graph (Core ATen IR)	Functionalization, decompositions, partitioning, post-grad passes	compile-trace-aot, pytorch-aot
Inductor	FX graph (Core ATen)	Kernel code	Lowering, fusion, scheduling, codegen	compile-trace-inductor, pytorch-inductor

Operator IR Hierarchy

torch.compile transforms operations through multiple operator-level IR layers. These are distinct from Inductor's loop-level IR, Scheduler IR, and Triton IR (covered in pytorch-inductor skill).

Operator IR Level	Description	Created By
Full ATen IR	Complete ATen API (includes in-place, out variants)	Dynamo graph capture
Core ATen IR	Functional subset, no mutations/aliases	AOT functionalization
Prims IR	Primitive operators with explicit broadcasting	Decompositions

Operator transformation pipeline:

Full ATen (Dynamo output)
    ↓ Functionalization (AOT Stage 2)
Core ATen (functional ops)
    ↓ Decompositions (AOT + Inductor)
Prims or simpler Core ATen
    ↓ Lowerings (Inductor)
Inductor IR (loop-level: Buffer, Pointwise, Reduction, etc.)

TORCH_LOGS Flags

Dynamo Stage

Flag	What It Shows
dynamo	Capture and graph construction
graph_breaks	Break locations and reasons
graph_code	Generated FX graph code
guards	Guard checks and failures
recompiles	Recompilation events
pre_grad_graphs	Before/after pre-grad passes
bytecode	Bytecode transformations

AOT Stage

Flag	What It Shows
aot	General AOT logging
aot_joint_graph	Joint forward+backward graph
aot_graphs	Partitioned forward/backward graphs
post_grad_graphs	Before/after post-grad passes

Inductor Stage

Flag	What It Shows
fusion	Fusion decisions and scheduling
schedule	Scheduling decisions
ir_post_fusion	Post-fusion IR dump
output_code	Generated Triton/C++ code
cudagraphs	CUDA graphs capture/replay

All Stages

TORCH_LOGS="dynamo,graph_breaks,aot,fusion,schedule,output_code"

Output Files

Location: /tmp/torchinductor_$USER/ (or $TORCH_COMPILE_DEBUG_DIR)

Dynamo Files

File	Content
fx_graph_readable.py	Captured FX graph
fx_graph_transformed.py	After pre-grad passes

AOT Files

File	Content
model__*__joint_*.py	Joint forward+backward graph
model__*__forward_*.py	Partitioned forward graph
model__*__backward_*.py	Partitioned backward graph

Inductor Files

File	Content
ir_*.txt	Pre-fusion IR (IR nodes)
ir_post_fusion_*.txt	Post-fusion IR (LoopBody ops)
output_code.py	Generated Triton/C++ kernels

Debugging Tools

Compiler Bisector

Automatically isolate failures to backend/subsystem/operation:

python -m torch._inductor.compiler_bisector run repro.py

See compile-bisect skill for details.

Config Options

import torch._dynamo.config as dynamo_config
import torch._inductor.config as inductor_config

# Dynamo config
dynamo_config.verbose = True
dynamo_config.suppress_errors = False

# Inductor config
inductor_config.debug = True
inductor_config.trace.enabled = True

Fresh Cache

from torch._inductor.utils import fresh_cache

with fresh_cache():
    # Compilation happens with clean cache
    compiled_fn(x)

Common Workflows

Debugging graph breaks:

1. Enable TORCH_LOGS="graph_breaks"
2. Load compile-trace-dynamo
3. Check break reasons in output

Debugging fusion:

1. Enable TORCH_LOGS="fusion,schedule"
2. Load compile-trace-inductor
3. Check fusion decisions in output

Adding operator support:

1. Check Dynamo capture (VariableTracker support)
2. Add Inductor lowering in torch/_inductor/lowering.py
3. Verify with bisector

Performance investigation:

1. Check AOT partitioning (training memory)
2. Check Inductor fusion (kernel count)
3. Check generated code quality

Skills Overview

Skill	Type	Purpose
compile-debug	Active workflow	End-to-end debugging (bisect → investigate → fix)
compile-bisect	Tool reference	Bisector usage and routing
compile-overview	Documentation	This file - architecture and flags
compile-trace-dynamo	Stage tracing	Debug Dynamo capture
compile-trace-aot	Stage tracing	Debug AOT transforms
compile-trace-inductor	Stage tracing	Debug Inductor compiler
pytorch-dynamo	Implementation	Dynamo internals
pytorch-inductor	Implementation	Inductor internals

---

For active debugging: Use /compile-debug command For stage details: Load stage-specific trace/implementation skills