Introduction

Diffusion models are stable but slow; GANs are fast but hard to train. Flow Matching aims to get the best of both: stable training and faster sampling.

Over the years, generative models like GANs, VAEs, and autoregressive models have made big strides. Most recently, diffusion models became popular because they’re reliable and produce great images. The catch? They’re slow at sampling—you often need dozens to hundreds of small steps to turn noise into a final image.

GANs flip that trade-off. They generate an image in one forward pass, so sampling is fast. But training is a delicate adversarial game that can be unstable and sometimes collapses to a few modes. Flow Matching tries to bridge these worlds. Instead of denoising in many tiny steps (diffusion) or playing an adversarial game (GANs), it learns a velocity field: a rule for how to move a point of noise toward a real image continuously over time. You train it with a simple regression loss (no adversary, no simulation of the ODE during training). At test time, you solve a deterministic ODE to get an image—often in far fewer steps than diffusion needs.

In this blog, we’ll explain the idea, compare it to GANs and diffusion, present the key equations, and walk through image generation from noise to image with a clean PyTorch implementation on CIFAR-10[2].

How Diffusion Models Work (and Why They’re Slow)

Big idea. Diffusion models learn to clean static off pictures. During training, we take real images and progressively add TV-like noise until they’re unrecognizable. The model sees these noisy versions and learns the small cleanup move that would make each one a bit clearer. Because it practices at many noise levels—from “slightly grainy” to “pure snow”—it becomes a reliable cleaner.

Training phase (why it’s stable).

• Each example is a straightforward cleanup task: “Here’s a noisy image → nudge it closer to a clean one.”

• There’s no adversarial game; it’s supervised and predictable, so training is usually steady.

• Learning across many noise levels teaches the model both big early cleanups and tiny late refinements.

Sampling phase (why it’s slow).

• To generate, you start from pure noise—there’s no picture underneath.

• You apply the learned cleanup move step by step, gradually revealing shapes, then edges, then texture.

• Each step is intentionally small to avoid smearing details, so you need dozens to hundreds of steps.

• If you add guidance (e.g., “make it look more like a cat”), many pipelines run the model multiple times per step, further increasing compute.

• High-resolution images make each step heavier, multiplying the total time.

Why people still use diffusion.

• Reliable training without adversarial instability.

• Excellent image quality: the coarse-to-fine cleanup naturally builds structure first and details later.

• Flexible control at sampling time (class labels, text prompts, style strength, etc.).

Figure 1. Diffusion sampling pipeline—starting from pure noise (t = T), the model iteratively denoises in reverse time to produce a clean generated image (t = 0), with intermediate refinements shown.

How Flow Matching Works

We want a generator that maps samples from an easy source distribution p0 (e.g., standard Gaussian noise) to a complex target distribution p1 (e.g., natural images). Rather than denoising in many tiny steps (diffusion) or training a generator against a discriminator (GANs), Flow Matching learns a time-dependent vector field f_theta(x, t). This vector field prescribes how any point x should move at time t so that, if we start from x(0) ~ p0 and follow the ordinary differential equation below, the distribution of states at t=1 matches the data distribution p1.

Objective (conditional flow matching) :

Training recipe (per minibatch). We construct short, supervised problems by choosing endpoints and an intermediate time, then asking the network to predict the correct velocity at that point along the path:

• Sample endpoints x0 ~ p0 and x1 ~ p1 (optionally with class label y for conditional generation).

• Sample an interpolation time t ~ U(0,1) and form the intermediate point x_t = (1 - t) x0 + t x1 (a straight-line path; spherical/OT paths are also possible).

• The ground-truth velocity along this straight path is constant: v* = x1 - x0 (the direction and speed needed to continue from x_t toward x1).

• Train f_theta by minimizing a simple mean-squared error to this target velocity:

Where

Why this is attractive. The loss is supervised regression—no adversary and no need to simulate the ODE during training. Because (x0, x1, t) are sampled independently for each example, the model learns many local constraints that glue together into a coherent global flow. In practice, this produces diffusion-like training stability while avoiding multi-step noise schedules.

Implementation tips. Encode time t with a small MLP or Fourier features; include class embeddings if using labels. Use standard optimizers (AdamW), cosine LR schedules, and gentle gradient clipping. The target velocity v* and the interpolated point x_t must be computed for each sample; omitting x_t (or the interpolation step) leads to a mismatched objective.

Sampling with flow matching:

After training, generation reduces to solving an initial value problem from t = 0 to 1 using the learned velocity field. Sampling is deterministic given the random seed for x0 (and the label y, if any). You may use a small number of fixed steps for speed, or an adaptive ODE solver for higher fidelity.

Discrete Euler sampling step (most compact form):

• x_t: current sample at time t.

• Δt: step size (e.g., Δt = 1/N for N steps over [0,1]).

• f_theta(x_t, t[, y]): learned velocity—how to move x_t at time t (optional class label y).

• x_{t+Δt}: the next point after one forward step using the predicted velocity.

Integrator choices. Euler is fast and often sufficient (e.g., 20–50 steps). Higher-order solvers like Heun or Runge–Kutta (RK45) can improve quality with moderate extra cost. During sampling, lightweight clamping or normalization of x may improve numerical stability.

Position in the landscape. Sampling typically needs far fewer steps than diffusion (which removes noise in many tiny increments) and is slower than GANs (which produce an image in one forward pass). Flow Matching offers a pragmatic middle ground: stable supervised training and faster, ODE-based sampling.

Flow Matching on CIFAR-10: A Code Walkthrough

1. Setup and Imports

We start with the usual suspects: PyTorch, Torchvision, SciPy for ODE solvers, and Matplotlib for visualization.

2. Attention and Residual Blocks

3. Flow Matching Model

4. Training Objective

The training loop uses AdamW, gradient clipping, and cosine learning rate scheduling.

5. Sampling Images

Two sampling strategies are implemented:

ODE Solver Sampling

6. Visualization

Training Progress: Loss Curve

The first figure shows the training loss curve over 100 epochs. We can see a steady decline, meaning the model is learning to better match the flow between random noise and real CIFAR-10 images. The curve stabilizes after about 60 epochs, suggesting that the model converges to a point where further training only brings marginal improvements. This steady decrease is a good indicator of stable training, especially compared to GANs where loss curves often oscillate unpredictably.

Real vs Generated Ships

The second figure compares real ship images (top row) from CIFAR-10 with generated ship samples (bottom row) produced by our flow matching model. While the generated ships are still slightly blurry compared to the real samples, they clearly capture essential characteristics such as hull structure, decks, and the surrounding water. This demonstrates that the model has learned to transform random noise into meaningful visual patterns that resemble ships, showing the effectiveness of flow matching on a challenging dataset like CIFAR-10.

Real vs Generated Airplanes

The third figure shows a similar comparison for the airplane class. Real airplanes (top row) are sharp and diverse, while the generated ones (bottom row) successfully capture the overall shape, wings, and orientation of airplanes. Some generated samples still lack fine detail, but the results highlight the model’s ability to capture class-specific features. With more training or larger architectures, the generated airplanes could become nearly indistinguishable from real CIFAR-10 samples.

Together, these results show how flow matching produces visually coherent images while maintaining stable and predictable training dynamics, bridging the gap between GANs’ speed and diffusion’s stability.

References

最新情報をメールで取得

© Couger Inc. All rights reserved.