Seven Beautiful Variants of Score Matching

While most researchers including me are focusing on diffusion and flow matching, there are other paradigms that are equally important but are less popular. So I want to bring them up here and explain it quickly:

1. Score Matching (SM)

2. Denoising Score Matching (DSM)

3. Sliced Score Matching (SSM)

4. Noise Conditional Score Networks (NCSN)

5. Implicit Score Matching (ISM)

6. Discrete Score Matching

Let’s unpack Discrete Score Matching (DiscSM) — how score matching extends to discrete data like text, graphs, or binary vectors. DiscSM learns how the model’s probability changes when we perturb the discrete input. similar to how continuous score matching learns how probability changes locally in space. So, DiscSM trains a model to capture local transition structure of the discrete data manifold.

7. Graph Score Matching

Graph Score Matching (GSM) is the natural extension of score matching to graphs, where data are structured, possibly discrete, and permutation-invariant.