Margin distribution

[Data-Designer]

Hi, excellent job. Could you provide some ideas for marginal distribution visualization in Figure 2?
Thanks for your help.

[juntongshi48]

Hi,

Thank you for your question!

Each round circle represents a numerical feature (column), and the color brightness only means to show the feature's noise level $\sigma_t$. The numerical features are diffused in the Variance Exploding manner as proposed in [1]. That is the data distribution is smoothly interpolated with a Gaussian prior centered at 0 with variance $\sigma_{\text{max}}$. Under this interpolation, $x_t^{\text{num}}$ can be found in close form as the top equation in the figure.

Each column of square radio represents a categorical feature in one-hot, with the bottom row representing the [MASK] category. The discrete diffusion smoothly interpolates the data distribution with the prior with all probability mass aggregated at the [MASK] category. Under this interpolation, $x_t^{\text{cat}}$ can be found in close form as the bottom equation in the figure.

Hope this solve your question

[1] Song et al., (2021) "Score-Based Generative Modeling through Stochastic Differential Equations".