We often want to divide two measures, that is, find the density (or Radon-Nikodym derivative) of one measure with respect to another. It would be nice to have a tool that performs this operation correctly in different scenarios.
For example, we divide measures when comparing them. We compare continuous measures, as in the classic Gaussian mixture model, by comparing their densities with respect to the Lebesgue measure. But to compare mixtures of continuous and discrete measures, as in Russell’s GPA problem, we have to be careful, and compare densities that are derived with respect to a different measure. We desire a tool that reasons uniformly across both scenarios.
We also divide measures when writing MCMC proposals as probabilistic programs. In the classic Metropolis-Hastings MCMC kernel, the acceptance ratio divides two densities, each with respect to a measure that is left implicit. For many proposals, this implicit measure can be a product of Lebesgue measures, but that won’t work for the single-site proposal (which mixes continuous and discrete measures). We desire a tool that derives the correct densities for both these scenarios as well.
We present such a tool, which takes two measures as input, checks that the first has a density with respect to the second, and produces that density as output. We go further, and develop this tool to need only the first measure as input: it automatically infers the second measure and corresponding density. This tool readily generalizes from density to disintegration, helping us derive correct posterior measures for useful scenarios.