Importance sampling is related to rejection sampling, which I looked at in the last post. Here is a short demo.

https://gist.github.com/drbenvincent/b0549ff1ae428a3428b1

A problem of rejection sampling is that many samples could be evaluated in regions of low probability mass. This then lead to a high rate of attrition, with many samples being rejected. In importance sampling, this seems like less of an issue in terms of ending up with a large number of samples for an accurate representation of the distribution. Although the same basic problem is there in that the probability is being evaluated for many points in parameter space with very low or zero probability.

rejection sampling demo

So let’s do the same thing from the last post and use this to do parameter estimation.

https://gist.github.com/drbenvincent/745adfa06cc96925fc10

Which results in this

importance sampling param est

Here is a nice little figure I found that helped with the intuition.

importance-sampling-schematic

Join the conversation

2 Comments

  1. In your first code you don’t need to do ‘proposal = 1/N;’ because you normalise the weights anyway.

Leave a comment

Your email address will not be published. Required fields are marked *