Suggested Project: Probabilistic Programming Examples
This is the first in a series of posts, where we suggest some personal projects, that you may take on during the course. Preferably, we will do the projects in small teams of 2-3 people.
For an overview of the various projects - you may look here.
Background
Probabilistic programming is one of the promising and refreshing approaches to probabilistic modelling and to artificial intelligence nowadays.
It is a fascinating field, lying in the intersection of programming language theory, compilation, probability and Bayesian statistics. It calls us to put aside our usual programmer mind, and be open to new ways of thinking (in a sense, thinking backwards).
Clojure has several libraries for probabilistic programming. The main ones are Anglican, Bayadera, Distributions and Metaprob. During our meetings, we will study a little bit of Anglican, and meet the main person behind its implementation, David Tolpin.
The project
On this project, you will study one of those libraries and extend its documentation by writing a series of examples.
Suggested tentative plan
- Choose a probabilistic programming library. We recommend one of the following:
- Anglican – if you are interested in flexibility, language design and expressive power.
- Bayadera – if you are interested in high performance and GPU computing (note that it is rather under-documented, and may be a bit challenging to study)
- Choose a some example-oriented textbook about Bayesian Methods.
- Bayesian Methods for Hackers by Cameron Davidson-Pilon is one good candidate, as it is very code-oriented.
- Reading through the book, see if you can reimplement the examples in the library that you have chosen.
- Document your examples, preferably in some form of literate programming (Gorilla-REPL / Jupyter / org-mode / ...).
- Prepare a talk about your work.
- Write a blog post about your experience and conclusions.
Recommended materials
Reading
- Anglican's website.
- A draft of an upcoming book about probabilistic programming (by Jan-Willem van de Meent, Brooks Paige, Hongseok Yang and Frank Wood)
Talks
- Frank Wood about Anglican - a bit outdated
- Dragan Djuric about Bayadera - also recently here
- Michael Lindon about Distributions
- Vikash Mansinghka about Metaprob