Sometimes you just need some elements of the inverse of a sparse matrix. Sometimes you’re working in C++. This is that time.| Un garçon pas comme les autres (Bayes)
I was writing a longer thing, but it was too long, so hey. Let’s just do this for a change| Un garçon pas comme les autres (Bayes)
I am, once again, in a bit of a mood. And the only thing that will fix my mood is a good martini and a Laplace approximation. And I’m all out of martinis. To be honest I started writing this post in February 2023, but then got distracted by visas and jobs and all that jazz. But I felt the desire to finish it, so here we are. I wonder how much I will want to re-write1 The post started as a pedagogical introduction to Laplace approximations (for reasons I don’t fully remember), but it rapid...| Un garçon pas comme les autres (Bayes)
If you gaze for long into an abyss, the abyss creates a cutting-edge AI method. So saddle up your Ganzfeld effect, girls. We ride at dawn.| Un garçon pas comme les autres (Bayes)
Gaussian processes are lovely things. I’m a big fan. They are, however, thirsty. They will take your memory, your time, and anything else they can. Basically, the art of fitting Gaussian process models is the fine art of reducing the GP model until it’s simple enough to fit while still being flexible enough to be useful. There’s a long literature on effective approximation to Gaussian Processes that don’t turn out to be computational nightmares. I’m definitely not going to summarise...| Un garçon pas comme les autres (Bayes)
The time has come once more to resume my journey into sparse matrices. There’s been a bit of a pause, mostly because I realised that I didn’t know how to implement the sparse Cholesky factorisation in a JAX-traceable way. But now the time has come. It is time for me to get on top of JAX’s weird control-flow constructs. And, along the way, I’m going to re-do the sparse Cholesky factorisation to make it, well, better. In order to temper expectations, I will tell you that this post does ...| Un garçon pas comme les autres (Bayes)
Just the other day1 I was chatting with a friend2 about MCMC and he asked me a fundamental, but seldom asked, question: What happens my acceptance probability is a bit off?. This question comes up a bunch. In this context, they were switching from double to single precision3 and were a little worried that some of their operations would be a bit more inexact than they were used to. Would this tank MCMC? Would everything still be fine? What is Markov chain Monte Carlo Markov chain Monte Carlo (...| Un garçon pas comme les autres (Bayes)
Sometimes it’s the parable of the barren fig tree. Sometimes you’re just pissed at a shrub. Paradoxes and counterexamples live in statistics as our morality plays and our ghost stories. They serve as the creepy gas station attendants that populate the roads leading to the curséd woods; existing not to force change on the adventurer, but to signpost potential danger.1 As a rule, we should also look in askance at attempts to resolve these paradoxes and counterexamples. That is not what the...| Un garçon pas comme les autres (Bayes)
If you’re not a machine learner (and sometimes if you are), Gaussian processes need priors on their parameters. Like everything else to do with Gaussian processes, this can be delicate. This post works through some options.| Un garçon pas comme les autres (Bayes)
Eliza knows a little something about monkeys. This will become relevant in a moment. In about 2016, Almeling et al. published a paper that suggested aged Barbary macaques maintained interest in members of their own species while losing interest in novel non-social stimuli (eg toys or puzzles with food inside). I’d never come across the concept of a Graphical Abstract before, but here is the one for this paper. Graphic design is my passion. Source This is where Eliza—who knows a little som...| Un garçon pas comme les autres (Bayes)
At some point in the distant past, I wrote three posts about prior distributions. The first was very basic, because why not. The second one talked about conjugate priors. The third one talked about so-called objective priors. I am suddenly1 of a mood to write some more on this2 topic. The thing is, so far I’ve only really talked about methods for setting prior distributions that I don’t particularly care for. Fuck that. Let’s talk about things I like. There is enough negative energy3 in...| Un garçon pas comme les autres (Bayes)
Imagine you have a target probability distribution and you want to estimate the expectation . That’s lovely and everything, but if it was easy none of us would have jobs. High-dimensional quadrature is a pain in the arse. A very simple way to get an decent estimate of is to use importance sampling, that is taking draws , from some proposal distribution . Then, noting that we can use Monte Carlo to estimate the second integral. This leads to the importance sampling estimator This all seems m...| Un garçon pas comme les autres (Bayes)
Welcome to part six!!! of our ongoing series on making sparse linear algebra differentiable in JAX with the eventual hope to be able to do some cool statistical shit. We are nowhere near done. Last time, we looked at making JAX primitives. We built four of them. Today we are going to implement the corresponding differentiation rules! For three1 of them. So strap yourselves in. This is gonna be detailed. If you’re interested in the code2, the git repo for this post is linked at the bottom an...| Un garçon pas comme les autres (Bayes)
This is part five of our ongoingseriesonimplementing differentiable sparse linear algebra in JAX. In some sense this is the last boring post before we get to the derivatives. Was this post going to include the derivatives? It sure was but then I realised that a different choice was to go to bed so I can get up nice and early in the morning and vote in our election. It goes without saying that before I split the posts, it was more than twice as long and I was nowhere near finished. So probably...| Un garçon pas comme les autres (Bayes)
Just some harmeless notes. Like the ones Judy Dench took in that movie.| Un garçon pas comme les autres (Bayes)
This is part three of an ongoing exercise in hubris. Part one is here.Part two is here. The overall aim of this series of posts is to look at how sparse Cholesky factorisations work, how JAX works, and how to marry the two with the ultimate aim of putting a bit of sparse matrix support into PyMC, which should allow for faster inference in linear mixed models, Gaussian spatial models. And hopefully, if anyone ever gets around to putting the Laplace approximation in, all sorts of GLMMs and non-...| Un garçon pas comme les autres (Bayes)
Come for the details, stay for the shitty Python, leave with disappointment. Not unlike the experience of dating me.| Un garçon pas comme les autres (Bayes)
Back in the early days of the pandemic I though “I’ll have a pandemic project”. I never did my pandemic project. But I did think briefly about what it would be. I want to get the types of models I like to use in everyday life efficiently implemented inside Stan. These models encapsulate (generalised) linear mixed models1, (generalised) additive models, Markovian spatial models2, and other models. A good description of the types of models I’m talking about can be found here. Many of th...| Un garçon pas comme les autres (Bayes)
Every day a little death, in the parlour, in the bed. On the lips and in the eyes. In the curtains in the silver, in the buttons, in the bread, in the murmurs, in the pauses, in the gestures, in the sighs.Sondheim The most horrible sound in the world is that of a reviewer asking you to compare your computational method to another, existing method. Like bombing countries in the name of peace, the purity of intent drowns out the voices of our better angels as they whisper: at what cost. Before ...| Un garçon pas comme les autres (Bayes)
But I got some ground rules I’ve found to be sound rules and you’re not the one I’m exempting. Nonetheless, I confess it’s tempting. – Jenny Toomey sings Franklin Bruno It turns out that I did something a little controversial in last week’s1 post. As these things always go, it wasn’t the thing I was expecting to get push back from, but rather what I thought was a fairly innocuous scaling of the prior. One commenter (and a few other people on other communication channels) point...| Un garçon pas comme les autres (Bayes)