This post was originally published here. Motivation: we often want to call LLMs in Starlette based apps (FastHTML, FastAPI, etc.) apps, and we don’t want to block the server on network calls to APIs. This post documents my explorations of various approaches that can run tasks in the background without blocking the main process. Tip You can see this notebook here. Using A SQL Database As A Queue We will use fastlite as the interface to our SQL database. Why You are often already using a data...
