I have a (somewhat unnecessarily) custom setup for managing my dotfiles and I made a nice improvement to it today. How it works The dotfiles are managed by a Python script. In broad strokes, the script will: read a TOML file locate the configured paths create symlinks, based on custom marker text in the filenames, for files in subdirectories under the configured paths1 If there’s a conflict (i.e. two configured paths provide the same symlink target location), the TOML file contains the reso...| pradyunsg.me
The pip team has released pip 24.1b1 which contains a lot of significant improvements and bug fixes. I’d like to highlight a major change in this blog post: the removal of support for “legacy” versions and dependency specification, or as I like to call it, nonsensical versions and dependencies. We’re releasing this as a beta to get feedback from the community on how this change affects their workflows. Nonsensical versions and dependencies For a bunch of historical reasons, pip has al...| pradyunsg.me
I’m someone who gets a lot of GitHub notifications – this week had >100 individual notifications. The problem The “new” GitHub Notifications UI1 was based on Octobox. It was a step in the right direction to help with my notification “workload”, although I recognise that it was not perceived as such by many others. That said, they copied over the exact same design issue I had with Octobox: the action buttons are on the completely opposite side from where my attention is.| pradyunsg.me
I maintain quite a few Python packages and they all have development workflows as well as release processes that are different in various ways. This is basically my research document as I am exploring what I want the release process (and supporting development workflow) to look like for the Python packages I maintain, so that I can reduce the overhead caused by the various differences in these projects as well as the need to make all these decisions myself.| pradyunsg.me
I’m trying to post these roughly once a month. Here’s the December post. I am working on open source projects, as part of an internship at FOSSEE and as a part of grant-funded work on pip’s dependency resolver. Work I did (Dec 11 - Jan 5) Technical Wrore and adopted vendoring for use in pip. Initiated conversation for pipenv to switch to it too. Made progress toward TOML 1.0, via useful clarifications in the specification Continued tackling pip’s build logic refactoring.| pradyunsg.me
I’m trying to post these roughly once a month. Here’s the November post. This update has been a little late, since I have been traveling and recovering from an illness. I also have good news: I will be working on open source projects related to Python Packaging (full time) starting early Dec 2019 till May/Jun 2020. Work I did (Nov 6 - Dec 10) Technical Worked to separate pip’s dependency vendoring logic from rest of pip This would enable other projects that use similar mechanisms (eg.| pradyunsg.me
I’m trying to post these roughly once a month. Here’s the October post. Work I did (Oct 6 - Nov 5) Technical Wrapped up refactoring metadata generation within pip. (yay!) Started refactoring wheel building logic within pip. A few minor fixes/cleanup within pip. Helped out with pip 19.3’s release. Significantly simplified pip’s release processes. Reviewed >50 PRs. Fixed a “major” regression in pip’s detection of virtual environments. Got some work done, progressing closer to TOML 1.| pradyunsg.me
bugs.python.org is the issue tracker for CPython. “bpo”, as it is commonly referenced, is where implementation bugs, smaller feature requests, and documentation issues are tracked as part of Python’s development. PEP 581 proposes sunsetting bugs.python.org, in favor of GitHub issues. “Why not focus on improving Roundup / bpo?” section made me wonder: What is needed to improve Roundup / bpo? This blog post is my attempt at summarizing the state of bpo and how it could be improving in...| pradyunsg.me
I’m trying to post these roughly once a month. Here’s the September post. Work I did (Sept 6 - Oct 5) Technical Figured out a rough structure for approaching pip’s build logic refactor. Discussed and sketched a rough “goal” model for pip’s build logic, to adopt after initial cleanups and decoupling refactor work. Made significant progress on refactoring pip’s metadata handling and generation logic. (yay!) Identified a quirk in how pip’s current resolution logic, which makes it...| pradyunsg.me
I’m trying to post these roughly once a month. Here’s the August post which includes the work I was planning to do this month. Work I did (Aug 6 - Sept 5) Technical Began working on splitting the code for handling of the two kinds of source distributions. Wrap up pip 19.2, with pip 19.2.3 release, unblocking Python 3.8’s final beta. Hooray! Fix documentation builds for pip, on ReadTheDocs – this took way longer to figure out than I’d expected.| pradyunsg.me
I’m going to try to post these roughly once a month. Here’s the late June post. Work I was planning to do Technical Clearly determine what parts of the build system are intertwined with pip’s resolution logic. Communication Get confirmation from my college about required dates/duration of internships in final semester. Work I did (24 June - 6 Aug) Technical Identified portion of build system intertwined with pip’s resolver: “Get abstract distribution”: building installable artifac...| pradyunsg.me
Note: I’ve updated this to reflect how this happened, on recommendation from PDM’s author. PDM claims to implement PEP 582. However, if you look at what it implements, it is something completely different from the standard. PDM’s file system structure __pypackages__ 3.10 bottle myscript.py PDM will scan for the __pypackages__ directory up to 5 folders above the “current” one. PEP 582’s file system structure __pypackages__ lib python3.10 site-packages bottle myscript.| pradyunsg.me
The way the Python packaging community is organised is something that I’ve explained in multiple places, in multiple contexts. I figure that it’ll be useful to actually write it down in a single place, so that I don’t have to repeat myself. The Python Packaging Authority The Python Packaging Authority (PyPA) is a fairly loose group of projects that happen to be related to Python packaging. While the PyPA has a formal governance model, there’s no “oversight” or “enforcement” on...| pradyunsg.me
In late Dec 2022, I played around with ChatGPT in an evening. Since it was on the top of my mind1, I decided try to nudge ChatGPT into the direction of giving me a useful answer about error message design for command line tools.2 Now, tell me what an excellent error message from a command line tool should contain. An excellent error message from a command line tool should contain the following elements:| pradyunsg.me
A false claim was made on the about page for a few days.| pradyunsg.me
When installing with pip (or from PyPI in general), wheels are much faster than source distributions, even for pure-Python projects. Packages with native code are a clearer win, because the wheel file will contain pre-compiled binaries for the platform you’re installing on. This means that you don’t need to have a compiler and non-Python build dependencies installed, and you don’t need to wait for the compiler to do its thing.| pradyunsg.me
Oh hey, backfilling is fun.| pradyunsg.me
Oh hey, backfilling is fun.| pradyunsg.me
April was a somewhat relaxed month.| pradyunsg.me
Woah, time flies!| pradyunsg.me
Making things look better and work better!| pradyunsg.me
New hardware + more time = more stuff done!| pradyunsg.me
A whole lot of logistics.| pradyunsg.me
Release all the things!| pradyunsg.me
Hopping onto the hype train and migrating to Hugo.| pradyunsg.me
More (new) Sphinx stuff and rewriting other things.| pradyunsg.me
Alrighty! Let’s start doing this again.| pradyunsg.me
I’m actually really good at procrastinating things when I’m uncomfortable. - me, back in Feb 2020| pradyunsg.me
This is an attempt to summarize the broader software architecture around dependency resolution in pip and how testing is being done around this area. The motivation behind writing this, is to make sure all the developers working on this project are on the same page, and to have a written record about the state of affairs. Architecture The “legacy” resolver in pip, is implemented as part of pip’s codebase and has been a part of it for many years.| pradyunsg.me
I’m trying to post these roughly once a month. Here’s the January post. I am working on open source projects, as part of an internship at FOSSEE and as a part of grant-funded work on pip’s dependency resolver. Work I did (Jan 6 - Feb 5) Technical Co-worked with another developer, in person, for 1 week, on pip! Triaged pip’s issue tracker (a lot). Spend some time improving pip’s test suite infrastructure.| pradyunsg.me
I’m glad you asked. What’s up? Over the past few months, I’ve worked on a prototype for a resolver in pip, spent some time planning what rolling out a new dependency resolver in pip might look like, and continue to work on pip’s build logic to decouple it from the dependency resolution logic. The prototype has helped me better understand the relationships and contracts between various portions of the codebase. This understanding has led me to conclude that it is a worthwhile investmen...| pradyunsg.me
*tap* *tap* Seems like this is reaching you. Hello there! I’m Pradyun. You probably knew that already. Thanks for spending the time to reach this random piece of writing by me. Acknowledgements This blog used to be built using Jekyll and the theme was based off Pixyll. The theme was heavily restyled to be more in line with how I wanted my site to look like. I ended up making changes to almost every element of the design, though the layout and colours were solid in Pixyll.| pradyunsg.me
My response to the discussion topic posed in Python Packaging Strategy Discussion Part 1 had become quite long, so I decided to move it to write a blog post instead. This post then started absorbing various draft posts I’ve had on this topic since this blog was started, morphing to include my broader thoughts on where we are today. Note: I’ve updated this to cover an aspect of the recent LWN article on the topic as well.| pradyunsg.me