Introducing Envelope.dev

tldr: I just released Envelope.dev as a private package index for Python libraries. It's a simple way to host your internal packages that doesn't break the bank. It's also the first generally available product built on Mountaineer. Check it out.

Since the jump we've been building our products at MonkeySee with Python. It lets us move as quickly as we can: our average rate of major feature development is about 6 features per week with just me at the keyboard. We've been able to do this because of the incredible ecosystem of packages available on PyPI. But we've also built out our internal frameworks and libraries to help us move even faster.

Envelope started as an admittedly premature optimization. I looked up after a few months of dev work and realized we had a few codebases that we'd want to reuse across a few different projects: billing, authentication, web parsing, etc. Most were too coupled to the rest of our architecture to be open-sourced. Others covered our core IP. Even though we had no other projects already in the pipeline, it felt like it was worth doing the refactor before we'd need them.

I started as simply as I could: I just refactored the code into separate packages in the same repo. This let us use our existing CI pipelines, since every downstream clone of the repo would also receive the private packages. Doing the install from the main web service could easily be simlinked within our pyproject.toml:

[tool.poetry.dependencies]
mountaineer-billing = { path = "./billing" }
mountaineer-auth = { path = "./auth" }

This worked for as long as we only had one project. But when we needed to start on a second project, we had a heavy lift. Do we want to use a mono-repo for all projects? Probably not. So we had to:

• Refactor the packages to be even more generic. This also required adding tests that could be run in isolation without our web service logic.
• Place the packages in a separate repo or multiple of them.
• Git clone the repo and install the packages everywhere we needed to use them.

This git clone approach worked pretty well. By convention, we cut a new git tag whenever we wanted to release a new version of the package:

[tool.poetry.dependencies]
mountaineer-billing = { git = "https://github.com/piercefreeman/mountaineer_billing.git", branch = "v1.0.0" }
mountaineer-auth = { git = "https://github.com/piercefreeman/mountaineer_auth.git", branch = "v1.0.0" }

This worked for a while until we had to publish wheels. Each wheel needs to be built for multiple architectures: our arm64 laptops, our x86_64 servers, and different versions of Python. CI handles this easily - but where do we store them once they're built? Poetry¹ isn't able to dynamically pull wheels from the Assets section of a Github release. If you want this behavior you'll have to install the fully qualified URL of the wheel explicitly on each machine - which changes depending your CPU architecture and Python version.

At the end of the day, what I really needed was just a private PyPI almost at feature parity. Luckily there's a well-documented API for Python package managers specified in PEP 503. Approved back in 2015 it acted as a standardization proposal for how local package managers can fetch remote repository definitions and their included files. Some extensions afterward have clarified how to deal with wheels and other markup to help with performance and local caching. At its simplest, a remote index is just a cascading list of HTML packages that include the a hrefs of different versions to install:

<div>
    <a href="/simple/mountaineer-billing/">
    <a href="/simple/mountaineer-auth/">
</div>

<div>
    <a href="/simple/mountaineer-billing/v1.0.0/">
    <a href="/simple/mountaineer-billing/v1.0.1/">
</div>

The easiest approach seemed to be just generating a static site with the proper HTML structure and hosting it on Github. But the question of how to gate access to the packages remained, since static sites don't support password-protected directories. And the second I was looking at spinning up an additional service to handle this, I realized I was just building a package registry. Might as well do it properly.

So I built a simple package registry to host these internal packages. I called it Envelope. It was a simple Mountaineer app that hosted this private PyPI index. It started as just a glorified static site that would pull from the existing github releases every hour, but it quickly grew to include:

• Full local caching & remote CDN support
• Package uploading
• Customizable access control with keys (limit in time or package version numbers)
• Full login system

Overkill? Probably. But here we are. Ironically it become the first service (aside from our primary one) that shared the codebase of internal libraries. So it was the first one that actually would have needed itself.²

At this point, we were still the only ones using it. But if you were to add billing it's basically a fully built out product. It was easy enough to import that billing component and get off to the races. So I decided to make it public. On the landing page I included a brief note about why I'm doing this. I spent as much time thinking about this as on designing the rest of the landing page:

So here’s my core promise:

• I’m focusing on Python. There are existing options for other languages.
• I’m not in this to make money (I do that at my main company). Your subscription here pays for storage and a portion of server costs, that’s it.
• What you see is what you get. I’m not trying to build 100 new features and charge you for ones you don’t need. Judge us by our current features and not future promises.

Way too much software these days are feature complete and keep getting more bloated. People still just buy them for one or two features. So Envelope from the start is going to stay that way. It's a simple package registry for Python packages. Subscription fees go to paying for storage, CDN, bandwidth, and server costs. That's pretty much it.

It's not going to be a unicorn - it might not even get profitable. But we're using every day and love it. And that remains the software that makes me happiest to use and to build, when someone has clearly solved the problem first for themselves. If this particular problem reasonates with you, check it out at Envelope.dev. And if it doesn't - that's cool too. Hopefully you too are building something you love.