Introducing Envelope.dev

I just released Envelope.dev as a private package index for Python libraries. It's a simple way to host your internal packages that doesn't break the bank. It's also the first generally available product built on Mountaineer.

"Just an internal tool"

Since the jump we've been building our products at MonkeySee with Python. It lets us move as quickly as we can: our average rate of major feature development is about 6 features per week with just me at the keyboard. We've been able to do this because of the incredible ecosystem of packages available on PyPI. But we've also built out our internal frameworks and libraries to help us move even faster.

Envelope started as an admittedly premature optimization. I looked up after a few months of dev work and realized we had a few codebases that we'd want to reuse across a few different projects: billing, authentication, web parsing, etc. Most were too coupled to the rest of our architecture to be open-sourced. Others covered our core IP. Even though we had no other projects already in the pipeline, it felt like it was worth doing the refactor before we'd need them.

I started as simply as I could: I just refactored the code into separate packages in the same repo. This let us use our existing CI pipelines, since every downstream clone of the repo would also receive the private packages. Doing the install from the main web service could easily be simlinked within our pyproject.toml:

[tool.poetry.dependencies]
mountaineer-billing = { path = "./billing" }
mountaineer-auth = { path = "./auth" }

This worked for as long as we only had one project. But when we needed to start on a second project, we had a heavy lift. Do we want to use a mono-repo for all projects?³ Probably not. So we had to:

• Refactor the packages to be even more generic. This also required adding tests that could be run in isolation without our web service logic.
• Place the packages in separate repos.
• Git clone the package repos and install them in their downstream services everywhere we needed to use them.

This git clone approach worked pretty well. By convention, we cut a new git tag whenever we wanted to release a new version of the package:

[tool.poetry.dependencies]
mountaineer-billing = { git = "https://github.com/piercefreeman/mountaineer_billing.git", branch = "v1.0.0" }
mountaineer-auth = { git = "https://github.com/piercefreeman/mountaineer_auth.git", branch = "v1.0.0" }

This worked for a while until we had to publish wheels. Each wheel needs to be built for multiple architectures: our arm64 laptops, our x86_64 servers, and different versions of Python. CI handles this easily - but where do we store them once they're built? Poetry¹ isn't able to dynamically pull wheels from the Assets section of a Github release. If you want this behavior you'll have to install the fully qualified URL of the wheel explicitly on each machine - which changes depending your CPU architecture and Python version.

PEP 503

At the end of the day, what I really needed was just a private PyPI almost at feature parity. Luckily there's a well-documented API for Python package managers specified in PEP 503. Approved back in 2015 it acted as a standardization proposal for how local package managers can fetch remote repository definitions and their included files. Some extensions afterward have clarified how to deal with wheels and other markup to help with performance and local caching. At its simplest, a remote index is just a cascading list of HTML packages that include the a hrefs of different versions to install:

<div>
    <a href="/simple/mountaineer-billing/">
    <a href="/simple/mountaineer-auth/">
</div>

<div>
    <a href="/simple/mountaineer-billing/v1.0.0/">
    <a href="/simple/mountaineer-billing/v1.0.1/">
</div>

The easiest approach seemed to be just generating a static site with the proper HTML structure and hosting it on Github. But the question of how to gate access to the packages remained, since static sites don't support password-protected directories. And the second I was looking at spinning up an additional service to handle this, I realized I was just building a package registry. Might as well do it properly.

Envelope.dev

Enter: Envelope.dev. It's a simple package registry to host these internal packages. Following the Linux philosophy, it does one thing and does it well.

The original Envelope was just a static site. We would tag releases as we normally do in Github and have the CI build action upload artifacts to the release. Every 15 minutes a cron job would pull from the release artifacts and build a new password-protected static site with the PEP 503 convention. We would point all our pyproject installers to the static site path.

That initial scope grew to include:

• Full local caching & remote CDN support
• Package uploading
• Customizable access control with keys (limit in time or package version numbers)
• Full login system

Ironically Envelope become the first service (aside from our primary one) that shared the codebase of internal libraries. So it was the first one that actually would have needed itself.² Now we have 5 different products including Envelope that all pull from the MonkeySee common code, so it was really worth doing the refactor when we did.

Originally I had intended for it to stay private: easier to manage and less overhead. But it occurred to me that if you were to add billing it's basically a fully built out product. It was easy enough to import that mountaineer-billing component and get off to the races.

WYSIWYG

On the landing page I included a brief note about why I'm doing this. I spent as much time thinking about this as on designing the rest of the landing page:

So here’s my core promise:

• I’m focusing on Python. There are existing options for other languages.
• I’m not in this to make money (I do that at my main company). Your subscription here pays for storage and a portion of server costs, that’s it.
• What you see is what you get. I’m not trying to build 100 new features and charge you for ones you don’t need. Judge us by our current features and not future promises.

Way too much software these days are feature complete and keep getting more bloated. People still just buy them for one or two features. So Envelope from the start is going to stay that way. It's a simple package registry for Python packages. Subscription fees go to paying for storage, CDN, bandwidth, and server costs. That's pretty much it.

It's not going to be a unicorn - it might not even get profitable. But we're using it every day and love it. That remains the software that makes me happiest to use: when someone has clearly solved the problem first for themselves. It's hard to match the labor of love that comes from building something you use every day. You sweat a lot more of the details. If this particular problem reasonates with you, check it out at Envelope.dev.

And if it doesn't resonate - that's cool too. Hopefully you're also building something you love. I hope to see it on the Internet soon.