Stupid simple serverless with usage-only pricing
While discovering a small repo that we made public by mistake, we wondered how could we prevent this or at least react sooner. We immediately devised a solution based on a polling agent that would repetitively query the GitHub API for our organization’s public repos and track the changes in time, hooked up with some notification system (Slack channel in our case). Later on we chose to handle the specially designed GitHub hooks for this, but for the sake of the exercise, it’s still interesting to think about the polling agent with history persistence approach.
Because this is the kind of program that runs from time to time and because a tech company gets to build numerous such automations, it makes sense to deal with it systematically. The distilled list of requirements below:
Minimal setup costs - don’t pay per application, don’t pay per developer, don’t waste time configuring much, if anything at all; if the transactional cost of starting up such an app is low, we’ll get the chance to automate more
Minimal running costs - no dedicated computational resources, so that means no recurrent costs per unit of time, which basically means it should be usage-based
General purpose - it should be in a conventional tech stack and it should be capable of supporting a diverse set of needs (computation, persistence, networking)
Looking at our exiting tech stack, there are a few contenders:
AWS Lambda - excluded because we don’t consider AWS to be a developer-friendly platform which means setup costs aren’t minimal; we use it extensively, but shield our application developers from it
Heroku - excluded because of running costs; for every automation we develop in isolation, we’d need one web dyno running, regardless of throughput
A Linux VM, e.g. DigitalOcean droplet or EC2 machine - excluded because the setup is very involved - secret isolation, multi tenancy of apps and the low-level nature of the approach make it unattractive
Although there are ways to work around these issues, as a great Python programmer once said, there must be a better way!
Serverless platforms to the rescue
Some of the more popular serverless computing platforms we took into account
One interesting approach that piqued our interest was Serverless.com. Because they rely on a cost / developer pricing model and we have hundreds of developers across the organization, it's an option that’s hard to digest, however, Cloudflare, one of our service providers, already has a solution in that space, with the added benefit of running the code close to the users with a very reasonable usage-based pricing. So we gave it a try. They are by no means the only ones that do it, the landscape being filled with solutions, like Firebase Cloud Functions, however, Firebase’s pricing model is not as simple and, apparently, not as low.
In part 2, we describe test driving Cloudflare Workers with our implementation of the GitHub automation.