Speeding infra deployments on a slow connection

I've been overseas travelling the last few months and therefore have been using 4G as my primary internet connection.

For most things this has been fine - but where I encountered real issues was doing infrastructure deploys with Terraform / Pulumi. The deploys would take forever, the internet slowing down to a crawl and becoming unusable.

What I put this down to was a lot of uploads happening in a short time frame was causing my internet provider to throttle my requests (they tend to be less generous with uploads than they do downloads as the common consumer's internet activity is primarily downloading - to an ISP heavy uploads are often a symptom of something more dubious - like seeding torrents). I put up with it for a short while until I really couldn't take it any more.

What I ended up doing was roughly:

  1. Spinning up a low tier droplet on DigitalOcean - (A droplet is DO's equivalent of an EC2 instance)
  2. SSHing into it, and doing some basic set up (securing the instance, setting up credentials etc)
  3. Cloning my repo and simply deploying from the instance itself.

Cloud providers have really really fast internet so uploading a few docker images, and making hundreds of API calls ended up taking a tiny fraction of the time it was taking on my crappy 4G connection. If you want to ensure break neck speed, the ideal scenario is to have your deployment instance living in the same cloud provider and same region that your infra lives on. That way, the outgoing request may not even need to leave the network.

The Gold Plated Solution

If I'm being completely honest, the above is somehwat of a bandaid solution to a greater problem - not having a proper CI/CD pipeline set up!

If you set up CI/CD pipeline (hosted with a cloud provider) you get this all for free - you push some code to a branch, the pipline picks up the change and deploys it from a cloud instance.

Still - if you're just working on a solo project or have a small team, you might just want a quick and dirty solution to speed up deploys, in which case just spinning up an instance, manually SSHing into it and deploying from there might be a good way to get a quick win.