r/devops•
ankittv

Deployment in Spot instances [Spot Fleet] AWS

Hi, Wassup guys. I am working with a Big Data Startup and right now, the majority of out INFRA works on Spot nodes(80%). This is a huge deal as we don't want to spend money on INFRA *cough AWS EC2.

We use spot fleets to manage spot nodes. The way the deployment happens right now is kind of Hacky. Jenkins is used as build and release server.

Right now I make a TAR ball with configs, artifact, scripts and push it to S3. I have a script in all the spot nodes to pull the TAR ball on reboot so that all the nodes have the latest artifact.

Script downloads the TAR Ball updated the configs, Artifact and restart the service. We have 20-30 services running right now and it becomes very painful to make the TAR BALLS and we reboot the servers using Ansible. I need some good framework around spot fleet. I am familiar with the tools like packer that makes AMI, but I don't want to have AMI for every single release.

Another approach, that we are trying right now has a scripts/ansible playbook as a part of your code repository itself and provision/deploy the service.

Right now I am the only DevOps and I need to take care of Automation / Monitoring / Deployment exploring new services etc so this becomes very cumbersome to manage the release process for this many services.

My goal is to have Infra structure as a code first in the company and don't spend time on configuring systems/deployment. I am thinking of writing a framework tool on all this but I just want to know if something already exists to solve this problem.

At the EOD wanna make release engineering as easy as possible. How do you guys are managing your release process? Should I explore hosted solution like code ship / Travis etc? What I want is complete automation that does build/release and deployment in spot nodes seamlessly with as much as less DevOps efforts.

Posted Jul 16, 2017, 10:26 PM UTC

74
Loading comments…