We reduced by over 60% the EC2 instance cost of our 1000+ node globally-distributed log-producing application by migrating it entirely to EC2 Spot Fleet. We discuss some of the issues we faced and also present the log data recovery mechanism which helped enable this migration.| NextRoll
The attribution team at AdRoll computes metrics out of petabytes of data every night. This is accomplished using a batch processing pipeline that submits jobs to AWS Batch. In this blog post we discuss how this is organized and orchestrated with Luigi. We also announce Batchiepatchie, a job monitoring tool for AWS Batch.| NextRoll