Getting the best performance out of Amazon EFS

tl;dr: EFS is NFS. Networked file systems have inherent tradeoffs over local filesystem access—EFS doesn't change that. Don't expect the moon, benchmark and monitor it, and you'll do fine.

On a recent project, I needed to have a shared network file system that was available to all servers, and able to scale horizontally to anywhere between 1 and 100 servers. It needed low-latency file access, and also needed to be able to handle small file writes and file locks synchronously with as little latency as possible.

Amazon EFS, which uses NFS v4.1, checks all of those checkboxes (at least, to a certain extent), and if you're already building infrastructure inside AWS, EFS is a very cost-effective way to manage a scalable NFS filesystem. I'm not going to go too much into the technical details of EFS or NFS v4.1, but I would like to highlight some of the painful lessons my team has learned implementing EFS for a fairly hefty CMS-based project.

Quick way to check if you're in AWS in an Ansible playbook

For many of my AWS-specific Ansible playbooks, I need to have some operations (e.g. AWS inspector agent, or special information lookups) run when the playbook is run inside AWS, but not run if it's being run on a local test VM or in my CI environment.

In the past, I would set up a global playbook variable like aws_environment: False, and set it manually to True when running the playbook against live AWS EC2 instances. But managing vars like aws_environment can get tiresome because if you forget to set it to the correct value, a playbook run can fail.

So instead, I'm now using the existence of AWS' internal instance metadata URL as a check for whether the playbook is being run inside AWS:

Self-Publish, don't write for a Publisher

I'm not a writer. I'm a software developer who communicates well. Because I'm a developer and software architect, I spend time evaluating solutions to find the best one. There are often multiple good options, but I try to pick the best among them.

When I chose to write a book two years ago, I evaluated whether to self-publish or seek out a publisher. I spent a lot of time evaluating my options, and chose the self-publishing route.

Because I'm asked about this a lot, I decided to summarize my reasons in a blog post, both to posit why self-publishing is almost always the right option for a beginning author, and to challenge publishers to convince me I'm wrong.

Sending emails to multiple receipients with Amazon SES

After reading through a ton of documentation posts and forum topics for Amazon SES about this issue, I finally found this post about the string list format that helped me be able to send an email with Amazon SES's sendmail API to multiple recipients.

Every way I tried getting this working, I was receiving errors like InvalidParameter for the sender, Unexpected list element termination for the error code, etc.

Normally, when sending email, you can either pass a single address or multiple addresses as a string, and you'll be fine:

Subscribe to RSS - amazon