Do I Smell Burning?

Mostly technical things...

Using Amazon S3 to Host Your Django Static Files

(Note, if you haven’t read it already, I recommend my previous article on Django and Static Files to get an understanding of the fundamentals)

Pretty much every Django project I deploy, I use Amazon’s Simple Storage Service (S3) for hosting my static files. If you aren’t particularly familiar with it, then the salient points are:

  • It’s part of the excellent Amazon Web Services (AWS) offering
  • It’s essentially a cloud file store. You have a bucket. You create, read, update and delete files in that bucket.
  • You can make files in your buckets web accessible
  • Amazon are probably better at this than you
  • It’s fairly cheap
    • Storage costs approximately $0.13 per month per GB stored up to 1 TB
    • Inbound data transfer is free
    • Outbound data transfer is free up to 1 GB per month
    • Outbound data transfer between 1 GB and 10 TB per month costs approximately $0.13
    • The cost to the average reader: under $0.15, or free if they’re covered by the AWS Free Usage Tier

Why don’t I use nginx or Apache or whatever webserver I have in front of my Django deployment for static file hosting? Three things:

  • Specialisation - while I have no doubts about the abilities of nginx and Apache to host static files, S3 will inevitably do it far better for far less effort, and it means one less thing for them to do
  • I frequently deploy to Heroku where I don’t have access to the configuration of the httpd layer
  • I find it pretty simple - not much more than half a dozen lines added to `settings.py`

So, first things first, you’ll need django-storages for the STATICFILES_STORAGE class (see my previous article for the role of storages), and boto which is the (excellent) Python AWS library that the unsurprisingly-namedS3BotoStorage uses to communicate with S3.

Assuming you’re inside a virtualenv, this should be pretty straightforward:

1
2
3
4
5
6
7
$ pip install django-storages boto
Downloading/unpacking django-storages
# Snip
Downloading/unpacking boto
# Snip
Successfully installed django-storages boto
Cleaning up...`

(Also if you have a requirements.txt file or setup.py, don’t forget to update them!)

Now that’s all installed, add it to INSTALLED_APPS in your settings.py:

1
INSTALLED_APPS += ('storages',)

You’ll need an S3 bucket to push files to, so head over to the AWS Management Console for S3 and “Create Bucket”, giving it some appropriate name, and picking the most appropriate geographical region for you. You’ll be offered the option to set up logging et cetera, but can happily skip this by just clicking “Create”:

Amazon S3 'Create Bucket' dialog

You’ll also need to get this name into your settings. We’ll do this from os.environ, because we’ve all read the twelve-factor opinions on config right? (Go read it, so if you turn up on IRC for help and I ask you to pastebin your settings.py, you don’t need to go on an extensive redacting spree / expose sensitive information / both)

1
AWS_STORAGE_BUCKET_NAME = os.environ['AWS_STORAGE_BUCKET_NAME']

If you’re on Heroku, you’ll want to add that to your config:

1
$ heroku config:add AWS_STORAGE_BUCKET_NAME='your_bucket_name_here'

Finally, all you need is to have Django use the right static configuration. I tend to wrap this in an if not DEBUG block because I don’t want it while developing (I include the AWS_STORAGE_BUCKET_NAME in that block too, so I don’t need to be too specific about my environment at dev time):

1
2
3
if not DEBUG:
    AWS_STORAGE_BUCKET_NAME = os.environ['AWS_STORAGE_BUCKET_NAME']
    STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'

Voila. Now, with DEBUG set to False, I just need to collectstatic and my static files will be uploaded to S3:

1
2
3
4
5
6
7
8
9
$ python manage.py collectstatic
You have requested to collect static files at the destination
location as specified in your settings.

This will overwrite existing files!
Are you sure you want to do this?

Type 'yes' to continue, or 'no' to cancel: yes
#Snip

And there you have it. Hopefully you shouldn’t have any problems following this guide, but if you have any questions, issues, or feedback (always appreciated!) then please leave a comment, find me on IRC, or catch me on Twitter.

Did this help? Check out my book

Ok so I’m still writing the book so you can’t buy it just yet. But if you want to make sure that you’re serving your static files in the best way possible, you’ll want it:

Check out the book