Eddy Chan

giving back to the hacker community

3 notes &

Minimum Viable Ops - Deploying your first Django App to Amazon EC2

So you’ve learnt a bit of django, you’ve built some small, simple apps, you’re pretty confident with “./manage.py runserver” and the results at http://localhost:8000 are starting to make sense. Where to next? In this How-To I’m going to walk you through what I call ‘Minimum Viable Ops’ to get your app deployed onto Amazon EC2 for the first time.

You want to do this because:

  1. You want to show your friends and family what you’re working on by sharing a URL with them
  2. You’ll need to learn enough Ops stuff to build a proper production environment in the future, this is the first step
  3. You want to validate your prototype with prospective customers/beta testers by watching them use your app in the context of their own devices and browser. (similar to 1)
  4. It just feels awesome to deploy, especially if you turn this into a regular habit

So close your IM client, email, turn off your phone and close the HN and Reddit tabs in chrome, let’s get started…

*Disclaimer: all stack technology choices made in this post were made on the rationale of being maximally mainstream and minimally viable.

Launch your Free EC2 Amazon Machine Image (AMI)

First sign-up for Amazon’s EC2 service, it’s a pretty straightforward process where they ask you for some personal details, credit card and verify your identity via an automated phone call. Once you’re confirmed, you can log in to the AWS Management Console and go to the ‘EC2’ tab to fire up a free (for the 1st year) micro instance in the region that’s closest to you.

Now you’ll need to go through the classic wizard, in the ‘Choose an AMI’ step we’re going to use a Community AMI from Alestic. The folks at Alestic have been kind enough to setup Ubuntu to run securely on Amazon EC2 and create an image from that setup which we can simply load onto our instance.

  1. Go to the ‘Community AMIs’ tab and search for “ami-6da8f128” - that’s the ID for a 64 bit Ubuntu image in the US West-1 location. If you want to use a different location you’ll need to look up the correct ID on the Alestic page.
  2. In ‘Instance Details’ you’ll want 1 instance and you’ll need to choose the Micro Instance Type so it qualifies for the free tier.
  3. Next you’ll need to create and download a key pair which will enable you to SSH into your machine later.
  4. For the rest of the ‘Launch Instance’ wizard you can choose the default options presented and skip through to the end to launch it.

Allocate Elastic IP Address and Update Security Groups

Now we’re going to create allocate an elastic IP address to the machine instance we just launched so we can access the machine directly (via HTTP or SSH) over the internet. Go to the ‘Elastic IPs’ link underneath ‘Network and Security’ in the left sidebar and ‘Allocate New Address’. Now go to ‘Associate Address’ and choose the one and only machine that will appear in the dropdown.

This will give you an Associated IP address with which you can access your machine that you launched a moment ago. But before you do that you’ll need to edit your default Security Group so that it allows incoming SSH and HTTP requests. Go to ‘Security Groups’ then click on default and the ‘Inbound’ tab so you add the new rules for HTTP and SSH (If you have a static IP address for your home/office you should put that address into the ‘Source’ field for the SSH rule instead of 0.0.0.0/0). Don’t forget to ‘Apply Rule Changes’.

At this point, it’s a good idea to go to your domain registrar (like Godaddy) to set the A record for one of your domains to point to your Elastic IP Address so you (and the people you show your app to) have something more friendly to work with rather than entering an IP address directly into their URL bar.

Ok - let’s test it out. You should be able to open a terminal, go to the directory the downloaded keypair resides in and SSH into your instance by doing:

chmod 400 MyKeyPair.pem; 
ssh -i MyKeyPair.pem ubuntu@mydomain.com

With the Alestic AMI, the user that’s setup for you to login as is called ‘ubuntu’. You can either use your domain or the IP address where I’ve typed ‘mydomain.com’. You should see something like this:

Install the Required Applications onto the EC2 Instance

Since we’re now able to SSH into our EC2 instance this is where the fun begins. We get to install the technology stack. For this tutorial we’re going setup Apache, Python/Django, PostgreSQL monolithically on the same machine to keep it as simple as possible and also stay within the free tier at Amazon.

Get latest package database and upgrade existing programs to latest version.

sudo apt-get update; sudo apt-get upgrade

Install Apache2 with WSGI

sudo apt-get install apache2 libapache2-mod-wsgi

Install Python Package Installer

sudo apt-get install python-pip

Install Django

sudo pip install django

Install Build Essential

sudo apt-get install build-essential

Install PostgreSQL server/client and PyGreSql, a postgresql module for python.

sudo apt-get install postgresql postgresql-client postgresql-contrib python-pygresql

Install Postgresql-dev-all (extension build tool) so you can build the psycopg extension (next)

sudo apt-get install postgresql-server-dev-all

Install Psycopg, Python Dev 2.7 (so our Django App can connect to PostgreSQL)

sudo apt-get install python2.7-dev; sudo easy_install psycopg2

At this point you Amazon instance now has a full technology stack (OS, web server, database server, Python/Django) that will allow you to run your app. The next 2 hurdles will involve configuring the web server and database server.

Configure Apache To Work With Django App

We’re going to configure Apache so it uses WSGI. Put simply WSGI is the middleware that allows Apache to serve up dynamic content from Django. We’ll also enable Apache to serve static files inside your project directly.

To do this you’ll need to edit /etc/apache2/httpd.conf. I kinda know my way around vim so I’ll be using that as the editor.

sudo vim /etc/apache2/httpd.conf

When you’ve opened the file in your editor you’ll need to add the following:

Alias /static/ /home/ubuntu/my-django-app/static/

<Directory /home/ubuntu/my-django-app/static>
Order deny, allow
Allow from all
</Directory>

WSGIScriptAlias / /home/ubuntu/my-django-app/apache/django.wsgi

Save the file and quit vim. What’s happening here is we’re telling Apache to serve the static files inside ‘my-django-app/static’. We’re also telling Apache to look inside my-django-app/apache/django.wsgi for the WSGI script. As you can probably tell we’ll be deploying ‘my-django-app’ directly into the home directory for ‘ubuntu’ but you can change this if you want.

Setup PostgreSQL Database for Django App

Now you need to setup a postgresql database for your app to use so it doesn’t use SQLite in your ‘production like’ environment (just for a bit of fun really).

Open PostgreSQL client from Linux command line

sudo su postgres -c psql template1

Change ‘postgres’ users password to ‘password’ and then quit the client:

template1=# ALTER USER postgres WITH PASSWORD 'password';
template1=# \q

In the terminal change the linux user ‘postgres’ password to ‘password’:

sudo passwd -d postgres
sudo su postgres -c passwd

Now login again to PostgreSQL client again

sudo su postgres -c psql template1

Create a database user and a database that your app will use:

postgres=# CREATE USER mydjangoapp WITH PASSWORD 'mypassword';
CREATE ROLE
postgres=# create database mydjangoappdb;
CREATE DATABASE
postgres=# grant all privileges on database mydjangoapp to mydjangoappdb;
GRANT
postgres=# \q

Edit the pgSQL conf file

sudo vim /etc/postgresql/9.1/main/pg_hba.conf 

Add following line to the bottom

local   mydjangoappdb     mydjangoapp                         md5

Restart the Postgresql server

sudo service postgresql restart

This sequence of commands will have created a database for which you’ll be able to put the details into the settings.py file of your app.

‘Deploy’ and Configure Your App

Chances are, if you’ve been developing and running Django apps on your local machine you probably haven’t thought much about how to get this code onto an Amazon server like the one you just built. The simplest way to do it is to use a code repository service like BitBucket with Mercurial. That way you can push the code from your own machine to BitBucket and then SSH into your Amazon machine to clone the whole repository or subsequently pull the updates from BitBucket.

Instructions for setting up a BitBucket repository and pushing your code there for the first time is outside the scope of this how-to. On your EC2 instance you’ll need to install Mercurial which will give you the command line tools to clone/pull your code:

sudo apt-get install mercurial

Then can clone your project by running this command in /home/ubuntu:

hg clone https://bitbucket.org/myusername/mydjangoapp

Before we get too excited just yet, there’s a couple of things we need to edit inside our app namely the settings.py file and adding the apache/django.wsgi file.

settings.py

You’re using a postgres database now so we’ll need to change our database settings:

DATABASES = {
    'default': {
         'ENGINE': 'django.db.backends.postgresql_psycopg2',
         'NAME': 'mydjangoappdb',
         'USER': 'mydjangoapp',
         'PASSWORD':'password',
         'HOST' : '',
         'PORT' : '',
     }
}

You also setup Apache to serve your static files from your domain so you’ll need to ensure the STATIC_ROOT and STATIC_URL are set correctly

SITE_ROOT = os.path.dirname(os.path.realpath(__file__))
STATIC_ROOT = os.path.join(SITE_ROOT, 'static')
STATIC_URL = 'http://mydjangoappdomain.com/static/'
STATICFILES_FINDERS = (
    'django.contrib.staticfiles.finders.FileSystemFinder',
    'django.contrib.staticfiles.finders.AppDirectoriesFinder',
    'django.contrib.staticfiles.finders.DefaultStorageFinder',
    )

ADMIN_MEDIA_PREFIX = '/static/admin/'

Save and close your settings.py file.You’ll notice that I put ADMIN_MEDIA_PREFIX down the bottom, what we’re going to do here is symlink the Django Admin static assets to our static directory so they can be served too by Apache.

ln -s /usr/lib/pymodules/python2.7/django/contrib/admin/media /home/ubuntu/appointly/static/admin

apache/django.wsgi

Create a folder in your django app called apache and then create a file called django.wsgi (remember that our httpd.conf for Apache will be looking for this file). Then copy the following into the file:

import os
import sys

path = '/home/ubuntu/my-django-app'
if path not in sys.path:
    sys.path.append(path)

os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'

import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

Save and close the file, we’re almost there… let’s do a syncdb in the app root directory so your tables get created in the postgres database.

./manage.py syncdb

And finally you need to restart Apache so it will serve up your app with wsgi.

sudo /etc/init.d/apache2 restart

Hooray! With any luck you should be going to http://mydjangoappdomain.com and see you your app running just like it does at http://localhost:8000 but now it’s running on a server in the cloud and accessible via the internet. Pretty f@#%$^ epic huh?

Where to from here???

So as I said at the beginning, this is a minimally viable setup for you to learn more about Ops and to do customer validation with your new app, you could possibly plug along like this and sign-up a few free pilot customers while spending most of your time on app dev before you need to come back to make changes to take this setup to the next level.

It’s in no way truly ‘web-scale production ready’, we’ve only taken the first step, there’s heaps of considerations that I haven’t taken into account here like:

  • Should I use Nginx or Apache for my webserver?
  • I need to serve static assets from a different machine, how do I do that?
  • Should I continue with Postgre or move onto Amazon RDS (MySQL), or should I setup MongoDB (a big architectural change)
  • Security concerns - to be honest I don’t know what I don’t know
  • Database Backups and disaster recovery

But you know what? I don’t have enough paying customers to start asking those questions yet, I don’t even have a single paying customer. Until then I’m spending my time building my MVP and doing customer validation.

Join the conversation at Hacker News.

Filed under amazon ec2 ops how-to

0 notes &

aptive asked: Hi Eddy, Read your interesting blog entry about learning python and django, found it on Hacker News. What caught my eye was that you are located in Sydney and doing a Startup. I wanted to ask if you are involved in meetups with other enterpreneurs, and whether it's worth it. I am located in Melbourne, but wanted to make the move to Sydney and also bit of a career change to a smaller company. Cheers, Ivo.

Hey Ivo, to be honest I haven’t been huge on going to meet-ups, not as yet anyway simply because I need to be focusing all my time on building my product right now to get to MVP.

I’m actually building a SaaS online booking/reservation tool for small businesses but doing it right unlike all the other bazillion offerings that are out there right now. Once I’ve got a beta to show people and a story to tell then (2 most important things for networking) I’ll be hitting the streets and going to things like Barcamp, Pycon, Open Coffee meetup and Silicon Beach drinks as well. But in my mind this comes 2nd to spending time meeting customers in my list of priorities. Meeting customers is more important than meeting other entrepreneurs.

My personal startup network consists of a few friends who were in Startmate 2011 and built Noosbox, I also know some of the ‘Aussie Mafia’ in Silicon Valley cos I used to work with them in SF.

Hope that helps!

Eddy

66 notes &

How I Learnt enough Python/Django to be Dangerous in 1 Month

Reading the umpteeth ‘Son, you need to learn to code if you’re going to be a web startup founder’ article I knew it was the first step I had to take. After all, it makes perfect sense right?

You have a vague product idea which you want to turn into a prototype or even possibly a V1; you’re either going to need to pay a good developer market rates to do it for you (1000s of dollars of precious startup capital with no guarantee the output will be satisfactory) or you could do it yourself. There is no ‘Technical Co-founder’, it’s a mythical creature that never makes itself known to mere ‘Ideas Guys’. There are just ‘Co-founders’ and all of them need to be building the product at least in the early days.

This is how I went about picking up the skills I required in 1 month. First, a disclaimer: I will note that I have an unfair advantage over the general populace because I took Computer Science at UNSW in the early 2000s but since graduation I had never touched a line of code except for hacking some VB macros in Excel in 2007. I held a series of Project and Product Management roles and by 2011 I was very much a ‘Product Guy’.

Week 1: Familiarizing Myself with Python

Every man and his dog are building their applications in either Python or Ruby. These 2 languages allow you build more functionality with less code and your time from 0 to deploying a working app is much shorter. I chose to learn Python simply because I perceived there to be more online learning material and a bigger coding community from which to get help. By the same token it felt like I was drinking from the firehose so I hope this post solves that for other newbies. 

My first port of call was Google’s Python Class. I needed to learn the Python syntax, the basic data structures like lists, dictionaries and tuples, control statements and looping. I downloaded the exercises and did them like I was 18 and in Programming 101 again, no peeking at the answers! I had all the resources in front of me, I just had to solve the problems. Google even gives you tests that you can run on your code to give you some ‘woohoo’ moments as you pass them.

Week 2: Introduction to Django

So at the end of week 1, I knew a bit of Python but I couldn’t do anything useful except solve well defined well structured programming problems (like a student). To build a web-app I knew I needed to learn about a web development framework. For Python there’s Django and Pylons among others, again I chose the one I perceived to have the most documentation and support which was Django.

Learning about Django was straightforward. I worked through the first 8 chapters of the Django book in the 2nd week. I didn’t just read, or copy and paste, I worked through and typed out every single code example they gave and made it work to make sure I understood what was going on. Beyond chapter 9 the book was getting too advanced to be useful 2 weeks in so I switched gears to another resource.

Weeks 3 and 4: Django By Example

This set of examples will get you to the point where you are feeling confident that you can build something yourself, even if it’s simple. It took me 2 weeks full-time to go through the examples and type out all the code (don’t copy and paste) into my own projects and work out why it did what it did so it wasn’t ‘magic’ anymore.

This is where the steepest part of the learning curve is but you will build a simple Blog, Forum, Photo Sharing App etc and in the process get a lot of practice in building a Django app starting with the data model, then the views and then the template and then deploying it. Each app required a new useful feature that brings in a slightly higher level of complexity. I highly recommend doing this (but only after the first 8 chapters of the Django Book).

Now You’re Dangerous!

It was Week 5 when I finally felt comfortable in laying down the model for my own app. I’m not saying this is all you need to learn build your app, far from it, this was just the beginning. Currently, I’m making it habit to learn something new from ‘The Hidden Features of Python’ everyday. However, you’re hopefully well on your way to being able to contribute on building your own App, I know I am.

You’ll still need another type of co-founder, the ‘Designer Co-founder’ but that’s a post for another day. Stay tuned for more, as I’m building my App I’m learning heaps of interesting stuff to be posted here.