App Engine and Django Notes, Round 2

Project: Get Django & Google App Engine to play nice.

Date: 10-Jan-2015

Status: A good start

I've successfully deployed python App Engine code before, and I've used Flask and Django before. However, not in a repeatable way. This time I'm going to start from scratch, without any meta-frameworks to help (until and unless they're needed).

Requirements:

• Repeatable
• An ORM that works (I don't want to embed SQL statements in my code now anymore than I did before I started using django in 2007).
• A current (supported) version of django. App Engine supports 1.5, but that version is insecure.
• Debuggable (in PyDev)
• Free (to start with, anyway)

If you're following along, you'll need python 2.7, virtualenv, pip, and the Google App Engine SDK.

Configuration, configuration

This question on stackoverflow seems very relevant. With that as inspiration I'll give this a shot.

Project Skeleton

mkdir -p ~/root/projects/<app-id>/application
cd ~/root/projects/<app-id>/application
touch app.yaml
mkdir lib
pip install -t lib "django<1.8"

Make app.yaml use the django that I installed:

application: <app-id>
version: 1
runtime: python27
api_version: 1
threadsafe: true

libraries: 
- webapp2
  version: 2.5.2

appengine_config.py

Also, we'll need to create an appengine_config.py file, to let App Engine know that we have libraries in the lib folder.

import sys
sys.path.append('lib')

This file is probably important for other things, but that's good enough for now.

Virtual environment

Create a virtual environment. This is a little touchy, since Google App Engine does it's own environment configuration. However, in order to use django, we need a virtual environment with the right version of django in it. Also, we don't want to have to install packages in more than one directory.

cd ~/root/development/env
python -m virtualenv <app-id>
. <app-id>/bin/activate
cd <app-id>lib/python2.7/site-packages

Now, use a .pth file to tell python that we want to add the application's lib directory to our site-packages:

echo $HOME/root/projects/<app-id>/application/lib > aelib.pth

And to confirm that it works:

(<app-id>)machine:application$ python
>>> import django
>>> django.__file__
'/Users/<you>/root/projects/<app-id>/application/lib/django/__init__.pyc'

Good enough. Now anything that is pip installed into the app engine's lib directory will show up in the virtual environment.

Start up a django project

Django was installed with -t, which means that pip didn't create symlinks to the django command line tools into our virtualenv's bin directory (and the directory didn't exist when django was installed). All this means that we have to use the path to the installed django package's bin directory. Fortunately, you only have to create a project once.

(<app-id>)machine:application$ ./lib/django/bin/django-admin.py start-project <app-id>

Much better. Now, our folder structure looks like this:

~/root/projects/<app-id>/application/
    app.yaml
    appengine_config.py
    lib/
    <app-id>/
        manage.py
        <app-id>/
            __init__.py
            settings.py
            urls.py
            wsgi.py

We ned a small modification in order to get things going, which is to move the django project up a level. Move manage.py, and <app-id>/<app-id> up one, and now it will look like this:

~/root/projects/<app-id>/application/
    app.yaml
    appengine_config.py
    lib/
    manage.py
    <app-id>/
        __init__.py
        settings.py
        urls.py
        wsgi.py

This would be unecessary if we had started a project called application, but that would have been difficult given that we needed application/lib directory first to hold django.

Structure

The directory structure is important, since it influences how you work with the project day to day. A good structure makes it so you don't have to guess about who's getting what resource from where. This one is pretty close to a standard Django layout, and looks like it will work for app engine.

Handler for app.yaml

Now, add a handler to app.yaml and we'll see if we can make something work.

handlers:
- url: .*
  script: <app-id>.wsgi.application

One additional step, comment out the DATABASES entry in settings.py. We'll set up a data store a bit later.

Running

Now, run dev_appserver.py app.yaml.

Good, this gives me the App Engine console on http://localhost:8000, and django's "It Worked!" page on http://localhost:8080.

And the final test, upload this to App Engine and see if it works there too...

(<app-id>)machine:application appcfg.py update .

Poof, now <app-id>.appspot.com is showing me the "It Works!" page. This is a good start.

We should be all set for now.

Results

All in all, I'd say this is a pretty good start. At the very least, it's way more successful than my previous attempt, in which I followed someone else's instructions and failed miserably.

We have:

• A Django 1.7 application
• It's running in App Engine
• We're not paying for anything (yet)

However, we don't have:

• PyDev configuration. PyDev has support for App Engine, so there's hope there.
• An ORM. There's a limit to how much you can do without data storage.