Django at the Ann Arbor Software Development Study Group

I was supposed to lead a jam last December at the Ann Arbor Software Development Study Group, but a client project postponed it. On Tuesday, February 2, we’ll try one more time. The topic is “Zero to Django: Writing Web Apps with Python Using the Django Framework”. By the end of the session, you will be collecting form data and redisplaying it on a webpage.

NOTE: This exercise is meant to get you collecting data quickly and I’ll be skipping a lot of explanations. Feel free to ask questions or research any areas that you may find particularly tricky.

On with the jam!

Pre-Requisites:

Though there is only one requirement for this session - installing Django - you'll want to get this done ahead of time. It's not difficult, but there are a few steps and it will take enough time that it could keep you from completing all the exercises. To install Django, go to http://docs.djangoproject.com/en/dev/topics/install for a detailed description of what you need to do. Before following that link, here's a couple of suggestions:

• Be sure to install Python 2.5 (Python 2.6 will probably be fine). Django will not run with the 3.x series of Python.
• You can skip installing Apache and mod-wsgi. This session is only about how to use the framework which can be done completely within the development environment provided by Django.
• You can also skip the section on getting a database running. We'll be using SQLite which is baked into Python 2.5 and later versions.
• Be sure to install an official release of Django. Under "Installing an  official release", click on "download page" in line item 1. Use the version under "Option 1: Get the latest official version".

The Exercise:

Create a folder where all of your Django apps can you live. I'm using Windows and I'm putting all of my Django projects in the C:\django directory.

1. From the command line, navigate to the directory you just created
2. from the command line, run "python django-admin.py startproject people". You may have to explicity give the path of the django-admin.py file which can be found in python25/Lib/site-packages/[django dir]. If you copied the file as suggested by the installation directions, just use that version of the file instead. Also, you may get a "permission denied" message. If that's the case, check your permissions on the folder.
3. If you successfully completed step 3, you should now have a "people" directory. On my machine, the path is c:\django\people.
4. Navigate to the directory mentioned in Step 4 and run "python manage.py runserver"
5. Open up a browser and navigate to "http://localhost:8000/people". You should see a message congratulating you on "your first Django-powered Page"

Now that you’ve created a page, let’s configure our site so that we can do something useful.

• Django Projects contain one or more Apps (OK, they don’t have to contain Apps, but if you want to connect to a database they do). To create an app, run “python manage.py startapp dataentry”. The “manage.py” file can be found in the root of the people directory you created earlier. On my machine, the path is C:\django\people\manage.py.
• In the “settings.py” file contained within the project root, configure the database. For this exercise we’ll use SQLite since it comes bundled with Python 2.5 and all later versions:

  DATABASE_ENGINE = 'sqlite3'
  DATABASE_NAME = 'peopledb'
  DATABASE_USER = ''
  DATABASE_PASSWORD = ''
  DATABASE_HOST = ''
  DATABASE_PORT = ''

• In the settings file, you need to add your app to the INSTALLED_APPS section. Your INSTALLED_APPS section should look like the following (note the “people.dataentry” on the last line):

  INSTALLED_APPS = (
      'django.contrib.auth',
      'django.contrib.contenttypes',
      'django.contrib.sessions',
      'django.contrib.admin', 
      people.dataentry',
  )

• Again in the settings file, add “"/templates",” to the TEMPLATE_DIRS setting.
  
  Don’t worry about the extra lines. Those go beyond the scope of this session. If they really bother you, you can remove them. Just be aware that you’ll need add the extra comma (,) at the end so Python understands that this is a single item tuple.

Now that the configuration is done, it’s time to setup your database. To do that, we’ll need to open up model.py file inside your application directory. On my machine, that’s “c:\django\people\dataentry\model.py”. Your model should like the following:

from django.db import models

class Person(models.Model):
    T_SHIRT_SIZE_CHOICES = (
        ("WS","Women's Small"),
        ("WM","Women's Medium"),
        ("WL","Women's Large"),
        ("S","Small"),
        ("M","Medium"),
        ("L","Large"),
        ("XL","X-Large"),
        ("2XL","2X-Large"),
        ("3XL","3X-Large"),
    )
    name = models.CharField(max_length=30)
    t_shirt_size = models.CharField(max_length=10,
        choices=T_SHIRT_SIZE_CHOICES)
    special_dietary_concerns = models.BooleanField(
        default=False)
    special_dietary_concerns_comments = models.TextField(null=True,
        blank=True)

    def __unicode__(self):
        return "<Person %s %s %s>" % (
            self.name,
            self.t_shirt_size,
            self.special_dietary_concerns)

Now that your model is in place, let’s make sure that it’s reflected in the database.

1. From the command line in the directory where the model.py file lives (“c:\django\people\dataentry\model.py” on my machine), run “python manage.py syncdb”. A series of tables will be created.
2. You will be prompted to create a superuser. Type “yes” and continue through the prompts.

That’s it! The Person class is now in the database. But how do you use it.? To test it out, run “python manage.py shell” from the command line. You’ll be placed into Python’s Interactive Console. Now run the following commands

1. from dataentry.models import Person
2. p = Person()
3. p.name = “Me”
4. p.t_shirt_size = “L”
5. p.save()
6. Person.objects.get(id=1)

That last command should have printed “<Person Me L False>” in the console. Assuming it did, how do we get this form on a webpage? Before we do that, let’s make a webpage first.

1. Create a new folder called “templates” in the “c:\django\people\dataentry” directory.
2. In the “c:\django\people\dataentry\templates” directory, create a new file called “template.html”
3. In the template.html file, paste the following snippet “<h1>{{ hello }}</h1>” . Save the file.

You’ve just created a very basic template. To use it, make sure that your views.py file looks like the following:

from django.shortcuts import render_to_response

def hello(request):
    hello = "Hello, World"
    return render_to_response("template.html", locals())

Open “c:\django\people\urls.py” and make sure the following code is in it:

from django.conf.urls.defaults import *
from people.dataentry import views

urlpatterns = patterns('',
    (r'^hello/$', views.hello)
)

All the pieces should be in place for you to actually view your webpage. From the command line, run “python manage.py runserver”. Open your favorite web browser and navigate to http://localhost:8000/hello. You should be rewarded with “Hello, World” in a very large font. That’s all well and good, but what about that model we created? Let’s put it to good use right now. Let’s create a list.html in our template directory that looks something like the following:

<html>
    <head>
        <title>{{ title }}</title>
    </head>
    <body>
        <table>
            <tr>
                <th>Name</th>
                <th>T Shirt Size</th>
                <th>Special Diet</th>
                <th>Special Diet Comments</th>
            </tr>
        {%  for person in people %}
            <tr>
                <td>{{ person.name }}</td>
                <td>{{ person.t_shirt_size }}</td>
                <td>{{ person.special_dietary_concerns }}</td>
                <td>{{ person.special_dietary_concerns_comments }}</td>
            </tr>
        {% endfor %}
    </body>
</html>

Now let’s create a new method in our view so that our views.py file looks like the following:

from django.shortcuts import render_to_response
from people.dataentry.models import Person

def hello(request):
    hello = "Hello, World"
    return render_to_response("template.html", locals())

def list_people(request):
    title = "List of People"
    people = Person.objects.all()
    return render_to_response("list.html", locals())

Switch to the urls.py file and wire a url to our new method. Our updated urls.py file should look like the following:

from django.conf.urls.defaults import *
from people.dataentry import views

urlpatterns = patterns('',
    (r'^hello/$', views.hello),
    (r'^list/$', views.list_people),
)

From your web browser, navigate to http://localhost:8000/list. You should see the person you entered in the Interactive Console. Cool! By now you should see a pattern on how to create a new page in Django: create a template, create a new method in the view and then edit the urls.py file. Let’s add a bit more complexity by creating a form. Create a new forms.py file in the “C:\django\people\dataentry” directory. Put the following code block in that new file:

from django import forms
from models import Person

class PersonForm(forms.ModelForm):
    class Meta(object):
        model = Person

Create a new template in the template folder called “form.html” and put the following html inside it:

<html>
    <head>
        <title>Adding a Person</title>
    </head>
    <body>
        <form method="post">
            <table>
                {{ form.as_table }}
            </table>
            <input type='submit'>
        </form>
    </body>
</html>

Switch over to your view.py file and modify it so that it looks like the following:

from django.shortcuts import render_to_response, HttpResponseRedirect
from people.dataentry.models import Person
from people.dataentry.forms import PersonForm

def hello(request):
    hello = "Hello, World"
    return render_to_response("template.html", locals())

def list_people(request):
    title = "List of People"
    people = Person.objects.all()
    return render_to_response("list.html", locals())

def add_person(request):
    title = "Add Person"
    if 'name' in request.POST:
        form = PersonForm(request.POST)
        if form.errors:
            errors = form.errors
        else:
            form.save()
            return HttpResponseRedirect("../list")
    else:
        form = PersonForm()
    return render_to_response("form.html", locals())

Now all you have to do is wire up the url in the urls.py file. Now when you successfully add a new person using the form you will be redirected back to the list of people where you will see your entry at the bottom of the list. 

IterHelper Released

I’ve mentioned IterHelper in my last few posts and I’m glad to say that I finally released a version of it that I feel is solid. I’ve run my unit tests on Python versions 2.5 and 2.6 and also on IronPython 2.0 and 2.6.

It doesn’t do anything earth-shattering at this point, but it does have some functions I find particularly useful such as a couple of filter methods and a number of “skip” and “take” methods. For a complete rundown of what it does, just run “help(IterHelper)”

I’ve used LaunchPad to manage my project and the experience has been good thus far. If you want to download it, you’ll need to do so from the trunk. I’m eventually moving to Python eggs, but until I work out the particulars, you’ll have to do include it in your project manually.

Euler Problem 1 Revisited

The other day at the Software Development Study Group in Ann Arbor, Chris Marinos was demonstrating F#, a functional programming language targeting the .NET framework. Of course, this got me thinking about programming styles in Python. So what better way to compare and contrast programming styles than with Euler problems? As you probably don’t recall, the first Euler problem is:

If we list all the natural numbers below 10 that are multiples of 3 or 5, we get 3, 5, 6 and 9. The sum of these multiples is 23.

Find the sum of all the multiples of 3 or 5 below 1000.

The traditional approach to this might look something like the following:

def traditional_approach():
    total = 0
    for i in range(1,1000):
        if i % 5 == 0 or i % 3 == 0:
            total += i
    return total

It’s not a bad approach, but there are more concise approaches. Here is my original solution using a list comprehension:

isValid = lambda x: x % 5 == 0 or x % 3 == 0

def with_list_comprehension():
    filteredValues = [x for x in range(1,1000) if isValid(x)]
    return sum(filteredValues)

I like list comprehensions, but this feature does introduce a lot of noise characters. If you use list comprehensions, you probably like this solution, otherwise you’re looking for something else. The next approach uses the itertools module (and the isValid lambda from the previous example).

def with_itertools():
    filteredValues = itertools.ifilter(isValid, range(1,1000))
    return sum(filteredValues)

This approach removes a couple of noise characters and is structured in a way familiar to most developers. My last solution uses a module called IterHelper that I proposed in a previous post. As I mentioned in that post, IterHelper sits on top of the itertools module and implements some of the recipes from the official itertools documentation. It’s largely inspired by my desire to use LINQ from both Python and IronPython. Here’s my solution to Euler Problem 1 using IterHelper:

def with_iterhelper():
    return IterHelper(range(1,1000))\
        .where(isValid)\
        .sum()

Thanks to the Python continuation character, a backslash, I can put discreet units of functionality on their own line, making it easier for me to consume.

If you decide to download and try out IterHelper, be aware that I’m not even considering it alpha yet. It’s still very much in the R & D phase and I’m planning on changing some of the method names.

Note to Self 12 – Lessons Learned from Our Latest Project

I know I’m supposed to be releasing my IterHelper tool right now, but I’ve been on a bit of death march lately and have had no time. Though the project hasn’t ended yet, there are some items that I need to store away for future use before they’re forgotten.

• On small teams, non-development related responsibilities should be divided among the team. Then the single non-dev responsibility (hopefully it’s the only one) should always take precedence over development responsibilities. It sounds backwards, but no matter what my priorities, I’m always going to find a way to write code. The same isn’t true of project management.
• I can’t say enough about having a non-developer - who I’ll refer to as the Customer Advocate – on the project team. Things we had problems making time for – client meetings and acceptance testing for instance – fell to the Customer Advocate which takes a lot of pressure off everyone else.
• Story points are great, but in order to get value out of them you actually have to use them – obvious, I know. Estimating tasks and then tracking how many points were completed daily, turned an out of control project into a manageable one.
• Speaking of scoring, I wish we would have re-scored each task after we completed it. It would have taken minimal effort while at the same time providing us with enough information to improve our estimates.
• We started using Agile Zen at the beginning of our project, but eventually abandoned it in favor of physical notecards. This introduced a lot of challenges, but it did allow us to grow our process exactly the way we wanted it. The added benefit is we’re in a much better position to evaluate the various kanban applications for our next project. To be clear, Agile Zen is a great product it just wasn’t what we needed at the time.
• We didn’t use Selenium or Watir to run acceptance tests. Even if we couldn’t run all the tests we wanted, it would have been a huge time saver. A costly mistake that won’t be repeated.

Does IronPython need LINQ?

Before I get started, I think I need to clear up some misconceptions that the Python community might have about LINQ. From the posts I’ve read, Python developers have dismissed LINQ as only a series of language enhancements that allow for SQL-like querying of lists. Though it is true that both C# and VB.NET have made changes to their respective languages for working with lists, you can still use LINQ without the aforementioned language enhancements. Python developers would be better served to think of LINQ as wrappers around the map, filter and reduce functions. In fact, LINQ is not much different than the “recipes” found in the official itertools documentation.

So the question still stands: does IronPython need LINQ? In my last post, I described how to use LINQ from an IronPython application and implied that I didn’t think it was the best idea. Don’t get me wrong, I really enjoy writing LINQ statements, I just don’t think LINQ fits well with IronPython. Why not? The first problem I have with the IronPython/LINQ combination is put on prominent display in this code sample.

print Enumerable.Average[object](
    list, Func[object, int](lambda x:x))

If you are a static language aficionado, you probably are wondering what the problem is. If you’re a Python developer, however, the fact that I’m explicitly naming the “object” and “int” types is sending up giant, red flags. Python folks don’t really care what type of object it is they’re using, they only want to know what the object can do and what it contains (i.e., duck-typing). It’s much easier – and more readable - to simply write a completely Python version of the Average function. Consider the following chunk of LINQ code from C#:

List<string> values = new List<string>("the quick brown fox jumped over the lazy dogs".Split());

double avgLengthOfLongWords = values
                .Select(x => x.Length)
                .Where(x => x > 3)
                .Average();

Console.WriteLine(avgLengthOfLongWords)

To convert that to IronPython, we’d be forced to write something like the following:

import clr
clr.AddReferenceToFileAndPath("c:\users\darrell\desktop\System.Core.dll")

import System.Linq
from System import Func
from System.Linq import Enumerable

stuff = "the quick brown fox jumped over the lazy dogs".split()

def Select(col, func):
    return Enumerable.Select[object, object](col, Func[object, object](func))
def Where(col, func):
    return Enumerable.Where[object](col, Func[object, bool](func))
def Average(col):
    return Enumerable.Average[object](
    col, Func[object, float](lambda x:x))

avgLengthOfLongWords = Average(Where(Select(stuff, lambda x:len(x)), lambda x:x > 3))
print avgLengthOfLongWords

Before I’m even able to write useable code, I need to add a reference to a dll, import a namespace and a couple of classes and then write some helper functions; all so that I can write LINQ statements without the benefits of chaining. That’s right – I have to nest my LINQ statements. No chaining. That’s because when we use IronPython, the Select and Where don’t return an IEnumerable or an IQueryable. Instead they returna WhereSelectEnumerableIterator.

But do you know what really rankles me about using LINQ in IronPython? It contributes absolutely nothing to the Python community. The only way you can use LINQ is if you are using IronPython, not CPython, not Jython. I want to engage the entire Python community, not just my corner of the world. If ever there was a case where implementing the principle trumps migrating the implementation, this is it.

Enter the itertools module. Itertools is a Python standard library module containing tools for working with iterators. It also will serve as the cornerstone for my implementation of LINQ. When you visit the official itertools documentation at python.org, you’re even presented with code “recipes” that could be described as LINQ-ish. The only problem is that the recipes only accomodate nesting of statements. Why is this a problem? Consider the differences in readability between nested and chained in the following example:

stuff = "the quick brown fox jumped over the lazy dogs".split()

getLength = lambda x:len(x)
isNotTooLong = lambda x:x<6
isNotTooShort = lambda x:x>3

nested = ifilter(isNotTooShort, ifilter(isNotTooLong, imap(getLength, stuff)))

chained = IterHelper(stuff).select(getLength).where(isNotTooLong).where(isNotTooShort)

As always is the case with nesting methods, you have to read the statement populating nested backwards. That is, the outermost function call is the last to be run and you work your way inwards from there to find the origin. This is not a big deal to most of us since this is something we had to get used to very early in our careers. But when you compare that to the statement populating chained, I know I can read it from left to right. My eyes aren’t jumping back and forth to understand what is happening in what should be a relatively simple statement.

So what does IterHelper look like?

class IterHelper(object):
    def __init__(self, someIterable):
        self.someIterable = someIterable
    def __iter__(self):
        return self.someIterable
    def next(self):
        return self.someIterable.next()
    def select(self, func):
        return IterHelper(itertools.imap(func, self.someIterable))
    def where(self, predicate):
        return IterHelper(itertools.ifilter(predicate, self.someIterable))

This extremely simplistic version of IterHelper is really only a wrapper around functions found in the itertools module. This is great news since any application written using Python 2.3 or later can make use of this pattern with little effort. There’s no special libraries AND everyone is invited regardless of your flavor of Python.

My plan is to release IterHelper as an open source project by sometime by mid-October. I have a well-tested, basic implementation now, but I still have questions regarding licensing, hosting and deployment. I will be doing a bit of research on these topics and be posting what I’ve learned in my Notes to Self series.