Screencast on Arunrocks

Understanding Test Driven Development with Django

Thu, 06 Mar 2014 22:15:00 +0530

Test-driven Development (TDD) has been getting a lot of attention these days. While I understand the importance of testing, I was sceptical of Test-Driven Development for a long time. I mean, why not Development-Driven Testing or Develop Then Test Later? I thought figuring out the tests before one can write even a single line of code would be impossible.

I was so wrong.

Test-driven Development will help you immensely in the long run. We will soon see how. We will approach TDD from a sceptical viewpoint and then try to create a simple URL shortener like bit.ly using TDD. I will conclude with my evaluation of the pros and cons of the technique; to which you may directly jump to as well.

What is TDD?

Test-driven development (TDD) is a form of software development where you first write the test, run the test (which will fail first) and then write the minimum code needed to make the test pass. We will elaborate on these steps in detail later but essentially this is the process that gets repeated.

This might sound counter-intuitive. Why do we need to write tests when we know that we have not written any code and we are certain that it will fail because of that? It seems like we are not testing anything at all.

But look again. Later, we do write code that merely satisfies these tests. That means that these tests are not ordinary tests, they are more like Specifications. They are telling you what to expect. These tests or specifications will directly come from your client’s user stories. You are writing just enough code to make it work.

For instance, your client needs a website (in Django) with a Article model that specifies a headline and a body. The very first thing to do in TDD would be to write a test which checks this specification. Then you run the test. Watch it fail. Then write three lines of code in models.py to create an Article class with a headline and body.

But wait a minute. Isn’t this cheating? Your client actually wanted you to create a Blog application. But your three lines are no where near the functionality of a blog. This is an uncomfortable thought that bothers you when you start with TDD.

Just ignore it for now. It is not that much of a big deal. In fact, even your client wouldn’t be too bothered about it. They would be happy that you are working on their application one spec at a time. They can actually watch the progress (and trust me they love that, especially management).

In any form of engineering, this is a common way of building things - there is a plan and we build it bit-by-bit based on it. Before a building is constructed, a blueprint is drawn and progress is made floor-by-floor. Prior to every mobile phone being manufactured, there are detailed CAD models to ensure that every part fits each other perfectly. So why should software engineering be any different?

How to do TDD?

Now, let’s see how test-driven development is done. Just follow this simple procedure:

Decide what the code will do: This is usually told to you by the client.
Write a test: The test should pass only if the code does that.
Run the test: It will fail.
Write some code: Just enough to make it pass.
Run the test: If it fails go to step 4.
Refactor the code: Tests will ensure that it doesn’t break specs.
Rinse and Repeat: Take another spec/user-story/feature and go to step 1.

Now that you know that tests are like specifications, you might be seeing some method in this madness. But this method is more familiar to you than you might think.

This is, in fact, quite similar to the Scientific method which is the basis of modern science. Let’s recollect what scientific method teaches us:

Define a question
Gather information and resources (observe)
Form an explanatory hypothesis
Test the hypothesis by performing an experiment and collecting data in a reproducible manner
Analyse the data
Interpret the data and draw conclusions that serve as a starting point for new hypothesis (go to step 3)
Publish results
Retest (frequently done by other scientists)

Compare this with the previous steps for TDD and notice the similarities. Tests in TDD take the role of experiments in Science. Your theory is only good if the experiments are repeatable and verifiable. You will see that the same will hold good for tests in your project’s source code when you work with other developers.

In fact, let’s look at collaborative software development happening in large open source projects. Almost all of them would have a good collection of tests. While writing test cases seem like a good idea, are there any good reasons to write tests before code?

Why do TDD?

So it seems that that TDD is not a arbitrary practice after all. But we still don’t have to follow it. There are plenty of ways to develop software.

But TDD does come with its own set of advantages, some of which are obvious and some which are not:

It is live documentation that grows, lives and changes with your code.
Improves design
Catches future errors
Long-term time savings
Reduces technical debt and hence risk
Avoid manual one-off tests. Eventually, you will add and re-add test data to test by hand. Too hackey.

These are advantages gathered from various developers who practice TDD. Each of them merit a detailed explanation. But to summarise, TDD brings with it a lot of benefits of testing by making it a mandatory part of your development cycle. You might think that you will add tests later, but sometimes you never get around to doing it.

Code written by passing simple, focused test cases tend to be more modular and hence better designed. It is a pleasant side effect of the process but you will certainly notice it.

How is TDD done?

To understand TDD better we will try to implement an entire Django project by writing test cases first and then the code. We will be creating a URL shortening services which takes a long url and converts it into a short one (possibly for fitting into a twitter message).

You can also watch the screencast below to see how the site is created.

User Stories

Imagine that after a long call with your client, you have distilled their needs into the following user stories:

The short URL must be always smaller than the original URL.
If you give the short URL, you must be able to recover the original URL.
The home page must have a form to enter the long URL.
Submitting the home page form should show the short URL.
Clicking on the short URL must redirect to the original (long) URL.

We have also roughly ordered the stories so that the core functionality comes first.

Create the Project

Note: You will need at least Python 2.7 and Django 1.6 to follow the rest of this post. Earlier versions of Python and Django had some differences in unit testing tools.

Typically URL shortener sites will have a short name like http://ti.ny. So let’s call our project tiny

django-admin.py startproject tiny
cd tiny
./manage.py startapp shorturls

Configure DATABASES in tiny/settings.py to use a simple file-based sqlite3 database and add the app shorturls in INSTALLED_APPS as well. Next, synchronise the database with:

./manage.py syncdb

Writing the First Test

Add your first test case to shorturls/tests.py:


from django.test import TestCase
from .models import Link


class ShortenerText(TestCase):
    def test_shortens(self):
        """
        Test that urls get shorter
        """
        url = "http://www.example.com/"
        l = Link(url=url)
        short_url = Link.shorten(l)
        self.assertLess(len(short_url), len(url))

This test creates a simple Link object given a (long) URL. It creates a short URL by using a class method called shorten() and asserts that it is shorter in length. Note that we are not saving the Link object into the database. Whenever a test can avoid touching the database, it must jump at the opportunity. Django uses an in-memory database while testing but even that can take some time. Faster the unit tests are, the more likely you are to use them.

Also, notice that we get a better error message when we use the assert...() functions (see the Sidenote below).

Run:

./manage.py test shorturls

As expected, it fails. Now build a model for this to work in shorturls/models.py:

from django.db import models

class Link(models.Model):
    url = models.URLField()

    @staticmethod
    def shorten(long_url):
        return ""

We are cheating by returning a zero length string but we know that this will pass the test.

./manage.py test shorturls

Sidenote: Choosing the right assertion

There are several assert functions provided by the unittest module in Python and several more provided by Django in the TestCase class. Initially, they might feel redundant. After all, assert is a keyword built into Python. Every conceivable assertion function can be replaced by an assert keyword checking for the truth of a Python expression.

The real difference is when the assertion fails. Here are three equivalent assertions along with the typical error message when it fails. Compare them yourself:

    assert len(url) < len(docs_url)
AssertionError

    self.assertTrue(len(url) < len(docs_url))
AssertionError: False is not true

    self.assertLess(len(url), len(docs_url))
AssertionError: 101 not less than 58

Clearly, the self.assert...() functions have a clearer error message. So it is worthwhile to familiarise ourselves with the assert API unless you plan to you something like pytest.

Second Test: Recovering the url

“But you wouldn’t clap yet. Because making something disappear isn’t enough; you have to bring it back. That’s why every magic trick has a third act, the hardest part, the part we call “The Prestige”.” — Christopher Priest, The Prestige

Add another test to shorturls/tests.py:

def test_recover_link(self):
    """
    Tests that the shortened then expanded url is the same as original
    """
    url = "http://www.example.com/"
    l = Link(url=url)
    short_url = Link.shorten(l)
    l.save()
    # Another user asks for the expansion of short_url
    exp_url = Link.expand(short_url)
    self.assertEqual(url, exp_url)

Our bluff gets called. We now need a real way to shorten urls and recover them. Unlike what might come to you intuitively, the URL is not mapped to a more compact encoding like a zip file. The allowable character set for a URL is pretty limited. Any kind of ‘string compression’ would reach its limits.

Instead we do something very simple. We know that, once saved into the database, each Link object can be uniquely identified by an integer - its primary key. Simple add this primary key to the domain’s URL and we have a short URL that can be mapped back to the original URL with a database lookup.

Change shorturls/models.py to this:

from django.db import models

class Link(models.Model):
    url = models.URLField()

    @staticmethod
    def shorten(link):
        l, _ = Link.objects.get_or_create(url=link.url)
        return str(l.pk)

    @staticmethod
    def expand(slug):
        link_id = int(slug)
        l = Link.objects.get(pk=link_id)
        return l.url

Now the tests should pass.

Third Test: Home Page With a Form

Add to shorturls/tests.py:

from django.core.urlresolvers import reverse
...

def test_homepage(self):
    """
    Tests that a home page exists and it contains a form.
    """
    response = self.client.get(reverse("home"))
    self.assertEqual(response.status_code, 200)
    self.assertIn("form", response.context)

This test fails because we don’t have any views mapped in our urls.py. In the spirit of minimum effort, let’s use Django’s class based view. Since the submission of a form would create a new Link object, let’s use a CreateView instead of a TemplateView. A CreateView will generate the form for free and it will be useful later on.

Replace contents of shorturls/views.py with:

from django.views.generic.edit import CreateView
from .models import Link


class LinkCreate(CreateView):
    model = Link
    fields = ["url"]

Create shorturls/templates/shorturls/link_form.html:

<form method="post">{% csrf_token %}
{{ form.as_p }}
<input type="submit" value="Shorten" />
</form>

Replace contents of tiny/urls.py with:

from django.conf.urls import patterns, include, url
from shorturls.views import LinkCreate

urlpatterns = patterns('',
    url(r'^$', LinkCreate.as_view(), name='home'),
)

Now the test should pass.

Fourth Test: Form Returns a Short URL

Add to shorturls/tests.py:

def test_shortener_form(self):
    """
    Tests that submitting the forms returns a Link object.
    """
    url = "http://example.com/"
    response = self.client.post(reverse("home"),
                                {"url": url}, follow=True)
    self.assertEqual(response.status_code, 200)
    self.assertIn("link", response.context)
    l = response.context["link"]
    short_url = Link.shorten(l)
    self.assertEqual(url, l.url)
    self.assertIn(short_url, response.content)

(This test is designed to work with Django URLField’s default behaviour to add trailing slashes. This needs to be agreed with your client, of course. In my case, I simply had to ask the mirror.)

Now we need to think of what gets shown when the form is submitted. Obviously, there would be the short URL. Once again we can use a ready made class based view for this - the DetailView.

Add to shorturls/views.py:

from django.views.generic import DetailView
...

class LinkCreate(CreateView):
    model = Link
    fields = ["url"]

    def form_valid(self, form):
        # Check if the Link object already exists
        prev = Link.objects.filter(url=form.instance.url)
        if prev:
            return redirect("link_show", pk=prev[0].pk)
        return super(LinkCreate, self).form_valid(form)


class LinkShow(DetailView):
    model = Link

Change tiny/urls.py to:

from shorturls.views import LinkCreate
from shorturls.views import LinkShow

urlpatterns = patterns('',
    url(r'^$', LinkCreate.as_view(), name='home'),
    url(r'^link/(?P<pk>\d+)$', LinkShow.as_view(), name='link_show'),
)

Now add the template for the DetailView. Create shorturls/templates/shorturls/link_detail.html with the following contents:

<p> Short Link: /r/{{ object.id }}
<p> Original Link: {{ object.url }}

Note that we haven’t created the short link redirection code for /r/ yet.

Only hitch we have now is that Django doesn’t know where to go after the form is submitted. One way to solve that is by adding get_absolute_url() to the Link model.

Change shorturls/models.py to:

from django.db import models
from django.core.urlresolvers import reverse


class Link(models.Model):
    url = models.URLField()

    def get_absolute_url(self):
        return reverse("link_show", kwargs={"pk": self.pk})

Now if the form is submitted, it gets redirected to the new DetailView. The tests should pass.

Fifth Test: Short URL Must Redirect To The Long URL

Our next and final test actually tests if the short URLs work:

def test_redirect_to_long_link(self):
    """
    Tests that submitting the forms returns a Link object.
    """
    url = "http://example.com"
    l = Link.objects.create(url=url)        
    short_url = Link.shorten(l)
    response = self.client.get(
        reverse("redirect_short_url",
                kwargs={"short_url": short_url}))
    self.assertRedirects(response, url)

The final bit of the puzzle is the reverse lookup or redirected the user with the short URL to the original URL. As you might have guessed, time for yet another class based view - RedirectView.

Add to shorturls/views.py:

from django.views.generic.base import RedirectView
...


class RedirectToLongURL(RedirectView):

    permanent = False

    def get_redirect_url(self, *args, **kwargs):
        short_url = kwargs["short_url"]
        return Link.expand(short_url)

Add to tiny/urls.py:

from shorturls.views import RedirectToLongURL
...

    url(r'^r/(?P<short_url>\w+)$', RedirectToLongURL.as_view(),
              name='redirect_short_url'),

Now the tests should pass.

Refactoring For Shorter URLs

We will now see how tests prevent regression. The short URLs we generate are nothing but primary keys of Link instances in the database. This scheme works swimmingly well until a smart guy notices that we are wasting too many characters compared to other URL shorteners.

A shortener like bit.ly creates short urls which are a mix of alphabets (lower and upper case), numbers and symbols (like hyphens and underscores). However, we use only numbers. Mathematically speaking, we can use a base higher than 10 if we have more than ten symbols. This can lead to shorter numbers. For e.g. the decimal “255” can be represented as “FF” in hexadecimal, which saved 1 byte!

Notice that we have deliberately avoided testing the short URL format. This gives us flexibility to use any short URL representation we like and we are not limited to just decimals. So we create a module which can convert a decimal to a higher base with more symbols and back.

Create a new file shorturls/basechanger.py with the following contents:

CHARS = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGUIJKLMNOPQRSTUVWXYZ"
BASE = len(CHARS)


def decimal2base_n(n):
    if n >= BASE:
        return decimal2base_n(n // BASE) + CHARS[n % BASE]
    else:
        return CHARS[n]


def base_n2decimal(n):
    if len(n) > 1:
        return base_n2decimal(n[:-1]) * BASE + CHARS.index(n[-1])
    else:
        return CHARS.index(n[0])

If you suspect anything wrong, just play along ;) . Since all the URL shortening and expansion logic resides in the models (as it should), we need to change that as well.

Change shorturls/models.py to:

from django.db import models
from django.core.urlresolvers import reverse
from .basechanger import decimal2base_n, base_n2decimal


class Link(models.Model):
    url = models.URLField()

    def get_absolute_url(self):
        return reverse("link_show", kwargs={"pk": self.pk})

    @staticmethod
    def shorten(link):
        l, _ = Link.objects.get_or_create(url=link.url)
        return str(decimal2base_n(l.pk))

    @staticmethod
    def expand(slug):
        link_id = int(base_n2decimal(slug))
        l = Link.objects.get(pk=link_id)
        return l.url

Now running your tests should work as if nothing happened. Your smart guy will be pleased that the short URLs are now shorter. It is a win-win, folks!

Note that we have not removed the hard-coded short URL path in our DetailView template. Let’s do that as well.

First, add a method to shorturls/models.py:

def short_url(self):
    return reverse("redirect_short_url",
                   kwargs={"short_url": Link.shorten(self)})

Change shorturls/templates/shorturls/link_detail.html:

<p> Short Link: <a href="{{ object.short_url }}">{{ object.short_url }}</a>
<p> Original Link: {{ object.url }}

Subtle Bug: More Tests Needed

Everything goes great for a while until user testing, when you get a strange bug report:

Critical: Some short URLs get redirected to the wrong site.

By following TDD, the only way to fix your code would be to create a test to reproduce your bug. Since the problem only happens after a certain number of short URLs are created, you design a test to create a large number of short URLs and compare it with the original URL.

Add a new test case to shorturls/tests.py:

import random, string
...


def test_recover_link_n_times(self):
    """
    Tests multiple times that after shortening and expanding
    the original url is recovered.
    """
    TIMES = 100
    for i in xrange(TIMES):
        uri = "".join(random.sample(string.ascii_letters, 5))
        url = "https://example.com/{}/{}".format(i, uri)
        l = Link.objects.create(url=url)
        short_url = Link.shorten(l)
        long_url = Link.expand(short_url)
        self.assertEqual(url, long_url)

Running the test gives you a mysterious error.

AssertionError: 'https://example.com/55/KPAOz' != u'https://example.com/42/mDcHw'

The hint of what is wrong is in the URLs themselves. After running a debugger, you realise that the 55th character is same as the 42nd character - the symbol ‘H’. A simple typo that was actually overlooked when I was writing this article.

I learnt two lessons from this. First, to never underestimate testing. The more tests you can write, the better your code becomes. Second, to never attempt to list the alphabets by hand. We are no longer in kindergarten and we are certainly not expected to remember all of it. That’s why Python has string module with such silly strings kept ready for you.

So the fix is a simple change in the first line of shorturls/basechanger.py:

import string
CHARS = string.digits+string.ascii_lowercase+string.ascii_uppercase

So once again, we have a 100% success rate in our tests.

The completed source code can be found at Github

So What’s Wrong With TDD?

The biggest problem I faced while learning TDD was wearing two different hats alternatingly. The Tester Hat first makes you think the most specific way to break your code and the Code Hat wants you to write terse code that works in the most general way possible. Perhaps, after some practice, the cognitive load was greatly reduced. But, I would leave you with this pithy statement for some thought:

“Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.” — Brian Kernighan

Of course, TDD practitioners themselves ackowledge that TDD is not for exploratory programming. In other words, you cannot create a map if you don’t know where you are or where you want to go. It is also expected that you have a certain amount of expertise in the language (in this case, Python/Django), to clearly see how the test case should be written in advance. So I wouldn’t expect a beginner to learn TDD along with learning programming.

It, of course, goes without saying due to its counter intuitive nature, it takes some time to get comfortable with TDD.

Conclusion

TDD is a design technique that needs a little bit of extra time for planning ahead. Some studies put this between 15-35% increase in development time. So, I wouldn’t use it for one time scripts or a quick work. But I will strongly consider it whenever I need to build production quality sites.

I might also not consider it for version 1 of something that I am working on. Maybe version 2 onwards, when the development will get faster and I am more familiar with the domain.

It doesn’t necessarily improve the client’s experience. If they are more interested in development time rather than the quality of code, then they might not appreciate TDD designed code itself. This is when BDD is much more effective in engaging them and “showcasing” the effectiveness of the methodology.

Django tests are pretty fast. So I am not too slowed down by the runs. I work on the models first, then the views, then the system tests and finally the templates. I prefer smaller, faster, non-redundant, black-boxy and functionality-oriented tests. Perhaps you want to read that again. It takes a while to design a good test case.

However, Pytest is a better option without learning all the legacy JUnit functions. It is a lot better that the default Django testing tool and I would prefer to use Pytest as my default test runner.

TDD relies on the inherent human nature to fix things. Writing tests are important but not essential. We might forget them. TDD will actually show a ‘broken’ status if any test fails, this is a great incentive to fix tests.

I tend to make a lot of design decisions while writing test cases. This is a good side effect of TDD. Lot of corner case which are missed while coding get importance upfront and by the time you are coding you are aware of them, rather than other way around. This is why the design is better for TDD.

It takes a lot of time to be good at TDD.

Building a Hacker News clone in Django - Part 4 (AJAX and Mixin)

Mon, 19 Aug 2013 22:50:00 +0530

You are reading a post from a four-part tutorial series

Yes, all good things do come to an end. It gets even better when the ending is good. Steel Rumors was a project to help Django beginners progress to the next level from basic tutorials. It had elements which would be useful for most practical sites like user registrations and making CRUD views.

Honestly, I don’t like most video tutorials myself as they need a considerable amount of time to watch. However, if they come with a full transcript then I can skim through the text and decide if it contains tips which are worth watching on video. Sometimes, the additional commentary explaining the context is well worth the watch.

So, I set out to create Steel Rumors as something that everyone including me would enjoy watching. But it turns out creating the transcript is much harder than recording a quick video. In fact, it gets monotonous at times (I really sympathise those who work in Medical Transcription!).

The videos also get difficult to record in a longer, complex project such as this. I don’t edit out any mistakes or typos I make, since debugging those presents a valuable learning opportunity for beginners. But bringing all the details together while maintaining the continuity can get incredibly demanding.

Also most tutorials would try to show you how to use the most popular package or the easiest way to implement a feature. I deliberately avoided that, perhaps inspired by Learn Python the Hard Way. It might be okay to reinvent the wheel the first time because it will help you understand how wheels work for a lifetime. So, despite many comments telling me that it is easier to use X than Y, I stuck to the alternative which helps you learn the most.

In this tutorial, we would cover some interesting areas like how you can make Django forms work with AJAX and how a simple ranking algorithm works. As always you can choose to watch the video or read the step by step description below or follow both.

I would recommend watching all the previous parts before watching this video.

Did you learn quite a bit from this video series? Then you should sign up for my upcoming book “Building a Social News Site in Django”. It explains in a learn-from-a-friend style how websites are built and gradually tackles advanced topics like testing, security, database migrations and debugging.

Step-by-step Instructions

This is the transcript of the video. In part 3, we created a social news site where users can post and comment about rumours of “Man of Steel” but cannot vote.

The outline of Part 4 of the screencast is:

Voting with FormView
Voting over AJAX
Mixins
Display Voted Status
Ranking algorithm
Background tasks

Voting with FormView

We will add an upvote button (with a plus sign) to each headline. Clicking on this will toggle the user’s “voted” status for a link i.e. voted or did not vote. The safest way to implement it is using a ModelForm for our Vote model.

Add a new form to links/forms.py:
```
    from .models import Vote
    ...

    class VoteForm(forms.ModelForm):
        class Meta:
            model = Vote
```

We will use another generic view called FormView to handle the view part of this form. Add these lines to links/views.py

    from django.shortcuts import redirect
    from django.shortcuts import get_object_or_404
    from django.views.generic.edit import FormView
    from .forms import VoteForm
    from .models import Vote
    ...

    class VoteFormView(FormView):
        form_class = VoteForm

        def form_valid(self, form):
            link = get_object_or_404(Link, pk=form.data["link"])
            user = self.request.user
            prev_votes = Vote.objects.filter(voter=user, link=link)
            has_voted = (prev_votes.count() > 0)

            if not has_voted:
                # add vote
                Vote.objects.create(voter=user, link=link)
                print("voted")
            else:
                # delete vote
                prev_votes[0].delete()
                print("unvoted")

            return redirect("home")

        def form_invalid(self, form):
            print("invalid")
            return redirect("home")

Those print statements will be removed soon and they are definitely not recommended for a production site.

Edit the home page template to add a voting form per headline. Add lines with ‘+’ sign (removing the ‘+’ sign) to steelrumors/templates/links/link_list.html:

    {% for link in object_list %}
    + <form method="post" action="{% url 'vote' %}" class="vote_form">
        <li> [{{ link.votes }}]
      +  {% csrf_token %}
      + <input type="hidden" id="id_link" name="link" class="hidden_id" value="{{ link.pk }}" />
      + <input type="hidden" id="id_voter" name="voter" class="hidden_id" value="{{ user.pk }}" />
      + <button>+</button>
        <a href="{% url 'link_detail' pk=link.pk %}">
          <b>{{ link.title }}</b>
        </a>
        </li>
    + </form>

Add this view in steelrumours/urls.py:
```
    from links.views import VoteFormView

    url(r'^vote/$', auth(VoteFormView.as_view()), name="vote"),  
```
Refresh the browser to see the ‘+’ buttons on every headline. You can vote them as well. But you can read the voting status only from the console.

Voting with AJAX

You have already copied the static folder of the goodies pack in the previous part. But in case you haven’t, then follow this step.

Create a folder named ‘js’ under steelrumors/static our javascript files. Copy jquery and vote.js from the goodies pack into this folder.
```
    mkdir steelrumors/static/js
    cp /tmp/sr-goodies-master/static/js/* ~/proj/steelrumors/steelrumors/static/js/
```

Add these lines to steelrumors/templates/base.html within <head> block:

      <title>Steel Rumors</title>
      <link rel="stylesheet" type="text/css" href="{{ STATIC_URL }}css/main.css" />
    +  <script src="{{ STATIC_URL }}js/jquery.min.js"></script>
    +  <script src="{{ STATIC_URL }}js/vote.js"></script>
    </head>
    <body>

In views.py delete the entire class VoteFormView and replace with these three classes. We are using a mixin to implement a JSON response for our AJAX requests:

    import json
    from django.http import HttpResponse
    ...

    class JSONFormMixin(object):
        def create_response(self, vdict=dict(), valid_form=True):
            response = HttpResponse(json.dumps(vdict), content_type='application/json')
            response.status = 200 if valid_form else 500
            return response

    class VoteFormBaseView(FormView):
        form_class = VoteForm

        def create_response(self, vdict=dict(), valid_form=True):
            response = HttpResponse(json.dumps(vdict))
            response.status = 200 if valid_form else 500
            return response

        def form_valid(self, form):
            link = get_object_or_404(Link, pk=form.data["link"])
            user = self.request.user
            prev_votes = Vote.objects.filter(voter=user, link=link)
            has_voted = (len(prev_votes) > 0)

            ret = {"success": 1}
            if not has_voted:
                # add vote
                v = Vote.objects.create(voter=user, link=link)
                ret["voteobj"] = v.id
            else:
                # delete vote
                prev_votes[0].delete()
                ret["unvoted"] = 1
            return self.create_response(ret, True)

        def form_invalid(self, form):
            ret = {"success": 0, "form_errors": form.errors }
            return self.create_response(ret, False)

    class VoteFormView(JSONFormMixin, VoteFormBaseView):
        pass

Showing the Voted state

We need some indication to know if the headline was voted or not. To achieve this, we can pass ids of all the links that have been voted by the logged in user. This can be passed as a context variable i.e. voted.

Add this to LinkListView class in links/views.py:

    class LinkListView(ListView):
    ...

        def get_context_data(self, **kwargs):
            context = super(LinkListView, self).get_context_data(**kwargs)
            if self.request.user.is_authenticated():
                voted = Vote.objects.filter(voter=self.request.user)
                links_in_page = [link.id for link in context["object_list"]]
                voted = voted.filter(link_id__in=links_in_page)
                voted = voted.values_list('link_id', flat=True)
                context["voted"] = voted
            return context

Change the home page template again. Add lines with ‘+’ sign (removing the ‘+’ sign) to steelrumors/templates/links/link_list.html:

        <input type="hidden" id="id_voter" name="voter" class="hidden_id" value="{{ user.pk }}" />
      + {% if not user.is_authenticated %}
      + <button disabled title="Please login to vote">+</button>
      + {% elif link.pk not in voted %}
        <button>+</button>
      + {% else %}
      + <button>-</button>
      + {% endif %}
        <a href="{% url 'link_detail' pk=link.pk %}">

Now, the button changes based on the voted state of a headline. Try it on your browser with different user logins.

Calculating Rank Score

We are going to change the sorting order of links from highest voted to highest score. Add a new function to models.py to calculate the rank score:

    from django.utils.timezone import now
    ...

    class Link(models.Model):
    ...

        def set_rank(self):
            # Based on HN ranking algo at http://amix.dk/blog/post/19574
            SECS_IN_HOUR = float(60*60)
            GRAVITY = 1.2

            delta = now() - self.submitted_on
            item_hour_age = delta.total_seconds() // SECS_IN_HOUR
            votes = self.votes - 1
            self.rank_score = votes / pow((item_hour_age+2), GRAVITY)
            self.save()

In the same file, change the sort criteria in the LinkVoteCountManager class. The changed line has been marked with a ‘+’ sign.

    class LinkVoteCountManager(models.Manager):
        def get_query_set(self):
            return super(LinkVoteCountManager, self).get_query_set().annotate(
    +             votes=Count('vote')).order_by('-rank_score', '-votes')

Ranking Job

Calculating the score for all links is generally a periodic task which should happen in the background. Create a file called rerank.py in the project root with the following content:

#!/usr/bin/env python
import os

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "steelrumors.settings")
from links.models import Link

def rank_all():
    for link in Link.with_votes.all():
        link.set_rank()

import time

def show_all():
    print "\n".join("%10s %0.2f" % (l.title, l.rank_score,
                         ) for l in Link.with_votes.all())
    print "----\n\n\n"

if __name__=="__main__":
    while 1:
        print "---"
        rank_all()
        show_all()
        time.sleep(5)

This runs every 5 secs in the foreground.

Turn it into a background job
```
(nohup python -u rerank.py&)
tail -f nohup.out
```
Note that this is a very simplistic implementation of a background job. For a more robust solution, check out Celery.

Watching the News Dive

It is fun to watch the rank scores rise and fall for links. It is almost as fun as watching an aquarium except with numbers. But the ranking function set_rank in models.py has the resolution of an hour. This makes it quite boring.

To see a more dramatic change in rank scores change the SECS_IN_HOUR constant to small value like 5.0. Now submit a new link and watch the scores drop like a stone!

Final Comments

Steel Rumors is far from being a complete Hacker News clone. But it supports voting, submission of links and user registrations. In fact, it is quite useable at this point.

Check out a demo of Steel Rumors yourself.

Hope you enjoyed this tutorial series as much as I did while making them. If you get stuck anywhere make sure you check the github source first for reference. Keep your comments flowing!

Resources

Full Source on Github

Building a Hacker News clone in Django - Part 3 (Comments and CRUD)

Tue, 09 Jul 2013 09:44:00 +0530

You are reading a post from a four-part tutorial series

It feels like ages since part 2 was out. Meanwhile, the movie Man of Steel has actually been released. So there is really no need to check for rumors about what that movie is all about. Probably the industry is now abuzz with rumours about its sequel instead.

In this tutorial, I would be showing how to use features like comments and CRUD views which are integral to a social site. You can choose to watch the video or read the step by step description below or follow both. The goodies pack which was introduced in the last part has been updated and would be used again to save time to create templates.

This video would be a continuation of the previous video and I recommend watching it. Click on the image below to watch the screencast or scroll down to read the steps.

Enjoyed this tutorial? Then you should sign up for my upcoming book “Building a Social News Site in Django”. It tries to explain in a learn-from-a-friend style how websites are built and gradually tackles advanced topics like testing, security, database migrations and debugging.

Step-by-step Instructions

Here is the text version of the video for people who prefer to read. In part 2, we showed you how to create a beta-like site to publish rumors about “Man of Steel” where users can sign-up and create their own profiles.

The outline of Part 3 of the screencast is:

Comments framework
Create/Read/Update/Delete of a Link
Pagination

Get the goodies pack again

The goodies pack has changed since the last tutorial, so I would recommend downloading it again.

Download sr-goodies-master.zip to any convenient location. On Linux, you can use the following commands to extract it to the /tmp directory.
```
    cd /tmp
    wget https://github.com/arocks/sr-goodies/archive/master.zip
    unzip master.zip
```
Explore the extracted files in /tmp/sr-goodies-master

Pagination

So far we have been seeing just the first page of the list of links. But we are using Django’s ListView which provides pagination. Let’s implement a simple ‘Next’ link to visit the next page.

Add the following snippet to steelrumors/templates/links/link_list.html just before {% endblock %}:

    {% if is_paginated %}
    <div class="pagination">
        {% if page_obj.has_next %}
        <a href="?page={{ page_obj.next_page_number }}">More &raquo;</a>
        {% endif %}
    </div>
    {% endif %}

In the same template file we’ll need to change the first <ol> tag to ensure that the line numbers in page 2 and later appear correctly. Replace that line with <ol> with these lines:
```
    {% if is_paginated %}
    <ol start="{{ page_obj.start_index }}">
    {% else %}
    <ol>
    {% endif %}
```
Now you can visit every page and read every submitted link!

CRUD - Create and Read Links

We have been using the admin interface to create/update/delete links. But this is only accessible to staff members. To allow users to submit links, we will need a new form, a new view class (generic CBV) and a new template with a form.

Add this new ModelForm to `links/forms.py`:

    from .models import Link
    ...

    class LinkForm(forms.ModelForm):
        class Meta:
            model = Link
            exclude = ("submitter", "rank_score")

Time to implement the “C” of CRUD by importing CreateView and the form you just created:

Add the `LinkCreateView` class to `links/views.py`:

    from django.views.generic.edit import CreateView
    from .forms import LinkForm
    ...

    class LinkCreateView(CreateView):
        model = Link
        form_class = LinkForm

        def form_valid(self, form):
            f = form.save(commit=False)
            f.rank_score = 0.0
            f.submitter = self.request.user
            f.save()

            return super(CreateView, self).form_valid(form)

Copy link_form.html from goodies to steelrumors/templates/links/link_form.html:

    cp /tmp/sr-goodies-master/templates/links/link_form.html \
       ~/proj/steelrumors/steelrumors/templates/links/

Add this view in steelrumours/urls.py:

    from links.views import LinkCreateView        

    url(r'^link/create/$', auth(LinkCreateView.as_view()),
        name='link_create'),

Visit http://127.0.0.1:8000/link/create/ and submit a new link. Remember, you’ll need to be logged in to submit links.

If you try to submit a link, you will see an error message asking you to “Either provide a url or define a get_absolute_url method on the Model.” So let’s define the get_absolute_url method.

Add the following method in the Link class:
```
    from django.core.urlresolvers import reverse
    ...

    def get_absolute_url(self):
        return reverse("link_detail", kwargs={"pk": str(self.id)})
```

Create a DetailView in links/views.py. This is the “R” of CRUD.

    from django.views.generic import DetailView
    ...

    class LinkDetailView(DetailView):
        model = Link

Copy link_detail.html from goodies to steelrumors/templates/links/link_detail.html:

    cp /tmp/sr-goodies-master/templates/links/link_detail.html \
       ~/proj/steelrumors/templates/links/

Add this detail view in steelrumours/urls.py:

    from links.views import LinkDetailView        

    url(r'^link/(?P<pk>\d+)/$', LinkDetailView.as_view(),
        name='link_detail'),

Try submitting the add link form again and it should take you to the detail page, without error.

For convenience, let’s add the urls to these new views in our template so that users can easily find them. Add only the line with a + sign to base.html (remove the + sign):

    {% if user.is_authenticated %}
    +   <a href="{% url 'link_create' %}">Submit Link</a> | 
        <a href="{% url 'logout' %}">Logout</a> |

Make the following change (changed line starts with a + sign) to steelrumors/templates/links/link_list.html:

    {% for link in object_list %}
        <li> [{{ link.votes }}]
    +   <a href="{% url 'link_detail' pk=link.pk %}">
          <b>{{ link.title }}</b>

Refresh the steelrumours site on your browser and check if all the links work correctly.

CRUD - Update and Delete

The remaining two views for Update and Delete are straight forward and we can add them together. We are going to reuse the LinkForm, so let’s start with views.

Add these view classes to links/views.py:

    from django.core.urlresolvers import reverse_lazy
    from django.views.generic.edit import UpdateView
    from django.views.generic.edit import DeleteView
    ...

    class LinkUpdateView(UpdateView):
        model = Link
        form_class = LinkForm

    class LinkDeleteView(DeleteView):
        model = Link
        success_url = reverse_lazy("home")

Copy link_confirm_delete.html from goodies to steelrumors/templates/links/link_confirm_delete.html:

    cp /tmp/sr-goodies-master/templates/links/link_confirm_delete.html \
       ~/proj/steelrumors/templates/links/

Add these views in steelrumours/urls.py:

    from links.views import LinkUpdateView
    from links.views import LinkDeleteView

        url(r'^link/update/(?P<pk>\d+)/$', auth(LinkUpdateView.as_view()),
            name='link_update'),
        url(r'^link/delete/(?P<pk>\d+)/$', auth(LinkDeleteView.as_view()),
            name='link_delete'),

Finally, for convenience, add these lines (with + sign) to steelrumors/templates/links/link_detail.html:

    <h2><a href="{{ object.link }}">{{ object.title }}</a></h2>
    + {% if object.submitter == user %}
    +  <a href="{% url 'link_update' pk=object.pk %}">Edit</a> | 
    +  <a href="{% url 'link_delete' pk=object.pk %}">Delete</a>
    + {% endif %}

Now, you can create, read, update and delete Link objects. Try it!

Enabling Comments

We are going to add comments to the link detail pages using the built-in Django comments framework. First add this applications in steelrumors/settings.py:
```
    INSTALLED_APPS = (

        'django.contrib.admin',
    +    'django.contrib.comments',
```
Run syndb to create the tables required by the comments app:
```
    ./manage.py syncdb
```

Copy the new link_detail.html page from the goodies pack:

    cp /tmp/sr-goodies-master/templates/links/link_detail2.html \
       ~/proj/steelrumors/templates/links/link_detail.html

We need to show comment counts in the front page itself. So add the following lines to steelrumors/templates/links/link_list.html at the beginning and middle of the template:

    {% extends "base.html" %}
    + {% load comments %}
    ...

    <a href="{% url 'link_detail' pk=link.pk %}">
    <b>{{ link.title }}</b>
    +  {% get_comment_count for link as comment_count %}
    +  {{ comment_count }} comment{{ comment_count|pluralize }}
    </a>

Add this line to steelrumours/urls.py for wiring up the comments app:
```
        url(r'^comments/', include('django.contrib.comments.urls')),
```
Now, open any link detail page and have fun writing comments!

Fun with Random Gossip

Add a mixin class called RandomGossipMixin in links/views.py before LinkListView class:

from django.contrib.comments.models import Comment
...

class RandomGossipMixin(object):
    def get_context_data(self, **kwargs):
        context = super(RandomGossipMixin, self).get_context_data(**kwargs)
        context[u"randomquip"] = Comment.objects.order_by('?')[0]
        return context

Change the class declaration of LinkListView to include this mixin as a base class:

class LinkListView(RandomGossipMixin, ListView):

Add the following lines to steelrumors/templates/links/link_list.html before the endblock line:

<blockquote style="background-color: #ddd; padding: 4px; border-radius: 10px; margin: 10px 0; color: #666; font-size: smaller; text-shadow: rgba(255,255,255,0.8) 1px 1px 0;">
{{ randomquip.comment|truncatechars:140 }}
</blockquote>

Now refresh the home page and enjoy a random comment appear at the bottom of the page.

Final Comments

We have a lot more feature-complete social news site at this point. Users can actually submit links and comment about them. In the next and concluding part, we will cover writing mixins and ranking algorithms in Django. With this users will be able to vote and influence the ranking of links.

That concludes Part 3. Follow me on Twitter at @arocks to get updates about upcoming parts.

EDIT: Check out Part 4!

Resources

Full Source on Github
Goodies pack on Github

Building a Hacker News clone in Django - Part 2 (User Profiles and Registrations)

Thu, 06 Jun 2013 00:17:00 +0530

You are reading a post from a four-part tutorial series

It has been more than a week since the first part of this tutorial series was posted and I’ve been getting positive feedback from several channels. Ironically, it was extremely popular on Hacker News. It was even translated into Chinese.

To be sure, the objective was not to create a full featured clone of any website. The real objective is to learn Django using a medium-sized project that utilises it to the fullest. Many tutorials fall short of bringing together various parts of Django. Compared to a microframework like Flask (which is also great, btw), Django comes with a lot of batteries included. If you are short on time, this makes it ideal for completing an ambitious project like this.

This tutorial would show you how to implement social features like supporting user registration and profile pages. We will leverage Django’s class based views for building CRUD functionality. There is a lot of ground to be covered this time.

As before, there is a text description of the steps if you do not prefer to watch the entire video. There is also a goodies pack with templates and other assets included, which would be required if you are following this tutorial.

This video would be a continuation of the previous video and I recommend watching it. Click on the image below to watch the screencast or scroll down to read the steps.

Step-by-step Instructions

Here is the text version of the video for people who prefer to read. In part 1, we showed you how to create a private beta-like site to publish rumors about “Man of Steel”.

The outline of Part 2 of the screencast is:

Better branding and templates
Custom login/logout
Sociopath to actually social - django-registrations
Simple registration
User Profiles

Open the goodies pack

So far, the appearance of the website looks a bit bland. Let’s use some assets which are pre-designed for the tutorial.

Download sr-goodies-master.zip to any convenient location. On Linux, you can use the following commands to extract it to the /tmp directory.
```
    cd /tmp
    wget https://github.com/arocks/sr-goodies/archive/master.zip
    unzip master.zip
```
Explore the extracted files in /tmp/sr-goodies-master
Copy the entire static directory from the extracted files to steelrumors/steelrumors. Also, overwrite the extracted sr-goodies-master/templates/base.html template into steelrumors/steelrumors/templates/
```
    cp -R /tmp/sr-goodies-master/static ~/proj/steelrumors/steelrumors/
    cp /tmp/sr-goodies-master/templates/base.html ~/proj/steelrumors/steelrumors/templates/
```

Add to steelrumors/urls.py:

    url(r'^login/$', 'django.contrib.auth.views.login', {
        'template_name': 'login.html'}, name="login"),
    url(r'^logout/$', 'django.contrib.auth.views.logout_then_login',
        name="logout"),

Add the login/logout URL locations to steelrumors/settings.py:

    from django.core.urlresolvers import reverse_lazy

    LOGIN_URL=reverse_lazy('login')
    LOGIN_REDIRECT_URL = reverse_lazy('home')
    LOGOUT_URL=reverse_lazy('logout')

Now copy login.html from the goodies pack to the templates directory:
```
    cp /tmp/sr-goodies-master/templates/login.html ~/proj/steelrumors/steelrumors/templates/
```
Refresh your browser to view the newly styled pages.

Using django-registrations

We will be using the “simple” backend of django-registrations since it is easy to use and understand.

Currently, the version of django registration on the the Python Package Index doesn’t work well with Django 1.5. So we will use my forked version using pip:
```
    pip install git+git://github.com/arocks/django-registration-1.5.git
```
Or if you don’t have git installed, use:
```
    pip install https://github.com/arocks/django-registration-1.5/tarball/master
```
Edit settings.py to add registration app to the end of INSTALLED_APPS
```
    'registration',
    )
```
Run syncdb to create the registration models:
```
    ./manage.py syncdb
```

We need to use the registration form template from goodies pack. Since, this is for the registration app we need to create a registration directory under templates:

    mkdir ~/proj/steelrumors/steelrumors/templates/registration/
    cp /tmp/sr-goodies-master/templates/registration/registration_form.html
       ~/proj/steelrumors/steelrumors/templates/registration/

Add to urls.py:
```
    url(r'^accounts/', include('registration.backends.simple.urls')),
```
Visit http://127.0.0.1:8000/accounts/register/ and create a new user. This will throw a “Page not found” error after a user is created.

Create a user’s profile page

Add UserProfile class and its signals to models.py:

    class UserProfile(models.Model):
        user = models.OneToOneField(User, unique=True)
        bio = models.TextField(null=True)

        def __unicode__(self):
            return "%s's profile" % self.user

    def create_profile(sender, instance, created, **kwargs):
        if created:
            profile, created = UserProfile.objects.get_or_create(user=instance)

    # Signal while saving user
    from django.db.models.signals import post_save
    post_save.connect(create_profile, sender=User)

Run syncdb again to create the user profile model:
```
    ./manage.py syncdb
```

Add these to admin.py of links app to replace/extend the default admin for User:


    from django.contrib.auth.admin import UserAdmin
    from django.contrib.auth import get_user_model
    ...
    class UserProfileInline(admin.StackedInline):
        model = UserProfile
        can_delete = False

    class UserProfileAdmin(UserAdmin):
        inlines=(UserProfileInline, )

    admin.site.unregister(get_user_model())
    admin.site.register(get_user_model(), UserProfileAdmin)

Visit http://127.0.0.1:8000/admin/ and open any user’s details. The bio field should appear in the bottom.

Add to views.py of links apps:

    from django.views.generic import ListView, DetailView
    from django.contrib.auth import get_user_model
    from .models import UserProfile
    ....
    class UserProfileDetailView(DetailView):
        model = get_user_model()
        slug_field = "username"
        template_name = "user_detail.html"

        def get_object(self, queryset=None):
            user = super(UserProfileDetailView, self).get_object(queryset)
            UserProfile.objects.get_or_create(user=user)
            return user

Copy user_detail.html from goodies to steelrumors/templates/:

    cp /tmp/sr-goodies-master/templates/user_detail.html \
       ~/proj/steelrumors/steelrumors/templates/

Let’s add the urls which failed last time when we tried to create a user. Add to urls.py:
```
    from links.views import UserProfileDetailView
    ...
    url(r'^users/(?P<slug>\w+)/$', UserProfileDetailView.as_view(), name="profile"),
```
Now try to create a user and it should work. You should also see the profile page for the newly created user, as well as other users.

You probably don’t want to enter these links by hand each time. So let’s edit base.html by adding the lines with a plus sign ‘+’ (omitting the plus sign) below:

      {% if user.is_authenticated %}
        <a href="{% url 'logout' %}">Logout</a> |
      +  <a href="{% url 'profile' slug=user.username %}"><b>{{ user.username }}</b></a> 
      {% else %}
      +  <a href="{% url 'registration_register' %}">Register</a> |

Refresh the browser to see the changes.

Edit your profile details

Add UserProfileEditView class to views.py in links app:

    from django.views.generic.edit import UpdateView
    from .models import UserProfile
    from .forms import UserProfileForm
    from django.core.urlresolvers import reverse

    class UserProfileEditView(UpdateView):
        model = UserProfile
        form_class = UserProfileForm
        template_name = "edit_profile.html"

        def get_object(self, queryset=None):
            return UserProfile.objects.get_or_create(user=self.request.user)[0]

        def get_success_url(self):
            return reverse("profile", kwargs={'slug': self.request.user})

Create links/forms.py:

    from django import forms
    from .models import UserProfile

    class UserProfileForm(forms.ModelForm):
        class Meta:
            model = UserProfile
            exclude = ("user")

Add the profile edit view to urls.py. Protect it with an auth decorator to prevent unlogged users from seeing this view.

    from django.contrib.auth.decorators import login_required as auth
    from links.views import UserProfileEditView
    ...
    url(r'^edit_profile/$', auth(UserProfileEditView.as_view()), name="edit_profile"),

Copy edit_profile.html from goodies to steelrumors/templates/:

    cp /tmp/sr-goodies-master/templates/edit_profile.html \
       ~/proj/steelrumors/steelrumors

Add the following lines to templates/user_detail before the final endblock:

    {% if object.username == user.username %}
    <p><a href='{% url "edit_profile" %}'>Edit my profile</a></p>
    {% endif %}

Now, visit your profile page and try to edit it.

Easter Egg Fun

Add the following lines to user_detail.html before the endblock line:

{% if "zodcat" in object.userprofile.bio %}
<style>
html {
  background: #EEE url("/static/img/zod.jpg");
}
</style>
{% endif %}

Now mention “zodcat” to your bio. You have your easter egg!

Final Comments

We have a much better looking site at the end of this tutorial. While anyone could register to the site, they will not be part of the staff. Hence, you cannot submit links through the admin interface. This will be addressed in the next part.

That concludes Part 2. Follow me on Twitter at @arocks to get updates about upcoming parts.

EDIT: Check out Part 3!

Resources

Full Source on Github
Goodies pack on Github

Building a Hacker News clone in Django - Part 1

Wed, 29 May 2013 02:27:00 +0530

You are reading a post from a four-part tutorial series

There is no better way to learn something than to watch someone else do it¹. So, if you have been waiting to go beyond the basics in Django, you have come to the right place.

In this video tutorial series, I would take you through building a social news site called “Steel Rumors” from scratch in Django 1.5. In case you don’t like videos and prefer to read the steps, you can find them here too.

Even though we will start from the basics, if you are an absolute beginner to Django, I would suggest reading the tutorial or my previous screencast on building a blog in 30 mins

The completed site would support user signups, link submissions, comments, voting and a cool ranking algorithm. My approach would be to use as many builtin Django features as possible and use external apps only when absolutely necessary.

Check out a demo of Steel Rumors yourself.

Click on the image below to watch the screencast or scroll down to read the steps.

If you liked this tutorial, then you should sign up for my upcoming book “Building a Social News Site in Django”. It tries to explain in a learn-from-a-friend style how websites are built and gradually tackles advanced topics like database migrations and debugging.

Step-by-step Instructions

Here is the text version of the video for people who prefer to read. We are going to create a social news site similar to Hacker News or Reddit. It will be called “Steel Rumors” and would be a place to share and vote some interesting rumors about “Man of Steel”.

The outline of Part 1 of the screencast is:

Objective
VirtualEnv - Start from Scratch!
Model Managers - Dream Job #78
Basic Template
Generic Views - ListView and DetailView
Pagination - For free!

Setup Virtual Environment

We will create a virtual development environment using virtualenv and virtualenvwrapper. Make sure you have installed them first:
```
    mkvirtualenv djangorocks
```
I use an Ubuntu variant called Xubuntu in my screencast. But you should be able to replicate these steps in other OSes with minimal changes.
Install Django (make sure you already have pip installed)
```
    pip install Django==1.5
```
You can also use Django 1.5.1. The latest Django version may or may not work with our code, hence it is better to specify a version to follow this tutorial.

Create Project and Apps

Create a project called steelrumors:

    cd ~/proj
    django-admin.py startproject steelrumors
    cd steelrumors
    chmod +x manage.py

Open steelrumors/settings.py in your favourite editor. Locate and change the following lines (changes in bold):

1. 'ENGINE': 'django.db.backends.__sqlite3__'
2. 'NAME': '__database.db__',
3. At the end of INSTALLED_APPS = (     __'django.contrib.admin',__

Next, change steelrumors/urls.py by uncommenting the following lines:

    from django.conf.urls import patterns, include, url
    from django.contrib import admin
    admin.autodiscover()

    urlpatterns = patterns('',
        url(r'^admin/', include(admin.site.urls)),
    )

Sync to create admin objects and enter the admin details:
```
    ./manage.py syncdb
```
Open a new tab or a new terminal and keep a server instance running (don’t forget to issue workon djangorocks in this terminal):
```
    ./manage.py runserver
```
Visit the admin page (typically at http://127.0.0.1:8000/admin/) and login.
Create links app:
```
    ./manage.py startapp links
```

Enter the following two model classes into links/models.py:

    from django.db import models
    from django.contrib.auth.models import User

    class Link(models.Model):
        title = models.CharField("Headline", max_length=100)
        submitter = models.ForeignKey(User)
        submitted_on = models.DateTimeField(auto_now_add=True)
        rank_score = models.FloatField(default=0.0)
        url = models.URLField("URL", max_length=250, blank=True)
        description = models.TextField(blank=True)

        def __unicode__(self):
            return self.title

    class Vote(models.Model):
        voter = models.ForeignKey(User)
        link = models.ForeignKey(Link)

        def __unicode__(self):
            return "%s upvoted %s" % (self.voter.username, self.link.title)

Create the corresponding admin classes. Enter the following into links/admin.py:

    from django.contrib import admin
    from .models import Link, Vote

    class LinkAdmin(admin.ModelAdmin): pass
    admin.site.register(Link, LinkAdmin)

    class VoteAdmin(admin.ModelAdmin): pass
    admin.site.register(Vote, VoteAdmin)

Enter the following into links/views.py:

    from django.views.generic import ListView
    from .models import Link, Vote

    class LinkListView(ListView):
        model = Link

Insert following lines into steelrumor/urls.py:

    from links.views import LinkListView
    ...
    urlpatterns = patterns('',
        url(r'^$', LinkListView.as_view(), name='home'),

Create a new templates directory and enter the following at steelrumors/templates/links/link_list.html:

    <ol>
    {% for link in object_list %}
        <li>
        <a href="{{ link.url }}">
          <b>{{ link.title }}</b>
        </a>
        </li>
    {% endfor %}
    </ol>

Edit settings.py to add our two apps to the end of INSTALLED_APPS = (
```
    'links',
    'steelrumors',
    )
```

Sync to create link objects:

    ./manage.py syncdb

Visit http://127.0.0.1:8000/admin/ and add a couple of Link objects. Now if you open http://127.0.0.1:8000/ you should see the added Links

Add Branding

Create a common base template at steelrumors/templates/base.html:

    <html>
    <body>
    <h1>Steel Rumors</h1>

    {% block content %}
    {% endblock %}

    </body>
    </html>

Modify steelrumors/templates/links/link_list.html and surround previous code with this:

    {% extends "base.html" %}

    {% block content %}
    ...
    {% endblock %}

VoteCount Model Manager

We need a count of votes within our generic ListView. Add these to links/models.py:

    from django.db.models import Count

    class LinkVoteCountManager(models.Manager):
        def get_query_set(self):
            return super(LinkVoteCountManager, self).get_query_set().annotate(
                votes=Count('vote')).order_by('-votes')

Insert these two lines into the Link class in links/models.py:

    class Link(models.Model):
    ...

        with_votes = LinkVoteCountManager()
        objects = models.Manager() #default manager

Edit links/views.py and insert these two lines into the LinkListView class:

    class LinkListView(ListView):
    ...

        queryset = Link.with_votes.all()
        paginate_by = 3

Crazy Fun

You can add 100 votes to random headlines using the following lines in the django shell:

$ ./manage.py shell
>>> from links.models import Link, Vote
>>> from django.contrib.auth.models import User
>>> a = User.objects.all()[0]
>>> for i in xrange(100): Vote(link=Link.objects.order_by('?')[0],voter=a).save()

Now visit http://127.0.0.1:8000/admin/ to find lots of Votes objects.

Final Comments

In case, you are wondering if this version of the site would be useful, I would say that it works well for a private beta version. Any new user would have to be added by the admin interface manually. They must be of the staff kind if they have to login via the admin interface. Staff can vote by manually creating Vote objects.

The public facing part of the site can still show the top rumors based the votes received by the staff. Based on how well designed the templates are, this version could be also used to get feedback about the site’s design and branding.

That concludes Part 1. Follow me on Twitter at @arocks to get updates about the following parts.

EDIT: Check out Part 2!

Resources

Full Source on Github (repository has changed!)

This became very controversial. Most people learn best by doing it themselves. But they need to read/hear/see it first from someone. I am actually a self-taught programmer. I learnt most of programming from books and trying it myself. Learning by doing is certainly the best way to learn. But among source of learning, watching an expert is probably the best. ↩︎

Bonus Screencast - HTML5 makeover for the Django blog

Wed, 07 Mar 2012 02:33:51 +0530

Thank you for the great feedback on my first screencast about building a blog in Django 1.3. A lot of people told me that they found the video helpful in understanding what Django is capable of.

As promised earlier, I have recorded a short video (7 mins) that explains how you can use an HTML5 template to improve the appearance of your blog. You can use any free template or create your own.

Django’s (version 1.3) static files app is being used here. Unlike prior versions, there are no changes to urls.py required here. Linking to the admin site is also demonstrated.

It is best to watch it directly at Youtube in HD.

UPDATE: The blog with the updated template has been uploaded to github

Building a blog in 30 mins with Django (Screencast)

Mon, 05 Mar 2012 22:40:05 +0530

Just uploaded a quick screencast showing how to build a blog in Django in just 30 minutes (plus a couple of seconds :) ). It shows off a lot of new features that we have in Django since 1.2.

It basically covers:

Using django.contrib.admin to create fully-featured admin pages
Adding tags to posts easily using taggit app
Class-based generic views for rapidly building pages (New in Django 1.3)
Using template inheritance and filters
Leveraging django.contrib.syndication for a simple feed

It is best to watch it directly at Youtube in HD. This was my first screencast so I apologize if something is not properly done.

I plan to add a short bonus video soon demonstrating how this bare-bones blog can be restyled into a modern-looking HTML5 site (UPDATE: Bonus video is up now).

UPDATE: The source code for the created project has been uploaded to github

Screencast on Arunrocks

Understanding Test Driven Development with Django

What is TDD?

How to do TDD?

Why do TDD?

How is TDD done?

User Stories

Create the Project

Writing the First Test

Sidenote: Choosing the right assertion

Second Test: Recovering the url

Third Test: Home Page With a Form

Fourth Test: Form Returns a Short URL

Fifth Test: Short URL Must Redirect To The Long URL

Refactoring For Shorter URLs

Subtle Bug: More Tests Needed

So What’s Wrong With TDD?

Conclusion

Building a Hacker News clone in Django - Part 4 (AJAX and Mixin)

Step-by-step Instructions

Voting with FormView

Voting with AJAX

Showing the Voted state

Calculating Rank Score

Ranking Job

Watching the News Dive

Final Comments

Resources

Building a Hacker News clone in Django - Part 3 (Comments and CRUD)

Step-by-step Instructions

Get the goodies pack again

Pagination

CRUD - Create and Read Links

CRUD - Update and Delete

Enabling Comments

Fun with Random Gossip

Final Comments

Resources

Building a Hacker News clone in Django - Part 2 (User Profiles and Registrations)

Step-by-step Instructions

Open the goodies pack

Custom Login page

Using django-registrations

Create a user’s profile page

Edit your profile details

Easter Egg Fun

Final Comments

Resources

Building a Hacker News clone in Django - Part 1

Step-by-step Instructions

Setup Virtual Environment

Create Project and Apps

Add Branding

VoteCount Model Manager

Crazy Fun

Final Comments

Resources

Bonus Screencast - HTML5 makeover for the Django blog

Building a blog in 30 mins with Django (Screencast)