python3 on Arunrocks

Django Release Schedule and Python 3

Tue, 30 Jan 2018 06:59:49 +0530

Do long term releases confuse you? For the longest time I was not sure which version of Ubuntu to download - the latest release or the LTS? I see a number of Django developers confused about Django’s releases. So I prepared this handy guide to help you choose (or confuse?).

Which Version To Use?

Django has now standardized on a release schedule with three kinds of releases:

Feature Release: These releases will have new features or improvements to existing features. It will happen every 8 months and will have 16 months of extended support from release. They have version numbers like A.B (note there’s no minor version).

Long-Term Support (LTS) Release: These are special kind of feature releases, which have a longer extended support of three years from the release date. These releases will happen every two years. They have version numbers like A.2 (since every third feature release will be a LTS). LTS releases have few months of overlap to aid in a smoother migration.

Patch Release: These releases are bug fixes or security patches. It is recommended to deploy them as soon as possible. Since they have minimal breaking changes, these upgrades should be painless to apply. They have version numbers like A.B.C

Django roadmap visualized below should make the release approach clearer:

The dates are indicative and may change. This is not an official diagram but something that I created for my understanding.

The big takeaway is that Django 1.11 LTS will be the last release to support Python 2 and it is supported until April 2020. Subsequent versions will use only Python 3.

The right Django version for you will be based on how frequent you can upgrade your Django installation and what features you need. If your project is actively developed and the Django version can be upgraded at least once in 16 months, then you should install the latest feature release regardless of whether it is LTS or non-LTS.

Otherwise, if your project is only occasionally developed then you should pick the most recent LTS version. Upgrading your project’s Django dependency from one feature release to another can be a non-trivial effort. So, read the release notes and plan accordingly.

In any case, make sure you install Patch Releases as soon as they are released. Now, if you are still on Python 2 then read on.

Python 3 has crossed tipping point

When I decided to use Python 3 only while writing my book “Django Design Patterns and Best Practices” in 2015, it was a time when Python 2 versus Python 3 was hotly debated. However, to me Python 3 seemed much more cleaner without arcane syntax like class methods named __unicode__ and classes needing to derive from object parent class.

Now, it is quite a different picture. We just saw how Django no longer supports Python 2 except for the last LTS release. This is a big push for many Python shops to consider Python 3.

Many platforms have upgraded their default Python interpreter. Starting 1st March 2018, Python 3 is announced to be the default “python” in Homebrew installs. ArchLinux had completely switched to Python 3 since 2010.

Fedora has switched to Python 3 as its system default since version 23. Even though python command will launch python3, the symlink /usr/bin/python will still point to python2 for backward compatibility. So it is probably a good idea to use #!/usr/bin/env python idiom in your shell scripts.

On 26 April 2018, when Ubuntu 18.04 LTS (Bionic Beaver) will be released, it is planned to be have Python 3.6 as default. Further upstream, the next Debian release in testing - Debian 10 (Buster) is expected to transition to Python 3.6.

Moving to packages, the Python 3 Wall of Superpowers shows that with the backing of 190 out of 200 packages, at the time of writing, we have nearly all popular Python packages on Python 3. The only notable package remaining is supervisor, which is about to turn green in supervisor 4.0 (unreleased).

Common Python 3 Migration Blockers

You might be aware of atleast one project which is still on Python 2. It could be open source or an internal project, which may be stuck in Python 3 for a number of reasons. I’ve come across a number of such projects and here is my response to such reasons:

Reason 1: My Project is too complex

Some very large and complex projects like NumPy or Django have been migrated sucessfully. You can learn the migration strategies of projects like Django. Django maintained a common codebase for Python 2 and 3 using the six (2 × 3=6, get it?) library before switching to Python 3 only.

Reason 2: I still have time

It is closer than you think. Python clock shows there is a little more than 2 years and 2 months left for Python 2 support.

In fact, you have had a lot of time. It has been ten years since Python 3 was announced. That is a lot of overlap to transition from one version to another.

In today’s ‘move fast and break things’ world, a lot of projects decide to abruptly stop support and ask you to migrate as soon as a new release is out. This is a lot more realistic assumption for enterprises which need a lot more planning and testing.

Reason 3: I have to learn Python 3

But you already know most of it! You might need about 10 mins to learn the differences. In fact, I have written a post to guide Django coders to Python 3. Small Django/Python 2 projects need only trivial changes to work on Python 3.

You might see many old blog posts about Python 3 being buggy or slow. Well, that has not been true for a while. Not only it is extremely stable and bug-free, it is actually used in production by several companies. Performance-wise it has been getting faster in every release, so it is faster than Python 2 in most cases and slower in a few.

Of course, there are lot of awesome new features and libraries added to Python 3. You can learn them as and when you need them. I would recommend reading the release notes to understand them. I will mention my favourites soon.

Reason 4: Nobody is asking

Some people have the philosophy that if nobody is asking then nobody cares. Well, they do care if the application they run is on an unsupported technology. Better plan for the eventual transition than rush it on a higher budget.

Are you missing out?

To me, the biggest reason to switch was that all the newest and greatest features were coming to Python 3. My favourite top three exclusive features in Python 3 are:

asyncio: One of the coolest technologies I picked up recently. The learning process is sometimes mind-bending. But the performance boost in the right situations is incredible.
f-strings: They are so incredibly expressive that you would want to use them everywhere. Instant love!
dict: New compact dict implementation which uses less memory and are ordered!

Yes, FOMO is real.

Apart from my personal reasons, I would recommend everyone to migrate so that the community benefits from investing efforts into a common codebase. Plus we can all be consistent on which Python to recommend to beginners. Because…

There should be one– and preferably only one –obvious way to do it. Although that way may not be obvious at first unless you’re Dutch.

This article contains an excerpt from the upcoming second edition of book "Django Design Patterns and Best Practices" by Arun Ravindran

Hangman in more than three lines of Python

Tue, 16 Feb 2016 20:03:00 +0530

Recently, Danver wrote about a brilliant implementation of the Hangman game in just three lines of Python. But the shortened code might be hard to understand. In this post we will rewrite his code in a more idiomatic Python 3 style.

Spoiler Alert: If you prefer a fun exercise, try doing it yourself first and then compare with my version.

Danver’s implementation cleverly uses the system dictionary (pre-installed in Linux and Mac, but we’ll see how it can work in Windows too) for picking a random word. It also displays a neat ASCII-art hangman at every step. Except for a few minor improvements, I have tried to retain the essence of Danver’s original in my rewrite below.

# -*- coding: utf-8 -*-
"""A simple text-based game of hangman

A re-implementation of Hangman 3-liner in more idiomatic Python 3
Original: http://danverbraganza.com/writings/hangman-in-3-lines-of-python

Requirements:
  A dictionary file at /usr/share/dict/words

Usage:
  $ python hangman.py

Released under the MIT License. (Re)written by Arun Ravindran http://arunrocks.com

"""

import random

DICT = '/usr/share/dict/words'
chosen_word = random.choice(open(DICT).readlines()).upper().strip()
guesses = set()
scaffold = """
|======
|   |
| {3} {0} {5}
|  {2}{1}{4}
|  {6} {7}
|  {8} {9}
|
"""
man = list('OT-\\-//\\||')
guesses_left = len(man)

while not guesses.issuperset(chosen_word) and guesses_left:
    print("{} ({} guesses left)".format(','.join(sorted(guesses)), guesses_left))
    print(scaffold.format(*(man[:-guesses_left] + [' '] * guesses_left)))
    print(' '.join(letter if letter in guesses else '_' for letter in chosen_word))
    guesses.update(c.upper() for c in input() if str.isalpha(c))
    guesses_left = max(len(man) - len(guesses - set(chosen_word)), 0)

if guesses_left > 0:
    print("You win!")
else:
    print("You lose!\n{}\nHANGED!".format(scaffold.format(*man)))
print("Word was", chosen_word)

You can view or download the gist on Github as well.

HO_ IT _ORKS

The code makes excellent use of Python’s built-in set data structure and string format function.

Right after a boring module docstring and the import statement, we initialize the following variables:

DICT: Path to a dictionary file
chosen_word: Randomly picked line (aka. a word) from DICT
guesses: Set of letters guessed by the user, so far
scaffold: Hangman drawn in ASCII art as a format string
man: Missing ASCII characters for scaffold
guesses_left: Remaining letter guesses

The while loop, whose exit condition will be explained soon, is our main game loop. The first print function reminds you the guesses you have entered so far (in alphabetic order) and the number of guesses left. The next print function shows the ASCII hangman with enough missing parts from man based on the number of wrong guesses.

Finally, the third print function shows the chosen_word by revealing only the characters from your guesses and replacing the rest with underscores. In the following line, we read a character or even several characters from the user, uppercase them, filter out the non-alphabets and update the guesses set, all in a single line:

guesses.update(c.upper() for c in input() if str.isalpha(c))

Next, we calculate the number of letters we got wrong. As the diagram below shows, this is the difference of the sets guesses and chosen_words (In the diagram, the letters - C,M,T). We find how many guesses are left by subtracting the set of letters of man from the letters we got wrong. The max function ensures that we don’t go below zero.

Eventually, you could win if your guesses cover all the right letters. In the language of sets, this is when guesses set grows large enough to become a super set of chosen_word set. This is indicated by the dashed purple circle. Hence, the while loop will continue until this happens or we run out of remaining guesses.

Finally, we show a win or lose message based on whether we ran out of guesses. In either case we reveal the initial chosen_word.

IMPRO_EMEN_S

For comparison, here is Danver’s original three-liner:

license, chosen_word, guesses, scaffold, man, guesses_left = 'https://opensource.org/licenses/MIT', ''.join(filter(str.isalpha, __import__('random').choice(open('/usr/share/dict/words').readlines()).upper())), set(), '|======\n|   |\n| {3} {0} {5}\n|  {2}{1}{4}\n|  {6} {7}\n|  {8} {9}\n|', list('OT-\\-//\\||'), 10
while not all(letter in guesses for letter in chosen_word) and guesses_left: _, guesses_left = map(guesses.add, filter(str.isalpha, raw_input('%s(%s guesses left)\n%s\n%s:' % (','.join(sorted(guesses)), guesses_left, scaffold.format(*(man[:10-guesses_left] + [' '] * guesses_left)), ' '.join(letter if letter in guesses else '_' for letter in chosen_word))).upper())), max((10 - len(guesses - set(chosen_word))), 0)
print 'You', ['lose!\n' + scaffold.format(*man), 'win!'][bool(guesses_left)], '\nWord was', chosen_word

Got it? Okay, maybe it needs a bit of explanation. Python is a whitespace-aware language designed for maximum readability. So it takes a lot of questionable tricks to write one-liners, some of which I have used myself in my post “Python One-liner Games”. This time we are going in the opposite direction.

Probably, the most visible changes are:

License: Mentioned in a doc string rather than as a string
Import: No longer need to use the internal __import__ function
Scaffold: Multi-line string rather than a one-liner
Print: Changed from a statement to a function, as in Python 3
Raw_input: Changed to an input function, as in Python 3
C-style %s formats: Replaced with str.format everywhere

But, there are also some less evident code changes:

Loop condition: Replaced list comprehension in while condition with a superset check
Negative index: No need of man[:10-guesses_left], just man[:-guesses_left]
Set update: Replace map/filter with a set.update i.e. from this:

    map(guesses.add, filter(str.isalpha, raw_input...

with this:

    guesses.update(c.upper() for c in input() if str.isalpha(c))

Magic numbers: The number 10 is no longer hard-coded and replaced with len(man)

As you can see, I have preferred set functions over list comprehensions. The choice of set functions like my_set.update over set operators like my_set += was intentional as the former allows any iterable as an argument and not just sets. In most cases, I have used generator expressions for efficiency.

W_Y PYT_ON 3?

I must admit that the actual reason I rewrote the Hangman was because it didn’t work on my Arch Linux box, which runs on Python 3. Over the last few months, I have completely switched over all my projects (and my book) to Python 3. So far I have had no reason to complain.

Apart from the syntactic differences, you will find some solid advantages of using Python 3 versus Python 2:

Unicode files: Since the dictionary file path can be changed, I created a test file with a few Malayalam words in separate lines. The Python 2 version just exits with a win message. This is probably because str.isalpha filters out all the characters. Whereas, in Python 3, everything just works.
Removes arcane forms: No more raw_input (who uses the old input?) or print statements

If you are on Windows, you can download pretty much any of the dictionaries online (even the offensive ones) and change the path in DICT to something like dict.txt. Or, you can create a simple text file with one word on each line. Pro Tip: Hangman is great for learning foreign languages so try creating a Unicode file with some foreign language words.

Before any of the Python 2 fans lash out on me, let me note in the passing that this code will work perfectly on Python 2.7+ if you replace input with raw_input. Cheers!

FI_AL COMME_T

This post is by no means a critique of Danver’s code. In fact, I loved his implementation and totally didn’t waste my weekend playing Hangman (27 wins out of 30 games!!!). It had a lot of fun mapping the problem to sets and making small improvements.

Hope you found some new Python best practices or just an awesome word game to kill time!

Python 3 Cheatsheet for Djangonauts

Wed, 20 May 2015 19:08:00 +0530

If you are already convinced to use Python 3 then you can directly jump to the section “Python 2 vs Python 3” for the cheatsheet

There is an intense debate whenever Python 3 is mentioned. Some think it is an unnecessary shift and Python 2 series should be continued. However, I chose Python 3 for my book because I strongly believe that it is the future of the language. Besides, the divide might not be as large you think.

Why Python 3?

Think of your favourite programming language and what you love about it. Now think of things that you don’t like about it or wish was improved. Sometimes these improvements might break older codebases but you badly want them.

Now imagine if the creator of the language takes upon himself/herself to revamp the language and rid of its warts. This is what actually led to Python 3 - Guido led the efforts to clean-up some fundamental design flaws with the 2.x series based on over fifteen years of experience developing a successful language.

While the development of Python 3 started in 2006, its first release, Python 3.0, was on December 3, 2008. The main reasons for a backwards incompatible version were – switching to Unicode for all strings, increased use of iterators, cleanup of deprecated features like old-style classes and some new syntactic additions like the nonlocal statement.

Python 2.7 development was supposed to end in 2015 but was extended for five more years through 2020. There will not be a Python 2.8. Soon all major Linux distributions would have completely switched to using Python 3 as a default. Many PaaS providers like Heroku already support Python 3.4.

Python 2 and Python 3 have been in parallel development by the core development team for a number of years. Eventually, Python 3 is expected to be the future of the language.

Porting Django

The reaction to Python 3 in the Django community was rather mixed. Even though the language changes between version 2 and 3 were small (and over time, reduced), porting the entire Django codebase was a significant migration effort.

Django has been supporting Python 3 since version 1.5. In fact, the strategy was to rewrite the code into Python 3 and deal with Python 2 as a backward compatibility requirement.

Most of the packages listed in the Python 3 Wall of Superpowers have turned green (indicating they have support for Python 3). Almost all the red ones have an actively developed Python 3 port. Also worth noting is the Django Wall of Superpowers with over 67% of the top 200 Django packages having a Python 3 port.

Python 3 Advantage

Many newcomers to Django wonder whether to start using Python 2 or 3. I recommend starting with Python 3 for the following reasons:

Better syntax: It fixes a lot of ugly syntax like izip, xrange and __unicode__ with the cleaner and more straightforward zip, range and __str__.
Sufficient third-party support: Of the top 200 third party libraries, more than 80% have Python 3 support.
No legacy code: Since they are creating a new project rather than dealing with legacy code which needs to support an older version.
Default in Modern Platforms: It is already the default Python interpreter in Arch Linux. Ubuntu and Fedora plan to complete the switch in a future release.
It is easy: From a Django development point of view, there are so few changes that it can be learnt in a few minutes.

The last point is important. Even if you know Python 2, you can pick up Python 3 in a short time.

Python 2 vs Python 3

This section covers the most important changes in Python 3 from a Django developer’s perspective. To understand the full list of changes, please refer to the recommended reading section in the end.

The examples are given in both Python 2 and Python 3. Depending on your installation, all Python 3 commands might need to be changed from python to python3 or python3.4 to invoke the interpreter.

Change all unicode methods into str

In Python 3, the __str__() method is called for string representation of your models rather than the awkward sounding __unicode__() method. This is one of the most evident ways to identify Python 3 ported code.

Python 2	Python 3
class Person(models.Model): name = models.TextField() def unicode(self): return self.name	class Person(models.Model): name = models.TextField() def str(self): return self.name

This reflects the difference in the way Python 3 treats strings. In Python 2, the human readable representation of a class could be returned by __str__() (bytes) or __unicode__() (text). However, in Python 3 the readable representation is simply returned by __str__() (text).

All Classes Inherit from object

Python 2 has two kinds of classes – old-style (classic) and new-style. New-style classes are classes which directly or indirectly inherit from object. Only new style classes can use Python’s advanced features like slots, descriptors and properties. Many of these are used by Django. However, classes were still old-style by default for compatibility reasons.

In Python 3, the old-style classes don’t exist anymore. As seen below, even if you don’t explicitly mention any parent classes the object class will be present as a base. So, all classes are new-style.

Python 2	Python 3
>>> class CoolMixin: … pass >>> CoolMixin.bases ()	>>> class CoolMixin: … pass >>> CoolMixin.bases (<class ‘object’>,)

Calling super() is Easier

The simpler call to super(), without any arguments, will save you some typing in Python 3.

Python 2	Python 3
class CoolMixin(object): def do_it(self): return super(CoolMixin, self).do_it()	class CoolMixin: def do_it(self): return super().do_it()

Specifying the class name and instance is optional, thereby making your code DRY and less prone to errors while refactoring.

Relative Imports Must be Explicit

Imagine the following directory structure for a package named app1:

/app1
  /__init__.py
  /models.py
  /tests.py

Now, in Python 3, let us run the following in the parent directory of app1:

$ echo "import models" > app1/tests.py
$ python -m app1.tests
Traceback (most recent call last):
   ... omitted ...
ImportError: No module named 'models'
$ echo "from . import models" > app1/tests.py
$ python -m app1.tests

Successfully import-ed

Within a package, you should use explicit relative imports while referring to a sibling module. You can omit __init__.py in Python 3, though it is commonly used to identify a package.

In Python 2, you could use import models to successfully import models.py module. But it was ambiguous and could accidentally import any other models.py in your Python path. Hence this is forbidden in Python 3 and discouraged in Python 2 as well. Use from . import models instead.

HttpRequest and HttpResponse have str and bytes Types

In Python 3, according to PEP 3333 (amendments to the WSGI standard), we are careful not to mix data coming from or leaving via HTTP, which will be in bytes; as opposed to text within the framework, which will be native (Unicode) strings.

Essentially, for HttpRequest and HttpResponse objects:

Headers will be always str objects
Input and output streams will be always byte objects

Unlike Python 2, the string and bytes are not implicitly converted while performing comparisons or concatenations with each other. Strings mean Unicode strings only.

Exception Syntax Changes and Improvements

Exception handling syntax and functionality has been significantly improved in Python 3.

In Python 3, you cannot use the comma separated syntax for the except clause. Use the as keyword instead:

Python 2	Python 3
try: pass except BaseException, e: pass	try: pass except BaseException as e: pass

The new syntax is recommended for Python 2 as well.

In Python 3, all exceptions must be derived (directly or indirectly) from BaseException. In practice, you would create your custom exceptions by deriving from the Exception class.

As a major improvement in error reporting, if an exception occurs while handling an exception then the entire chain of exceptions are reported:

Python 2	Python 3
>>> try: ... print(undefined) ... except Exception: ... print(oops) ... Traceback (most recent call last): File "", line 4, in <module> NameError: name 'oops' is not defined	>>> try: ... print(undefined) ... except Exception: ... print(oops) ... Traceback (most recent call last): File "<stdin>", line 2, in <module> NameError: name 'undefined' is not defined During the handling of the preceding exception, another exception occurred: Traceback (most recent call last): File “<stdin>”, line 4, in <module> NameError: name ‘oops’ is not defined

Python 2

Python 3

>>> try:
...   print(undefined)
... except Exception:
...   print(oops)
...
Traceback (most recent call last):
  File "", line 4, in <module>
  NameError: name 'oops' is not defined

>>> try:
...   print(undefined)
... except Exception:
...   print(oops)
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  NameError: name 'undefined' is not defined
During the handling of the preceding
exception, another exception occurred:
Traceback (most recent call last):
File “<stdin>”, line 4, in <module>
NameError: name ‘oops’ is not defined

Once you get used to this feature, you will definitely miss it in Python 2.

Standard Library Reorganized

The core developers have cleaned-up and better organized the Python standard library. For instance SimpleHTTPServer now lives in the http.server module:

Python 2	Python 3
$ python -m SimpleHTTPServer Serving HTTP on 0.0.0.0 port 8000 ...	$ python -m http.server Serving HTTP on 0.0.0.0 port 8000 ...

New Goodies

Python 3 is not just about language fixes. It is also where bleeding-edge Python development happens. This means improvements to the language in terms of syntax, performance and built-in functionality.

Some of the notable new modules added to Python 3 are:

asyncio : Asynchronous I/O, event loop, coroutines and tasks
unittest.mock : Mock object library for testing
pathlib : Object-oriented filesystem paths
statistics : Mathematical statistics functions

Even if some of these modules might have backports to Python 2, it is more appealing to migrate to Python 3 and leverage them as built-in modules.

Pyvenv and Pip are Built-in

Most serious Python developers prefer to use virtual environments. virtualenv is quite popular to isolate your project setup from the system-wide Python installation. Thankfully, Python 3.3 is integrated with a similar functionality using the venv module.

Since Python 3.4, a fresh virtual environment will be pre-installed with pip, a popular installer:

$ python -m venv djenv
[djenv] $ source djenv/bin/activate
[djenv] $ pip install django

Notice that the command prompt changes to indicate that your virtual environment has been activated.

Other Changes

We cannot possibly fit all the Python 3 changes and improvements in this post. But the other commonly cited changes are:

print is now a function : Previously it was a statement i.e. arguments were not in parenthesis.
Integers don’t overflow : sys.maxint is outdated, integers will have unlimited precision.
Inequality operator <> is removed : Use != instead.
True Integer Division : In Python 2, 3 / 2 would evaluate to 1. It will be correctly evaluated to 1.5 in Python 3.
Use range instead of xrange : range() will now return iterators as xrange() used to work before.
Dictionary keys are views : dict and dict-like classes (like QueryDict) will return iterators instead of lists for keys(), items() and values() method calls.

This article is an excerpt from the book "Django Design Patterns and Best Practices" by Arun Ravindran

Further Information

Read What’s New In Python 3.0 by Guido https://docs.python.org/3/whatsnew/3.0.html
To find what is new in each release of Python, read _What’s New in Python_https://docs.python.org/3/whatsnew/
For richly detailed answers about Python 3 read Python 3 Q & A by Nick Coghlan http://python-notes.curiousefficiency.org/en/latest/python3/questions_and_answers.html

Introducing Edge - a Modern Django Project Template

Mon, 22 Sep 2014 08:19:00 +0530

A newer version has been announced

Have you ever begun creating a web app only to abandon it while setting it up? I have done it more times than I would have liked.

Many programming environments involve so much ritual to setup that we tend to forget what we set out to create. Recently, I had to setup an Android development environment from scratch. Despite downloading the latest version of the IDE, I waited until several additional updates downloaded to patch it to the latest version. The trend of packages getting more granular and updates being made several times a day is here to stay.

It is like if you have an idea for a painting then you have to go shopping for all the art-supplies first. But the moment you step out, you have lost your muse. Scientists call this encoding specificity or as a famous paper calls it “Walking through doorways causes forgetting.”

Django Projects

Of course, in Django I use startproject which builds an excellent but minimal project structure. But once I start coding I inevitably tend to install packages like the django braces. Most of my project use Bootstrap, so I end up downloading the bootstrap assets as well. Any non-trivial web app needs some login functionality, so I end up adding that too.

In short, by the time I have setup everything to actually start building the app, I have crossed the doorway and forgotten most of the idea. And lost a bit of the enthusiasm too. Edge - for Django Projects in 2014

So, I created Edge. I believe Edge is a great starting point for Django projects. It comes with Bootstrap assets (even for the admin), basic user sign up flows and a sensible set of default settings. Most importantly, it is built for Django 1.7 and Python 3.4, the latest versions at the time of writing.

As soon as you start your project in edge, you will notice that it has a great looking and functional home page rather than Django’s familiar blue “It Worked!” page. The appearance is much closer to what you expect as a starting point for your app. Check it out yourself in the short demo below.

Motivations

Things I found missing or inadequate with the default Django project structure

Iconic Omissions

It started with the small things. Doesn’t it always? Every time I noticed a 404 while developing a Django application I had to calm myself down - don’t worry, it is just a missing favicon.

I know I would eventually add a favicon in production. But I kind of turned blind to errors on my test server logs. The fix was to add a favicon, of course.

Template and Static paths

Another one of those small things that I would always need to specify to my settings.py. I understand why the paths are not there. But during developing, it would be so much more convenient, if these directories were present and added to settings.

Pathlib

Talking about paths, the awkward os.path.join(BASE_DIR, “templates”) call is needed anymore. Edge uses the new pathlib library in Python 3.4 instead. The new syntax BASE_DIR / “templates” is much more convenient and readable.

Bootstrap

It is 2014 and hardly anybody starts with a blank page (I mean, blank HTML and CSS) anymore. In fact, most people I know start with a minimal Bootstrap 3 template.

There are many good packages for using bootstrap in Django. But edge doesn’t use any of them because it is becomes another dependency that sometimes lags behind the rapid development cycle of Bootstrap. Instead, bootstrap assets have been directly included in the static folder. Updating is now easier, just copy the latest versions into the appropriate paths.

User Authentication

Django has a built-in user model. It is the base for the admin interface and countless other packages. But a simple user sign-up, login and logout workflow needs quite a bit of work to setup.

Django registrations was often used for this but has been inactive for a while. Even though Django allauth is an excellent solution in production, I would hardly use social login in my development environment.

In edge, a simple sign-up and login form has been provided using crispy forms and class based generic views. It does not do email verification or need a dummy mail server setup. But it helps you prototype and build web apps that can be shown to users for early testing.

Admin

“What is this Django Administration?” If only I had a nickel every time someone asked me this. Most projects continue to have their admin interface titled that way even in production.

The admin interface’s appearance also looks quite antiquated. Try opening it on a mobile device. The edge template uses django-admin-bootstrapped to give it a Bootstrap 3 theme. Also, it customises the heading to “MyProject Administration”.

Alternate Project Templates

There are many good project templates already available. I did try using django-twoscoops-project, django-base-template, django-project-skel etc. Some of them did not work properly in Python 3. Others seemed too bloated.

Project templates are a subjective choice. While I prefer a template with all the necessary settings to start focusing on the problem at hand, I find packages like Celery in the bells and whistles category. Using a project template generator like cookiecutter was also something I never preferred.

My solution - edge

Edge has everything what I expect to be in a Django 1.7 and Python 3.4 project. I have used edge in a couple of projects and look forward to using it in my future projects. It is my new baseline.

More details about the features and future plans are on the wiki. The project itself is open source and on Github.

I would love to hear your feedback.

Recreating the "Building a Blog in Django" Screencast

Sun, 06 Jul 2014 18:56:00 +0530

Few days back, Django 1.7 release candidate 1 was announced. I have never been so excited about a release in recent memory. It has a new app loading framework and schema migrations baked right in!

It is also the significant release that I had been waiting for to update my original Django screencast. With more than 77 thousand views, it is by far the most popular Django screencast on YouTube.

Back in 2012, when I was working in Django, there were a lot of comparisons between Rails and Django. One of the biggest selling points of Ruby on Rails was their official 15-minute blog screencast. It was fascinating to watch an expert Rails developer build an entire site from scratch especially how deftly they use TextMate, which could nearly read his mind.

Armed with a Linux terminal and Emacs, I set out to create a similar screencast for anyone interested in Django. Live coding was something that I had not tried before. I remember starting at 11 pm one night, hoping to complete my recording by midnight.

But by the time I finished a recording without any major goofups it was 5 am in the morning. So with barely any editing, I directly uploaded the raw footage to YouTube and hit the bed.

Over time, a lot of people have loved the screencast, often asking how they can setup their development environment to resemble mine. Emacs despite not being an IDE is often as productive or more, depending on how well you customise it.

Why the Remake?

I have done several more Django screencasts after that, preferring to demonstrate entire projects in Django rather than focus on a specific feature. But the original still remains a favourite as it is beginner-friendly and short.

Plenty of new cool features like Class-based Views had come to Django since then. I also noted that there were very few tutorials which used Python 3. For starting a new Django project, I would definitely recommend Python 3.

So, I decided to remake it but even shorter. Crazily enough, I wanted to cover a lot more than the first screencast. Here are the topics I wanted to touch upon (ones which were not covered in the first screencast are in bold):

New 1.7 defaults
Syntax changes in Python 3.4
QuerySets and Custom Managers
Admin and ModelAdmin
Rich-text posts with django-markdown
Generic Class-based Views
Tags
RSS Feeds
Migrations
Running testcases

Thankfully, Django 1.7 comes with great project defaults like a predefined SQLite 3 database and admin URLs setup, which gives you a running start. Migrations is a real time-saver as you don’t have to keep re-entering your test data.

The entire recording clocked slightly above 15 mins. Here is the final video and the text version below it:

People have translated my tutorials into various languages in the past. So I have added subtitles to this screencast. Enjoy the automatic translations in your preferred language!

Text Version

Some prefer to read the transcript of the screencast. Assuming you already have Django 1.7 installed in Python 3.4, you can follow these steps.

Start the project

    django-admin startproject qblog
    cd qblog
    ./manage.py migrate
    ./manage.py createsuperuser
    ./manage.py runserver

Add an app called blog

    ./manage startapp blog

Open blog/models.py and change its contents to:

    from django.db import models
    from django.core.urlresolvers import reverse


    class EntryQuerySet(models.QuerySet):
        def published(self):
            return self.filter(publish=True)


    class Entry(models.Model):
        title = models.CharField(max_length=200)
        body = models.TextField()
        slug = models.SlugField(max_length=200, unique=True)
        publish = models.BooleanField(default=False)
        created = models.DateTimeField(auto_now_add=True)
        modified = models.DateTimeField(auto_now=True)

        objects = EntryQuerySet.as_manager()

        def __str__(self):
            return self.title

        class Meta:
            verbose_name = "Blog Entry"
            verbose_name_plural = "Blog Entries"
            ordering = ["-created"]

In qblog/settings.py add "blog" to INSTALLED_APPS and migrate:

    ./manage.py makemigrations blog
    ./manage.py migrate blog

Change blog/admin.py to:

    from django.contrib import admin
    from . import models


    class EntryAdmin(admin.ModelAdmin):
        list_display = ("title", "created")
        prepopulated_fields = {"slug": ("title",)}

    admin.site.register(models.Entry, EntryAdmin)

Go to admin. Add an unpublished post and published one. Then open `./manage shell` and check the difference between `Entry.objects.all()` and `Entry.objects.published()`.

Markdown

Install django-markdown

    pip install django-markdown

Add "django_markdown" to INSTALLED_APPS.
Add a second line to qblog/urls.py:

    url(r'^markdown/', include('django_markdown.urls')),

Change blog/admin.py to:

    from django_markdown.admin import MarkdownModelAdmin


    class EntryAdmin(MarkdownModelAdmin):
        ...

Enter test entries now.

Markdown Editor Toolbar Not Showing in Admin?

Despite this being a Python 3.x tutorial, many have tried this in Python 2.x. Which is great, but the snazzy Markdown Editor in admin seems to be missing for them.

When I had checked, I found that the Markdown widget's media i.e. javascript and stylesheets were not getting added in admin. A workaround is to explicitly define form_overrides as follows.

  from django_markdown.widgets import AdminMarkdownWidget
  from django.db.models import TextField
class EntryAdmin(MarkdownModelAdmin):
list_display = (“title”, “created”)
prepopulated_fields = {“slug”: (“title”,)}
# Next line is a workaround for Python 2.x
formfield_overrides = {TextField: {‘widget’: AdminMarkdownWidget}}

Index View

Change blog/views.py to:

    from django.views import generic
    from . import models


    class BlogIndex(generic.ListView):
        queryset = models.Entry.objects.published()
        template_name = "home.html"
        paginate_by = 2

Create a url mapping in qblog/urls.py:

    urlpatterns = patterns(
        '',
        url(r'^admin/', include(admin.site.urls)),
        url(r'^markdown/', include('django_markdown.urls')),
        url(r'^', include('blog.urls')),
    )

Create blog/urls.py:

    from django.conf.urls import patterns, include, url
    from . import views

    urlpatterns = patterns(
        '',
        url(r'^$', views.BlogIndex.as_view(), name="index"),
    )

Index Template

Make template and static directories. Then, copy the files:

    mkdir templates
    cp -R /somewhere/templates/* templates
    mkdir static
    cp -R /somewhere/static/* static

Create templates/home.html with:

    {% extends "base.html" %}
    {% load django_markdown %}

    {% block blog_entries %}
    {% for object in object_list %}
      <div class="post">
        <h2>{{ object.title }}</h2>
        <p class="meta">{{ object.created }}</p>
        {{ object.body|markdown }}
      </div>
    {% endfor %}
    {% endblock %}

Add to qblog/settings.py:

    TEMPLATE_DIRS = (os.path.join(BASE_DIR, "templates"), )
    STATICFILES_DIRS = (os.path.join(BASE_DIR, "static"), )

Refresh the home page now. Enjoy the first non-admin page.

Feed

Create blog/feed.py with:

    from django.contrib.syndication.views import Feed
    from blog.models import Entry


    class LatestPosts(Feed):
        title = "Q Blog"
        link = "/feed/"
        description = "Latest Posts"

        def items(self):
            return Entry.objects.published()[:5]

Add to blog/urls.py (last but one line):

    from . import views, feed
    
    url(r'^feed/$', feed.LatestPosts(), name="feed"),

Mention in the templates/base.html template’s HEAD:

    <link href="/feed/" rel="alternate" type="application/rss+xml" title="Q Blog Feed" />

Entry View

Add to blog/views.py:

    class BlogDetail(generic.DetailView):
        model = models.Entry
        template_name = "post.html"

Add to blog/urls.py:

    url(r'^entry/(?P<slug>\S+)$', views.BlogDetail.as_view(), name="entry_detail"),

Add to blog/models.py:

    objects = EntryQuerySet.as_manager()

    def get_absolute_url(self):
        return reverse("entry_detail", kwargs={"slug": self.slug})

Entry Template

Open templates/home.html and save it as post.html while removing the for loop, like this:

    {% extends "base.html" %}
    {% load django_markdown %}

    <div class="post">
      <h2><a href="{% url "entry_detail" slug=object.slug %}">{{ object.title }}</a></h2>
      <p class="meta">
        {{ object.created }} |
        Tagged under {{  object.tags.all|join:", " }}
      </p>
      {{ object.body|markdown }}
    </div>

Update `templates/home.html` with the same `<h2>` line.

Testing

Add these two test cases in blog/tests.py:

    from django.test import TestCase
    from django.contrib.auth import get_user_model
    from .models import Entry


    class BlogPostTest(TestCase):

        def test_create_unpublished(self):
            entry = Entry(title="Title Me", body=" ", publish=False)
            entry.save()
            self.assertEqual(Entry.objects.all().count(), 1)
            self.assertEqual(Entry.objects.published().count(), 0)
            entry.publish = True
            entry.save()
            self.assertEqual(Entry.objects.published().count(), 1)


    class BlogViewTests(TestCase):
        def test_feed_url(self):
            response = self.client.get('/feed/')
            self.assertIn("xml", response['Content-Type'])

Run the tests by ./manage.py test blog and they should both pass.

Final Notes

The entire source is available on Github. If you face any issues, make sure that you are running the same Python/Django versions and compare your code with mine.

As always, I would love to hear your comments and feedback.

An Easy Guide to Install Python or Pip on Windows

Mon, 30 Dec 2013 21:49:00 +0530

[Updated 2017-04-15] These steps might not be required in latest Python distributions which are already shipped with pip.

I enjoy studying other people’s development environments. They give me plenty of insights into their aesthetic choices and their productivity shortcuts. But I have seen a lot of horrible environments cobbled together by flaky batch files that are one update away from blowing up, often leaving no option but to reinstall everything from scratch. Unfortunately, they have all been Windows environments.

Recently, I tried to install Python and pip on a Windows laptop. Though I have installed Python on Windows XP and Windows Servers for several years till 2010; there have been a lot of changes both in the Windows world such as Powershell and the Python world. So there is a lot of confusing information out there for the Python beginner. And then there is Python 3. Sigh!

Suffice to say I wanted to share what I learnt in a video aimed at the Python beginner or anyone struggling to install Python on Windows. The video and the high-level transcript follows:

What is covered?

Installing Python 2 or Python 3 on Windows 7
Installing pip from PowerShell
Setting up a virtualenv
Installing via pip

How to install Python / Pip on Windows 7 (or 8)

Download the MSI installer from http://www.python.org/download/. Select 32 bit or 64 bit based on the System Settings which opens by pressing Win+Break
Run the installer. Be sure to check the option to add Python to your PATH while installing.
Open PowerShell as admin by right clicking on the PowerShell icon and selecting ‘Run as Admin’
To solve permission issues, run the following command:
```
Set-ExecutionPolicy Unrestricted
```
Enter the following commands in PowerShell to download the bootstrap scripts for easy_install and pip:
```
mkdir c:\envs
cd c:\envs

(new-object System.Net.WebClient).DownloadFile('https://bootstrap.pypa.io/ez_setup.py',   'c:\envs\distribute_setup.py')

(new-object System.Net.WebClient).DownloadFile('https://raw.github.com/pypa/pip/master/contrib/get-pip.py', 'c:\envs\get-pip.py')

python c:\envs\distribute_setup.py
python c:\envs\get-pip.py
```
Once these commands run successfully, you can delete the scripts get-pip.py and distribute_setup.py

HTTP Issue with distribute_setup.py?
[Updated 2015-03-05] The script distribute_setup is no longer available at its old location. Thanks to Rudhra for sharing the new location.
Now typing easy_install or pip should work. If it doesn’t it means the Scripts folder is not in your path. Run the next command in that case (Note that this command must be run only once or your PATH will get longer and longer). Make sure to replace c:\Python33\Scripts with the correct location of your Python installation:
```
setx PATH "%PATH%;C:\Python33\Scripts"
```
Close and reopen PowerShell after running this command.
To create a Virtual Environment, use the following commands:
```
cd c:\python
pip install virtualenv
virtualenv acme
.\acme\Scripts\activate.ps1
pip install IPython
ipython3
```
That’s it! I suppose the same steps work for Windows 8 as well. But I don’t have a machine to try it out. Do let me know if this worked for you.