So, with front-end pretty much designed right, we now move to backend design for our per-paragraph commenting app and we will work with Django

---

So it finally begins. Here, we now dive into the backend, having our front end fairly finished. What I would like to do though, since I've been reading and learning about this TDD (Test Driven Development) approach, is that we do one step at a time, and also follow the test driven approach to make this app. Okay?

So, in this post, we would do some groundwork by setting up the backend app prerequisites and indulge in some design blabber.

Just as a reminder, we had seen what all data we'll like to pull from the backend for each annotation. It looked like:

{
  content_id: "1",
  paragraph_id: "1",
  annotation_body: "Lorem Ipsum",
  user : {
    user_id: "1",
    user_name: "John Doe",
    user_gravatar: "images/male.png",
    user_url: "#",
    },
}

We could straightaway use this as a basis of our app. But first, the preliminaries - In its inception, I designed this app to be fairly modular and independent. I even experimented with 'on the fly' magic, where one doesn't have to tell the other app about this app at all, and it would still work. But it didn't quite happen.

There are a few reasons, or rather limitations which I don't know how to circumvent for now. Salient ones are:

1. Suppose we were to make a completely decoupled app, that would mean we wouldn't need to make any template changes to the parent app too. Then we'll be hooking on to content rendered by the other app. But how do we know which part in the page is the one we must work on. We needed that 'commentable-container' right? So, some kind of template intrusion is there already.

2. We also need those unique IDs on the content blocks, they're not going to produce themselves? In fact, we developed such functionality in our 'blogging' app just to cater to this feature, and it works quite well. You can try creating an annotation on this page itself to see how it works.

3. We could go with Django's AJAX capabilities, but that would be too much work. Wasn't Django all about DRY? and in fact, so is Pirate Learner! So, I evaluated TastyPie and Django Rest Framework, and though Tastypie seemed a little more tastier, reviews and developer feedback said Django Rest Framework is more... bendable, so to speak. But for Django REST framework, we needed to put a generic relation in the target app too. The good news is, it doesn't go into the DB. The bad news is, it still goes into their model, to ease up the reverse mappings and URL generation.

So, if you can see why I was not able to make a completely independent module, I can purport that it is a loosely coupled app. Whenever you are ready, we can dive into developing it backend.

Test first, Code later.

'Test first, Code later' - That's what TDD philosophy is. Make a test (which will fail), then write the minimal amount of code which will cause the test to pass, next test, repeat. Let's see how well can I do it.

In this tutorial, we'll set up an environment and get our packages.

Preparing the Environment:

Inside a folder, lets say, ' project ' create a virtual environment using virtualenv

  virtualenv env --no-site-packages

We'd like to add that --no-site-packages so that our virtual environment does not load any globally installed libraries. Why? Because that will ensure that I am tracking my dependent modules in my requirements.txt file well.

In case you don't have it installed, you can install it using

  sudo pip install virtualenv

To install ' pip ', get the latest get_pip.py script. Note that I am working on a Linux machine, and you might need to google a lot if your's is not a *nix family OS.

  Django==1.6.8
  pi-blogging >=0.1.0b1
  #blogging requirements
  django-classy-tags==0.4
  html5lib==1.0b1
  django-mptt==0.6
  django-sekizai==0.7
  six==1.3.0
  django-ckeditor==4.4.4
  south==1.0.2
  # Optional, recommended packages
  Pillow
  django-filer==0.9.5
  django-reversion==1.8.5
  django_select2
  easy_thumbnails
  django-taggit
  django-crispy-forms
  beautifulsoup4
  lxml
  #annotation requirements
  djangorestframework
  selenium

You can install them all by putting them in a requirements.txt and using ' pip install -r requirements.txt '

NOTE: installing pillow seems to cause installation problems on many systems. For that, try the following:

  sudo ln -s /usr/include/freetype2 /usr/local/include/freetype
  sudo apt-get build-dep python-imaging

as suggested on this linkseems to fix it on one of my systems.

Further, you'll also need development versions of libxml2 and libxslt :

  sudo apt-get install libxml2-dev libxslt1.1 libxslt1-dev libxml2

Thereafter, we configure our Django in our virtual environment:

  django-admin.py startproject demo

This should create a project directory structure resembling this:

demo/
|-- demo
|   |-- __init__.py
|   |-- settings.py
|   |-- urls.py
|   `-- wsgi.py
`-- manage.py

In the settings.py , setup the blogging app and its prerequisites, and finally our annotations app:

  INSTALLED_APPS = (
  ...
  'blogging',
  'mptt',
  'sekizai',
  'reversion',
  'django_select2',
  'easy_thumbnails',
  'filer',
  'taggit',
  'crispy_forms',
  'ckeditor',
  'annotations',
  ...
  )

  TEMPLATE_CONTEXT_PROCESSORS = (
  'django.contrib.auth.context_processors.auth',
  'django.contrib.messages.context_processors.messages',
  'django.core.context_processors.i18n',
  'django.core.context_processors.debug',
  'django.core.context_processors.request',
  'django.core.context_processors.media',
  'django.core.context_processors.csrf',
  'django.core.context_processors.tz',
  'sekizai.context_processors.sekizai',
  'django.core.context_processors.static',
  )

  TEMPLATE_DIRS = (
  os.path.join(PROJECT_PATH, "templates"),
  )
  STATIC_URL = '/static/'
  MEDIA_ROOT = PROJECT_PATH+'/media'
  MEDIA_URL = '/media/'
  STATICFILES_DIRS = (
  PROJECT_PATH+"/static",
  )

With this done, we can start writing our first test, fail and and then move ahead. Since we are sitting on top of a page, we assume that most of the rendering of content is in place. We will not be testing other apps in this tutorial, but just our annotation app.

So, lets see if we can get our app to resolve in our first test case.

In tests.py :

  from django.test import TestCase
  from django.core.urlresolvers import resolve
  #from django.http import request
  # Create your tests here.
  class basic_setup(TestCase):
      def test_can_resolve_app_url(self):
          obj = resolve('/annotations/')

And run:

  python manage.py tests

Resolver404: {u'path': 'annotations/', u'tried': [[
<regexurlpattern home="">], [
<regexurlresolver> (admin:admin) ^admin/>]]}
----------------------------------------------------------------------
Ran 1 test in 0.027s
FAILED (errors=1)
</regexurlresolver>
</regexurlpattern>

Oh, we haven't added the url pattern. Add it in demo/urls.py :

  url(r'^annotations/', include('annotations.urls', namespace='annotations')),        

Note that we must have the blogging app set up properly, which means a url for that in the urls.py . So if you haven't, add it now:

  url(r'^blogging/', include('blogging.urls',namespace='blogging')),

Run the tests again:

ImportError: No module named urls
----------------------------------------------------------------------
Ran 1 test in 0.025s
FAILED (errors=1)

Oh, yes, we haven't made a URL router for our app yet. Do we need it? Usually we will need a urls.py file in our app if there are more URL patterns that we want to deduce, that is, there is more hierarchy in our app. In our current case, what do we expect?

There will be either GET or a POST request. using the GET request, we can get all the annotations for some content, and with POST , we can add new annotations for the particular post. This means that we don't really need it.

Thus, we can change our code by directly calling the view of our app.

  url(r'^annotations/', 'annotations.views.home'),

Okay, run the tests again:

ViewDoesNotExist: Could not import annotations.views.home. View does not exist in module annotations.views.
----------------------------------------------------------------------
Ran 1 test in 0.027s
FAILED (errors=1)

New error, telling us that we haven't yet created a view. Let us begin with a dummy:

In views.py of annotations:

  def home():
      pass

Running the tests passes the result.

Next test, the function returned and what we expect must be the same:

  self.assertEqual(obj.func, home)

This too shall pass, meaning that we are indeed returning the desired function. Let us think of the next step now. What would the user find if he simply visited the /annotations/ page? Above all, should he visit the annotations page at all, or should it be like magic?

So, here is what the app must do:

1. It must allow creation of new annotations on posts. This needs only POST content where we'll have content object type, its ID and body.
2. It must allow retrieving annotations for posts. This could be a GET method, or we could have a URL binding, of type /annotations/content_type/object_id/
3. It must allow retrieving annotations made by the current user, and their parent post URLs. This could be a GET method, or another URL binding like /annotations/users/<user>/
4. It must allow retrieving annotations made by another user which are marked as public. Same as above
5. It must allow modifications of annotations by the author. /annotations/<id>/
6. It must allow deletion of annotations by the author.
7. It must allow moderation of annotations by administrators or moderators.

Looking at this list above, no-where do we find any reason that the base page url must yield anything without a GET request, or a POST request. Also, since this would work behind the scenes, we don't expect users to be directly using these URLs, but via Ajax calls only. It seems a better thing to me to have a single URL where we can filter out the request by GET parameters or POST parameters. Except for the updation of annotations, where it would be better if we know what annotations are we updating. So, for that one case, a URL scheme is justified.

  urlpatterns = patterns('',
  url(r'^$', views.home, name='home'),
  url(r'^/(?P<id>\d+)/', views.update, name='update'))

For now, we'll build up from 'no-ajax' to 'Ajax' to 'Ajax-via REST', and develop accordingly.

So, let us first create a few blogging articles, I'll do this with their fixtures (which comes bundled with the package).

First, create database tables:

  python manage.py syncdb

then,

  python manage.py loaddata fixtures.json

We'll do a manual test to see if they have been created by starting the server. [We could have used selenium for that, but it is a small test. We'll not bring the big guns for now]. Hmm, everything seems to load, but I can't see any images. I had encountered this problem fairly recently, so I know what is wrong. The development server is not serving up static files. Why? Because we haven't told it to. How do we tell it to? By adding a rule in the urls.py .

  from django.conf.urls.static import static
  from django.conf import settings

The `settings` include is needed to find our MEDIA_ROOT which we had defined in our settings file (or else Django cannot find it here)

  urlpatterns = urlpatterns+ static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)

Restarting the server (it autorestarts, by the way), and lo! the images now load as desired. Let us also have a look at the markup of an article (does it contain unique IDs?). It does! Great! Also, we have already done the formating of the front end. So, why not hook it up here directly?

We need to load our script. Let us just load it on every page for now (since it is easy to hack into the demo site's base html rather than every other app's templates.) Our previous tutorial's scripts and CSS will be used. It seems good practice to me to bundle the static files along with the app. So, let us make the static folder in our annotation app and put the JS and CSS files in there.

So, now, annotations app folder structure looks like this:

annotations/
|-- admin.py
|-- forms.py
|-- __init__.py
|-- models.py
|-- static
|   |-- css
|   |   |-- styles.css
|   |   `-- styles.scss
|   `-- js
|       `-- script.js
|-- tests.py
`-- views.py

Start the server, see if it looks good? It does, mostly. Good enough to go ahead for now, in my case (Screenshot below). Now, back to design. But in the next tutorial!