Had a bit of fun today with my site after discovering nearly 2000 spam comments. Thankfully the comments go through an approval system first before they are visisble.

I also threw django-grappelli on for a pretty admin area. It looks quite neat.

By adding a newer jQuery and jQuery-UI to the admin section I also broke the Zinnia tag autocomplete. I rectified this by replacing the autocomplete_tags.js which utilised Jörn Zaefferer's jQuery plugin with code to call jQuery UI autocomplete.

Up early this morning (noisy kids).

I enabled the PowerShell and Shell-Session lexers in Pygments:

Import-Csv -Path New-VMs.csv | `
ForEach-Object {
  New-VM -VMHost $_.VMHost -Name $_.Name -Template $_.Template -Datastore $_.Datastore
}

[timd@fedorabox ~]$ yum install eric
Loaded plugins: fastestmirror, langpacks, refresh-packagekit
You need to be root to perform this command.

update wiki_wikipage set article = replace(article,'</code>','{/code}');

Today I looked at an issue I was having with the syntax hilighting on my site. The way it currently existed, you needed to exit the blog entry html directly to get code in the post using a <code> tag. I've now switched to code in square brackets. This means that the you can add code snippets in the GUI editor, however, TinyMCE comes along and says "I'm helping" and replaces all of your spaces with nbsp and all of your carriage returns with break tags. So, I've found you need to reverse that with a replace prior to parsing the code in pyments. The end result is pulling a bunch of stuff out, parsing and hilighting, then putting it back.

Pygments Python Syntax Hilighter Rules!

        lexer_string = ''
        if match_obj.group(3) is not None:
            lexer_string = match_obj.group(3).lower()
            if lexer_string in ['c','cpp','c++','cpplexer']:
                lexer = CppLexer()
            elif lexer_string in ['py','python','pythonlexer']:
                lexer = PythonLexer()

#include <htc.h>
#define _XTAL_FREQ 20000000
__CONFIG (CP_OFF & CCPMX_RB0 & DEBUG_ON & WRT_OFF & CPD_OFF & LVP_OFF & BOREN_ON & MCLRE_ON & PWRTE_OFF & WDTE_OFF & FOSC_HS);

void putc(unsigned char dt);
void led_digit(unsigned char digit, unsigned char position);
void led_readouts(void);
void led_integer(unsigned int number);
void led_raw(unsigned char raw);
void led_hold(void);
unsigned int get_adc(unsigned char chan);

    bcf       STATUS,RP1
    bsf       STATUS,RP0                        ; Select Bank 1 of data memory
    movlw     0xD7
    movwf     OPTION_REG
    bcf       STATUS,RP0
    bcf       INTCON,GIE                        ; Disable all interrupts.

Ooops. I's got a big bug in my twitter API code for the site =/

$ psql
Type "help" for help.

=# select count(id) from mytweets_mytweet;

count
-------
62479

(1 row)

EDIT:

It turns out that I was relying on a DB constraint that didn't make it through the server migration - so the code was just jamming records in after each Twitter API GET. The insert is wrapped in a try/catch so that it will not bail when there is an integrity error (duplicate keys). With the field not set with a unique constraint - the code just keeps happily sticking records in there... All good now.

I've spent today hacking in some new features for the gallery. Unfortunately, not public facing. Just stuff to assist in uploading photos.

The most comprehensive change is the new photo selector and having a stockphoto (django module) photo attached to all blog posts. There was some messing about with column types and casting and alterting tables in postgres to make it play nice. The field for an article image used to be just a character varying field, it's now an integer field with a proper foreign key relationship to the stockphoto module.

In the admin section, there is now a nice jquery-ui based dialog for selecting from group to gallery to photo and updating the field. This is much better for photos because the standard django select widget is not up to the task. I mean it works fine, but you really don't want to have to choose through a list of "IMG_1235.jpg" etc etc - you really want a thumb.

Getting this custom functionality was just a matter of overriding the admin template for that app by creating an <app>/templates/<app>/<model>/change_form.html/. Secondly, the forms.Widget widget is wrapped so that we can append whatever we want in the admin section - in my case just a text box and a "browse" button. The real work is done in jquery code in the template.

Sadly, updating jquery-ui has borked a few things like tag autocompletion.

Hrm... facepalm... "default=datetime.now()" != "default=datetime.now" #python

I have the date in the title of my blog entries, recently  I added some code to do this automatically. When defining the model originally I had default=datetime.now(). This set the default title of my blog to the date and time when the model was evaluated. So when I came back the next day to add an entry the date was for the day I added the code and touched the .wsgi file. By passing the function, not the result it means that the function is evaluated when a record is made.

I really should stop adding features to my new website and put it live. It will never be done!

I found a great site demonstrating how to add reCaptcha to the existing comments app from django.contrib.

http://www.betterprogramming.com/recaptcha-for-djangos-comments-framework.html

So, there is blog, wiki and gallery, thats all the old featured covered. Time for a new feature. Today I wrote an app which pulls my tweets from the twitter API and stores them in the database and also display the last few in the side bar. The app has one new comonent which I don't normally use - a management interface.This allows an external call using manage.py running on crontab to go and fetch the last few messages.

I also ran through a lot of old blogs and tagged them - the tag cloud is looking a lot healthier now.

I have been working on this site on and off for a week now and having great fun hacking some python.

As a base for the blog I am using Django. Django is a framwork which is python based and tends to be very modular. The main modules are called applications or apps. There are millions of apps to do different jobs. As a starting point I used Zinnia. Zinnia looks after blogging and includes tagging and CRM type stuff. It was as simple matter of concocting some sqlfu and pulling the data out of mysql and sticking it in postgres. Blog entries are very simple.

Being a bit rusty on Django, I opted to start with stockphoto, a simple gallery app. The ultimate aim is to replicate all of the overall functionality that my current PHP based site has. The original gallery was gallery2 based with over 4000 images in it. There was no way that I was going to upload those again! I ended up doing a lot of hacking on the stockphoto model. They used a number of field names that match postgres keywords like "date" for the creation date field and "desc" for the description field.

The existing gallery2 instance has about 200 galleries with nesting up to 4 levels deep from the root. I was willing to cut this down to just 2 levels. To facilitate this I created the "group" model, which is really just another "gallery" model which lists galleries. That way there are groups in the root, then galleries, then photos. I also customised the admin areas a bit with extra list columns. I wrote two templatetags for stockphoto, first the index page, which shows the list of groups in the sidebar, the second is the random image block. Next on the agenda is to add in tagging.

On a roll now, I decided to tackle the wiki. The existing site had a mediawiki instance with lots of stuff on it which I didn't particulary want to rewrite. Once I worked out how to get the latest revisions of each pages out of the DB, I dumped them all. Next I replicated the schema in postgres and imported the wiki entries. I was then able to create wiki records from the mediawiki data. There was a lot of messing around required, because the wiki module is designed to be used with markdown. I modified the wiki app to incude the wikimarkup python library. I then wrote a new filter which used that instead of the markdown filter.

The mediawiki has a feature currently which allows me to include code and have it syntax hilighted. To replicate this feature I made another filter based on python-pygments. There was all sorts of messing around required because the two filters werent' playing nice together. I found in the end that I needed to run the field through the pygments filter first, then the wikimarkup filter. It's still not perfect. I'm also having trouble with something eating newlines on blog entries code tags.

I also wrote a template tag for the wiki which dumps out a link to the index, as well as a list of featured pages. This involved another model change to include the models.BooleanField() for featuring it.

All of this happend over a week with lots of back and forward and adjustments and tearout-and-reinstall of apps. It's been good fun!

May Already?!

Kel had lot to do today so I looked after the Kids. If only I could get them to dig in their sand pit and not in the dirt.

I showed Liam an episode of The Mysterious Cities Of Gold, both he Emily were GLUED to it. The Mysterious Cities of Gold is a cartoon that was on when I was little in the late 80s. I used to like it alot. Liam even said that he likes it better than BEN 10!

I had some time today to build an ICSP attachment for programming 18 pin pics. My JDM serial programmer doesn't seem to cut it any more. I designed the link cable from the ICD2 so that it fits the headers on the attachments for the JDM so I can now do 18, 28 and 40 pin pics. The JDM has become unreliable for some reason, I have trouble verifying the code once it's written - and it takes aaages to write. The ICD2 takes only a few seconds.

I built up another MAX232CPE serial prototype circuit on the breadboard for my next round of tinkering with the 16f88. Although, tonight I did manage to get the UBW firmware D (USB CDC) to compile after lots and lots of messing around on and off over the past few weeks. If I can get that built and the bootloader then I can go direct to USB rather than serial. I have a Sparkfun UBW with the 18f4550 on it and also two 18f2550s for breadboard work.

The workflow using the ICD 2 is going to be different. I am used to pulling the chip out of circuit to program it. The next target is to test the ICD functionality - I've never tried it before.

I added mod_python to apache on my server at home so I can play with Django. I went through the first two parts of the tutorial and it was pretty cool (using the dev server). Under mod_python the paths are screwy - I must have misconfigured it. Will go back to it later! Oh - the built in dynamic admin functionality is just awesome. It takes so much crap work out of putting up the back end of a site - forms, fields, validation etc etc (all the tedious stuff)