I ‘m already using a good email spam filter: SpamBayes, so I decided to test bayesian filtering for the spam on this blog too.

I decided to give Reverend a try:

from reverend.thomas import Bayes

SPAM_DB='spam.bayes'
guesser = Bayes()

# load the spam DB
try:
    guesser.load(SPAM_DB)
except IOError:
    print "Creating a new spam filter database"
    guesser.save(SPAM_DB)

def train_spam(text):
    guesser.train('spam',text)
    guesser.save(SPAM_DB)

def train_ham(text):
    guesser.train('ham',text)
    guesser.save(SPAM_DB)

# try to guess the spam / ham ratio of a text
def guess(text):
    spam = 0
    ham = 0
    value = guesser.guess(text)
    for o in value:
        if o[0] == 'ham': ham = o[1]
        if o[0] == 'spam': spam = o[1]
    return (ham,spam)

Small, and really simple module no ? The next step, simply add a ‘spam’ and ‘ham’ attributes on your comment post. And add two methods to train the comment as a spam or a ham.. And of course, only display comments which have a good ratio ( >1) ham/spam. This took me about 1 hour to implement…

After a week, of train, this is working very fine, not a single false positive, and it filter every spam since the first trains. As I get around 20 spams post per day, this is quite a good news ;)

Enjoy Bayesian ?

With time, Python really became a powerfull tool for hardware hackers :) That’s perhaps why I like it so much ?

I have to admit that I used Python for quite everything, and that I’ve ever done some strange stuff with it (a IPTV packet scrapper, firewall shapping .. ) but this guys
pushed the limit far beyong from what I’m seeing like usual python use. And the best part: they wrote some powerfull tools ( and use them to “debug skype”), and publish them.

Look at here, you will find a bunch of really backhat python tools written by the Secdev team.

/Enjoy

I decided to look at this a little deeply, and decided to use /dev/rtc to do the trick. Here a little example of the stuff.

from  fcntl import ioctl
import os, time

# some linux kernel header
RTC_IRQP_SET = 1074032652
RTC_PIE_ON =  28677
RTC_PIE_OFF = 28678

# open the realtime clock
f = open ("/dev/rtc", "r")
fd = f.fileno()

# 2048 freq
ioctl(fd,RTC_IRQP_SET,2048)
ioctl(fd,RTC_PIE_ON,0)

t0 = time.time()
interupt = 0

# the read will block until the next interrupt.
while interupt < 1000 :
    os.read(fd,4)  # 4 = size of double
    interupt = interupt + 1

delta = time.time() - t0
print interupt / delta

# drop the scheduler
ioctl(fd,RTC_PIE_OFF,0)

This is quite accurate enought, the only issue is that I’m unable to call the sched_setscheduler() from Python right now. So, if the host isn’t loaded I get the 2048 interrupts.. but it can be less :(

Update:
The sched_setscheduler can be done w/ a simple pyrex extension:

cdef extern from "sched.h":
struct sched_param:
int sched_priority
int sched_setscheduler(int pid, int policy,sched_param  *p)

SCHED_FIFO = 1

def switchRTCPriority(nb):
cdef sched_param sp
sp.sched_priority = nb
sched_setscheduler (0,SCHED_FIFO , &sp);

After this quick Pyrex hack, I now have a accurate realtime interrupt in Python ;)

Now, this is a little reminder for people (like me) which doesn’t read binary op fluently.

# This will off the bits according to value
x &= ~(value) 

# Here a little reverse: This will set bits on according to value
x |= value

Have fun w/ bits :)

Anyway, as this become usual, (grrr) there is a bunch of projects that try to clone the rails features. The well know python proverb can be used:
Divide to loose

Here ‘s a little list:

But, how can I send a message to my Gmail Talk in less 10 lines of code. I decided to look at xmpppy .. Ok I get a strange issue w/ secure login ..

As I already use gajim, I decided to look at this code.. Hum, look strange it has the same xmpp module, with the same API, but this one is working.

import xmpp

login = 'Your.Login' # @gmail.com
pwd   = 'YourPassword'

cnx = xmpp.Client('gmail.com')
cnx.connect( server=('talk.google.com',5223) )

cnx.auth(login,pwd, 'botty')

cnx.send( xmpp.Message( "YourFriend@gmail.com" ,"Hello World form Python" ) )

As you can see this is small and simple :) The only issue is to find the right module, and hope the xmpppy will fix that soon.

(In fact gajim team seems to fork the xmpp code due to lack of support. This bug already exists in the xmpppy bug list )

It looks like pyxfce meet my needs, so I decided to give it a try. The first thing to do is to build the latest svn of xfce. Next step play with the python wrapper.

Update: You don’t need the latest version (svn) of xfce. pyxfce works w/ xfce 4.2.1. The debian package is broken, as rely on a old libxfcegui (4.1)

Update 2: Debian package is fixed right now :)

Here is a little example of what you can do without too much pain. This script wait for every new window, and stack it on the same workspace as previous other windows of the same app. For example, the emacs windows is launched of the workspace ’4′, all other emacs windows will be displayed on this workspace.

import sys
import xfce4
import gtk

black_list = ['xfrun4','Terminal']

def newWinHandler(screen,win):
    name =  win.get_resource_name()
    if name not in black_list:
       for w in screen.get_windows():
           if w != win and w.get_resource_name() == name:
              dest = w.get_workspace()
              win.move_to_workspace( dest )
              break

screen = xfce4.netk.screen_get_default()
screen.connect("window-opened",newWinHandler)
gtk.main()

The major issue of pyxfce is the lack of documentation, you can do a lot of stuffs with it like creating custom xfce applications, but i didn’t find anything about this.

from elementtree import ElementTree

data = open('Kismet-May-22-2005-4.xml','r').read()

detection = ElementTree.XML(data)

for node in detection.getchildren():
try:
print "SSID: " + node.find('SSID').text,
except AttributeError:pass  #hidden SSID
print "BSSID: " +  node.find('BSSID').text,
gps = node.find('gps-info')
print "Lon: " + gps.find('max-lon').text +  " Lat: "  + gps.find('max-lat').text

Really simple, and seems to offer really good speed. I now have another weapon for every day work.