The main issue, is that I haven’t unset the “joinable” flag, so everybody can create a account. By the way password are sent, spammers doesn’t manage to post stuff in the personnal area.

With 300 users, and only 4 valids, I need to find a way to clean the acl_user folder. Last time I used twill, this works really great, so I decided to use the same stuff for Zope.

Here my little script:

#!/usr/bin/python

LOGIN='foo'
PASSWORD='far'

LOGIN_URL='http://your_server/manage_main'
USER_URL='http://your_server/acl_users/manage_users'
GOOD_USERS = ['jkx','foobar']

import twill

def delUsers():
   twill.commands.go(USER_URL)
   form = twill.commands.showforms()[0]

   usernames = form.possible_items('names:list')
   for u in usernames:
      if u not in GOOD_USERS:
         twill.commands.formvalue(1,'names:list',u)

      twill.commands.showforms()
      twill.commands.submit(3)

def doLogin():
   twill.commands.go(LOGIN_URL)
   forms = twill.commands.showforms()

   twill.commands.formvalue(2,'__ac_name',LOGIN)
   twill.commands.formvalue(2,'__ac_password',PASSWORD)
   twill.commands.submit()

if __name__ =='__main__':
  doLogin()
  delUsers()

This will drop all user account (not the ones in GOOD_USERS) ..

I ‘m already using a good email spam filter: SpamBayes, so I decided to test bayesian filtering for the spam on this blog too.

I decided to give Reverend a try:

from reverend.thomas import Bayes

SPAM_DB='spam.bayes'
guesser = Bayes()

# load the spam DB
try:
    guesser.load(SPAM_DB)
except IOError:
    print "Creating a new spam filter database"
    guesser.save(SPAM_DB)

def train_spam(text):
    guesser.train('spam',text)
    guesser.save(SPAM_DB)

def train_ham(text):
    guesser.train('ham',text)
    guesser.save(SPAM_DB)

# try to guess the spam / ham ratio of a text
def guess(text):
    spam = 0
    ham = 0
    value = guesser.guess(text)
    for o in value:
        if o[0] == 'ham': ham = o[1]
        if o[0] == 'spam': spam = o[1]
    return (ham,spam)

Small, and really simple module no ? The next step, simply add a ‘spam’ and ‘ham’ attributes on your comment post. And add two methods to train the comment as a spam or a ham.. And of course, only display comments which have a good ratio ( >1) ham/spam. This took me about 1 hour to implement…

After a week, of train, this is working very fine, not a single false positive, and it filter every spam since the first trains. As I get around 20 spams post per day, this is quite a good news ;)

Enjoy Bayesian ?

With time, Python really became a powerfull tool for hardware hackers :) That’s perhaps why I like it so much ?

I have to admit that I used Python for quite everything, and that I’ve ever done some strange stuff with it (a IPTV packet scrapper, firewall shapping .. ) but this guys
pushed the limit far beyong from what I’m seeing like usual python use. And the best part: they wrote some powerfull tools ( and use them to “debug skype”), and publish them.

Look at here, you will find a bunch of really backhat python tools written by the Secdev team.

/Enjoy

I decided to look at this a little deeply, and decided to use /dev/rtc to do the trick. Here a little example of the stuff.

from  fcntl import ioctl
import os, time

# some linux kernel header
RTC_IRQP_SET = 1074032652
RTC_PIE_ON =  28677
RTC_PIE_OFF = 28678

# open the realtime clock
f = open ("/dev/rtc", "r")
fd = f.fileno()

# 2048 freq
ioctl(fd,RTC_IRQP_SET,2048)
ioctl(fd,RTC_PIE_ON,0)

t0 = time.time()
interupt = 0

# the read will block until the next interrupt.
while interupt < 1000 :
    os.read(fd,4)  # 4 = size of double
    interupt = interupt + 1

delta = time.time() - t0
print interupt / delta

# drop the scheduler
ioctl(fd,RTC_PIE_OFF,0)

This is quite accurate enought, the only issue is that I’m unable to call the sched_setscheduler() from Python right now. So, if the host isn’t loaded I get the 2048 interrupts.. but it can be less :(

Update:
The sched_setscheduler can be done w/ a simple pyrex extension:

cdef extern from "sched.h":
struct sched_param:
int sched_priority
int sched_setscheduler(int pid, int policy,sched_param  *p)

SCHED_FIFO = 1

def switchRTCPriority(nb):
cdef sched_param sp
sp.sched_priority = nb
sched_setscheduler (0,SCHED_FIFO , &sp);

After this quick Pyrex hack, I now have a accurate realtime interrupt in Python ;)

Now, this is a little reminder for people (like me) which doesn’t read binary op fluently.

# This will off the bits according to value
x &= ~(value) 

# Here a little reverse: This will set bits on according to value
x |= value

Have fun w/ bits :)

Plone 2.0.5 (python 2.3.4)

Total transferred:      239130 bytes
HTML transferred:       234620 bytes
Requests per second:    5.52 [#/sec] (mean)
Time per request:       181.261 [ms] (mean)
Time per request:       181.261 [ms] (mean, across all concurrent requests)
Transfer rate:          128.54 [Kbytes/sec] received

Plone 2.1 (python 2.3.5)

Total transferred:      240740 bytes
HTML transferred:       237440 bytes
Requests per second:    3.54 [#/sec] (mean)
Time per request:       282.476 [ms] (mean)
Time per request:       282.476 [ms] (mean, across all concurrent requests)
Transfer rate:          83.19 [Kbytes/sec] received

Grr, here is two point to read:

• The 2.1 version is really slower look at the transfer rate..
• But this is astomish slow

The bench was done on:

• pure zope httpd (no apache or squid frontend)
• without debug enable
• with default HTTPCache and RAMCache enable (default install setting)
• a dual Xeon at 2.6Ghz with 1Go of RAM ! (I know python only use 1 CPU but !!!)

I first think my setup is buggy, but no.. nobody with a good CMS in stock ?

Anyway, as this become usual, (grrr) there is a bunch of projects that try to clone the rails features. The well know python proverb can be used:
Divide to loose

Here ‘s a little list:

But, how can I send a message to my Gmail Talk in less 10 lines of code. I decided to look at xmpppy .. Ok I get a strange issue w/ secure login ..

As I already use gajim, I decided to look at this code.. Hum, look strange it has the same xmpp module, with the same API, but this one is working.

import xmpp

login = 'Your.Login' # @gmail.com
pwd   = 'YourPassword'

cnx = xmpp.Client('gmail.com')
cnx.connect( server=('talk.google.com',5223) )

cnx.auth(login,pwd, 'botty')

cnx.send( xmpp.Message( "YourFriend@gmail.com" ,"Hello World form Python" ) )

As you can see this is small and simple :) The only issue is to find the right module, and hope the xmpppy will fix that soon.

(In fact gajim team seems to fork the xmpp code due to lack of support. This bug already exists in the xmpppy bug list )