Square Wheel Blog

Zenkaku_Hankaku bug

Thu, 21 Jan 2010 19:34:15 GMT

I've just run into a weird bug on my Gentoo box. After some recent update the grave key stopped working. It did produce tildes when pressed with shift, but without a modifier it produced no output. Analysis with xev showed that the key was recognized as Zenkaku_Hankaku. Some more research revealed that it was due to this line in /usr/share/X11/xkb/keycodes/evdev and /usr/share/X11/xkb/keycodes/xfree86:

alias <HZTG> = <TLDE>;

Posting this just in case somebody else is struggling with the same problem.

After RuPy 2009

Mon, 09 Nov 2009 21:44:16 GMT

I'm back from RuPy 2009 in Poznań. The conference was really good, much better than I anticipated. Unfortunately some talks that seemed interesting were cancelled.

The talk by David Goodger on communities (and puzzles) was really great. Kristian Rother's talk on development in Bioinformatics lab provided some insights on software development processes in groups of non-professional developers, followed with some interesting discussions during coffee breaks.

On the second day, the talk about RabbitMQ was nice, though neccessarily it covered just pretty basic stuff. The lecture on Python packaging ecosystem (by Tarek Ziade) was very informative and useful. Also, Tarek answered questions about customizing distutils to needs of specific projects (I will definitely try them out).

I took a shot at doing a lightning talk during the second session, but screwed it up; should have prepared the talk better.

Besides official talks, the chats during breaks made the conference even better.

Also, thanks to all the nice people at Duszpasterstwo Akademickie (University Ministry) at Order of Saint Dominic for helping Marcin and me with companionship, hot tea and even a tour of Poznań on Sunday evening.

nginx and an undefined variable

Mon, 02 Nov 2009 00:28:08 GMT

Just a couple of days ago I've run into a tricky bug with nginx. One of the virtual servers defined a variable in its configuration (via set keyword). When creating a new virtual host by copying the site file I've removed the set line, but not the usage (fastcgi_param line). This caused a random sequence of bytes to be put into that variable. PHP serving those FCGI requests handled it gracefully, but I spent quite a long time trying to figure out why this variable was garbled.

A toy ORM in PHP

Fri, 30 Oct 2009 20:26:45 GMT

I've released yet another toy project: an attempt to create a PHP ORM. It is hosted on BitBucket

The project grew out of disappointment with Doctrine and its numerous bugs (problems with isset, saving already deleted object etc). It uses mostly well-known patterns: Unit of Work, Identity Map, Mapper and more. Internal methods are unfortunately public due to some limitations of PHP (no package access nor friend classes), with alternatives (using reflection or classkit or much more sophisticated design) felt like unneccessary complication.

There is almost no documentation yet, feel free to drop a line if you are interested in the project.

TwillManager

Thu, 20 Aug 2009 00:04:46 GMT

TwillManager is a side project created at our company (Code Sprinters), written in Python (with CherryPy, Mako, sqlite - and, of course, Twill - scripted headless web browser).

Its main purpose is to monitor web applications - every few seconds (interval can be defined) it runs the script, which can load up a page, verify links, submit forms etc. - and if the script fails, sends out an e-mail notification. The status of all the defined watches is also available through a simple web interface. The application is in very early stage, though usable nonetheless. The code can be downloaded from Github: http://github.com/codesprinters/twillmanager/tree/master.

Credit due to Adam Byrtek for the idea of the application and Krzysztof Walo, together with whom I've implemented this app.

Academia

Mon, 03 Aug 2009 23:35:18 GMT

Two interesting posts (rants?) about current status of Computer Science in the academic world: Why I am Not a Professor and What's wrong with CS research. I have to agree with most points in the first one (while not with the second one).

I've also witnessed, at my University, the practice of scaling/normalizing exam results, the inflation of grades – and sometimes poor quality of the courses themselves. So... any reason to stay in the academia – except for the possibility to do the few courses I didn't manage to do earlier due to sharing my time between studying and working at Code Sprinters?

Edit (2009-08-22):

Also, http://www.phdcomics.com/comics.php?f=1215

Routes in PHP.

Wed, 22 Jul 2009 00:10:32 GMT

Yet another small piece of code: implementation of simple route mechanism in PHP: Bitbucket repository.

It doesn't try to be a dispatch mechanism - just returns the list of matching variables (or null if no match) and allows url generation. Couple it with whatever way of finding controllers (classes, functions) you want.

Screw the anonymous

Fri, 19 Jun 2009 23:53:52 GMT

Yet another post not related to programming (I'll try to post something about programming soon).

A good rule of thumb:

Don't send anonymous messages; regard authors of anonymous letters as either ill-mannered or cowards.

Recently I received two letters (the first one is more a manifesto I guess...); both sent by a group of people dissatisfied with the way one of local organizations is governed. It's nice that those people care about our little society... but I believe those letters should be signed with real names, not with vague term "group of concerned people".

I understand there are some situations where protecting one's identity is necessary - eg. corporate whistleblowers or witnesses of some crimes (where criminals could get their revenge on them). But in case of local politics? In my opinion - no way. Similarly, not in discussion on the Internet. I believe (you might disagree with me) that if you have the courage to call other people idiots or morons then you should have the guts to show your real name. Assume some responsibility for what you write.

And yes, Anonymous, this includes you.

Post Scriptum:

OK, this is a kind of rant. I know. And I know that a couple of years ago I was posting on some messageboards with shortened nicks not containing my full name. Guess I qualified in "ill-mannered" crowd then.

Wielkości względne (in Polish)

Tue, 02 Jun 2009 14:42:10 GMT

W kr�tkiej przerwie w pisaniu pracy magisterskiej...

Gazeta Wyborcza pisze (O płacę minimalną na noże):

Płaca minimalna osiągnie połowę płacy średniej za 14 lat - zakłada Ministerstwo Pracy.

Przeformułujmy nieco to zdanie:

Ministerstwo Pracy ostrzega: za 14 lat średnia płaca spadnie do dwukrotności płacy minimalnej.

I na tym polega urok wielkości względnych.

Primitive captcha added.

Fri, 29 May 2009 14:57:30 GMT

I've just had to add a primitive captcha to this blog. Turns out that even though only a few people read it spambots try to add advertising comments to the articles. Hope this fixes the problem - if not I'll have to add ReCaptcha here.

Releasing "onion"

Tue, 26 May 2009 19:19:29 GMT

As mentioned in the first post and on my old Wordpress blog I'm doing some research into web applications for my thesis — and in the process I'm writing my own framework. Well, the part that is not related to my thesis (dealing with continuations) was just made public. You can read the API documentation and download it from http://squarewheel.pl/onion/.

The name? Well, the framwork follows a layered architecture. And onions have layers. Just like ogres.

The basic idea was to try to get a framework as minimalistic as possible to avoid it getting in my way when doing real thesis-related work. It's almost bare WSGI, but instead of having each component in the chain accept (start_response, environ) tuple it accepts WebOb Request object and returns Response. It also contains simple dependency lookup code to help with decoupling different application components.

I can't publish the part that is directly related to my thesis (and the code is almost finished now :) ) until I finish the thesis and resolve some issues with university holding copyright on it.

By the way — this blog is powered by that code (minus the thesis part), so it's enough to prove that it's possible to write an application in that framework.

Guessing is bad (sometimes)

Tue, 07 Apr 2009 22:00:08 GMT

“Be strict with what you generate, be permissive with what you accept” is a quite popular opinion when it comes to input/output from libraries, network protocols etc. But is it such a good idea? Because of it we are stuck with web as it is now, with bad markup everywhere – because it works (mostly). And yet even more problems arise.

A week ago I was hit by similar situation, but at a library API level. Zend_Date constructor accepts string representation of date and a format specifier (optional). Without explicitly specified format, it tries to guess the correct one.

Thus it interpreted '2009-03-31' correctly as 31st March. But the very next day unit tests for one piece of code broke. Zend_Date interpreted '2009-04-01' as 4th January instead of 1st April.

To quote the Zen of Python:

In the face of ambiguity, refuse the temptation to guess.

A little known tool: newgrp

Thu, 02 Apr 2009 14:43:38 GMT

Documentation of various linux programs often states that after adding user to some unix group you need to re-login in order to use new privileges. Although this is definitely the easiest option, it is still possible to enhance user's groupset without re-logging.

After adding user to the group you can use newgrp group_name. This is an old tool (from the time when unix users could use privileges of only one group at a time) that changes the current group for user. Additionally, it tries to add the group to the current groupset for the user.

For example:

$ id
uid=1000(pawel) gid=1000(pawel)
grupy=6(disk),10(wheel),11(floppy),18(audio),
19(cdrom),27(video),80(cdrw),85(usb),100(users),
443(plugdev),1000(pawel),1004(www),1005(ssmtp),
1006(vboxusers),1007(kvm)

$ newgrp wireshark
$ id
uid=1000(pawel) gid=1008(wireshark)
grupy=6(disk),10(wheel),11(floppy),18(audio),
19(cdrom),27(video),80(cdrw),85(usb),100(users),
443(plugdev),1000(pawel),1004(www),1005(ssmtp),
1006(vboxusers),1007(kvm),1008(wireshark)

Unfortunately, the current group changed. To have it back to user's default group just run newgrp again:

$ newgrp
$ id
uid=1000(pawel) gid=1000(pawel)
grupy=6(disk),10(wheel),11(floppy),18(audio),
19(cdrom),27(video),80(cdrw),85(usb),100(users),
443(plugdev),1000(pawel),1004(www),1005(ssmtp),
1006(vboxusers),1007(kvm),1008(wireshark)

This only affects the current shell, but should be enough to just run a single program (like wireshark or some cd burninig software like k3b).

Oh, and just plain su - mylogin would also work - but it asks for a password :)

Playing with design: Object-oriented HTML helpers

Sat, 21 Mar 2009 17:27:04 GMT

HTML helpers are simple functions, that generate HTML fragments. They were made popular by Ruby on Rails (eg. link_to helper). Similarly they were adopted in Pylons as Webhelpers.

In theory helpers should be just simple functions – the output should depend just on arguments to the function. This works well for most cases, but sometimes there is a need to reach for values provided by other classes – eg. it would be nice if form-generating helper could obtain the value of CSRF-protecting field from the object that provides such protection, or access the URL-generating object (eg. in a link() helper)

One could just avoid those situations by forcing user to supply those arguments – but it can become inconvenient at some point. Another solution would be to just call functions that provide those values (URL generator or CSRF token generator) directly, but this would tie helpers to specific objects/implementations (what if user switches to another URL scheme and wants to use his own generator?).

In OOP world standard practice for such situations is to use either dependency injection or dependency lookup – but when using plain functions a new problem arises – how to inject dependency to a function? In python this is doable, one could just inject it to a module-global variable. In the toy framework I'm writing now I wanted to avoid this, because it prevents sharing the same module between two applications running in the same interpreter. I also wanted to avoid thread_local variables as much as possible.

The obvious solution I used is to make those helpers methods of some object – it is then possible to inject dependencies into that object. My current implementation has a HelperSet class that has a handle to dependency-lookup code and an abstract Helper class. HelperSet is constructed during application initialization and various helpers are added to it – depending which of them programmer wants to use:

helpers = HelperSet(config)
helpers.add_helper(FormsHelper)
helpers.add_helper(ListsHelper)

That object is later injected into view and available to templates:

${h.forms.textarea_obj(post_form['body'], id='post-body', rows=40, style='width:100%')}

${h.forms.submit('submit', 'Save')}

A call to add_helper method constructs a new instance of given helper class and exposes it through an attribute on HelperSet object. The constructed instance also gets a reference to the set so it can use it's handle to dependencies registry.

Deprecating sliding_sessions

Wed, 18 Feb 2009 18:13:50 GMT

As Alberto pointed out in one of the posts on my old wordpress blog, my sliding_sessions plugin will not work with Rails 2.3.

After short investigation it turns out the plugin is now completely obsolete and replaced with functionality in Rails itself.

If you need to set expiration period for sessions through all controllers in your application, simply add the following option to your config/intializers/session_store.rb file:

:expire_after => 60.minutes

If you need to set different expiration time in different controllers or actions, use the following code in action or some before_filter:

request.session_options = request.session_options.dup
request.session_options[:expire_after] = 5.minutes
request.session_options.freeze

Duplication of the hash is needed only because it is already frozen at that point, even though modification of at least :expire_after is possible and works flawlessly.

The old plugin (for Rails <= 2.2) is here: sliding_sessions.zip

Some problems with MySQL and collation

Thu, 12 Feb 2009 19:10:30 GMT

Just had a minor "WTF" moment with MySQL, related to collation. It turns out that in the default unicode collation (utf8_general_ci) accents are ignored (but not all of them). Yes, I should have checked in the documentation.

This is a problem for us, as UNIQUE indices do not allow two different words, because mysql consider them equal. This means we need to use utf8_bin — utf8_polish_ci won't work for the task.

Here are samples of some queries in different collations:

mysql> SET @@collation_connection = 'utf8_general_ci';
Query OK, 0 rows affected (0.00 sec)

mysql> SELECT "Może" = "Moze"; -- Polish Ż is considered equal to Z
+------------------+
| "Może" = "Moze"  |
+------------------+
|                1 |
+------------------+
1 row in set (0.00 sec)

mysql> SELECT "Ławka" = "Lawka"; -- But Ł is different from L
+--------------------+
| "Ławka" = "Lawka"  |
+--------------------+
|                  0 |
+--------------------+
1 row in set (0.00 sec)

mysql> SET @@collation_connection = 'utf8_unicode_ci';
Query OK, 0 rows affected (0.00 sec)

mysql> SELECT "Może" = "Moze";
+------------------+
| "Może" = "Moze"  |
+------------------+
|                1 |
+------------------+
1 row in set (0.00 sec)

mysql> SELECT "Ławka" = "Lawka";
+--------------------+
| "Ławka" = "Lawka"  |
+--------------------+
|                  0 |
+--------------------+
1 row in set (0.00 sec)

mysql> SET @@collation_connection = 'utf8_bin';
Query OK, 0 rows affected (0.00 sec)

mysql> SELECT "Ławka" = "Lawka";
+--------------------+
| "Ławka" = "Lawka"  |
+--------------------+
|                  0 |
+--------------------+
1 row in set (0.00 sec)

mysql> SELECT "Może" = "Moze";
+------------------+
| "Może" = "Moze"  |
+------------------+
|                0 |
+------------------+
1 row in set (0.00 sec)

mysql> SET @@collation_connection = 'utf8_polish_ci';
Query OK, 0 rows affected (0.00 sec)

mysql> SELECT "Ławka" = "Lawka";
+--------------------+
| "Ławka" = "Lawka"  |
+--------------------+
|                  0 |
+--------------------+
1 row in set (0.00 sec)

mysql> SELECT "Może" = "Moze";
+------------------+
| "Może" = "Moze"  |
+------------------+
|                0 |
+------------------+
1 row in set (0.00 sec)

So now it's time to change most VARCHARS in about 100 tables.

A new blog!

Fri, 30 Jan 2009 22:38:20 GMT

The "Square Wheel blog", previously available at Wordpress has moved to a new site – http://squarewheel.pl

Reasons? Well, first of all, I'm developing a new python web framework (and yes, I know there are hundreds of those already) for my Master's thesis. I'll probably publish the part not related to my research soon, and the part directly related to the thesis as soon as possible (probably six months after graduation). And when writing a framework I need an application to use it, just to make sure it works. Well, this blog is the first attempt at such application, to test the basic part of the framework (not the part directly related to my research topic).

Moreover on wordpress.com I couldn't host .zip files, so I had to put them on my university web server. This blog will not have such limitation.

I know this blog is still in it's infancy – no RSS feed, ugly UI and so on. I hope it will improve. Also, I'll try to post more regularly.

Edit:

RSS Feed added — experimental version at http://squarewheel.pl/posts/feed