El Tramo | Psi

Trying out Git

2008-08-26T00:00:00+02:00

A while ago, the Psi development team switched from Darcs to Subversion for version control, because the Darcs pros (distributed, extremely simple and elegant) did not compensate for the cons any longer (slowness, non-scalability, ‘infinite’ merges, lack of community and tools, ...). Our development was pretty central anyway at that time, so we decided that Subversion was good enough. However, we started to miss local commits more than we thought we would, and some of us are working on their own forks, which makes Subversion a suboptimal choice. We are therefore currently trying out Git as a replacement, which should bring us all the good stuff from Darcs, combined with the speed and portability of Subversion. Note that during the experiment, we will not be updating our Subversion branch any more (which will soon cause breakage, since Subversion automatically updates changes to the external Iris repository). If you're interested in following our latest developments from our experimental Git tree, simply clone our official repository through the following steps:

git clone git://git.psi-im.org/psi.git
cd psi
git submodule init
git submodule update

To get the latest changes after an initial clone, execute the following commands

git pull
git submodule update

Note that you need Git 1.5 or later in order to be able to do these commands. If you're interested in knowing more about using Git, check out Scott Chacon's excellent Git talk (and accompanying book). For fans of social networking, we also have a mirror of the official Psi repository on GitHub. And while we're on the topic of web interfaces: while waiting for an official web interface to our Git repository, you can check out an experimental one here.

Going Agile with Google Summer of Code

2008-06-26T00:00:00+02:00

Although Psi has had a fair number of succesful Google Summer of Code projects so far, we have experienced some failures as well: the summer before last, 3 out of 6 projects didn't make the final deadline. A project's failure was typically due to not having anything really usable at the end of the summer, regardless of the good work that was done during the past months. To reduce the risk of such surprises, I decided to take an Agile Development approach for this year's ‘Roster improvement’ project. Here are some of the principles of Agile software development (as written down in the Agile Manifesto), and how we apply them to Adam's GSoC project:

The developer frequently delivers working software to the customer. In our case, Adam sends me a working roster every monday, which I can play around with. I then tell him what I do and don't like about it, and he processes my feedback during the next iteration (i.e. the next week). Of course, since the roster is made from scratch, the backend of the roster is completely mocked up during the first weeks. As the project progresses, the backend will become more 'real'.
Working software is the primary measure of progress. This avoids the kind of surprises we had before, where both student and mentor believed everything was going fine during the project, but where nothing could be delivered at the end of the project.
Code is kept as simple as possible: if you don't need something, leave it out. By initially focusing on the roster frontend (while keeping the backend framework mocked up), we ensure that our backend will only be as complex as we actually need it to be. Moreover, we can constantly change our mind about the requirements throughout the whole project without having to throw away the complete backend. This contrasts with the previous attempts we had at reworking the Psi roster, where we started by working out a complex roster backend, but never got to the point where we had something working with this (over-)designed framework, and had to start over all again.
Code is written with testability in mind. In principle, all new code needs to be tested (and preferably even unit tested before it's written), which allows you to quickly refactor while remaining sure that your code still works as before. However, given the very strict time constraints, I'm not sure yet that we're going to apply this Test-driven Development approach (although some claim that the extra time you spend in tests will even pay off in the short term).
Good designs make sure that you can easily adapt your code to changing requirements. For example, a Model-View-Controller-based design seems to lend itself well to the roster, making the (rather complex) roster logic both testable, extensible, and easy to understand.

I'm convinced that these principles will ensure that both Adam and I (and of course all Psi users) will get a succesful result at the end of the project, and that we'll have had a fun time during these 3 months.

PsiCon 2008 @ San Francisco Bay Area, CA

2008-05-16T00:00:00+02:00

This year, we held the first ever International Psi Conference in the San Francisco Bay Area. The event was spread out over 2 days, with different venues to keep things interesting. The conference was a big success, with attendees from different countries (including the Netherlands, Belgium, and the U.S.), and the presence of several big companies. The opening day of the conference took place at 'Aqui Cal-Mex' in Campbell. This day included a keynote talk by Steve Pao from Barracuda Networks, talking about the Barracuda IM solutions and open source. Other interesting discussions included possible opportunities for Psi in the embedded market, and how we could surf the Web 3.0 wave. The second day of the conference was held at 'The Cheesecake Factory' in Palo Alto, and mainly consisted of breakout sessions with Justin Karneges and Jason Kim, where we (amongst others) brainstormed about possible future projects based on Psi, discussed the plans for upcoming releases, and talked about how to streamline the interactions with Psi-based projects. The conference ended with the social event: a guided tour of Mineta San José International Airport. This year's local chair was Justin Karneges, who provided us with the supplies (notebook, pen, ...), eco-friendly transportation, and who carefully picked the venues to provide both good food and a good infrastructure to make the 38ºC (100ºF) heat wave more bearable. Thanks Justin! Translation: Had lunch with some guys from Barracuda, had dinner with Justin and Jason, took a wrong turn and ended up at the airport, wheather was hot, had jetlag this morning, was bored to death, wrote something.

Mimicking Jaiku with Psi

2008-05-02T00:00:00+02:00

The day before yesterday, Peter Saint-Andre sent out a couple of Jaiku invites to all Jabber Google Summer of Code students and their mentors, including me. Never having looked at microblogging before, I toyed around with it a bit, and it quickly reminded me that I still had something on my Psi wish-list for a while now: a flat, live log of all Jabber events in your network. Since I had a long weekend, I quickly coded up a prototype, and hooked it into Psi. The result looks a bit like this:

If you know Jaiku, you'll probably notice that this looks very similar to the Jaiku web interface. Besides status messages, there are all kinds of (extended) presence events from your contacts, such as the currently playing tune or his/her current mood. Groupchat (`channel' in Jaiku) and directed messages are interleaved with the events, and get a hyperlink which, when clicked, opens up the corresponding groupchat or chat dialog. This type of event log allows you to have a good overview of everything that is happening in your Jabber network. And if your log gets cluttered with groupchat events, you can always disable groupchat events (or any other type of event) at the top of the dialog, with a more compact log as a result. Finally, just as with the Jaiku Jabber bot, you can quickly reply to the last event from a certain user at the bottom of the dialog. When will this prototype be production-ready, you ask? Well, I'm actually not planning to invest any more time in it in the near future. The reason is that Aleksey Palazchenko (aka AlekSi) will create a brand new history system for Psi for his Google Summer of Code project. I'm pretty sure his new history system will enable us to get a global live history of events, together with filtering based on type. And if we still need some extra functionality, we could always create a plugin.

Improving Psi's roster

2008-05-01T00:00:00+02:00

For a while now, Psi users have been requesting several changes and additions to the roster (or `contact list'). These requests include grouping contacts into meta-contacts, nested roster groups, and displaying user avatars in the roster. We have been postponing all these changes to the roster as much as possible, because none of us wanted to touch the roster code, for reasons I'll explain below. This year, Psi is fortunate enough to have Adam Czachorowski (aka Gislan), a student from the Google Summer of Code, to work on roster improvements. So far, the major cause of all Psi developers staying clear from doing substantial roster changes is probably the fact that all roster-related classes are very tightly coupled to each other. This has several consequences:

It's hard to get a good understanding of which piece of code does what, since there is no separation of logic between the different classes, and none of them work without the other one.
The code is very fragile: if you change one tiny piece, you might break something completely different. Moreover, it's hard to tell what you broke, since the functionality is spread out across the different classes.
The code is untestable: Since there is no decent separation of the roster logic (i.e. the structure of the roster) and the roster user interface (i.e. how it is shown), it has been impossible so far to create some form of automated tests for the roster code. Because of the tight coupling between the various classes, it is impossible to test each part of the roster functionality in isolation, making unit testing impossible. Given that the roster is the most central part of the Psi user interface, it is actually unacceptable that we have no form of testing whether it (still) functions correctly.
There can be no reuse of any of the roster code in other parts of the UI (such as the list of participants in the MUC dialog), or even for other IM clients that are based on the back-end of Psi.

Because of these fundamental issues, a complete makeover of the roster code is in order. More specifically, we want to have a clear separation of anything that has to do with UI, and the actual logic of the roster. Additionally, we want the (untestable) UI layer to be as thin as possible, pushing as much down to the logic layer as we can. Finally, we want to achieve a full coverage of the logic layer using only unit tests. What Gislan will exactly do in his Google Summer of Code project still has to be worked out in detail. He will be responsible for a major part (if not all) of the roster rewrite, and get some new functionality in there as well (since that will be a breeze with the new code). You can follow his progress on his blog, and get more detailed technical information on the project page for his GSoC project.

Introducing Greem

2007-10-14T00:00:00+02:00

After a short hiatus, I finally resumed work on my new Jabber/XMPP client project, which I christened `Greem'. The main goal of the project is to create a mobile Jabber/XMPP client for the Qtopia platform. The nice thing about Qtopia is that its target audience keeps on expanding: besides running on the GreenPhone (of which Trolltech was kind enough to provide me with one), Qtopia has recently been ported to the Neo 1973 (OpenMoko), and even Windows CE and Windows Mobile. In this post, I briefly describe what the expectations and the goals are for Greem, and how Psi fits into the picture. The goal of Greem is to be a clean, simple Qt/Qtopia client, built completely from scratch, using lessons learned from the Psi project. Since it is my intention to keep Greem as simple and clean as possible, some advanced features from Psi will not be available: multiple accounts, customizations that are only relevant to 1% of the population, ... However, development will be focused on things that Psi also needs in the near future:

Clean, unit testable APIs.
New and improved roster (for meta-contacts, avatars, ...)
Server-side message archiving
Reliable communication
Simplicity

A lot of this will either flow directly back into Psi (which should be possible thanks to the decoupled APIs), or will be used as a guide to refactor certain parts of Psi that need it (e.g. the roster). For the first release(s), not every part of Greem will be written from scratch, though. Although I already have a major part of the low-level XMPP server connection handling coded up, Greem will initially use Iris as its backend for connecting to a server. Anything beyond the connection (i.e. XMPP data structures, stanza handling, ...) will be done outside of Iris. This way, the major focus can be on the front-end and general frameworks, without having to worry about the low-level implementation details (which can typically be a big hassle to get right). In the future, the Iris backend, which is connected through a very thin layer with the front-end, will be replaced by my own implementation of XMPP streams. An update on the current status of the project will follow soon, together with some preliminary screenshots.

Testing Psi

2007-10-01T00:00:00+02:00

While the last bugs are being squeezed out of Psi 0.11's release candidates, work on 0.12 has already begun. One thing I'm excited about as a developer is the fact that we're making the Psi codebase `testable', which has some nice consequences. The first part of our effort to make Psi testable is creating unit tests for most of our classes (using CppUnit), which should provide quick checks that our classes behave (and keep behaving) the way they should. However, the fact that Psi was not designed to be testable from the ground up makes this a non-trivial job, requiring quite some refactoring. This is something that will happen incrementally over time. A side effect of this refactoring will be an increase in modularity, allowing us to use different frontends (e.g. Cocoa) for the same code. The second part we're working on is creating UI tests for parts that are not automatically testable. For this, we are decoupling all our dialogs from the rest of the application, such that we can create a standalone application that allows you to toy with the dialog. This decoupling not only means that it becomes very easy to test dialogs with your own scenarios (without having to start Psi, connect to a server, ...), but also means that the dialogs become completely reusable for other applications. We still have a long way to go until every part of Psi is testable (either automatically or not), but I am very much looking forward to the day where all of our code is nicely decoupled, and we can run make check on every commit to ensure that we didn't break anything silly.

Customizable XEP-0076 implementation

2007-04-01T00:00:00+02:00

It is time for us to be honest: the reason Psi has not had a release in the past year and a half is because we have secretly been working on one of the most controversial and least implemented features in the Jabber world: XEP-0076 (Malicious Stanzas). We have allocated two full-time developers for achieving this: Machekku has done the groundbreaking work, implementing the main processing loop, statistic gathering, and user interfaces for this type of stanzas, whereas I have been concentrating mostly on backend issues. Although our work is still in a highly experimental stage, we decided to release the full source code in order to get useful feedback from the community. Besides a Psi implementation, we also provide an Openfire server-side implementation for malicious stanza tagging as an extension of the content filter, targeted at fixing non-XEP-0076-compliant behavior of entities. You can find Machekku's and my Psi work here and here respectively. Apply these patches to a clean development version of Psi. In order to use this work, please modify the <evil> section of your options.xml accordingly. XEP-0076 statistics can be found in the options dialog, in a separate tab. The patch against the latest development version of Openfire can be found here. This will add an extra checkbox to the content filter pane, allowing messages to be tagged as malicious.

Revamped account dialogs

2007-03-16T00:00:00+01:00

We recently made Psi's dialogs related to account creation, registration, and modification a bit easier to use. The account registration process is now a wizard-like dialog, where a user first needs to select a server (from a centralized on-line list), and then needs to provide the information requested by the server (according to XEP-0077: In-Band Registration). We have also introduced new default settings that should avoid the need to tweak any account settings when adding an existing account. This means that you should be able to log into any XMPP server (including Google Talk) just by entering your JID and password. Some more details and screenshots below. Removing the need to tweak account settings was done by a combination of new 'Encrypt connection' and 'Allow plaintext authentication' options. For 'Encrypt connection', there is a choice of 'Always', 'Never' (mostly used for debugging), 'Automatically' (which uses encryption if TLS support is available), or 'Legacy SSL', where the default option is to encrypt traffic when it is available. For 'Allow plaintext authentication', the options consist of 'Always', 'Never', and 'Over an encrypted connection', the latter being the default. A combination of these 2 defaults makes that you should always be able to connect to a server that is compliant to the XMPP1.0 specification, without allowing anything less secure than what the spec allows. One more thing on our TODO list is to allow importing (self-signed) server certificates. At the moment, most users still change the option to ignore all SSL warnings, which is less than desirable from both the security and the usability point of view. Below are some screenshots of the reworked account dialogs.

Receiving Google Talk files with libjingle

2007-02-04T00:00:00+01:00

As Google released a new version of libjingle last friday, I decided to experiment with their file transfer implementation in Psi this weekend. I added some code that allows someone to accept files sent by Google Talk, and tested it with a few Google Talk contacts, with a perfect success rate (so far). For the brave who wish to experiment with this, see the instructions below. Note that this code is unofficial, untested, only uses a very rudimentary UI (there is no integration with the existing file transfers), still suffers from at least two bugs (on Mac OS X: one that segfaults when receiving images, and one where CPU skyrockets to 100% after a while, which we had with older libjingles as well), and has no support from the Psi developers whatsoever. Comments should therefore go directly to me. To compile Google File Transfer support in Psi, get the latest development version from Subversion. First, use a custom configure script to configure Psi:

./configure-jingle --enable-google-ft

(note that you may need to provide a path to expat). DO NOT enable other Jingle features, as both conflict at the moment. Next, download and build the new version of libjingle by going into the third-party/libjingle.new subdir, and executing the following commands:

make -f Makefile.libjingle qmake make

Finally, go back to the root dir, and build Psi. After the build has finished, your version of Psi should be able to receive files from Google Talk, putting them in ~/googletalk_files (or ~/Desktop/googletalk_files if you are on Mac OS X).