XMPP Scripting with Sluift

Did you ever want to find out what XMPP clients people in your contact list are using? Do you want to migrate your contact list from one server to another, but don’t want to provide your password to some on-line service to do that? Do you have some XMPP-related task you quickly want to write a script for, but don’t want to deal with complex asynchronous APIs? Well, Sluift may be just the thing you are looking for!

Sluift is a Lua-based script layer on top of the Swiften XMPP library. It provides a simple API to do common XMPP tasks, either interactively (through an XMPP console), or by running a script in batch mode. In this post, we’ll go through some examples of what you can already do with Sluift today.

Continue reading post

More squishy data

After Tobias Markmann told me that he was running into resource limitations with a Swiften-based tool for testing server load, I decided to do a small experiment myself. I created a small benchmarking tool, and ran it through the memory allocation profiler from Apple’s Instruments. It turned out that the combination of TLS and ZLib compression (aka “squishy data”) was causing a much higher memory usage than I would have expected.

Continue reading post

Beautiful (XMPP) Testing

O’Reilly recently released the book Beautiful Testing, a collection of essays about testing and QA in general. As I mentioned earlier, I wrote an article in that book on (unit) testing XMPP protocols, using Swift as a motivating example. Since the book’s scope may (oddly enough) not always be as interesting for developers in general, I released my article under a Creative Commons Attribution license (thanks to the good folks from O’Reilly for encouraging us to do this), which you can find here (or directly from my Git repository). The original excerpt from the book (including the index, list of biographies, and all the fancy artwork) is also available for download under the same license.

I of course encourage you to buy either the PDF or dead tree version of the book, as all the proceeds of the book go to charity.

“XMPP: The Definitive Guide” Code Examples

Although the primary focus of XMPP: The Definitive Guide is explaning the XMPP protocol and all its extensions through text and illustrations, we also included a few Python code examples to help people get started with implementing their own ideas. In fact, we devoted a whole chapter to building an XMPP application, starting out with a simple bot implementation, but gradually extending the application into a full server component. For people who want to try this out for themselves, we’re releasing the source code of all code examples, including a simple echo bot, and different variants of the CheshiR microblogging platform XMPP interface.

Continue reading post

“Beautiful Testing” XMPP Chapter

Adam Goucher and Tim Riley (Director of QA at Mozilla) announced a few months ago that they are putting together a Beautiful Testing book for O’Reilly. I took the opportunity to write a chapter about testing in the context of XMPP (more specifically, about testing protocol implementations in Swift), and just submitted the final draft for technical review. The book is expected to be released this August.

Continue reading post

Final revision of the XMPP book submitted

After a few weeks of heavy labour and long nights, Peter, Kevin, and I just submitted the final revision of “XMPP: The Definitive Guide” to the folks at O’Reilly. All the feedback from our (thorough) reviewers has been processed, we added quite a few extra bits and clarifications (58 pages to be exact), polished the whole thing up, and went through the resulting manuscript with a fine toothed comb. We hope the people who will read this book will be as satisfied with the end result as we are. If all goes according to plan, the book should roll out of the presses in about 2 months. In the mean time, you can expect an update to the on-line rough cut version of the book in the next couple of days.

Trying out Git

A while ago, the Psi development team switched from Darcs to Subversion for version control, because the Darcs pros (distributed, extremely simple and elegant) did not compensate for the cons any longer (slowness, non-scalability, ‘infinite’ merges, lack of community and tools, …). Our development was pretty central anyway at that time, so we decided that Subversion was good enough. However, we started to miss local commits more than we thought we would, and some of us are working on their own forks, which makes Subversion a suboptimal choice. We are therefore currently trying out Git as a replacement, which should bring us all the good stuff from Darcs, combined with the speed and portability of Subversion. Note that during the experiment, we will not be updating our Subversion branch any more (which will soon cause breakage, since Subversion automatically updates changes to the external Iris repository).

Continue reading post

We're writing an XMPP book

I’m excited to announce that Peter, Kevin, and I recently got the green light from O'Reilly to start writing a book about Jabber/XMPP. The book will be targeted at a diverse public: on one hand, people who want to get acquainted with XMPP and will get an introduction and a general overview of XMPP, its workings, and its possibilities. On the other hand,  software engineers who want to integrate XMPP into their products will get a guide to implementing different use cases of XMPP through a series of different developer stories. The book is expected to be available in 2009, so start making some room on your bookshelf!

Going Agile with Google Summer of Code

Although Psi has had a fair number of succesful Google Summer of Code projects so far, we have experienced some failures as well: the summer before last, 3 out of 6 projects didn’t make the final deadline. A project’s failure was typically due to not having anything really usable at the end of the summer, regardless of the good work that was done during the past months. To reduce the risk of such surprises, I decided to take an Agile Development approach for this year's ‘Roster improvement’ project.

Continue reading post

Mimicking Jaiku with Psi

The day before yesterday, Peter Saint-Andre sent out a couple of Jaiku invites to all Jabber Google Summer of Code students and their mentors, including me. Never having looked at microblogging before, I toyed around with it a bit, and it quickly reminded me that I still had something on my Psi wish-list for a while now: a flat, live log of all Jabber events in your network. Since I had a long weekend, I quickly coded up a prototype, and hooked it into Psi.

Continue reading post

Improving Psi's roster

For a while now, Psi users have been requesting several changes and additions to the roster (or ‘contact list’). These requests include grouping contacts into meta-contacts, nested roster groups, and displaying user avatars in the roster. We have been postponing all these changes to the roster as much as possible, because none of us wanted to touch the roster code, for reasons I’ll explain below. This year, Psi is fortunate enough to have Adam Czachorowski (aka Gislan), a student from the Google Summer of Code, to work on roster improvements.

Continue reading post

Introducing Greem

After a short hiatus, I finally resumed work on my new Jabber/XMPP client project, which I christened ‘Greem’. The main goal of the project is to create a mobile Jabber/XMPP client for the Qtopia platform. The nice thing about Qtopia is that its target audience keeps on expanding: besides running on the GreenPhone (of which Trolltech was kind enough to provide me with one), Qtopia has recently been ported to the Neo 1973 (OpenMoko), and even Windows CE and Windows Mobile. In this post, I briefly describe what the expectations and the goals are for Greem, and how Psi fits into the picture.

Continue reading post

Qtopia Greenphone Grant

A month or 2 ago, I applied for the Qtopia Greenphone Innovation Grant Program, an initiative from TrollTech to promote the development of applications for their Linux-based Qtopia Greenphone. I probably won’t surprise anyone by saying that I sent in a proposal about writing a good, cross-platform, mobile Jabber/XMPP client. Anyway, I was very excited to receive a mail from TrollTech yesterday, stating that my proposal was accepted by their review panel! As an applicant, I will be receiving a shiny new Greenphone, together with a Qtopia SDK to develop against. Deadline for submitting my application: October 31st. Let the coding begin.

Customizable XEP-0076 implementation

It is time for us to be honest: the reason Psi has not had a release in the past year and a half is because we have secretly been working on one of the most controversial and least implemented features in the Jabber world: XEP-0076 (Malicious Stanzas). We have allocated two full-time developers for achieving this: Machekku has done the groundbreaking work, implementing the main processing loop, statistic gathering, and user interfaces for this type of stanzas, whereas I have been concentrating mostly on backend issues. Although our work is still in a highly experimental stage, we decided to release the full source code in order to get useful feedback from the community. Besides a Psi implementation, we also provide an Openfire server-side implementation for malicious stanza tagging as an extension of the content filter, targeted at fixing non-XEP-0076-compliant behavior of entities.

Continue reading post

Revamped account dialogs

We recently made Psi’s dialogs related to account creation, registration, and modification a bit easier to use. The account registration process is now a wizard-like dialog, where a user first needs to select a server (from a centralized on-line list), and then needs to provide the information requested by the server (according to XEP-0077: In-Band Registration). We have also introduced new default settings that should avoid the need to tweak any account settings when adding an existing account. This means that you should be able to log into any XMPP server (including Google Talk) just by entering your JID and password. Some more details and screenshots below.

Continue reading post


Kev and I will be attending FOSDEM on February 24th and 25th, and we’ll be joining other XMPP developers at the XMPP Interop Event and DevCon the day after. So, if you want to meet up, or have some outstanding protocol issues with Psi, now is a good time to tell us :)

Receiving Google Talk files with libjingle

As Google released a new version of libjingle last friday, I decided to experiment with their file transfer implementation in Psi this weekend. I added some code that allows someone to accept files sent by Google Talk, and tested it with a few Google Talk contacts, with a perfect success rate (so far). For the brave who wish to experiment with this, see the instructions below. Note that this code is unofficial, untested, only uses a very rudimentary UI (there is no integration with the existing file transfers), still suffers from at least two bugs (on Mac OS X: one that segfaults when receiving images, and one where CPU skyrockets to 100% after a while, which we had with older libjingles as well), and has no support from the Psi developers whatsoever. Comments should therefore go directly to me.

Continue reading post

Psi Subversion repository

As many of you know, Psi has been using Darcs for version controlling since the development of 0.10. However, although Darcs works on most platforms, some people have been experiencing problems with getting Darcs to work for them. Because of this, and some other inconveniences (not being able to checkout subdirectories of the development tree, long checkout times, the lack of a scalable web-interface, …), we have decided to create a Subversion mirror our development repository (using Tailor).

Continue reading post


I have to confess, i’m kind of a fan of very simple Jabber clients that do not overwhelm you with menuitems, toolbars, and options. This is why i like clients such as Google Talk and iChat so much. I always hoped Psi could one day be such a simple client, but the fact is that we have a very broad userbase, of which many are power-users that do not want to give up any option or feature whatsoever. I never really understood why people needed to send plain messages (instead of chats) in IM, why there is such a thing as ‘Extended Away’ and ‘Free for chat’ in a chat system where you have status messages, why someone would want to hide contacts that are ‘Away’, and the list goes on. However, we’ve had many discussions about this before, and the conclusion is always that someone on this planet wants this, so we never touched these (IMO) silly features. Until now …

Continue reading post

Running aMUCk

The Psi feature frenzy continues! With my PhD draft text delivered last week, I finally had some more time over the weekend to implement some neat stuff. And not just any stuff: after many years of being blamed for not keeping up with the times, we finally have support for JEP-0045 (Multi User Chat) in our mainline development branch. Sure, maybe some small aestetic modifications can still be done here and there, but all of the functionality should be there. This means you don’t have any excuse whatsoever to be anti-social anymore, so start joining all the fun chatrooms such as psi@conference.jabber.ru.

Continue reading post

Libjingle for Mac OS X

As Kev announced earlier today, we took the dust of our Libjingle branch of Psi, and merged it into the mainline development branch (mainly to ease code management). It is not unusual to discover things when you go through stuff you haven’t seen in a while, and this time was no different: we found out, much to our surprise, that we actually had a working support for Libjingle on Mac OS X all this time! Support for OS X was written several months ago, by defining a new sound card type in mediastreamer (a third-party framework included in libjingle), making use of the cross-platform PortAudio API to do interaction with the audio hardware. We never tested it, so we assumed it didn’t work. Fortunately, a quick test proved that wrong ! So now we are down to one unsupported platform: Windows. Because PortAudio is cross-platform, it should in theory work on other platforms as well, including Windows. However, we have not had the time to test this theory, so it might require some more work. But I’m sure Kev will keep us posted on that !

Nicks and Pics

Since the beta2 release of Psi a few days ago, some new stuff has hit our development branch. First of all, Psi now has support for avatars. These can be set and displayed using the (current version of the) PEP-based JEP-84 (User Avatar), and there is read-only support of the historical JEP-153 (vCard-based Avatars), supported by such clients as Google Talk, iChat, and the Py* transports. Secondly, the remote controlling interface of Psi has been extended with forwarding of unread messages (thanks to Norman Rasmussen). And finally, events coming from unlisted contacts (such as subscriptions or messages) now include the nickname of the contact through JEP-172 (User Nickname). A consequence of this is that mass-adds from gateways will go a lot smoother. And now we’re really stopping this feature frenzy, we promise !

Standards and Features

There has been a slight shift in the roadmap of Psi. While the only goal of 0.11 was to port it to Qt4 (do i hear a booooring somewhere?), we decided to make 0.11 also fully XMPP-compliant. We therefore merged the current ‘next generation’ branch (which amongst others supports privacy lists) with the mainline development branch. A consequence of this is that other features were pulled into mainline as well (yay!), so it’s not unacceptable to start looking forward to the next release.

Continue reading post

Chat States

A few weeks ago, Tom Germeau from MozChat brought to my attention that recent clients such as Google Talk, Gajim, Pandion (and of course MozChat) are all using JEP-0085 (Chat State Notifications) for doing ‘Contact is typing …’ notifications. There have been many discussions in the past on whether or not to prefer this JEP over the old JEP-0022 (Message Events), which most of the other clients out there implement. This is why Psi didn’t implement chat sates yet. However, we recently decided to put this (cleaner, standards track) JEP into our mainline development version after all.

Continue reading post

Personal Eventing Protocol

Magnus Henoch announced a few days ago that he was working on an ejabberd implementation for JEP-163 (Personal Eventing Protocol), the protocol formerly known as SPPS. I found a bit of spare time today to finish the real (as opposed to pseudo-) PEP implementation in Psi, while Magnus fixed some things on the server side. The implementation isn’t available publically yet (since no server supports it anyway), but interested server developers are of course always welcome to ask for a snapshot.

Continue reading post

Entity Capabilities

Support for JEP-115 (Entity Capabilities) has just hit the Psi mainline development branch. This doesn’t mean much to the user, except that it can display the client version of contacts without the burst of client-version queries each time you connect. However, there is still the overhead of the extra tag in every presence packet sent and received, but stream compression (either through TLS or using JEP-138) should take care of this in the future. JEP-115 also describes a server optimization to reduce traffic, but as far as we know, no server implements this. We hope that if more clients start supporting entity capabilities, the servers will follow as well.

Continue reading post