I’m fed up with booting my Linux-based smartphones like desktop-systems. Two major developments will help me accomplish enormous improvements in boot speed:

• devtmpfs — kernel support for the /dev file system
• dbus system activation — on-demand launching of dbus-based services

I’m going to carry out the following two tasks in OE:

1. Writing fso-boot, a small executable written in C, which mounts the filesystems, brings up DBus and (optionally) launches X11
2. Setting fso-boot as new init process, that way you still have sysvinit and udev in your root file system, but they’re not active unless explicitly asked for

I’ll do that for the freesmartphone.org adaptation for the HTC Dream (T-Mobile G1, Google ADP-1), which I’m running on 2.6.32 (necessary for devtmpfs) — stay tuned for the first benchmarks.

As mentioned, the freesmartphone.org team and community has taken the challenge to put the FSO stack on the Palm Pre which is out next month. The goal is to manage a voice call with the FSO stack within four weeks.

The idea behind this is a very important one. With only the Openmoko FreeRunner as a platform, the FSO stack is doomed into oblivion sooner or later, since its a very limited hardware platform — in quantity, but considering the closed alternatives also in quality. Hence, we need to proof that FSO can run on current, competitive hardware — to embrace companies that want to adopt FSO in their niche.

The Palm Pre is currently our major hope — all other hardware being either too closed (yes, this includes the Nokia N900) or already outdated.

Something wonderful has happened! Jürg Billeter — mastermind of Vala — pushed support for server-side async dbus into Vala. I hope I didn’t annoy him too much (having continuesly pestered for almost a year now), but the net effect is that we can now continue working on fsogsmd, the Vala implementation of our dbus GSM server (see http://docs.freesmartphone.org for an overview of the API). Yay!

I’m on my way to LinuxTag 2009. Instead of a “real booth” like last year, we settled on a developer table in the hacking area — there we can present our Linux on mobile projects such as

• GPE
• FSO
• OpenEZX
• OpenMoko
• Gnufiish

in a more relaxed way — giving room to dive into some technical issues, when interested folks come around.

Find me there, if you’re interested in any of the aforementioned projects. I’ll be there until Friday afternoon.

I felt it’s time to recap the stuff that kept me busy the last months and give you an overview over the achievements planned for this year — always focusing the free software movement, of course.

freesmartphone.org

Let’s start with the major project I’ve been working on, the freesmartphone.org project, funded by Openmoko, Inc. FSO grows, and it grows in the right directions. We get more API customers — notably the SHR project and the Paroli project — and refine our API and the reference implementation. The 5th milestone has just been released and apart from a major foobar with read-only partitions, it’s pretty good. We are going to fix this OE-inheritance and release a milestone 5.1 in a couple of days.

fso-abyss (GSM 07.10 Multiplexing)

For some modems — e.g. the TI Calypso (see my previous post on ogsmd and its modems) — until now we have relied on pyneo’s gsm0710muxd. Over the last weeks we found some severe problems (race conditions, buffer overflows) with this though, so I thought I have a shot at developing my own GSM 07.10 Multiplexer.

The result is called fso-abyss and is — as with all our software — available at git.freesmartphone.org under a free software license. The major difference to gsm0710muxd is the architecture (and maintainability). While gsm0710muxd combines talking to the serial ports, the pty’s, handling dbus queries, and doing modem specific things, fso-abyss went a different route.

At the heart there is a minimal protocol engine implementing GSM 07.10. Since there was already something available in Qtopia — even nicely seperated without any external dependencies — I took that one and factored it out in a dedicated project called libgsm0710 (available in git as well). The idea here is that different interest groups can collaborate on getting the protocol engine right, since not everyone wants a DBus frontend such as implemented in fso-abyss. The next step was writing a VAPI file for glueing the protocol engine to Vala (more about that one in a bit), which has been used to develop the upper layers of fso-abyss.

Last but not least, there was the pty implementation, the serial port communications abstraction, and finally the dbus server. The DBus API originally designed in cooperation with pyneo has been enhanced to feature the additional features (only) present in fso-abyss. Apart from the architecture, fso-abyss also can handle virtual serial port signalling, 07.10 test commands, automatic session handling, has a wakeup service, and more. Next up is adding support for the Cinterion mc75i which has some proprietary extensions to GSM 07.10 Basic Multiplexing.

dbus-hlid (DBus High Level Introspection Daemon

Modern DBus APIs are pretty dynamic, i.e. objects can come and go at any time. Depending on the hardware, you may find more or less objects of a certain kind. You can now add infrastructure to query the objects (essentially a duplication of what DBus should provide), or just rely on the existing DBus introspection API. Unfortunately this API is missing some critical features to make it really usable, such as querying objects that implement a certain interface.

So I took the plunge and factored this out of the freesmartphone.org frameworkd, since it has broader use. This is the API for it (as introspected by mdbus):

root@om-gta02:~# mdbus -s org.freesmartphone.DBus /org/freesmartphone/DBus
[METHOD] org.freesmartphone.DBus.ListBusNames() -> ( as:result )
[METHOD] org.freesmartphone.DBus.ListObjectPaths( s:busname ) -> ( ao:result )
[METHOD] org.freesmartphone.DBus.ListObjectsByInterface( s:busname, s:iface ) -> ( ao:result )


Here are examples of how you can use it (demonstrated within a Python shell):

>>> hlid.ListBusNames()
[ 'org.freedesktop.DBus',
'org.freesmartphone.omuxerd',
':1.21',
'org.bluez',
'org.tichy.launcher',
':1.13',
':1.0',
'org.freesmartphone.frameworkd',
':1.14',
':1.1',
':1.2',
':1.3',
':1.4',
'org.freesmartphone.ogsmd',
':1.6',
'org.freesmartphone.DBus']

>>> hlid.ListObjectPaths("org.freesmartphone.ogsmd")
['/org/freesmartphone/GSM/Device', '/org/freesmartphone/GSM/Server']

>>> hlid.ListObjectPaths("org.freesmartphone.odeviced")
[ '/org/freesmartphone/Device/Audio',
'/org/freesmartphone/Device/CPU',
'/org/freesmartphone/Device/Display',
'/org/freesmartphone/Device/Display/0',
'/org/freesmartphone/Device/Display/gta02_bl',
'/org/freesmartphone/Device/IdleNotifier/0',
'/org/freesmartphone/Device/Info',
'/org/freesmartphone/Device/Input',
'/org/freesmartphone/Device/LED/gta02_aux_red',
'/org/freesmartphone/Device/LED/gta02_power_blue',
'/org/freesmartphone/Device/LED/gta02_power_orange',
'/org/freesmartphone/Device/LED/neo1973_vibrator',
'/org/freesmartphone/Device/PowerControl/Bluetooth',
'/org/freesmartphone/Device/PowerControl/UsbHost',
'/org/freesmartphone/Device/PowerControl/WiFi',
'/org/freesmartphone/Device/PowerSupply/ac',
'/org/freesmartphone/Device/PowerSupply/adapter',
'/org/freesmartphone/Device/PowerSupply/apm',
'/org/freesmartphone/Device/PowerSupply/battery',
'/org/freesmartphone/Device/PowerSupply/usb',
'/org/freesmartphone/Device/RealTimeClock/0',
'/org/freesmartphone/Device/RealTimeClock/rtc0']

>>> hlid.ListObjectsByInterface("org.freesmartphone.odeviced", "org.freesmartphone.Device.LED")
[ '/org/freesmartphone/Device/LED/gta02_aux_red',
'/org/freesmartphone/Device/LED/gta02_power_blue',
'/org/freesmartphone/Device/LED/gta02_power_orange',
'/org/freesmartphone/Device/LED/neo1973_vibrator']


fso-monitord

While working on implementing GSM time(zone) support for ogsmd, we found we had too few samples, especially since time(zone) information are only sent by few providers all over the world. Moreoever, we missed a generic means to record all the data the frameworkd is sending out via its signals, such as:

• Usage statistics
• Location Updates
• Diagnostic Data

To support this (and more), we came up with fso-monitord, which is available from git as well. fso-monitord logs its data to a flat file format that you can send to us to improve our databases or for debugging. We also figured this would be the best place to add a generic frameworkd watchdog — monitoring all fso components — shutting down or restarting components as necessary and also logging incidents such as API violations.

What’s next in FSO?

For milestone 5.5 (due end of march), we have two major features on the roadmap, namely bluetooth networking (headset profile) and extended PIM support. Milestone 6 will then sport full-fledged networking.

Beyond milestone 6 — apart from one major thing, which I’ll cover in a second — we only have some rough plans, such as revamping or refining the subsystems we’re not perfectly happy with (oeventsd and opreferencesd come to mind). Also, alsa audio scenario handling is broken by design, but this is something we have to take up with upstream.

The freesmartphone.org reference implementation has been progressing incredibly fast. This is partly due to choosing Python as the implementation language (which has been a wise choice) of our DBus APIs. Now you all know that although I truely love Python (I even wrote a book about it) and try to use it everywhere it fits, I’m very well aware that for the future of the freesmartphone.org project, it might be important to come up with a frameworkd reimplementation in a compiled language — to reduce the footprint and squeak every possible bit of performance out of the (embedded) system.

This is why I have decided to encourage a second reference implementation. This one will be written in Vala (I might have mentioned it before, did I?) which is an incredible combination of elegance and performance, featuring a complete lack of any runtime penalties and additional dependencies. It’s simply amazing and I’m seriously thinking about writing an introductionary book about Vala later this year.

Anyways, back to the topic, the first bits of this Vala implementation has landed in the freesmartphone.org git in the form of the very successful GSoC project odeviced, written by Sudarshan S. Stay tuned for some amazing FSO runtime speedups coming in autumn and winter this year to your device.

XeTex

Next to writing software for the freesmartphone.org project, I also found some time to pick up working with my favourite writing tool LyX. LyX, which could be described as a LaTeX frontend, nowadays features integration with the new LaTeX variant XeTex. In contrast to other incarnations such as pdfLaTeX, XeTeX can utilize system fonts such as AAT or OpenType, which are the latest technology in computer-assisted typesetting.

I can now use my “corporate” fonts FF Meta and FF Meta Serif from LyX — amazing!

Conferences

Although still working on cutting down my travelling, I can’t miss some conferences this year. I managed to skip FOSDEM, which made me a bit sad, but I’ll be compensated by attending

• Chemnitzer Linux-Tage
• FOSS Aalborg
• OpenExpo 2009
• LinuxTag 2009

and possible some more… This year my main topics will be OpenEmbedded and freesmartphone.org — both dedicated to reducing the fragmentation of Linux-based embedded systems and to ease writing software for mobile devices running free and open source software. I hope we’ll bump into each other at one of these occasions.

Stay tuned!

Although traditionally the Chaos Computer Congress’ schedule is slightly suboptimal for me (12/26th is my birthday), I’m going to be in Berlin from 12/28th to 12/30th and will visit CCC on the 3th day (12/29th). I’m going to attend Harald’s talk about GSM base stations, so if you want to talk to me, just pick me up afterwards.

好久不見! Three weeks passed within a blink. Last sunday, we smoothly landed in Frankfurt/Main after 14 hours of a calm flight. 4/5 of the Openmoko Framework Team (while Stefan was on vacation in .au and missed all the fun) met in Taipei to tackle some outstanding issues and synchronize with the plans for the next major Openmoko release.

For a start, please refer to the blog postings of Charlie and Daniel, who mentioned some of the things we did in detail. Let me cover the current status and what we are going to work on in the remainder of this year and then step back and talk a bit about the meaning of all this. Right now the framework offers you support for the following tasks, everything accessible via consistent and convenient DBus interfaces:

• Device control (Backlight, Peripheral Power, Real Time Clock, USB, Switches, Buttons, Audio) covers the basics for embedded systems. Thanks to kernel 2.6, most of these interfaces are working on all devices — adding a new device should require almost 0 work. In fact the only machine specific module we have in odeviced is neo-powercontrol which takes care of the Openmoko specific peripheral devices.
• GSM Telephony (SIM access, SMS, Network, Calls, Supplemental Services, GPRS) is more or less feature complete minus some of the more esoteric features such as advice of charge (AOC) and GSM time (zone) reporting, which are only supported by few providers. Right now we have support for the following modems:
• Generic singleline and multiplexed multiline, serving as start points for your customizations,
• TI Calypso, as found in Openmoko devices and some HTC ones,
• Sierra USB, as found in Lenovo Thinkpads,
• Freescale Neptune, as found in Motorola EZX phones.

Support for the SIEMENS / Cinterion MC75i as well as for the Qualcomm-based HTC devices is on our list next.

• Resource Control for peripheral subsystems, such as GSM, GPS, Bluetooth, Wifi, etc. This provides you with reference counted resource management allowing for a maximum of user level power management (kernel level power management is not covered here).
• (A)GPS support covering NMEA and UBX devices with downloadable ephemeris and almanac for greatly improved warm- and coldstart performance.
• Preferences, Events, and Rules. This triade supports a simple way of configuring application preferences as well as defining system behaviour (display dimming, ringtones, sms notifications, battery notifications, and much more) using a set of extendable and customizable rules.

The two major things missing until we officially declare a 0.9 release are PIM and Networking. For the former, we’re attempting to integrate the results of a Google Summer of Code project (opim API), for the latter, we’re (still) evaluating whether NetworkManager, Moblin Connman, or Exalt can fit our usecases and needs — plus a very limited set of convenient calls on top.

All of the above is of course complementing the freedesktop.org initiative and should serve as a natural addition in order to help standardizing important Linux-based embedded APIs, as found in Maemo, Openmoko, LiMo, Moblin, OpenEZX, etc. We are commited to cooperate with said platforms to help defragmenting the mobile device world so that application programmers have it easier to target multiple platforms.

Along this line, the recently posted weekly Openmoko engineering newsletter adressed the issue of Openmoko and its relationship to freesmartphone.org. The bottom line is that Openmoko is funding the freesmartphone.org initiative to help fighting fragmentation. Previously Openmoko’s strategy was breadth-first to show the amazing versatility of open devices. Their next strike is going deep in one direction to create something that is both attracting developers (thanks to the dbus service level framework) as well as users (thanks to a pleasingly simple, extensible, phone application). We will see both of this in the forthcoming Openmoko 2009 distribution which is going to be the first FSO-compliant distribution ever — with more (e.g. Debian with pkg-fso, SHR and Rasterman’s work) being underway. Lots of them built out of OpenEmbedded, of course. (Speaking of OpenEmbedded –  the long awaited switch to git is happening this week. Open the flood gates and embrace our new revision control system :))

Last but not least some personal remarks. With me being part of the Openmoko family for more than two years now, it was time to reevaluate and redefine our relationship. I’m really glad to announce that Openmoko supports my direction of stepping a bit back from being overall Openmoko platform architect, allowing me to concentrate on the freesmartphone.org framework — making this part of the Openmoko platform as strong as possible.

As said, the three weeks passed too quickly and we found out we need to synchronize more often — hence I’m looking forward to increase the frequency a bit and next year stay more often at the Openmoko headquarters. 乾杯!

Although trying (really!) to cut down travelling, it’s still a lot. Here’s a sweeping swipe of what happened during the past couple of months and what’s going to happen soon.

Froscon

FrOSCon took place at its usual place on the last weekend in August and it was a very well organized conference — even better and more streamlined than the previous year was. I had the pleasure to listen to the Minix3 talk from Prof. Tanenbaum, which was very entertaining. Unfortunately my Openmoko talk was right after his one, so I had to take people kind of down to earth

Since I was feeling pretty weak at this weekend I could only attend the first day. Looking forward to next year’s session.

Braunschweig

Been travelling from Frankfurt to Braunschweig (to work with the Openmoko students on the framework) for a couple of times and I have started to actually use my Openmoko FreeRunner as a GPRS-forwarding device for my laptop. Using the freesmartphone.org framework it’s a breeze to do that. I just have to issue the dbus command ActivateContext(”internet.eplus.de”, “”, “”) and wait until the context goes online. Then I use iptables to enable NAT and forwarding for the laptop on the FreeRunner and it’s done.

GPRS is very solid on the FreeRunner — it works for hours without any disconnections or other interruptions. If the data connection is not 100% loaded you even get incoming call signalling and can take phone calls in between

Framework

Talking about the Openmoko / freesmartphone.org framework… we had a successful milestone3 release of it, debuting PDU-mode for SMS and phonebook data as well as lots of bugfixes over the place. We also have some nice reference docs now — i’m still working on introductionary type docs.

Unfortunately despite me trying to educate, lots of people still don’t get the point of the framework releases — I get frequent comments on the testing UI zhone, but rarely anyone is actually contributing to the dbus API specification and implementation discussion I seriously ponder whether to release console images in the future to make this 100% clear. I say it again: FSO is not about user interfaces, it’s about a strong independent dbus service level framework to facilitate 3rd party development.

That said, there are four important contributions in the freesmartphone.org world: pkg-fso, fso-gpsd, frameworkd-glib, downloads.freesmartphone.org:

1. pkg-fso is a team coordinating the packaging of any software from the FSO initiative (and, widely, any software related to Openmoko).
2. fso-gspd is a program offering a compatibility layer for the org.freedesktop.gypsy implementation of ogpsd. There are a lot of programs using the gpsd interface and with this compatibility layer, those programs will still work, but benefit from the improved accuracy of the UBX-based ogpsd.
3. frameworkd-glib is a C library offering bindings to the freesmartphone.org framework APIs. Handling modern dbus APIs can be cumbersome in C (think a{sv} and friends), so this library offers you convenience functions for that.
4. We now have official feeds hosted by a machine living in the same rack that serves kernel.org. Thanks to our friends at NSLU2-Linux.org and OSUOSL.org.

All software goodies live in our git repository.

Mobile Developer Days ‘08

Last week I had the honor to give an invited talk about OpenEmbedded and Qt-integration into OE for the Mobile Developer Days 08 conference in Berlin. This conference is pretty unique in that it adopts a platform-agnostic approach, i.e. you will find people working on Symbian, Qt, PalmOS, Windows Mobile there. I even spotted iPhone folks. In my opinion, such a holistic approach is important for the future of development on mobile devices. Congrats, folks.

Speaking about the iPhone… true readers of this column may remember that I have been a MacOS user since early this year. To add up on this, lately I acquired an iPhone to gain some experience with this exciting new development platform. I have just been looking into what this system provides. I can already say that there’s a whole lot of stuff where FOSS can learn and I’m glad to be a part of both worlds, so I can try to be a catalysator.

Taipei ‘08

On Sunday I’m going to fly over to Taipei. It’s been a while (12 months to be exact) since I met the folks in person there and there’s lots of stuff to catch up on. Now that the framework approaches its 0.9 release at the end of the year, we need to discuss the plan for the next 6 months. Can’t wait to meet all the engineers again! It’s going to be three interesting weeks. Stay tuned

Coming soon…

With the new Openmoko Framework initiative (as posted in previous installments of this column) facing its first milestone release (nothing but solid phone calls, so don’t be disappointed. If you have no Openmoko device, check out the video in the same directory), we are now facing three different major software stacks for the Neo family (there are special-purpose variants, but I won’t go into details here).
As there is quite a chance that developers might be confused about that, I want to use this chance to sketch the big picture and answer some of the questions around the future of these stacks.

Products

Openmoko is selling hardware products. Openmoko funds work on software stacks, so that the actual hardware can be more than just a developer board, but rather approaching a useful mobile compagnion. As with all kinds of products, Openmoko products have a — more or less specific — target audience. However, as we learned during all these months since we sketched the first product in the nice summer of 2006, even this specific target audience is not completely homogenous. The existance of the three software stacks is both due to the fact that we all are still learning how to write software for mobile devices, but also because there are quite substantial differences in what people expect from and want to do with their Openmoko devices.

Stacks

So — what are these three stacks, where are the actual differences and who should run which stack? In a nutshell, it boils down to the following:

1. The Openmoko 2007.2 Stack, utilizing GTK+ and assorted applications. 2007.2, since it was the 2nd iteration of the GTK+ user interface that we released in 2007.
2. The ASU Stack, the combination of a classical smartphone stack based on Trolltech’s Qtopia ported to X11 and enhanced with an EFL-based launcher and new applications. You may have seen the term Illume which is the launcher of ASU.
3. The FSO Stack, also known as the Openmoko Framework initiative. This one is called FSO, because it’s an implementation of the freesmartphone.org APIs. You may also have seen the term Zhone which describes the framework testing user interface and is a minor part of this stack.

Openmoko 2007.2 is for people who are familiar with the GNOME Mobile initiative and who want to write applications that run on multiple devices running (parts of) GNOME Mobile. This includes Maemo, which runs on the Nokia Internet Tablets. The strength of the GTK+ stack is a UI and programming environment similar to what you run on your Linux desktop, if you’re into GNOME. The GTK+ has PIM applications based on the Evolution Data Server and runs the gsmd phone server. Although you can use them, the applications are still pretty rough und unfinished. Some people have problems with the stability of the phone server.

ASU has been started to integrate the Qtopia stack — ported to X11 — with a new set of graphically pleasing applications based on the Enlightenment Foundation Libraries. Qtopia is a more mature product than the GNOME Mobile stack and you can expect all the standard feature phone applications to work in a solid way. It uses the Qtopia phone server. Since — contrary to standard Qtopia — it does not directly use the framebuffer, non-Qt applications can safely share the screen with Qt applications, that is until you are writing applications that do not communicate with the framework. If you want to integrate, then you’re back to C++ and Qt.

FSO has been started to overcome the deficiencies both of the 2007.2 and the ASU stack, namely to come up with an extensible framework that gives developers the infrastructure they need to create solid and exciting software products based on the Openmoko platform. An infrastructure that supports competing UIs while we can collaborate on developing services, making the framework strong . Here, the focus is on stable highlevel services that you can access from whatever language or UI that supports dbus. People report that despite its infancy, e.g. the phone server part in FSO is already more solid than anywhere else.

Future

Right now, Openmoko’s priority is getting the ASU to a point where it is stable and satisfies the every-day use with the FreeRunner product. In parallel, Openmoko recognizes the framework initiative as critical for further iterations of their software stacks. The goal is to be able to take one application from one stack and replace it with an application from another stack. Once all applications are framework-aware, this goal can be reached — which also implies userdata compatibility between software stacks, of course!

In practice, this means once the framework has reached a certain extent (see http://trac.freesmartphone.org for our tasks, issues, and milestones), we can expect it to be seen in all kinds of products, likely including further releases of the ASU. Until then, the FSO image is already your best bet if you want to write something that is tailored for your special usecase. If you want, you could also help us shaping our framework-testing-toy Zhone into something that fulfills the daily needs

Of course, Openmoko would also love to see Openmoko 2007.2 getting more love, but they don’t have the resources to do it. If you are interested in working on it, here’s a strategy I’d recommend:

1. Remove neod, gsmd, and phonekit.
2. Integrate the framework phone subsystem and make the dialer talk fso.org OTAPI (substituting gsmd and phonekit).
3. Integrate the framework device subsystem and make the launcher use it (substituting neod).
4. Debug and polish the PIM applications.
5. (optional and only when it’s ready) Integrate the framework PIM subsystem and make the PIM applications talk framework.

I hope that answered most of your questions. If not, feel free to add more via commenting this article.