Sid Player Update
Our Sid Player application has been on sale in the AppStore for three weeks now. We received quite a nice amount of feedback considering that this is a niche application -- thanks to all of you who downloaded and reviewed or mailed us!
Behind the curtains, we have been analysing the feedback and have just started to work on the first update which we expect to be available within the next 4 weeks.
This next release is mostly a database update, since we will add the missing songs from the HVSC directories 'GAMES' and 'DEMOS'. This adds about 2000 more classy titles that were left out due to not being present in the appropriate 'MUSICIANS' subdirectories.
Apart from that we will add an NTSC-override in the settings tab and -- if all goes well -- add some download-all-the-good-stuff buttons ;-)
Stay tuned for more exciting things coming to the Sid Player this year!
Visiting 25c3 for one day
Although traditionally the Chaos Computer Congress' schedule is slightly suboptimal for me (12/26th is my birthday), I'm going to be in Berlin from 12/28th to 12/30th and will visit CCC on the 3th day (12/29th). I'm going to attend Harald's talk about GSM base stations, so if you want to talk to me, just pick me up afterwards.
Sid Player on iPhone
You all know how much I adore the C64 sound aesthetics. This is the #1 program I was missing in my iPhone.
ogsmd and its modems
Recently, I have been working on adding more support for various types of modems to ogsmd. "Why's that?", I hear you say, "isn't that already done since long?". Yes and no. There are two dimensions where modems might differ:
- The way how the AT interface is exposed,
- The implemented AT command set.
Both dimensions come with their own set of problems:
Exposing an AT interface
As you might know, there are basically three different means for exporting an AT interface:
- A single line without mulitplexing support,
- Multiple premultiplexed lines,
- A single line with multiplexing mode.
Before we go into the details, some notes about what I mean with multiplexing mode. The 3GPP standard 07.10 defines a binary multiplexing protocol that exposes a number of virtual channels over which you can talk AT as defined in v250, 05.05, 07.07, etc. The concrete number of channels modem-specific -- it usually varies between three and four.
Let's come back to the three ways of exporting an AT interface now:
A single line without multiplexing
A single line without multiplexing support is the easiest, however it is also the most limiting way. The reason for that being that quite a lot of commands might take a long time to complete -- during that time the single line is blocked and you can not perform additional commands nor receive the so-called unsolicited responses from the network. These messages usually inform you about changes in the environmental GSM network condition, but also occur as notifications for incoming SMS or calls. In that scenario you will not be able to handle GPRS and voice calls at the same time.
Multiple premultiplexed lines
In the most simple form, a modem might support two discrete lines, e.g. one for all AT commands, the other one for GPRS connections. Some modems support three, four or even many more lines, however this usually comes with some strings attached. Multiple premultiplex lines usually are non-homogenous, that is you can not perform every AT command on any line, rather there is a line for SIM access, a line for voice calls, a line for network commands, etc.
A single line with multiplexing mode
In contrast to the premultiplxed line, these kinds of modems support an optional multiplexing mode over the single line transport (see explanation above). While this means that you need support code for the demultiplexing, a usually nice side effect is that the multiplexed lines are homogenous, e.g. the application (ogsmd in that case) can chose how to operate the lines. If you have more than two lines, you might want to dedicate one to unsolicited responses only, so you can react to network events even while dialling out. You might want to reserve one line for GPRS data connections and -- if you have a modem that does not buffer SIM access -- you might want to dedicate one line for the (potentially slow) SIM card retrieval commands.
The AT commandset
The second dimension of differences is the set of supported AT commands. The GSM AT command set is relatively "old" (in terms of computer science) and has its root in the (even older) command set (e.g. v25ter / v.250) designed for analogue terminal adapters. The GSM standards such as 3GPP 05.05, 07.07 define commands that serve as addition to the ones defined in v.250.
Unfortunately, the lion's share of these commands has been tagged "optional" -- also some of the GSM functionality is severely underspecified. This can lead to trouble such as follows:
- Multiple response format are to be considered "legal", which makes it tougher for your AT command parser.
- Some important status commands (such as the call status) are not covered by the specification at all. This means you need to rely on heuristics and timers to periodically poll the status.
- Vendors have extended the standard command set with dozens of additional commands. Your modem abstraction needs to handle these addition commands, but not rely on them.
- Some vendors even violate the standard to feature additional parameters. Obviously this is actually worse than inventing new commands (not to speak about implementing a multi-vendor modem class here...).
Briefly, some examples:
TI Calypso
The TI Calypso is a 2.5G (no EDGE) modem from Texas Instruments, as found in the Openmoko mobile phones Neo1973 and the FreeRunner. It has a fairly comprehensive AT implementation. All the commands support the query parameter and there are few standard violations. TI invented about 50 extracommands, of which ogsmd is using about 10 -- mainly for extended call progress information and network status.
Freescale Neptune
The Freescale Neptune is a 2.5G (EDGE) custom modem as appearing in the widely successful Motorola EZX Linux mobile phone family. The AT implementation is a nightmare, there are dozens of standard violations, implicit behaviour all over the place, and as if that wasn't enough, it has about 100 completely undocumented custom commands. And yes, 90% of the commands do NOT support the query format...
Cinterion mc75i
The Cinterion (previously SIEMENS) modem is one of the latest in a huge family of industry modems, deployed in millions of stationary and mobile devices. It will appear in the next-generation Openmoko device (codename GTA03). Its AT implementation is conforming to all standards and has been canonically enhanced to fill out the missing spots in the specifications. The documentation is a dream.
Bringing it all together
Supporting an additional modem in ogsmd can take from a day to some weeks, depending on the discussed two dimensions "way of exposing the interface" and "commandset implementation".
By the way, one particular ugly area is the call handling. As I have mentioned, the GSM standard only supports little information here (due to the compatibility with analogue modems), so most vendors have added their own unsolicited commands to take care of that. For those who have not though, we have a generic abstraction in ogsmd that uses timings and additional status requests (+CCLC to the rescue) to overcome this limitation. However, imagine a singleline modem with no vendor supported extra commands blocking during an ATD command... welcome to the dark.
So all this has motivated you to add a new modem abstraction to ogsmd? Excellent, be my guest! It now boils down to first identifying the way how to talk to it and then either chose to derive from the generic 'singleline' or the 'muxed4line' modem class. If the latter, you can adjust the number of individual channels created based on the capabilities of your modem. If your AT interface is premultiplexed, there's nothing else to do but match the channels to the right device nodes. If your AT interface supports a multiplexing mode, use the gsm0710muxd inbetween ogsmd and your modem (take a look at the TI Calypso implementation which is doing that as well).
Second, you need to find out where the standard AT implementation does not work with your modem or is not giving you all the information your modem could provide (think special commands). For such cases, in your new modem class you want to override specific mediators and add new unsolicited handlers. I'll briefly explain what these two are:
Custom Mediators
In ogsmd, a mediator is a small class that encapsulates a dbus command such as:
ogsmd's abstract mediator module (ogsmd/modems/abstract/mediator.py) contains mediators that are using standardized AT commands to perform their job. If your modem does not support these commands -- or has better commands to do that -- you should override said mediator (ogsmd/modems/mediator.py) to issue different commands in the 'trigger' method. Then, collect the results in 'responseFromChannel', and forward it to dbus -- that's all.
Custom Unsolicited Handlers
Unsolicited handlers specify what happens on incoming messages that originate from the GSM network, e.g. a +CRING that informs you about an incoming call. The handler then can perform additional queries or just trigger a dbus signal.
If your modem has additional (non-standard) unsolicited commands (or violates the 3GPP standards, which happens often enough) that you want to process, write a handler function in the unsolicited module (ogsmd/modems/unsolicited.py) for each of these.
Epilogue
I hope you have an idea what it means to add a new modem support class to ogsmd now. If you have more questions or want to help, drop a mail to smartphones-userland@linuxtogo.org.
I hope you enjoyed this months' tech talk ;-)
Pornophonique
Thanks to my framework team buddy Daniel 'Alphaone' Willmann, I just fell in love with the german band Pornophonique. Everyone appreciating the 8-bit chipsounds we had in the old days of computing will absolutely enjoy their work. My favourite song is 'Sad Robot' from the album 8-bit-Lagerfeuer -- it sounds a bit Everlast-inspired, but hey, that's a very common 4-chord-loop.
Did I mention that their music is creative commons licensed? Hats off, guys -- you rock!
CI61 06:50 FRA
::: {.img-shadow}
{width="200"}
:::
好久不見! Three weeks passed within a blink. Last sunday, we smoothly landed in Frankfurt/Main after 14 hours of a calm flight. 4/5 of the Openmoko Framework Team (while Stefan was on vacation in .au and missed all the fun) met in Taipei to tackle some outstanding issues and synchronize with the plans for the next major Openmoko release.
For a start, please refer to the blog postings of Charlie and Daniel, who mentioned some of the things we did in detail. Let me cover the current status and what we are going to work on in the remainder of this year and then step back and talk a bit about the meaning of all this. Right now the framework offers you support for the following tasks, everything accessible via consistent and convenient DBus interfaces:
- Device control (Backlight, Peripheral Power, Real Time Clock, USB, Switches, Buttons, Audio) covers the basics for embedded systems. Thanks to kernel 2.6, most of these interfaces are working on all devices -- adding a new device should require almost 0 work. In fact the only machine specific module we have in odeviced is *neo-powercontrol* which takes care of the Openmoko specific peripheral devices.
- GSM Telephony (SIM access, SMS, Network, Calls, Supplemental Services, GPRS) is more or less feature complete minus some of the more esoteric features such as advice of charge (AOC) and GSM time (zone) reporting, which are only supported by few providers. Right now we have support for the following modems: - Generic singleline and multiplexed multiline, serving as start points for your customizations, - TI Calypso, as found in Openmoko devices and some HTC ones, - Sierra USB, as found in Lenovo Thinkpads, - Freescale Neptune, as found in Motorola EZX phones. Support for the SIEMENS / Cinterion MC75i as well as for the Qualcomm-based HTC devices is on our list next.
- Resource Control for peripheral subsystems, such as GSM, GPS, Bluetooth, Wifi, etc. This provides you with reference counted resource management allowing for a maximum of user level power management (kernel level power management is not covered here).
- (A)GPS support covering NMEA and UBX devices with downloadable ephemeris and almanac for greatly improved warm- and coldstart performance.
- Preferences, Events, and Rules. This triade supports a simple way of configuring application preferences as well as defining system behaviour (display dimming, ringtones, sms notifications, battery notifications, and much more) using a set of extendable and customizable rules.
The two major things missing until we officially declare a 0.9 release are PIM and Networking. For the former, we're attempting to integrate the results of a Google Summer of Code project (opim API), for the latter, we're (still) evaluating whether NetworkManager, Moblin Connman, or Exalt can fit our usecases and needs -- plus a very limited set of convenient calls on top.
All of the above is of course complementing the freedesktop.org initiative and should serve as a natural addition in order to help standardizing important Linux-based embedded APIs, as found in Maemo, Openmoko, LiMo, Moblin, OpenEZX, etc. We are commited to cooperate with said platforms to help defragmenting the mobile device world so that application programmers have it easier to target multiple platforms.
Along this line, the recently posted weekly Openmoko engineering newsletter adressed the issue of Openmoko and its relationship to freesmartphone.org. The bottom line is that Openmoko is funding the freesmartphone.org initiative to help fighting fragmentation. Previously Openmoko's strategy was breadth-first to show the amazing versatility of open devices. Their next strike is going deep in one direction to create something that is both attracting developers (thanks to the dbus service level framework) as well as users (thanks to a pleasingly simple, extensible, phone application). We will see both of this in the forthcoming Openmoko 2009 distribution which is going to be the first FSO-compliant distribution ever -- with more (e.g. Debian with pkg-fso, SHR and Rasterman's work) being underway. Lots of them built out of OpenEmbedded, of course. (Speaking of OpenEmbedded -- the long awaited switch to git is happening this week. Open the flood gates and embrace our new revision control system :))
Last but not least some personal remarks. With me being part of the Openmoko family for more than two years now, it was time to reevaluate and redefine our relationship. I'm really glad to announce that Openmoko supports my direction of stepping a bit back from being overall Openmoko platform architect, allowing me to concentrate on the freesmartphone.org framework -- making this part of the Openmoko platform as strong as possible.
As said, the three weeks passed too quickly and we found out we need to synchronize more often -- hence I'm looking forward to increase the frequency a bit and next year stay more often at the Openmoko headquarters. 乾杯!
St.Augstin, Braunschweig, Berlin, Taipei
Although trying (really!) to cut down travelling, it's still a lot. Here's a sweeping swipe of what happened during the past couple of months and what's going to happen soon.
Froscon
FrOSCon took place at its usual place on the last weekend in August and it was a very well organized conference -- even better and more streamlined than the previous year was. I had the pleasure to listen to the Minix3 talk from Prof. Tanenbaum, which was very entertaining. Unfortunately my Openmoko talk was right after his one, so I had to take people kind of down to earth ;)
Since I was feeling pretty weak at this weekend I could only attend the first day. Looking forward to next year's session.
Braunschweig
Been travelling from Frankfurt to Braunschweig (to work with the Openmoko students on the framework) for a couple of times and I have started to actually use my Openmoko FreeRunner as a GPRS-forwarding device for my laptop. Using the freesmartphone.org framework it's a breeze to do that. I just have to issue the dbus command ActivateContext("internet.eplus.de", "", "") and wait until the context goes online. Then I use iptables to enable NAT and forwarding for the laptop on the FreeRunner and it's done.
GPRS is very solid on the FreeRunner -- it works for hours without any disconnections or other interruptions. If the data connection is not 100% loaded you even get incoming call signalling and can take phone calls in between
Framework
Talking about the Openmoko / freesmartphone.org framework... we had a successful milestone3 release of it, debuting PDU-mode for SMS and phonebook data as well as lots of bugfixes over the place. We also have some nice reference docs now -- i'm still working on introductionary type docs.
Unfortunately despite me trying to educate, lots of people still don't get the point of the framework releases -- I get frequent comments on the testing UI zhone, but rarely anyone is actually contributing to the dbus API specification and implementation discussion :( I seriously ponder whether to release console images in the future to make this 100% clear. I say it again: FSO is not about user interfaces, it's about a strong independent dbus service level framework to facilitate 3rd party development.
That said, there are four important contributions in the freesmartphone.org world: pkg-fso, fso-gpsd, frameworkd-glib, downloads.freesmartphone.org:
- pkg-fso is a team coordinating the packaging of any software from the FSO initiative (and, widely, any software related to Openmoko).
- fso-gspd is a program offering a compatibility layer for the org.freedesktop.gypsy implementation of ogpsd. There are a lot of programs using the gpsd interface and with this compatibility layer, those programs will still work, but benefit from the improved accuracy of the UBX-based ogpsd.
- frameworkd-glib is a C library offering bindings to the freesmartphone.org framework APIs. Handling modern dbus APIs can be cumbersome in C (think a{sv} and friends), so this library offers you convenience functions for that.
- We now have official feeds hosted by a machine living in the same rack that serves kernel.org. Thanks to our friends at NSLU2-Linux.org and OSUOSL.org.
All software goodies live in our git repository.
Mobile Developer Days '08
Last week I had the honor to give an invited talk about OpenEmbedded and Qt-integration into OE for the Mobile Developer Days 08 conference in Berlin. This conference is pretty unique in that it adopts a platform-agnostic approach, i.e. you will find people working on Symbian, Qt, PalmOS, Windows Mobile there. I even spotted iPhone folks. In my opinion, such a holistic approach is important for the future of development on mobile devices. Congrats, folks.
Speaking about the iPhone... true readers of this column may remember that I have been a MacOS user since early this year. To add up on this, lately I acquired an iPhone to gain some experience with this exciting new development platform. I have just been looking into what this system provides. I can already say that there's a whole lot of stuff where FOSS can learn and I'm glad to be a part of both worlds, so I can try to be a catalysator.
Taipei '08
On Sunday I'm going to fly over to Taipei. It's been a while (12 months to be exact) since I met the folks in person there and there's lots of stuff to catch up on. Now that the framework approaches its 0.9 release at the end of the year, we need to discuss the plan for the next 6 months. Can't wait to meet all the engineers again! It's going to be three interesting weeks. Stay tuned :)
{width="200"}
{width="200"}
{width="200"}GTK, ASU, FSO? TMTLA!
With the new Openmoko Framework initiative (as posted in previous installments of this column) facing its first milestone release (nothing but solid phone calls, so don't be disappointed. If you have no Openmoko device, check out the video in the same directory), we are now facing three different major software stacks for the Neo family (there are special-purpose variants, but I won't go into details here).
As there is quite a chance that developers might be confused about that, I want to use this chance to sketch the big picture and answer some of the questions around the future of these stacks.
Products
Openmoko is selling hardware products. Openmoko funds work on software stacks, so that the actual hardware can be more than just a developer board, but rather approaching a useful mobile compagnion. As with all kinds of products, Openmoko products have a -- more or less specific -- target audience. However, as we learned during all these months since we sketched the first product in the nice summer of 2006, even this specific target audience is not completely homogenous. The existance of the three software stacks is both due to the fact that we all are still learning how to write software for mobile devices, but also because there are quite substantial differences in what people expect from and want to do with their Openmoko devices.
Stacks
So -- what are these three stacks, where are the actual differences and who should run which stack? In a nutshell, it boils down to the following:
- The Openmoko 2007.2 Stack, utilizing GTK+ and assorted applications. 2007.2, since it was the 2nd iteration of the GTK+ user interface that we released in 2007.
- The ASU Stack, the combination of a classical smartphone stack based on Trolltech's Qtopia ported to X11 and enhanced with an EFL-based launcher and new applications. You may have seen the term Illume which is the launcher of ASU.
- The FSO Stack, also known as the Openmoko Framework initiative. This one is called FSO, because it's an implementation of the freesmartphone.org APIs. You may also have seen the term Zhone which describes the framework testing user interface and is a minor part of this stack.
Openmoko 2007.2 is for people who are familiar with the GNOME Mobile initiative and who want to write applications that run on multiple devices running (parts of) GNOME Mobile. This includes Maemo, which runs on the Nokia Internet Tablets. The strength of the GTK+ stack is a UI and programming environment similar to what you run on your Linux desktop, if you're into GNOME. The GTK+ has PIM applications based on the Evolution Data Server and runs the gsmd phone server. Although you can use them, the applications are still pretty rough und unfinished. Some people have problems with the stability of the phone server.
ASU has been started to integrate the Qtopia stack -- ported to X11 -- with a new set of graphically pleasing applications based on the Enlightenment Foundation Libraries. Qtopia is a more mature product than the GNOME Mobile stack and you can expect all the standard feature phone applications to work in a solid way. It uses the Qtopia phone server. Since -- contrary to standard Qtopia -- it does not directly use the framebuffer, non-Qt applications can safely share the screen with Qt applications, that is until you are writing applications that do not communicate with the framework. If you want to integrate, then you're back to C++ and Qt.
FSO has been started to overcome the deficiencies both of the 2007.2 and the ASU stack, namely to come up with an extensible framework that gives developers the infrastructure they need to create solid and exciting software products based on the Openmoko platform. An infrastructure that supports competing UIs while we can collaborate on developing services, making the framework strong . Here, the focus is on stable highlevel services that you can access from whatever language or UI that supports dbus. People report that despite its infancy, e.g. the phone server part in FSO is already more solid than anywhere else.
Future
Right now, Openmoko's priority is getting the ASU to a point where it is stable and satisfies the every-day use with the FreeRunner product. In parallel, Openmoko recognizes the framework initiative as critical for further iterations of their software stacks. The goal is to be able to take one application from one stack and replace it with an application from another stack. Once all applications are framework-aware, this goal can be reached -- which also implies userdata compatibility between software stacks, of course!
In practice, this means once the framework has reached a certain extent (see http://trac.freesmartphone.org for our tasks, issues, and milestones), we can expect it to be seen in all kinds of products, likely including further releases of the ASU. Until then, the FSO image is already your best bet if you want to write something that is tailored for your special usecase. If you want, you could also help us shaping our framework-testing-toy Zhone into something that fulfills the daily needs :)
Of course, Openmoko would also love to see Openmoko 2007.2 getting more love, but they don't have the resources to do it. If you are interested in working on it, here's a strategy I'd recommend:
- Remove neod, gsmd, and phonekit.
- Integrate the framework phone subsystem and make the dialer talk fso.org OTAPI (substituting gsmd and phonekit).
- Integrate the framework device subsystem and make the launcher use it (substituting neod).
- Debug and polish the PIM applications.
- (optional and only when it's ready) Integrate the framework PIM subsystem and make the PIM applications talk framework.
I hope that answered most of your questions. If not, feel free to add more via commenting this article.
Momentum
Momentum is something really strange! It's hardly predictable and you have to run to catch it before it goes away -- but when it's there, things are progressing like there's no tomorrow :-)
There's a whole lot of momentum present in some of the projects I care about:
OpenEmbedded
- OpenEmbedded will abandon monotone and move to git as its primary SCM. This will increase the wide-spread adoption of OE and attract new people. We will have more (shortlived) branches and merges will be easier. We will also revamp the commit policies a bit to introduce more stability to both the stable and the unstable branches.
- Setting up the non-profit organization (german registered association e.V.) is progressing and we will soon have our legal entity.
- Some couraged people are revamping our website into something that's more accurate, structured, (and pretty). Good for both OE-novices and experts.
OpenEZX
- OpenEZX developers started to push for mainline inclusion. This will greatly increase the visibility of our project.
- OpenEZX developers are working on a 2nd stage bootloader that overwrites the MOTO kernel, but leaves the rest of the flash file system untouched -- by this we can boot both an OpenEZX kernel from SD as well as the original system (with its kernel on SD), which makes testing a relief.
- We found security holes in the MOTOMAGX kernel, which may enable us to put our own code on these systems.
- Motorola has released new devices, apparantly running EZX. More devices for the platform!
- Openmoko's Framework and Zhone phone UI is going to provide a slick looking -- working -- featurephone userland for OpenEZX -- the first releases are just a couple of days away. It's now important we get kernel work finished, so we can release something that is a real alternative to the closed source MOTO system on EZX devices.
Openmoko
- Openmoko successfully went into mass production of the Neo FreeRunner (GTA02) device. Finally, people will have the 2nd generation of the first truly open source hardware platform in their hands.
- The Qtopia/Enlightenment based next generation software update is progressing nicely. I expect a release in the next couple of weeks.
- The new dbus-centric Openmoko framework initiative as well as the Zhone bread-and-butter application will see a release within the next 48 hours.
Exciting times for Linux on mobile platforms, n'est-ce pas?