Software Tools

Transferring SMS messages from Windows Mobile to Blackberry devices

UPDATE (14 Feb 2015): Due to some outstanding issues with the SMS2IPD tool such as Unicode support and message extraction issues on some devices, which will most likely forever remain unfixed because I no longer have any Windows Mobile 6.x devices (which are no longer supported by Microsoft), I have decided to stop maintaining SMS2IPD and removed the download from this article.

For a more robust tool to restore text messages to Blackberry devices from a CSV (comma-separated values) backup, refer to my other tool, CSV2IPD.

The original article, written in 2010, is archived below for informational purposes only.

---

This may seem an easy task - one may immediately think of using the Device Switch Wizard in Blackberry Desktop Manager or do a quick search for some SMS backup/restore software which will do the job. However, while I was doing it, I came across some challenges which turns this into a difficult job:

1. The Device Switch Wizard found in the Blackberry Desktop Manager simply does not migrate the SMS messages.

2. There are plenty of applications that perform SMS backup/restore for Windows Mobile but there are none for Blackberry.

The first challenge is expected as many users simply do not need to migrate text messages to their new phones and it is therefore understandable that the Device Switch Wizard ignores SMS during the migration process but the second challenge disappoints me. I was realistic - I never expected to find a tool that transfers text messages on-the-fly, perhaps with the tool running on Windows, the Blackberry connected via Blackberry Desktop Manager and the Windows Mobile device connected via ActiveSync. BUT I was hoping to find some software that can restore Blackberry SMS from a backup, perhaps XML-formatted, which can be created by a tool running on Windows Mobile.

I was wrong - there are no such applications for Blackberry. This is explained by the fact that RIM does not provide an API for Blackberry applications to programmatically access the SMS messages store (there are, however, APIs to access email messages).

My approach

My ideas come from the fact that Blackberry Desktop Manager allows user to backup device memory (including contacts, calendar and SMS) into an IPD file, and restore from the IPD backup. Although the IPD file format is not documented by RIM, it is easy to reverse engineer (for example, by using a hex-comparison tool) due to its unencrypted and tokenized nature. There are previous attempts to do this, for example ipddump and MagicBerry. The latter is extremely useful as it allows you to view, edit, merge and split IPD files.

Keeping the above in mind, I came up with a tool that runs on Windows Mobile device and backs up all text messages into an IPD file. The file created can later be opened using Blackberry Desktop Manager and the text messages can be restored back onto the Blackberry device.

Final product: SMS2IPD

In the lack of a better name, let me call it SMS2IPD, short for SMS to IPD Converter. Heres how to use it:



1. Run the tool on your Windows Mobile device. You will need .NET Compact Framework 3.5 and above.

2. Select the correct time offset so that SMS timestamps will be correctly displayed. For example, if you are in GMT+8, select -8 for the time offset. The selection is needed as I am not sure which field to set to specify the timezone of the SMS messages.

3. Click on Start, choose the location to save the IPD file. Give the application some time to run, when completed, it will produce the IPD file.

4. Connect your Blackberry and start Desktop Manager, choose Backup/Restore, then click on Restore again. Select the IPD file created in step 1 and Desktop Manager will proceed to transfer all SMS to the device.

Note: this process may take a long time during which Desktop Manager may seem to hang. It will also overwrite all existing text messages on the Blackberry device, so back them up and merge them to the IPD first using MagicBerry. MagicBerry may also seem to hang when loading a large IPD file, so be patient.

Limitations and future improvements

I myself reverse engineer the IPD file format by using a hex editor, a hex comparison tool and MagicBerry, with some references to ipddump and MagicBerry source code, so there will be cases not handled properly. These include SMS with special characters, unicode SMS, and SMS sent from a timezone different from your device timezone.

Upon imported on the Blackberry device, all messages may appear as unread. You can mark all as read by scrolling to the top of the messages list (which displays the date), press the Blackberry key and choose Mark Prior Opened.

Some Blackberry devices by default will delete text messages older than 30 days. This behaviour can be changed by opening Options in Messages, choose General Options, and select a longer period for "Keep Messages".

I attached the source code for those interested to improve the application. You can modify and re-distribute the application and its source code as you like, but please mention that the source code was taken from my blog. You may not, however, use the source code and/or the application for any commercial purposes.

Credits

To read the SMS messages from a Windows Mobile device, I am using MAPIDotNet library found on codeproject, written by rwt33. The version I used was downloaded some time back, so it may differ significantly from the current version found in the project article.

Downloads

SMS2IPD CAB Installer for Pocket PC and Smartphone, Windows Mobile 5 and above with .NET Compact 3.5 (updated 10 Oct 2010) - removed as SMS2IPD is no longer maintained. See notice at the beginning of this article.

Get

DLNA Streaming from Linux 1st Gen Sony GoogleTV

While cleaning up my wifes tablet I found a few hundred photos/videos from 2010-2011 time frame.  I copied them all to offline storage.  My kids were having a ball watching younger versions of themselves over my shoulder, so I thought, why cant I just stream this stuff?

The Sony GoogleTV already supports streaming from other DLNA devices.  This generally includes a lot of devices, like Windows laptops, that appear on the network and make videos and pictures available to DLNA clients.

I tried a few different approaches in Linux before I found one that just worked right out of the box.

First, theres MiniDLNA (now called ReadyMedia).  This project is "sort of" supported by Ubuntu with a separate repo.  However the objects in that old repo seem out of date, and the only way to get this up and running with the latest and greatest version is by building it from source.  I didnt want to mess with that this morning, so I moved on without even giving this one a go.

Next, I found MediaTomb.  While this was relatively easy to set up under Ubuntu, I couldnt get my Sony GTV to recognize it.  My phone w/ a DLNA client, however, *would* recognize it, but it refused to play any of the content.  I messed w/ the config file and uncommented some lines for the PS3, Googled around a bit, didnt find any solution.  Uninstalled, moving on.

I looked around for Plex, which I know is popular, but it is commerical and I didnt really like the license terms, so I moved on.

I finally found Rygel, which installed easily under Mint/Ubuntu and in literally minutes I had it running.  I also installed the companion app, rygel-preferences, which made it super easy to point to specific folders that had the content I wanted to publish. 

I did have to run "rygel &" from the commandline to start the server, but after that the Sony GTV discovered it immediately and the kids could look at all 6GB of short movies and photos.

Get

Capturing data from a Tektronix 1230 logic analyzer by emulating a parallel port printer

UPDATE: Refer to my latest article for a full review of the Tektronix 1230 logic analyzer.

As a fan of vintage test equipments such as oscilloscopes and logic analyzers, I sometimes encounter the problem of extracting data from these machines onto external devices for storage or further analysis. While some of these old devices allow saving data to floppy drives or feature serial or GPIO ports and associated protocols to communicate with PCs or other devices in order to extract measurement data or capture screenshots, many other devices only support printing data in hard-copy to parallel or serial printers, which are hard to find nowadays. Recently while working with an old Tektronix 1230 logic analyzer, I spent some time tackling this problem and this article will share some of the achievements I have made and other interesting findings.

The device

I acquired this Tektronix 1230 from eBay, a vintage logic analyzer in the 1980s but still useful for troubleshooting old 8-bit designs. The following photos show the logic analyzer, with a Centronics parallel port and a DB25 RS-232 port at the back:


A word of warning here for those intending to collect this vintage equipment. Make sure that your 1230 comes with probes (also known as pods) - you will want the P6444 or P6443 16-channel probe. Probes can be hard or expensive to purchase separately. It should also preferably come with RS-232 or Centronics ports for data output.

The challenge: transferring captured data

My challenge came when I wanted to export the capture data from this logic analyzer. Although it has a RS-232 port supporting up to 9600bps, communication with the device requires the S43R101 1230/PC application software made by Tektronix. During my research, a serial protocol for custom-made application to work with the logic analyzer via the serial port seems to be available and documented in the manual. Unfortunately, I could never locate a copy of the manual or the application software for this device available for download, apart from this information sheet. There is, however, plenty of information on the Internet for the Tektronix 1240, a later model. Extracting data via the serial port therefore seems infeasible.

The only other way is to print data via the Centronics parallel port. Although I do not have any parallel port printer nowadays, a thought suddenly came across my mind that I might be able to program a PIC microcontroller to act as a parallel port printer and save the output data to an SD-card for further actions. Well, as they say, thoughts are dangerous and within minutes I found myself soldering wires to a DB25 parallel connector to interface the logic analyzer with a PIC24FJ64GA002 microcontroller.

The Centronics printer protocol

This is the Centronics parallel port pinout:
Attempting this project requires knowledge of the Centronics protocol used for communication with parallel port printers, described in details on this page. For our purposes of emulating the parallel port printer. we will only need to care about the ACK, STROBE, BUSY signals and the D0-D7 lines. In the simpler polled mode of the Centronics protocol on a standard parallel port, the host (referring to the logic analyzer in our case) would send a pulse on the STROBE line to indicate its going to send printing data. After that, the printer (or the PIC microcontroller in our scenario) will need to set the BUSY line high while at the same time reading for an input byte from the D0-D7 lines. When processing is completed, the printer will send a pulse on the ACK line and set the BUSY line low, indicating it is ready for the next command. The process is illustrated in the following timing diagram:


To indicate that the emulated printer is available and ready for printing, Paper End (pin 12) should be connected to GND while Error (pin 15) and Select (pin 13) should be connected to 5V.

Result: a PIC-based virtual parallel port printer

Using the ST7735 1.8" color LCD (see my previous post) and the Microchip SD card library, I was able to build a working virtual printer. The device would listen for data being sent from the parallel port and save the print job to a .PRN file on the SD card. The following photos show the printer in action:



The 1.8" LCD shows the SD card volume label, file system type, total capacity and free space. The name of the file containing the last print job is also displayed on the LCD. 

As the PIC does not understand the data being sent and simply writes the data received on the SD card, there is an issue of telling when the print job finishes in order to start a new output file. On a real printer, the printer would detect the paper feed command and eject the page when a page has finished printing. In my case, I have chosen a simplified approach - assume that printing has ended and close the output file if no data is received after a certain period, e.g. 5 seconds. This would be sufficient for most purposes.

Reading the printer output

This emulated printer should work well with any application expecting a generic parallel port printer. It may not work if the application is expecting a specific modern parallel port printer model, in which case customized identification data might be sent via the parallel port in ECP or EPP mode to communicate with the application. For a simple test, executing the DOS command DIR > LPT1 or DIR > PRN with the printer connected to the LPT1 port should result in a .PRN file containing the list of files and sub-directories in the current directory being written to the SD card.

On the Tektronix 1230, printing can be done by double-pressing the NOTES key (to print a screen capture) or by pressing the D key (to print memory contents), as described in the help screen:


While pressing D will print the memory contents in text format, double-pressing the NOTES key (supported only on Epson-compatible printer) will print a graphics representation of the current screen. Although printing of memory contents will not hang the logic analyzer and the printer status will be displayed as "Printing", the logic analyzer will stop responding while the screen is being printed.  Also during my experiment, if the status lines on the Centronics parallel port report conflicting information, the logic analyzer will refuse to boot up with the POWER indicator turned off, making it seem as if the device is dead. Removing the parallel cable and the unit will power on just fine. This seems to be a bug in the device firmware.

The screen print output, as captured by my virtual printer, is in binary format and contains Epson escape codes. A quick inspection of the output file and comparison with the Epson escape codes documentation shows that only 4 escape codes are used:

• ESC 65 and ESC 50 - set line spacing
• ESC 108 - set left margin
• ESC 42 - select bit image for graphics printing

I quickly managed to write a tool which converts the output escape codes back to a 450x250 bitmap file:


A check-box "optimize for printing" is provided. If checked, the output will be black-on-white (instead of green-on-black) and can be printed using a normal printer. The printout after conversion looks like below:

With the virtual PIC-based printer and the escape code converter tool, I am able to copy data from the logic analyzer to my PC for other purposes.

Replacing the RTC batteries 

As with other old electronics equipments, I needed to replace the 3V batteries which keeps the device clock and and settings. The 1230 is using two CR2330 3V batteries for this purpose:


The notice near the batteries reads, CAUTION: REFER BATTERY REPLACEMENT TO QUALIFIED TECHNICIAN. Am I a qualified technician? Well, at least not from the administrative perspective - I am not certified by Tektronix to open up this device! Would there be any implications if I am not? Are there any specific instructions to be performed before the battery is replaced or must the batteries be replaced in a specific order to avoid loss of data? Unable to find any information on this on the Internet, I proceeded to replace nevertheless and the logic analyzer still seems to be working well ever since. 

Interestingly, although the time settings on this device allow years between 1900-2099, the year would jump back to 1914 even if 2014 is selected after a reboot, indicating some sort of Y2K issues. The rest of the date and time remains correct. The day of week also needs to be selected manually and is not calculated automatically like many other devices. This was probably done to save precious code space for other stuff, or perhaps the algorithm is too complicated to be implemented on a Z80 processor (used by most Tektronix logic analyzers of this generation) efficiently.

Downloads

The source code of the printer and the bitmap converter tool can be downloaded here:
MPLAB 8 PIC24FJ64GA002 project for the virtual printer
Bitmap converter for Tektronix 1230 Epson printer output

Read my other article for a full review of the Tektronix 1230:
Tektronix 1230 Logic Analyzer

The following Youtube videos provide useful information on operating this logic analyzer:
Tektronix 1230 training (part 1)
Tektronix 1230 training (part 2)

Some interesting reverse-engineering information on the Tektronix 1240, a later model:
Repairing and understanding a Tek1240

Get