BlackBerry
Skip Top Navigation
Developers Partners
North America [change region] Products Solutions Purchasing Support
Developers


BlackBerry Developer Journal

Creating a RSS/RDF Push Service for BlackBerry

Mark Sohm, Research In Motion

These days, more and more web sites are offering Really Simple Syndication[1] (RSS) or Resource Description Framework[2] (RDF) services. These services allow RSS and RDF readers to connect to a website and retrieve a list of headlines that can be presented to the user. Combining feeds from multiple sites gives a reader a quick, one-stop location where they can browse the latest information from all of their favorite web sites. More information on RSS and RDF can be found on their official home pages. You can find links to these pages in "Appendix A - Site References" below.

Topics within this section include:

Selecting the Technology

BlackBerry is an ideal platform for easily taking advantage of these services. I thought that having a small, always-available list of updated headlines of favorite web sites would be a popular BlackBerry push service. I decided to create a browser channel push, which involves pushing an HTML or WML page to a BlackBerry handheld through the BlackBerry Mobile Data Service (MDS). A new icon will appear on the home screen of the BlackBerry handheld once the push arrives. The HTML or WML page will open in the BlackBerry Browser when a user clicks on the icon.

Since this service would not require an application to reside on the BlackBerry handheld, I was free to choose from a variety of technologies to build the application. The service as a whole is required to perform the following tasks:

  • manage a list of subscribers and web sites to which they have subscribed
  • download and parse RSS/RDF pages
  • send a customized HTML page to the user using the BlackBerry MDS

After considering my options, I chose to make use of the following technology. MySQL[3] as the database server to store all of the user, server, site and subscription information. For the user front end, I chose to make use of Apache[4] and php[5] to create a web site where users can manage their subscriptions, and add new RSS/RDF sites. To handle the download, RSS/RDF parsing, subscriber page generation and pushing to the MDS, I decided to create a server application using Java (J2EE).

Please read on for a journal-like description of the process behind the creation of this application. Full source code is included in the Appendices at the end of this document. I suggest jumping to the appendix and reading through the source files as you work your way through this article. I have refrained from including code samples inside this article to conserve space; most samples would be quite large and I believe it is more beneficial to view the source files in their entirety to see how the application flows and works together.

The Database Design

The first step in creating this system was to design a database that would hold all of the data used by the pieces of this application. The first and most obvious table required would hold the list of users or subscribers to the service. This would contain the following:

  • email address used to push the HTML page to their BlackBerry handheld
  • personalized HTML page that is pushed to them
  • hash of their personalized page
  • flag that is set when a user unsubscribes from the service
  • timestamps of when their personalized page was last updated
  • time of the last successful push to MDS

The next required table would be one that holds information about the RSS/RDF sites on the system. This table would store:

  • the site name
  • its URL
  • user number of the user who added the site to the system
  • a cache of the RSS/RDF page itself
  • hash of the RSS/RDF page
  • timestamp of when this page was last updated
  • last modified date/time
  • ETag header information from the web server that houses the RSS/RDF page

The ETag and last modified HTTP header information from the last HTTP request are sent with every subsequent request to access the RSS/RDF page. This allows the web server to determine whether the page has been updated since last retrieval, and not to send the whole page every time (it will simply reply with a Not Modified: 304 code instead). This is a feature that should be implemented with every RSS/RDF reader as it greatly reduces the load placed on a web server when many RSS/RDF readers connect simultaneously.

I also needed a table to store subscription information to know what sites users would want pushed to their BlackBerry handheld. The basic requirements of this table called for two columns:

  • the site number
  • the user number of the subscriber

Expanding on this, I also included some columns to store user preference information about the sites they subscribe to:

  • an integer field to store the maximum number of articles to return from the site. This is useful for sites that include a large number of headlines in their RSS/RDF page.
  • a flag that allows subscribers to select whether they want the article description to be sent along with the headline - for sites that include a description
  • an integer field to allow a subscriber to specify the order they wish their subscribed sites to appear in on their BlackBerry handheld

The final table contains a list of BlackBerry Mobile Data Service (MDS) servers that will be used to push data to the subscribers. The hostname and port of each BlackBerry MDS server are stored here. This allows MDS servers to be added and updated as the BlackBerry environment grows.

With all the tables set up to hold the data, it was time to create something that could be used to populate them. I was now ready to begin creating the PHP and HTML pages that would be used by potential subscribers to sign up for the RSS/RDF push service. For the complete SQL commands used to create the tables described above, please see "Appendix B - RSS/RDF Push Service MySQL Table Structures".

The Front End

My decision to design the front end using web-based technologies was based on a number of factors. Almost every computer user is familiar with the web, and connecting via a web browser wouldn't require any additional software to be loaded on a subscriber's PC. In this way, they could even manage their subscription using their BlackBerry handheld! The RSS/RDF push service front end is made up of a very small number of files. There is a main index HTML page and four PHP pages.

The main index.html page simply prompts the user for their email address, and submits it to the rssmain.php page after verifying that the user has entered a properly formatted email address. Most of the work of the front end is done on the rssmain.php page. When a user first loads this page, it verifies whether their email address is already associated with a user entry in the user table. If it is not, one is added. It will then list all RSS/RDF pages that have been added to the server. This allows the user to manage subscriptions. From this the user can set the order they will be received, the number of headlines to be shown for each site, and whether or not the description will be shown with the headline if it is available. Users are also able to add additional RSS/RDF sites to the system from the rssmain.php page.

The remaining three pages have a much simpler function. The rssunsubscribe.php page is posted to from rssmain.php, and supplies the user number for the subscriber that is unsubscribing. The user is then removed from all entries of the subscriptions table, and the SendDelete flag in the users table is set to 1. The showpage.php also accepts the user number as a parameter and will retrieve and display the user's customized HTML page from the users table that is sent to them via the channel push. The last page, header.php, is included by the other pages and creates the connection to the MySQL database server.

Now that the front end was complete and I had a mechanism to enter subscriber and site information, I was ready to start building the push service application. To view the complete source code for the PHP pages, please see "Appendix C - RSS/RDF Push Service php & HTML Pages".

The Push Application

The next step would be to create the application that would take care of all the dirty work. It would need to:

  • download the RSS/RDF pages
  • parse them
  • create a customized HTML page based on a user's subscription
  • push that page out to their BlackBerry handheld through MDS

I created four Java classes to handle this operation:

  • a database connector
  • an RSS/RDF XML parser
  • a push class
  • a main controlling class that ties it all together

The first Java class I created, RSSDBConnector, was one that connects to the MySQL database server. Making a separate class for the database class made sense because if I need to change the database server, I just have to create a new dbconnector class. I made use of the MySQL Connector/J[6] JDBC driver when connecting to MySQL. The RSSDBConnector class allows connections to be created and closed to MySQL, as well as executing a query passed to it. I created two separate methods to handle queries, one for select statements that return a result set, and one for insert and update statements that return the number of rows affected by the query.

The next class on the list, RSSParser, needed to parse the xml RSS/RDF documents and break them apart, extracting the site name and titles, links and descriptions for every item. The dom4j[7] framework was used to handle the XML. This class is fairly simple in that you pass it a string of XML and get back a Vector containing the extracted information to be used in creating a subscriber's customized HTML page.

The RSSPusher class handles the actual push of data to MDS has been modelled after the BrowserPushDemo application that is included as a sample application with the BlackBerry Java Development Environment (JDE). Since the main framework was already in place, I was able to quickly make some minor changes to customize the application to suit my needs. This class allows me to set all the parameters for a push and send it out to the specified MDS Server.

With all of the individual pieces in place, it was time to build the main controlling class that would utilize the database connector, RSS/RDF parser and push classes. The RSSPush class will create a connection to MySQL through RSSDBConnector and retrieve a listing of all RSS/RDF pages on the system. It will then go through each page to see if has been updated since the last check, then updates the page in the database if needed. Once that has been completed, the HTML pages for all subscribers are updated if pages they subscribe to have been updated. After all of the customized HTML pages are up to date they are pushed out to the subscribers. It then disconnects from MySQL and sleeps for 45 minutes before starting the process all over again.

With the Java push application in place, my BlackBerry RSS/RDF push service was almost ready to go. To see the full Java source code for the Java push application, please see "Appendix D - RSS/RDF Push Service Java Source Code". The final pieces of the puzzle are the icons that will be shown on the home screen of the BlackBerry after the push has taken place.

Graphics

Since I was using a browser channel push, the BlackBerry Browser would take care of rendering the HTML page that is pushed out to the subscribers. This means I didn't have to create an application to reside on the BlackBerry handheld. However, this push would place an icon on the BlackBerry handheld home screen. There are default icons used for channel pushes if an icon is not specified, but I wanted something that would easily identify my push page. I created a pair of 28x28 pixel images in .png format that are used for the read and unread icons. The icons for a channel push change between two states; read and unread, depending on whether or not a new page has been pushed since the user last viewed it. These images are placed on the web server and links to them are included with the push itself.

Finishing Up

With all of the pieces of the RSS/RDF push service ready to go, all that remained was to populate some data. I added the MDS servers and ports of the machines that are set to be the MDS Push Servers in my environment and added a handful of RSS/RDF sites I thought would be popular in attracting subscribers. As a matter of convenience, I set up the Java push application to run as a Windows Service using the Java Service Wrapper[8], allowing it to run at startup without requiring someone logged into the computer. With that in place I had a service setup to allow me, and any other BlackBerry users, to read the latest headlines from popular web sites, updated automatically, on our BlackBerry handhelds.


Appendix A
Site References

  1. Really Simple Syndication (RSS)
  2. Resource Description Framework (RDF)
  3. MySQL
  4. Apache
  5. php
  6. MySQL Connector/J
  7. dom4j
  8. Java Service Wrapper


Appendix B
RSS/RDF Push Service MySQL Table Structures


Download the script


Appendix C
RSS/RDF Push Service php & HTML Pages


Download the source files


Appendix D
RSS/RDF Push Service Java Source Code


Download the source files


Top |  Table of Contents |  Journal in PDF Format |  Legal Disclaimer
 
     
 Home | Products | Solutions | Purchasing | Support | Developers | Worldwide | News | About Us | Contact Us | Site Map
 Legal | Copyright © 2008 Research In Motion Limited, unless otherwise noted.