Updating a Database in a Deployed Mobile App

One of the issues I had to deal with in building my mobile app was the evolving database schema as more features were added and the app grew.  I needed a way to update the database scheme without blowing away the old one and forcing the app to redownload all the data.  While I could have done that without any loss of data (there is no user supplied data to be lost), the main data set is over 8 MB in size and that isn’t nice to users’ wireless plans.  What I needed was a way to incrementally change the schema as the app was updated.

First Attempt

My first attempts just used if() statements.  i.e. if the current app version was one value and the new app version was a different value, apply this update.  This worked well enough for my development versions where I was strictly controlling the upgrade versions.  But I quickly realized that if I were to try to do that with versions of the app out in the wild, where I had no control over when it was updated, I’d quickly have a mess of conditionals to deal with.  So I scrapped that idea and went looking for something new.

Final Solution

I don’t think this solution is new to me as I’m sure I saw it somewhere to give me the idea but I couldn’t tell you where I saw it off the top of my head.  The answer was a switch() statement, with each case() block having a bit of the upgrade path from old to new schemas.  In order for this to work, you need to track a version number for the database schema that increases each time you make a change.  Simply start at one and every time you change the schema increment the value.  This value, which I called current_version, is simply a coded variable.

Additionally, the app needs a second value that it stores as part of the application data which contains the database version that is currently installed (installed_version).  Then, each time the application is run, you check these values.  There are three possibilities[1]:

  1. There is no stored value for installed_version. This should only be true the very first time the application is run.  In this case, you simply need to create the entire database schema as it now stands and store the current_version as the installed_version.
  2. The current_version equals the installed_version.  This is the simplest case of all.  Since the database schema is all up-to- date you don’t have to do anything.  Continue on to the main part of the application.
  3. The current_version is greater than the installed_version.  In this case there is work to do and we now need to do the update.

To make the update we simply enter a switch() statement based on the installed_version.  Each case() block should have arguments that start at 1 and increase by one each time.  The first case() block (i.e. case 1:) would hold the commands that were necessary to upgrade from version 1 to version 2.  The second would hold the commands modify the schema from version 2 to version 3, and so forth.

None of the case statements should have the break command associated with them so that after execution they would fall through to the next one and cascade all the way to the end.  That way if a user skips one or more app updates that include database upgrades, each one will be applied in turn and none will be skipped.

Finally, the default() block at the end of the switch statement should contain the code up update the value of the stored installed_version so that it is also up-to-date for the next time the application is run.

Here is a bit of sample code that shows how this works.  It is lifted in modified (with additional comments added) from my application which is written in using Titanium Studio’s Mobile SDK.  It should be noted that the c object is a configuration object that holds a lot of information such as the database name, the table names, and column names among other things.

// Check that we have a stored value for the installed database version
if(!Ti.App.Properties.hasProperty('installed_db_version')){
  // if not, the database has never been created
  createDB();
} else {
  var installed_version = Ti.App.Properties.getInt('installed_db_version');
  if (c.current_version > installed_version){
    Ti.API.info("Performing an upgrade from database version "+old_version+" to version "+c.version);
    var db = Ti.Database.open(c.name);
    // We need to do an upgrade to the new version based on the old version.
    // We'll do this incrementally adding in all the changes since the installed
    // version was created.
    switch(installed_version) {
      case 1:
        db.execute("REPLACE INTO daily_data (source,start,duration,flux_1000_300000,error_1000_300000,ul_1000_300000," +
        "flux_300_1000,error_300_1000,ul_300_1000,flux_100_300000,error_100_300000,ul_100_300000,test_statistic)" +
        "VALUES (1,56000,86400,10,1,0,9,2,0,1,0.5,0,32)");
        db.execute("REPLACE INTO daily_data (source,start,duration,flux_1000_300000,error_1000_300000,ul_1000_300000," +
        "flux_300_1000,error_300_1000,ul_300_1000,flux_100_300000,error_100_300000,ul_100_300000,test_statistic)" +
        "VALUES (1,57000,86400,12,2,0,10,1.5,0,2,0.5,1,32)");
      case 2:
        db.execute("DELETE FROM " + c.T_DAILY);
        db.execute("DELETE FROM " + c.T_WEEKLY);
        Ti.App.Properties.setString('last_lc_data_update','0');
      case 3:
        db.execute(DATABASE_CREATE_GCN);
        Ti.App.Properties.setString('last_GCNNotice_date', '0');
      case 4:
      ...
      ...
      default:
        Ti.App.Properties.setInt('installed_db_version',c.current_version);
        break;
    }
  db.close();
  }
}

In version 1, I was using some dummy data that I was setting explicitly.  In version 2, I had updated the code to get the data from a web service so I needed to remove all the old data and add a stored variable to track the date of the last data point.  In version 3, I added a table to the database and an internal stored variable to track the date that data was last updated.

At the beginning of the code block, I check to see if the database is installed at all.  If not I call a createDB() function.  This function simply contains the code that creates the current version of the database schema.  Another option would be to just set the installed_version to 0 and have a case 0: block that creates the original database schema that all the modifications are applied to but I believe it is faster and cleaner to just create the current schema rather than run through all the old ones applying one updated at a time.  While probably not an issue if you only have a few changes, as the number of modifications grow, so does the time involved in the creation of the new database.

And there you have it.  A simple straightforward way to make sure that your database updates go smoothly across updates of your application code.

One final note is that you don’t have to call this each time the app runs, but only the first time the database is used.  Now I suspect in most cases, the database is used every time the application is run but if there are sections of your app that don’t use the database, this check can be delayed until first use of the database.  This allows the app to load up a bit faster, although the difference may not be noticeable.


[1] There is a fourth possibility and that is that current_version is less than installed_version.  But if you’re properly incrementing current_version then this should never happen unless you roll back a distributed version of your application.  If you do that, all bets are off and you’ll need to find another way to handle the update.  Of course this is only a worry if the older code can’t work with the newer schema.

This post originally appeared on my old Programming Space blog.

Headaches of Switching to Mac OS X for Development

When you’ve been working primarily in one environment for nearly two decades, switching to a new one is never painless.

This is the fifth of several articles about my  journey developing an application for NASA’s Fermi Gamma-ray Space Telescope mission.  Earlier articles can be found here:

Let me say right from the beginning that I have nothing against OS X.  In fact, there are many nice things about the operating system and it definitely looks nice.  It’s just that I’ve been using some form of Linux (mostly RedHat and it’s derivatives) since 1997 and was using VAX and Solaris before that.  That was for school and work.  At home I run Linux and Windows.  I’ve never owned a Macintosh computer although I’ve occasionally used one here and there through the years.

I knew when I started developing the mobile app, I would eventually find myself on a Mac for development work because of Apple’s restriction that iOS apps can only developed on OS X.  And I knew there were going to be adjustments to be made.  Here are the ones I ran into that caused me the most grief.

Focus follows mouse

The first was the lack of “focus follows mouse” behavior on OS X.  I’ve spent the last 15 years with my windows set up so that when I move the mouse, the window it is over has focus and I can immediately start typing.  Many times I’ll have multiple terminal windows open (sometimes with just a line or two showing) and slide the mouse from window to window and launch programs.

No longer having that as an option definitely caused me several false starts and a bunch of retyping.  It wasn’t so much of an issue for the mobile development and that was mostly being done in a single IDE plus the emulators, but for my other Mac project, which involved a lot of building and compiling and running tools in the terminal windows, it was definitely causing a hassle.

I can understand completely why OS X doesn’t have this as an option.  The windowing system design with the menu bar always at the top of the screen and not attached to the individual windows definitely makes the idea of focus follows mouse undesirable.  It’s just something I’m used to that I miss on the Mac.

Command vs Control Key and Muscle Memory

I think of all the adjustments this one has caused me the most grief.  My hands have been trained, over the past two decades, to know exactly where the Control Key is located for all my keyboard shortcuts and the spacing between that key and the other keys in the command.  All of those same short cuts, on a Mac, use the command key which is one key over and which I still continue to miss for the first hour or two working on the Mac unless I make a conscious effort.  Eventually, I remember that I’m on a Mac and hit the right key, but in the mean time, since my IDE responds differently to Control-C versus Command-C for example, I’m having to hit a lot of extra key strokes to recover form my mistake.

Middle Mouse Button ≠ Paste

This one I use extensively and it’s absence drives me batty.  Instead of doing this:

  • highlight selection
  • move mouse to target window
  • click the middle button

I have to do the following:

  • highlight selection
  • Command- (not Control-) C to copy
  • move to the new window
  • click to activate the window (remember, no focus follows mouse)
  • Command- (again not Control-) V to paste.

In my Linux windowing environment, this works everywhere, no questions asked.  In fact, I can’t think of a single place where I’ve not been able to use it.

On the Mac, it does work (sort of) in the terminal windows.  But in the Titanium IDE it doesn’t work at all.  Maybe it works in other places but I don’t use much else right now and where I do spend my time, it doesn’t work.

In the terminal windows, it works like on Linux but with one exception.  When you middle click, it doesn’t count that as clicking to activate the window.  A lot of times I’m pasting in a command and I then want to hit Enter and have it execute.  On the Mac, the Enter key keystroke is captured by the window you copied from, not the one you just middle clicked in.  You have to middle click and then left click to activate the window you just pasted to.  Maybe Linux does the same with click to focus window behavior but with focus follows mouse enabled, the window I’m in is the active one so it’s not an issue.

Fuzzy Screen

I’ve seen flame wars and huge discussions on the font rendering engine on the Mac vs. Windows vs. Linux and I don’t intend to reignite those, but for me, the Mac screen is fuzzy.  I should say that I’m using an external 24″ 1920×1200 monitor on the Mac (it’s an HP LA2405wg)  and it is being driven by the Mac at it’s native resolution (the Mac is a MacBook with a 1440×900 LCD screen that is just way too small for a guy that uses dual 1920×1200 screens on all his other systems).  I get used to it after a bit but whenever I have to be switching back and forth between systems, its definitely an irritant.

Slow Response

I don’t think this really has anything to do with the fact that the computer is a Mac, per se, other than the fact that the machine has an old Core 2 Duo processor where my other systems are a Intel i7 and an AMD Phenom which are both quad-core and much faster.  I’m running Mountain Lion on a system that originally shipped with Leopard and I can feel it.  Luckily, this is soon to be rectified and I’m getting a new MacBook in a month or two with a faster processor and more RAM.

The other area of slow response is when I do switch between systems.  I have all my computers hooked up to a pair of monitors, keyboard, and mouse via a KVM switch.  For some reason the Mac has issues with the switch.  Many times it doesn’t connect the keyboard and mouse and sometimes just doesn’t grab the keyboard.  It also doesn’t seem to respond very well to the video switch when the screen saver is on.  Again, I think these are mainly issues of the old hardware and are only an issue when I switch to the Mac for the first time any given day.

Final Thoughts

Overall, the experience of working on the Mac has been fine.  I don’t have adapters to do the dual screen setup like I do with my other computers (only one video out on the MacBook) but that typically isn’t an issue since my work on the Mac is fairly focused.

Would I ever switch to a Mac as my primary system?  Probably not, although it’s not out of the question.  If I was doing mobile development full time and had to target iOS, then I might just out of necessity.  But while that’s only a side project, I think I’ll stick with what I’ve been using longer and am more familiar with.

I think the biggest thing I learned by starting to do development on the Mac is how ingrained some habits are and what I took for granted in my usual environment.  It has helped me to realize what is core to my workflow and what is extras provided by the environment I’m working on.

This post originally appeared on my old Programming Space blog.