Getting syncing right
Background info
To my long time customers, I don't need to tell you how many times I've promised I'm bringing syncing to TimeTag, and then haven't delivered. I feel that I owe a detailed explanation why.
Before I do, though, the great news: Syncing is *actually* coming. Seriously. I'm in the middle of writing the code right now and it's working great. Why now? Because I've finally found a service that works so well for it. (Parse.com)
It began this summer when I decided to undertake a new project from scratch, called NoteTag which is going to ship with the launch of iOS 7. It's a fun note taking app that uses color tags instead of regular text based ones like most apps use. I knew that it needed to have syncing because what use is a note taking app if it's only available on 1 device? Since I had experience using Parse before with another project, I figured I'd build a whole syncing backend from scratch and use their platform to do it.
The results were great. It syncs so well from what I can tell. I still need to test a few scenarios (users are great at finding that one thing that will break a system) but so far it just works which is something that I could never get working using iCloud or other third party providers.
So what makes syncing so complicated?
The issue is that there are just so many scenarios that can occur, and if you want to write an app that works well and doesn't frustrate the hell out of users, then you need to code against all these scenarios. Here's a fun scenario:
- Joe pulls out his iPhone, opens up NoteTag, and creates a note. He tags it with 3 tags: "Recipe", "Food", and "Shopping".
- The note is now created online, and in the process of saving the note, the system has to ask: a) Do these three tags exist online? they should, but if for some odd reason they don't, do we need to create them? b) If they do exist, we should create a relationship between them and the note.
- Ok, so now a note exists online with 3 tags, and on his device with 3 tags. But Joe decides that "Shopping" tag is not one he wants to use for this note. So he untags Shopping. But he doesn't stop there. He also, in this moment, decides to add a new tag instead called "Delicious".
- The app now tells the cloud backend: Hey, this note has 3 tags. But wait, it's not the same 3 as before! It's now "Recipe", "Food" and "Delicious". It runs though each. Ok, found recipe. Ok, found food. Wait, I found something called "Shopping" that's not there. Delete that relationship. And I didn't find this new one, "Delicious". I guess I need to make that one and form the relationship.
If you followed all that, you can see that things start to get tricky. The key is that as an app maker/designer, you want all this stuff to 1) happen instantly and 2) be invisible to the user. These are HUGE challenges to a programmer.
Don't get me started on what happens when devices go offline. I personally like to make apps that work both online and offline, because 1) not every device is an iPhone with a cell connection and 2) even if they all are iPhones, not every cell connection/wifi connection is operable 24/7.
So when you introduce offline operations into an app, you need a way to store all these changes and then push them in order of when they occurred. What gets really fun is when two devices are both offline, both make changes, and then both come online at either the same time, or in 'reverse' order of when the human made changes. What information wins? Do you make copies of conflicting records, or mush them together, or do you just kinda pick one at random and hope that's what the user wanted?
This is the kind of thinking that goes into building a really solid backend for an app, so that records stay the same across numerous devices. And since I'm just a lone programmer with a lot of projects (we actually added a second programmer recently!), it's a lot of work to do with my limited time.
Ok Ok, syncing is tough but...syncing is coming to TimeTag soon, right?
Yes. I hope so. I really do, because I want TimeTag to have syncing just as much as my users do.