Is Ember Data just unfinished or fundamentally broken?

The Ember team has big goals. And in a lot of ways, they’re succeeding: with the latest revision of its router API, the core of Ember is stable, flexible, powerful and easy to use.

Unfortunately, the same can’t be said for Ember’s data-management and server-communication component, Ember Data. In fairness, it’s clearly marked NOT PRODUCTION READY. But after attempting to use it for almost three months, I’m starting to wonder if the entire premise behind Ember Data is flawed.

As I understand it, that premise — which I’ll call the “store abstraction” — is something like this:

  • On the client side, you have a mini-database of the models that maps to the models persisted on your server. This local database is called the store.
  • Your app should be able to query the store and not really care if the data has been fetched from the server already or not. If it hasn’t, the store will give you a stub while it fetches the data for you.
  • When it’s time to save your changes, the store should bundle together all of the relevant changes (e.g. a new model and its associations) into a transaction, so that the models in that transaction can be saved or reverted together.

That sounds good, but it turns out that the store abstraction is way too leaky. And not only that, but it doesn’t provide a useful abstraction: the kind that emphasizes what matters and hides what doesn’t for you and your users.

For example, it’s usually pretty important to know when you’re up to date with the “source of truth” (typically, the server - forget about apps that use LocalStorage) and when you’re only seeing data that you happen to have fetched at some point since the session began.

Second, it’s even more important to know when your local changes are persisted in the source of truth. In other words, you frequently need save callbacks of some kind. And 99% of the time, you actually want to treat non-persisted data entirely differently (e.g. an unsaved record shouldn’t appear in a list with existing records, which you can do in Ember but you always have to think about it).

Finally, the conceptual overhead of transactions isn’t very useful when your protocol, like most REST APIs, doesn’t actually let you roll back changes across different model types if one of those models fails to save.

These are just some of the issues that I’ve run into while trying to implement a few forms with a moderately complex model structure and UI. Sometimes, I can work around the framework but that’s a lot of extra work. I shouldn’t need to read every line of the implementation code beneath the abstraction just to get something done, and Ember Data is failing badly on that score.

The trouble with the store abstraction is even clearer if you browse the Ember Data issues list on Github. Many of the “bugs” aren’t of the “you forgot to type var in front of a variable declaration” variety. Instead, they arise from the myriad ways in which the store can fail to correctly represent the actual state of your users’ data. They won’t be easy to fix.

Even if these particular issues (144 as I write this) are fixable, the time spent fixing them will be time not spent addressing the most basic feature requirements. Ember Data still won’t provide validation or server error handling (features that as far as I can tell have been on the roadmap for a while).

It also won’t be time spent future proofing Ember Data for more complex interaction scenarios. Right now, it’s practically impossible to ensure that Ember Data’s store reflects changes that are made on the server by another user or by the server itself. There’s no mechanism for propagating changes to the client using WebSockets or even polling. For example, if a record is deleted on the server, there’s no API for removing that record from the local store. And there are plenty of pitfalls if you try to implement this yourself — largely because the change might conflict with a local transaction state. As a result, Ember Data is really challenging to use for anything collaborative — and if that’s the case, why the heck are you writing a web app?

So is Ember Data doomed? I hope not. There’s been a ton of work done on making it a great way to represent and query data on the client, and the core components (e.g. finding a model and loading its associations) are functional and even pleasant. The big problems largely arise from the complexity of the interaction between the local store and the server, and the conceptual confusion created by using transactions that are more notable for behaving modally (e.g. any one model can only be in one transaction at a time, so too bad if your UI isn’t modal) than behaving transactionally.

While wrapping my head around these issues, I’ve started to wonder if Ember Data would actually work better if it strengthened the concept of transactions into something more like an “editing context” that would let you isolate sets of client-side data changes until they are actually persisted. Models in the base store would not be editable by the user directly — they could only be updated by the server (e.g. after a model is saved or if changes are propagated through WebSockets).

If you’re displaying data, you’d typically want to use the store — for example, in a FilteredRecordArray (basically, a live query). When you want to make changes to a record (say in a “new” or “edit” view) you’d have to create an “editing context” and add models to it, either by creating them within the context or copying existing models into the editing context manually.

The key differences between this and Ember Data’s existing transaction approach would be that:

  • Models could be in more than one editing context at a time, so that you can provide more flexibility in the UI — in the current approach, edit modes almost have to be, uh, modal, because otherwise you’re likely to accidentally commit other changes when the user clicks save on an unrelated form,

  • Using a specific editing context would be mandatory — unlike the global “defaultTransaction” currently in Ember Data, and

  • Changes would not be propagated across editing contexts or to the store (and from there to other views) until they are committed to the server and therefore both validated and persisted.

Although enforcing this distinction between client-side changes and persisted data might seem cumbersome at first, I think it would map better to our concerns as developers and users and, as a result, it would be easier to use.

There’d have to be some kind of notification mechanism so that editing contexts would know when a model has changed in the store (e.g. if you start editing a post and someone else replies to it or updates it on the server).

You might also want the option of either saving in the background (i.e. update the store before the editing context has confirmed its changes with the server) or monitoring the saving state in the foreground (my own preference). Again, you’re introducing a choice that the developer has to think about, but I think this would be a clearer tradeoff and easier to reason about than the current “every change is automatically propagated immediately but may not actually be persisted or validated” approach.

Anyway, I wish I had the skill to implement this idea and see how it works in a real app. The current Ember Data implementation is not working for me, and I’m frankly not sure what to do about that because I love Ember itself.

(Discuss this post on Hacker News.)