Why You Need a Strategic Data Service

27 points by jasonkolb 11 years ago | 8 comments
  • VonIgelfeld 11 years ago
    Now I understand the value of data, but what most companies miss is the value of consumer privacy. We like our privacy and will pay for it if we need to.
    • ianfernz 11 years ago
      Typical data capture and analytics isn't like Prism. Seriously, you overestimate your own importance. Facebook doesn't care what you individually do, it's all about collecting statistics about everyone so they can better reach the groups you're a part of.
      • hengheng 11 years ago
        I know, and I'd rather the service be paid for by myself than by a secret service.
    • jrullmann 11 years ago
      The author says that different data stores are good at different things, so we need to use multiple data stores in our application. He proposes that a data service layer can abstract these implementations, making it easier to swap out data stores as needed. I think that's a good idea. Separating our applications from data store specifics is a big reason why we use ORMs and ODMs today.

      However, I think there are challenges with this polygot data store architecture that he doesn't address. Each addition requires due diligence to understand its CAP trade-offs (which the author mentions briefly), scalability and performance characteristics, how to configure, etc. These are non-trivial concerns even for a single database. It's important to consider these challenges when building out a data store or data service.

      I'd propose an architecture where the data services layer itself exposes different data models to the application, all of which are persisted in a single data store. Given that many data stores use a key-value store under the covers anyway, translating the specific data model down to a single, persistent data store would simplify operations while exposing the desired data model to the application. (As a caveat, this multi-model approach requires ACID transactions to ensure strict consistency when translating between data models.) This approach provides operational simplicity with just one data store and application efficiency by exposing the "right" data model API.

      Full disclosure: I'm an engineer at FoundationDB, a database that provides ACID-compliant polygot data models on a single key-value data store (http://www.foundationdb.com).

      • abekarpinski 11 years ago
        Predictive analytic is the thing I'm most interested in right now. Trying to use past data to predict the success of future product launches. I can say that having your data decoupled will save you some major headaches later on.
        • k0 11 years ago
          Great topic Jason. It seems the footnote links [2], [3], etc are missing something...I'm getting "File Not Found"/custom 404.
          • cylentwolf 11 years ago
            I had the same issue and was coming here to post the same thing.
          • mdda 11 years ago
            Where did the footnotes go?