Snowsuit Zine // issue 05

Table of Contents

NoSQL is Also About Availability and Latency

In the 1980s, Sun Microsystems coined the phrase "the network is the computer". While it's unclear exactly what they meant at the time, the vision was one of thin clients and centralized computation, or mobile phones and SaaS as it is today. In the era of The Cloud, the dream of centralized computation is being realized. The burden of centralized computation, however, is not just that services might need to scale well but that the services need to be highly available.

When it comes to NoSQL, many people focus exclusively on scalability. Outspoken critics argue that most companies do not need the scalability that NoSQL claims to offer. For example, recently Olery blogged about their transition from MongoDB to PostgreSQL and some of the comments on reddit and Hacker News reflect this.

dasil003:

The excuse of poor scalability is just a rationalization that some kids who didn't understand SQL used to justify using something that has a shiny marketing page that they ignorantly found more viscerally appealing.

shadowdude777:

The lead architects keep assuring everyone that it's more "scalable" this way, but you can tell everyone is aware of the fact that we'd be far better off with Postgres.

When it comes to scalability, the critics are probably correct. Most fledgling companies do not have scalability problems and may never have them. However, not every problem is a scalability problem. The consolidation of computing means that services need to be available and responsive as much as possible to give users an enjoyable experience even if the number of users is small.

Existing relational databases do not have a compelling story for SaaS. Their design goal is consistency at the cost of availability or latency in a clustered setup. That is not to say one cannot keep a relational database up for long periods of time or that they have high-latency, but rather that they become more expensive and complicated as the system grows geographically for availability.

Take the standard multi-machine relational database setup which consists of a single master and multiple slaves. All writes will be funneled through the master even if they originate on the other side of the globe, increasing the latency. Unless losing acknowledged writes is acceptable during a failure, the master will synchronously replicate transactions to slaves, incurring another latency cost if the slaves are in different data centers. If anyone is partitioned from the master, they cannot perform writes. Finally, handling automatic failover at a global level with relational databases is a non-trivial problem.

Facebook and Google have famously addressed these issues with MySQL. However, those solutions work by building layers on top of MySQL rather than being out-of-the-box. They also look similar, with similar trade-offs, to some NoSQL solutions.

Not all NoSQL databases address availability and latency. MongoDB, for example, prefers consistency (allegedly) over availability. Redis Cluster is neither consistent nor available. However, Dynamo-inspired databases such as Riak and Cassandra aim to be highly available. As Sun found out too late, centralized computation changes the landscape. Rather than organizing more data on single, big machines, it's spread out geographically on many small machines. Rather than having a few, highly reliable, machines it's many machines with questionable uptimes. Does every SaaS need to be on a NoSQL database? Probably not. However, when contemplating the choice, one should consider not just the scalability of the system but also the SLAs around availability and latency.

Articulations

Eating A Different Apple

Apple, as we see them in 2015, stands apart from its peers in computing. By the standards held by pro-business thinkers, Apple is unmatched in terms of both performance and gains. By the standards held by artistic minds, Apple's beauty and craftsmanship are the unmatched traits. For the hackers, the experience is mostly a Unix thing. Most of what we care about works fine if we stay inside the Terminal.

Developing for Apple used to mean one accepts Objective-C and Xcode and goes for it. The release of Swift is a much needed upgrade. Both work as examples of Apple putting their opinion first. Ruby comes on Macs by default, as does Python. Apple provides them knowing that these tools don't compete with their own. To build for Apple is to put Emacs and freedom of language choice away to open Xcode and write Swift.

To understand how non-developers think about the closed system, consider another device: a luxury car, like a BMW. A BMW comes with a curated set of features, like German engineering and leather seats, which are sold to a small but lucrative segment of the car market. Not being able to easily hook an iPhone up to the stereo is seen as though BMW just haven't figured out what the experience should be yet, but probably will soon.

Now consider it while looking at Apple. They have standards for what can be placed in their App store with an app review process. Some see this as closed, but a typical user sees it as though Apple curated the products available in their store, a notion we already accept from just about every other kind of store. A strong brand comes from a strong sense of what the brand believes in, so from Target to Apple to BMW, the brands express their personality by what they sell.

Apple devs have a history of the same

Apple's developer community is used to this. They've been using Xcode and Objective-C for years. Further, Apple exhibits behaviors hostile to the open source community, such as implementing their own form of anything that begins to thrive. AFNetworking created a way of thinking that Apple brought into their tools, along with improvements only they could make, such as making HTTP calls in the background when the app is not in focus.

The result is an almost non-existent open source community. From the OS, to the developer tools, to the MVC pattern provided by Apple's UIKit team, Apple has chosen the one right way to do everything. If holes in their thinking are found, they'll find a way to take absorb both the functionality and credit anyway.

Open source thrives when other conditions are true, such as an embrace of many different approaches. It is amusing to consider that a company that prides itself on thinking different also goes to great lengths to control the experience of being creative.

Perhaps BMW engineers feel something similar.

Things might be changing

Along with a change in leadership comes a change in culture. Overall, Tim Cook has been overwhelmingly more inclusive, bringing more external talent inside Apples processes than Jobs ever did. For example, Cook acquired 29 companies in first 9 months of 2014. A quick glance at the public record of Apple's acquisition history to see the huge uptick. Note: only the publicly announced acquisitions are listed.

Cocoapods and Carthage are both attempting to create an open source ecosystem, differing mostly in the packaging of code. Cocoapods is a Ruby system that does a lot, including some of what Xcode does. Its approach is basically to create a lib directory and put code there in a way that is visible to Xcode. Carthage is more minimal, written in Swift, and builds framework files, Apple's format for including bundles in projects. Either way, the packaging problems common in any open source community are finding solutions.

The arrival of Swift couldn't be better timed. Functional programmers feel at home and Objective-C programmers are beginning to appreciate the safety provided by the type system and the huge reduction in code that it takes to express an idea.

Meanwhile, Apple as a company has doubled in value since Cook took over and they keep releasing new devices, all of which will have API's for Swift. One can program phones, pads, laptops, watches, all in Swift. It seems likely we'll see a future where those who advocate monads today can express them in Swift while writing software the transports living beings.

Monthly Consumption

Books

  • Thomas Jefferson: The Art of Power by Joe Meacham (link)
  • The Soul of A New Machine by Tracy Kidder (link)
  • It's Complicated: The Social Lives Of Networked Teams (link)
RSS