Snowsuit Zine // issue 06

Table of Contents

What is the Long Term Cost of Clouds?

Google, Microsoft, and Amazon are battling it out over being the preferred cloud provider among developers. In the end, it's unlikely there will be a clear winner, however even if all three providers have an equal share of the market it means that a significant chunk of computation in the world will be owned and operated by three companies. Consolidation gives the cloud vendors the ability to take advantage of economies of scale and offer customers low prices. The cost, however, is freedom.

In the 90s, a start-up would have to spend a substantial amount of its capital on infrastructure components like storage and databases. The emergence of open source changed all that. In contrast to the 90s, only a few years ago the idea of not using an open source database would be nearly unthinkable for most start-ups. Not only would the cost of using a proprietary database threaten to bankrupt a small company but open source databases provided a quality solution at a low price that one could trust without vendor lock-in. As a result there has been a surge of start-ups that have changed the world (for better and for worse) that have built their foundation on open source.

Today, many organizations use proprietary solutions such as Amazon's RedShift, DynamoDB, or Google's BigTable. These databases are battle tested at scale and the feedback cycle on changes is short because they are provided as a service. Amazon has provided solutions for other portions of infrastructure as well, such as Kinesis (Kafka), SQS (message queue) and ELB (HAProxy). Microsoft and Google are providing their own versions as well.

In the short-term, the services provided by cloud vendors have been a boon for innovators. It's cheaper than ever to start a technology company and it will only get cheaper with the three companies working hard to beat each other. However, the long-term may not be so rosy. While Microsoft has gotten a personality make-over, it has a long history of exploiting its position of power to is own benefit at the cost of others. Google is famous for putting "Do No Evil" in their charter, but is currently under investigation in the European Union for alleged anti-competitive practices. Amazon cares a lot about their customers, but has a history of treating their employees poorly.

Google and Amazon owe their current positions, in part, to open source. Microsoft's attitude adjustment is, in a large part, thanks to open source products chipping away at the empire. Yet the future these companies are giving developers is one of cheap services under their proprietary solutions. Prisons many developers have been happy to lock themselves in. Why developers that would scoff at using Oracle are happy to use DynamoDB seems to be a mix of cheapness and novelty, with something like DynamoDB being cheap and new. For many people it appears that open source is not an ideology but a cheap source of high quality software and will use any option that can reduce the overall cost-of-ownership.

One can see a possible vision of the future of software in other existing markets. Intel processors are ubiquitous and become more closed source each year. Apple makes it extremely difficult to run any other operating system on their devices. Graphics card manufacturers are notorious for producing low-quality drivers for non-Windows environments and not releasing the specifications. The innovations which have given rise to countless advances are ossifying.

There is little question that Amazon, Microsoft, and Google can provide cheap and high-quality services at scale. Their size gives them the ability to exploit efficiencies that smaller players cannot. However, there are examples of people preferring to add inefficiencies into a process in order to preserve some freedom. The organization of the United States government is an example of this. It is designed such that it is expensive for various arms of it to interact in order to ensure it does not become too powerful. In a similar vein, many countries have strict anti-monopoly laws. In these cases, the people have decided that giving a small group of people excessive power is more expensive than not.

Human nature is, if anything, known for being cyclic. Even if open source is brought to its knees by the cloud providers, chances are in a decade or two it will be revived by the next generation, who is always looking for something different than the status quo.

Monthly Consumption

Books

  • Flash Boys: A Wall Street Revolt by Michael Lewis (link)
  • Making Money: A Novel of Discworld by Terry Pratchett (link)

Papers

  • Time, Clocks, and the Ordering of Events in a Distributed System by Leslie Lamport (link)
  • Functional Pearl: La Tour D'Hanoi by Ralf Hinze (link)

Articles

  • The Log: What every software engineer should know about real-time data's unifying abstraction by Jay Kreps (link)
RSS