TLDR nr.3: The cathedral IN the bazaar

2019. június 14. - Floorshrink

TLDR: Open Source SW development has changed. Once a protest against closed source SW with questionable price-performance characteristics, initiated by amazing fellows in dorm rooms like Linus Torvalds, today it is driven by a handful mega scale SaaS players to support their own services and used as a tractor beam to pull traditional enterprise IT closer to their core offerings. As a byproduct this change is going to turn the entire Open Source movement upside down and will render some of the top HW players largely irrelevant in the next ten years. The cathedral walked into the bazaar and transformed it.

CATB statements with question marks - something has changed

Eric S. Raymond published his essay The Cathedral and the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary in 1999 (CATB for short). He used Linux as an example (besides his own experience with Fetchmail) and came to the conclusion that Open Source SW development has a fair chance to produce better products than its traditional closed source counterparts. I sail to stormy waters with the observations below, but I risk them anyway. (Parts in italic are quotes from CATB)

“The development of the Linux operating system by a loose confederation of thousands of programmers -- without central project management or control -- turns on its head everything we thought we knew about software project management.” Today most impactful OSS projects (see Appendix 1 and 2) start their life as an in-house development effort targeting technical challenges related to the core business of SaaS giants. Later – once the problem is addressed – they are shared with the broader world as open sourced technologies. In fact, most contributors to the Linux kernel today come from IBM (Red Hat), Intel, Samsung, Facebook and Huawei. Even Raymond acknowledges this: “It’s fairly clear that one cannot code from the ground up in bazaar style.” Linux was built on the foundation of Minix, an early Unix port to i386 machines.
‘‘Given enough eyeballs, all bugs are shallow.’’ Take a look at Heartbleed or the CVE vulnerability lists, and you will find that some Linux distros have more distinct vulnerabilities than the once despised Windows. The really bad thing is that governing bodies (PMCs) of certain Open Source projects make decisions that break contracts (the API-s used by other projects) that render the end product - using this component - fail. Raymond’s argument works WITHIN a project but falls victim of the Conway’s law when it gets to inter-component cooperation.
‘‘Plan to throw one away; you will, anyhow.” (Fred Brooks, The Mythical Man-Month, Chapter 11) Or, to put it another way, you often don’t really understand the problem until after the first time you implement a solution. The second time, maybe you know enough to do it right. True, with a caveat: for each building block you will end up with 2-3 functionally overlapping, but slightly different and surely incompatible implementations. When someone thinks that innovation in a given project stalled, she forks the project thinking little about compatibility or a smooth migration path from the old to the new. Tough luck, Mr. Customer.
“Hackerdom has long explicitly recognized "egoboo" (ego-boosting, or the enhancement of one's reputation among other fans) as the basic drive behind volunteer activity.” This phenomenon is still at work albeit with major problems: when egos of developers working on different OSS projects collide, the product and eventually the customer suffers. Even more painful when developers ignore the voice of the customer coming through product management. The mirroring of this sentiment is when product management reports INTO Engineering. Auch …

The root cause for these changes is not in technology nor in process, but in the business model. We have to differentiate between the value creation and revenue creation since the two are not identical in this case.

The most important OSS projects are infrastructure building blocks (operating systems, containers, databases, SW development frameworks, scheduling, provisioning and monitoring tools) BUT NOT core business applications.
The revenue streams of the greatest contributors do not come from selling these innovations – while they could not create their mega scale core services without these technologies. Facebook, Netflix, Twitter, Lyft, Uber etc. are not “de jure” SW companies, even Google, the nr. 1 OSS contributor makes the bulk of its revenue from advertising, not from SW subscriptions. “De facto” all of them are.

The original value creation model

The following chart is from the presentation titled Open Source Projects and Product Management - Need, Pain or Useless? by Patrick Maier and Holger Dyroff. Here the OSS project is driven by delighted individuals striving for freedom, in most instances contributors to the code are users of the same code (in sync with CATB). The product uses the project as a source of innovation and give back stability (bugfixes) in return. The revenue comes from customers of the product (not detailed if this is from support or subscription) The investment is through improvements into the code base (anything that the community may not find interesting, therefore would not touch, while customer do care about them).

The new value creation model

In the new model SaaS and public cloud providers create an SaaS offering that is the core of their business. They are after millions of consumers, hence they run into technical problems that no Enterprise IT have ever experienced. (their scale dwarfs even the largest enterprise IT shops). The new challenges require new solutions, hence the foundation of scale out architectures, containers, new development frameworks and big data platforms emerge. The mega players fund the creation of these platform components from their core business and pass them on to the open source community and specialists. Enterprise IT who have been engaged in a never-ending struggle to justify infrastructure investments, jump on the free cool stuff and bake them into their environment, thus paving the way that eventually will lead to moving their payloads to the very same cloud providers.

As always, I will appreciate any feedback or comment.

_____________________________________________________________________________________________

Appendix 1: Major contributions to the top-level OS projects by original vendor:

Facebook: React, React Native, Presto, RocksDB, Cassandra, Torch, PyTorch
Google: Android (an ARM based Linux fork for smartphones), Chromium (web browser), Tensorflow (math library for machine learning), Kubernetes (container orchestration system), Angular (development framework), Polymer (JS library), js (JS framework), BorgMon (forked as Prometheus (Kubernetes monitoring and alerting))
Yahoo: Hadoop (based on the MapReduce white paper from Google), Pulsar (pub-sub messaging system)
Red Hat (now IBM): OpenShift (container management platform)
Microsoft: Visual Studio Code (source code editor), .Net Core (developer platform)
Netflix: Chaos Monkey (service resiliency checking tool)
Airbnb: Airflow (workflow scheduler)
Oracle/Sun: MySQL (RDBMS), Hudson that forked to Jenkins, (build automation server)
Lyft: Envoy (distributed services proxy for applications)
Twitter: Zipkin (distributed tracing for latency problems in microservices)
Uber: Jaeger (distributed tracking for microservices based applications)
University of Berkeley: PostgreSQL (RDBMS)

Appendix 2: The digging I did before I dared to state anything

There are multiple ways to answer the question who contributes the most to the open source movement. One way is to measure the number of contributions (commits on Github) or the number of employees contributing. Another way is to track the finalists of the BOSSIE awards and reconcile them with several lists compiled by various organizations like CNCF and Datamation.

BOSSIE Awards (Best of Open Source Software Award) from Infoworld:

Rankings from Felipe Hoffa and Fil Maj based on the number of employees of a given firm contributing to OSS repos vs. the total number of contributions.

A handful of subjective lists:

A few articles and talks on the OSS business model

Examples to prove that OS SW is not exempt from bugs: