The open source ecosystem, part 1

It is no news that open source has challenged many established paradigms in the software world, and outside of it too. Behind these two simple words, "open" and "source", lies a complete philosophy of collaboration, transparency and scholarship, that does not only apply to the production of software:

• open source hardware makes available blueprints of electronic and mechanical devices for public usage
• open documentation makes available the source documents for ebooks and other digital documents
• open source music provides the building blocks for music works: samples, tracks, sessions
• open science where scientific experiments results are not hampered by hidden agendas and intellectual property protection

Beyond the availability of blueprints, the open source ecosystem relies on a process that can be described as "inside-out" in comparison to traditional development processes in any of the above disciplines. Here are its most obvious characteristics as they apply to software:

• Need-based: every successful open source project can be traced back to a programmer scratching his own itch. Because we're all humans, this initial need is necessarily felt by other people, programmers or not, who adopt and enhance the software to fulfill it. The need is often of pragmatic nature: if a prior system exists that satisfactorily fulfills the need, the incentive to build a new system - a taxing endeavour - will not be there. Best to use the existing system and move on to achieve whatever the original need was there for.
• Community-based: beyond the core development team, which usually acts as a gatekeeper to any modifications made to the source, lies a diverse community of users who play different roles in the evolution of the system. Beyond using it to fulfill their pragmatics needs, a fraction of the community is actively involved in enhancing the software because they recognize that participation is key to obtain the specific features that they individually need. These active members bring their own skills to the development of the system, including testing, documenting, coding, design, marketing, administration, legal counsel, etc. One of the main factors when evaluating an open source system is measuring the size and activity of its community.
• Feedback-based: with community comes feedback, debate and discussion. Features, bugs and suggestions are discussed on publicly-available forums, mailing lists and issue trackers. Without feedback from the community, the core team has no way to focus on the important (pragmatically) issues.
• Cooperation-based: cooperation occurs among the community, who are otherwise perfect strangers, but also across open source projects. In fact, an open source system is rarely designed to perform its function without the presence of dependencies, which are other open source systems, complete with their ecosystems, that constitute its building blocks. This entails cooperation across core teams and communities to achieve the pragmatic needs. In rarer instances, we also find cooperation among communities that work on "competing" systems. Cooperation here takes the form of knowledge sharing and work on common infrastructures.
• Nonlinear: many threads of activities are typically going on simultaneously in an open source ecosystem. Because community members routinely undertake new initiatives, it is not possible to centralize management and oversight of all these activities. It is unlikely for any one member to be aware of all activities. To avoid complete chaos and system instability, the core team defines a release management process that is constantly being refined - in the successful cases - as the ecosystem grows.
• Internet-based: Open source would not exist without the Internet. The near-global availability of information is the key enabler to community-forming and inter-system linkage.
• Meritocracy-based: based on the achievements of community members, their authority rises among the community. The core team accepts new members who have demonstrated both initiative and technical skill.

To me, these characteristics are indicative of an evolutionary, organic software development process that is at odds with most tenets of software engineering, as it is practiced in private organizations. The adoption of open source systems in the commercial and governmental domains is a witness to the success of this approach.

In the next parts, I'll distinguish between classes of software and where open source fits in the software stack. I'll also talk about the viability of business models for commercial companies that adopt and produce open source software.