frankie-tales

SM-Tools, Copernicus, FOSS and the reasons of inevitable choices and drifts

2026-02-25T15:00:00Z

Here at work, we develop a series of tools for geospatial processing with multiple goals, expected maintenance durations, different scopes, generalization needs, and motivations. Note that about 10 years ago, here in Europe, a completely new approach to upstream and downstream services for Earth Observation began: ESA changed its data licenses, and distribution and access modalities entered the Big Data era.

That even changed the data approach for academic institutions and triggered a major shift in the daily work of researchers, with access to a tremendous volume of weekly data available almost just in time and worldwide. Of course, such a change also impacted us, and we had to adapt our processing and storage capabilities to the new era.

One of my side projects in that regard is SM-Tools, which consists primarily of a collection of support tools for running our internally developed soil moisture algorithm using SAR satellite data (SMOSAR). One such tool (now named smt_copernicus) began (and evolved with multiple restarts-from-scratch) more than 10 years ago, when the Sentinel constellation started operations. Its purpose is to search for and download satellite products from Copernicus archives using multiple criteria, and to maintain an internal geospatial database of these products, along with all derived maps and ancillary data. This is only one component of a system that should be able to process large quantities of multi-source data on selected areas of interest, create downstream products, and calibrate and analyze results by comparing them with field data. The clear final goal is to achieve new findings in satellite data analysis, supported by extended processing worldwide, and to introduce new algorithms.

This is a long-term goal that unfortunately runs up against short- to mid-term difficulties of accessing archives that are not under our direct control. The sad reality is that in the last 4 years, the Copernicus archive access modality changed 3 times, and in the previous period, Copernicus also changed policies and modalities in progress (e.g., by introducing online and offline products, changing formats, etc.). Geospatial communities are small enough to encounter more practical difficulties than expected in such operational conditions, and this is now an almost weekly experience. We now have to chase other parties’ changes more often than we did in the past, rather than working on our own goals instead.

For instance, until 2023, the main package used for accessing the Copernicus archive was the Sentinelsat Python package (developed by an handfull of willing scholars starting in the summer of 2015). It became abandonware that year when people discovered that the protocol changes required rewriting most of the package from scratch, including all test code and mockings. That happened again at the end of last year. Incredibly enough, all access protocols have been an ongoing affair since 2023, even for non-secondary details, and required frequent adjustments to avoid unexpected breakage in download and processing pipelines. That for sure does not encourage community-supported FOSS solutions. Exactly in 2023, when I discovered our Sentinel-related tool had to be deeply changed because of the lack of the obsoleted Sentinelsat package, I decided that enough was enough and managed to migrate from Python to a fully self-supported Perl reimplementation: one of the things I always hated in the Python ecosystem is the excessive (for me and our purposes at least) speed in deprecation of consolidated packages and features, and the prospective of having to chase after unexpected changes in both Copernicus AND Python sides was out of question. My experiences with Perl have been much less annoying in this regard, with scripts running perfectly even after 20 years since they were written. Let’s consider this the old-school approach: if something is working, don't touch it without a more than valid reason, and even then, think twice before you touch.

In the meantime, around 2019, another independent effort started to support Copernicus access, along with a few other data providers. That’s about 4 years after the original Sentinelsat project. Timing in this case is essential to be considered. EODag is a single-company FOSS product that has been actively developed since then, but may have been considered stable 1-2 years later. Of course, it only provides the usual access layer, and using that package implied, at the time, replacing Sentinelsat with EODag as a base layer for searching and downloading only, while performing other tasks afterward with other self-made code. Before 2023, using Sentinelsat or EOdag was equivalent in order to perform the same task, with very little advantages.

Note that both tools were, in any case, pretty adequate but not enough for our goals, and also had a few defects (or a lack of flexibility, to say better) to manage in some creative way. That has been one of the reasons for not replacing Sentinelsat with EOdag in 2023. The other major one was that the idea of replacing a small package with another (as for Sentinelsat, there are just a couple of main contributors to the codebase, along with a good number of pending issues and PRs for such a kind of product) is probably not the safest one to avoid problems in the near future, when Copernicus will change things (again, see the next Earth Observation Processing Framework EOPF data format — Zarr). And of course, EODag is written in Python, and I already expressed my concerns about that.

Even if you like it or not, nowadays, the concrete alternative to adopting small FOSS projects to perform basic tasks is to use AI tooling to create a perfectly (or almost so) tailored implementation for the target task. While in 2023 I had to rewrite from scratch (in maybe a month of work with some fixes to Geo::GDAL::FFI package for a multi-threading issue) an implementation of a multi-threaded tool for accessing the Copernicus archive and maintaining an internal geospatial database of products consistently among multiple generations of the archive, I implemented the STAC protocol variant in a few hours instead, thanks to Claude Code-based patching and my reviewing and tests for the resulting codebase. As said in this post, currently, the cons of adopting a small third-party FOSS solution outweigh the pros, particularly regarding the resulting technical debt, compared with a well-conducted self-consistent AI-based development process.

I’m seeing in my own side projects exactly the mirror of what will probably be the reality of FOSS projects in the near future, in practice, as I mentioned in the previous post. Relatively few major/interesting projects will be adopted by others and attract contributions, while most codebases will become pure one-man shows, with AI tooling.

A significant part of geospatial processing involves data procurement and processing, i.e., refining and preparing data and images in order to collect, filter, and process large volumes of data for subsequent analysis. This is the most annoying and repetitive part of the process, and also often the most time-consuming. In my experience, working on those tasks is probably the most effective way to use LLMs through a spec-and-test-driven design. Whether you like it or not, it is the most immediate way to produce working code by iterating with a chain of thoughts and accurate reviewing of results, including a decent test coverage. As observed in Antirez's experience, the AI agent also had the nice ability to retain my Perl style (which is not secondary, given that Perl has many programming flavors and variants).

Maybe the final results will be an increase in quick-and-dirty codebases, but for many scholars, it will be a major simplification of their lives. In most domains of science, coding activities have been seen as an inevitable evil: they are a tool, not the primary goal, and even before the advent of LLMs, most scientific codebases were far from something to be proud of. The Copernicus attitude to FAFO will also encourage such approaches. Simply because scholars don't have time to waste chasing changes introduced by this or that data provider or company when contracts change.

AI sloping attitude? No, simple survival instinct.

Is AI driven coding the start of the end of mainstream FOSS?

2026-02-04T20:00:00Z

Someone on Mastodon (I’m sorry, but I don’t remember who exactly) published a short post that pointed to a rather technical economic study of the impact of AI on FOSS software development [1]. It is no secret that the AI debate is highly polarized, and the enthusiasts for the current trend in AI applications in the IT domain are at least as numerous as those who are concerned/skeptical. What is certain is that no one can, in the long term, prospectively evaluate the impact of AI on society, particularly in the IT world.

The main thesis of the paper is that AI-based code production will end the mainstreaming of FOSS software, as we have learnt over the last 15-20 years. The paper begins with well-known episodes from recent history (specifically, the Tailwind saga [2] and Stack Overflow's near-death experience [3]).

Of course, the paper presents a theoretical economic model to evaluate a possible impact scenario for the FOSS production model, which could or could not come to fruition, depending on the assumptions made.

My honest opinion is that a conscious and accurate use of AI can accelerate development. That is, in a bad and good sense, I mean directly on the basis of the experience and skills of people who use such models. Therefore, we are both seeing slop and high-profile creations with the aid of AI. Maybe slop contributions are more prevalent simply because mediocre developers are the majority, and mediocrity is the backbone of enterprise production (because it is the most replicable and independent of contributors and their capacities).

Like it or not, modern software industries do not need, and fight against, too much creative approaches. Enterprises need aurea mediocritas, not isolated geniuses. Also, depending on third-party creations, it apparently reduces the enterprise's technical debt because it is typically shifted onto someone else's shoulders. Of course, this is an approach that works until it fails miserably when such a third party disappears, changes its license model, changes its mind about the product, changes its APIs, and so on.

That said, one clear consequence of using AI helpers in coding appears to be the progressive disappearance of many packages, modules, and libraries, which can be easily replaced by AI-generated creations tailored to the task. Just to cite one practical example, Tailwind nowadays could be easily replaced by CSS and simple JavaScript components, with the obvious advantage of not depending on yet another third-party-controlled piece of code that could be subject to abrupt changes from one version to another without notice and break existing codebases. At the same time, Tailwind themes can be generated by AI without even consulting its documentation (which apparently had an immediate impact on the company's revenues).

Another advantage is that AI-based, tailored solutions would reduce the amount of code from external dependencies that solve problems for others, instead of focusing on the minimal set of features for your own needs (with all the implications of possible breakages arising from such an anti-minimalistic pattern).

Of course, using AI helpers in this way does not reduce the effort required to understand and create new software, but it probably raises the required competence to a higher level, which could be better in the long term, while encouraging quick-and-dirty approaches in the near term. The so-called vibe coding is not a black-and-white concept; it has a lot of grey tones directly depending on the awareness, responsibility, and skills of the developer: as said, it can accelerate in many senses - even to crash against a wall - increasing in an uncontrolled manner the technical debt when in the hands of the wrong individual. Even about that, Anthropic recognizes that AI abuse can negatively impact coding skills and debugging capabilities [6].

Add to this the current very high infrastructure load many networks are reporting, for which the AI bots currently seem to be the culprit [4]. This seems like very strange behavior for such botnets, given that web crawlers have been around since the 90s and should be able to handle infrastructure load fairly well by now. It seems that AI companies simply aren't fair enough on their own, or that the training phases of neural nets are definitely more demanding. Maybe both?

So, what do I see as the future for FOSS development as a whole? I am not a pessimist as in the cited article. For sure, I see fewer small contributions in the long term. Today, there is a massive production of AI-slop-based contributions to many prime-time projects, but I see this as incidental. In recent years, GitHub-based path of honors has been a major self-promotion channel for junior developers, which explains the drift toward low-quality contributions: devs are (were?) strongly motivated to contribute and find in AI slop an easy path to that, by creating personal portfolios. That’s also true for fake security-related reports (see the well-known Curl project case [5] and others). This is, of course, annoying, but in my view, that’s the result of current AI hype and should normalize in the mid-term.

Also, in the near future, I see less and less relevance in FOSS projects that are not sustained by a strong architectural idea, innovation-grade, a large community, and a consistent development effort (much bigger than a few weeks or months of work). That kind of project will become mainly background noise, let me say. Maybe that could impact whole categories of FOSS software: it is not a secret that many language hubs are full of packages/modules of dubious quality, often used because they are available just a use/import directive away. In many cases, such products will simply be replaced by an AI-based reimplementation. If the final result will be better or worse in average quality, only the future will show. For sure, the AIAD will cause a progressive democratization/popularization of the development process, giving average users access to possibilities once unavailable to them: we will probably see the production of a plethora of small tools and workflows built on agents rather than finished, refined products, like it or not.

The result could be an increase in FOSS products at the cost of lower average generality and code quality, with a few high-end, tailored products for mainstream applications instead. But was this really so different in the past? I don’t think so. The true difference is probably the increase in quantity in both sets of products, as potentiated by AI tools: if one does not do her/his homework, the result is clearly garbage, but that was true before AI, too.

“AI gives us the worst and the best - simultaneously.”

(Daniel Stenberg, the Curl Mantainer)

References

How to trust FOSS players and the security implications

2026-01-27T17:30:00Z

More and more, recent (and not too recent) episodes [1-5] nowadays show a hard truth we already discovered in the Debian project since the end of the 90s. A key security principle in FOSS code development is ensuring the trustworthiness of all parties involved, and that’s unfortunately also the weakest part of the whole chain.

Debian adopted a long time ago a tentative GnuPG-based screening of people involved in the project through the so-called Web of Trust. All developers need to participate in eyeball meetings to sign each other's public keys after verifying personal IDs. The more signatures, the more trustworthy. That is generally mandatory before being granted the privilege of uploading to the main archive without review. Of course, trust is not automatic, and each volunteer must demonstrate they have the required skills and good intentions, and embrace the Debian social contract. This is the annoying, often multi-year process to be accepted as a contributor.

This process is far from perfect and may also be subject to abuse, as Martin Krafft demonstrated at a past DebConf, a long time ago. One of the main issues is that maintainers work on upstream software that generally does not undergo such a process to accept contributors. As the well-known sad case of the xz utils demonstrated [6] a couple of years ago, an initial review of pull requests is not generally enough to ensure that the developer or group does not have evil intentions in the mid- to long-term. Also, with the best intentions of sane upstream developers, evil parties can make very creative efforts to hide their malware code. This is magistrally demonstrated in that episode, which did not cause major security issues, only by casualties.

The sad reality is that none of the main programming language hubs is really trustworthy, because literally anyone can register anonymously to upload code and participate in development teams. Core teams at least review pull requests before accepting them to avoid major abuses. To address this, enforcing a worldwide web of trust for all core projects and possibly all software hubs should be considered a mandatory step to improve security and accountability.

It does not resolve all problems, but it helps. A central authority is not the answer, as it could create more problems than it solves. Instead, enhance trustworthiness by encouraging ongoing cross-review by multiple parties. Establish processes that build developers' trust over time through active participation and transparent peer review. While key hijacking remains a risk, this is a separate issue requiring distinct protective measures.

I've written about this related issue before. The shift from distributions to language and upstream hubs shifts software management onto developers and users, increasing the risk of security incidents from malicious contributions.

That said, like it or not, most FOSS products out there are created/maintained by single individuals and micro-development teams with no warranties and questionable durability. I wonder how billion-dollar companies can seriously consider basing their core business in such conditions, a problem directly connected to the broader sustainability challenges for FOSS projects. The progressive spread of the AGPL license and other similar licenses is a symptom of this type of malaise and should be taken into consideration, as they are different aspects of the same problem. Security concerns are another key point that we, as a community, should try to manage better, but my honest thought is that nowadays, predatory big (and not so big, even) companies (as well as public bodies too) that use community-driven FOSS code in an unfair manner, without returning a cent for development and maintenance, are not more acceptable.

Therefore, FOSS communities are not perfect, but many of the culprits are nowadays on the shoulders of companies and public bodies that are still looking out their windows instead of being active stewards and promoting reciprocal collaboration among all involved parties.

Come on, put the money and effort into the sources of your digital profits.

References

Too many eyes or too few efforts?

2025-12-07T16:00:00Z

I recently read a post by Jack Poller about the end of FOSS optimism in creating software in recent years. His thesis is that the myth that the more eyes that look at a piece of software, the higher its quality, is indeed a myth, and that nowadays it is also a dangerous illusion when we concentrate the analysis on security. Commercial software, on the other hand, has processes and resources dedicated to managing security, which in these times of active AI use could make the difference.

I agree with such a thesis at large, but with some differences in declination and sense. It is absolutely true that security requires special skills that most developers do not have and (above all) are absolutely not interested in having. This is amplified in the FOSS communities, where many programs and libraries are not specifically written with security in mind. On the other hand, currently most AI-generated security reports are fake and pure hallucinations (e.g., ask the Curl maintainer, about the overwhelming number of reports in the last few years that are clearly generated with help of AI and are pure garbage). Therefore, having new AI-based tools for screening does not simplify the problem neither for software creators nor for infosec experts. Basically, they still need to do their time-consuming jobs with resources and efforts: there is not an easy escape for that.

Even, if you like it or not, nowadays FOSS software production is organic to the commercially supported one, and they are strictly connected. If the security (and, more generally, the sustainability) of so many FOSS projects today is a problem, it is also a problem for their commercial counterparts. In other words, FOSS development has been mainstream for at least the last 20 years: that implies that the sustainability (and security) problem extends to the entire supply chain for both FOSS and commercial software.

I already dealt with some of those topics in past posts (e.g., have a look here, or here), not referring to security, but to the sustainability of the FOSS ecosystem of independent-but-interdependent projects. This is a matter of responsability for both FOSS mantainers and companies, and not something that could be avoided as a problem for others.

As already written in the past, I can't see a recipe for success, but I'm quite sure that overcomplication of information systems does not help, but is part of the problem. Too many dependencies, complex frameworks, and architectures are sure highways for disasters. And that's true for both FOSS and commercial software. The shattering of code hubs among multiple sources of binaries, packages, images, and distributions is another sure quarry of problems, because they imply a multiplication of possible origins of issues, as well as the need of sustaining multiple teams and replicating efforts instead of concentrating them.

There is great chaos under heaven; the situation is excellent! (Mao Zedong) .

DebianGis anniversary and the power of being a community

2025-10-16T18:15:00Z

A few days before today, 21 years ago, I sent this message to the debian-devel-announce mailing list to solicit helpers in packaging and to oversee the geospatial software stack included in the main Debian archive. After so many years, still there.

At that time, the Debian project had already been around for more than 10 years and even had a few releases behind, but the typical maintenance of software was still a one-person show, except for a few large and complex pieces of software. Even the kernel was managed by a single developer, Herbert Xu. Each developer was responsible for implementing updates, fixing bugs, and releasing updates. Often jealous of such prerogatives and their stated possession of the package or task. While there was already a quality assurance team and an orphaning process for abandoned packages (or inactive developers), these processes were not very common and were even relatively slow and imperfect.

Since that time, team-based maintenance has become the standard approach in the Debian community for properly managing the most complicated software collections. That's at least to ensure proper long-term maintenance, because the average developer can provide only a limited continuity to their efforts, and real life is generally more complicated than the digital one. It is not secondary that packaging tasks can be tedious in the long term, and it is easy to lose motivation. The presence of a working team can reduce the risk of burnout and allow each developer to step down when needed. The most effective team should be small enough to coordinate more easily and avoid lagging in changes and migrations, but not too small: presenting a one-person show as a team work is not a great idea. But too many people in the same team is equally not a great idea for the opposite reason.

In the specific case of Debian, the system is so modular and lacks many interdependencies that it favors the creation of fully independent management groups for hundreds of components and ecosystems of packages. It is not casual that most of the bugs and inconsistencies are concentrated where too many parts need to interact appropriately with perfectly aligned programming interfaces.

Talking about DebianGis and the geospatial software, the key motivation at the time was the lack of a coordinated effort to build and collect together, with consistency, a lot of different libraries and programs that were (and still are) specialized enough and based on hundreds of dependencies, often not within the perimeter of competence of the geodata user. At that time, piling up the software stack of a typical geospatial application was not an easy task for the faint of heart, and most (all?) of the Linux distributions lacked in one way or another.

Anecdotally, I can remember about 15 years ago, the company that sold us a Suse Enterprise-based solution for a geospatial information system that had so many problems in completing the required setup that I finally created a chroot-based environment with a Debian stable plain install to run a working PostGIS DBMS. That was the time when containerized solutions were still far from being supported, so the chroot environment was the most immediate solution to such a problem. A little win for a community-based distribution and its tiny geospatial team, which provides a measure of the problems at the time. A giant step for the whole FOSS concept.

I'm currently much less active in packaging tasks, but still seeing the current team alive and capable of releasing well-supported products more than twenty years later gives me reason to be proud of such a community.

AI artifacts, copyright and electric sheep dreaming

2025-04-22T12:30:00Z

My last post captured the attention of my old fellow Sandro 'strk' Santilli on Mastodon, who sent a provocation about the whole AIAD thing. So, the challenge is accepted.

First and foremost, the whole AIAD issue is a complex and hotly debated topic. The question of whether the training practices of the past and present should be considered fair use is a matter of contention. This is particularly true for existing code bases on well-known repositories and other types of content, a complexity that keeps us all intellectually engaged.

In the specific case of code, the basis for deciding about such a question was formerly stated in the accompanying licenses. Honestly, there is nothing in the current formulation of OSI licenses (BSD and GPL, among others) that precludes such a very special activity. The non-discriminating conditions apply to any human activity, and the neural net training - undoubtedly a human-driven task - is not excluded. The training is a learning type of activity or a sort of.

That said, the NN training is, of course, a massive and intensive kind of learning. But let me consider any artifact created by an AI model as a direct derivative product (this is a forcing of the concept for me, but let me axiomatically accept it). That could be an 'original' product created from scratch (i.e., by direct prompting or by limited documental context) or a direct derivate one (because it is based on previous code proposed as part of the AI context). Suppose you ask any of the big models for a direct clarification of the terms of use. In that case, you will discover that any of their artifacts retain the copyright ownership of the product, but there is a very permissive use that perfectly adheres to the four fundamental freedoms of FOSS licenses. Of course, the results could be based on a previous code base. In that case, its license still applies for such a derivative work, even with the additional copyright of Anthropic, Google, or OpenAI.

This could pose a significant problem if the final software product needs to retain one specific copyright holder, as is the case with most proprietary software or some FOSS ones. Understanding the potential impact of copyright transfer is a key consideration in this context, and it's crucial for us to be fully informed and aware.

Is the process of an AI participating in development really that different from what any average hacker does when participating in FOSS projects? I don't think so. You add your copyright to the existing ones for the parts that are under your direct control and accept conditions of use already defined in a license. The true challenge, in my humble opinion, lies in the changing of licenses. Any ex-post change should start with a note of acceptance by all copyright holders, including Anthropic, Google, etc.

I consider this difficulty a feature, not a bug. I advise against participating in projects with copyright transferring because of the potential for changing the license later when your contributions move out of your control. It happened in the past, and it will happen again.

One should also consider the licenses of many other sources of knowledge and inspiration for developers. How many people know that the default license for StackOverflow code snippets is CC BY-SA? Indeed, how many developers actually add an acknowledgment in their software for such snippets? Or even for snippets taken from sites, blogs, books, or manuals without considering that such sources are even more restrictive for use and creating derivative work? Isn't our full daily work the result of a long learning phase, based on our education and training by books, experiences of others, as well as trial-and-error processes?

That said, let me spend some words about the elephant in the room. In the context of AI and copyright law, the 'elephant' represents the complex nature of AI models and their potential to create original works. Do AI models dream of electric sheep? Well, I don't think current models are pure stochastic parrots, to be honest. I think there are probably dozens or hundreds of cognitive forms that govern what we call generically intelligence, including some emotive and empathic forms that one can also find in a dog, a cat, or a dolphin. One of those forms is probably captured from the neural network model of functional representation, which is also perhaps in common with part of our mind. We are at an average level, much more effective and efficient in those regards, and to be fair, I would also say that hallucinating is a common experience even on the human side. We are much more complete and retain contexts that are wide, like a lifetime. A few people are terrified by this observation and seek refuge in negation or faithful certainties.

We are complex organisms with probably still partially known processes that govern our so-called intelligence, which is physically based on cells and energy in our brains, whether we like it or not. We found a way to mimic part of this complex process, with all limitations of the case. Is this intelligence? I don't know, but at the end of the day, what is intelligence? When none asks me about that, I know perfectly what is, but if you ask me about that, I don't know anymore.

FOSS toxicity, burnout and governance (again)

2025-03-23T15:40:00Z

I recently read with interest the post where Hector Martin resigned as Asahi Linux leader. As possibly well-known, Asahi Linux is the very first Fedora-based distribution where all the hard work to support the Apple ARM M* chip series in the Linux world found its way.

Dismissing the whole thing as another episode of burning out for a FOSS developer would be ungenerous. Some elements are typical in such cases, such as the excessive users pressure on the project. That caused a lot of pain and over-reactions in other contexts, but a big part of the Hector experience is related to issues with the upstream Linux kernel within the more general Rust-in-Linux saga.

Of course, harsh behaviours and bad relations with other people are common patterns in all human activities and often require a very thick skin and big diplomatic capabilities to get positive results. This is not specifically true for FOSS developers only (even if I say that a lot of developers are quite peculiar on the side of human relations, often in a semi-pathological way).

In my life, I have had the opportunity to deal with another very peculiar human category, such as cavers. Even when no money is involved, human beings can create bad relations, internal wars, and unhealthy atmospheres for their fellows. So, I guess it is a standard part of all human-to-human interactions, not a FOSS ecosystem peculiarity. A lot of people are strongly opinionated, and when also passionate, they tend to become intractable.

That said, I will concentrate on a specific part of Hector's post.

[...]
Back in 2011, Con Kolivas left the Linux kernel community. An anaesthetist by
day, he was arguably the last great Linux kernel hobbyist hacker. In the years
since it seems things have, if anything, only gotten worse. Today, it is
practically impossible to survive being a significant Linux maintainer or
cross-subsystem contributor if you’re not employed to do it by a corporation.
Linux started out as a hobbyist project, but it has well and truly lost its
hobbyist roots.
[...]

This is indisputable, IMHO, and it is likely the root of many problems that afflict the Linux kernel community and others. When people start working side-by-side with others who are motivated and sustained by different goals, it could be the seed of big social issues. Linux is not more just for fun for years, and that can create friction among developers. That includes perception about what should be considered acceptable for project sustainability and what should not.

These days, all that includes the use of profiling features for projects that are directly exposed to end-users and definitively the acquisition of personal habits and/or information about the users. This is not the case of Linux kernel, but it is a concrete issue for organizations such as the Mozilla Foundation and its products, and could become a divisive issue for OS distribution projects, as already happened in the immediate past.

Another problem is the wrong perception that a FOSS project should always accept contributions and enhancement proposals, while often it is not due or obvious. Like it or not, no acceptance of contribution is due, and I would add that FOSS projects could (or should) be very conservative in those regards, i.e. answering thanks-but-no-thanks more often than I have seen in recent years. IMHO, this gives evidence to what, for me, is an urgency for too many projects to list here, including, in practice, all programming languages out there.

Adding features over features on a piece of software is almost never a good policy because it increases the entropy of any software product and decreases its usability. The original spirit and goals of the software that possibly decreed its success should always be preserved, and that has been the basis of the original success of the Unix operating system approach, as an ensemble of compact and independent tools that solve single problems in the most general, simple and elegant ways, tools which are able to glue together to solve complex tasks. Now, this simple vision is already a source of friction among developers because maintainers are so often obliged to answer negatively to pull requests and patches for either respecting quality levels or opportunity.

This is a delicate trade-off that should be translated for every minimally articulated project in a written Social Contract to state and explain the goals and rules to respect for both users and developers. In a word, the governance principia of the project need to be defined ex-ante and not ex-post, to avoid misunderstandings and inner fights, as well as avoidable pressures during time. Unfortunately, many FOSS projects tend to delay ad libitum the definition of an explicit Social Contract or introduce that - more often, only a Code of Conduct for contributors is contemplated - when it is too late. Many big projects do not even have such explicit statements, and it is a pity. Users and developers should always be aware of such rules and goals in the life of any FOSS project.

One of the biggest problem nowadays is having such governance clearly established for life, but another is also ensuring that users and developers know its existnce and respect it, the two side of the same problem.

FOSS governance and sustainability in the third millennium

2024-10-11T17:45:00Z

I have long participated in the FOSS community. My first public contribution was the YardRadius project in 1995, a consolidation of the old Livingston Radius daemon and a series of add-ons written by Christian Gafton (RIP) and me. That was some years before the more significant FreeRadius project. At that time, I ran for a period an ISP just before the dotcom bubble exploded, but that's another story...

That was when most of the code was written in C (sometimes C++) and a handful of parvenu scripting languages such as Tcl, Perl (version 5 would be released one year later), or the still fresher Python.

At that time, the GNU project had already been active for more than ten years, the Linux kernel was less than five years old, and the whole FOSS thing was in its infancy for most of the world, but for a group of hackers and visionaries. Indeed, it was still seen with suspicion and fear in the business area, if not explicitly fought.

Being a FOSS company was an exception, not a viable opportunity. In those years, the only credible company was probably RedHat Inc. in the USA, while other companies (such as MySQL AB in Sweden) still had to gain momentum. To my memory, almost no FOSS company was probably located in the EU at the time. Sourceforge or Freshmeat still had to appear, and FOSS tarballs had to be merely searched via Google search through rigorously indie websites.

Fast-forward almost thirty years. FOSS development has become mainstream, including for the same companies that used FUD at the time to fight it (Microsoft, yes, I'm talking about you). Every developer proudly creates their FOSS portfolio of tiny or more considerable project contributions through Github to collect gadgets and show their own skills. There are more FOSS projects than single developers, and so on.

Apparently, all has gone well, but for a couple of details, now, like then. Those details are the governance and sustainability of FOSS projects, which are strictly connected to their credibility. There are very few (maybe no one) articulated projects that are conducted by a handful of developers for a significant duration, and even fewer just for free (like beer).

In those regards, I can consider SQLite and its well-known geospatial extension Spatialite, as well as a few other projects in the past, such as OpenSSL. They all survived a pretty long time, with quite regular releases, but not without sustainability problems from time to time. See, for instance, the issues of Spatialite. That could be specifically grave when security is involved, and the software is a reverse dependency for many others (as happened to OpenSSL).

The hard truth is that any complex software requires years of work to solve issues and add rock-solid features. Proper architectural choices and automation tools can make the whole effort more manageable and light, but there is no silver bullet to change this fact of life.

If you belong to a tiny pool of developers or a single small company, there is a good probability that you can't simply maintain the same level of effort for long enough. Even in most cases, proper software maintenance is simply a tedious activity. Let me call that with its correct name: it is a grunt work that merely needs to be done. The genuinely creative part is often overwhelmed by tons of boring issues fixing, communication activities, documentation writing, etc.

Of course, the only decent solution is an excellent modular architecture with many people working on independent parts in a decentralized way. But for that, you need excellent organization and tooling, with project governance that ensures the right level of involvement and satisfaction for people. The developers could be paid or not, but in the second case, they must enjoy participating in the project; otherwise, they will leave for others. Paid people at least are paid to work on demand (not based on their goodwill), but they need to enjoy their work equally. Otherwise, the sh*t load will rise to high peaks. That's the reason for having excellent governance, and it is also the reason why I rarely saw a single company FOSS project that was durable.

It is under the eyes of anyone that the most durable FOSS projects are conducted by a multi-actor foundation with a well-defined set of rules (without exceeding bureaucracy) that include some democratic process to make fundamental decisions. Unfortunately, such governance cannot be stated from night to day. The project requires a critical mass of developers/companies and years to accomplish such a result. Most of the FOSS projects are simply not relevant enough to that and should find an umbrella foundation and a more general community that could be helpful for sustainability (let's say FSF, Linux Foundation, or OSGEO for the geospatial world). This type of organization will attract funding and ensure mid to long-term sustainability. This is what I would call a condition sine qua non to ensure the success of a FOSS project.

To be clear, most FOSS projects are doomed to fail and close after a few years because not enough developers, users, or companies are interested in either working on them or sponsoring them to cover their expenses and development time. The alternative for the most conservative and stable is a de facto freezing in the state they are in with only minimal maintenance. Indeed, the Debian archive is full of such programs, which are good enough for use but miss any future evolution plan, if not sporadically. Fortunately, multiple software programs would become obsolete enough to be ignored and replaced by new, more flexible ones. This can be especially true for tooling and languages, but even in general.

Let me now concentrate on the elephant in the room. Who does what in the FOSS world?

Pure FOSS products companies

I will be clear. Most of the companies whose core business is concentrated on creating and maintaining FOSS tools and products have not had an easy life, and that will not change in the future. They have had a bumping road since the very beginning because the sources of returns are very limited: sponsorships, public funding, customer contracts on side extra modules, etc. In the last fifteen years, all this has become even more complicated due to complex relationships with major cloud providers. This is evident if one considers the latest known cases of RedHat (acquisition by IBM), Hashicorp (changes of licensing models and successive acquisition by IBM), Elasticsearch (again, changes of licensing models), MySQL AB (acquisition by Sun and then Oracle), and MariaDB (acquisition by K1 private equity), which reveal constant financial issues with this type of model, in a way or another.

I'm pretty pessimistic about the sustainability of such a single-holder FOSS project model. I even consider it neither ethical nor convenient for independent developers to sign copyright transfers and other side agreements whose purpose is maintaining complete control of the source code by the copyright holding company. That continuously exposes the possibility of the project or license hijacking.

Companies that also work on FOSS products

That is the easy way, followed by the big companies, that have plenty of incoming sources that are not directly dependent on FOSS products. The FOSS projects are not their core business, but they can be heavily dependent on several of them. They participate in large projects and can even be influential on the goals and directions of the projects because those products are heavily integrated within their workflows. They can sustain internal teams that work on multiple FOSS projects and pay for costs, meetings, sponsorship, and so on. They can even tolerate working with other companies and a community of independent developers for common goals and even with concurrents sometimes. This is the stablest situation and probably optimal, at least until company goals do not change, which could possibly upset the project (ask ex-Google Python core contributors for comments, but there is a long list of examples).

Companies that work with FOSS products

Most companies and users are merely passive users of multiple FOSS projects, with little skills and will to participate in development and costs. In those regards, things are slowly changing, but the level of awareness of the whole FOSS ethic and development process is still low. Many companies still have an evident predatory (let me simplify, sorry) approach to FOSS software use, too: they like the free as beer part, to say briefly. Their role in the sustainability of the project is not relevant, but they are part of the users community and globally determine its success. They are inevitable and part of life and could be more fair from time to time, thanks to micro-sponsorships and minor contributions (even bug reports).

Large FOSS projects with an ample community, a tiny minority

Of course, the Linux kernel is an immediate example of this kind of project, which includes a large set of contributors, a large group of companies, and independent developers. Unfortunately, it is a tiny minority of the entire FOSS project family. Often (always?) governed by a foundation but also with ample and diversified funding because the users community and interests are both broad. If all projects were of such kind, the rise of FOSS projects would be delightful. Unfortunately, this is not the general case. The result is that participating in a FOSS project is generally a choice for life. If you are the initial owner, it is a big responsibility. There could be a steep rise in these years of mainstream FOSS development because there are tons of open-source (more or less) projects out there, often conducted by big companies, too. Gaining interest and funding are challenging tasks. For the average developer team, finding a solid umbrella foundation is the easiest choice, or thinking twice before starting. For sure, as a developer, I would never sign any additional/conditional agreement about copyright holding and contribution rules with a single company (a small or a big one, that's not the point) because the future of such a project is always uncertain. Sure, any FOSS project can always be forked, but forking governance is never an easy task.

So what? In the short run, FOSS projects sustainability is a voting machine; in the long run, it is a weighing machine: don't feed the hype. Concentrating efforts on solid and well-conducted projects is a must, even more than considering their immediate technical merits or popularity. That's true for developers as for users. If your preferred FOSS project is a single-company show, it is probably doomed. You are warned, these are my elementary 2 cents suggestions to navigate the magic FOSS ocean.