frankie-tales

SM-Tools, Copernicus, FOSS and the reasons of inevitable choices and drifts

2026-02-25T15:00:00Z

Here at work, we develop a series of tools for geospatial processing with multiple goals, expected maintenance durations, different scopes, generalization needs, and motivations. Note that about 10 years ago, here in Europe, a completely new approach to upstream and downstream services for Earth Observation began: ESA changed its data licenses, and distribution and access modalities entered the Big Data era.

That even changed the data approach for academic institutions and triggered a major shift in the daily work of researchers, with access to a tremendous volume of weekly data available almost just in time and worldwide. Of course, such a change also impacted us, and we had to adapt our processing and storage capabilities to the new era.

One of my side projects in that regard is SM-Tools, which consists primarily of a collection of support tools for running our internally developed soil moisture algorithm using SAR satellite data (SMOSAR). One such tool (now named smt_copernicus) began (and evolved with multiple restarts-from-scratch) more than 10 years ago, when the Sentinel constellation started operations. Its purpose is to search for and download satellite products from Copernicus archives using multiple criteria, and to maintain an internal geospatial database of these products, along with all derived maps and ancillary data. This is only one component of a system that should be able to process large quantities of multi-source data on selected areas of interest, create downstream products, and calibrate and analyze results by comparing them with field data. The clear final goal is to achieve new findings in satellite data analysis, supported by extended processing worldwide, and to introduce new algorithms.

This is a long-term goal that unfortunately runs up against short- to mid-term difficulties of accessing archives that are not under our direct control. The sad reality is that in the last 4 years, the Copernicus archive access modality changed 3 times, and in the previous period, Copernicus also changed policies and modalities in progress (e.g., by introducing online and offline products, changing formats, etc.). Geospatial communities are small enough to encounter more practical difficulties than expected in such operational conditions, and this is now an almost weekly experience. We now have to chase other parties’ changes more often than we did in the past, rather than working on our own goals instead.

For instance, until 2023, the main package used for accessing the Copernicus archive was the Sentinelsat Python package (developed by an handfull of willing scholars starting in the summer of 2015). It became abandonware that year when people discovered that the protocol changes required rewriting most of the package from scratch, including all test code and mockings. That happened again at the end of last year. Incredibly enough, all access protocols have been an ongoing affair since 2023, even for non-secondary details, and required frequent adjustments to avoid unexpected breakage in download and processing pipelines. That for sure does not encourage community-supported FOSS solutions. Exactly in 2023, when I discovered our Sentinel-related tool had to be deeply changed because of the lack of the obsoleted Sentinelsat package, I decided that enough was enough and managed to migrate from Python to a fully self-supported Perl reimplementation: one of the things I always hated in the Python ecosystem is the excessive (for me and our purposes at least) speed in deprecation of consolidated packages and features, and the prospective of having to chase after unexpected changes in both Copernicus AND Python sides was out of question. My experiences with Perl have been much less annoying in this regard, with scripts running perfectly even after 20 years since they were written. Let’s consider this the old-school approach: if something is working, don't touch it without a more than valid reason, and even then, think twice before you touch.

In the meantime, around 2019, another independent effort started to support Copernicus access, along with a few other data providers. That’s about 4 years after the original Sentinelsat project. Timing in this case is essential to be considered. EODag is a single-company FOSS product that has been actively developed since then, but may have been considered stable 1-2 years later. Of course, it only provides the usual access layer, and using that package implied, at the time, replacing Sentinelsat with EODag as a base layer for searching and downloading only, while performing other tasks afterward with other self-made code. Before 2023, using Sentinelsat or EOdag was equivalent in order to perform the same task, with very little advantages.

Note that both tools were, in any case, pretty adequate but not enough for our goals, and also had a few defects (or a lack of flexibility, to say better) to manage in some creative way. That has been one of the reasons for not replacing Sentinelsat with EOdag in 2023. The other major one was that the idea of replacing a small package with another (as for Sentinelsat, there are just a couple of main contributors to the codebase, along with a good number of pending issues and PRs for such a kind of product) is probably not the safest one to avoid problems in the near future, when Copernicus will change things (again, see the next Earth Observation Processing Framework EOPF data format — Zarr). And of course, EODag is written in Python, and I already expressed my concerns about that.

Even if you like it or not, nowadays, the concrete alternative to adopting small FOSS projects to perform basic tasks is to use AI tooling to create a perfectly (or almost so) tailored implementation for the target task. While in 2023 I had to rewrite from scratch (in maybe a month of work with some fixes to Geo::GDAL::FFI package for a multi-threading issue) an implementation of a multi-threaded tool for accessing the Copernicus archive and maintaining an internal geospatial database of products consistently among multiple generations of the archive, I implemented the STAC protocol variant in a few hours instead, thanks to Claude Code-based patching and my reviewing and tests for the resulting codebase. As said in this post, currently, the cons of adopting a small third-party FOSS solution outweigh the pros, particularly regarding the resulting technical debt, compared with a well-conducted self-consistent AI-based development process.

I’m seeing in my own side projects exactly the mirror of what will probably be the reality of FOSS projects in the near future, in practice, as I mentioned in the previous post. Relatively few major/interesting projects will be adopted by others and attract contributions, while most codebases will become pure one-man shows, with AI tooling.

A significant part of geospatial processing involves data procurement and processing, i.e., refining and preparing data and images in order to collect, filter, and process large volumes of data for subsequent analysis. This is the most annoying and repetitive part of the process, and also often the most time-consuming. In my experience, working on those tasks is probably the most effective way to use LLMs through a spec-and-test-driven design. Whether you like it or not, it is the most immediate way to produce working code by iterating with a chain of thoughts and accurate reviewing of results, including a decent test coverage. As observed in Antirez's experience, the AI agent also had the nice ability to retain my Perl style (which is not secondary, given that Perl has many programming flavors and variants).

Maybe the final results will be an increase in quick-and-dirty codebases, but for many scholars, it will be a major simplification of their lives. In most domains of science, coding activities have been seen as an inevitable evil: they are a tool, not the primary goal, and even before the advent of LLMs, most scientific codebases were far from something to be proud of. The Copernicus attitude to FAFO will also encourage such approaches. Simply because scholars don't have time to waste chasing changes introduced by this or that data provider or company when contracts change.

AI sloping attitude? No, simple survival instinct.

Is AI driven coding the start of the end of mainstream FOSS?

2026-02-04T20:00:00Z

Someone on Mastodon (I’m sorry, but I don’t remember who exactly) published a short post that pointed to a rather technical economic study of the impact of AI on FOSS software development [1]. It is no secret that the AI debate is highly polarized, and the enthusiasts for the current trend in AI applications in the IT domain are at least as numerous as those who are concerned/skeptical. What is certain is that no one can, in the long term, prospectively evaluate the impact of AI on society, particularly in the IT world.

The main thesis of the paper is that AI-based code production will end the mainstreaming of FOSS software, as we have learnt over the last 15-20 years. The paper begins with well-known episodes from recent history (specifically, the Tailwind saga [2] and Stack Overflow's near-death experience [3]).

Of course, the paper presents a theoretical economic model to evaluate a possible impact scenario for the FOSS production model, which could or could not come to fruition, depending on the assumptions made.

My honest opinion is that a conscious and accurate use of AI can accelerate development. That is, in a bad and good sense, I mean directly on the basis of the experience and skills of people who use such models. Therefore, we are both seeing slop and high-profile creations with the aid of AI. Maybe slop contributions are more prevalent simply because mediocre developers are the majority, and mediocrity is the backbone of enterprise production (because it is the most replicable and independent of contributors and their capacities).

Like it or not, modern software industries do not need, and fight against, too much creative approaches. Enterprises need aurea mediocritas, not isolated geniuses. Also, depending on third-party creations, it apparently reduces the enterprise's technical debt because it is typically shifted onto someone else's shoulders. Of course, this is an approach that works until it fails miserably when such a third party disappears, changes its license model, changes its mind about the product, changes its APIs, and so on.

That said, one clear consequence of using AI helpers in coding appears to be the progressive disappearance of many packages, modules, and libraries, which can be easily replaced by AI-generated creations tailored to the task. Just to cite one practical example, Tailwind nowadays could be easily replaced by CSS and simple JavaScript components, with the obvious advantage of not depending on yet another third-party-controlled piece of code that could be subject to abrupt changes from one version to another without notice and break existing codebases. At the same time, Tailwind themes can be generated by AI without even consulting its documentation (which apparently had an immediate impact on the company's revenues).

Another advantage is that AI-based, tailored solutions would reduce the amount of code from external dependencies that solve problems for others, instead of focusing on the minimal set of features for your own needs (with all the implications of possible breakages arising from such an anti-minimalistic pattern).

Of course, using AI helpers in this way does not reduce the effort required to understand and create new software, but it probably raises the required competence to a higher level, which could be better in the long term, while encouraging quick-and-dirty approaches in the near term. The so-called vibe coding is not a black-and-white concept; it has a lot of grey tones directly depending on the awareness, responsibility, and skills of the developer: as said, it can accelerate in many senses - even to crash against a wall - increasing in an uncontrolled manner the technical debt when in the hands of the wrong individual. Even about that, Anthropic recognizes that AI abuse can negatively impact coding skills and debugging capabilities [6].

Add to this the current very high infrastructure load many networks are reporting, for which the AI bots currently seem to be the culprit [4]. This seems like very strange behavior for such botnets, given that web crawlers have been around since the 90s and should be able to handle infrastructure load fairly well by now. It seems that AI companies simply aren't fair enough on their own, or that the training phases of neural nets are definitely more demanding. Maybe both?

So, what do I see as the future for FOSS development as a whole? I am not a pessimist as in the cited article. For sure, I see fewer small contributions in the long term. Today, there is a massive production of AI-slop-based contributions to many prime-time projects, but I see this as incidental. In recent years, GitHub-based path of honors has been a major self-promotion channel for junior developers, which explains the drift toward low-quality contributions: devs are (were?) strongly motivated to contribute and find in AI slop an easy path to that, by creating personal portfolios. That’s also true for fake security-related reports (see the well-known Curl project case [5] and others). This is, of course, annoying, but in my view, that’s the result of current AI hype and should normalize in the mid-term.

Also, in the near future, I see less and less relevance in FOSS projects that are not sustained by a strong architectural idea, innovation-grade, a large community, and a consistent development effort (much bigger than a few weeks or months of work). That kind of project will become mainly background noise, let me say. Maybe that could impact whole categories of FOSS software: it is not a secret that many language hubs are full of packages/modules of dubious quality, often used because they are available just a use/import directive away. In many cases, such products will simply be replaced by an AI-based reimplementation. If the final result will be better or worse in average quality, only the future will show. For sure, the AIAD will cause a progressive democratization/popularization of the development process, giving average users access to possibilities once unavailable to them: we will probably see the production of a plethora of small tools and workflows built on agents rather than finished, refined products, like it or not.

The result could be an increase in FOSS products at the cost of lower average generality and code quality, with a few high-end, tailored products for mainstream applications instead. But was this really so different in the past? I don’t think so. The true difference is probably the increase in quantity in both sets of products, as potentiated by AI tools: if one does not do her/his homework, the result is clearly garbage, but that was true before AI, too.

“AI gives us the worst and the best - simultaneously.”

(Daniel Stenberg, the Curl Mantainer)

References

AI training, copyright and the future of contents creation

2026-01-11T21:00:00Z

I have already addressed the implications of modern LLMs, specifically their training, in the context of copyright and licenses for both code and original content. A 'IANAL' disclaimer applies to this post, but my honest opinion is that such training is a legitimate type of reading and learning after study, unless explicitly excluded in licenses among the licensee's rights.

Following the exploitation of LLMs and the AI boom that began in 2022, several lawsuits and litigations emerged among multiple parties, with a few reaching a significant milestone through the first court rulings. Note that every country has a bit different regulations about copyright and fair use, so the current lawsuites could be only the starting point of a long list of legal actions.

While most of the current lawsuits seem to demonstrate that Anthropic or Meta had the right to use books bought (in paper or digital form) for LLMs training (on the basis of the fair use principle), the most problematic aspect instead is the apparent use of pirated books taken from LibGen and other known piracy websites, which - if confirmed - can result in potentially destructive damange for the companies, to compensante authors and pay fees in the order of hundreds of billions.

The same problems are present in the coding parts: again, using FOSS-licensed code for training could fall under fair use, but training using private codebases, as well as proprietary ones, could be equally destructive for the same companies, as well as for GitHub and Microsoft. The key point would be demonstrating, without any doubt, the unfair use of private or pirated content, of course.

Of course, I'm quite sure future licenses for FOSS codebases and documentation could include an explicit exclusion clause for AI training, which could jeopardize the legitimation of use even for future FOSS code. I would expect such a license change, as some projects already explicitly exclude AI-based contributions. My opinion about such a question is that it could represent shooting oneself in the foot, due to the pervasivity of AI tools among developers currently. Adoption of AIAD could represent a boost in development time if adopted with a healthy dose of skepticism (i.e., a human-in-the-loop approach). About that, I'm quite convinced of Linus Torvald's point of view: the point is not who writes the code, but who is technically responsible for it and ensures the required quality review and supervision.

Moreover, an implication of the current polarization in the AI hype is the future (present?) crisis of traditional web content providers. A symptomatic case is the StackOverflow crisis, which will, with high probability, lead to the end of the service as we know it in the near future.

That will have an impact on future AI training, too, for sure, because SO has been for years a huge source of knowledge about multiple fields in IT. What if fewer and fewer people will contribute to Wikipedia and general web content? What if more and more sources of information were to reserve the right to use their information for pure human-driven study? Knowledge has not been static in human history; AI models will need to continuously enrich their training sets and stay up to date.

It would be grotesque if the whole AI hype were brought to a halt by such copyright-based legal questions (even if I'm pretty sure a fully fair training would be possible now for such companies, who knows the impact of a more limited approach on the final result?). Surely, this seems the most serious threat to the future of such companies and the whole AI-based solutions.

The only true solution to such a threat is finally having a true open training model, which details sources and the whole process of training with full transparency, something that even the so-called open AI models are still far to be ready to provide.

References

Still, no silver bullet

2025-08-24T18:30:00Z

I recently re-read the seminal book by Fred Brooks about software engineering, entitled "The Mythical Man-Month" or MM-M for brevity. Specifically, I read the paper version of the 20th anniversary, which was revised and reprinted in 1995, after the first edition of 1975. I did that on purpose, firstly because it is always a fantastic read, and secondly to understand how much of its contents is still valid today, exactly thirty years later since its last revision.

Fred missed in 2022; otherwise, it would still be interesting to know his thinking nowadays after the LLMs boom and the birth of the AIAD (AI Aided Development) as a new revolutionary (or often seen as such) tool. Hi, Fred, wherever you are. It is worth mentioning that AI was already taken into consideration by Brooks at the time, even if limited to expert systems and other rule-based variants, which seemed promising and often sold as revolutionary before the mid-90s. A lot of the book's contents remained in the history of software engineering, including the famous Brooks' Law, and the whole book is still an excellent source of inspiration for any management and organization of complex intellectual projects (not necessarily limited to software systems), that heavily includes large teams of individuals.

One of the main theses of the latest book edition is that in the short term of 10 years from its proposition (the original essay was dated 10 years after the first edition of the book), he did not expect a silver bullet. That means no significant technological or managerial development was expected to be able to improve our productivity in programming by one order of magnitude. Ten years later, he confirmed the same idea, even considering exceptional tools like old-generation AI, visual programming, CASE tools, and so on. Is this thesis contradicted in 2025 by the existence of current AIAD tools, including chatbots, agents, and AI-empowered IDEs? My honest idea is no. I mean not now and not for the foreseeable future. The reason is exactly the same as Fred posed at the time. Reducing accidental problems in software creation (what AI is able to do) cannot be confused with the essential problems in software creation: the complexity of defining an articulated task, its analytical specifications, and an algorithmic solution to solve it. First of all, ignore the simplistic case of asking an AI engine to implement a very simple program. Here, the word simple means truly that. If you can specify a request by a whatever large manageable token context and formulate your request in terms of a brief question (let me suppose a question of some dozens of rows), well, that's probably an example of a simple (or dumb) problem. Too few, too easy. We are talking about a whole system that is generally difficult to describe, even in thousands of pages of specifications and documentation, written collectively by large teams of developers, architects, and domain experts.

The hard truth is that most of the real-world information systems out there cannot simply be specified in such a way. We are not able to define an unambiguous and complete enough specification to describe such systems, not to mention being truly able to write a complete and neat documentation of it, including its inner workings and use. We live in a deep illusion about that. The context size to get the required level of details to avoid bugs and ambiguities in specification would be impractical for current and even future tools, as well as for any humans. We would get in any case buggy (i.e., incomplete or misunderstood) results even if the AI engine were able to avoid hallucinations (which is not the case) and had no limitations for context size. The presence of AI hallucinations is only accidental in this regard.

In the current AI tooling, we are simply moving the complexity from the writing of a formal language step by step to using a natural language with a higher level of abstraction to express the problem. The complexity is still there, and it is inherent to the problem. Again, we resolved an accidental difficulty in a creative manner, not different from moving from assembly to using a modern programming language. Now the difficulty has moved elsewhere, but it is still there, and natural language is even more complicated to use compared to formal language. These difficulties translate into multiple refinements and trials to try to be more precise and get sensible answers and code in a continuous iteration. Isn't that so similar to the whole ordinary process of developing a program? In the most simplistic approach, such a process becomes vibe coding, and iteration could tend to infinity, with a forever loop. The smarter programmer for an easy task will do that in a reasonable (limited, hopefully) number of iterations, instead. Is that a significant improvement of one order of magnitude? I think not, as in most cases for the past. As in the case of high-level languages instead of assembly, they improved efficiency in coding as asserted in MM-M, but not by a whole order of magnitude. The AIAD is again another helper to solve accidental difficulties. The problem and all its complexity are still there. Thinking that we found the silver bullet is again (and again) an illusion or pure marketing.

So why do many CEOs insist on predicting a bright yet unlikely future of AI agents instead of having developers create applications? Brooks already wrote about that: there is a profound confusion in exchanging months and men, and an excess in optimism when approaching software development, even among techies, but that becomes paroxysmal among managers. None can seriously provide even a decent and reasonable evaluation of development time starting from incomplete, ambiguous, or vague specifications: the same simply happens systematically in overestimating the capabilities of current AI tools.

So what? AIAD is simply yet another tool among those available to developers, but the management problem of dominating complex projects is still there, with all its inherent difficulties. And the possibility to use natural language instead of a high-level formal one is only an apparent simplification of the process. It looks more familiar and easier, but it is also much more ambiguous, and the so-called prompt engineering is again a pure optimistic illusion, an euristhic approach to try to overcome our totally insufficient capabilities of dominating nuances and semantics.

Again about AI, copyright, uses and abuses

2025-04-29T20:00:00Z

My last post dealt with some ideas about copyright that need more in-depth analysis. First, as it was common in the old good days, IANAL applies to this post and the whole topic. The final results of current litigations in courts that touch on some of the primary companies involved in the whole AI thing could ultimately differ from what is now the common sense point of view (mine). This post could become rapidly obsolete, so another disclaimer is due for this aspect, too.

A nice summary of the matter in the summer of 2024 can be read in [1].

I have to partially fix the assertion I made about copyright by Anthropic/Google and whatever company in a certain sense. At least for US copyright law, anything directly produced by non-humans is (or seems currently) not admissible for copyright coverage [2]. As you know, there are a few differences between different jurisdictions, specifically the US and the rest of the world. That's the reason why, indeed, Joseph Borg and Galyna Podoprikhina by WH Partners [3] say:

Despite the "common" belief that a work can be only be
protected by copyright if it is created by a human, one must
bear in mind that copyright laws are not uniform around the
world, especially when it comes to AI-generated work, or a
work created with the assistance of AI. Currently, apart
from the UK as described above, AI artwork is also subject
to copyright in Ireland, India, and New Zealand.

If a photo taken with a camera is admissible for copyright, the same could be claimed for any other tool, one could say. In the past centuries, even photography would have been indubitably not initially admitted to copyright and the artistic scene: indeed, a painting is a lot different from a photo. A modern photographer would not agree with such mortification of her/his work. Today, the reality is quite different, and photography is fully accepted among modern arts, with all implications about intellectual property and copyright.

Most authors agree with the significance of the human contribution to the work to define a copyrightable work, but how this contribution could be quantified is obscure. The number of prompts and replies, as well as the size of context contributions, are significant and sufficient efforts, or should they be quantified in LOCs and the size of direct patches to the AI artifacts? And in that case, what is the percentage of human-driven contribution that represents the threshold for deciding if the work is copyrightable or not? I'm afraid that the final verdict is something to decide in a court, as in cases of plagiarism.

But for AI, for ages, we also had RAD and no-code/low-code utilities that seemed the future of development for specific applications, with all their limitations (not too different from AI ones, to be fair). Even in those cases, copyright claims could be problematic.

That said, there is also the problem of training possibly performed without authorization. While most of the FOSS software is covered by one of the OSI licenses, not all licenses are compatible with each other, so the resulting LLM model is questionable and possibly unfair. I will ignore, for decency, the eventual use of proprietary content for the purpose of training, which already seems to be the subject of lawsuits by multiple parties in some contexts: see for instance [4] and [5].

As written in [2], the final destination of the whole topic is still foggy and unclear. Multiple parties are involved, and a series of lawsuits and claims are pending. This seems to be the reason why some companies explicitly deny the possibility of using AI tools in their developers' daily work. Due to their pervasive diffusion at multiple levels, this becomes increasingly difficult to avoid. As often in the past, tools are still forward than rules and sh*t could happen in a not so far future.

References

AI artifacts, copyright and electric sheep dreaming

2025-04-22T12:30:00Z

My last post captured the attention of my old fellow Sandro 'strk' Santilli on Mastodon, who sent a provocation about the whole AIAD thing. So, the challenge is accepted.

First and foremost, the whole AIAD issue is a complex and hotly debated topic. The question of whether the training practices of the past and present should be considered fair use is a matter of contention. This is particularly true for existing code bases on well-known repositories and other types of content, a complexity that keeps us all intellectually engaged.

In the specific case of code, the basis for deciding about such a question was formerly stated in the accompanying licenses. Honestly, there is nothing in the current formulation of OSI licenses (BSD and GPL, among others) that precludes such a very special activity. The non-discriminating conditions apply to any human activity, and the neural net training - undoubtedly a human-driven task - is not excluded. The training is a learning type of activity or a sort of.

That said, the NN training is, of course, a massive and intensive kind of learning. But let me consider any artifact created by an AI model as a direct derivative product (this is a forcing of the concept for me, but let me axiomatically accept it). That could be an 'original' product created from scratch (i.e., by direct prompting or by limited documental context) or a direct derivate one (because it is based on previous code proposed as part of the AI context). Suppose you ask any of the big models for a direct clarification of the terms of use. In that case, you will discover that any of their artifacts retain the copyright ownership of the product, but there is a very permissive use that perfectly adheres to the four fundamental freedoms of FOSS licenses. Of course, the results could be based on a previous code base. In that case, its license still applies for such a derivative work, even with the additional copyright of Anthropic, Google, or OpenAI.

This could pose a significant problem if the final software product needs to retain one specific copyright holder, as is the case with most proprietary software or some FOSS ones. Understanding the potential impact of copyright transfer is a key consideration in this context, and it's crucial for us to be fully informed and aware.

Is the process of an AI participating in development really that different from what any average hacker does when participating in FOSS projects? I don't think so. You add your copyright to the existing ones for the parts that are under your direct control and accept conditions of use already defined in a license. The true challenge, in my humble opinion, lies in the changing of licenses. Any ex-post change should start with a note of acceptance by all copyright holders, including Anthropic, Google, etc.

I consider this difficulty a feature, not a bug. I advise against participating in projects with copyright transferring because of the potential for changing the license later when your contributions move out of your control. It happened in the past, and it will happen again.

One should also consider the licenses of many other sources of knowledge and inspiration for developers. How many people know that the default license for StackOverflow code snippets is CC BY-SA? Indeed, how many developers actually add an acknowledgment in their software for such snippets? Or even for snippets taken from sites, blogs, books, or manuals without considering that such sources are even more restrictive for use and creating derivative work? Isn't our full daily work the result of a long learning phase, based on our education and training by books, experiences of others, as well as trial-and-error processes?

That said, let me spend some words about the elephant in the room. In the context of AI and copyright law, the 'elephant' represents the complex nature of AI models and their potential to create original works. Do AI models dream of electric sheep? Well, I don't think current models are pure stochastic parrots, to be honest. I think there are probably dozens or hundreds of cognitive forms that govern what we call generically intelligence, including some emotive and empathic forms that one can also find in a dog, a cat, or a dolphin. One of those forms is probably captured from the neural network model of functional representation, which is also perhaps in common with part of our mind. We are at an average level, much more effective and efficient in those regards, and to be fair, I would also say that hallucinating is a common experience even on the human side. We are much more complete and retain contexts that are wide, like a lifetime. A few people are terrified by this observation and seek refuge in negation or faithful certainties.

We are complex organisms with probably still partially known processes that govern our so-called intelligence, which is physically based on cells and energy in our brains, whether we like it or not. We found a way to mimic part of this complex process, with all limitations of the case. Is this intelligence? I don't know, but at the end of the day, what is intelligence? When none asks me about that, I know perfectly what is, but if you ask me about that, I don't know anymore.

Coding with AI, the good and the bad

2025-04-20T17:40:00Z

Like many other developers, I recently started using some LLM-based AI systems as helpers for coding in a few languages. I'm not a fan of VSCode, and I prefer a more traditional approach to coding: I hate to cope with code completion servers and use one of my preferred editors, Vim or Emacs. Navigating by tags is more than enough for me. That said, this is the summary of my current experience in the new world of AI-aided approach to coding (i.e., AI-aided development or AIAD for brevity).

There's a clear divide among developers when it comes to AI tools: some love them, while others are more skeptical. If you're keen to delve deeper into this topic, Salvatore Sanfilippo, also known as Antirez, shares some insightful perspectives on his YouTube channel. He provides comprehensive evaluations of the leading AI models and systems, offering a balanced view of their strengths and limitations.

Based on my experience, while both Anthropic and Google top models are equally good enough for use, a big difference is due to their relative UX, which is incomparable among various systems. Luckily, things are continuously changing in those areas. Copying and pasting without a clean and accessible per chat/project assets catalog is definitively a pain. The most effective approach is sharing pieces of a git-based code base, which is not always possible with all AI chatbots. Even Salvatore shared a few impressive results about his experiences, and I can definitively assert that, at least currently, some of those systems can be quite helpful for some tasks if (and only if) used with a grain of salt.

First and foremost, the most effective use of AI tools starts from very circumscribed tasks to perform in a step-by-step approach after a well-stated creation of the context to make choices. In other words, any model must be conducted by hand in the right direction. It is crucial to use what Claude AI defines as projects to create a clear context with a general detailed architecture description and key assets for orienteering the linguistic model. It is also essential to maintain order meticulously. Being verbose enough and precise is the key point, and that's typically what a senior profile should be able to do. Each resulting asset needs accurate reviewing and testing; it is simply impossible to assume that a simple change in human terms corresponds to perfectly valid changes in AI assets. What is easy for us could not be for the AI model and viceversa.

The review should be both stylistic and functional because complicated programming patterns could be completely unmatched by the AI model. That is specifically true when documentation about APIs is unprecise or missing the point. The models even tend to generate redundant code or hallucinated code snippets that cannot be compiled or interpreted. Sometimes, the AI model generates incredibly good code at large but presents silly or subtle oversights. At the end of the day, there is not always a gain in development time; it is only something different: you spend more time reviewing and fixing bugs than writing code from scratch. It is essential to cope with the limits of current AI systems to develop a handy way to compose multiple parts together and keep track of improvements and fixes in multiple iterations of the process. During iterations, it could reach the limits of the AI context and get incomplete or truncated files, and it is fundamental to have a way to regain the path with minimal effort. In other words, one has to design a well-formed idea of the resulting products and all intermediate artifacts. That's the reason why seniority and experience are fundamental to using such tools effectively: sorry, Mr. CEO, engineers are still here to stay and being paid for good work.

Tasks that can be easily covered by current AI models are:

Creation of general documentation and/or comments in existing code.
Creation of simple tests from a structured codebase.
Translation of code from one programming language to another, including documental formats, such as XML, JSON, and YAML.
Creation or improvement of auxiliary tools and boilerplates.
Reviewing and improving existing code by steps.
Help in identifying common issues and possibly interesting features.

The average results are good enough for an initial base of successive analysis and development, and I found the AIAD especially helpful with activities that I judge seriously dull and time-consuming. It's a relief to have AI tools that can handle the whole web front-end-related activities, which I personally find less interesting.

Using multiple AI models to cross-review a code base and possibly prepare multiple improvement alternatives while purging the obvious misunderstandings is also an interesting opportunity. At the end of the day, I judge AIAD as effective enough, but for the most significant limitation of having a not-too-extended context size (at least with basic pro/premium profiles). This intrinsic limit is the source of major problems in the UX, along with still rough or incomplete interfaces: if you need to review and modify a large code base, you can easily crash against such limitations and have to apply intensively a divide-and-conquer strategy to govern hundreds of thousands of LOCs (but diving into any unknown big code base is anyway so different?).

Of course, the correct approach is understanding that its use changes programmers from pure creators to reviewers. In those regards, I generally consider AIAD a Stackoverflow with steroids: anyone who used SO in the past found valid and interesting answers to questions along with perfectly misleading suggestions, and that's not different, and mostly better.

Disclaimer: I mostly used Claude and Gemini Pro, much less ChatGPT and Deepseek due to their intrinsic limits for UX.

Knowledge dispersion, AI and fragmentation of the information sources

2025-01-06T21:00:00Z

One of the coding projects I currently maintain is Autodir, a not-so-known little daemon based on autofs that can be used to create automagically users or group directories at their first use. It is specifically helpful when some kind of shared accounts system is adopted for multiple hosts and the related home directories need to be created optionally and on demand. Well, I recently moved the old repository from SourceForge to GitHub, and that has also been the occasion for me to update the old Docbook howto document for Autodir initially written by the original Autodir developer, Venkata Ramana Enaganti. I mostly maintained the project as a Debian package in the last 20 years or so, with only little interest in feature improvements: it basically just works, and that's more than enough for my use cases.

That has been for me the occasion to re-discover the Linux Documentation Project and the Docbook format used for documentation. Well, for old-school users like me, LDP has been, in the past, a Holy Bible sort of reference. So, what a disappointment for me to find out that the project is frozen under all regards and has missed any update for years. Many of the howtos have become obsolete and never touched in the last 10-15 years or more. Any update to the LDP git repository - if any - is not even reflected in the LDP website. That's a pity because the Autodir howto is still one of the main items pointed out in any web search, but it is sadly obsolete, with little possibility of changing its status. In brief, LDP is not more neither authoritative nor reliable as an information source.

This personal episode is the original reason for this post, which is also about the status of documentation and information provided for many FOSS communities and the possible future of content creation and solid knowledge building for the IT domain as a whole.

Of course, there are still some happy isles where the provided documentation and manuals are a pleasure to consult and represent the primary source of trustable knowledge about a topic. I can think about the BSD or Guix handbooks and some community-driven wikis, such as the Arch one, as well as some Debian guides. Selected tools, libraries and languages also have very lovely and complete documentation that is up-to-date and enjoyable to read.

This is not the case for the average tech topic because, in recent years, building a decent and deep knowledge of any IT argument, library or tool has become a nightmare. Information sources have become dispersed among tons of small websites, blogs, magazines, courses, books, small manuals, papers, podcasts, news articles, or even videos. Often, they are hidden among a plethora of low-quality stuff that is confused among the very few truly informative references. We suffer from an infodemic that every day becomes worse, not only for generalistic information but also for specialistic content. Even paying for information is not more of a guarantee of quality, with books, courses, and e-learning resources often hastily prepared for shallow consumption and which become obsolete at the speed of light (when they are not completely wrong from their beginning). I'm a long standing subscriber of the O'Reilly learning platform and I know what I say.

Sadly, our prospects are even worse thanks to the new generative AI systems, with a probable intensification of automagically created content without even a minimal quality checking. This is something that is already happening at every level, but it is specifically grave in the case of the IT domain, where the combination of AI coding tools, the progressive reduction of seniority and expertise in some fields could upset the whole process of building knowledge and solid experiences for creating a problem-solving attitude from scratch in a few years. Even in the scientific domain, this progressive enshittyfication of the quality level of publications is sadly tangible, therefore some editors already are requiring explicit exclusion of AI tools abuse for publication preparation and reviewing.

Today, junior profiles are probably condemned to marginalization in the IT world thanks to massive generative AI use in coding, but someone should ask themselves how senior profiles could be prepared in the future in this way. For sure even in the FOSS ecosystem it is already very difficult suggesting introductory references and authoritative repositories of reasoned documentation for newbies.

Optimists will say that the whole process has simply changed in the last few years and will change again with the current AI trends: nothing is lost, and I'm merely affectionate to the old-school approaches that need to change and will do again in the future.

Maybe the answer will simply be one-to-one mentorship-based training, as in the far past, to find patiently the right path in the current information jungle. I have no certainty about that.

I hope so, my Padawan. Really, I hope so.