Computer engineers and programmers have long relied on reverse engineering as a way to copy the functionality of a computer program without copying that program's copyright-protected code directly. Now, AI coding tools are raising new issues with how that "clean room" rewrite process plays out both legally, ethically, and practically.

Those issues came to the forefront last week with the release of a new version of chardet, a popular open source python library for automatically detecting character encoding. The repository was originally written by coder Mark Pilgrim in 2006 and released under an LGPL license that placed strict limits on how it could be reused and redistributed.

Dan Blanchard took over maintenance of the repository in 2012 but waded into some controversy with the release of version 7.0 of chardet last week. Blanchard described that overhaul as "a ground-up, MIT-licensed rewrite" of the entire library built with the help of Claude Code to be "much faster and more accurate" than what came before.

Speaking to The Register, Blanchard said that he has long wanted to get chardet added to the Python standard library but that he didn't have the time to fix problems with "its license, its speed, and its accuracy" that were getting in the way of that goal. With the help of Claude Code, though, Blanchard said he was able to overhaul the library "in roughly five days" and get a 48x performance boost to boot.

Not everyone has been happy with that outcome, though. A poster using the name Mark Pilgrim surfaced on GitHub to argue that this new version amounts to an illegitimate relicensing of Pilgrim's original code under a more permissive MIT license (which, among other things, allows for its use in closed-source projects). As a modification of his original LGPL-licensed code, Pilgrim argues this new version of chardet must also maintain the same LGPL license.

"Their claim that it is a 'complete rewrite' is irrelevant, since they had ample exposure to the originally licensed code (i.e., this is not a 'clean room' implementation)," Pilgrim wrote. "Adding a fancy code generator into the mix does not somehow grant them any additional rights. I respectfully insist that they revert the project to its original license."

Whose code is it, anyway?

In his own response to Pilgrim, Blanchard admits that he has had "extensive exposure to the original codebase," meaning he didn't have the traditional "strict separation" usually used for "clean room" reverse engineering. But that tradition was set up for human coders as a way "to ensure the resulting code is not a derivative work of the original," Blanchard argues.

In this case, Blanchard said that the new AI-generated code is "qualitatively different" from what came before it and "is structurally independent of the old code." As evidence, he cites JPlag similarity statistics showing that a maximum of 1.29 percent of any chardet version 7.0.0 file is structurally similar to the corresponding file in version 6.0.0. Comparing version 5.2.0 to version 6.0.0, on the other hand, finds up to 80 percent similarity in some corresponding files.

"No file in the 7.0.0 codebase structurally resembles any file from any prior release," Blanchard writes. "This is not a case of 'rewrote most of it but carried some files forward.' Nothing was carried forward."

Blanchard says he was able to accomplish this "AI clean room" process by first specifying an architecture in a design document and writing out some requirements to Claude Code. After that, Blanchard "started in an empty repository with no access to the old source tree and explicitly instructed Claude not to base anything on LGPL/GPL-licensed code."

There are a few complicating factors to this straightforward story, though. For one, Claude explicitly relied on some metadata files from previous versions of chardet, raising direct questions about whether this version is actually "derivative."

For another, Claude's models are trained on reams of data pulled from the public Internet, which means it's overwhelmingly likely that Claude has ingested the open source code of previous chardet versions in its training. Whether that prior "knowledge" means that Claude's creation is a "derivative" of Pilgrim's work is an open question, even if the new code is structurally different from the old.

And then there's the remaining human factor. While the code for this new version was generated by Claude, Blanchard said he "reviewed, tested, and iterated on every piece of the result using Claude. ... I did not write the code by hand, but I was deeply involved in designing, reviewing, and iterating on every aspect of it." Having someone with intimate knowledge of earlier chardet code take such a heavy hand in reviewing the new code could also have an impact on whether this version can be considered a wholly new project.

Brave new world

All of these issues have predictably led to a huge debate over legalities of chardet version 7.0.0 across the open source community. "There is nothing 'clean' about a Large Language Model which has ingested the code it is being asked to reimplement," Free Software Foundation Executive Director Zoë Kooyman told The Register.

But others think the "Ship of Theseus"-style arguments that can often emerge in code licensing dust-ups don't apply as much here. "If you throw away all code and start from scratch, even if the end result behaves the same, it’s a new ship," Open source developer Armin Ronacher said in a blog post analyzing the situation.

Old code licenses aside, using AI to create new code from whole cloth could also create its own legal complications going forward. Courts have already said that AI can't be the author on a patent or the copyright holder on a piece of art but have yet to rule on what that means for the licensing of software created in whole or in part by AI. The issues surrounding potential "tainting" of an open source license with this kind of generated code can get remarkably complex remarkably quickly.

Whatever the outcome here, the practical impact of being able to use AI to quickly rewrite and relicense many open source projects—without nearly as much effort on the part of human programmers—is likely to have huge knock-on effects throughout the community.

"Now the process of rewriting is so simple to do, and many people are disturbed by this," Italian coder Salvatore "antirez" Sanfilippo wrote on his blog. "There is a more fundamental truth here: the nature of software changed; the reimplementations under different licenses are just an instance of how such nature was transformed forever. Instead of combating each manifestation of automatic programming, I believe it is better to build a new mental model and adapt."

Others put the sea change in more alarming terms. "I'm breaking the glass and pulling the fire alarm!" open source evangelist Bruce Perens told The Register. "The entire economics of software development are dead, gone, over, kaput! ... We have been there before, for example when the printing press happened and resulted in copyright law, when the scientific method proliferated and suddenly there was a logical structure for the accumulation of knowledge. I think this one is just as large."

Read full article

Comments

Your Data Is Made Powerful By Context (so stop destroying it already)

In logs as in life, the relationships are the most important part. AI doesn’t fix this. It makes it worse.

(cross-posted)

After twenty years of devops, most software engineers still treat observability like a fire alarm — something you check when things are already on fire.

Not a feedback loop you use to validate every change after shipping. Not the essential, irreplaceable source of truth on product quality and user experience.

This is not primarily a culture problem, or even a tooling problem. It’s a data problem. The dominant model for telemetry collection stores each type of signal in a different “pillar”, which rips the fabric of relationships apart — irreparably.

Your observability data is self-destructing at write time

The three pillars model works fine for infrastructure1, but it is catastrophic for software engineering use cases, and will not serve for agentic validation.

But why? It’s a flywheel of compounding factors, not just one thing, but the biggest one by far is this:

✨Data is made powerful by context✨

The more context you collect, the more powerful it becomes

Your data does not become linearly more powerful as you widen the dataset, it becomes exponentially more powerful. Or if you really want to get technical, it becomes combinatorially more powerful as you add more context.

I made a little Netlify app here where you can enter how many attributes you store per log or trace, to see how powerful your dataset is.

• 4 fields? 6 pairwise combos, 15 possible combinations.
• 8 fields? 28 pairwise combos, 255 possible combinations.
• 50 fields? 1.2K pairwise combos, 1.1 quadrillion (2^250) possible combinations, as seen in the screenshot below.

When you add another attribute to your structured log events, it doesn’t just give you “one more thing to query”. It gives you new combinations with every other field that already exists.

Note that this math is exclusively concerned with attribute keys. Once you account for values, the precision of your tooling goes higher still, especially if you handle high cardinality data.

Data is made valuable by relationships

“Data is made valuable by context” is another way of saying that the relationships between attributes are the most important part of any data set.

This should be intuitively obvious to anyone who uses data. How valuable is the string “Mike Smith”, or “21 years old”? Stripped of context, they hold no value.

By spinning your telemetry out into siloes based on signal type, the three pillars model ends up destroying the most valuable part of your data: its relational seams.

AI-SRE agents don’t seem to like three pillars data

I posted something on LinkedIn yesterday, and got a pile of interesting comments. One came from Kyle Forster, founder of an AI-SRE startup called RunWhen, who linked to an article he wrote called “Do Humans Still Read Logs?”

In his article, he noted that <30% of their AI SRE tools were to “traditional observability data”, i.e. metrics, logs and traces. Instead, they used the instrumentation generated by other AI tools to wrap calls and queries. His takeaway:

Good AI reasoning turns out to require far less observability data than most of us thought when it has other options.

My takeaway is slightly different. After all, the agent still needed instrumentation and telemetry in order to evaluate what was happening. That’s still observability, right?

But as Kyle tells it, the agents went searching for a richer signal than the three pillars were giving them. They went back to the source to get the raw, pre-digested telemetry with all its connective tissue intact. That’s how important it was to them.

Huh.

You can’t put Humpty back together again

I’ve been hearing a lot of “AI solves this”, and “now that we have MCPs, AI can do joins seamlessly across the three pillars”, and “this is a solved problem”.

Mmm. Joins across data siloes can be better than nothing, yes. But they don’t restore the relational seams. They don’t get you back to the mathy good place where every additional attribute makes every other attribute exponentially more valuable. At agentic speed, that reconstruction becomes a bottleneck and a failure surface.

Our entire industry is trying to collectively work out the future of agentic development right now. The hardest and most interesting problems (I think) are around validation. How do we validate a change rate that is 10x, 100x, 1000x greater than before?

I don’t have all the answers, but I do know this: agents are going to need production observability with speed, flexibility, TONS of context, and some kind of ontological grounding via semantic conventions.

In short: agents are going to need precision tools. And context (and cardinality) are what feed precision.

Production is a very noisy place

Production is a noisy, rowdy place of chaos, particularly at scale. If you are trying to do anomaly detection with no a priori knowledge of what to look for, the anomaly has to be fairly large to be detected. (Or else you’re detecting hundreds of “anomalies” all the time.)

But if you do have some knowledge of intent, along with precision tooling, these anomalies can be tracked and validated even when they are exquisitely minute. Like even just a trickle of requests2 out of tens of millions per second.

Let’s say you work for a global credit card provider. You’re rolling out a code change to partner payments, which are “only” tens of thousands of requests per second — a fraction of your total request volume of tens of millions of req/sec, but an important one.

This is a scary change, no matter how many tests you ran in staging. To test this safely in production, you decide to start by rolling the new build out to a small group of employee test users, and oh, what the hell — you make another feature flag that lets any user opt in, and flip it on for your own account.

You wait a few days. You use your card a few times. It works (thank god).

On Monday morning you pull up your observability data and select all requests containing the new build_id or commit hash, as well as all of the feature flags involved. You break down by endpoint, then start looking at latency, errors, and distribution of request codes for these requests, comparing them to the baseline.

Hm — something doesn’t seem quite right. Your test requests aren’t timing out, but they are taking longer to complete than the baseline set. Not for all requests, but for some.

Further exploration lets you isolate the affected requests to a set with a particular query hash. Oops.. how’d that n+1 query slip in undetected??

You quickly submit a fix, ship a new build_id, and roll your change out to a larger group: this time, it’s going out to 1% of all users in a particular region.

The anomalous requests may have been only a few dozen per day, spread across many hours, in a system that served literally billions of requests in that time.

Precision tooling makes them findable. Imprecise tooling makes them unfindable.

How do you expect your agents to validate each change, if the consequences of each change cannot be found?[3]

Well, one might ask, how have we managed so far? The answer is: by using human intuition to bridge the gaps. This will not work for agents. Our wisdom must be encoded into the system, or it does not exist.

Agents need speed, flexibility, context, and precision to validate in prod

In the past, excruciatingly precise staged rollouts like these have been mostly the province of your Googles and Facebooks. Progressive deployments have historically required a lot of tooling and engineering resources.

Agentic workflows are going to make these automated validation techniques much easier and more widely used; at the exact same time, agents developing to spec are going to require a dramatically higher degree of precision and automated validation in production.

It is not just the width of your data that matters when it comes to getting great results from AI. There’s a lot more involved in optimizing data for reasoning, attribution, or anomaly detection. But capturing and preserving relationships is at the heart of all of it.

In this situation, as in so many others, AI is both the sickness and the cure[4]. Better get used to it.

In 2021, being a good software engineer felt great. The world was full of software, with more companies arriving every year who needed to employ engineers to write their code and run their systems. I knew I was good at it, and I knew I could keep doing it for as long as I wanted to. The work I loved would not run out.

In 2026, I’m not sure the software engineering industry will survive another decade. If it does, I’m certain it’s going to change far more than it did in the last two decades. Maybe I’ll figure out a way to carve out a lucrative niche supervising AI agents, or maybe I’ll have to leave the industry entirely. Either way, the work I loved is going away.

Tasting our own medicine

It’s unseemly to grieve too much over it, for two reasons. First, the whole point of being a good software engineer in the 2010s was that code provided enough leverage to automate away other jobs. That’s why programming was (and still is) such a lucrative profession. The fact that we’re automating away our own industry is probably some kind of cosmic justice. But I think any working software engineer today is worrying about this question: what will be left for me to do, once AI agents have fully diffused into the industry?

The other reason it’s unseemly is that I’m probably going to be one of the last to go. As a staff engineer, my work has looked kind of like supervising AI agents since before AI agents were a thing: I spend much of my job communicating in human language to other engineers, making sure they’re on the right track, and so on. Junior and mid-level engineers will suffer before I do. Why hire a group of engineers to “be the hands” of a handful of very senior folks when you can rent instances of Claude Opus 4.6 for a fraction of the price?

Overshooting and undershooting

I think my next ten years are going to be dominated by one question: will the tech industry overshoot or undershoot the capabilities of AI agents?

If tech companies undershoot - continuing to hire engineers long after AI agents are capable of replacing them - then at least I’ll hold onto my job for longer. Still, “my job” will increasingly mean “supervising groups of AI agents”. I’ll spend more time reviewing code than I do writing it, and more time reading model outputs than my actual codebase.

If tech companies tend to overshoot, it’s going to get a lot weirder, but I might actually have a better position in the medium term. In this world, tech companies collectively realize that they’ve stopped hiring too soon, and must scramble to get enough technical talent to manage their sprawling AI-generated codebases. As the market for juniors dries up, the total number of experienced senior and staff engineers will stagnate, driving up the demand for my labor (until the models get good enough to replace me entirely).

Am I being too pessimistic?

Of course, the software engineering industry has looked like it was dying in the past. High-level programming languages were supposed to let non-technical people write computer code. Outsourcing was supposed to kill demand for software engineers in high-cost-of-living countries. None of those prophecies of doom came true. However, I don’t think that’s much comfort. Industries do die when they’re made obsolete by technology. Eventually a crisis will come along that the industry can’t just ride out.

The most optimistic position is probably that somehow demand for software engineers increases, because the total amount of software rises so rapidly, even though you now need fewer engineers per line of software. This is widely referred to as the Jevons effect. Along these lines, I see some engineers saying things like “I’ll always have a job cleaning up this AI-generated code”.

I just don’t think that’s likely. AI agents can fix bugs and clean up code as well as they can write new code: that is, better than many engineers, and improving each month. Why would companies hire engineers to manage their AI-generated code instead of just throwing more and better AI at it?

If the Jevons effect is true, I think we would have to be hitting some kind of AI programming plateau where the tools are good enough to produce lots of code (we’re here already), but not quite good enough to maintain it. This is prima facie plausible. Every software engineer knows that maintaining code is harder than writing it. But unfortunately, I don’t think it’s true.

My personal experience of using AI tools is that they’re getting better and better at maintaining code. I’ve spent the last year or so asking almost every question I have about a codebase to an AI agent in parallel while I look for the answer myself, and I’ve seen them go from hopeless to “sometimes faster than me” to “usually faster than me and sometimes more insightful”.

Right now, there’s still plenty of room for a competent software engineer in the loop. But that room is shrinking. I don’t think there are any genuinely new capabilities that AI agents would need in order to take my job. They’d just have to get better and more reliable at doing the things they can already do. So it’s hard for me to believe that demand for software engineers is going to increase over time instead of decrease.

Final thoughts

It sucks. I miss feeling like my job was secure, and that my biggest career problems would be grappling with things like burnout: internal struggles, not external ones. That said, it’s a bit silly for software engineers to complain when the automation train finally catches up to them.

At least I’m happy that I recognized that the good times were good while I was still in them. Even when the end of zero-interest rates made the industry less cosy, I still felt very lucky to be a software engineer. Even now I’m in a better position than many of my peers, particularly those who are very junior to the industry.

And hey, maybe I’m wrong! At this point, I hope I’m wrong, and that there really is some je ne sais quoi human element required to deliver good software. But if not, I and my colleagues are going to have to find something else to do.

edit: This post got some comments on Hacker News. Some commenters are doubtful, either because they don’t think AI coding is very good, or because they think human creativity/big-picture thinking/attention to detail will always be valuable. Others think ten years is way too optimistic. The top comment repeats the irony that I describe in the third paragraph of this post.

edit: This post also got some comments on the Serbian r/programming subreddit, some excellent comments on Tildes, which is a new one to me, and some more comments on lobste.rs.

ALT

John Berkey, “The Sightless Bird”

On this week’s installment of “Jeep Thoughts With David Tracy,” I want to talk about why I think the 1945 Willys CJ-2A may be the greatest off-road vehicle platform of all time.

The World War II Jeep may be the greatest Jeep of all time, but it’s not the best Jeep of all time. In fact, its successor, the civilian CJ-2A, was better. I am not disparaging the mighty Willys MB; it helped allies win World War II, and it set the standard for every single 4×4 to follow. The Land Rover Defender? The Ford Bronco? The Toyota Land Cruiser? The Nissan Patrol? All of these legends were inspired by the WWII Jeep.

But if I had to choose one vehicle to use both around the farm and off-road, I’d pick the civilian model that debuted for the 1945 model-year, the CJ-2A. It got rid of the Willys MB’s low-hanging fuel tank:

That tank, by the way, is pretty annoying to fill:

It also deleted the needlessly complex fuel line that goes from the tank, up the firewall, around the engine bay, and finally into the carburetor:

The CJ-2A bolstered the MB’s frame quite a bit by boxing the C-channel, it added a super useful rear tailgate where the MB had none, and — crucially — the CJ-2A’s T90 manual transmission is significantly more robust than the WWII Jeep’s.

Yes, the CJ-2A is slightly better than the WWII Jeep, a vehicle that I’ve argued remains one of the great off-road platforms of all time.

This has led me to conclude that the CJ-2A may very well be the greatest off-roader ever. What I mean by that is, I believe it is capable of traversing more obstacles than any other vehicle. Yes, some modern vehicles will get up dunes the Jeep can’t handle. Some will be able to power through deep mud pits the Jeep can’t handle. But when you factor in every type of terrain out there — rocks, mud, tight forests, brush — I believe the CJ-2A, especially if you added in a locking differential or two, can traverse the highest percentage of challenging obstacles. Here’s why.

Why The CJ-2A Is The GOAT Over Modern Off-Roaders

You might be thinking to yourself: How can an 80 year-old vehicle possibly still be the best off-roader when the world has the brand new Jeep Wrangler, Ford Bronco, and Toyota Land Cruiser?

The answer to that requires a look at what makes a vehicle great off-road. I’ve written this about many times, but the most important attribute of a great off-road vehicle is favorable geometry. That means short overhangs, good ground clearance, a small belly, light weight, and small overall dimensions. After geometry, things like traction, torque, underbody protection, and articulation become important.

Most modern off-roaders excel in the latter areas. Lockers have become commonplace on many SUVs, tires are better than they’ve ever been, engines are more powerful than ever (and transfer case/axle ratios are nice and short), disconnecting sway bars allow for great articulation, and skid plates abound. But what most modern off-roaders struggle with is weight (a two-door Wrangler weights two tons!) and overall vehicle geometry. Let’s focus on the latter.

Because of demand for increased interior volume (and also the need to package more features), vehicles have become larger. In order to achieve great approach, departure, and breakover angles (plus ground clearance), many modern vehicles have to resort to humongous tires. Seriously, the very smallest tire you can get on a modern Jeep Wrangler is 32-inches (see above). The big-dog Jeep Wrangler Rubicon TJ from just 20 years ago offered 31s, and those were considered big:

With tall tires, the overall vehicle becomes taller, and to achieve decent stability you need a wide enough stance, which is why a Jeep Wrangler JL is a full seven inches wider than a Jeep Wrangler TJ from 20 years ago.

So yes, modern off-roaders have great ground clearance and approach/departure angles, but to pull that off they have to make significant compromises on overall vehicle geometry.

“But what about that TJ you keep mentioning? That’s small, right?” And the answer is yes. The Jeep Wrangler TJ is one of the best off-roaders ever because it combines compact overall size with the amazing suspension pioneered by the Jeep Cherokee XJ and ZJ.

The TJ Rubicon is absolutely one of the off-road kings, with factory lockers, a 4:1 low range in the transfer case, skid plates, great approach/departure/breakover angles, and small overall dimensions. But I still think the CJ-2A is better.

Why The CJ-2A Is The GOAT Over Other Jeep CJs/Early Wranglers

It’s fairly obvious why I’d say the Willys is a stronger off-road platform than a new vehicle given how large vehicles have become and how heavily laden with safety tech they are. But what about other CJs like the CJ-5, CJ-7, and what about the Wrangler YJ and TJ?

For one, I can tell you as a matter of fact that the CJ-2A is better off-road than the YJ, because I have owned both, and the YJ does not come close. Its track bars limit articulation from the four leaf springs, its lower profile limits ground clearance, and frankly this results in an off-road experience that involves lots of scraping and lots of tire-lifts. I love my YJ, and it’s fun off-road, but a 1945 Willys CJ-2A crushes it on the hard stuff.

Even if you removed those track bars, the YJ couldn’t hold up. CJ-3Bs, CJ-5s, and CJ-7s may all offer more power than a CJ-2A, but the one area that they — along with any Jeep after 1953 (and also Land Cruisers and the like) — struggle with is top-heaviness.

Having short overall length is a boon for an off-roader. It allows the vehicle to push the wheels out towards the bumpers to create great approach and departure angles, without yielding a huge belly that creates a poor breakover angle. Being short is good.

The issue is that, in order to maintain stability up steep grades, a short vehicle has to have a low center of gravity. It’s the reason why the vehicles that you often see roll over when off-roading are Jeep CJ-5s, CJ-7s, Jeep YJs, and TJs. They’re not as short as an old CJ-2A, but they’re still quite short, and with their overhead valve engines, their center or gravity is taller than that of early, flathead engine-equipped CJs.

 

View this post on Instagram

 

A post shared by The Autopian (@theautopian)

This really began in 1953 with the CJ-3B. Overhead valves are a great thing for engine efficiency/power, but when we’re talking about pure off-road capability, you really don’t need a ton of power — you need thrust at the wheels. The CJ-2A creates this with a fairly torquey engine and also super short gearing in the axles: 5.38 to 1.

In 1953, the CJ-3B introduced the F-head overhead valve engine, which I quite love. But in order to package this engine, Willys-Overland had to extend the hood by about four inches, which required extending the cowl, which raised the windshield, which required a higher seating position. It was a chain reaction that led to a significantly taller machine. This can be an issue when climbing steep grades.

The other downside that many Jeep CJs and Wranglers have over the Willys CJ-2A is that they feature a roll bar. This is great for safety, of course, but if we’re talking about pure capability, being able to lower the windshield and create a slim profile that you can maneuver under downed trees and other low obstacles is a significant help.

The CJ-2A Has The Perfect Off-Road Silhouette

The CJ-2A is far from a truly perfect off-road vehicle. The leaf spring shackles limit approach and departure angle, the leaf springs themselves don’t flex as well as a coil spring setup, the frame flexes well but not quite as well as a flimsy WWII Jeep frame, and the Go-Devil under the hood doesn’t make a ton of power.

But the geometry is pretty much perfect. The rocker panels are way up high, as are most of the mechanical bits (many other Jeeps may technically have more ground clearance, but their rockers and more of their bellies are lower). The folding windshield and lack of a roll bar keep the profile low, the overall length is super short, the flathead motor keeps the center of gravity down, the width is really narrow, all the mechanicals are tucked up high, and weight is only 2,500 pounds. Combine this amazing geometry with a torquey engine, a low-range transfer case, and solid axles that flex well enough, and you’re basically a set of lockers away from being invincible.

It’s very unlikely that there will be a future vehicle that is as geometrically perfect as the CJ-2A. Gas engines have gotten too large, so hoods will be high. EVs will never come with two solid axles. A flat gas engine might solve this, but even then, the number of safety mandates in modern cars pretty much means that there will never be a vehicle as simple, as small, as lightweight, and as low as a CJ-2A.

There are plenty of vehicles that are great off-road, and many are right there with the CJ-2A. With lockers, many will outperform the CJ-2A on certain terrain. But as a platform, the CJ-2A (and its CJ-3A and M38 derivatives) is as close to off-road geometric perfection as there is. And there may not be a single vehicle out there that, with nothing but a simple mod or two (like a Lock Right locker), can handle a diversity of off-road obstacles as well as this first civilian Jeep.

The post Why The Jeep CJ-2A Is The Best Off-Roader Ever, And Possibly Always Will Be appeared first on The Autopian.