Why unmoderated AI patches are a pain in the ass - a lesson in code hygiene

[kiozen]

Moving the discussion from #1017 into the discussion area to keep the ticket clean.

@kiozen you say that the Brouter thing wasting your time but the Brouter Problem was first reported in 2021 - enough time to solve the problem but I know I impossible to do that when you can't reproduce the problem. The question is, what is better the invest time to solve a problem for all QMS User with the help of AI or to do nothing because not every User has this Problem.

The problem we are talking about is code hygiene. Those AI generated solutions solve a problem for sure, with the cost of adding unnecessary complexity and questionable code patterns. In fact it's a bit like a novice trying to write complex software. Many "if" many "else" and other conditions. The code is tinkered here and there until it works. In the end you get a software no one understands and no one wants to change because of the fear breaking something else. I have seen that too often in my career.

I just want to remind you of your 1000 lines of changes experiment in CMainWindow.cpp that solved a problem that turned out to be solvable in just a few lines. And I see the same problem in the BRouter PR.

In my experience there are two types of bugs:

• Small bugs that can be solved in a few lines of code.
• Conceptual bugs. They are much harder to fix. The temptation is to fix these with little patches here, little patches there. In the end we get the "novice code" I was talking about. The real fix is to analyze the problem, analyze the requirements, properly define how it should work and then rewrite the shit with a clean, simplified concept.

A project of the size of QMapShack inevitably ends up in the complexity trap. A lot of code, a lot of dependencies within the code. Many contributions of different quality. After a couple of years no one comprehends the code in total. Including me. That is why it is so important to keep the code base and the concepts simple and thus easy to understand. Anyone picking up a problem or a feature in the code has to read and to understand the existing code first. Even me. If the code is too confusing, too chaotic everyone will stop immediately and move on to other topics.

And this is why code hygiene is so important. If we do not pay attention on that, bad code and concepts will reach a tripping point and no one will be able to contribute.

Also keep in mind I am the only constant in this project. This is running now for more than 20 years. Contributors come and go. This is completely normal as focus and interests change over time. In the end I have to deal with the results. And that is why I pay that much attention on that topic.

This BRouter code is a fine example. It wasn't written by me. It was a huge PR at that time and compared to the Routino router - implemented by me - it's working different. At that time the concept seemed to be fine. Now there are some issues. The original author is gone. The root cause of the reported bugs is a conceptual one. The AI solution is fixing stuff but it makes the code even harder to understand. Therefore if no one else is picking the topic to rewrite the code with a clean concept, it will inevitably land upon me. So please understand that I am not willing to accept code that is too hard to read or too chaotic.

Speaking of AI. There is nothing wrong with AI. In fact I read the AI answer on Google instead of working my way thru all the Bla Bla and blown egos at Stackoverflow. The delegate stuff was implemented with a lot of AI help. But my experience is that AI produces a lot of bull shit, too. Same question asked on different days, two solutions. Both solutions work, however one is completely overengineered the other one is clean and straight forward. This teaches me, that you can't take the answers blindly. If you have a lot of experience you can get most out of the AI generated code. Either by teaching the AI or by refactoring the solution manually.

There is an interesting article on Heise.de (German):

https://www.heise.de/news/KI-Code-Schneller-geschrieben-langsamer-getestet-11215818.html

This is exactly what I experience. Especially this statement is fitting quite perfectly the current situation:

"A study by the research institute METR (Model Evaluation and Threat Research) shows that experienced developers working in familiar code environments with AI tools take an average of 19 percent longer—primarily due to additional verification and correction steps. One reason is that, when errors occur, they first have to familiarize themselves with the code generated by the AI."

From my point of view using AI to produce code is fine if:

• It's a small project like a script to do something
• You isolate a problem, get a solution and use that pattern in your more complex code.
• You ask the AI to review your code. I have done that with copilot lately. Most of the suggestions are garbage. But there are 1 or 2 good catches. Again, it needs expertise to make this a fruitful experience.

I have never tried to code a complex software using just AI. Never bother of the code, just tell the AI what you want. And report bugs. Would be interesting how this ends up.

[eyzdh]

i understand your point, and for project like QMS you need have a structure and rules.
After I put one PR, I learned my lesson.
That's why I called the brouter code as inspiration. Now I would call it as AInspiration because to make clear is just one step.
Other steps is compare the codes and write the best code for QMS that need to be done by an human and not ai.
I did the ai stuff because is the only thing what I can do for QMS as open source.

I don't now how this ends up with ai, but its interesting to test ai, and yes ai makes things wrong and you need more than one prompt.

I played in the last days just for fun with ai - I know is only good enough for my because code hygiene of ai.

• add for macOS .sh to install QT directly form QT without build with cmake
• Brouter Profil Editor like brouter.de
• Brouter Mouse Setting and splitting long routes (not much faster then expected)
• Routino Profile Editor and Mouse Setting

[kiozen]

This is a position paper of the QGis maintainers regarding the use of AI tools in PRs:

https://github.com/qgis/QGIS-Enhancement-Proposals/blob/master/qep-408-ai-tool-policy.md

I agree with that quite a lot. Especially the part about "Extractive Contributions".