Informal policy on use of generative AI / assistance tools

[Lestropie]

@MRtrix3/mrtrix3-devs

Currently contemplating the prospect of having to overcome my clankophobia (how is this not a term yet?) and try to make use of AI coding tools to make a dent in the list of bugs / feature requests.

But before I even make an attempt, I want to run some thoughts past the team.

---

My biggest concern with the utilisation of these tools specifically in the context of research software is the prospect of erroneous code written by an AI having a deleterious effect on external scientific research.
We can quote:

Covered Software is provided under this License on an "as is" basis, without ... warranties that the Covered Software is free of defects ... (or) fit for a particular purpose ...

all we want, but I still feel a large burden of responsibility for ensuring quality and robustness in what is provided here. When some issue is identified, I put a lot of effort into backtracking through git blame and GitHub to establish what went wrong and when, including who wrote & reviewed the code, and establishing bidirectional links for anyone to follow along.

AI throws a spanner into the works here. It is going to become increasingly easy for both core developers and external researchers to generate and contribute functional code through prompt rather than hard labour; indeed one day C++ changes could be proposed by individuals with no knowledge of the language. We can be diligent about the addition of tests to verify additions or changes (though I'm conscious of AI being used to generate tests also, which could lead to a death spiral of refuse if one is not careful), but to me that's not enough.

Prior to myself or anyone else making changes to MRtrix through such tools, I'd like to establish clear policies around the documentation of such, integrated into the contribution instructions.

Here's what I think needs forethought:

1. Tracking

   We need the capability to retrospectively identify:

   • If the use of specific tools consistently leads to unexpected problems.
   • How much of the code base is AI- versus human-generated.

   This to me means documenting within individual commit messages that such tools have been used.

2. Accountability

   We need to be able to infer retrospectively whether the introduction of a particular problem was an error of code / communication written by a human versus an error resulting from imperfect review of code that was written by an AI.

   I've publicly self-flagellated on the importance of accountability for research software errors, so I don't like the thought of lazy AI use having deleterious effects on external scientific projects for which responsibility is deferred to silicon.

3. Attribution

   I want for the statistics on contributions toward the project to not be undermined by this novel capacity for generation of voluminous content for small effort.

   We already have the @MRtrixBot account for attributing auto-generated content to mitigate distortion of author line counts. Generative AI shares certain features with this: yes it's not just running a shell script, it involves prompt engineering / adequate problem definition, but it's nevertheless a mismatch between how much effort was invested and the reporting / perception of the proportion of one's contribution to the software.

The first thought that came to mind here for me was the way in which I often utilise the @MRtrixBot account for auto-generated content, where I generate three commits:

1. Authored exclusively by myself, adding or modifying the code responsible for the auto-generation
2. Authored exclusively by @MRtrixBot, contributing the generated content
3. Authored exclusively by myself, performing any requisite review / cleanup / revision

A similar approach to generative AI use, where one commit contains verbatim what was produced by the AI agent and then subsequent commits contain the human revisions to such, would to me be the most faithful encoding. It would however be a fair bit of overhead, which others might not be willing to take on. And it only really works for a singular prompt-response process and not for more dynamic pair-programming. I also have some recollection that in the early LLM boom popular opinion was that AI should not be considered an "author" of code, though I'm not finding any such arguments at the current time and don't recall the exact reasoning.

The other related technique is the use of git trailers, specifically "Co-authored-by". This is added automatically if accepting explicit code suggestions from PR code review within GitHub---personally I also use this if I manually make a change based on a suggestion from another dev that wasn't formatted as such---and co-contributor icons will show up in the GitHub interface.

---

Putting all of this together, I think the most sensible choice is to expand the use of git trailers.

My initial searching on the topic led me to this post (which is now also captured in the AI response to web search):

• Assisted-by means I wrote the code and AI helped either through prompts or inline completions up to roughly 33% generated code.
• Co-authored-by is the 50/50-ish bucket ranging from 34–-66% generated code.
• Generated-by means the majority of this code came from AI--—roughly 67-–100%.
• Commit-generated-by means AI summarized a conventional commit message for me (or similar trivial contribution) but none of the code was AI-modified in any meaningful way.

Other posts suggest even greater metadata, such as the precise model and prompt used. If this were to be embedded as part of a standardised integrated automated workflow I'd be down; but in the absence of such I think that's too intrusive. Tools like Git AI and GitButler seek to provide accountability on a per-line basis within commits, but I don't think we should be demanding such stringent requirements from anyone looking to contribute to the project. Whereas git trailers are not a massive request, and could be retrospectively added through rebasing if need be.

Completely open to opinions and alternative suggestions. I've near zero experience with these tools so am not speaking from a position of expertise. I am however well-versed in technical debt and protection against perverse incentives, so am invested in ensuring this does not turn out in the future to be a complete disaster.

[bjeurissen]

About accountability: Regardless of whether AI was used to generate code, responsibility remains with the person who submits it. I don’t see how the involvement of AI changes this.

About attribution: At this stage, I’m not overly concerned about attribution. If a contribution is of high quality and aligns with the project, I’m happy to attribute it to the submitter, since they remain responsible for correctness and robustness. A possible exception might be large-scale AI-assisted refactoring that is mostly cosmetic in nature. More fundamentally, though, the real issue here is the use of "code volume" as a metric for contribution/attribution, something that has always been muddy, even without AI, and that generative tools merely make more obvious rather than newly problematic.

About tracking: This will be hard to bring into practice. Many developers now use IDEs with Copilot or similar tools enabled, meaning that almost all coding could be considered AI-assisted to some degree, at least “Assisted-by,” if not “Co-authored-by.” The “Generated-by” category seems appropriate for cases like a single class performing a well-defined task that was fully produced by AI in response to a prompt, but then if it is part of a larger commit that includes human-generated code (or "Assisted-by" code ;-)), then things easily get complicated.

[daljit46]

My biggest concern with the utilisation of these tools specifically in the context of research software is the prospect of erroneous code written by an AI having a deleterious effect on external scientific research.

I'm struggling to understand what the worry is here. As @bjeurissen mentioned, erroneous code, whether it's generated by an LLM or manually typed, is always the responsibility of the person submitting the code. I don't think labeling the code as generated by AI will help in this regard. If the intention is to be able to somehow track how a mistaken piece of code was created, the usefulness of bookeeping generating prompts is rather limited fundamentally due to LLMs being non-deterministic: the same prompt will not generate the same code on successive runs.
I do think there may be a compelling argument from an ethical standpoint for developers to disclose whether a substantial amount of the code they've written is by LLMs, but I also believe that's a completely personal choice and can't really be enforced.

[Lestropie]

responsibility remains with the person who submits it
is always the responsibility of the person submitting the code

Correct.
But there's a reason why git distinguishes between author and committer.
Just like how the problem with students submitting AI-generated work is specifically about their putting their own name at the top of the document and/or their checking of the "this is my own work" box.

What I don't want is:

• People expecting me to filter their AI-generated code.
• People claiming personal responsibility for contributions for personal gain, then attempting to deflect responsibility to AI when something goes wrong; can't have your cake and eat it too.

Given one of the patch updates was delayed by ~ 9 months due to nobody reviewing PRs with single-digit line counts, you'll have to forgive me for being pessimistic about people's diligence in reviewing mass AI-generated code.

Since I refuse to spend my time reviewing somebody else's AI-generated code, if people decline to flag what is AI and what is not the only reasonable course of action is to assume the former in all circumstances and refuse to perform any code review. In which case the only reasonable way to preserve software quality I think is to enforce test coverage; even "tests required" isn't adequate as tests themselves could be slop. So #2773 might need to be prioritised.

More fundamentally, though, the real issue here is the use of "code volume" as a metric for contribution/attribution

Again, you'll have to forgive me for being a little salty about working myself to an early grave trying to keep the project out of abandonware status only to now be told that metrics for such don't matter.

If I were ever to catch anyone quoting authorship statistics having made the explicit choice to omit AI attribution there would be hell to pay.

I do think there may be a compelling argument from an ethical standpoint for developers to disclose whether a substantial amount of the code they've written is by LLMs, ...

If it is documented that such divulgence is necessary, and then at some point in the future someone attempts to deflect responsibility for fault based on their non-divulged use of AI, then you would at the very least be able to hold their feet to the fire for violation of contribution instructions. It's not much, but it's something.

And as per above I reject fundamentally the description of such as "code they've written".

... but I also believe that's a completely personal choice ...

There are ethical standards in research that aren't opt-in.
Ethics may have some catching up to do to figure out what the hell to do with the interaction of research and software in the LLM era, but my personal instinct is to stay ahead of the curve. The copyright message is conservative for safety reasons, not as a modus operandi.

... and can't really be enforced.

I'm not looking for a bullet-proof solution; I'm trying to reduce the unintended consequences of the boom by introducing rules that mitigate perverse incentives.
I'll be myself conforming to what I think is ethical, and we can only wait and see if the domain catches up.

Many developers now use IDEs with Copilot or similar tools enabled, meaning that almost all coding could be considered AI-assisted to some degree, at least “Assisted-by,” if not “Co-authored-by.”

If an IDE has always-on paired AI programming, surely automating insertion of a git trailer to all commits is not an insurmountable technical challenge, even if not perfectly conforming to the rule-of-thirds.