Really? You don't think it's a matter of updating the prompt and expanding the context to include more of the code?

I don't know the exact method of how they generated these comments, but I think just focusing a prompt on a single method with the entire codebase in context would yield much better results.

   isUserAMonkey

   Added in API level 8

   public static boolean isUserAMonkey ()

   Returns "true" if the user interface is currently being messed with by a monkey.

A product that scans your repo and identifies mis-named functions would be compelling, though.

It reflects back on them really well.

I've never used Spring Boot, but I have to give them credit for this. I don't know that I'd have similar restraint

Consider this real example from the pull request:

  /**
  * Returns the plugin version property.
  * @return the plugin version property
  */
  @Input
  public Property<String> getPluginVersion() {
      return this.pluginVersion;

Here's a sample of the crap you get, for 3000+ files, multiple comments per file, all this inane, that anyone with two brain cells could figure out just by reading the name of the function. If the project didn't already have these sort of comments, then adding them now is a choice that the maintainer of standards would have to make, not some random contributor.

The second point doesn't seem like a positive on the whole, except maybe for the company selling it, in the "there's good money to be made in being part of the problem" sense.

Edit: I noticed one comment that seemed like it added some useful context: For CheckAdditionalSpringConfigurationMetadata.Report::iterator it says "Each line represents a source code file and its associated problems." However, this turns out to be completely wrong, if I'm reading the code right, because a line could represent a filename, or a problem, or just be a blank line. The comment gives the appearance of being helpful while actually just adding confusion and wasting everyone's time.

Always up-to-date and succinct auto-generated LLM-generated 'documentation' that describes a method in terms of who calls it (and perhaps why?) and the behaviour of the whole deep call graph inside the function seems like something that could be quite useful when looking at a method that doesn't have any human written documentation available, and isn't obvious... but it shouldn't masquerade as if it's actual API documentation that describes a durable contract.

If this synthesised analysis could flag incompatibilities between the contract described in API documentation and the actual code behaviour that could be useful (similar to what tools do today for nullable/non-nullable-annotated args/returns, except on fuzzier natural language descriptions of higher level behaviours). That seems like a much harder reasoning problem for the LLM to solve, though.