LLMs for automated documentation generation

Automated documentation generation using large language models has become a vital aspect of software development, as it helps maintain code quality and reduces manual effort by automatically creating and updating documentation. One of the primary challenges developers face is keeping documentation up-to-date and in sync with the codebase. In this post, we will survey approaches for generating documentation from code, focusing on quality control and synchronization, and explore how to overcome the limitations of automated documentation generation.

Approaches to Automated Documentation Generation

Several approaches can be employed to generate documentation from code, including:

• JSDoc/TSDoc comments: These comments provide a way to document functions, classes, and variables directly in the code, allowing large language models to parse these comments and generate documentation.
• README generation: Large language models can generate README files based on the code structure and content, providing a high-level overview of the project, including its purpose, dependencies, and usage examples.
• API reference docs: By analyzing the code, large language models can generate API reference documentation, including endpoint descriptions, parameter information, and response formats, to help developers understand how to use the API.
• Changelogs: Large language models can also generate changelogs by comparing different versions of the codebase and highlighting changes, such as new features, bug fixes, and deprecations.

Example: Generating Documentation with JSDoc Comments

For example, consider a TypeScript function documented with JSDoc comments:

/**
 * Calculates the sum of two numbers.
 * @param {number} a The first number.
 * @param {number} b The second number.
 * @returns {number} The sum of a and b.
 */
function add(a: number, b: number): number {
  return a + b;
}

A large language model can parse these comments to generate documentation, including function descriptions, parameter information, and return types, using libraries such as jsdoc or typedoc.

Quality Control and Synchronization

To ensure the quality and accuracy of generated documentation, it is essential to implement quality control measures, such as:

• Regularly reviewing and updating JSDoc/TSDoc comments to reflect changes in the codebase, using tools like git hooks to automate the process.
• Using large language models to generate documentation in a continuous integration/continuous deployment (CI/CD) pipeline, ensuring that documentation is always up-to-date, with tools like GitHub Actions or Jenkins.
• Implementing automated testing to verify the accuracy of generated documentation, using frameworks like jest or mocha, to catch any errors or inconsistencies.

Overcoming Limitations

While automated documentation generation can significantly reduce manual effort, it is not without its limitations. For example:

• Large language models may struggle to understand complex code structures or nuances, leading to inaccurate or incomplete documentation, which can be mitigated by using more advanced models or fine-tuning them on specific codebases.
• Generated documentation may not always be concise or easy to read, requiring manual review and editing, which can be improved by using techniques like summarization or text simplification.

To overcome these limitations, developers can use techniques like:

(async () => {
  const { generateDocumentation } = await import("documentation-generator");
  const code = "path/to/codebase";
  const documentation = await generateDocumentation(code);
  console.log(documentation);
})();

This code uses an async IIFE to generate documentation from a codebase using a documentation-generator library, and logs the result to the console.

In conclusion, automated documentation generation using large language models can greatly improve code maintainability and reduce manual effort. By surveying approaches to documentation generation and implementing quality control measures, developers can ensure that their documentation is accurate, up-to-date, and in sync with the codebase. Start simple, use JSDoc/TSDoc comments, and integrate large language models into your CI/CD pipeline to automate documentation generation and improve overall code quality.