Using Hypothesis with AI Agents

AI coding agents (Claude Code, Gemini CLI, etc.) can read and write Hypothesis annotations through the Hypothesis MCP server. This lets agents participate in documentation review workflows — reading feedback left by team members and responding to or resolving comments while editing source files.

This page covers two workflows:

1. Commenting on published documentation — an agent annotates pages on the live site.
2. Reading and responding to comments during authoring — an agent working in the documentation/ repository fetches annotations, addresses feedback in source files, and replies to threads.

Prerequisites

Before using these workflows:

1. Hypothesis account — follow the How to Comment on Documentation guide to create an account and join the arda-products group.
2. Hypothesis MCP server — must be configured in Claude Code. See the Hypothesis MCP Setup for full instructions. The key requirements are:
   • An .npmrc in the workspace root with @arda-cards registry pointing to GitHub Packages (uses gh auth token for credentials — no separate PAT needed).
   • A Hypothesis API token stored in 1Password as op://Private/Hypothesis-API/credential. Generate a token at hypothes.is/account/developer.
   • The MCP server entry in settings.json (already included in the Arda workspace settings).

Workflow 1: Commenting on Published Documentation

An agent can leave annotations on any page of the published documentation site at https://arda-cards.github.io/documentation/.

Example prompts

Search for existing feedback on a page:

Search Hypothesis annotations on https://arda-cards.github.io/documentation/domain/information-model/
for any comments tagged "question" in the arda-products group.

Leave a review comment:

Create a Hypothesis annotation on https://arda-cards.github.io/documentation/current-system/architecture/design-pattern-index/
with the text "This section should reference the new State Engine pattern added in v2.4"
and tags ["suggestion", "architecture"].

Read all team annotations on a page:

Get all Hypothesis annotations for the arda-products group on
https://arda-cards.github.io/documentation/process/craft/project-management/project-planning/
and summarize the open discussion threads.

How it works

The agent uses these MCP tools:

• search_annotations with uri set to the page URL and group set to the group’s public ID.
• create_annotation with the page URL, comment text, and optional tags. Annotations default to the arda-products group.
• get_group_annotations to fetch all annotations across the group (useful for a broad review sweep).

Workflow 2: Reading and Responding During Authoring

When editing documentation source files, an agent can fetch annotations left on the published version of that page, address the feedback by modifying the source, and then reply to the annotation thread.

Example prompts

Fetch and address feedback:

I'm editing documentation/src/content/docs/domain/information-model/index.md.
Check Hypothesis for any annotations on the published version of this page
(https://arda-cards.github.io/documentation/domain/information-model/).
For each annotation tagged "suggestion" or "typo", make the fix in the source file
and reply to the annotation with what you changed.

Review sweep across a section:

Search Hypothesis for all annotations in the arda-products group tagged "question".
For each one, tell me which source file it corresponds to and summarize the question.

Resolve a discussion:

Update annotation <annotation-id> to add the tag "RESOLVED"
and reply with "Fixed in commit abc1234".

Mapping URLs to source files

The published URL https://arda-cards.github.io/documentation/<path>/ maps to the source file:

documentation/src/content/docs/<path>/index.md

or for non-directory pages:

documentation/src/content/docs/<path>.md

The agent can derive this mapping automatically from the annotation’s uri field.

Tool Reference

Tool	What it does	When to use it
get_groups	Lists groups you belong to	Find the arda-products group public ID
search_annotations	Search with filters (group, URI, text, user, tag)	Find annotations on a specific page or by a specific reviewer
get_annotation	Get full details of one annotation	Read the complete text and quote context of a specific comment
get_group_annotations	Fetch all annotations in a group (paginated)	Broad review sweep across all documented pages
create_annotation	Create a new annotation	Leave feedback on a published page
update_annotation	Update text or tags on an annotation	Mark a thread as resolved, or edit a previous comment

Tips

• Use tags consistently — question, suggestion, typo, RESOLVED are the established conventions (see How to Comment on Documentation).
• Default group — The MCP server defaults to the arda-products group (pubid e4e5jGAx) when no group is specified. Pass group: "__world__" only if you intend to create a public annotation. Use get_groups to discover pubids for other groups.
• Batch operations — get_group_annotations paginates internally and can fetch up to 2000 annotations in a single call. Use it for sweeps; use search_annotations for targeted queries.
• Permission prompts — Each MCP tool triggers a permission prompt on first use in a session. Read tools are safe to approve broadly. Write tools (create_annotation, update_annotation) modify annotations visible to the team.