Inconsistency about the Agent mode between README and the implementation

[yukukotani]

README says that

For automation and scheduled tasks without trigger checking.

• Triggers: Always runs (no trigger checking)

and the example here is triggered by pull_request event, while the implementation here requires workflow_dispatch or schedule events to trigger.

Which is right?

[yukukotani]

I'm wondering which to use for label-based workflows

[burkematthew]

Seconding this observation. My workflow file looked like this:

name: Claude Assistant
on:
  pull_request:

permissions:
  pull-requests: write
  contents: read

jobs:
  claude-dependabot-review:
    runs-on: [self-hosted, linux, x64]
    if: github.event.pull_request.user.login == 'burkematthew'
    steps:
      - name: Checkout routing-service
        uses: actions/checkout@v4

      - uses: anthropics/claude-code-action@beta
        with:
          claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
          github_token: ${{ secrets.GITHUB_TOKEN }}
          mode: agent
          direct_prompt: <Example prompt>

Digging further, this looks like the documented behavior changed here.

Once I rolled back to commit 7c5a98d for my Github Action, I was able to see triggers working automatically.

[burkematthew]

A little more information on my digging here. Even rolling back to the previous version results in some issues because agent mode isn't setup to create the initial tracking comment:

  shouldCreateTrackingComment() {
    return false;
  },

But, then when getting further into the workflow, we try to output the commentId to claude_comment_id to set CLAUDE_COMMENT_ID. So then when we get to update_claude_comment, we run into the error of CLAUDE_COMMENT_ID environment variable is required.

[yukukotani]

it looks like that the concept of Execution Modes is still under development and unstable. Let's stay tuned.

[km-anthropic]

Thanks for reporting this inconsistency @yukokotani @burkematthew!

1. Documentation Fix: I've just created PR docs: clarify agent mode only works with workflow_dispatch and schedule events #378 to update the documentation to accurately reflect the current implementation. Agent mode indeed only works with workflow_dispatch and schedule events, and the docs now make this clear.

2. New Review Mode: In PR feat: add review mode for automated PR code reviews #374, we're introducing a new review mode that might better suit your needs if you're looking to automatically review PRs. This new mode is specifically designed for automated PR reviews (with inline comments) and will work with pull_request events. It provides a cleaner solution for the use case shown in the example.

So while agent mode will continue to be limited to workflow_dispatch and schedule events (as per the implementation), the new review mode will offer the automated PR review functionality that many users are looking for.

[rahim-figs]

I was using the agent mode for this use case, which is now broken.

• Invoke Claude code on pull_request as one of the jobs, and after analyzing the changes in the PR, Claude selects a bunch of E2E tests from our suite which are relevant to the changes.
• We run only the tests selected by Claude, instead of running the entire suite (for speed, and cost - our tests are AI driven).

Any idea how to get this to work? @km-anthropic