Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 11 additions & 10 deletions examples/todomvc/.claude/agents/playwright-test-generator.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
name: playwright-test-generator
description: Use this agent when you need to create automated browser tests using Playwright. Examples: <example>Context: User wants to test a login flow on their web application. user: 'I need a test that logs into my app at localhost:3000 with username [email protected] and password 123456, then verifies the dashboard page loads' assistant: 'I'll use the generator agent to create and validate this login test for you' <commentary> The user needs a specific browser automation test created, which is exactly what the generator agent is designed for. </commentary></example><example>Context: User has built a new checkout flow and wants to ensure it works correctly. user: 'Can you create a test that adds items to cart, proceeds to checkout, fills in payment details, and confirms the order?' assistant: 'I'll use the generator agent to build a comprehensive checkout flow test' <commentary> This is a complex user journey that needs to be automated and tested, perfect for the generator agent. </commentary></example>
tools: Glob, Grep, Read, Write, mcp__playwright-test__browser_click, mcp__playwright-test__browser_drag, mcp__playwright-test__browser_evaluate, mcp__playwright-test__browser_file_upload, mcp__playwright-test__browser_handle_dialog, mcp__playwright-test__browser_hover, mcp__playwright-test__browser_navigate, mcp__playwright-test__browser_press_key, mcp__playwright-test__browser_select_option, mcp__playwright-test__browser_snapshot, mcp__playwright-test__browser_type, mcp__playwright-test__browser_verify_element_visible, mcp__playwright-test__browser_verify_list_visible, mcp__playwright-test__browser_verify_text_visible, mcp__playwright-test__browser_verify_value, mcp__playwright-test__browser_wait_for, mcp__playwright-test__test_setup_page
tools: Glob, Grep, Read, Write, mcp__playwright-test__browser_click, mcp__playwright-test__browser_drag, mcp__playwright-test__browser_evaluate, mcp__playwright-test__browser_file_upload, mcp__playwright-test__browser_handle_dialog, mcp__playwright-test__browser_hover, mcp__playwright-test__browser_navigate, mcp__playwright-test__browser_press_key, mcp__playwright-test__browser_select_option, mcp__playwright-test__browser_session_log, mcp__playwright-test__browser_snapshot, mcp__playwright-test__browser_type, mcp__playwright-test__browser_verify_element_visible, mcp__playwright-test__browser_verify_list_visible, mcp__playwright-test__browser_verify_text_visible, mcp__playwright-test__browser_verify_value, mcp__playwright-test__browser_wait_for, mcp__playwright-test__test_setup_page
model: sonnet
color: blue
---
Expand All @@ -17,7 +17,7 @@ Your process is methodical and thorough:
expected outcomes and validation points

2. **Interactive Execution**
- For each scenario, start with the `test_setup_page` tool to set up page for the scenario
- Start with the `test_setup_page` tool to set up page for the scenario
- Use Playwright tools to manually execute each step of the scenario in real-time
- Verify that each action works as expected
- Identify the correct locators and interaction patterns
Expand All @@ -26,15 +26,16 @@ Your process is methodical and thorough:

3. **Test Code Generation**

After successfully completing the manual execution, generate clean, maintainable
@playwright/test source code that follows following convention:
After successfully completing the manual execution

- One file per scenario, one test in a file
- Use seed test content (copyright, structure) to emit consistent tests.
- File name must be fs-friendly scenario name
- Test must be placed in a describe matching the top-level test plan item
- Test title must match the scenario name
- Includes a comment with the step text before each step execution
- Read seed test content (copyright, structure) to emit consistent tests.
- Retrieve code snippets with the `test_browser_session_log`
- Based on the seed test and code snippets, generate clean, maintainable @playwright/test source code:
- One file per scenario, one test in a file
- File name must be fs-friendly scenario name
- Test must be placed in a describe matching the top-level test plan item
- Test title must match the scenario name
- Includes a comment with the step text before each step execution

<example-generation>
For following plan:
Expand Down
21 changes: 11 additions & 10 deletions examples/todomvc/.github/chatmodes/🎭 generator.chatmode.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
description: Use this agent when you need to create automated browser tests using Playwright.
tools: ['createFile', 'createDirectory', 'fileSearch', 'textSearch', 'listDirectory', 'readFile', 'test_browser_click', 'test_browser_drag', 'test_browser_evaluate', 'test_browser_file_upload', 'test_browser_handle_dialog', 'test_browser_hover', 'test_browser_navigate', 'test_browser_press_key', 'test_browser_select_option', 'test_browser_snapshot', 'test_browser_type', 'test_browser_verify_element_visible', 'test_browser_verify_list_visible', 'test_browser_verify_text_visible', 'test_browser_verify_value', 'test_browser_wait_for', 'test_setup_page']
tools: ['createFile', 'createDirectory', 'fileSearch', 'textSearch', 'listDirectory', 'readFile', 'test_browser_click', 'test_browser_drag', 'test_browser_evaluate', 'test_browser_file_upload', 'test_browser_handle_dialog', 'test_browser_hover', 'test_browser_navigate', 'test_browser_press_key', 'test_browser_select_option', 'test_browser_session_log', 'test_browser_snapshot', 'test_browser_type', 'test_browser_verify_element_visible', 'test_browser_verify_list_visible', 'test_browser_verify_text_visible', 'test_browser_verify_value', 'test_browser_wait_for', 'test_setup_page']
---

You are a Playwright Test Generator, an expert in browser automation and end-to-end testing.
Expand All @@ -14,7 +14,7 @@ Your process is methodical and thorough:
expected outcomes and validation points

2. **Interactive Execution**
- For each scenario, start with the `test_setup_page` tool to set up page for the scenario
- Start with the `test_setup_page` tool to set up page for the scenario
- Use Playwright tools to manually execute each step of the scenario in real-time
- Verify that each action works as expected
- Identify the correct locators and interaction patterns
Expand All @@ -23,15 +23,16 @@ Your process is methodical and thorough:

3. **Test Code Generation**

After successfully completing the manual execution, generate clean, maintainable
@playwright/test source code that follows following convention:
After successfully completing the manual execution

- One file per scenario, one test in a file
- Use seed test content (copyright, structure) to emit consistent tests.
- File name must be fs-friendly scenario name
- Test must be placed in a describe matching the top-level test plan item
- Test title must match the scenario name
- Includes a comment with the step text before each step execution
- Read seed test content (copyright, structure) to emit consistent tests.
- Retrieve code snippets with the `test_browser_session_log`
- Based on the seed test and code snippets, generate clean, maintainable @playwright/test source code:
- One file per scenario, one test in a file
- File name must be fs-friendly scenario name
- Test must be placed in a describe matching the top-level test plan item
- Test title must match the scenario name
- Includes a comment with the step text before each step execution

<example-generation>
For following plan:
Expand Down
20 changes: 11 additions & 9 deletions packages/playwright/src/agents/generator.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ tools:
- playwright-test/browser_navigate
- playwright-test/browser_press_key
- playwright-test/browser_select_option
- playwright-test/browser_session_log
- playwright-test/browser_snapshot
- playwright-test/browser_type
- playwright-test/browser_verify_element_visible
Expand All @@ -38,7 +39,7 @@ Your process is methodical and thorough:
expected outcomes and validation points

2. **Interactive Execution**
- For each scenario, start with the `test_setup_page` tool to set up page for the scenario
- Start with the `test_setup_page` tool to set up page for the scenario
- Use Playwright tools to manually execute each step of the scenario in real-time
- Verify that each action works as expected
- Identify the correct locators and interaction patterns
Expand All @@ -47,15 +48,16 @@ Your process is methodical and thorough:

3. **Test Code Generation**

After successfully completing the manual execution, generate clean, maintainable
@playwright/test source code that follows following convention:
After successfully completing the manual execution

- One file per scenario, one test in a file
- Use seed test content (copyright, structure) to emit consistent tests.
- File name must be fs-friendly scenario name
- Test must be placed in a describe matching the top-level test plan item
- Test title must match the scenario name
- Includes a comment with the step text before each step execution
- Read seed test content (copyright, structure) to emit consistent tests.
- Retrieve code snippets with the `test_browser_session_log`
- Based on the seed test and code snippets, generate clean, maintainable @playwright/test source code:
- One file per scenario, one test in a file
- File name must be fs-friendly scenario name
- Test must be placed in a describe matching the top-level test plan item
- Test title must match the scenario name
- Includes a comment with the step text before each step execution

<example-generation>
For following plan:
Expand Down
4 changes: 2 additions & 2 deletions packages/playwright/src/mcp/browser/browserServerBackend.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ export class BrowserServerBackend implements ServerBackend {
}

async initialize(server: mcpServer.Server, clientInfo: mcpServer.ClientInfo): Promise<void> {
this._sessionLog = this._config.saveSession ? await SessionLog.create(this._config, clientInfo) : undefined;
this._sessionLog = new SessionLog(this._config, clientInfo);
this._context = new Context({
config: this._config,
browserContextFactory: this._browserContextFactory,
Expand All @@ -66,7 +66,7 @@ export class BrowserServerBackend implements ServerBackend {
try {
await tool.handle(context, parsedArguments, response);
await response.finish();
this._sessionLog?.logResponse(response);
await this._sessionLog?.logResponse(response);
} catch (error: any) {
response.addError(String(error));
} finally {
Expand Down
8 changes: 5 additions & 3 deletions packages/playwright/src/mcp/browser/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ export const defaultConfig: FullConfig = {
viewport: null,
},
},
capabilities: [],
network: {
allowedOrigins: undefined,
blockedOrigins: undefined,
Expand All @@ -98,6 +99,7 @@ export type FullConfig = Config & {
launchOptions: NonNullable<BrowserUserConfig['launchOptions']>;
contextOptions: NonNullable<BrowserUserConfig['contextOptions']>;
},
capabilities: NonNullable<Config['capabilities']>,
network: NonNullable<Config['network']>,
saveTrace: boolean;
server: NonNullable<Config['server']>,
Expand Down Expand Up @@ -307,13 +309,13 @@ export function outputDir(config: FullConfig, clientInfo: ClientInfo): string {
?? path.join(tmpDir(), String(clientInfo.timestamp));
}

export async function outputFile(config: FullConfig, clientInfo: ClientInfo, fileName: string, options: { origin: 'code' | 'llm' | 'web', reason: string }): Promise<string> {
const file = await resolveFile(config, clientInfo, fileName, options);
export function outputFile(config: FullConfig, clientInfo: ClientInfo, fileName: string, options: { origin: 'code' | 'llm' | 'web', reason: string }): string {
const file = resolveFile(config, clientInfo, fileName, options);
debug('pw:mcp:file')(options.reason, file);
return file;
}

async function resolveFile(config: FullConfig, clientInfo: ClientInfo, fileName: string, options: { origin: 'code' | 'llm' | 'web' }): Promise<string> {
function resolveFile(config: FullConfig, clientInfo: ClientInfo, fileName: string, options: { origin: 'code' | 'llm' | 'web' }): string {
const dir = outputDir(config, clientInfo);

// Trust code.
Expand Down
4 changes: 2 additions & 2 deletions packages/playwright/src/mcp/browser/context.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,13 +38,13 @@ const testDebug = debug('pw:mcp:test');
type ContextOptions = {
config: FullConfig;
browserContextFactory: BrowserContextFactory;
sessionLog: SessionLog | undefined;
sessionLog: SessionLog;
clientInfo: ClientInfo;
};

export class Context {
readonly config: FullConfig;
readonly sessionLog: SessionLog | undefined;
readonly sessionLog: SessionLog;
readonly options: ContextOptions;
private _browserContextPromise: Promise<BrowserContextFactoryResult> | undefined;
private _browserContextFactory: BrowserContextFactory;
Expand Down
144 changes: 80 additions & 64 deletions packages/playwright/src/mcp/browser/sessionLog.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,26 +41,30 @@ type LogEntry = {

export class SessionLog {
private _folder: string;
private _file: string;
private _ordinal = 0;
private _pendingEntries: LogEntry[] = [];
private _entries: LogEntry[] = [];
private _sessionFileQueue = Promise.resolve();
private _flushEntriesTimeout: NodeJS.Timeout | undefined;
private _mode: 'disk' | 'memory' | 'none';
private _includeSnapshots: boolean;

constructor(sessionFolder: string) {
this._folder = sessionFolder;
this._file = path.join(this._folder, 'session.md');
constructor(config: FullConfig, clientInfo: mcpServer.ClientInfo) {
this._folder = outputFile(config, clientInfo, `session-${Date.now()}`, { origin: 'code', reason: 'Saving session' });
this._mode = config.saveSession ? 'disk' : config.capabilities.includes('session') ? 'memory' : 'none';
this._includeSnapshots = !!config.saveSession;
}

static async create(config: FullConfig, clientInfo: mcpServer.ClientInfo): Promise<SessionLog> {
const sessionFolder = await outputFile(config, clientInfo, `session-${Date.now()}`, { origin: 'code', reason: 'Saving session' });
await fs.promises.mkdir(sessionFolder, { recursive: true });
// eslint-disable-next-line no-console
console.error(`Session: ${sessionFolder}`);
return new SessionLog(sessionFolder);
serializedLog(): string {
const lines: string[] = [];
for (const entry of this._entries)
this._serializeEntry(entry, lines);
return lines.join('\n');
}

logResponse(response: Response) {
async logResponse(response: Response) {
if (this._mode === 'none')
return;

const entry: LogEntry = {
timestamp: performance.now(),
toolCall: {
Expand All @@ -70,15 +74,19 @@ export class SessionLog {
isError: response.isError(),
},
code: response.code(),
tabSnapshot: response.tabSnapshot(),
tabSnapshot: this._includeSnapshots ? response.tabSnapshot() : undefined,
};
this._appendEntry(entry);
this._entries.push(entry);
await this._flushEntries();
}

logUserAction(action: actions.Action, tab: Tab, code: string, isUpdate: boolean) {
if (this._mode === 'none')
return;

code = code.trim();
if (isUpdate) {
const lastEntry = this._pendingEntries[this._pendingEntries.length - 1];
const lastEntry = this._entries[this._entries.length - 1];
if (lastEntry?.userAction?.name === action.name) {
lastEntry.userAction = action;
lastEntry.code = code;
Expand All @@ -87,7 +95,7 @@ export class SessionLog {
}
if (action.name === 'navigate') {
// Already logged at this location.
const lastEntry = this._pendingEntries[this._pendingEntries.length - 1];
const lastEntry = this._entries[this._entries.length - 1];
if (lastEntry?.tabSnapshot?.url === action.url)
return;
}
Expand All @@ -104,74 +112,82 @@ export class SessionLog {
downloads: [],
},
};
this._appendEntry(entry);
this._entries.push(entry);
this._scheduleFlushEntries();
}

private _appendEntry(entry: LogEntry) {
this._pendingEntries.push(entry);
private _scheduleFlushEntries() {
if (this._flushEntriesTimeout)
clearTimeout(this._flushEntriesTimeout);
this._flushEntriesTimeout = setTimeout(() => this._flushEntries(), 1000);
}

private async _flushEntries() {
if (this._mode === 'disk')
await this._flushEntriesToDisk();
}

private async _flushEntriesToDisk() {
await fs.promises.mkdir(this._folder, { recursive: true });
clearTimeout(this._flushEntriesTimeout);
const entries = this._pendingEntries;
this._pendingEntries = [];
const entries = this._entries;
this._entries = [];
const lines: string[] = [''];
for (const entry of entries)
this._serializeEntry(entry, lines);
const file = path.join(this._folder, 'session.md');
this._sessionFileQueue = this._sessionFileQueue.then(() => fs.promises.appendFile(file, lines.join('\n')));
}

for (const entry of entries) {
const ordinal = (++this._ordinal).toString().padStart(3, '0');
if (entry.toolCall) {
private _serializeEntry(entry: LogEntry, lines: string[]) {
const ordinal = (++this._ordinal).toString().padStart(3, '0');
if (entry.toolCall) {
lines.push(
`#### Tool call: ${entry.toolCall.toolName}`,
`- Args`,
'```json',
JSON.stringify(entry.toolCall.toolArgs, null, 2),
'```',
);
if (entry.toolCall.result) {
lines.push(
`### Tool call: ${entry.toolCall.toolName}`,
`- Args`,
'```json',
JSON.stringify(entry.toolCall.toolArgs, null, 2),
entry.toolCall.isError ? `- Error` : `- Result`,
'```',
);
if (entry.toolCall.result) {
lines.push(
entry.toolCall.isError ? `- Error` : `- Result`,
'```',
entry.toolCall.result,
'```',
);
}
}

if (entry.userAction) {
const actionData = { ...entry.userAction } as any;
delete actionData.ariaSnapshot;
delete actionData.selector;
delete actionData.signals;

lines.push(
`### User action: ${entry.userAction.name}`,
`- Args`,
'```json',
JSON.stringify(actionData, null, 2),
entry.toolCall.result,
'```',
);
}
}

if (entry.code) {
lines.push(
`- Code`,
'```js',
entry.code,
'```');
}
if (entry.userAction) {
const actionData = { ...entry.userAction } as any;
delete actionData.ariaSnapshot;
delete actionData.selector;
delete actionData.signals;

lines.push(
`#### User action: ${entry.userAction.name}`,
`- Args`,
'```json',
JSON.stringify(actionData, null, 2),
'```',
);
}

if (entry.tabSnapshot) {
const fileName = `${ordinal}.snapshot.yml`;
fs.promises.writeFile(path.join(this._folder, fileName), entry.tabSnapshot.ariaSnapshot).catch(logUnhandledError);
lines.push(`- Snapshot: ${fileName}`);
}
if (entry.code) {
lines.push(
`- Code`,
'```js',
entry.code,
'```');
}

lines.push('', '');
if (entry.tabSnapshot) {
const fileName = `${ordinal}.snapshot.yml`;
fs.promises.writeFile(path.join(this._folder, fileName), entry.tabSnapshot.ariaSnapshot).catch(logUnhandledError);
lines.push(`- Snapshot: ${fileName}`);
}

this._sessionFileQueue = this._sessionFileQueue.then(() => fs.promises.appendFile(this._file, lines.join('\n')));
lines.push('', '');
}
}
Loading
Loading