Skip to content

Conversation

benceruleanlu
Copy link
Member

@benceruleanlu benceruleanlu commented Sep 24, 2025

Summary

UseNewMenu has been defaulted to Top in the app for over a year; Playwright’s test default lagged behind. This PR aligns the test default with reality and keeps legacy specs stable.

Changes

  • tests(e2e): default to 'Top' via fixture; specs that previously relied on the old implicit default now explicitly set 'Comfy.UseNewMenu' to 'Disabled'.
  • docs(browser-tests): remove outdated README note suggesting tests set 'Top' manually.

Review Focus

  • Intentional uses of 'Top' and 'Bottom' remain unchanged.
  • Confirm ComfyPage default remains 'Top' (see browser_tests/fixtures/ComfyPage.ts).

Screenshots (if applicable)

N/A

┆Issue is synchronized with this Notion page by Unito

@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Sep 24, 2025
Copy link

github-actions bot commented Sep 24, 2025

🎭 Playwright Test Results

⚠️ Tests passed with flaky tests

⏰ Completed at: 09/25/2025, 01:10:56 AM UTC

📈 Summary

  • Total Tests: 457
  • Passed: 424 ✅
  • Failed: 0
  • Flaky: 4 ⚠️
  • Skipped: 29 ⏭️

📊 Test Reports by Browser

  • chromium: View Report • ✅ 417 / ❌ 0 / ⚠️ 4 / ⏭️ 29
  • chromium-2x: View Report • ✅ 2 / ❌ 0 / ⚠️ 0 / ⏭️ 0
  • chromium-0.5x: View Report • ✅ 1 / ❌ 0 / ⚠️ 0 / ⏭️ 0
  • mobile-chrome: View Report • ✅ 4 / ❌ 0 / ⚠️ 0 / ⏭️ 0

🎉 Click on the links above to view detailed test results for each browser configuration.

@DrJKL DrJKL self-assigned this Sep 24, 2025
DrJKL
DrJKL previously approved these changes Sep 24, 2025
@DrJKL DrJKL assigned benceruleanlu and unassigned DrJKL Sep 24, 2025
@benceruleanlu
Copy link
Member Author

the only failing test is just flaky

image

@benceruleanlu benceruleanlu added the New Browser Test Expectations New browser test screenshot should be set by github action label Sep 24, 2025
Copy link
Contributor

@christian-byrne christian-byrne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM + 2 questions!

try {
await comfyPage.setupSettings({
'Comfy.UseNewMenu': 'Disabled',
'Comfy.UseNewMenu': 'Top',
Copy link
Contributor

@christian-byrne christian-byrne Sep 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question (non-blocking): would it work to just remove this line altogether since this is the default value?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah I think so, good point, will make this change after merge since we have the setting to dismiss approvals after commits

Comment on lines -59 to -66
**Most tests require the new menu system** - Add to your test:

```typescript
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.setSetting('Comfy.UseNewMenu', 'Top')
})
```

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question (non-blocking): should we just change this to say like "Some tests require the old menu system"? Or maybe not because that should never be necessary anymore?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be clear enough, since it would say that explicitly by setting it to Disabled, so I don't think this would be needed

@benceruleanlu benceruleanlu merged commit bc45492 into main Sep 25, 2025
21 checks passed
@benceruleanlu benceruleanlu deleted the bl-new-menu-top-default branch September 25, 2025 01:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
New Browser Test Expectations New browser test screenshot should be set by github action size:L This PR changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants