-
Notifications
You must be signed in to change notification settings - Fork 10.5k
Avoid pausing circuit simultaneously with disposing of it #63010
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR fixes race conditions that occur when attempting to pause a circuit after it has already been disposed, preventing exceptions from being thrown when resolving services from disposed DI scopes.
- Wraps pause logic in the circuit's single-threaded dispatcher to prevent concurrent execution with disposal operations
- Adds a disposal check before attempting to resolve services during pause operations
- Includes comprehensive test coverage for various race condition scenarios
Reviewed Changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
File | Description |
---|---|
src/Components/Server/src/Circuits/CircuitPersistenceManager.cs | Wraps pause logic in dispatcher invoke and adds disposal check |
src/Components/Server/src/Circuits/CircuitHost.cs | Adds internal method to expose disposal state |
src/Components/Server/src/Circuits/CircuitRegistry.cs | Changes method visibility from private to internal for testing |
src/Components/Server/test/Circuits/CircuitRegistryTest.cs | Adds comprehensive test coverage for race condition scenarios |
Co-authored-by: Copilot <[email protected]>
e2e on 3rd run passed, re-merging |
src/Components/Server/src/Circuits/CircuitPersistenceManager.cs
Outdated
Show resolved
Hide resolved
nice |
Prevent scope-related operations on disposed circuit
Description
The problem occurs only when we try to pause the circuit after it was already disposed. It can happen in scenarios:
The test call these actions synchronously (they await), so that the reproduction is stable. The real race was observed on CI with a frequency of ~20 hits per week.
Fix
aspnetcore/src/Components/Server/src/Circuits/CircuitHost.cs
Line 200 in 0f963e9
Wrapping pause logic in the dispatcher invoke prevents it from running concurrently with disposal that comes from e.g. termination or eviction.
- Even if pause will not run concurrently with disposal, it can still run just after disposal happened - see the tests. To prevent it from trying to discover services in a disposed DI scope, we checkremoved after the discussion - I could not find any evidence that pause can happen after disposal in real life.CircuitHost
disposal flag.Fixes - let's keep it open for some time: #62672
Frequency of failures on CI before this fix for reference:
