-
Notifications
You must be signed in to change notification settings - Fork 810
Fix race condition that caused inprogress connect to be canceled #2618
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
Comments suppressed due to low confidence (2)
src/Grpc.Net.Client/Balancer/Subchannel.cs:269
- Using Debug.Assert here may not enforce the invariant in production builds. Consider adding a runtime check that handles the unexpected case when connectionRequested is false.
Debug.Assert(connectionRequested, "Ensure that only expected state made it to this point.");
test/Grpc.Net.Client.Tests/Balancer/ConnectionManagerTests.cs:686
- [nitpick] Consider unsubscribing from the MessageLogged event or scoping the handler to the duration of the test to avoid potential interference with other tests if the event is raised multiple times.
testSink.MessageLogged += (w) =>
|
||
// There is already a connect in-progress on this transport. | ||
// Don't cancel and start again as that causes queued requests waiting on the connect to fail. | ||
if (_connectContext != null && !_connectContext.Disposed) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had to check the file to see if this had a race condition (i.e. can _connectContext
become null
; it can't) - it isn't, but may I suggest:
if (_connectContext is { Disposed: false })
so that it is obviously not a race (i.e. single-read)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
seems fine; very minor style thing - not a hill either of us needs to die on
Updated [Grpc.Net.Client](https://github.com/grpc/grpc-dotnet) from 2.65.0 to 2.71.0. <details> <summary>Release notes</summary> _Sourced from [Grpc.Net.Client's releases](https://github.com/grpc/grpc-dotnet/releases)._ ## 2.71.0 ## What's Changed * Remove old dotnet-core feed by @JamesNK in grpc/grpc-dotnet#2611 * Bump axios from 1.7.4 to 1.8.4 in /testassets/InteropTestsGrpcWebWebsite/Tests by @dependabot in grpc/grpc-dotnet#2615 * Bump @babel/helpers from 7.25.0 to 7.27.0 in /testassets/InteropTestsGrpcWebWebsite/Tests by @dependabot in grpc/grpc-dotnet#2616 * Bump tar-fs from 3.0.6 to 3.0.8 in /testassets/InteropTestsGrpcWebWebsite/Tests by @dependabot in grpc/grpc-dotnet#2619 * Fix race condition that caused inprogress connect to be canceled by @JamesNK in grpc/grpc-dotnet#2618 * Bump tools package to 2.71 by @apolcyn in grpc/grpc-dotnet#2621 * Update NuGet package versions by @JamesNK in grpc/grpc-dotnet#2620 * bump version to 2.71.0-pre1 by @apolcyn in grpc/grpc-dotnet#2622 * Bump version on 2.71 for final release by @apolcyn in grpc/grpc-dotnet#2627 **Full Changelog**: grpc/grpc-dotnet@v2.70.0...v2.71.0 ## 2.71.0-pre1 ## What's Changed * Remove old dotnet-core feed by @JamesNK in grpc/grpc-dotnet#2611 * Bump axios from 1.7.4 to 1.8.4 in /testassets/InteropTestsGrpcWebWebsite/Tests by @dependabot in grpc/grpc-dotnet#2615 * Bump @babel/helpers from 7.25.0 to 7.27.0 in /testassets/InteropTestsGrpcWebWebsite/Tests by @dependabot in grpc/grpc-dotnet#2616 * Bump tar-fs from 3.0.6 to 3.0.8 in /testassets/InteropTestsGrpcWebWebsite/Tests by @dependabot in grpc/grpc-dotnet#2619 * Fix race condition that caused inprogress connect to be canceled by @JamesNK in grpc/grpc-dotnet#2618 * Bump tools package to 2.71 by @apolcyn in grpc/grpc-dotnet#2621 * Update NuGet package versions by @JamesNK in grpc/grpc-dotnet#2620 * bump version to 2.71.0-pre1 by @apolcyn in grpc/grpc-dotnet#2622 **Full Changelog**: grpc/grpc-dotnet@v2.70.0...v2.71.0-pre1 ## 2.70.0 ## What's Changed * update ArgumentNullException.ThrowIfNull usage by @WeihanLi in grpc/grpc-dotnet#2563 * use nameof for CallerArgumentExpression by @WeihanLi in grpc/grpc-dotnet#2562 * Correctness: Make some private & internal classes sealed where possible by @Henr1k80 in grpc/grpc-dotnet#2559 * Bump vue from 2.6.14 to 3.0.0 in /examples/Spar/Server/ClientApp by @dependabot in grpc/grpc-dotnet#2565 * Bump cross-spawn from 7.0.3 to 7.0.6 in /testassets/InteropTestsGrpcWebWebsite/Tests by @dependabot in grpc/grpc-dotnet#2574 * [vote]Added Active maintainers into MAINTAINERS.md. by @subhraOffGit in grpc/grpc-dotnet#2449 * Refactor: Use `await using` for `packageVersionStream` to ensure proper disposal of async resources by @dexcompiler in grpc/grpc-dotnet#2521 * Performance microoptimizations by @Henr1k80 in grpc/grpc-dotnet#2558 * Complete health checks watch service on server shutting down by @JamesNK in grpc/grpc-dotnet#2582 * Avoid using ConcurrentDictionary for channels with few methods by @JamesNK in grpc/grpc-dotnet#2597 * Bump elliptic from 6.6.0 to 6.6.1 in /examples/Spar/Server/ClientApp by @dependabot in grpc/grpc-dotnet#2599 * Move updating connectivity state outside of subchannel lock by @JamesNK in grpc/grpc-dotnet#2601 * Bump Grpc.Tools dependency by @apolcyn in grpc/grpc-dotnet#2603 * bump version on v2.70.x branch by @apolcyn in grpc/grpc-dotnet#2604 * Change version to 2.70.0 by @JamesNK in grpc/grpc-dotnet#2610 ## New Contributors * @Henr1k80 made their first contribution in grpc/grpc-dotnet#2559 * @subhraOffGit made their first contribution in grpc/grpc-dotnet#2449 * @dexcompiler made their first contribution in grpc/grpc-dotnet#2521 **Full Changelog**: grpc/grpc-dotnet@v2.67.0...v2.70.0 ## 2.67.0 ## What's Changed * precompile condition clean by @Varorbc in grpc/grpc-dotnet#2528 * Log server cancellation errors at info level by @JamesNK in grpc/grpc-dotnet#2527 * Update logging to use generated logs by @wabalubdub in grpc/grpc-dotnet#2531 * Bump serve-static from 1.14.2 to 1.16.2 in /examples/Spar/Server/ClientApp by @dependabot in grpc/grpc-dotnet#2536 * Update to Grpc.Tools 2.67.0-pre1 by @JamesNK in grpc/grpc-dotnet#2547 * Cleanup gRPC unit testing helpers in tester sample by @JamesNK in grpc/grpc-dotnet#2548 * Fix UpdateBalancingState not called when address attributes are modified by @JamesNK in grpc/grpc-dotnet#2553 * Update Grpc.Tools to 2.67.0 by @JamesNK in grpc/grpc-dotnet#2554 * Fix System.Text.Json vulnerability warning by @JamesNK in grpc/grpc-dotnet#2556 * Update package dependencies to 9.0 RC2 by @JamesNK in grpc/grpc-dotnet#2560 * Bump elliptic from 6.5.7 to 6.6.0 in /examples/Spar/Server/ClientApp by @dependabot in grpc/grpc-dotnet#2567 * Update to .NET 9 RTM by @JamesNK in grpc/grpc-dotnet#2571 ## New Contributors * @wabalubdub made their first contribution in grpc/grpc-dotnet#2531 **Full Changelog**: grpc/grpc-dotnet@v2.66.0...v2.67.0 ## 2.67.0-pre1 ## What's Changed * precompile condition clean by @Varorbc in grpc/grpc-dotnet#2528 * Log server cancellation errors at info level by @JamesNK in grpc/grpc-dotnet#2527 * Update logging to use generated logs by @wabalubdub in grpc/grpc-dotnet#2531 * Bump serve-static from 1.14.2 to 1.16.2 in /examples/Spar/Server/ClientApp by @dependabot in grpc/grpc-dotnet#2536 * Update to Grpc.Tools 2.67.0-pre1 by @JamesNK in grpc/grpc-dotnet#2547 * Cleanup gRPC unit testing helpers in tester sample by @JamesNK in grpc/grpc-dotnet#2548 * Fix UpdateBalancingState not called when address attributes are modified by @JamesNK in grpc/grpc-dotnet#2553 * Update Grpc.Tools to 2.67.0 by @JamesNK in grpc/grpc-dotnet#2554 * Bump version for 2.67 RC by @apolcyn in grpc/grpc-dotnet#2555 ## New Contributors * @wabalubdub made their first contribution in grpc/grpc-dotnet#2531 **Full Changelog**: grpc/grpc-dotnet@v2.66.0...v2.67.0-pre1 ## 2.66.0 ## What's Changed * Bump version on master to 2.66.0-dev by @stanley-cheung in grpc/grpc-dotnet#2491 * Fix failure to create GrpcChannel under Wine compatibility layer (including Steam Proton and Apple Game Porting Toolkit) by @mayuki in grpc/grpc-dotnet#2496 * Update .NET 9 SDK and resolve warnings by @sebastienros in grpc/grpc-dotnet#2502 * Bump braces from 3.0.2 to 3.0.3 in /testassets/InteropTestsGrpcWebWebsite/Tests by @dependabot in grpc/grpc-dotnet#2504 * Bump axios from 1.6.2 to 1.7.4 in /testassets/InteropTestsGrpcWebWebsite/Tests by @dependabot in grpc/grpc-dotnet#2505 * Update puppeteer by @JamesNK in grpc/grpc-dotnet#2507 * Remove internal_ci flag from interop test script by @JamesNK in grpc/grpc-dotnet#2509 * Fix Google auth interop test by @JamesNK in grpc/grpc-dotnet#2512 * [testing] improve sanity check in jwt_token_creds interop test by @apolcyn in grpc/grpc-dotnet#2513 * Add HTTP version configuration to GrpcChannelOptions by @JamesNK in grpc/grpc-dotnet#2514 * Bump grpc.tools version to 2.66 by @apolcyn in grpc/grpc-dotnet#2523 * Bump webpack from 5.76.0 to 5.94.0 in /examples/Browser/Server/wwwroot by @dependabot in grpc/grpc-dotnet#2522 * Bump elliptic from 6.5.4 to 6.5.7 in /examples/Spar/Server/ClientApp by @dependabot in grpc/grpc-dotnet#2525 * Bump micromatch from 4.0.7 to 4.0.8 in /testassets/InteropTestsGrpcWebWebsite/Tests by @dependabot in grpc/grpc-dotnet#2524 * Bump v2.66.x branch to 2.66.0.pre1 by @apolcyn in grpc/grpc-dotnet#2526 * Bump v2.66.x to v2.66.0 by @apolcyn in grpc/grpc-dotnet#2539 **Full Changelog**: grpc/grpc-dotnet@v2.65.0...v2.66.0 ## 2.66.0-pre1 ## What's Changed * Bump version on master to 2.66.0-dev by @stanley-cheung in grpc/grpc-dotnet#2491 * Fix failure to create GrpcChannel under Wine compatibility layer (including Steam Proton and Apple Game Porting Toolkit) by @mayuki in grpc/grpc-dotnet#2496 * Update .NET 9 SDK and resolve warnings by @sebastienros in grpc/grpc-dotnet#2502 * Bump braces from 3.0.2 to 3.0.3 in /testassets/InteropTestsGrpcWebWebsite/Tests by @dependabot in grpc/grpc-dotnet#2504 * Bump axios from 1.6.2 to 1.7.4 in /testassets/InteropTestsGrpcWebWebsite/Tests by @dependabot in grpc/grpc-dotnet#2505 * Update puppeteer by @JamesNK in grpc/grpc-dotnet#2507 * Remove internal_ci flag from interop test script by @JamesNK in grpc/grpc-dotnet#2509 * Fix Google auth interop test by @JamesNK in grpc/grpc-dotnet#2512 * [testing] improve sanity check in jwt_token_creds interop test by @apolcyn in grpc/grpc-dotnet#2513 * Add HTTP version configuration to GrpcChannelOptions by @JamesNK in grpc/grpc-dotnet#2514 * Bump grpc.tools version to 2.66 by @apolcyn in grpc/grpc-dotnet#2523 * Bump webpack from 5.76.0 to 5.94.0 in /examples/Browser/Server/wwwroot by @dependabot in grpc/grpc-dotnet#2522 * Bump elliptic from 6.5.4 to 6.5.7 in /examples/Spar/Server/ClientApp by @dependabot in grpc/grpc-dotnet#2525 * Bump micromatch from 4.0.7 to 4.0.8 in /testassets/InteropTestsGrpcWebWebsite/Tests by @dependabot in grpc/grpc-dotnet#2524 **Full Changelog**: grpc/grpc-dotnet@v2.65.0...v2.66.0-pre1 Commits viewable in [compare view](grpc/grpc-dotnet@v2.65.0...v2.71.0). </details> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Fixes #2612
A previous fix to stop deadlocks - #2601 - caused transient request errors. If a subcannel's not connected, and multiple gRPC calls queue at the same time, it's possible for a connection to be rapidly started and canceled. Although the state of the subchannel is quickly resolved, the queued calls could get an error result.
The fix is to check if a connection request is in progress later in the connection pipeline. If there is already a request then interrupt the delay if it is waiting and exit.