System.NullReferenceException: at Microsoft.AspNetCore.Http.DefaultHttpRequest.get_HasFormContentType

[Valdeminas]

Is there an existing issue for this?

• I have searched the existing issues

Describe the bug

Sometime I got the error System.NullReferenceException at Microsoft.AspNetCore.Http.DefaultHttpRequest.get_HasFormContentType() for several GET API.

Looks similar to this: #41574 But this has no conclusion.

Expected Behavior

No error

Steps To Reproduce

Intermitten issue, unclear cause

Exceptions (if any)

System.NullReferenceException:
   at Microsoft.AspNetCore.Http.DefaultHttpRequest.get_HasFormContentType (Microsoft.AspNetCore.Http, Version=6.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)
   at Microsoft.AspNetCore.Mvc.ModelBinding.FormValueProviderFactory.CreateValueProviderAsync (Microsoft.AspNetCore.Mvc.Core, Version=6.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)
   at Microsoft.AspNetCore.Mvc.ModelBinding.CompositeValueProvider+<CreateAsync>d__3.MoveNext (Microsoft.AspNetCore.Mvc.Core, Version=6.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.AspNetCore.Mvc.ModelBinding.CompositeValueProvider+<TryCreateAsync>d__4.MoveNext (Microsoft.AspNetCore.Mvc.Core, Version=6.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.AspNetCore.Mvc.Controllers.ControllerBinderDelegateProvider+<>c__DisplayClass0_0+<<CreateBinderDelegate>g__Bind|0>d.MoveNext (Microsoft.AspNetCore.Mvc.Core, Version=6.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker+<<InvokeInnerFilterAsync>g__Awaited|13_0>d.MoveNext (Microsoft.AspNetCore.Mvc.Core, Version=6.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker+<<InvokeFilterPipelineAsync>g__Awaited|20_0>d.MoveNext (Microsoft.AspNetCore.Mvc.Core, Version=6.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker+<<InvokeAsync>g__Logged|17_1>d.MoveNext (Microsoft.AspNetCore.Mvc.Core, Version=6.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker+<<InvokeAsync>g__Logged|17_1>d.MoveNext (Microsoft.AspNetCore.Mvc.Core, Version=6.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.AspNetCore.Routing.EndpointMiddleware+<<Invoke>g__AwaitRequestTask|6_0>d.MoveNext (Microsoft.AspNetCore.Routing, Version=6.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware+<Invoke>d__6.MoveNext (Microsoft.AspNetCore.Authorization.Policy, Version=6.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at MyProject.Api.Middleware.CurrencyMiddleware+<InvokeAsync>d__2.MoveNext (MyProject.Api, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null: /opt/atlassian/pipelines/agent/build/MyProject.Api/Middleware/CurrencyMiddleware.cs:51)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at MyProject.Api.Middleware.ClientPublishedDateMiddleware+<Invoke>d__2.MoveNext (MyProject.Api, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null: /opt/atlassian/pipelines/agent/build/MyProject.Api/Middleware/ClientPublishedDateMiddleware.cs:22)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware+<Invoke>d__5.MoveNext (Swashbuckle.AspNetCore.SwaggerUI, Version=6.2.3.0, Culture=neutral, PublicKeyToken=4232c99127b3c254)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware+<Invoke>d__4.MoveNext (Swashbuckle.AspNetCore.Swagger, Version=6.2.3.0, Culture=neutral, PublicKeyToken=62657d7474907593)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at MyProject.Api.Middleware.RateLimiting.MyProjectUserIdRateLimitMiddleware+<InvokeAsync>d__2.MoveNext (MyProject.Api, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null: /opt/atlassian/pipelines/agent/build/MyProject.Api/Middleware/RateLimiting/MyProjectUserIdRateLimitMiddleware.cs:42)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at MyProject.Api.Middleware.RateLimiting.MyProjectIpRateLimitMiddleware+<InvokeAsync>d__2.MoveNext (MyProject.Api, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null: /opt/atlassian/pipelines/agent/build/MyProject.Api/Middleware/RateLimiting/MyProjectIpRateLimitMiddleware.cs:43)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at MyProject.Api.Middleware.ExceptionHandling.ExceptionHandlingMiddleware+<InvokeAsync>d__2.MoveNext (MyProject.Api, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null: /opt/atlassian/pipelines/agent/build/MyProject.Api/Middleware/ExceptionHandling/ExceptionHandlingMiddleware.cs:31)

.NET Version

6.0.3

Anything else?

Host (useful for support):
Version: 6.0.3
Commit: c24d9a9c91

.NET SDKs installed:
No SDKs were found.

.NET runtimes installed:
Microsoft.AspNetCore.App 6.0.3 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 6.0.3 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

Azure App service on Linux with AppInsights

[Valdeminas]

I know from other issues, that a crash dump would be useful. If someone has guidance, how to setup one for a Linux App Service on Azure I would be grateful.

[davidfowl]

Are you using app insights?

[Valdeminas]

Yes. Do you need more info about how it's configured in our app?

[Valdeminas]

@davidfowl

We also get this error, which I think might be related.

System.NullReferenceException:
   at Microsoft.AspNetCore.Http.DefaultHttpRequest.get_Path (Microsoft.AspNetCore.Http, Version=6.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)
   at MyProject.Api.Program+<>c.<Main>b__1_6 (MyProject.Api, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null: /opt/atlassian/pipelines/agent/build/MyProject.Api/Program.cs:124)
   at Microsoft.AspNetCore.Builder.UseWhenExtensions+<>c__DisplayClass0_1.<UseWhen>b__1 (Microsoft.AspNetCore.Http.Abstractions, Version=6.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)
   at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware+<Invoke>d__6.MoveNext (Microsoft.AspNetCore.Authorization.Policy, Version=6.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at MyProject.Api.Middleware.CurrencyMiddleware+<InvokeAsync>d__2.MoveNext (MyProject.Api, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null: /opt/atlassian/pipelines/agent/build/MyProject.Api/Middleware/CurrencyMiddleware.cs:51)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at MyProject.Api.Middleware.ClientPublishedDateMiddleware+<Invoke>d__2.MoveNext (MyProject.Api, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null: /opt/atlassian/pipelines/agent/build/MyProject.Api/Middleware/ClientPublishedDateMiddleware.cs:22)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware+<Invoke>d__5.MoveNext (Swashbuckle.AspNetCore.SwaggerUI, Version=6.2.3.0, Culture=neutral, PublicKeyToken=4232c99127b3c254)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware+<Invoke>d__4.MoveNext (Swashbuckle.AspNetCore.Swagger, Version=6.2.3.0, Culture=neutral, PublicKeyToken=62657d7474907593)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at MyProject.Api.Middleware.RateLimiting.MyProjectUserIdRateLimitMiddleware+<InvokeAsync>d__2.MoveNext (MyProject.Api, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null: /opt/atlassian/pipelines/agent/build/MyProject.Api/Middleware/RateLimiting/MyProjectUserIdRateLimitMiddleware.cs:42)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at MyProject.Api.Middleware.RateLimiting.MyProjectIpRateLimitMiddleware+<InvokeAsync>d__2.MoveNext (MyProject.Api, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null: /opt/atlassian/pipelines/agent/build/MyProject.Api/Middleware/RateLimiting/MyProjectIpRateLimitMiddleware.cs:43)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at MyProject.Api.Middleware.ExceptionHandling.ExceptionHandlingMiddleware+<InvokeAsync>d__2.MoveNext (MyProject.Api, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null: /opt/atlassian/pipelines/agent/build/MyProject.Api/Middleware/ExceptionHandling/ExceptionHandlingMiddleware.cs:31)

this get s thrown at

POST /signalR/negotiate

In our codebase, there seems to be only one place, where Path is not handled directly in middleware, and gets passed to services, but instead of sending the HttpContext directly, the values get copied in the following object.

    public abstract class RateLimitRequestBaseDTO
    {
        public RateLimitRequestBaseDTO(HttpContext httpContext)
        {
            RequestMethod = httpContext.Request.Method.ToUpper();
            RequestPath = httpContext.Request.Path.Value.ToLower();
            IsAdmin = httpContext.User.Identity.IsAuthenticated && httpContext.User.IsAdmin();
        }
        public abstract string ValueToCheck { get; set; }
        public bool IsAdmin { get; set; }
        public string RequestMethod { get; set; }
        public string RequestPath { get; set; }
    }

Do you need any additional info from our end?

[davidfowl]

It's likely that there's other code running concurrently accessing the http context when this happens. The code properly using the http context is just happens to be running concurrently with the code that be using it.

I would audit anything that is using the IHttpContextAccessor.

[Valdeminas]

@davidfowl

Regarding IHttpContextAccessor - all access to it in our app is of the form:

public Constructor(
            IHttpContextAccessor httpContextAccessor)
        {
            _httpContextAccessor = httpContextAccessor;
        }

As I understand, this is the recommended way, so can it still be what's causing the issue?

There's one more thing I suspect.

We recently added httpContext.Items usage to our middleware.

Example pseudocode:

In Middleware.cs

public async Task InvokeAsync(HttpContext httpContext)
        {
            httpContext.Items["Key"] = "value";
            await _next(httpContext);
        }

And then in ControllerBase.cs

        protected object ItemValue
        {
            get
            {
                if (HttpContext.Items.TryGetValue("key", out object itemValue))
                {
                    return itemValue;
                }
                return null;
            }
        }

And then in Controller.cs

        [ProducesResponseType(200)]
        [HttpGet]
        public async Task<ActionResult> GetMethod()
        {
            var itemValue = ItemValue;
            return Ok();
        }

Could it be what's causing the issue?

[davidfowl]

Injecting the IHttpContextAccesor isn’t the problem. It’s when you access the HttpContext. If you do any Task.Run or any work dispatching that introduces concurrency from the request then it can cause problems

[Valdeminas]

@davidfowl

ok. thanks. gotcha.

I reviewed the code again.

All of our IHttpContextAccessor usage comes from clases implementing IAuthorizationHandler.

HttpContext is only accessed in HandleRequirementAsync and not propogated further.

For example:

public AuthorizationHandlerBase(
            IHttpContextAccessor httpContextAccessor)
        {
            _httpContextAccessor = httpContextAccessor;
        }

        protected override Task HandleRequirementAsync(AuthorizationHandlerContext context, T param)
        {
            var authcontext = _httpContextAccessor.HttpContext;
            string queryAccessToken = authcontext.Request.Query["Query"];
            if (queryAccessToken != null && "Value"== queryAccessToken)
            {
                context.Succeed(param);
            }
            else
            {
                context.Fail();
            }
            return Task.CompletedTask;
        }

Could this be the cause?

Also, should I understand, that our httpContext.Items usage mentioned earlier is correct and should not cause problems?

Thanks!

[davidfowl]

Yes this looks like an OK usage. Do you have any code that runs before this in middleware that might be logging telemetry? Are you using entity framework or SQL before this?

[adityamandaleeka]

Once this is understood, I think we can treat this as a duplicate of #42040

[Valdeminas]

While typing this out, I started to get the suspicion that most likely this will have something to do with SQL.

Still, I can't pinpoint the exact problem, so I'm leaving my full message and waiting for your comments.

Thanks again for your help @davidfowl

---

Regarding telemetry - we have 2 different scenarios. Not sure which one is relevant, so I'll mention both.

1:

At the beggining of our middleware pipeline, we have ApplicationInsightsMiddleware which we use to enrich our app insights data.

Here is some example code:

public async Task InvokeAsync(HttpContext context)
        {
            var requestTelemetry = context.Features.Get<RequestTelemetry>();
            requestTelemetry.Name = $"{context.Request.Method} {context.Request.Path}";
            requestTelemetry.Url = Uri.TryCreate(context.Request.GetDisplayUrl(), UriKind.RelativeOrAbsolute, out var requestUrl) ? requestUrl : null;

            requestTelemetry.Context.Session.Id = context.GetCorrelationId();
            requestTelemetry.Context.GlobalProperties["_Stuff"] = context.GetStuff();
            
            //Can include code for these functions if you think it's relevant
            await AddStuffRequestProperties(context, requestTelemetry);
            AddOtherStuffToRequestProperties(context, requestTelemetry);

            await _next(context);

            if (context.Response != null)
            {
                requestTelemetry.ResponseCode = context.Response.StatusCode.ToString();
                requestTelemetry.Success = context.Response.StatusCode >= 200 && context.Response.StatusCode <= 299;
            }
            else
            {
                requestTelemetry.Success = false;
            }
        }

2:

We have a few middleware classes which inject ITelemetryClientService into their InvokeAsync method.

Here is some example code for ITelemetryClientService:

public class TelemetryClientService : ITelemetryClientService
    {
        private readonly TelemetryClient _telemetryClient;

        public TelemetryClientService(TelemetryClient telemetryClient)
        {
            _telemetryClient = telemetryClient;

        }

        public void TrackEvent(string eventName)
        {
            _telemetryClient.TrackEvent(eventName);
        }
     }

And an example of what our middleware does:

public async Task InvokeAsync(HttpContext httpContext, ITelemetryClientService telemetry)
        {
            try
            {
                await _next(httpContext);
            }
            catch (Exception ex)
            {
                telemetry.TrackException(ex);

                //Can include code for these functions if you think it's relevant
                await HandleExceptionAsync(httpContext, ex);
            }
        }

Regarding SQL - we do have one Middleware which sends requests to the database.

Don't want to include actual code yet, bet essentially it looks something like:

public async Task InvokeAsync(HttpContext httpContext, IService service)
        {
            var ipAddress = httpContext.GetIpAddress();
            var stuff= await service.GetStuff(ipAddress);
            httpContext.Response.OnStarting(() =>
                    {
                        httpContext.Response.Cookies.Append("Cookie", stuff, StuffOptions());
                        return Task.CompletedTask;
                    });
            await _next(httpContext);
        }     

public IService (
            IRepository repository,
            ITelemetryClientService telemetryClient)
        {
            _repository = repository;
            _telemetryClientService = telemetryClient;
        }

 public async Task<Stuff> GetStuff(string ipaddress)
        {
             var stuff= await _repository.GetStuff(ipaddress);

            _telemetryClientService.TrackEvent(
                new Dictionary<string, string>() {
                    { "stuff", stuff}
                },
                "stuff");

            return new Stuff{ StuffValue = stuff};
        }

public Repository(
            Context context
        {
            _context = context;
        }

        public async Task<Stuff> GetStuff(string ipaddress)
        {
            return await _context .Stuff
                .Where(x => x.IpAddress== ipaddress)
                .FirstOrDefaultAsync();
        }

[davidfowl]

@Valdeminas Can you reproduce the issue locally?

[Valdeminas]

@davidfowl Not yet. We only see it in production and only sometimes. Therefore it does seem like it is some sort of a concurrency issue. The trouble is finding out, which part of the code is causing it.

Any ideas or tips for further investigation from the code provided above?