NeuroAgent

Fix Azure Functions YARP Stream Consumed Error

Resolve the 'Stream was already consumed' error in Azure Functions with YARP reverse proxy. Learn proper stream handling and middleware configuration.

#azure-functions #dotnet-development #http-proxy #middleware #stream-handling #troubleshooting #yarp-proxy

11/09/2025, 04:21 PM

Azure Functions CreateLessonPlan API fails with “Stream was already consumed” during YARP proxy forwarding

I’m encountering a specific issue with the CreateLessonPlan endpoint in my Azure Function App that fails with a “Stream was already consumed” error when being proxied from the host to the worker. All other APIs in the same project are working correctly.

Endpoint Details:

Method: POST
URL: http://localhost:7071/api/lessonplans

Request Body:

json

{
  "SubjectId": 1,
  "TeacherId": "628d0f85-6915-4df6-b769-c636e433abab",
  "School": "Trường THPT Kim Sơn B",
  "Class": "12A1",
  "LessonTitle": "Bài 1. Định nghĩa – Quy tắc tính đạo hàm",
  "Time": "1 tiết"
}

Observed Behavior:

Function host calls Functions.CreateLessonPlan
Request authentication succeeds but userId and role are null
Proxy retries 10 times before failing
Error shows that the request stream was already consumed

Error Logs:

System.Net.Http.HttpRequestException: An error occurred while sending the request.
 ---> System.InvalidOperationException: Stream was already consumed.
  at Yarp.ReverseProxy.Forwarder.StreamCopyHttpContent.SerializeToStreamAsync(...)

Expected Behavior:
The API should execute normally and return a generated lesson plan response, similar to other endpoints in the project.

Additional Context:

Warning messages about missing decimal precision (HasColumnType, HasPrecision, etc.) appear unrelated to the crash
The proxy error occurs immediately after authentication and before EF Core writes any data
This issue may be caused by reusing the request body stream or a custom middleware that reads the stream twice

How can I resolve the “Stream was already consumed” error specifically for the CreateLessonPlan endpoint in my Azure Functions with YARP reverse proxy setup?

NeuroAgent

The “Stream was already consumed” error in your Azure Functions CreateLessonPlan endpoint occurs when YARP reverse proxy attempts to forward the request, but the request body stream has already been read by middleware or authentication logic before proxying. This typically happens when custom middleware reads the request body for logging, validation, or authentication purposes without properly buffering or cloning the stream for downstream use. The solution involves implementing proper stream handling techniques in your middleware and ensuring request streams remain available for YARP’s proxying process.

Understanding the Stream Consumption Problem
Common Causes in Azure Functions with YARP
Diagnostic Steps to Identify the Root Cause
Solutions and Workarounds
Prevention Best Practices
Implementation Examples

Understanding the Stream Consumption Problem

The “Stream was already consumed” error specifically indicates that System.Net.Http.HttpRequestContent.SerializeToStreamAsync() is being called on a stream that has already been read or disposed. In the context of YARP reverse proxy working with Azure Functions, this typically occurs when:

Middleware reads the request body: Any middleware component that accesses context.Request.Body without proper stream handling will consume the stream, making it unavailable for YARP’s proxying process.
Authentication logic consumes stream: If your authentication middleware reads the request body to validate tokens or extract user information, it can leave the stream in a consumed state.
Azure Functions stream behavior: The isolated process model in Azure Functions has specific stream handling requirements that differ from traditional ASP.NET Core applications.

As Microsoft’s YARP documentation explains, YARP routes and transforms request URLs and headers but needs the request body stream to remain available for transparent proxying. When the stream is consumed before proxying, YARP cannot forward the request data to the downstream service.

Common Causes in Azure Functions with YARP

Authentication Middleware Stream Consumption

Your observation that “userId and role are null” suggests authentication logic is running but potentially consuming the request stream. Authentication middleware often needs to read request bodies for token validation, especially in custom authentication schemes.

Body-Reading Middleware for Logging or Validation

Middleware designed to log request bodies or perform validation commonly encounters this issue. According to Microsoft’s guidance, when telemetry initializers or middleware read request bodies, they can leave streams in an unusable state.

Form Data Consumption

As noted in YARP issue #1412, accessing context.Request.Form consumes the request body, and since the body is not buffered by default, YARP cannot proxy the request correctly afterward.

Azure Functions Isolated Process Stream Handling

The Azure Functions isolated process model has unique stream characteristics. As shown in GitHub issue #1636, accessing HttpRequestData.Body in middleware requires special handling to avoid stream consumption issues.

Diagnostic Steps to Identify the Root Cause

1. Enable Detailed Logging

Add comprehensive logging to track when and where the request body is being accessed:

csharp

app.Use(async (context, next) =>
{
    var originalBody = context.Request.Body;
    Log.Information("Request body stream accessed at: {StackTrace}", Environment.StackTrace);
    
    try
    {
        await next();
    }
    finally
    {
        // Reset stream position if possible
        if (originalBody.CanSeek)
        {
            originalBody.Position = 0;
        }
    }
});

2. Check Middleware Order

Review your middleware pipeline configuration. The authentication and any body-reading middleware should be positioned correctly relative to YARP’s proxy middleware.

3. Stream State Inspection

Add middleware to inspect the stream state before proxying:

csharp

app.Use(async (context, next) =>
{
    var canRead = context.Request.Body.CanRead;
    var canSeek = context.Request.Body.CanSeek;
    var position = context.Request.Body.Position;
    
    Log.Information("Stream state - CanRead: {CanRead}, CanSeek: {CanSeek}, Position: {Position}", 
        canRead, canSeek, position);
    
    await next();
});

4. Request Body Cloning Test

Temporarily implement request body cloning to isolate the issue:

csharp

app.Use(async (context, next) =>
{
    // Clone the request body
    var clonedBody = await CloneRequestBody(context.Request.Body);
    
    // Replace the original body with the clone
    context.Request.Body = clonedBody;
    
    await next();
});

async Task<Stream> CloneRequestBody(Stream originalBody)
{
    var memoryStream = new MemoryStream();
    await originalBody.CopyToAsync(memoryStream);
    memoryStream.Position = 0;
    return memoryStream;
}

Solutions and Workarounds

Solution 1: Implement Proper Stream Buffering

Create a middleware class that buffers the request body without consuming it:

csharp

public class StreamBufferingMiddleware
{
    private readonly RequestDelegate _next;

    public StreamBufferingMiddleware(RequestDelegate next)
    {
        _next = next;
    }

    public async Task Invoke(HttpContext context)
    {
        // Only buffer for POST/PUT requests with content
        if (context.Request.Method == "POST" || context.Request.Method == "PUT")
        {
            context.Request.EnableBuffering();
            
            // Store the original stream position
            var originalPosition = context.Request.Body.Position;
            
            try
            {
                await _next(context);
            }
            finally
            {
                // Restore the original position
                context.Request.Body.Position = originalPosition;
            }
        }
        else
        {
            await _next(context);
        }
    }
}

Solution 2: Use Context Items for Data Passing

Instead of reading the request body in middleware, store data in context items as recommended in Azure Functions GitHub issue #1636:

csharp

app.Use(async (context, next) =>
{
    if (context.Request.Method == "POST")
    {
        var requestBody = await new StreamReader(context.Request.Body).ReadToEndAsync();
        context.Items["RequestBody"] = requestBody;
        
        // Reset the stream for downstream consumption
        context.Request.Body = new MemoryStream(Encoding.UTF8.GetBytes(requestBody));
        context.Request.Body.Position = 0;
    }
    
    await next();
});

// In your Azure Function
public class CreateLessonPlan
{
    [Function("CreateLessonPlan")]
    public HttpResponseData Run([HttpTrigger(AuthorizationLevel.Function, "post")] HttpRequestData req)
    {
        var requestBody = req.HttpContext.Items["RequestBody"]?.ToString();
        // Use the stored request body
    }
}

Solution 3: YARP-Specific Stream Handling

Configure YARP to handle stream consumption more gracefully by using the ForwarderRequestConfig:

csharp

services.AddReverseProxy()
    .LoadFromConfig(Configuration.GetSection("ReverseProxy"))
    .AddTransforms(transformBuilderContext =>
    {
        transformBuilderContext.RequestTransforms.Add(transformContext =>
        {
            // Ensure the request body is available for proxying
            if (transformContext.ProxyRequest.Content != null)
            {
                transformContext.ProxyRequest.Content = new StreamCopyHttpContent(transformContext.HttpContext.Request.Body);
            }
            return ValueTask.CompletedTask;
        });
    });

Solution 4: Custom Stream Wrapper

Implement a stream wrapper that allows multiple reads:

csharp

public class ReusableStream : Stream
{
    private readonly Stream _originalStream;
    private readonly MemoryStream _buffer;
    private bool _buffered = false;

    public ReusableStream(Stream originalStream)
    {
        _originalStream = originalStream;
        _buffer = new MemoryStream();
    }

    public override async Task<int> ReadAsync(byte[] buffer, int offset, int count, CancellationToken cancellationToken)
    {
        if (!_buffered)
        {
            _buffered = true;
            await _originalStream.CopyToAsync(_buffer);
            _buffer.Position = 0;
        }

        return await _buffer.ReadAsync(buffer, offset, count, cancellationToken);
    }

    // Implement other Stream methods...
}

Prevention Best Practices

1. Middleware Design Guidelines

Avoid reading request bodies unless absolutely necessary
Use context items for data passing instead of stream reading
Implement proper stream disposal and position management

2. YARP Configuration Best Practices

Configure appropriate timeouts and retry policies
Use EnableBuffering() for requests that might need stream manipulation
Consider request size limits for memory buffering

3. Azure Functions Integration

Leverage Azure Functions’ built-in middleware capabilities
Use the isolated process model’s stream handling features
Consider using the new .NET 8 Azure Functions improvements for better stream management

4. Testing Strategy

Implement comprehensive stream state testing
Add unit tests for middleware components that interact with streams
Use integration tests to verify YARP proxy behavior with different request types

Implementation Examples

Complete Middleware Solution

Here’s a complete middleware implementation that handles stream consumption:

csharp

public class SafeStreamMiddleware
{
    private readonly RequestDelegate _next;

    public SafeStreamMiddleware(RequestDelegate next)
    {
        _next = next;
    }

    public async Task Invoke(HttpContext context)
    {
        // Only process POST/PUT requests with JSON content
        if (ShouldBufferRequest(context))
        {
            await BufferAndProcessRequest(context);
        }
        else
        {
            await _next(context);
        }
    }

    private bool ShouldBufferRequest(HttpContext context)
    {
        return (context.Request.Method == "POST" || context.Request.Method == "PUT") &&
               context.Request.ContentType?.Contains("application/json") == true;
    }

    private async Task BufferAndProcessRequest(HttpContext context)
    {
        // Enable buffering
        context.Request.EnableBuffering();
        
        // Read the body
        var requestBody = await new StreamReader(context.Request.Body).ReadToEndAsync();
        
        // Store in context items
        context.Items["RequestBody"] = requestBody;
        
        // Reset stream position
        context.Request.Body.Position = 0;
        
        try
        {
            await _next(context);
        }
        catch
        {
            // Clean up resources
            context.Request.Body.Dispose();
            throw;
        }
    }
}

// Registration in Program.cs
app.UseMiddleware<SafeStreamMiddleware>();

Azure Function with Stream Safety

Implement your Azure Function with stream safety considerations:

csharp

public class CreateLessonPlan
{
    [Function("CreateLessonPlan")]
    public HttpResponseData Run(
        [HttpTrigger(AuthorizationLevel.Function, "post")] HttpRequestData req)
    {
        try
        {
            // Get request body from context items (set by middleware)
            var requestBody = req.HttpContext.Items["RequestBody"]?.ToString();
            
            if (string.IsNullOrEmpty(requestBody))
            {
                // Fallback: read directly if middleware didn't run
                requestBody = new StreamReader(req.Body).ReadToEnd();
            }

            var lessonPlan = JsonSerializer.Deserialize<LessonPlan>(requestBody);
            
            // Process the lesson plan
            var result = ProcessLessonPlan(lessonPlan);
            
            var response = req.CreateResponse(HttpStatusCode.OK);
            response.Headers.Add("Content-Type", "application/json; charset=utf-8");
            response.WriteString(JsonSerializer.Serialize(result));
            
            return response;
        }
        catch (Exception ex)
        {
            var response = req.CreateResponse(HttpStatusCode.InternalServerError);
            response.WriteString($"Error processing lesson plan: {ex.Message}");
            return response;
        }
    }

    private LessonPlanResult ProcessLessonPlan(LessonPlan plan)
    {
        // Your business logic here
        return new LessonPlanResult { 
            Id = Guid.NewGuid(),
            Success = true,
            Message = "Lesson plan created successfully"
        };
    }
}

public class LessonPlan
{
    public int SubjectId { get; set; }
    public string TeacherId { get; set; }
    public string School { get; set; }
    public string Class { get; set; }
    public string LessonTitle { get; set; }
    public string Time { get; set; }
}

public class LessonPlanResult
{
    public Guid Id { get; set; }
    public bool Success { get; set; }
    public string Message { get; set; }
}

YARP Configuration with Stream Handling

Configure YARP to better handle stream consumption:

json

{
  "ReverseProxy": {
    "Routes": {
      "lessonplan-route": {
        "ClusterId": "lessonplan-cluster",
        "Match": {
          "Path": "/api/lessonplans",
          "Methods": ["POST"]
        },
        "Transforms": [
          { "RequestHeaderRemove": "Content-Length" },
          { "RequestHeaderRemove": "Transfer-Encoding" }
        ]
      }
    },
    "Clusters": {
      "lessonplan-cluster": {
        "Destinations": {
          "destination1": {
            "Address": "https://localhost:7071/"
          }
        }
      }
    }
  }
}

This comprehensive approach should resolve the “Stream was already consumed” error by implementing proper stream handling, using context items for data passing, and ensuring YARP has access to the request body stream when needed.

Sources

Conclusion

The “Stream was already consumed” error in your Azure Functions CreateLessonPlan endpoint with YARP reverse proxy is a common streaming issue that can be resolved through proper middleware design and stream handling. Key takeaways include:

Identify the root cause by adding diagnostic logging to track when and where request streams are being consumed.
Implement proper stream buffering using context.Request.EnableBuffering() and ensure stream positions are reset after reading.
Use context items for data passing instead of reading request bodies directly in middleware, as recommended by Azure Functions best practices.
Configure YARP appropriately by removing Content-Length and Transfer-Encoding headers when streams might be modified.
Test thoroughly with different request scenarios to ensure stream handling works correctly across all endpoints.

By following these guidelines, you can resolve the stream consumption issue while maintaining the functionality of your CreateLessonPlan API and ensuring all other endpoints continue to work properly. The solution balances proper stream management with the specific requirements of Azure Functions and YARP reverse proxy integration.

How to properly buffer request bodies in Azure Functions middleware without consuming the stream?What are the best practices for authentication middleware that doesn't interfere with YARP proxy forwarding?How to implement request body cloning in Azure Functions for multiple stream reads?Why does my Azure Functions isolated process model have different stream handling requirements?How to configure YARP reverse proxy to handle POST requests with large request bodies?What middleware order should I use in Azure Functions with YARP proxy integration?

Ask NeuroAgent

Fix Azure Functions YARP Stream Consumed Error

Contents

Understanding the Stream Consumption Problem

Common Causes in Azure Functions with YARP

Authentication Middleware Stream Consumption

Body-Reading Middleware for Logging or Validation

Form Data Consumption

Azure Functions Isolated Process Stream Handling

Diagnostic Steps to Identify the Root Cause

1. Enable Detailed Logging

2. Check Middleware Order

3. Stream State Inspection

4. Request Body Cloning Test

Solutions and Workarounds

Solution 1: Implement Proper Stream Buffering

Solution 2: Use Context Items for Data Passing

Solution 3: YARP-Specific Stream Handling

Solution 4: Custom Stream Wrapper

Prevention Best Practices

1. Middleware Design Guidelines

2. YARP Configuration Best Practices

3. Azure Functions Integration

4. Testing Strategy

Implementation Examples

Complete Middleware Solution

Azure Function with Stream Safety

YARP Configuration with Stream Handling

Sources

Conclusion