How to send an HTTP POST request without buffering or setting Content-Length?

[AArnott]

I have an ASP.NET Core web api that needs to be tested with requests that lack a Content-Length header (or with an incorrect one). Up till this change which upgraded Microsoft.AspNetCore.Mvc.Testing from 3.1.9 to 5.0.1, I was able to do this by setting the ByteArrayContent.Headers.ContentLength property to null, but now it seems that the header is specified anyway.
Even if I use a non-seekable stream with StreamContent, the Content-Length header is still set after I call HttpClient.PostAsync. It seems the request stream is buffered on the client and the determined length is then used as the header.

How can I suppress this behavior, thus sending a streaming request entity without the Content-Length header from my test?

My test content here:
https://github.com/AArnott/IronPigeon/blob/b762524b7049f02188a145a0513495f752f3505d/test/IronPigeon.Relay.Tests/BlobControllerTests.cs#L119-L134

[Tratcher]

This is due to #24984.

tested with requests that lack a Content-Length header (or with an incorrect one)

All HTTP/1.1 requests must have a valid Content-Length header or a Transfer-Encoding: chunked header. That's enforced by the server. TestServer was updated to more closely mirror a valid client and generate the expected headers.

It seems the request stream is buffered on the client and the determined length is then used as the header.

That shouldn't be happening. Can you show how you created the non-seekable stream? The ContentLength property is checked, but that shouldn't trigger full buffering. For a streaming request you can avoid any of the new logic by setting HttpRequestMessage.Headers.TransferEncodingChunked = true.

[AArnott]

Thank you. Adding the HttpRequestMessage.Headers.TransferEncodingChunked header as you prescribed both makes sense and solved the problem. I can upgrade to the latest package now.

To answer your prior question about my previous attempt:

That shouldn't be happening. Can you show how you created the non-seekable stream?

Sure:

    private class NonSeekableStream : Stream
    {
        private readonly Stream inner;

        public NonSeekableStream(Stream inner)
        {
            this.inner = inner;
        }

        public override bool CanRead => true;

        public override bool CanSeek => false;

        public override bool CanWrite => false;

        public override long Length => throw new NotSupportedException();

        public override long Position { get => throw new NotSupportedException(); set => throw new NotSupportedException(); }

        public override void Flush() => throw new NotSupportedException();

        public override int Read(byte[] buffer, int offset, int count) => this.inner.Read(buffer, offset, count);

        public override long Seek(long offset, SeekOrigin origin) => throw new NotSupportedException();

        public override void SetLength(long value) => throw new NotSupportedException();

        public override void Write(byte[] buffer, int offset, int count) => throw new NotSupportedException();
    }

I then use the non-seekable stream like this:

-        using var content = new ByteArrayContent(serializedAddressBookEntry)
+        using var nonSeekableStream = new NonSeekableStream(new MemoryStream(serializedAddressBookEntry));
+        using var content = new StreamContent(nonSeekableStream)

Then when I run the test, before my controller's action method is invoked, a breakpoint on my non-seekable stream's Read method drains the stream, and later my controller is invoked with a Content-Length header.
At the time my NonSeekableStream.Read method is called (the second time), I can see in the Parallel Stacks window that the caller is the RedirectHandler.

But I don't know why because I believe my test is creating the request with the URI that eventually shows up at the controller, so no actual redirect should have been required.

[Tratcher]

I see. RedirectHandler is higher in the stack than I usually work with. You can turn that off by setting AllowAutoRedirect = false.

The redirect handler has to copy the body before it's consumed just in case a redirect happens, trying to do it after it sees a redirect is too late. I'm not sure why it implemented the copy this way, it should have used HttpContent.LoadIntoBufferAsync to make the copy, or not supported bodies that couldn't be replayed.