Overview of the new System.IO.Pipelines namespace

The new System.IO.Pipelines namespace contains a new high-performance I/O API introduced with .NET Core. It was originally built for the Kestrel web server to handle network traffic efficiently, and it's now available for general use when you need fast streaming reads and writes.

Unlike the classic Stream class, which reads and writes byte arrays, Pipelines use reusable buffers and minimize copying. You can process data as it arrives, without waiting for the full payload.

Here's a minimal example that reads from a stream using a PipeReader:

using System.IO.Pipelines;

PipeReader reader = PipeReader.Create(stream);
while (true)
{
    ReadResult result = await reader.ReadAsync();
    ReadOnlySequence<byte> buffer = result.Buffer;

    foreach (var segment in buffer)
    {
        Console.WriteLine($"Read {segment.Span.Length} bytes");
    }

    reader.AdvanceTo(buffer.End);
    if (result.IsCompleted) break;
}

await reader.CompleteAsync();

You can also create a PipeWriter for efficient output, or use Pipe directly for producer-consumer pipelines. It's ideal for servers, parsers, and any scenario that needs to process streams of data at high speed.