Skip to content

Conversation

stephentoub
Copy link
Member

@stephentoub stephentoub commented Jun 18, 2024

Rather than using a manual loop to copy data from a stream to a buffer, use the Stream.CopyTo{Async} method. Downside is there's an extra 32 byte allocation for the temporary destination passthrough Stream object. Upside is it allows the source Stream's to optimize via its CopyTo{Async} override, e.g. MemoryStream.CopyTo just makes a single Write call with its buffer. The base Stream.CopyTo also uses a larger rented buffer size than these APIs were using.

Related to #103539

Before:

Method Kind Mean Allocated
Hash MemoryStream 79.76 us 568 B
Hash FileStream 837.08 us 568 B

After:

Method Kind Mean Allocated
Hash MemoryStream 59.76 us 600 B
Hash FileStream 131.51 us 600 B
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using System.IO.Hashing;
using System.Security.Cryptography;

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);

[MemoryDiagnoser]
[HideColumns("Job", "Error", "StdDev", "Median", "RatioSD")]
public class Tests
{
    private Stream _stream;
    private byte[] _bytes = RandomNumberGenerator.GetBytes(1024 * 1024);

    [Params("MemoryStream", "FileStream")]
    public string Kind { get; set; }

    [GlobalSetup]
    public void Setup()
    {
        switch (Kind)
        {
            case "MemoryStream":
                _stream = new MemoryStream(_bytes);
                break;

            case "FileStream":
                string path = Path.GetRandomFileName();
                File.WriteAllBytes(path, _bytes);
                _stream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read, 0, FileOptions.DeleteOnClose);
                break;
        }
    }

    [GlobalCleanup]
    public void Cleanup() => _stream.Dispose();

    [Benchmark]
    public ulong Hash()
    {
        _stream.Position = 0;
        var hash = new XxHash3();
        hash.Append(_stream);
        return hash.GetCurrentHashAsUInt64();
    }
}

@stephentoub stephentoub merged commit f8e575c into dotnet:main Jun 19, 2024
@stephentoub stephentoub deleted the hashcopyto branch June 19, 2024 23:46
Copy link
Contributor

Tagging subscribers to this area: @dotnet/area-system-io-hashing, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

@github-actions github-actions bot locked and limited conversation to collaborators Jul 20, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants