-
Notifications
You must be signed in to change notification settings - Fork 4
Description
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 22.04 |
Queue | TigerUbuntu |
Baseline | ebe6f541ca2adea41f4b1dfc97774af371654bab |
Compare | bc97b0aa42d0d173a6c9d0542f78f6825a1efb49 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
16.94 μs | 18.66 μs | 1.10 | 0.25 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>*'
Payloads
System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromStream(Mode: Reflection)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.656853571674155 > 18.202524704994957.
IsChangePoint: Marked as a change because one of 9/17/2023 8:50:51 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -4.324377044504235 (T) = (0 -18469.91616103519) / Math.Sqrt((527919.8142794342 / (12)) + (30647.349895304094 / (39))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (39) - 2, .025) and -0.052127249228207516 = (17554.831104872417 - 18469.91616103519) / 17554.831104872417 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 22.04 |
Queue | TigerUbuntu |
Baseline | ebe6f541ca2adea41f4b1dfc97774af371654bab |
Compare | bc97b0aa42d0d173a6c9d0542f78f6825a1efb49 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Collections.Perf_Frozen<ReferenceType>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
4.76 μs | 5.19 μs | 1.09 | 0.24 | False | ||||
4.58 μs | 5.03 μs | 1.10 | 0.07 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Perf_Frozen<ReferenceType>*'
Payloads
System.Collections.Perf_Frozen<ReferenceType>.Contains_True(Count: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.186892643921764 > 5.175669409540316.
IsChangePoint: Marked as a change because one of 9/17/2023 8:50:51 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -4.088637734271883 (T) = (0 -5176.666493709929) / Math.Sqrt((46886.50084835413 / (12)) + (1780.1690340760954 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.05226924097765024 = (4919.526573731598 - 5176.666493709929) / 4919.526573731598 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.Perf_Frozen<ReferenceType>.TryGetValue_True(Count: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.026876110690179 > 4.952556829089142.
IsChangePoint: Marked as a change because one of 9/18/2023 8:11:22 PM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -10.73472297851471 (T) = (0 -5023.409321515483) / Math.Sqrt((20470.7300748719 / (17)) + (1200.388415661769 / (32))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (32) - 2, .025) and -0.08143199209453031 = (4645.145842029404 - 5023.409321515483) / 4645.145842029404 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 22.04 |
Queue | TigerUbuntu |
Baseline | ebe6f541ca2adea41f4b1dfc97774af371654bab |
Compare | bc97b0aa42d0d173a6c9d0542f78f6825a1efb49 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in MicroBenchmarks.Serializers.Json_FromStream<Location>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
784.99 ns | 845.67 ns | 1.08 | 0.31 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream<Location>*'
Payloads
MicroBenchmarks.Serializers.Json_FromStream<Location>.SystemTextJson_Reflection_
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 845.6734513343799 > 818.4636302021297.
IsChangePoint: Marked as a change because one of 9/17/2023 8:50:51 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -4.304511120496314 (T) = (0 -852.6285122170425) / Math.Sqrt((1843.5789816860672 / (12)) + (165.34600686567887 / (39))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (39) - 2, .025) and -0.06772930712208318 = (798.5437006643424 - 852.6285122170425) / 798.5437006643424 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 22.04 |
Queue | TigerUbuntu |
Baseline | ebe6f541ca2adea41f4b1dfc97774af371654bab |
Compare | bc97b0aa42d0d173a6c9d0542f78f6825a1efb49 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Collections.ContainsTrue<Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
2.09 μs | 2.37 μs | 1.13 | 0.26 | False | ||||
15.69 μs | 19.13 μs | 1.22 | 0.28 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue<Int32>*'
Payloads
System.Collections.ContainsTrue<Int32>.HashSet(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3659668355518995 > 2.2817381928808294.
IsChangePoint: Marked as a change because one of 9/17/2023 8:50:51 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -6.929433911947136 (T) = (0 -2347.6672841623613) / Math.Sqrt((7869.336366717247 / (12)) + (1583.7446129484958 / (37))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (37) - 2, .025) and -0.0846146055601579 = (2164.5174904775413 - 2347.6672841623613) / 2164.5174904775413 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.ContainsTrue<Int32>.Queue(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.130935475752803 > 16.481330004570612.
IsChangePoint: Marked as a change because one of 9/17/2023 8:50:51 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -4.959125335683476 (T) = (0 -19049.135134854474) / Math.Sqrt((3210599.9739791197 / (12)) + (12772.159422256766 / (39))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (39) - 2, .025) and -0.1557225550015537 = (16482.44645950417 - 19049.135134854474) / 16482.44645950417 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 22.04 |
Queue | TigerUbuntu |
Baseline | ebe6f541ca2adea41f4b1dfc97774af371654bab |
Compare | bc97b0aa42d0d173a6c9d0542f78f6825a1efb49 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in BilinearTest
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
6.90 μs | 7.94 μs | 1.15 | 0.28 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'BilinearTest*'
Payloads
BilinearTest.Interpol_Scalar
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.936244053270042 > 7.291304184155571.
IsChangePoint: Marked as a change because one of 9/17/2023 8:50:51 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -4.20683166415137 (T) = (0 -7946.9810330994615) / Math.Sqrt((286943.0302363747 / (12)) + (27137.35000684864 / (39))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (39) - 2, .025) and -0.09056046606537817 = (7287.0613600833085 - 7946.9810330994615) / 7287.0613600833085 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 22.04 |
Queue | TigerUbuntu |
Baseline | ebe6f541ca2adea41f4b1dfc97774af371654bab |
Compare | bc97b0aa42d0d173a6c9d0542f78f6825a1efb49 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Globalization.Tests.Perf_DateTimeCultureInfo
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
81.87 ns | 95.82 ns | 1.17 | 0.29 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_DateTimeCultureInfo*'
Payloads
System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: )
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 95.8167789631484 > 88.5902541657734.
IsChangePoint: Marked as a change because one of 9/17/2023 8:50:51 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -5.223915256549826 (T) = (0 -94.16348424465751) / Math.Sqrt((23.831384057565607 / (12)) + (2.5516376493680837 / (39))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (39) - 2, .025) and -0.08631622273954104 = (86.6814673973938 - 94.16348424465751) / 86.6814673973938 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 22.04 |
Queue | TigerUbuntu |
Baseline | ebe6f541ca2adea41f4b1dfc97774af371654bab |
Compare | bc97b0aa42d0d173a6c9d0542f78f6825a1efb49 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Buffers.Text.Tests.Utf8ParserTests
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
44.75 ns | 47.50 ns | 1.06 | 0.52 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'
Payloads
System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: -3.4028235E+38)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 47.49865213016325 > 46.338653178239625.
IsChangePoint: Marked as a change because one of 9/17/2023 8:50:51 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -4.104857220705202 (T) = (0 -48.31220227283206) / Math.Sqrt((7.313863579690579 / (12)) + (3.307704011337144 / (39))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (39) - 2, .025) and -0.07619118348334322 = (44.891839864789056 - 48.31220227283206) / 44.891839864789056 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 22.04 |
Queue | TigerUbuntu |
Baseline | ebe6f541ca2adea41f4b1dfc97774af371654bab |
Compare | bc97b0aa42d0d173a6c9d0542f78f6825a1efb49 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Text.Json.Document.Tests.Perf_ParseThenWrite
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
1.44 μs | 1.58 μs | 1.09 | 0.26 | False | ||||
8.61 μs | 9.81 μs | 1.14 | 0.25 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_ParseThenWrite*'
Payloads
System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json400B)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.5757550359550363 > 1.540350972163248.
IsChangePoint: Marked as a change because one of 9/17/2023 8:50:51 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -4.82641681725024 (T) = (0 -1591.3896294551266) / Math.Sqrt((3841.6141945519853 / (12)) + (306.1044429406684 / (39))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (39) - 2, .025) and -0.05811766599529018 = (1503.9817220689047 - 1591.3896294551266) / 1503.9817220689047 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json4KB)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.811418391341585 > 9.750588457870741.
IsChangePoint: Marked as a change because one of 9/18/2023 2:28:59 PM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -6.217492523161725 (T) = (0 -9803.369509762757) / Math.Sqrt((254887.63657617578 / (16)) + (4297.266517646095 / (33))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (33) - 2, .025) and -0.087399999174593 = (9015.421663788991 - 9803.369509762757) / 9015.421663788991 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 22.04 |
Queue | TigerUbuntu |
Baseline | ebe6f541ca2adea41f4b1dfc97774af371654bab |
Compare | bc97b0aa42d0d173a6c9d0542f78f6825a1efb49 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in Benchstone.MDBenchI.MDNDhrystone
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
307.12 ms | 348.56 ms | 1.13 | 0.27 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchI.MDNDhrystone*'
Payloads
Benchstone.MDBenchI.MDNDhrystone.Test
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 348.5621192 > 325.84715390499997.
IsChangePoint: Marked as a change because one of 9/17/2023 8:50:51 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -4.381253426598594 (T) = (0 -347292827.06498533) / Math.Sqrt((411049464781716.56 / (12)) + (4765821167308.256 / (39))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (39) - 2, .025) and -0.07987398421034687 = (321604957.74786323 - 347292827.06498533) / 321604957.74786323 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | ubuntu 22.04 |
Queue | TigerUbuntu |
Baseline | ebe6f541ca2adea41f4b1dfc97774af371654bab |
Compare | bc97b0aa42d0d173a6c9d0542f78f6825a1efb49 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Text.Tests.Perf_Encoding
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
65.09 ns | 72.32 ns | 1.11 | 0.30 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'
Payloads
System.Text.Tests.Perf_Encoding.GetBytes(size: 512, encName: "utf-8")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 72.31876932012298 > 70.45546285171473.
IsChangePoint: Marked as a change because one of 9/17/2023 8:50:51 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -3.443064917913446 (T) = (0 -72.7002111242899) / Math.Sqrt((16.156374631906683 / (12)) + (2.204552512454975 / (39))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (39) - 2, .025) and -0.059428278277145856 = (68.62211686714252 - 72.7002111242899) / 68.62211686714252 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository