Version 0.12.0 release

Version 0.12.0 release

Author: Arkatufus

README.md

@@ -48,20 +48,74 @@ var serializer = new Serializer(options);
 
 This is essential for frameworks like Akka.NET where we need to be able to resolve live Actor References in the deserializing system.
 
+## Whitelisting Types On Deserialization
+
+Sometimes we need to limit the types that are allowed to be deserialized for security reasons. For this reason, you can either pass a class instance that implements the `ITypeFilter` interface into the `SerializerOptions` or use the `TypeFilterBuilder` class to build a `TypeFilter` that Hyperion can use to filter out any possibly harmful injection attack during deserialization.
+
+using the `ITypeFilter` interface:
+
+```c#
+public sealed class TypeFilter : ITypeFilter
+{
+    public ImmutableHashSet<string> FilteredTypes { get; }
+
+    internal TypeFilter(IEnumerable<Type> types)
+    {
+        FilteredTypes = types.Select(t => t.GetShortAssemblyQualifiedName()).ToImmutableHashSet();
+    }
+    
+    public bool IsAllowed(string typeName)
+        => FilteredTypes.Any(t => t == typeName);
+}
+```
+
+using the `TypeFilterBuilder` convenience builder:
+
+```c#
+var typeFilter = TypeFilterBuilder.Create()
+    .Include<AllowedClassA>()
+    .Include<AllowedClassB>()
+    .Build();
+
+var options = SerializerOptions.Default
+    .WithTypeFilter(typeFilter);
+
+var serializer = new Serializer(options);
+```
+
+### Convert Whitelist To Blacklist
+
+To do blacklisting instead of whitelisting a list of types, you will need to do a slight modification to the TypeFilter class.
+
+```c#
+public sealed class TypeFilter : ITypeFilter
+{
+    public ImmutableHashSet<string> FilteredTypes { get; }
+
+    internal TypeFilter(IEnumerable<Type> types)
+    {
+        FilteredTypes = types.Select(t => t.GetShortAssemblyQualifiedName()).ToImmutableHashSet();
+    }
+    
+    public bool IsAllowed(string typeName)
+        => FilteredTypes.All(t => t != typeName);
+}
+```
+
 ## Version Tolerance
 
 Hyperion has been designed to work in multiple modes in terms of version tolerance vs. performance.
 
 1. Pre Register Types, when using "Pre registered types", Hyperion will only emit a type ID in the output stream.
 This results in the best performance, but is also fragile if different clients have different versions of the contract types.
 2. Non Versioned, this is largely the same as the above, but the serializer does not need to know about your types up front. it will embed the fully qualified typename
-in the outputstream. this results in a larger payload and some performance overhead.
+in the output stream. this results in a larger payload and some performance overhead.
 3. Versioned, in this mode, Hyperion will emit both type names and field information in the output stream.
 This allows systems to have slightly different versions of the contract types where some fields may have been added or removed.
 
 Hyperion has been designed as a wire format, point to point for soft realtime scenarios.
 If you need a format that is durable for persistence over time.
-e.g. EventSourcing or for message queues, then Protobuf or MS Bond is probably a better choise as those formats have been designed for true version tolerance.
+e.g. EventSourcing or for message queues, then Protobuf or MS Bond is probably a better choice as those formats have been designed for true version tolerance.
 
 ## Performance
 

RELEASE_NOTES.md

@@ -1,3 +1,21 @@
+### 0.12.0 January 12 2022 ####
+
+* Allow explicit control over which types can be deserialized [#281](https://github.com/akkadotnet/Hyperion/pull/281)
+
+We've expanded our deserialization safety check to block dangerous types from being deserialized; we recommend this method as a best practice to prevent [deserialization of untrusted data](https://cwe.mitre.org/data/definitions/502.html). You can now create a custom deserialize layer type filter programmatically: 
+
+```c#
+var typeFilter = TypeFilterBuilder.Create()
+    .Include<AllowedClassA>()
+    .Include<AllowedClassB>()
+    .Build();
+var options = SerializerOptions.Default
+    .WithTypeFilter(typeFilter);
+var serializer = new Serializer(options);
+```
+
+For complete documentation, please read the [readme on filtering types for secure deserialization.](https://github.com/akkadotnet/Hyperion#whitelisting-types-on-deserialization)
+
 ### 0.11.2 October 7 2021 ####
 * Fix exception thrown during deserialization when preserve object reference was turned on 
   and a surrogate instance was inserted into a collection multiple times. [#264](https://github.com/akkadotnet/Hyperion/pull/264)

src/Hyperion.API.Tests/CoreApiSpec.ApproveApi.approved.txt

@@ -41,6 +41,11 @@ namespace Hyperion
         public void TrackDeserializedType([Hyperion.Internal.NotNull] System.Type type) { }
         public void TrackDeserializedTypeWithVersion([Hyperion.Internal.NotNull] System.Type type, [Hyperion.Internal.NotNull] Hyperion.TypeVersionInfo versionInfo) { }
     }
+    public sealed class DisabledTypeFilter : Hyperion.ITypeFilter
+    {
+        public static readonly Hyperion.DisabledTypeFilter Instance;
+        public bool IsAllowed(string typeName) { }
+    }
     public delegate object FieldInfoReader(object obj);
     public delegate void FieldInfoWriter(object obj, object value);
     public delegate void FieldReader(System.IO.Stream stream, object obj, Hyperion.DeserializerSession session);
@@ -49,6 +54,10 @@ namespace Hyperion
     {
         void BuildSerializer(Hyperion.Serializer serializer, Hyperion.ValueSerializers.ObjectSerializer objectSerializer);
     }
+    public interface ITypeFilter
+    {
+        bool IsAllowed(string typeName);
+    }
     [System.AttributeUsage(System.AttributeTargets.Field | System.AttributeTargets.All, AllowMultiple=false, Inherited=true)]
     public sealed class IgnoreAttribute : System.Attribute
     {
@@ -90,18 +99,24 @@ namespace Hyperion
     public class SerializerOptions
     {
         public static readonly Hyperion.SerializerOptions Default;
-        [System.Obsolete]
+        [System.Obsolete("This constructor is deprecated and will be removed in the future, please use the " +
+            "one with the packageNameOverrides argument")]
         public SerializerOptions(bool versionTolerance = false, bool preserveObjectReferences = false, System.Collections.Generic.IEnumerable<Hyperion.Surrogate> surrogates = null, System.Collections.Generic.IEnumerable<Hyperion.SerializerFactories.ValueSerializerFactory> serializerFactories = null, System.Collections.Generic.IEnumerable<System.Type> knownTypes = null, bool ignoreISerializable = false) { }
-        [System.Obsolete]
+        [System.Obsolete("This constructor is deprecated and will be removed in the future, please use the " +
+            "one with the disallowUnsafeTypes argument")]
         public SerializerOptions(bool versionTolerance, bool preserveObjectReferences, System.Collections.Generic.IEnumerable<Hyperion.Surrogate> surrogates, System.Collections.Generic.IEnumerable<Hyperion.SerializerFactories.ValueSerializerFactory> serializerFactories, System.Collections.Generic.IEnumerable<System.Type> knownTypes, bool ignoreISerializable, System.Collections.Generic.IEnumerable<System.Func<string, string>> packageNameOverrides) { }
+        [System.Obsolete("This constructor is deprecated and will be removed in the future, please use the " +
+            "one with the typeFilter argument")]
         public SerializerOptions(bool versionTolerance, bool preserveObjectReferences, System.Collections.Generic.IEnumerable<Hyperion.Surrogate> surrogates, System.Collections.Generic.IEnumerable<Hyperion.SerializerFactories.ValueSerializerFactory> serializerFactories, System.Collections.Generic.IEnumerable<System.Type> knownTypes, bool ignoreISerializable, System.Collections.Generic.IEnumerable<System.Func<string, string>> packageNameOverrides, bool disallowUnsafeTypes) { }
+        public SerializerOptions(bool versionTolerance, bool preserveObjectReferences, System.Collections.Generic.IEnumerable<Hyperion.Surrogate> surrogates, System.Collections.Generic.IEnumerable<Hyperion.SerializerFactories.ValueSerializerFactory> serializerFactories, System.Collections.Generic.IEnumerable<System.Type> knownTypes, bool ignoreISerializable, System.Collections.Generic.IEnumerable<System.Func<string, string>> packageNameOverrides, bool disallowUnsafeTypes, Hyperion.ITypeFilter typeFilter) { }
         public Hyperion.SerializerOptions WithDisallowUnsafeType(bool disallowUnsafeType) { }
         public Hyperion.SerializerOptions WithIgnoreSerializable(bool ignoreISerializable) { }
         public Hyperion.SerializerOptions WithKnownTypes(System.Collections.Generic.IEnumerable<System.Type> knownTypes) { }
         public Hyperion.SerializerOptions WithPackageNameOverrides(System.Collections.Generic.IEnumerable<System.Func<string, string>> packageNameOverrides) { }
         public Hyperion.SerializerOptions WithPreserveObjectReferences(bool preserveObjectReferences) { }
         public Hyperion.SerializerOptions WithSerializerFactory(System.Collections.Generic.IEnumerable<Hyperion.SerializerFactories.ValueSerializerFactory> serializerFactories) { }
         public Hyperion.SerializerOptions WithSurrogates(System.Collections.Generic.IEnumerable<Hyperion.Surrogate> surrogates) { }
+        public Hyperion.SerializerOptions WithTypeFilter(Hyperion.ITypeFilter typeFilter) { }
         public Hyperion.SerializerOptions WithVersionTolerance(bool versionTolerance) { }
     }
     public class SerializerSession
@@ -130,6 +145,18 @@ namespace Hyperion
     {
         public Surrogate(System.Func<TSource, TSurrogate> toSurrogate, System.Func<TSurrogate, TSource> fromSurrogate) { }
     }
+    public sealed class TypeFilter : Hyperion.ITypeFilter
+    {
+        public System.Collections.Immutable.ImmutableHashSet<string> FilteredTypes { get; }
+        public bool IsAllowed(string typeName) { }
+    }
+    public class TypeFilterBuilder
+    {
+        public Hyperion.TypeFilter Build() { }
+        public Hyperion.TypeFilterBuilder Include(System.Type type) { }
+        public Hyperion.TypeFilterBuilder Include<T>() { }
+        public static Hyperion.TypeFilterBuilder Create() { }
+    }
     public class TypeVersionInfo
     {
         public TypeVersionInfo() { }
@@ -591,6 +618,10 @@ namespace Hyperion.Internal
         public Hyperion.Internal.ImplicitUseTargetFlags TargetFlags { get; }
         public Hyperion.Internal.ImplicitUseKindFlags UseKindFlags { get; }
     }
+    public class UserEvilDeserializationException : Hyperion.Internal.EvilDeserializationException
+    {
+        public UserEvilDeserializationException(string message, string typeString) { }
+    }
     [System.AttributeUsage(System.AttributeTargets.Property | System.AttributeTargets.Field | System.AttributeTargets.Parameter | System.AttributeTargets.All)]
     public sealed class ValueProviderAttribute : System.Attribute
     {

src/Hyperion.Benchmarks/Program.cs

@@ -8,8 +8,16 @@ class Program
     {
         static void Main(string[] args)
         {
-            var benchmark = BenchmarkSwitcher.FromAssembly(Assembly.GetExecutingAssembly());
-            benchmark.RunAll();
+            var benchmark = BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly);
+
+            if (args.Length == 0)
+            {
+                benchmark.RunAll();
+            }
+            else
+            {
+                benchmark.Run(args);
+            }
         }
     }
 }

src/Hyperion.Benchmarks/SerializeClassesBenchmark.cs

@@ -8,6 +8,7 @@
 #endregion
 
 using System;
+using System.Runtime.CompilerServices;
 using BenchmarkDotNet.Attributes;
 
 namespace Hyperion.Benchmarks
@@ -20,9 +21,25 @@ public class SerializeClassesBenchmark : HyperionBenchmark
         private LargeSealedClass sealedObject;
         private GenericClass<int, string, bool, DateTime, Guid> genericObject;
 
+        private Serializer _filteredSerializer;
+        
         protected override void Init()
         {
-            Serializer = new Serializer(new SerializerOptions(preserveObjectReferences:true));
+            var baseOptions = new SerializerOptions(preserveObjectReferences: true);
+            Serializer = new Serializer(baseOptions);
+
+            var filteredOptions = baseOptions
+                .WithTypeFilter(
+                    TypeFilterBuilder.Create()
+                        .Include<CyclicClassA>()
+                        .Include<CyclicClassB>()
+                        .Include<VirtualTestClass>()
+                        .Include<LargeSealedClass>()
+                        .Include<GenericClass<int, string, bool, DateTime, Guid>>()
+                        .Include<TestEnum>()
+                        .Build());
+            _filteredSerializer = new Serializer(filteredOptions);
+            
             var a = new CyclicClassA();
             var b = new CyclicClassB();
             a.B = b;
@@ -45,9 +62,23 @@ protected override void Init()
         #endregion
 
         [Benchmark] public void Cyclic_References() => SerializeAndDeserialize(cyclic);
+        [Benchmark] public void Filtered_Cyclic_References() => SerializeAndFilteredDeserialize(cyclic);
         [Benchmark] public void Virtual_Classes() => SerializeAndDeserialize(virtualObject);
+        [Benchmark] public void Filtered_Virtual_Classes() => SerializeAndFilteredDeserialize(virtualObject);
         [Benchmark] public void Large_Sealed_Classes() => SerializeAndDeserialize(sealedObject);
+        [Benchmark] public void Filtered_Large_Sealed_Classes() => SerializeAndFilteredDeserialize(sealedObject);
         [Benchmark] public void Generic_Classes() => SerializeAndDeserialize(genericObject);
+        [Benchmark] public void Filtered_Generic_Classes() => SerializeAndFilteredDeserialize(genericObject);
+
+        [MethodImpl(MethodImplOptions.AggressiveInlining)]
+        private void SerializeAndFilteredDeserialize<T>(T elem)
+        {
+            Serializer.Serialize(elem, Stream);
+            Stream.Position = 0;
+            
+            _filteredSerializer.Deserialize<T>(Stream);
+            Stream.Position = 0;
+        }
     }
 
     #region test data types

src/Hyperion.Benchmarks/SerializeStructsBenchmark.cs

@@ -7,6 +7,7 @@
 // -----------------------------------------------------------------------
 #endregion
 
+using System.Runtime.CompilerServices;
 using System.Runtime.InteropServices;
 using BenchmarkDotNet.Attributes;
 
@@ -15,12 +16,25 @@ namespace Hyperion.Benchmarks
     public class SerializeStructsBenchmark : HyperionBenchmark
     {
         #region init
+        private Serializer _filteredSerializer;
+        
         private StandardStruct standardValue;
         private BlittableStruct blittableValue;
         private TestEnum testEnum;
 
         protected override void Init()
         {
+            base.Init();
+            
+            var filteredOptions = SerializerOptions.Default
+                .WithTypeFilter(
+                    TypeFilterBuilder.Create()
+                        .Include<StandardStruct>()
+                        .Include<BlittableStruct>()
+                        .Include<TestEnum>()
+                        .Build());
+            _filteredSerializer = new Serializer(filteredOptions);
+            
             standardValue = new StandardStruct(1, "John", "Doe", isLoggedIn: false);
             blittableValue = new BlittableStruct(59, 92);
             testEnum = TestEnum.HatesAll;
@@ -29,8 +43,21 @@ protected override void Init()
         #endregion
 
         [Benchmark] public void Enums() => SerializeAndDeserialize(testEnum);
+        [Benchmark] public void Filtered_Enums() => SerializeAndFilteredDeserialize(testEnum);
         [Benchmark] public void Standard_Value_Types() => SerializeAndDeserialize(standardValue);
+        [Benchmark] public void Filtered_Standard_Value_Types() => SerializeAndFilteredDeserialize(standardValue);
         [Benchmark] public void Blittable_Value_Types() => SerializeAndDeserialize(blittableValue);
+        [Benchmark] public void Filtered_Blittable_Value_Types() => SerializeAndFilteredDeserialize(blittableValue);
+
+        [MethodImpl(MethodImplOptions.AggressiveInlining)]
+        private void SerializeAndFilteredDeserialize<T>(T elem)
+        {
+            Serializer.Serialize(elem, Stream);
+            Stream.Position = 0;
+            
+            _filteredSerializer.Deserialize<T>(Stream);
+            Stream.Position = 0;
+        }
     }
 
     #region test data types

src/Hyperion.Tests.FSharpData/Hyperion.Tests.FSharpData.fsproj

@@ -9,7 +9,7 @@
   </ItemGroup>
 
   <ItemGroup>
-    <PackageReference Update="FSharp.Core" Version="6.0.0" />
+    <PackageReference Update="FSharp.Core" Version="6.0.1" />
   </ItemGroup>
 
 </Project>

src/Hyperion.Tests/UnsafeDeserializationExclusionTests.cs

@@ -1,7 +1,9 @@
-using System.IO;
+using System;
+using System.IO;
 using Hyperion.Extensions;
 using Hyperion.Internal;
 using Xunit;
+using FluentAssertions;
 
 namespace Hyperion.Tests
 {
@@ -22,5 +24,64 @@ public void CantDeserializeANaughtyType()
                     serializer.Deserialize<DirectoryInfo>(stream));
             }
         }
+
+        internal class ClassA
+        { }
+        
+        internal class ClassB
+        { }
+        
+        internal class ClassC
+        { }
+        
+        [Fact]
+        public void TypeFilterShouldThrowOnNaughtyType()
+        {
+            var typeFilter = TypeFilterBuilder.Create()
+                .Include<ClassA>()
+                .Include<ClassB>()
+                .Build();
+
+            var options = SerializerOptions.Default
+                .WithTypeFilter(typeFilter);
+
+            var serializer = new Serializer(options);
+            
+            using (var stream = new MemoryStream())
+            {
+                serializer.Serialize(new ClassA(), stream);
+                stream.Position = 0;
+                Action act = () => serializer.Deserialize<ClassA>(stream);
+                act.Should().NotThrow();
+                
+                stream.Position = 0;
+                Action actObj = () => serializer.Deserialize<object>(stream);
+                actObj.Should().NotThrow();
+            }
+
+            using (var stream = new MemoryStream())
+            {
+                serializer.Serialize(new ClassB(), stream);
+                stream.Position = 0;
+                Action act = () => serializer.Deserialize<ClassB>(stream);
+                act.Should().NotThrow();
+                
+                stream.Position = 0;
+                Action actObj = () => serializer.Deserialize<object>(stream);
+                actObj.Should().NotThrow();
+            }
+            
+            using (var stream = new MemoryStream())
+            {
+                serializer.Serialize(new ClassC(), stream);
+                stream.Position = 0;
+                Action act = () => serializer.Deserialize<ClassC>(stream);
+                act.Should().Throw<UserEvilDeserializationException>();
+                
+                stream.Position = 0;
+                Action actObj = () => serializer.Deserialize<object>(stream);
+                actObj.Should().Throw<UserEvilDeserializationException>();
+            }
+        }
     }
 }