Customizing Trimming in .NET 5

This post has been republished via RSS; it originally appeared at: Microsoft Developer Blogs.

Customizing Trimming

In the last blog post, we talked about how trimming in .NET 5 has been expanded to be able to trim types and members that are detected as not being used in the application, and that detection uses static analysis, it doesn’t run the code for the app, so it doesn’t know which branches won’t be taken or what values variables will actually contain at runtime.

The trimming can either be too little or too much – too little when code is included that won’t be used, and too much when code is removed that will actually be used.

There are two ways that the trimming can be customized:

  • Using attributes to provide hints to the trimmer
  • Using XML files to provide explicit directions to the trimmer

Custom Attributes for Trimming

The primary mechanism for providing hints to the trimmer is to use attributes. The attributes tell the trimmer what extra dependencies should be included, and how to treat methods that it can’t readily determine if they are safe to be trimmed. The advantage of using attributes is that they live with the code, so as the code morphs over time, the annotations are less likely to go stale. It also has the benefit that the annotations will be available to anybody who consumes the library.

We want the library ecosystem to take this into consideration for their libraries so that trimming can be used by apps using 3rd party libraries from Nuget etc.

If you don’t own a library and don’t have access to the source, then these same attributes can be applied using external XML files. More details on that are further below.

Annotating Reflection

Reflection is the main cause of dynamic code, and commonly that will involve passing around a System.Type or string variable representing the type’s name, then members of the type are accessed via name. When the trimmer sees the calls to those reflection APIs it doesn’t know what is going to be called, so considers that code path as unsuitable for trimming and will throw a warning. The trimming problem isn’t with the code doing the reflection, it’s with the code being reflected upon – the trimmer needs to know which type and members will be accessed. The application will be trimmed, but it may be missing code that can cause fatal exceptions at runtime.

This ambiguity can be solved, using the DynamicallyAccessedMembers attribute, which annotates the Type variable and says which members of that Type will be used. The trimmer can then reason about the referenced type and know which members need to be retained.

The following examples do the same thing, based either on a Type variable or a string containing the type name.

  public IEnumerable foo(
      [DynamicallyAccessedMembers(DynamicallyAccessedMemberTypes.All)] Type t)
  {
      return Activator.CreateInstance(t) as IEnumerable;
  }

  public IEnumerable foo(
      [DynamicallyAccessedMembers(DynamicallyAccessedMemberTypes.All)] string typename)
  {
      return Activator.CreateInstance("mscorlib", typename).Unwrap() as IEnumerable;
  }

Specifying dependencies

One of the problems with trimming is to ensure that types that are dynamically used are not accidentally trimmed. This can be solved with attribution using the DynamicDependencyAttribute. Essentially this attribute says, “when you include me, also include my friends”. The attribute is applied to methods, and specifies which other types, methods or members should be included.

The following example is a little contrived as it could easily reference the types directly, but ensures that if foo is kept, then both the Stack and Queue types are kept along with all their members.

  [DynamicDependency(DynamicallyAccessedMemberTypes.All, "System.Collections.Stack", "mscorlib")]
  [DynamicDependency(DynamicallyAccessedMemberTypes.All, "System.Collections.Queue", "mscorlib")]
  public IEnumerable foo(bool lifo)
  {
      string typename = (lifo) ? "System.Collections.Stack" : "System.Collections.Queue";
      return (IEnumerable) Activator.CreateInstance("mscorlib", typename).Unwrap();
  }

Dynamic dependency can be used to express a dependency on either a type and subset of members, or at specific members.

Suppressing Warnings

There can be cases where the trimmer isn’t smart enough to know that the specific circumstances for how the code is used, and what it flags as problematic is actually correct. In that case, an explicit suppression can be added to the code to tell it to ignore the warning. This will probably mostly be used by library authors creating systems based on reflection, and care needs to be taken to ensure that the code will truly work correctly after trimming.

The UnconditionalSuppressMessage takes the specific message to be suppressed, and is used in a similar way to suppressing static analysis rules.

For example, from System.Type:

  [UnconditionalSuppressMessage("ReflectionAnalysis", "IL2085:UnrecognizedReflectionPattern",
              Justification = "Literal fields on enums can never be trimmed")]
  // This will return enumValues and enumNames sorted by the values.
  private void GetEnumData(out string[] enumNames, out Array enumValues) { … }

Forcing Warnings

The opposite of suppressing warnings is to mark code with RequiresUnreferencedCode attribute. This will mark the method that it is not compatible with trimming, and will present a warning whenever the method is used. It also means that warnings will not be generated for code in that method as the responsibility is now shifted to the caller of the method.

Using XML files to control trimming

Using xml files for controlling trimming can be done external to the content individual assemblies – this has a couple of benefits and drawbacks. The big advantage is that you can provide trimming directives for assemblies without requiring the changing the code of the assembly – which is necessary if you are using libraries written by somebody else – for example as part of the framework or from Nuget packages. The trimming directives can be tweaked for the specific application, but it means that they need to be created and managed for each application that you wish to trim.

The XML files provide a convenient escape clause. If you trim your application and discover problems at runtime because required code has been trimmed, using the XML files you can customize the trimming as part of the publish to ensure the required code is included.

There are 3 scenarios for XML files:

  • Preservation
  • External Attribution
  • Feature switches

Each scenario uses its own individual XML files and command argument to specify the file, but they all follow the same basic format.

The xml files follow the structure below:

  <linker>
    <assembly fullname="Assembly">
      <type fullname="Namespace.A">
        <field name="MyNumericField" />
        <property name="Property1" />
        <method name="Method3" />
        <event name="Event2" />
      </type>
    </assembly>
  </linker>

Preservation

The trimmer will automatically include all code that it thinks can be reached by the application. The preservation scenario for using XML files is to tell the trimmer to “preserve” code and not to remove it, even if it doesn’t think it is used. This is great for code that is discovered and invoked by reflection.

The trimmer can be passed an xml description file specifying which assemblies, types and members need to be retained, even if the trimmer doesn’t think the code is reached. An XML file (or files) need to be authored using the structure above, and then added to the project file using the following tag:

  <ItemGroup>
    <TrimmerRootDescriptor Include="TrimmerRoots.xml" />
  </ItemGroup>

When an assembly, type or member are listed in the xml, the default action is preservation, which means that regardless of whether the trimmer thinks they are used or not, they will be preserved in the output. Preservation is additive, it will tell the trimmer what extra code that it doesn’t think is needed should be kept, if it thinks a type or member is needed then it will include it, even if it would not be included based on the preservation tags.

The preservation tags are ambiguously inclusive – if you don’t provide the next level of detail – it will include all the children. If an assembly is listed without any types, then all the assembly’s types and members will be preserved.

The following all mean the contents of AssemblyA will be preserved.

  <linker>
    <assembly fullname="Assembly">
  </linker>
  <linker>
    <assembly fullname="AssemblyA" preserve="all" />
  </linker>
  <linker>
    <assembly fullname="AssemblyA, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null">
  </linker>

If types are specified under an assembly, then unless preserve=all is specified at the assembly level, only the listed types and members will be flagged for preservation. Similarly, if the type doesn’t have a preserve attribute and it doesn’t list any children then all of its members will be flagged for preservation.

The following will preserve all the members of One & Two, but not other types of AssemblyA (unless they are needed by other code).

  <linker>
    <assembly fullname="AssemblyA">
      <type fullname="AssemblyA.One" preserve="all" />
      <type fullname="AssemblyA.Two" /> 

As an alternative to preserve="all" on a type, preserve can be set to "fields" or "methods".

If individual fields, properties, methods, events or nested types are listed, then only those specified will be flagged for preservation. Granular control over specifying members by name or signature (in the case of overloads) and which accessors of a property should be included are possible.

When types are specified within an assembly, or members within a type, the peers that were not mentioned will be implicitly set to required="false", which means they will only be included if referenced from other code. If desired, required="false" can be explicitly set on a type or member if you want to enumerate them, but has the same effect as not including the attribute.

Applying Attributes via XML

The problem with using attributes to annotate code with trimming information is that you need to have the source code and build the library with those attributes. As a workaround an XML configuration file can be used to provide a way to apply those annotations external to the code.

An XML file can be authored using an <attribute> tag as a children to assembly, type, member or parameter tags.

  <linker>
    <assembly fullname="AssemblyA">
      <type fullname="AssemblyA.One" preserve="all" >
        <method signature="AssemblyA.IFoo LoadAddon(System.String)">
          <attribute fullname="System.Diagnostics.CodeAnalysis.RequiresUnreferencedCodeAttribute" assembly="System.Runtime">
            <argument>LoadAddon is not trimmable as the assembly and type for the addon are unknown during trimming</argument>
  …

This attributes file needs to be passed to the trimmer by embedding the substitutions file with the name "ILLink.Subsitutions.xml"

  <ItemGroup>
  	<EmbeddedResource Include="ILLink.Subsitutions.xml"/>
  <ItemGroup>

Or using the _ExtraTrimmerArgs property as part of publish:

  dotnet publish -r linux-x64 -p:PublishTrimmed=True -p:TrimMode=Link -p:PublishSingleFile=true -p:_ExtraTrimmerArgs="--link-attributes foo.xml"

Reducing the application size

Using member level trimming during publish will have the biggest impact on the on-disk size of the application. This removes all the code from your application and references that are determined not to be reachable or called by the app.

As discussed in the previous trimming post, member level trimming is not the default, and presents a risk of overly aggressively trimming if dynamic code patterns are not correctly recognized.

Beyond member level trimming, there are two other factors that will impact the published size of the app:

  • Ready To Run
  • Feature Switches

Ready To Run

When you trim with <TrimMode>Link</Trimmode> and without <PublishReadyToRun>True</PublishReadyToRun>, it will remove the Ready To Run binary code from the assemblies. This will further reduce the size of the application, but at the cost of startup time. As the application is run, each method will need to be JITed the first time it is invoked. Subsequent calls within the process lifetime will faster as the method will already be JITted.

As an application author you will need to make a judgement call as to what is higher priority – application size or startup time. The size reduction seen with Ready To Run will depend on the amount of library code that your application uses, and can be dwarfed by the baseline size of including the runtime itself which cannot be dramatically changed.

Framework feature switches

One of the differences between the Mono and .NET Core frameworks is that Mono was designed for mobile applications where package size is a concern; .NET framework is fuller featured and so larger. In the move to consolidate the two we needed a mechanism to retain the size benefits of Mono.

One mechanism we have added that will allow us to conditionally remove code from applications is feature switches. Using this mechanism, an SDK (like Xamarin) or the developer can decide if a feature that would normally be available in the libraries can be removed, both at runtime and during linking. For example InvariantGlobalization which removes globalization support which may not be needed if the app is not presenting UI or needing to process globalized input (such as parsing different date formats).

The feature switches supported in .NET5 include:

MSBuild Property Name

Description

Trimmed when

DebuggerSupport

Trims code that enables better debugging experience - for example type visualizers

False

EnableUnsafeUTF7Encoding

Insecure UTF-7 encoding

False

EnableUnsafeBinaryFormatterSerialization

BinaryFormatter serialization

False

EventSourceSupport

EventSource related code and logic

False

InvariantGlobalization

All globalization specific code and data

False

UseSystemResourceKeys

localized resources for system assemblies, such as error messages

True

HttpActivityPropagationSupport

Distributed tracing support in System.Net.Http

False

The SDK for Blazor and other app targets configure default values for their environments. The feature switches can be toggled either from the command line using

  dotnet publish -p:EnableUnsafeUTF7Encoding=false

Or in the project file

  <PropertyGroup>
    <EnableUnsafeUTF7Encoding>false</EnableUnsafeUTF7Encoding>
  </PropertyGroup>

Feature switch definition

The design for feature switches is that the developer should toggle the feature on or off using MSBuild. The behavior for switch should be consistent regardless of whether the app is being trimmed or not. The functionality for a feature should be guarded by conditional logic that keys off the same MSBuild property. We do not want to end up in the situation where the behavior that was tested during development has a different behavior at runtime after the app has been published.

Feature switches work using a combination of pieces:

  • A property in code with a getter that indicates if the feature is available or not
  • A substitution for the linker to replace the property with a static value
  • MSBuild flags that expose a property for the feature wired together the code property and substitution.

To toggle the feature, a property is defined in code to state whether the feature is enabled/disabled. This property should be queried on each code path that would execute the feature, thereby guarding the functionality. The implementation of the property should read the value from runtimeconfig.json using AppContext.TryGetSwitch.

  public static bool EnableUnsafeUTF7Encoding
  {
    get => AppContext.TryGetSwitch("System.Text.Encoding.EnableUnsafeUTF7Encoding", ref s_enableUnsafeUTF7Encoding);
  }

This property is then queried to guard the functionality for UTF7 encoding.

MSBuild will set the value in the runtimeconfig using

  <RuntimeHostConfigurationOption Include="System.Text.Encoding.EnableUnsafeUTF7Encoding"
    Condition="'$(EnableUnsafeUTF7Encoding)' != ''"
    Value="$(EnableUnsafeUTF7Encoding)"
    Trim="true" /> 

Which creates the EnableUnsafeUTF7Encoding msbuild property, associates it with the System.Text.Encoding.EnableUnsafeUTF7Encoding key used by TryGetSwitch, and also toggles a feature switch for the trimmer using the same name.

To enable the trimmer to remove the feature, the substitutions capability is used which tells the trimmer to replace the code for methods with a stub. The stub can:

  • Do nothing
  • Throw an exception
  • or Return a constant value

The option to replace the property getter with a static value is used, returning a static value disabling the feature. When the trimmer then examines the conditional logic querying the property, it will see that the property returns false, determine that the code guarded by that property is not unreachable, so it will be trimmed.

The substitution XML ties the feature flag (System.Text.Encoding.EnableUnsafeUTF7Encoding) to the property getter, and replaces it with a static value of false.

  <linker>
    <assembly fullname="System.Private.CoreLib">
      <type fullname="System.LocalAppContextSwitches">
        <method signature="System.Boolean get_EnableUnsafeUTF7Encoding()" body="stub" value="false" feature="System.Text.Encoding.EnableUnsafeUTF7Encoding" featurevalue="false" />
      </type>
    </assembly>
  </linker>

Using Feature switches in your libraries

If your library has functionality that is large, but not always needed, then you could consider adding your own feature switches that disable functionality.

REMEMBER: these articles are REPUBLISHED. Your best bet to get a reply is to follow the link at the top of the post to the ORIGINAL post! BUT you're more than welcome to start discussions here:

This site uses Akismet to reduce spam. Learn how your comment data is processed.