.NET Native, SilverlightSerializer, MakeGenericType and MissingMetadataException

Recently I have ported my app Map Mania to the  Universal Windows Platform, and at the moment of this writing it is being certified. (that, I least I hope). It’s the first UWP app I actually submit to the Store. I have been doing quite some IoT Core stuff and that does not require much submitting. But it had to happen at one time, if only to dogfood my WpWinNl port to UWP, and it was an educating experience.

I learned the hard way that stuff that does work with debug settings, does not necessarily work when you compile stuff ‘for real’. I ran into quite a serious issue with generics. As soon as I turned my app over to the .NET Native toolchain, it even crashed on startup. Mind you, most of the ‘business code’ in this app and large parts of the XAML were still the same as the original Windows Phone7 and 8 app, as was the WpWinNl base library – all ported to UWP, but largely unchanged.

The root cause of the problem turned out to be SilverlightSerializer. This is a serialization framework made by Mike Talbot that I have been using to store app state since 2011. It’s name quite dates it origins. Unfortunately the link to the original article is dead – actually the whole blog seems to have disappeared - but it’s code has been sitting in WpWinNl and it’s predecessors ever since that time. I have ported it to Windows Phone 8, Windows RT, and now to UWP.

Deep down in the core it uses the Type.MakeGenericType method to make a generic type with two type parameters to store types. So if I have ViewModel with a double property, it makes a GetSetGeneric<ViewModel, double>. This, now, does not go down well with .NET Native. Runtime, it will pop up the following error:

Exception thrown: 'System.Reflection.MissingMetadataException' in System.Private.CoreLib.dll

Additional information: Reflection_InsufficientMetadata_EdbNeeded: WpWinNl.Utilities.GetSetGeneric<BeautyOfMaps.Models.TileMapSource,System.Boolean>. For more information, visit http://go.microsoft.com/fwlink/?LinkId=623485

Now, in your app, in  the “Properties” folder, there is a Default.rd.xml file that allows you to instruct the compiler not to ‘optimize things away’, to make sure this code still works. The insidious part is that this code is in my libary, downloaded from NuGet, and since it uses generics there is no ‘'generic’ way for me to instruct the compiler in the libary’s rd.xml not to optimize away code in the app that is using it.

If I wanted to fix this particular problem, I have to add the following line to the app’s rd.xml:

<Type Name="WpWinNl.Utilities.GetSetGeneric{BeautyOfMaps.Models.ViewModel.MainViewModel,System.Boolean}" Dynamic="Required All"/>

And guess what – then you get the next error. Now it it’s complaining about missing

WpWinNl.Utilities.GetSetGeneric<LocalJoost.Maps.TileSources.Google,
LocalJoost.Maps.TileSources.GoogleType>

So it apparently needs a line like this for every class type and property type combination that goes past MakeGenericType. So while you already defined in code what these properties are, you have to declare (again) in XML which ones you want to keep. I have been in contact with the .NET Native team about the apparent lack of logic behind this, but is seems that according to their date very little people actually use this kind of reflection. While this is true without any doubt – Microsoft telemetry does not lie - I think the critical thinking error that was made here is that those people might be using libraries that do – like mine! - and then they are thoroughly in deep… er, manure.

Trying to add manually all class/property type combinations is where madness lies, my friends. I came to about the 6th type and then I was kind of done. I am a developer, so I am lazy by nature, and in stead of actually doing it manually I added a weird property to SilverlightSerializer called RdXmlEntries. It’s usage is as follows:

• Run the app in DEBUG (so not .NET Native)
• Serialize all objects you want to serialize
• Directly behind the last serialization, add this code:

foreach (var entry in SilverlightSerializer.RdXmlEntries)
{
  Debug.WriteLine(entry);
}

• Copy and paste the resulting data in your Default.rd.xml
• Verify your code now runs under .NET Native.

This code is now in WpWinNlBasic version 3.0.5-alpha on NuGet and available for use. You now know why I keep this in the unstable branch – as long as I haven’t had the chance to dogfood all the stuff I have ported and created earlier, I want to thread carefully.

When I showed this solution to the .NET Native team I got feedback that I only needed to provide lines for when the second parameter is a value type, not when it’s a reference type. I have validated that in one case, but I serialize quite a complicated object graph and had spent more than enough time on it already, I did not really feel like experimenting further.

Bottom line – be very, very careful with advanced reflection in UWP apps. Don’t make the same mistake I made – test early under .NET Native and follow guidance provided here to maintain your sanity.

Special thanks to Tom Verhoeff for providing assistance based upon his earlier experience. And sorry, no code sample this time as this is more general guidance that a specific coding issue ;)

Build for both-Reflection bits & pieces for Windows Phone 8 and Windows 8

This is a bit of an odd post – it’s the things I learned while porting SilverlightSerializer 2 from Windows Phone to Windows 8 Store apps code. For those who don’t know what that is – it’s a very fast serializer, created by the brilliant Mike Talbot that’s able to store complex objects graphs in a very compact binary format. I use it to store an app’s state by simply serializing the view model graph to storage. For my (now) only Windows Store app I made a half-*ssed attempt to convert the much simpler SilverlightSerializer  v1 last year – but that’s slower and much less capable – especially my port. I was very much in a hurry then, so I cut out everything I did not need.

Anyway – SilverlightSerializer is a big chunk of reflection code. To get it to work in Windows Store apps, I had to change quite a lot of things – but then again a lot less than I thought. Even more shocking was the fact it worked without any problems when I linked the adapted code back to my Windows Phone project. That’s the power of the common Windows Runtime – no #if WINDOWS_PHONE preprocessor statements here!

So anyway, here’s what I learned. I wrote it in a kind of boring old-new style, but I found very little on this subject on the web, so I hope this is useful for people digging into reflection on Windows Store apps and hitting the wall when trying to do it ‘the new way’.

Properties and Fields

Get an object’s properties:

• Old code: Type.GetProperties()
• New code: Type.GetRuntimeProperties()

Get a single property:

• Old code: Type.GetProperty("MyProperty")
• New code: Type.GetRuntimeProperty("MyProperty")

Get a property’s get method

• Old code: PropertyInfo.GetGetMethod();
• New code: PropertyInfo.GetMethod;

Get a property’s set method

• Old code: PropertyInfo.GetSetMethod();
• New code: PropertyInfo.SetMethod;

Get public non-static properties that can be get and set

• Old code:Type.GetProperties(BindingFlags.Instance | BindingFlags.Public).
                        Where(p => p.GetSetMethod() != null));
• New code:Type.GetRuntimeProperties().
                           Where(p=> p.GetMethod != null && p.SetMethod != null &&
                                       !p.SetMethod.IsStatic && !p.GetMethod.IsStatic &&
                                       p.GetMethod.IsPublic && p.SetMethod.IsPublic);

Get an object’s Fields:

• Old code: Type.GetFields()
• New code: Type.GetRuntimeFields()

Get a single Field:

• Old code: Type.GetField("MyProperty")
• New code: Type.GetRuntimeField("MyProperty")

You can already see two patterns here – in stead of GetXYZ you do GetRuntimeXYZ. This “Runtime” indicates all an objects has available on runtime – that is, both it’s own properties, methods etc. and those of it’s parents. The other pattern is that in the Windows Runtime there is a tendency to use properties in stead of zero-parameter get methods.

Attributes

Determine whether a property has a custom attribute or not

• Old code: PropertyInfo.GetCustomAttributes(typeof(DoNotSerialize), true).Any()
• New code: PropertyInfo.GetCustomAttribute<DoNotSerialize>(true) != null

Type info

Determine if a type is an enum

• Old code: Type.IsEnum
• New code: Type.GetTypeInfo().IsEnum

Determine if a type is a generic type

• Old code: Type.IsGenericType
• New code: Type.GetTypeInfo().IsGenericType

Determine a type’s generic arguments

• Old code: Type.GetGenericArguments()
• New code: Type.GetTypeInfo().GenericTypeArguments

Find a type’s default constructor

• Old code:Type.GetConstructor(new Type[] { });
• New code: Type.GetTypeInfo().DeclaredConstructors.
                          FirstOrDefault(p => !p.GetParameters().Any());

Determine if a type implements an interface

• Old code: Type.GetInterface("IEnumerable", true) != null;
• New code: GetTypeInfo().ImplementedInterfaces.FirstOrDefault(
                       p => string.Compare(p.Name, name, ignoreCase?
                       StringComparison.OrdinalIgnoreCase : StringComparison.Ordinal)==0);

I must admit being a bit lazy on this one – this was used quite a number of times in SilverlightSerializer, so I made the following extension method

using System;
using System.Linq;
using System.Reflection;

namespace WpWinNl.Utilities
{
  public static class TypeExtensions
  {
    public static Type GetInterface(this Type type, string name, bool ignoreCase)
    {
      return type.GetTypeInfo().ImplementedInterfaces.FirstOrDefault(
        p => string.Compare(p.Name, name, ignoreCase? 
            StringComparison.OrdinalIgnoreCase : StringComparison.Ordinal)==0) ;
    }
  }
}

and was done with it ;-). Beware – this is not something I recommend – making the new API compatible with the old. Rather, make it the other way around. But for porting existing code – especially code you did not write yourself – this can be a helpful way to save time.

Miscellaneous

Creating delegates

• Old code: Delegate.CreateDelegate(typeof(DelegateType), MethodInfo);
• New code: MethodInfo.CreateDelegate(typeof(DelegateType));

A MethodInfo is something you would get out of for instance PropertyInfo.GetMethod. I must admit to code in SilverlightSerializer is quite complex at this point, and I don’t understand the whole detail. But this seems to work, and I suggest people more interested in the details to have a look at the code. I will post the whole shebang soon.

Sorry, no sample this time ;-)

Reflection in WinRT: use Rx Extensions for dynamically adding event handlers

Suppose you want to add an event handler, but you don’t know up front which event. Admittedly, this is a bit of an edge case, but I have used it in a behavior that starts a storyboard upon an event (I will publish that later). The designer can specify the event using a string value in XAML, so I cannot simply use object.MyEvent += (handler). We need reflection here.

So suppose you have found the event in the object using the GetRuntimeXXX methods like I described earlier today, and want to dynamically add a handler to the event “MyEvent”:

var evt = MyObject.GetType().GetRuntimeEvent("MyEvent");
evt.AddEventHandler(MyObject, 
  new EventHandler((p, q) => {// your code here }));

This will compile, and even run - unless the object is, for instance, a Grid, and the event “Loaded”. That’s apparently a “WinRT event”, whatever that may be, and this will result in a runtime exception:

"Adding or removing event handlers dynamically is not supported on WinRT events."

Googling Binging this message leads to all kind of complicated solutions using the WindowsRuntimeMarshall class (without any samples, alas), but the solution turns out to be very simple: use Rx Extensions Beta for Windows RT. For the really lazy reader: click tools/Library Package Manager/Package Manager Console and enter “Install-Package Rx-Main –Pre” in the Package Manager Console.

Now add

using System.Reactive.Linq;

to the top of the code file and then simply use the following code:

Observable.FromEventPattern<RoutedEventArgs>(MyObject, "MyEvent")
  .Subscribe(se => { // your code here });

and you are done. The difference between ordinary events and “WinRT events” is apparently swallowed up in the bowels of Rx. This library is great to begin with, but if it saves you the trouble of digging in yet another still not very thoroughly documented API, it’s even better. As a bonus, by very nature of Rx the handler is automatically disposed of when your calling object goes out of scope, which is not necessarily the case in my first code. But that did not work anyway ;-)

Reflection in WinRT: DeclaredProperties vs GetRuntimeProperties caveat

I suppose this is part of some FM I did not R, but I would like to point it out in case some other sod wastes just as much time on it as me.

I embarked upon a journey to port SilverlightSerializer and some behaviors to WinRT. All stuff is heavily making use of reflection. Now reflection has been hugely overhauled in WinRT, so I knew I was in for an interesting ride. Recently I was convinced I managed to do it, but it turned out I did not.

There are apparently two ways to use reflection in RT. What I did use for reading all properties of an object, was:

var properties = MyObject.GetType().GetTypeInfo().DeclaredProperties;

But it turns out this returns only the properties declared in the class itself. What it does not return, are the properties of possible parent classes!

What I should have used, apparently, is:

var allproperties = MyObject.GetType().GetRuntimeProperties();

When I feed a Grid into “MyObject”,  “properties” hold 6 properties, while “allproperties” holds 51.

I don’t know why this distinction is made, and I understand even less how come I missed this the first time. On my first meeting as an MVP, Dennis Vroegop, after welcoming me to the club, already warned me that the MVP award did not automatically come with super powers. Boy, was he right.