Windows 10 maps part 7 - using external maps with TMS/WMS

Intro
In this final episode of the planned series, posted on "Back to the Future" day, I will show you how to use external map sources and how to use them with the Windows 10 maps control. Specifically I will show you how to use Tile Map Services (TMS) - way of distributing digital maps designed by the Open Source Geospatial Foundation and popularized by Google Maps - and Web Map Services (WMS), and older, more complex protocol designed by the Open Geospatial Consortium in 1999.
I am using TMS in a loosely defined way here - I am defining it as a REST-based system that retrieves pre-rendered tiles of maps using fixed zoom levels based on zoom level and location (try to say that without stuttering). A TMS basically is a take-it-or-leave-it approach - you get a map rendered as the maker saw fit, whereas WMS offers you the possibility to select layers and determine the order in which those layers are retrieved - and sometimes they even support custom styling. In addition, it allows for arbitrary image sizes, whereas TMS typically services 256x256 images. TMS typically comes from a file system containing a very large number of small files, whereas a WMS is typically served from a spatial database. Consequently, WMS, while being more versatile, usually is a lot slower.

So what is this all about?
The are a very large number of servers out there that offer you maps that you can use - OpenStreetmaps, Google, NOAA, but also our own Dutch Rijkswaterstaat - a government agency that maintains the Dutch main road and water transport infrastructure. You can search for public endpoints of those servers, and when you implement a little code to translate the requests of you Windows 10 map control into the right URLs, maps will show up.
The sample code will show you how to use the following data:

• Google Maps
• OpenStreetMaps (a non-profit organization that has insanely detailed maps) - I use it often when I go on hiking holidays in Germany or the like, as even the most obscure hiking trails are displayed in more detail than any tourist map you can buy.
• Dutch Rijkswaterstaat maps
• NOAA weather and cloud maps

Be sure to read up on the terms of services of the map providers. Not every one of those providers have unlimited bandwidth or CPU power to serve up the needs of your app, or they may require you to pay for that. OpenStreetMaps is a non-profit organization that does not like you to let their servers burn. Some, like Google, require you to use their API in stead of directly accessing the URLs where their data reside. The sample code used in this project violates this TOS of Google, and is intended as an educational sample only. I strongly advise you not to use this in production environments - and to be honest, the HERE maps data is so good in most of the cases there is little need for doing that.

Viewing the demos
Start the app, click "Initial location". You will see this.

Now select "None" for Map style (the map will turn black), then select "OpenStreetMap" for Tile Map. The map will turn into as displayed below, and already shows some blue dotted hiking trails - even through the very neck of the woods where I live.

If you select Google Hybrid you will get the Google Satellite imagery with street labels on top of it. Once again, this is illegal, but it proves my point:

And why this might be useful... if you zoom all the way in and put the Google and Here imagery next to each other you will see that Google imagery in some places still has some more detail, although it's very outdated - my neighbor's house extension is not visible yet (it is on Here Maps) and I am now driving the 2nd car after the car that is still in front of my house - putting this back to the early days of 2009 at the very latest.

If you zoom out a little again, and select "RWS NWB" you will see the national road grid map from the Dutch Rijkswaterstaat, a simple line map for secondary roads, the major highway depicted as double red lines, with black dots showing the location of hectometer signs - the little signs that show you where you are on the roads, and that can be used to specify your location when your cars breaks down and you need to call for help. Amongst other things :)

Set style back to "Roads", and zoom out more or less the USA, the select NOAA Radar. This shows the rain radar of the USA, as observed by the NOAA


Looks like my friend Richard Hay is in for some rain over in Jacksonville. Or maybe it is just past him ;). Anyhow, if you select "Visible Img" you will get some real-time (or as near as real time as possible) weather satellite imagery in the visible light.

Now this is what I call a cloud service  ;)
Now believe it or not, but all those beautiful maps are created by these simple methods way down in MainPage.Xaml.cs

private void TileComboChanged(object sender, SelectionChangedEventArgs e)
{
  MyMap.TileSources.Clear();
  var tileSource = TileCombo.SelectedItem as ITileSource;
  if (tileSource != null)
  {
    MyMap.TileSources.Add(new MapTileSource(tileSource.TileSource));
  }
}

private void InitTileComboBox()
{
  TileCombo.Items.Add("None");
  TileCombo.Items.Add(new OpenStreetMapSource());
  TileCombo.Items.Add(new GoogleMapSource("y", "Google Hybrid"));
  TileCombo.Items.Add(new WmsTileSource("NOAA Radar",
   "http://nowcoast.noaa.gov/arcgis/services/nowcoast/radar_meteo_imagery_nexrad_time/MapServer/WmsServer?", 
    new[] { "1" }, "1.3.0", "CRS"));
  TileCombo.Items.Add(new WmsTileSource("NOAA Visible Img",
   "http://nowcoast.noaa.gov/arcgis/services/nowcoast/sat_meteo_imagery_goes_time/MapServer/WMSServer?", 
   new[] { "9" }, "1.3.0", "CRS"));
  TileCombo.Items.Add(new WmsTileSource("RWS NWB", 
    "http://geodata.nationaalgeoregister.nl/nwbwegen/ows?service=WMS", 
      new[] { "wegvakken", "hectopunten" }));
  TileCombo.Items.Add(new WmsTileSource("RWS NWB 4326", 
    "http://geodata.nationaalgeoregister.nl/nwbwegen/ows?service=WMS", 
    new[] { "wegvakken", "hectopunten"},"1.3.0", "CRS", 4326));

  TileCombo.SelectedIndex = 0;
}

And, of course, 'the supporting act' of some classes I wrote myself.

 
Tile sources
Welcome to the wonderful world of tile maps. Our friends of the Maps develop team have created a small class hierarchy to serve up tile maps to a Windows app map. Add any child of MapTileSource to your map's TileSource collection and you have created a new map. It's as simple as that. There are three kinds of map tiles sources, as depicted in the diagram below:

• HttpMapTileDataSource is meant to serve up map tiles via an URI to the web and download them on the fly
• LocalMapTileDataSource is meant to serve up map tiles via an URI to titles that are downloaded to local storage
• CustomMapTileDataSource is meant to cover all other cases - it does not ask for a remote or a local URI to a map tile, but asks you to return a 256x256 bitmap and it's up to you to determine where it comes from or how it's created.

My samples use HttpMapTileDataSource only, but of course the other classes are very interesting too, specifically LocalMapTileDataSource, as it opens the possibility to download and use tiles from any map to your device and use it from there, thereby creating your own Here-Maps-like offline experience.
HttpMapTileDataSource has a UriRequested event. You will need to attach a listener to it with this signature:

void MapUriRequested(
     HttpMapTileDataSource sender, 
     MapTileUriRequestedEventArgs args);

MapTileUriRequestedEventArgs has a Request property with four parameters:

• Request
• X
• Y
• Zoomlevel

The last three are input, based upon which the event listener must calculate a Uri, which needs to be put in Request - the output property. How this works in practice can be seen in for instance the OpenStreetMapSource:

private readonly static string[] TilePathPrefixes = { "a", "b", "c" };

protected override void MapUriRequested(HttpMapTileDataSource sender, MapTileUriRequestedEventArgs args)
{
  var deferral = args.Request.GetDeferral();
  // TilePathPrefixes - load balancing + caching
  args.Request.Uri = new Uri(
    $"http://{TilePathPrefixes[args.Y % 3]}.tile.openstreetmap.org/{args.ZoomLevel}/{args.X}/{args.Y}.png");
  deferral.Complete();
}

Where do you get this wisdom? You can be a GIS expert, or just look for these kinds of tricks on the internet, and mostly they can be found on the tile provider's site itself. Sufficient to say OpenStreetMaps has 3 servers, and then the X, Y and Zoomlevel are simply in the path and the filename.
The MapTileDataSource and it's children sport a weird oddity, though. The most logical approach would be to subclass HttpMapTileDataSource, right? This is possible, as they are not sealed. All is well, up until the moment you actually try to use such a child class in real life by adding it to the map TileSources collection and have the map display itself. Then you are greeted by
An exception of type 'System.Runtime.InteropServices.InvalidComObjectException' occurred in Manipulation_Drawing.exe but was not handled in user code
WinRT information: The text associated with this error code could not be found.
And good luck to you. Fortunately, where's a need, there's a workaround. I made a class containing a HttpMapTileDataSource, implementing an interface exposing that HttpMapTileDataSource. Meet BaseHttpTIleSource and it's friends.

ITileSource is pretty simple, as I alluded to:

interface ITileSource
{
  MapTileDataSource TileSource { get; }
}

It basically allows you to add YourTileSource.TileSource to the map's TileSource's collection without having aforesaid crash. BaseHttpTileSource is pretty simple, too.

public abstract class BaseHttpTileSource : ITileSource
{
  protected BaseHttpTileSource()
  {
    var t = new HttpMapTileDataSource();
    t.UriRequested += MapUriRequested;
    TileSource = t;
  }

  public MapTileDataSource TileSource { get; private set; }

  protected abstract void MapUriRequested(HttpMapTileDataSource sender, 
    MapTileUriRequestedEventArgs args);
}

It takes away most of the plumbing of using a HttpMapTileDataSource and the only thing you need to do is subclass this and override the MapUriRequested method. As shown for OpenStreetMaps, and there is also such an override for Google:

protected override void MapUriRequested(HttpMapTileDataSource sender, MapTileUriRequestedEventArgs args)
{
  var deferral = args.Request.GetDeferral();
  args.Request.Uri = 
    new Uri($"http://mt{(args.X%2)+(2*(args.Y%2))}.google.com/vt/lyrs={mapPrefix}&z={args.ZoomLevel}&x={args.X}&y={args.Y}");
  deferral.Complete();
}

Once again, this violates Google's TOS, but the trick is very old and I wrote first about it almost exactly five years ago. I still wait for Google's lawyers to turn up on my doorstep, at which point I will gladly discuss the lack of a decent API for Windows apps which enables us to use the data in a legitimate way.

WMS maps
Web Map Services is a prime example of the age-old wisdom stating a camel is a horse designed by a committee - the committee in this case being the Open Geo Consortium. OGC is - or at least was in that time - a group op GIS professionals of academic origin and they have clearly tried to a design a protocol that catered for a wide variety of analytic needs - of course using XML, which was the hot rage in the days it was designed. What they did not do was taking into account banal things like consistency, ease of use, performance, and other things that are valued by mere mortals who just want to have a bloody map in their web page or app. And if you think this is bad, try to read the specifications for vector data (WFS or Web Feature Service), a format so convoluted, bloated, with so much versions, inconsistencies and complexities that it almost requires a PhD to get a basic understanding of how it works - let alone use it. WMS servers used to be slow, bandwidth and processing power gorging monsters. And this in the early 2000s, the year where most web access was served over 56k6 dial up lines, and a top of the line computer had a Pentium 4 processor. No wonder it never gained much use outside of the specialist realm.
But I digress. I wrote a small class that will serve as a WMS client for the limited scenarios that play nice with the Windows map control, which require little knowledge of the actual plumbing for using it. This class is called WmsTileSource, How it's used, you can see in InitTileComboBox, where I initialize four instances of it. To explain how you get to such parameters, I will need to explain a bit first. The next paragraph is me getting on my GIS hobby horse. If you do not care about that, at least read the two bullet points halfway and the first sentence of the paragraph below the Africa picture.

Projections (and coordinate systems)

This may come as a surprise, but the Earth is not flat, yet every map since the beginning of map making pretends it is. This has various, mostly historical reasons, the most notable being the fact it's much easier to roll up a flat map of the of Earth's pieces you are interested in (or put a lot of them in a book) and take those along for the trip, than to lug around a true representation of the Earth - which most likely would contain a lot of details on stuff you don't need, and too little of the stuff you do. Yes kids, people used to lug atlases around, books with maps. They have the added bonus of never running out of battery.

So a map needs (or needed) to be flat, which makes it necessary to put the outside of what is essentially a sphere on a flat surface. That cannot be done without some dire consequences. Peel an orange and try to make the outside a continuous flat surface - and you will understand what I mean.
This is where projections come into play. These are ways to fit the sphere's outside on a flat surface. There are literally thousands of projections in use, both for the whole planet and for parts of it - so much they are mostly designated by numbers - so called EPSG numbers - not by name. To the left I show only a few ways to project Earth. The most well-known version is the Mercator projection, shown top-left. It is also known as WGS84 or EPSG:4326 (that is the number I meant). There is also a variant of this, hated by professional cartographers everywhere, but it became instantly and overnight the most used projection worldwide as it is used by Google Maps. It is referred to as 'popular Mercator', initially had the tongue-in-cheek designation EPSG:900913 (900913 being 'Google in "leet speak"), later officially adopted as EPSG:3857.
The point of this whole rambling paragraph is two-fold:

• The Windows 10 Maps control uses the EPSG:3857 projection. This means that only WMS servers that support delivering maps in EPSG:3857 / EPSG:900913 will correctly show data (that is, roads and stuff will appear on exactly the same places as your standard Here Maps data). EPSG:4326 will give 'mostly correct' results, provided you don't zoom out too much. All other projection systems won't work at all. At best they will show stuff in the wrong place, most likely it will appear to be wildly distorted as well.
• The Popular Mercator projection comes with some severe consequences. One of them is that the further you go from the equator (move closer to the poles), the more stuff gets stretched. A whole generation now has grown up believing Greenland is about the size of Canada (the biggest country in the world bar Russia), and the northern American continent is about the size of Africa. Well, have a look at this map - and have a reality check, too. Cheers!

One minor detail: another thing Google made us do was using the WGS84 coordinate system (aka "lat/lon", standing for latitude and longitude) for a projection is was not intended to, because that is so convenient when using it in conjunction with satellite navigation. There are also a lot of ways to designate locations on Earth other than just latitude and longitude, but outside the GIS inner circle hardly anybody knows - and even less people care. There is a lesson to be learned: if you design too academically oriented standards, they will be wiped out or at least bastardized by market parties more interested in practical appliances than scientific correct approaches.

Finding the right parameters for the WmsTileSource
Enough theory (and rambling) - we will now get down to business and actually use the WmsTileSource. First, you will need to know where the WMS server you want to use is actually located. This can be quite a challenge, but some institutions are really helpful. If you enter "noaa wms server" in Bing or Google and search a little around you will quite readily get to this site that shows you a whole range of interesting servers.

The important thing to know is that a WMS server can provide you with metadata about what it can and cannot do. These are called "capabilities" and you can get them but getting adding "?service=WMS&request=getcapabilities to the WMS URL. NOAA already has provided those links in their page - how thoughtful of them. You can click those links best using *cough*Google Chrome*cough* as that readily displays XML rather than trying to download it. So I clicked the third link (for "Recent GOES Weather Satellite Imagery", not visible in the image above) and then you get a rather depressingly long and hard to read XML potpourri. But don't despair, I will learn you a few tricks to quickly distill the things you need. We will need to find out:

1. Does this thing support ESPG:3857, EPSG:900913 or EPSG:4326?
2. What coordinate system tag is it using?
3. What layers are available?
4. What version of WMS is it running?

Find the EPSG
This one is easy. Just search for the number 3857, if you don't find that try 900913, then 4326. If you don't find anything, you're out of luck and the server is unusable. But good old NOAA does not fail us

So yes, we can use this server, it supports even the most optimal projection (and 4326 as well, but why use good if you can get perfect).

Find the coordinate tag
Over time the WMS standard has 'evolved'. In ye olden days the coordinate system was designated using the SRS tag, now it's mostly using the CRS tag. I think - but I am not sure - in WMS version 1.1.1 it was SRS, in 1.3.0 it's CRS. Anyway - you can quite easily find this by first searching for CRS, and if you cannot find that, SRS. See the image above - this one is clearly using CRS.

Find available layers

Guess what - you search for the text "Layer". Inside that Layer you will find a Title describing what it is and a Name that you will need to refer to it. NOAA have made this a bit complicated, but the important thing is to hunt for the Layer/Name.

So for Visible Imagery you will need a layer with the name "9". Usually a more descriptive name is used, but whatever.

Find the WMS version

Sometimes you can get this out of the URL, but if you cannot - well, you guessed it. Find the word "version" and that will give you give you most likely two possible outcomes: 1.3.0 or 1.1.1

Putting it all together
So know we have:

• EPSG = 3857
• Coordinate system tag = "CRS"
• Layer = "9"
• WMS version is "1.3.0"

This will allow us to construct a WMS layer like this:

new WmsTileSource("NOAA Visible Img",
"http://nowcoast.noaa.gov/arcgis/services/nowcoast/sat_meteo_imagery_goes_time/MapServer/WMSServer?", 
 new[] { "9" }, "1.3.0", "CRS"));

Notice that the layers are actually constructed as an array, as you can request for multiple layers on top of each other. This only makes sense when the layers are partially transparent, and visible imagery is not - so in this case it doesn't make sense. But if you want to use radar rain images on top of cloud data (see code inside InitTileComboBox) that makes perfect sense. Be aware that layers a drawn in order of appearance, so the last layer in the array will appear on top.
Also notice I did not provide any data for the EPSG. That is because that's the last parameter, and it's default value is 3857.

One more thing
In all cases - be it WMS to TMS - the device you are using goes out to the web to get tiles. That is why a HttpMapTileDataSource has an "AllowCaching" property, that is default set to true. So even if you don't download map tiles to your device, it still caches them - a nice feature for both your users and the map providers.

Conclusion
I have shown you the amazing versatility of the Windows Maps control in nearly all aspects in this series, and hope to have inspired you to look beyond the data that is offered by default with this last post. The world of geo data is a fascinating one, it is a shame so much is locked up behind the complexities of confusing, convoluted and/or outdated protocols.
Enjoy mapping! And let me know if you have found a cool map server to use in your app.

Borders on Xamarin Forms user interface elements

This is one of those blog posts, born out of utter frustration because looking for this yielded no usable results - while you would expect a lot of people would be wanting to do, as it's quite elementary IMHO. Maybe it's just my inexperience with Xamarin Forms, but for the life of me I could not find how to place borders around panel-like structures like a grid. Sure, there is Frame, but that places a meager 1 pixel hardly visible line around it.

Suppose I had this code

<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             x:Class="XamBorders.MainPage" BackgroundColor="White">
  <ContentView Padding = "20" VerticalOptions="Start" >
    <Label Text="I want a border here!" 
           FontSize="20" TextColor="Black" 
           HorizontalOptions="Center"></Label>
  </ContentView>
</ContentPage>

Result in in the UI below (on the Visual Studio Android Emulator)

How on Earth do you get something like this?

Turns out you need a couple of ContentViews inside each other like a bunch of Matryoshka dolls - one with the border color as background color, and inside that a slightly smaller ContentView with the same background color as the page. Like this:

<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             x:Class="XamBorders.MainPage" 
  BackgroundColor="{StaticResource BackgroundColor}">
  <ContentPage.Resources>
    <ResourceDictionary x:Name="AppDictionary">

      <Color x:Key="BackgroundColor">#FFFFFF</Color>
      <Color x:Key="BorderColor">#E1E1E1</Color>

      <Style x:Key="InternalViewStyle" TargetType="ContentView">
        <Setter Property="BackgroundColor" 
            Value="{StaticResource BackgroundColor}"/>
        <Setter Property="VerticalOptions" Value="Fill"/>
        <Setter Property="Padding" Value="10,10,10,10"></Setter>
      </Style>

      <Style x:Key="BorderStyle" TargetType="ContentView">
        <Setter Property="BackgroundColor" Value="{StaticResource BorderColor}"/>
        <Setter Property="Padding" Value="3,1,1,3"></Setter>
      </Style>

    </ResourceDictionary>
  </ContentPage.Resources>
  <ContentView Padding="20" VerticalOptions="Start" >
    <ContentView Style="{StaticResource BorderStyle}" >
      <ContentView Style="{StaticResource InternalViewStyle}">
        <Label Text="I want a border here!"
               FontSize="20" TextColor="Black"
               HorizontalOptions="Center"></Label>
      </ContentView>
    </ContentView>
  </ContentView>
</ContentPage>

So we have the internal view that has the same background color as the page, as well as a padding of 10 on every side to make the border not too tight around the text. Then the 'border contentview' around that has a padding 0 3,1,1,3 so that it's slightly larger bottom and right as to create some kind of shadow effect. If you don't want that, just make the padding equal. I defined the settings a styles as to make them easily reusable (they are in a app-wide resource dictionary in the app I am now developing).

Why it has to be this way - no idea, but I hope it will save some other Xamarin Forms newbie the frustration I experienced this afternoon. Maybe there are better alternatives - I really welcome comments. But this works

Sample project can be found here.

Windows 10 maps part 6 - scenes and camera

Intro
Picking up the thread where I left it rushing into an IoT series, which I wanted to have ready before the Microsoft devices event that took place on October 6th, I will now write bit about scenes and camera. These are features that are part are designed to work with the new 3D view, that already made an appearance in the previous part of this series.

The problem
Maps in were always 2D previously. You can play with the pitch and show landmarks on it, and that gets a bit of a 3D effect, but the map itself is still flat, and the view on your map is orthogonal (that is, straight from above), like you are holding a paper map. Only the zoom level (i.e. the distance from your eye to the map) and the point directly below you (the map center) determined what you look at. This approach no longer works when you use a 3D map like Windows 10 features. Now, you are basically floating in space next to a sphere (Earth, to be specific) and you can look in any direction you want - directly below the point you are floating next to (or above, whatever) but you can also look sideways, to the 'top' of the sphere, or even from it. You can rotate so South is up. So although the point directly below you and the distance between ('height') are still a factor in what you can see, they are no longer the only factor.

To illustrate what I mean, I can best show a few pictures. First, here we have Earth, and we are floating directly over the Netherlands. Basically, there's not much different from a normal orthogonal map, although this shows a bit of the curve of the Earth

Now let's play a little with the controls:

• First I changed the pitch using the canvas (the controls on the right of the map, the 2nd one from above)
• Then I rotated about 180 degrees using the top canvas control

You get a totally different view now!

Space, the final frontier ;) .

The solution
We need a new helper class, something that helps us create a view given a number of parameters. And luckily, Microsoft have provided not one but actually two: MapCamera and MapScene.

My demo solution actually does very little with both: the save scene button saves the current way you are looking at the map (Earth), and Restore brings you back to it.

private MapScene lastScene;

private void SaveScene(object sender, RoutedEventArgs e)
{
  lastScene = MapScene.CreateFromCamera(MyMap.ActualCamera);
}

private void RestoreScene(object sender, RoutedEventArgs e)
{
  if (lastScene != null)
  {
    MyMap.TrySetSceneAsync(lastScene);
  }
}

MyMap.TrySetSceneAsync is actually async so it could be awaited, but since this is a simple fire-and-forget method I can't be bothered with that.

If we put a breakpoint in SaveScene we can see the MapCamera's values

And it's location

So apparently I am looking almost due South (Heading 181), from a location that about 1600km above a point that is a bit west and a whole lot more north than the Netherlands (latitude and longitude for the center of our tiny spec by the sea is about 52, 5), pitched upward almost 40°, and rolled a wee bit to the right. My app does not support setting all camera features, but you if you set a breakpoint on SaveScene and change the roll value to 90 before you let it continue, you will get this effect on hitting restore - and you can imagine looking out of a forward-looking window of your spaceship in orbit - that is presumably in some pretty unlikely polar orbit considering the direction it's travelling into

The properties of the MapCamera are all excellently explained here in the official documentation and there's also a very good picture illustrating what I mean with 'floating next to a sphere and lookup down on it'

Now Microsoft have recognized the fact that not everyone is a spatial expert and that it may indeed be very hard to create a view in a 3D map that shows all what you want using just the camera, especially when there are blocking features. That's where the MapScene class comes into place. As you may already have noticed, the Map's Camera itself is read only, you can only use the camera to create a Scene from, then put it back to a map.

The intent though is that you create a Scene using one of the static methods the class provides. There is a method to create a camera from a bounding box, optionally specifying a heading an a pitch, a method to create one from a location and a radius, but the best I like is the one that makes sure any number of points in a list will show up. This is a great feature to show all your initial points in your 3D map app - for instance, all flying airplanes in 3D within the target area of your map.

Conclusion
Lots of talk this time, little code. MapCamera and (especially) MapScene make programmatically navigating around a map a lot easier, but it still take some experimenting to check if you get desired results. For me, having grown up in a world of 2D maps and having worked in 2D GIS for 20+ years, it's just as hard as you ;)