18 October 2018

My first Azure build & deploy pipelines for an Azure Function V2

Intro

For those who think I am breaking rank and suddenly move away from HoloLens and Mixed Reality to Microsoft Azure (like some *cough* these days) - don't worry. I am still Windows Development MVP with every intention to stay within the Mixed Reality area. But every Mixed Reality app - like basically every app these days that's more than a simple toy needs a backend - mobile apps in general, but Mixed Reality apps just as well. Walk the World uses an Azure V1 function and a Redis cache to validate and cache request, and AMS HoloATC uses an app service to do a lot of calculations in the cloud and share selected airplanes. The new version, almost finished, uses two V2 functions. So basically, there's always an Azure component. And I used AMS HoloATC's new backend as a guinea pig for the pipelines because it's better to try something new on private projects first than to experimentally blow a customer's deployment smithereens, right?

RIGHT?

Caveat emptor

Anyway, in this post I wrote down my experiences - simply 'how I did it', how I used Azure pipelines. This may not be the best way to do it. I don't doubt some real DevOps heroes will have better ideas. But potentially making a fool out of myself never stopped me blogging.

Function first

To be able to build an deploy a function, we of course first need one. So I made this little demo project "MyFunctionV2"

1-make function

I created a simple HTTP trigger with anonymous access rights, which I think in any serious scenario you should never do, but this a demo, so here goes:

2-httptrigger

And then I simplified the already simple code in it even more so that resulted in this:

using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Logging;

namespace MyFunctionV2
{
    public static class HelloWorld
    {
        [FunctionName("HelloWorld")]
        public static async Task<IActionResult> Run(
            [HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)] HttpRequest req,
            ILogger log)
        {
            log.LogInformation("C# HTTP trigger function processed a request.");

            return new OkObjectResult("Hello, world");
        }
    }
}

I stuck the sources in GitHub. Now we are ready to publish.

First publish

Publish? Weren't you going to use the Azure pipelines? Yes - but only to deploy. And since I am a lazy ******* I do the first publish just like in the old days (that is, up to last week) from Visual Studio. Although things have improved considerably - I used to make a function in the portal first, delete everything and then start to publish, because that was the only way to get it into a consumption plan. This way is quick and easy But I digress.

I publish a new "App Service", still apparently

4-publish1

Visual Studio selects all kinds of defaults that are not quite desirable: 

5-publish2

I want at least a new consumption based plan, and while I am at it, a separate storage account as well - although I am not going to use that in this sample (my AMS HoloATC functions does, for sure).

7-publish

image

When you have set everything to your liking, it should look like this:

8-publish

Notice the Export button. This allows you to export the whole created configuration in the form of a JSON document. This you can use to completely automate the creation of this as well in a release pipeline, as this 'script' contains a number of variables you can set (like names for the service, storage, the plan, etc.). This is a bit more complicated and I haven't gotten to that part yet, but I know it's possible.

And if you click create, you might possible get this:

9-publish

Click yes, and we are done with this part.

imageCreate a Build pipeline

This requires a project to be present in . If you have opted to store the source in  Azure DevOps (or Visualstudio.com) there's already a project of course, but since I put the source in GitHub I need to create it myself. I took "TestProject" (I am not very gifted in the creative naming department, I know).

The following screenshots are made with the preview settings for Azure DevOps, so things may look a bit different still if your UI settings are still for the 'old' VisualStudio.com settings. You can force the move forward to click your profile picture right top and select "Preview features" and go from there. You might as well get used to it as the new look & feel is rolling out as I type.

image

Okay, so let's click Pipelines/Builds and continue



If this is your first pipeline, it will show the picture on the left, by the second and following pipelines you will see a list of pipelines with this on top, but chances are as you are reading this, it will be your first pipeline ;)

imageimage







After that, it's just following the steps:

image

I am not a XAML typist nor very versed in YAML, so I click "Use the visual designer"

image

Since we are using GitHub, you might need to make a connection first.

image

I will call mine "LocalJoost" - click "Authorize using oAuth". That will show a GitHub popup authorizing Azure Pipelines. Then select your repository:

image

Hit select

image

And then continue. You will then need to select a template, and need to scroll down quite a bit

image

Select C# function and hit apply

image

Now let's be bold and hit "Save & queue". This will save nor queue, but give you a three-option drop-down in which "Save & queue" is one of the options. Click that option, and it will still not save, nor queue, but you will get yet another popup:

image

Hit "Save & queue" - third time's a charm, you will see

image

on top of your screen. If you click the build number, it will take you to a page where you can follow the sausage being made, and when it's done you should see about this:

image

So far so good. We have a build and it's working! And you should get an e-mail for confirmation as well.

Defining a build trigger

You might want to have a look at when and how a build is kicked off, because we don't want to do this manually every time something changes. A good developer is still a lazy developer. So click Edit in the screen above

image

And then click "Triggers"

image

We select the continuous trigger checkbox and select the proper values, although the defaults will probably be ok right away:

image

Hit "Save and queue" once again, but this time only select "Save" from the pop-out menu.

So, for fun, let's change the Readme.md, push a commit to the master and have a look at the builds again. And sure enough, a built is started immediately by the CI trigger.

image

And this, my friends, is all it takes to set up a build pipeline for a V2 function with sources hosted in GitHub.  Now for the final part - I want automated deploy as well.

Create a Release pipeline

Starts easy again:

image

This time it does a better suggestion:

image

Hit apply. That gives you this screen which I find a wee bit confusing:

image

Let's click "Add an artifact". That creates a kind of pop-over on the right side:

image

Select "Build" for source type, then set the Source Build pipeline to be the build pipeline we just created, and the Default Version to "Latest". Then we hit add. Then click the red exclamation mark on "Stage 1", to view the build tasks:

image

You will have to select the Azure subscription used (I have two) plus the App Service where you want to deployment to happen. To be able to use the subscription you will have to click the Authorize button first:

image

And you might want to disable your popup blocker for this site to expedite that process:)

image

Once this is done, you can select "Function App" (don't let it sit at "Web App") and select the actual function app you want to deploy to

image

And then you click "Save".

image

Uhm, yeah, click OK I guess ;) .This will give you this screen

image

So let's create a release, manually, to see if things are working

image

Click "Create" (don't change anything). You will go back to this screen, but you will see a yellow-greenish banner indicating a release has been started indeed:

image

Click the release name in that banner, that will take you to this progress screen:

image

If you click "In progress" you will once again see the sausage being made in more detail:

image

And it should finish with the "Deploy Azure App Services - succeeded".

For some final automation

We want this of course to kick off automatically. To that extent, we have to click "Edit pipeline" again. If find this screen not very intuitive, but it's not rocket science to find out what's the idea with some clicking around

image

We seem to have to click the lightning bolt icon - the left one - to set the "Continuous deployment trigger".

image

Flip the toggle to to Enabled, hit save, and then the final test.

So does it work?

I have published to app MyFunctionV2, so the URL to the site should be https://myfunctionv2.azurewebsites.net and to the specific function https://myfunctionv2.azurewebsites.net/api/HelloWorld

Hitting that link gives, as expected, "Hello World". Now let's change the HttpTrigger function we made all the way at the beginning to return the text "Hello again, World". We then commit and push the code to GitHub, and now we wait...

We are off to a promising start.

image

And then we see a release being triggered automatically

image

And when that's done, we hit the URL again and sure enough:

image

Mission accomplished.

Conclusion

So even an complete n00b like me can create a fully features continuous build-and-deploy pipeline, in a manner of a few minutes. I can tell you that retracing my steps and actually writing it down actually took a lot more time than the actual creation of the pipeline itself. I think Azure pipelines makes DevOps a lot easier, and the new UI is a lot more intuitive that the old VisualStudio.com. There are still a few places where there's room for improvement, but this is very usable.

Anyone wanting to have a look at the completely uninteresting test function can do so here. I would not bother, to be honest ;)

09 October 2018

Workaround: remote texture loading does not work with MRTK Standard Shader

imageThis is one of those things that can cost you hours to fix and drive you completely crazy. I was working on a new improved version of my HoloLens / Mixed Reality app AMS HoloATC, and this version was using a dynamic OpenStreetMap for geographical reference. So I stole the OpenStreetMap slippy map from my own work. But times have moved on, and I wanted to use the new Standard Shader from the Mixed Reality Toolkit in stead of the Unity Standard Shader, to be able to use the cool fluent effects like the Hover Light and stuff.

Unfortunately, I not only got some fluent options that worked beautifully, but also some problems. If I used my DynamicTextureLoader that I wrote about like 1.5 years ago to download map tiles from the internet, nothing happened. The texture is downloaded, applied - but it never shows up.

To show you what it looks like, I made a demo project:

image

And as you can see on the most right picture, I got it to work as well.

The trick is in the materials. The difference is hard to spot, but one of the panels using a MRTK standard already has a dummy texture at startup:

image

This is fairly easy to achieve, like this:

image

It's a file of literally one white pixel. This suggests there is something in the MRTK Standard Shader that look for textures at startup and if there is no texture present, subsequent changes of the texture are ignored. I don't know if it's a bug or intentional - some kind of performance optimization - but this tricks gives you textures and fluent stuff.

A little demo project showing this trick can be found here.

05 September 2018

Getting a quick Mixed Reality log file from someone else's computer or HoloLens

When the **** hits the fan

A quick tip this time, born from necessity. I guess we've all been here - you make a perfectly working Mixed Reality app for Immersive Headsets and/or the HoloLens,, you test the hell out of it, you submit it to the store, you get approved, and you sit back and relax to wait for download numbers to go up and five star reviews to roll in...

... except they don't. Between the five stars a few one stars appear. "Does not work". "Crashes immediately on my system". Especially on Immersive Headsets things like this can happen, for the simple reason there are like 8 different headsets out there, connected to Finagle knows how many differently configurated PCs. I've had this happen to me last year, on systems that were configured with gtx1080i-based systems and/or the Samsung Odyssey headset - those things were so bloody fast my app ran into timing issues. But I didn't have either so I could not reproduce. What to do?

Getting the log file from a PC

Turns out you can actually get a Unity Player logfile from someone's system pretty easily. If your user is using an Immersive Headset, simply ask them to open go to the desktop of the PC the headset is connected to, open the File Explorer and go to the following folder:

%USERPROFILE%AppData\Local\Packages

In that folder there are a number of app-specific folders, usually quite a lot. Suppose there's a problem with my app AMS HoloATC. In the folder, your user needs to find the folder that is specific for AMS HoloATC

image

So you user needs to look into 17852LocalJoost.AMSHoloATC_rt0w0x8frck66

In that folder there's a folder TempState, which contains your coveted UnityPlayer.log:

image

In that file there's a lot of stuff - basically everything you see passing by in your Visual Studio Output window if you are running the app from Visual Studio:

Direct3D:
    Version:  Direct3D 11.0 [level 11.1]
    Renderer: Intel(R) HD Graphics 630 (ID=0x591b)
    Vendor:   (null)
    VRAM:     2111 MB
Initialize engine version: 2018.1.0f2 (d4d99f31acba)
WARNING: Shader Unsupported: 'MixedRealityToolkit/InvisibleShader' - Pass '' has no vertex shader
UnloadTime: 1.484216 ms
TryGetGeometry always returns false.
 
(Filename: C:\buildslave\unity\build\Runtime/Export/Debug.bindings.h Line: 43)

Display is Opaque
 
(Filename: C:\buildslave\unity\build\Runtime/Export/Debug.bindings.h Line: 43)

NullReferenceException: Object reference not set to an instance of an object.
   at OpaqueDetector.get_IsOpaque()
   at InitialPlaceByTap.Start()
   at InitialPlaceByTap.$Invoke2(Int64 instance, Int64* args)
   at UnityEngine.Internal.$MethodUtility.InvokeMethod(Int64 instance, Int64* args, IntPtr method) 
(Filename: <Unknown> Line: 0)

And lo and behold - apparently my OpaqueDetector class is playing havoc on the user's machine. Now at least I know where my app is going wrong.

Getting the log file from a HoloLens

You can do the same thing on a HoloLens, but there are a few caveats there:

  • You can only do this for sideloaded apps, not apps installed from the Store
  • Your user will need to use the file explorer in the device portal.

image

In User Folders \ LocalAppData \ 17852LocalJoost.AMSHoloATC_2.1.9.0_x86__rt0w0x8frck66 \ TempState \ you will find the log file. But you will need to send the user experiencing the problem an Appx package first. Since HoloLenses are basically all the same, this scenario is not very likely to happen, but it can be useful if you are making a sideloaded business app and someone hits upon an unforeseen scenario making the app behave in an unforeseen way - this may help a quick diagnosis.

Important notes

  • The log file is overwritten every session. So if the user starts the app again after aforesaid unexpected behavior, the data is gone. So make sure people using or testing the app know this.
  • Make sure you only write none-sensitive stuff to the log stuff: API keys, cookie values, decoded stuff and other information you don't want to tell the word - don't write that in your log file.
  • This is not a replacement for instrumentation. But it can (and has been to me) a life saver if someone from the HoloDevelopers Slack community can repro the problem, can quickly send you some log files, you can send a fixed Appx back for them to test, and in a few rounds you can squash the bug and submit a fixed version - even if you don't have access to the system playing havoc.

Credits and thanks

Special thanks to freshly (finally!) minted MVP Jesse McCulloch, both for making me aware of this file and where to find it - as well as being that invaluable repro/tester for me when Walk the World for Immersive Headsets suddenly started to crash on certain configurations - including, fortunately, his