Can I call external executables from an Azure Function in an Azure App Service? Yes! (1/2)

This post has been republished via RSS; it originally appeared at: New blog articles in Microsoft Tech Community.

This is the first of two blog posts for this topic; this one will get you a local Node.js app running that is working against Azure Storage Blobs for image manipulation using a package that calls an external executable, and also set up some basic Azure infrastructure.  The next one will actually get the app running in an Azure Function.

First, let me do an important disclaimer, before I even talk about anything:

This Sample Code is provided for the purpose of illustration only and is not intended to be used in a production environment. THIS SAMPLE CODE AND ANY RELATED INFORMATION ARE PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY AND/OR FITNESS FOR A PARTICULAR PURPOSE. We grant You a nonexclusive, royalty-free right to use and modify the Sample Code and to reproduce and distribute the object code form of the Sample Code, provided that You agree: (i) to not use Our name, logo, or trademarks to market Your software product in which the Sample Code is embedded; (ii) to include a valid copyright notice on Your software product in which the Sample Code is embedded; and (iii) to indemnify, hold harmless, and defend Us and Our suppliers from and against any claims or lawsuits, including attorneys’ fees, that arise or result from the use or distribution of the Sample Code.

OK, now that that's out of the way...

I have a customer I'm working with that is doing an application migration and modernization project.  One of the critical parts of their application architecture is a service that runs in a Windows VM that performs specific image manipulation tasks.  This service today is a Node.JS service that has references to external executables to perform the image manipulation tasks.  It turns out, this is actually not that unusual.  There are even cases, such as in the case of this customer, where an application is using a library that is, underneath, shelling out to an executable installed on the server.  This posed a problem: how can they manipulate their images using tools they know today, that currently work with external files and external executables, in an Azure App Service?  The official Azure Samples repository on GitHub has functions-dotnet-migrating-console-apps/README.md at master · Azure-Samples/functions-dotnet-migrating-console-apps (github.com), but that's an older article that doesn't get into a lot of detail, and some of the UI has changed.

Here's one answer.  Note that I should again emphasize this is a reference discussion and is not meant to be "the one true architecture" or even necessarily "a good architecture" - but for the use case the customer had, with the scalability requirements the customer has and their knowledge base, this is a "good enough architecture".  The goal of this post is not to get into heavy architecture solutions, but instead just present one way to do this.  This is a very "rough and ready" post in the spirit of a blog post rather than a formal document.

This post is also not really presenting the long-term maintenance scenario here; there are a lot of CI/CD tools and multiple ways to deploy to Azure.  Because I'm focusing on the end goal, I'm going to do a lot of things in a more "manual" fashion.  It's an exercise for the reader to make a "proper" deployment out of this.

OK, so enough disclaimers.  Let's get to it.

First, we need the pieces installed locally.  It's out of the scope of this post to get into all the developer workstation setup details, but the pieces I had locally were:

• Visual Studio Code – I had 1.68.1 installed, but this should work on anything current
• The GraphicsMagick Image Processing System installed, as that's what is needed by the Node.JS package being used (customer's selection here for the tool, not mine, but that's fine, it works well and gets us to the point of the post!)  I had version 1.3.36 installed, but that’s not critical to the outcome.

Next, we need a place to run the application in Azure - I'll explain in a bit why this is coming up so early.  The customer's use case felt like an excellent use case for a Function App, so we'll start there.

1. In the Azure Portal, in the top search bar, search for function app, then select the Marketplace entry for Function App. 
   Azure screenshot showing "search" results for "function app"
2. Fill in everything as appropriate for your situation.  (Need help understanding about subscriptions and resource groups?  Check out the Azure Governance for ISVs sessions!)  Make sure to select Node.js for the Runtime stack:
   Azure screenshot showing the runtime settings
   I chose the long-term support version 16 option, as that was current when writing this, but the concept applies to whatever version you choose.
3. For the Operating system section, chose Windows for the OS, as that matches the scenario presented, but the overall concept isn’t Windows specific.
   Azure screenshot showing Windows as selected OS
4. In the Plan section, select the Consumption (Serverless) hosting plan, because this is a non-performance-sensitive scenario.  In other cases, you might choose a different hosting option.
   Azure screenshot showing the Consumption plan selected
5. On the Hosting tab, accept the default Storage account.
6. On the Networking tab, there was nothing to do, so move on.
7. On the Monitoring tab, set Application Insights set to Yes, accepting the defaults, because that allows better troubleshooting of the solution.  Again, the details on this are out of scope for this post, but trust me.
8. For this example, there’s no Tags, so move on from that tab.
9. On Review + create, created the Function App.  This won’t take too long, even with the Application Insights integration.
10. Once the app is created, use the Go to resource button to get to the Overview of the app.

OK, so great, now there's a place to host the app.  But there's no app yet!  So why do the host side first?  Because the host side automatically created an Azure Storage Account for hosting the Function App data.  This saves us from making one!  Of course, in production, you'd make a dedicated one, with proper HA/DR etc., but "rough and ready!"

So, let's go to Visual Studio Code now.  Create an empty folder where convenient, and have Visual Studio Code open that folder, telling Visual Studio Code you trust the folder when prompted.

Next, add a new file named app.js.  This gets us to a very basic Node.js app, although it doesn’t do anything yet. 

Then, add the Azure Blob Storage and GraphicsMagick packages, using npm in a Code Terminal window:

npm install @azure/storage-blob

npm install gm

Then, open the empty app.js file, and add the following non-production code:

Yes, this is not the best Node.js code, I know.  But again, the point is to show the concepts.  I fully expect someone that is a Node.js expert to do better. ;) 

So what's going on here?  Well, basically:

• Bring in the Azure Blob Storage module and configure it to talk to a storage account
• Bring in the GraphicsMagick module with a lightly documented (but supported, which is great for us!) parameter telling it where to find the external executable; note that the ending slash is critical here!
• Set up some references to where the pre-manipulation and post-manipulation images will be in the Azure storage account.
• Creates a function that actually does the image work.
• Call the app function to do the image work.

This is a little messier than I'd like because of the asynchronous nature of Node.js, but that very nature makes for better scalability and works well in a web environment, so it's worth putting up with it.  One very important thing here to notice is that the GraphicsMagick executable can read data piped in from stdin and send output to stdout, which in Node.js can be easily hooked up to Azure Blobs on both sides.  This is really cool, because it means no local files to worry about on the App Service filesystem for the image processing!  It also makes for some very clean code, ultimately.

OK, so there’s the code.  You’re not quite ready to test this yet, because lines 12 and 13 need a storage reference.  For now, you’ll just hard code those (this would be really bad in production but it lets the focus here be on the main goal of the external executable)... but what to use?  Remember earlier that there's a storage account out there for the Function App, so let's use that.  (Again, not production!)

The Azure Portal should still open on the Function App in the browser.  The name of the associated Storage Account and one of the two "god" keys for it can be found under Settings, Configuration in the Application settings value for AzureWebJobsStorage; the fastest way to pull that out is to select Advanced edit and in the JSON file, locate the value.  Being careful not to change anything, you can just copy the value to your clipboard:

Azure screenshot showing the Configuration view for the App Service

Azure Configuration "Advanced edit" screenshot

The AccountName and AccountKey in that string are what you want to put into the replacements for wat in the code.  (No, I won't give you mine!)  Here's an idea of what the lines will look like at the start:

"const accountName = 'stor" and "const accountKey = 'PD8Va"

OK, let's get a picture in a blob out there.  In the Azure Portal, search for the name of the storage account in the search box at the top to get quickly to your storage account.  Then, under Data storage, select Containers, then + Container, and create one named images.  Leave the Public access level at Private (no anonymous access).

Data storage, Containers

+ Container

New container, images, private

Note the code has this name and the blob name hard-coded but of course in real life, you'd be smarter than that I bet. ;) The container creates very quickly; once it's done, click/tap it on the list and you will be in the Portal's Container view.  (You could use Azure Storage Explorer for this next step but that's again, out of scope for now...)  Along the top command bar, select Upload, and upload any JPG you have lying around that happens to be named "test.jpg".  (Yes, you'll probably need to copy something to that name. ;) )

Guess what?  Almost there with a local copy of the app!  There's just one more step.  You can't do a global installation of something in the App Service environment, because you just can't do that in a Platform-as-a-Service world.  But, remember that the code is written to look for a local copy of the GraphicsMagick executable.  So let's give it one.  Back in Visual Studio Code, right-click/tap-and-hold the folder in the folder explorer, and select, and select Reveal in File Explorer.

Leaving that File Explorer open, open another one, however you'd like.  In the second one, find where you installed GraphicsMagick, probably something like C:\Program Files\GraphicsMagick-1.3.36-Q8.  Copy that whole folder into a utils folder of your project (this is why you have two File Explorer windows open!)... this will end up looking like this in your project:

Visual Studio Code folder window

OK, that should be it!  If you've done everything "right" here, you can now run the application in Visual Studio Code!  It will run for a short time, then exit.  You should then be able to look at the Container in the Azure Portal, and after doing a Refresh:

Refresh button

you should see your test.png output next to the test.jpg input!

ttest.png and test.jpg listed

If you click/tap on test.png and change to the Edit view, you can even see your picture right there in the Azure Portal!

sample test.png picture

Whew, that's a lot.  This is already a very long post!

In the next post, we'll take this code and get it running in an Azure Function!