File pull and push from Azure Blobs in Azure functions with Powershell.
I’ve completed a PoC to adapt a local script and turn it into a function that runs based on an HTTP request. It was a great opportunity to work with serverless architecture using Azure Functions. As I adapted it from a Powershell script made by one of my colleagues, the language option was preset (I would have gone with Python otherwise, knowing myself) The main feature of the function is that it interacts with rules of an Azure Firewall policy to backup or modify (add or delete) some specific rules. Originally the script downloaded a backup locally as a .JSON file and consumed another .JSON of the rules to be modified.
My goal was to make it serverless and reside completely in the customer’s infra Azure, so the most interesting part was moving the functionality of writing and reading the files to an Azure Blob. Maybe I didn’t search well enough (and I searched, I searched a lot…) but I found that the Azure Functions documentation for Blob input or output binding with Powershell was limited and the examples were few and very simple. Of course, you can always use the AzCLI module Az.Storage
, leverage directly the Connection String and interact with the Blob like from anywhere else, but where’s the fun in that? Why not use the bindings? I ran into several issues, so I’m going to put the relevant code here along with some comments, hopefully it will help someone.
Bindings:
The first thing is to declare in the function.json
the bindings that we want. In this case my function has:
- An HTTP trigger.
- An HTTP output binding.
- An output binding to a blob.
- An input binding from a blob (the same, but it doesn’t have to).
¿Where´s my connection String?
There can be as many inputs and outputs as we want, but only one trigger. Most relevant here is the connection
parameter in the blob bindings. This parameter will refer to a secret in the function app, in a common namespace to all the functions that we have under that app. Very similar to the secrets of code repositories like Github. This way, we can have credentials saved for your use without exposing them. As we can see, the file name can also be parameterized in the bindings (backupFile
and sourceFile
in this case)
{
"bindings": [
{
"authLevel": "function",
"type": "httpTrigger",
"direction": "in",
"name": "Request",
"methods": [
"get",
"post"
]
},
{
"type": "http",
"direction": "out",
"name": "Response"
},
{
"name": "outputBlob",
"type": "blob",
"path": "poc/{backupFile}", # Path is directly the blob/file we want
"connection": "blobCs", # This is the name of the secret containing the ConnectionString
"direction": "out"
},
{
"name": "inputBlob",
"type": "blob",
"path": "poc/{sourceFile}",
"connection": "blobCs",
"dataType": "binary", # Explicit typecast
"direction": "in"
}
]
}
The function
The function is here simplified, and the relevant part is the one that reads or writes to the blob bindings.
At the beginning, we declare the modules and parameters to use, in this case they are read from the body
of the HTTP request.
using namespace System.Net
# Input bindings are passed in via param block.
param($Request, $inputBlob, $TriggerMetadata)
Import-Module Az.Accounts
# Interact with query parameters or the body of the request.
$sourceFile = $Request.Body.sourceFile
$backupFile = $Request.Body.backupFile
Output Blob: Writing multiple files to an output
By design the output binding is directly considered a blob (a file). The solution to be able to write more than one is to generate a temporary folder, save the files there and zip it later to push it to the output. I took advantage of the fact that in Powershell everything is an object and managed each rule object, exported it to an .XML file in a temporary folder. The folder is then compressed. Temporary folders are created and deleted on each run (if they exist) since the function can persist the filesystem between runs.
Write-Host "Rules discovery completed"
# Create temp folder to store all rules and then zip it
if (Test-Path -Path .\caughtRules) {
Remove-Item -Path .\caughtRules -Recurse -Force | Out-Null
}
New-Item -ItemType Directory -Path .\caughtRules
# Have the rule collection, now lets catch the rules we targeted
foreach ($rule in $targetedRules) {
$ruleName = $rule.Name
$caughtRule = $existingrulecollection.GetRuleByName($ruleName)
$caughtRule | Export-Clixml .\caughtRules\$ruleName.xml
}
# Zip temp folder with the objects in xml
if (Test-Path -Path .\caughtRules.zip) {
Remove-Item -Path .\caughtRules.zip -Force | Out-Null
}
Compress-Archive -Path .\caughtRules\* -DestinationPath .\caughtRules.zip | Out-Null
# Copy zipped file to output blob
$zipPath = ".\caughtRules.zip"
$content = [System.IO.File]::ReadAllBytes($zipPath)
Push-OutputBinding -Name outputBlob -Value $content -Clobber
Reading the file
Also by design the input binding is directly considered a single file and it will also be provided as a string
, something quite problematic if what we have there is a .zip. The first life-saving step, which can be seen in the function.json
, is to typecast the input to a ByteArray (declared there as binary
).
Once the typecast was done, I just had to treat the input blob as a byteArray from which to read my .zip file. Of course I also had to create temporary folders to store the unzipped rules in the same way as I did with the output…
if (Test-Path -Path .\unzippedRules) {
Remove-Item -Path .\unzippedRules -Recurse -Force | Out-Null
}
# Pulling inutBlob string into a MemoryStream
$asByteArray = $inputBlob
$zipPath = ".\sourceRules.zip"
# Writing ByteArray into .zip file
[System.IO.File]::WriteAllBytes($zipPath, $asByteArray)
# Expand .zip and load unzipped rules
Expand-Archive -Path .\sourceRules.zip -DestinationPath .\unzippedRules -Force | Out-Null
$objXmls = gci .\unzippedRules
$ruleArray = @()
foreach ($xmlobj in $objXmls){
$importedObject = Import-Clixml $xmlobj
$ruleArray += $importedObject
}
Keypoints
Actually, even when including so many threatening-sounding words, reading and writing from Azure Functions in Powershell to an Azure Storage Account using input and output blobs is not that complicated. But the lack of simple examples that can be extrapolated to each particular case can make us spend more hours than we want to get it done. Let’s remember the most important thing:
- Correctly define our bindings (including the connection Strings if they are necessary to connect to another resource)
- Typecast the input if we want to treat it as something other than
string
- Read and write correctly. The input/output blob already represents the file.
Kudos
- Blob storage bindings overview in official docs
- Blob input binding for Powershell in official docs
- Blob output binding for Powershell in official docs