Nowadays automation is one of the most wanted adoptions in the Market.
How comfortable is to create a whole new environment with only one click?

Automation can be simple or complex depending on our needs and what we want to achieve. I have always believed that automation is a strategy and at the same time a way of making your life easier without having the need to check all over again for possible mistakes.

You click it, you get it!

To keep it simple and understandable, in this post I will demonstrate a solution where we will automate the process of creating a single Azure resource in 3 different Resource Groups using Azure DevOps Pipelines and some other tools.
To do this, we will…

  1. create an manual SP with Bash using cloud shell
  2. create the YAML Pipeline with stages
  3. override ARM parameters in the YAML
  4. use Azure DevOps predefined variables
  5. create a PowerShell script using functions to create Variable Groups in Azure DevOps Library
  6. create a PAT (personal Access Token)

For testing purposes we will create a Keyvault , because Keyvault is one of the quickest azure resources that can be created at a glance, so no need to wait for the deployment especially when you want to make some tests in your account.

Lets start!

Part 1

First let’s take some preliminaries out of the way assuming that we have already created an organization and a project in the Azure DevOps Environment.
(If you dont know how, you can follow this link https://docs.microsoft.com/en-us/azure/devops/organizations/projects/create-project?view=azure-devops&tabs=preview-page)

For this post I have created this project:

The next step is to create an ARM template for the Keyvault we want to deploy and uploaded it to the Azure Repos:

As we know, if we want to deploy resources on Azure with ARM templates we need to have at least two files:

  • The one where we declare the resource/s you want to have (in this case the Keyvault.json)
  • The other one for the parameters (Keyvault.paramaters.json)

As a result , inside this Repo I have these two ARM template files needed for the Keyvault:

Now let’s have a quick look in the Keyvault.json file:

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "keyvaultname": {
            "type": "string",
            "metadata": {
                "description": "Name of the keyvault"
            }
        }
    },
    "functions": [],
    "variables": {
        "tenantidvar": "111-111-111-111-111",
        "objectidvar": "111-111-111-111-111"
    },
    "resources": [
        {
            "name": "[parameters('keyvaultname')]",
            "type": "Microsoft.KeyVault/vaults",
            "apiVersion": "2016-10-01",
            "location": "[resourceGroup().location]",
            "tags": {
                "displayName": "[parameters('keyvaultname')]"
            },
            "properties": {
                "enabledForDeployment": true,
                "enabledForTemplateDeployment": true,
                "enabledForDiskEncryption": true,
                "tenantId": "[variables('tenantidvar')]",
                "accessPolicies": [
                    {
                        "tenantId": "[variables('tenantidvar')]",
                        "objectId": "[variables('objectidvar')]",
                        "permissions": {
                            "keys": [
                                "get",
                                "create"

                            ],
                            "secrets": [
                                "get",
                                "list",
                                "set"
                            ]
                        }
                    }
                ],
                "sku": {
                    "name": "standard",
                    "family": "A"
                }
            }
        }
    ],
    "outputs": {}
}

As you can easily see I have defined only one parameter with the name “Keyvaultname” (obviously for the keyvault name) and two variables:

  • Tenantidvar
  • Objectidvar

That is because tenant ID should be used for authenticating requests to the key vault (globally unique identifier) and the object ID of a user, service principal or security group in the Azure Active Directory tenant for the vault. The object ID must be unique for the list of access policies.
Tip: you can find both ID values in your Azure portal. The tenant ID by going to your Tenant page and the object ID of a user by going to your Azure Active Directory->Users ->  click on your account and then you will get this window:

find your Tenant and object ID

Now let’s have a look at the parameters file (Keyvault.parameters.json):

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "keyvaultname": {
            "value": "XXX"
        }
    }
}

You can see the “XXX” as a name for the Keyvault. This is because we don’t actually need to specify the name, and this is because our intention is to override this value inside the Azure DevOps every time we deploy this resource to each resource group. What we want to achieve is to have a different Keyvault name for each resource group without having to maintain three different versions of the ARM templates (this is one of the aspects of automation after all!).
Now we need to create a connection between the Azure DevOps environment and the Azure portal, and we will do this by creating a Service Principal.

When you try to create a SP (service principal) you have four choices:

Select authentication method for the SP

If we opt for automatic SP then we need to specify which Resource group we will use to deploy resource/s.

Automatic is selected

But we cannot choose this option because the Recourse Groups are not present yet, they are going to be created when the pipeline starts. One more reason we cannot choose this option is because we need to deploy resource/s to 3 different environments (so 3 different RGs) at the same time and not only to one RG, so in this case automatic SP is not an option.
We want the SP to have access to the entire subscription deploying resources to more than one RGs and will do that by creating a manual SP.
Important note: creating a manual SP it also gives us more granular control regarding the security part but this out of the scope for this post.
To create a manual SP we need to take some steps first:

  1. navigate to shell.azure.com and login.
  2. Make sure the Environment is set to Bash.

3.Type the following command:
az ad sp create-for-rbac –name myspnamehere

and we will get this:

4. Azure will generate an appID, which is the Service principal client ID used by Azure DevOps Server. It will also generate a strong password, which is the Service principal key. The final value of interest is the tenant, which is the Tenant ID. We copy these values to a notepad because we will need them at a later step.

Now we go back to our DevOps environment and we choose to create a manual Service connection.
(Project Settings->Pipelines->Service Connections->New Service Connection->Azure Resource Manager->Service Principal Manual).

There, we also need to fill out the required fields with the values taken from the Bash command we run previously:

Service principal client ID = appId
Service principal key = password
Tenant ID =
tenantid

When you finished, don’t forget to click on “Verify” to check that the connection between your DevOps environment and the Azure portal has been made.
For test purposes I have already created a manual SP with the name “MymanuallycreatedSP”, so we will use this one to continue.:

Let’s summarize what we have done until now:

  1. We created a Keyvault ARM template
  2. We setup the Azure DevOps environment
  3. We created a manual Service Principal

Part 2

In Azure DevOps we can organize pipeline jobs into stages. Stages are the major divisions in a pipeline: “deploy in the 1st RG”, “deploy in the 2nd RG”, and “deploy in the 3d RG” are good examples of stages. They are logical boundaries in the pipeline.
Having this in mind we will create 3 different stages in the pipeline. Each stage will deploy a Keyvault for each RG. For example:

1st stage will deploy a keyvault to test-rg1
2nd stage will deploy a keyvault to test-rg2
3d stage will deploy a keyvault to test-rg3.

We need to provide some values to these stages like the RG names they are going to use for the deployment, subscription ID and some other things. Just like any other programming language, we can use variables to store these values. Azure DevOps offers the “Library” which you can find in the Pipelines section.

There, we can create variables and variable groups and use them inside the YAML code so the next step for us would be to specify what variables we want to have and organize them into variable groups.

First, we need to have a variable group with the name “CommonVariables” where we will create and store common variables needed for all stages for example:

Location (the location our resources are going to be deployed to).
Subscription ID (our subscription ID from Azure).

Secondly, we need to have a variable group name for each stage.
For example:
GroupVar-1 for stage1
GroupVar-2 for stage2
GroupVar-3 for stage3

Now in each variable group (except the first one) we need to have the following variables: KeyvaultName (the keyvault name we want to have in each RG).
Resource group Name (the Resource group where the Keyvault will be deployed to).
So it will look like this:

Variable GroupVariable NameValue
CommonVariablesvar_locationwesteurope
 var_subscriptionId111-111-111
GroupVar-1KeyVaultName-1_varkeyvaultA0015
 RG-name-1_vartest-rg1
GroupVar-2KeyVaultName-2_varkeyvaultA0016
 RG-name-2_vartest-rg2
GroupVar-3KeyVaultName-3_varkeyvaultA0017
 RG-name-3_vartest-rg3
Organising variable groups, variables and values

Normally, we would create these variable groups, variables and values through the Azure DevOps Portal using GUI but we are talking about automation here so we will try to do this as part of the automation process using a PowerShell script. Later on we will configure the pipeline to run the script. Remember, we automate everything without touching the portal !

Now, let’s write some PowerShell code!
The PowerShell script has been divided in two parts:

  1. Create three Resource Groups
  2. Create the Variable Groups + Variables + Values

This is how we will create the three resource Groups we want:

$myResourceGroupName = "test-rg"
Get-AzResourceGroup -Name $myResourceGroupName -ErrorVariable notPresent -ErrorAction SilentlyContinue

For (($i = 1); ($i -lt 4); $i++)
{

  New-AzResourceGroup `
     -Name ($myResourceGroupName + $i) `
     -Location westeurope
}

The above script is using the For loop to create three resource groups with the name test-rg + number. So What we want to have in the portal is the following resource groups:

test-rg1
test-rg2
test-rg3

Next step in our script is to create three different Functions where we will pass parameters for each  Variable Group , variables and values we want to have:
For example the following Function creates the Group Variable: CommonVariables with the variables we need to have there: location (used to define the location for all the resources) and the subscriptionId :

Function New-VariableGroup-Common ($groupName, $groupDescription,#Define Group Variable Name and Group Description. 
                            $variableLocation, $variableLocationValue,#Define location variable and value for Common Group Variable 
                            $variableSubscriptionId, $variableSubscriptionIdValue <#Define subscriptionId variable and value.#> ) {

$organizationName = "dimitriskrallis"
$projectName = "MyAutomationProject"
$personalAccessToken = "dgdgdfgdfgdfgdgfdfgdfg254234wrt##$"


$variableGroupsUri = "https://dev.azure.com/" + $organizationName + "/" + $projectName + "/_apis/distributedtask/variablegroups?api-version=5.1-preview.1"
$base64AuthInfo    = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f "", $personalAccessToken)))
    $body = @{
 
        type        = "Vsts"
        name        = $groupName
        description = $groupDescription
        variables   = @{$variableLocation = $variableLocationValue
                        $variableSubscriptionId = $variableSubscriptionIdValue }
 
    }
 
    $json = $body | ConvertTo-Json
         
    Invoke-RestMethod -Uri $variableGroupsUri -Method POST -ContentType "application/json" -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo) } -Body $json
 
}

New-VariableGroup-Common CommonVariables Variables-For-CommonUse var_location westeurope var_subscriptionId 11111-111-1111-1111-1111111111

Now, the next function will create the Group Variable in the Library which will be used for the first Resource Group. Inside this Variable Group, we will need to declare two variables: 
the keyvault name and the Resource Group where the keyvault will be deployed to:

Function New-VariableGroup-RG1 ($groupName, $groupDescription,#Define Group Variable Name and Group Description for RG1 Group Variable
                                $variableKeyVaultName, $variableKeyVaultNameValue, #Define KeyVault Variable Name and value for TestA-RG Group Variable
                                $variableResourceGroupName, $variableResourceGroupNameValue <#Define Resource Group Name and Value for RG1 Group Variable#>) {

$organizationName = "dimitriskrallis0042"
$projectName = "MyAutomationProject"
$personalAccessToken = "jghjgjhgjhgjhguytgyutyutyutgjhghjghjgjgjgg"


$variableGroupsUri = "https://dev.azure.com/" + $organizationName + "/" + $projectName + "/_apis/distributedtask/variablegroups?api-version=5.1-preview.1"
$base64AuthInfo    = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f "", $personalAccessToken)))
    $body = @{
 
        type        = "Vsts"
        name        = $groupName
        description = $groupDescription
        variables   = @{$variableKeyVaultName = $variableKeyVaultNameValue
                        $variableResourceGroupName = $variableResourceGroupNameValue }
 
    }
 
    $json = $body | ConvertTo-Json
         
    Invoke-RestMethod -Uri $variableGroupsUri -Method POST -ContentType "application/json" -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo) } -Body $json
 
}
#We call the function and pass the parameters needed for the first RG:
New-VariableGroup-RG1 GroupVar-1 VariablesForRG1 KeyVaultName-1_var keyvaultA0015 RG-name-1_var test-rg1

We need 3 functions (each for one resource group) so the entire Powershell script will be like this:

<#what does this script:
It uses the For loop to create three different RGs with a specific name convention.
After the RGs have been created, the next step is to run 4 different PS functions.
Each function creates a Variable Group and some variables needed later for the YAML code in the Pipeline.
Also, in each function we need to include the PAT token#>

#First create the three RGs with the For loop:

$myResourceGroupName = "test-rg"
Get-AzResourceGroup -Name $myResourceGroupName -ErrorVariable notPresent -ErrorAction SilentlyContinue

For (($i = 1); ($i -lt 4); $i++)
{

     New-AzResourceGroup `
     -Name ($myResourceGroupName + $i) `
     -Location westeurope
}

<#Now we will use this function to create a Variable Group which will contain only the Common Variables like location (used to define the location for all the resources) and the subscriptionId.#>

Function New-VariableGroup-Common ($groupName, $groupDescription,#Define Group Variable Name and Group Description. 
                            $variableLocation, $variableLocationValue,#Define location variable and value for Common Group Variable 
                            $variableSubscriptionId, $variableSubscriptionIdValue <#Define subscriptionId variable and value.#> ) {

$organizationName = "dimitriskrallis0042"
$projectName = "MyAutomationProject"
$personalAccessToken = "ghghghghghghghggghgghghghghghghg"


$variableGroupsUri = "https://dev.azure.com/" + $organizationName + "/" + $projectName + "/_apis/distributedtask/variablegroups?api-version=5.1-preview.1"
$base64AuthInfo    = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f "", $personalAccessToken)))
    $body = @{
 
        type        = "Vsts"
        name        = $groupName
        description = $groupDescription
        variables   = @{$variableLocation = $variableLocationValue
                        $variableSubscriptionId = $variableSubscriptionIdValue }
 
    }
 
    $json = $body | ConvertTo-Json
         
    Invoke-RestMethod -Uri $variableGroupsUri -Method POST -ContentType "application/json" -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo) } -Body $json
 
}

New-VariableGroup-Common CommonVariables Variables-For-CommonUse var_location westeurope var_subscriptionId 111111-1111-1111-1111-1111111

<#The following function will create the Group Variable in the Library which will be used for reference for the first Resource Group.
Inside this Variable Group,we will need to declare two variables: the keyvault name and the Resource Group where the keyvault will be deployed to.#>

Function New-VariableGroup-RG1 ($groupName, $groupDescription,#Define Group Variable Name and Group Description for RG1 Group Variable
                                $variableKeyVaultName, $variableKeyVaultNameValue, #Define KeyVault Variable Name and value for TestA-RG Group Variable
                                $variableResourceGroupName, $variableResourceGroupNameValue <#Define Resource Group Name and Value for RG1 Group Variable#>) {

$organizationName = "dimitriskrallis"
$projectName = "MyAutomationProject"
$personalAccessToken = "ghghghghghghghggghgghghghghghghg"


$variableGroupsUri = "https://dev.azure.com/" + $organizationName + "/" + $projectName + "/_apis/distributedtask/variablegroups?api-version=5.1-preview.1"
$base64AuthInfo    = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f "", $personalAccessToken)))
    $body = @{
 
        type        = "Vsts"
        name        = $groupName
        description = $groupDescription
        variables   = @{$variableKeyVaultName = $variableKeyVaultNameValue
                        $variableResourceGroupName = $variableResourceGroupNameValue }
 
    }
 
    $json = $body | ConvertTo-Json
         
    Invoke-RestMethod -Uri $variableGroupsUri -Method POST -ContentType "application/json" -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo) } -Body $json
 
}
#We call the function and pass the parameters needed for the first RG:
New-VariableGroup-RG1 GroupVar-1 VariablesForRG1 KeyVaultName-1_var keyvaultA0015 RG-name-1_var test-rg1

<#The following function will create the Group Variable in the Library which will be used for reference for the second Resource Group.
Inside this Variable Group,we will need to declare two variables: the keyvault name and the Resource Group where the keyvault will be deployed to.#>

Function New-VariableGroup-RG2 ($groupName, $groupDescription,#Define Group Variable Name and Group Description for RG2 Group Variable
                                $variableKeyVaultName, $variableKeyVaultNameValue, #Define KeyVault Variable Name and value for RG2 Group Variable
                                $variableResourceGroupName, $variableResourceGroupNameValue <#Define Resource Group Name and Value for RG2 Group Variable#>) {

$organizationName = "dimitriskrallis"
$projectName = "MyAutomationProject"
$personalAccessToken = "ghghghghghghghggghgghghghghghghg"


$variableGroupsUri = "https://dev.azure.com/" + $organizationName + "/" + $projectName + "/_apis/distributedtask/variablegroups?api-version=5.1-preview.1"
$base64AuthInfo    = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f "", $personalAccessToken)))
    $body = @{
 
        type        = "Vsts"
        name        = $groupName
        description = $groupDescription
        variables   = @{$variableKeyVaultName = $variableKeyVaultNameValue
                        $variableResourceGroupName = $variableResourceGroupNameValue }
 
    }
 
    $json = $body | ConvertTo-Json
         
    Invoke-RestMethod -Uri $variableGroupsUri -Method POST -ContentType "application/json" -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo) } -Body $json
 
}
#We call the function and pass the parameters needed for the second RG:
New-VariableGroup-RG2 GroupVar-2 GroupVariablesForRG2 KeyVaultName-2_var keyvaultA0016 RG-name-2_var test-rg2

<#The following function will create the Group Variable in the Library which will be used for reference for the third Resource Group.
Inside this Variable Group,we will need to declare two variables: the keyvault name and the Resource Group where the keyvault will be deployed to.#>

Function New-VariableGroup-RG3 ($groupName, $groupDescription,#Define Group Variable Name and Group Description for RG3 Group Variable
                                $variableKeyVaultName, $variableKeyVaultNameValue, #Define KeyVault Variable Name and value for RG3 Group Variable
                                $variableResourceGroupName, $variableResourceGroupNameValue <#Define Resource Group Name and Value for RG3 Group Variable#>) {

$organizationName = "dimitriskrallis0042"
$projectName = "MyAutomationProject"
$personalAccessToken = "ghghghghghghghggghgghghghghghghg"


$variableGroupsUri = "https://dev.azure.com/" + $organizationName + "/" + $projectName + "/_apis/distributedtask/variablegroups?api-version=5.1-preview.1"
$base64AuthInfo    = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f "", $personalAccessToken)))
    $body = @{
 
        type        = "Vsts"
        name        = $groupName
        description = $groupDescription
        variables   = @{$variableKeyVaultName = $variableKeyVaultNameValue
                        $variableResourceGroupName = $variableResourceGroupNameValue }
 
    }
 
    $json = $body | ConvertTo-Json
         
    Invoke-RestMethod -Uri $variableGroupsUri -Method POST -ContentType "application/json" -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo) } -Body $json
 
}
#We call the function and pass the parameters needed for the third RG:
New-VariableGroup-RG3 GroupVar-3 GroupVariablesForRG3 KeyVaultName-3_var keyvaultA0017 RG-name-3_var test-rg3

Important: the above script is using an API to create the variable groups in the DevOps environment”:
$variableGroupsUri = “https://dev.azure.com/” + $organizationName + “/” + $projectName + “/_apis/distributedtask/variablegroups?api-version=5.1-preview.1&#8221;

Also as you can see a PAT (Personal Access Token) is needed in every function. You need to create this from the DevOps environment by going to the upper right corner:

Part 3

Now we have created the Powershell script, its time to create/modify the pipeline YAML code: For the test purposes I have already created a pipeline with the name MyautomationPipeline, let’s edit the Pipeline and break down the YAML code to see what we have here:

trigger:
- main

pool:
  vmImage: ubuntu-latest

variables:
- group: GroupVar-1
- group: GroupVar-2
- group: GroupVar-3
- group: CommonVariables


stages:
- stage: ShowGeneralInfo
  displayName: show the ENV VARS
  jobs:
   - job: DisplayEnvVars
     displayName: Use the powershell command to display the ENV vars
     steps:
       - powershell: Write-Host "We use the branch $env:SOURCE_BRANCH . The name of the build pipeline is $env:BUILD_DEFINITION_NAME. The repository NAME is $env:BUILD_REPO_NAME and the Requested mail is $env:MAIL_REQUESTED"
         env:
           BUILD_DEFINITION_NAME: $(Build.DefinitionName)
           BUILD_REPO_NAME: $(Build.Repository.Name)
           MAIL_REQUESTED: $(Build.RequestedForEmail)
           SOURCE_BRANCH: $(Build.SourceBranch)

- stage: DeployToRGNo1
  displayName: Deploy in the First Resource Group (test-rg1)
  jobs:
    - job: Deployto1stRG
      displayName: Deploy the Keyvault in the test-rg1
      steps: 
      - task: AzureResourceManagerTemplateDeployment@3
        inputs:
          deploymentScope: 'Resource Group'
          azureResourceManagerConnection: 'MymanuallycreatedSP'
          subscriptionId: $(var_subscriptionId)
          action: 'Create Or Update Resource Group'
          resourceGroupName: $(RG-name-1_var)
          location: $(var_location)
          templateLocation: 'Linked artifact'
          csmFile: 'KeyVault.json'
          csmParametersFile: 'KeyVault.parameters.json'
          deploymentMode: 'validation'
          overrideParameters: '-keyvaultname $(KeyVaultName-1_var)'

- stage: DeployToRGNo2 
  displayName: Deploy in the Second Resource Group (test-rg2)
  jobs:
    - job: Deployto2stRG
      displayName: Deploy the Keyvault in the test-rg2 Resource Group
      steps: 
      - task: AzureResourceManagerTemplateDeployment@3
        inputs:
          deploymentScope: 'Resource Group'
          azureResourceManagerConnection: 'MymanuallycreatedSP'
          action: 'Create Or Update Resource Group'
          resourceGroupName: $(RG-name-2_var)
          location: $var_location
          templateLocation: 'Linked artifact'
          csmFile: 'KeyVault.json'
          csmParametersFile: 'KeyVault.parameters.json'
          deploymentMode: 'validation'
          overrideParameters: '-keyvaultname $(KeyVaultName-2_var)'

- stage: DeployToRGNo3
  displayName: Deploy in the Third Resource Group (test-rg3)
  jobs:
    - job: Deployto3dRG
      displayName: Deploy the Keyvault in the test-rg3 Resource Group
      steps: 
      - task: AzureResourceManagerTemplateDeployment@3
        inputs:
          deploymentScope: 'Resource Group'
          azureResourceManagerConnection: 'MymanuallycreatedSP'
          action: 'Create Or Update Resource Group'
          resourceGroupName: $(RG-name-3_var)
          location: $var_location
          templateLocation: 'Linked artifact'
          csmFile: 'KeyVault.json'
          csmParametersFile: 'KeyVault.parameters.json'
          deploymentMode: 'validation'
          overrideParameters: '-keyvaultname $(KeyVaultName-3_var)'

Now let’s see the structure the YAML code:
We will use 4 stages:
The 1st for displaying some info using environment variables.
the 2nd for deploying the Keyvault in the 1st Resource group (test-rg1).
the 3nd for deploying the Keyvault in the 2nd Resource group (test-rg2) .
and the 4th for deploying the Keyvault in the 3d Resource group (test-rg3) .

As you can see, before stages I use the “- group” statement. This is because later on we will use some variables, so the pipeline needs to know where to get these variables from.

Stage to display some info:
In YAML pipelines, we can reference predefined variables as environment variables.
There are Agent, build, pipeline, deployment job, system and checks variables.
Here we will use some of them to get some useful information like:
what is the branch and Repo we use, what is the mail that we have defined for the project ?

So when the pipeline run we get something like:

Getting info from predefined variables

Tip: more information about predefined variables can be found here:
https://docs.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml

Stage to deploy to RG1:
Here we will use variables from 2 Variable Groups:
CommonVariables and GroupVar-1 and will also define the template files for the Keyvault (source and parameter file) and we will also override the value.
For example, in the Keyvault template we have the value “XXX” for the name, but here we say to YAML code: “for the Keyvault name, go get the value from the KeyVaultName-1_var which is in the GroupVar-1 variable group” (we don’t need to declare the variable group in the stage).
We also use the variable “var_location” from the CommonVariables variable group.

the same logic applies for the rest 2 stages:
DeployToRGNo2
DeployToRGNo3
we just need to change the variables that will be used for each stage but some variables remain the same like var_location.
Our pipeline is ready, but if we try to run it we will get the following error:

and this is normal because when the YAML code runs , first checks if there are any variables and variable groups, if yes, then these variables need to be there before hand. So it is clear that the variable groups within their variables and values, must be defined BEFORE the pipeline run. As I mentioned before, we could create the variable groups with values and the Resource Groups in the portal, but the goal is here to do it with automation without using GUIs.
You might think “ok, lets insert the PowerShell script in the beginning of the pipeline as a single task”. Unfortunately we cannot do this but what we can do is to create another pipeline with only one task, to run the MyScript.ps1 and then “connect” these pipelines together!

To refresh our memory, MyScript.ps1 script does two things:
Firstly creates the resource groups in the portal and secondly creates the variable groups + variables + values in the DevOps Environment so we need another pipeline for this job.
For testing purposes I have already created the new pipeline using the existing Service principal:

trigger:
- main

pool:
  vmImage: ubuntu-latest

- task: AzurePowerShell@5
  inputs:
    azureSubscription: 'MymanuallycreatedSP'
    ScriptType: 'FilePath'
    ScriptPath: 'MyScript.ps1'
    errorActionPreference: 'silentlyContinue'
    azurePowerShellVersion: 'LatestVersion'

So now we have one more Pipeline the next step is to create a dependency between these two pipelines because now we want first to run the “MycreateRGsANDVARSPipeline” and THEN the “MyautomationPipeline”.
To do this we need to edit the “MyautomationPipeline” and go to the “triggers” option:

There, in the Build completion we need to click on Add, then select the Pipeline that will run the MyScript.ps1 to create the variable groups + RGs and finally save.

So what we actually “say” here is “when the MycreateRGsANDVARSPpipeline is finished, THEN run the next pipeline”.

Let’s finally run this solution and check if it works!
Lets run the first pipeline:

Now the first pipeline ended lets see if we have the variables in the Library:

Yes we have!

Now the MyautomationPipeline starts automatically..

MyautomationPipeline has 4 stages, you can see them in action while the pipeline is running:

In the end, we can see that all 4 stages run successfully !

Now lets have a look in the portal also to see that we have the resources we need:

We should also have a keyvault in each RG, lets see if we have it:

Yes we have!

Have we touched anything in the portal or DevOps environment?
No..
(except of course creating the pipelines and the PAT, you cannot avoid doing this!).

This was a simple example of what automation is about.

Thank you very much for reading this long post !
Stay healthy and safe!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.