The goal of this post is to dynamically spin up an Azure Windows VM and copy files to it using Visual Studio Team Services (VSTS) build and/or release. Throughout this post we will be collecting a lot of values we will need later. Open Notepad or some other application so you can paste values into if for later use as we discover them.
Certificates
Communication to the VM from our agents is done via WinRM. To enable WinRM you must have a certificate to encrypt the HTTPS traffic we use to communicate. The certificate can be a self-signed test certificate that you can create yourself. We will start by creating the certificate. Open the Visual Studio Developer Command Prompt for Visual Studio 2015. The command prompt is installed with Enterprise, Professional and Community editions of Visual Studio 2015. We are going to create a certificate that we can use for any Windows VMs deployed into the West US region. When running the first makecert command you will be asked for a password. Make note of the password because it is what you will pass into the pvk2pfx command.
makecert -sv westus.pvk -n "cn=*.westus.cloudapp.azure.com" -r westus.cer
pvk2pfx -pvk westus.pvk -spc westus.cer -pfx westus.pfx -pi EnterPasswordHere
If you wanted to deploy to another region you simply need to change *.westus.cloudapp.azure.com to *.desiredRegion.cloudapp.azure.com.
Azure Key Vault
The certificate must be stored in Azure Key Vault. We will now create an Azure Key Vault to store our certificate. We need to switch to the Azure PowerShell. For this post we will be using version 1.0.1. However, you do not have to have 1.0.1 on your agent. At the time of this writing our hosted agents are running Azure Powershell 0.9.7 and can be used to deploy ARM Templates.
Navigate to the same folder you just created the certificates in. Now execute the following commands to create an Azure Key Vault.
Login-AzureRMAccount
Once you log in run:
Get-AzureRMSubscription
Note the subscription name, subscription id and tenant id that matches the subscription shown after you logged in. We will need the subscription name, subscription id and tenant id when we create our service endpoint from VSTS to Azure. If you have multiple subscriptions you can switch the default subscription with the Set-AzureRMContext cmdlet. If you need to view them for the selected subscription simply run the Get-AzureRMContext cmdlet.
We are going to place the key vault in its own resource group. We will create that resource group now.
New-AzureRmResourceGroup -Name MyKeyVaults -Location “West US”
With the resource group created we can create our key vault. For this key vault to work for provisioning new VMs in Azure we have to make sure the Enabled For Deployment property is enabled which it is not by default so we add the -EnabledForDeployment switch.
New-AzureRMKeyVault -VaultName VMsKeyVault -ResourceGroupName MyKeyVaults -Location “West US” -EnabledForDeployment
With our key vault created we need to store the key we created earlier into the vault as a secret. Although key vaults can store certificates as keys, to work with the Azure Deployment task it must be stored as a secret. This requires us to 64 bit encode the contents of the file and build a JSON object that we also 64 bit encode before converting into a secure string so we can store as a secret.
$file = Get-Content .\westus.pfx -Encoding Byte
$encoded = [System.Convert]::ToBase64String($file)
$json = @”
{
“data”: “$encoded”,
“dataType”: “pfx”,
“password”: “EnterPasswordHere”
}
“@
$jsonBytes = [System.Text.Encoding]::UTF8.GetBytes($json)
$jsonEncoded = [System.Convert]::ToBase64String($jsonBytes)
$secret = ConvertTo-SecureString -String $jsonEncoded -AsPlainText -Force
Set-AzureKeyVaultSecret -VaultName VMsKeyVault -Name WestUSCert -SecretValue $secret
Be sure and make note of the Id value of the secret along with the vault name and resource group. We will need that for our ARM Template.
Create Service Principal
From Visual Studio Team Services we are going to need to establish a connection with our Azure subscription. We are going to do this using a Service Principal. We are going to create a new Service Principal for use with this post. But to do that we must first register an application in the Azure Active Directory which we will do now. As of this writing this can only be done in the original Azure portal. However, many of us are being forced into the new portal upon login. If you are, simply click on Browse and select Active Directory and you will be redirected to the original Azure portal.
- Now click on Applications of the selected Active directory.
- Click the Add button at the button of the page.
- Enter a name for your application and make sure Web Application and/or Web API is selected.
- Enter two URLs based on your application name. They do not have to be real. I used the same value for both.
- Once the application is created click on Configure.
- Make note of the Client ID because we will need it in a moment.
- Select a key duration under the keys section and click save at the bottom of the page.
- Once the key is saved copy the value and place it with your Client ID. This will be your only chance to collect this value.
Grant access
Now that we have a service principal we need to give it access to create resources in your subscription. Return to the new Azure Portal.
- Click Browse and select Subscriptions.
- Select the subscription you are using.
- Click the Access button.
- Click Add.
- Select Contributor as the roll.
- Search and select the name of the application you just created.
- Click OK to grant the service principal access to your subscription.
There is a script that can do all of this for you here on GitHub.
ARM Template
At this point I feel this post is long enough already so we are simply going to use a sample template and save a deep dive into ARM Templates for a later time. We are going to use the WinRM on a Windows VM sample template from GitHub.
- Clone the https://github.com/Azure/azure-quickstart-templates.git repo.
- Open the azuredeploy.parameters.json file in the 201-winrm-windows-vm folder and update the values. Many of the resource types have odd naming conventions. Do yourself a favor and stick to just lowercase letters and numbers. Make a note of the storage account name, admin login and password you enter, we will need them during our build.
- Check the files into VSTS.
Service Endpoint
It is now time to start using the values I told you to take note of. Using the Subscription Name, Subscription Id, Tenant Id, Client Id and Key we are going to create a service endpoint to Azure from VSTS.
- Log in to VSTS and select a project.
- Click the manage project gear icon in the upper right hand corner of the page.
- Select the Services tab.
- Select Azure from the New Service Endpoint drop down.
- Select the Service Principal Authentication radio button.
- You may enter anything you like for the connection name.
- Most of the fields are self-explanatory with the exception of Service Principal Id which should have been labeled Client Id.
Build
With the service endpoint created we can now create our build.
- Return to your project in VSTS and click Build.
- Click the plus to create a new build definition.
- Select the Empty template and click Next.
- Select the location where you checked in the ARM Template we copied from GitHub and click Create.
- Click Add build step…
- From the Deploy section add the Azure Resource Group Deployment and Azure File Copy to your build.
- Select the Service Endpoint we just created for the Azure Subscription.
- Enter a name for the Resource Group
- Set the location to West US (or the region you created your cert for).
- Use the Browse buttons for the Template and Template Parameters files.
- Select the Azure File Copy task.
- Use the Browse button and select any file or folder you would like to copy to the VM.
- Select the Service Endpoint we just created for the Azure Subscription.
- Select Azure VMs for the Destination Type.
- Enter the storage account name you used in the parameters.json file.
- Enter a name for your resource group used on the Azure Resource Group Deployment task.
- Enter the Admin login and password you used in the parameters.json file. If you need to secure the password you can use an encrypted build variable.
- Enter the folder on the VM that you would like the files copied to. The folder will be created if it does not exist.
Now save and queue a build. This build has taken as long as 20 minutes to run so be patient. Once it is done you will have a new Windows VM in Azure with the files you selected copied to it.