It’s me again, blogging from a random hotel room.
Last time we had a look into the combination of OMS and
Azure in order to ensure availability of a Windows Server container hosted on a
virtual machine in Azure.
(The templates and scripts will be published soon).
This time, I would like to focus on another scenario I
think is very useful in the combination of Azure and OMS.
In this blog post, I would like to cover a common
scenario that we will run into every now and then when customers wants to
protect virtual machines automatically to Azure in a programmatic way.
Overview
More and more customers are looking into how they can
leverage the Azure cloud today, and one of the low-hanging fruits are services
that can easily be plugged into existing services on-premises that will easily
enable hybrid cloud scenarios, such as Business Continuity and Disaster
Recovery. These services can be harnessed directly from Azure but provides you
with a more comprehensive solution when used in conjunction with the entire OMS
suite – that includes these services as well.
In a couple of minutes, you can have your services and
applications running healthy as ever in an Azure region once this has been
configured, in case of a failover.
In this article I will not go into design principals
around the recovery processes (we’ll save that one for later), but rather cover
a scenario that automatically will take care of some heavy lifting for you.
Use case
Many organizations have Hyper-V running as their primary
hypervisor on-premises today, powering test, dev and production virtual
machines. Since Azure has been able to democratize disaster recovery with their
recovery services, people are looking into how to take advantage of this in a
streamlined and efficient way. My goal here is to show how you can onboard and
enable protection for newly created virtual machines on a Hyper-V host that has
been registered to your Recovery Services Vault in Azure, by combining events
that are logged into Log Analytics in OMS, monitored by a saved search that
also has an associated alert with remediation attached to it.
This will invoke a Runbook created in Azure Automation
that will enable replication on newly created virtual machines on that
particular Hyper-V host and replicate to Azure as the recovery site.
Breakdown of the
workflow
·
A VM gets created on the Hyper-V host.
Regardless of how it’s created, this will log a specific informational event in
the Hyper-V-VMMS-Admin log on the host, EventID 13002 that “A new virtual
machine was created”
·
The OMS agent deployed on the host will fetch
this and ingest this into Log Analytics
·
A search query is defined to monitor for this
specific event on this specific host
·
An alert is created and associated with this
query, so an e-mail will be sent when this occur
·
A runbook is created in Azure Automation that
will search for newly discovered virtual machines on the registered Hyper-V
host in the Azure Recovery Vault, look for VMs that isn’t protected and enable
protection for them.
·
This runbook is associated with the alert and is
part of the remediation process
Getting started
Before we can enable this scenario, we have to have some
prerequisites in place:
·
OMS Workspace
·
Azure Automation account
·
OMS agent installed on the applicable Hyper-V
hosts
·
Azure Recovery Vault in ARM
·
Runbooks
Assuming you have all of the above except the runbooks, I’ll
cover the creation of the Azure Recovery Vault and the Runbook and stitch
everything together.
Creating Azure
Recovery Vault with Azure Resource Manager
The following PowerShell cmdlets will enable Azure
Recovery Vault in your subscription and go through the creation of the Vault:
Note: You must
install and register the agent on your Hyper-V host manually in this process,
before proceeding with the rest of the script.
# Login to Azure and select Azure subscription
Login-AzureRmAccount -Credential (get-credential
-Credential kristian.nese@ledzeppelin.onmicrosoft.com)
Select-AzureRmSubscription –SubscriptionID $subID
# Check to see if your Azure subscription is enabled with ASR
resource providers
Get-AzureRmResourceProvider -ProviderNamespace "Microsoft.RecoveryServices"
Get-AzureRmResourceProvider -ProviderNamespace "Microsoft.SiteRecovery"
# Register ASR Resource Provider
Register-AzureRmResourceProvider -ProviderNamespace "Microsoft.SiteRecovery"
Register-AzureRmProviderFeature -FeatureName "betaAccess"
-ProviderNamespace "Microsoft.RecoveryServices"
Register-AzureRmResourceProvider -ProviderNamespace "Microsoft.RecoveryServices"
# Verify to see that the RP is registered correctly
Get-AzureRmResourceProvider -ProviderNamespace "Microsoft.RecoveryServices"
Get-AzureRmResourceProvider -ProviderNamespace "Microsoft.SiteRecovery"
# Create a new ASR Recovery Vault
mkdir -Path "c:\ASRDemo"
$RG = "DRGroup"
$Location = "west europe"
$vault = "HyperV"
$path = "c:\ASRDemo"
New-AzureRmResourceGroup -Name $rg -Location $location
$vault = New-AzureRmRecoveryServicesVault -Name $vault -ResourceGroupName $rg
-Location $location
$FilePath = Get-AzureRmRecoveryServicesVaultSettingsFile -Vault $vault -Path $path | % FilePath #
changed this to make sure we get the right filepath
Import-AzureRmSiteRecoveryVaultSettingsFile
-Path $FilePath
# Create a new Hyper-V Site in the ASR vault
$sitename = "KNHVSite"
New-AzureRmSiteRecoverySite -Name $sitename
# Query the job
Get-AzureRmSiteRecoveryJob
# Generate and download registration key for the site (Copy
the downloaded key to the Hyper-V host. You'll need the key to register the
Hyper-V host to the site)
$SiteIdentifier = Get-AzureRmSiteRecoverySite
-Name $sitename
| Select -ExpandProperty SiteIdentifier
Get-AzureRmRecoveryServicesVaultSettingsFile
-Vault $vault
-SiteIdentifier $SiteIdentifier
-SiteFriendlyName $sitename
-Path $Path
# Download and install the agent on the Hyper-V host(s) from
this URL: https://aka.ms/downloaddra
# Verify that the Hyper-V host is registered in ASR
Get-AzureRmSiteRecoveryServer
# Create storage account with Geo-redundant storage
$storageaccountID = New-AzureRmStorageAccount
-ResourceGroupName $RG
-Name "knasrdemo01"
-Type Standard_GRS
-Location $Location
# Create a replication policy and associate it with the
protection container
$ReplicationFrequencyInSeconds = "300"
$PolicyName = “DefaultPolicy”
$Recoverypoints = 1
$storageaccountID = Get-AzureRmStorageAccount
-Name "knasrdemo01"
-ResourceGroupName $RG
| Select -ExpandProperty Id
$PolicyResult = New-AzureRmSiteRecoveryPolicy
-Name $PolicyName
-ReplicationProvider “HyperVReplicaAzure”
-ReplicationFrequencyInSeconds $ReplicationFrequencyInSeconds -RecoveryPoints
$Recoverypoints -ApplicationConsistentSnapshotFrequencyInHours
1 -RecoveryAzureStorageAccountId
$storageaccountID
# Get the protection container and start the association
$protectionContainer = Get-AzureRmSiteRecoveryProtectionContainer
$Policy = Get-AzureRmSiteRecoveryPolicy -FriendlyName $PolicyName
$associationJob = Start-AzureRmSiteRecoveryPolicyAssociationJob -Policy $Policy -PrimaryProtectionContainer $protectionContainer
Once this has been created and the Hyper-V host(s) are
registered to Azure, you should be able to see the virtual machines on that
host by executing the following cmdlet:
Get-AzureRmSiteRecoveryProtectionEntity -ProtectionContainer
$protectionContainer | select friendlyname
FriendlyName
------------
ASRGEN2-01
ASRGEN2-03
ASRGEN2-02
Preparing Azure Automation
There’s a few things we need in order to successfully
create and run our runbook in Azure Automation
·
Assets
o Credentials
o Variables
o Modules
·
Runbook
For the assets, you can easily add the required assets by
using PowerShell.
In my case, I need the credentials to login to an Azure
subscription, and I also need to store the subscription ID in a variable.
This can be achieved using the following cmdlets:
Creating Automation
Variable
New-AzureRmAutomationVariable -Name SubscriptionID -Encrypted $false -Value $Subscription -ResourceGroupName $RGName -AutomationAccountName $AAName
Creating
Automation Credential
New-AzureRmAutomationCredential -Name AutoAdmin -ResourceGroupName $RGName -AutomationAccountName $AAName -Value $cred
Uploading
Automation Module
Normally you would use a cmdlet for uploading PowerShell
modules to Azure Automation as well, but since we just want to grab a few of
them directly from the available PowerShell Gallery in the AA itself, we
quickly head over to the portal and grab them from there.
Install the following modules in this specific order.
Note: it can
take several minutes before the modules are installed and ready. You might find
yourself a cup of coffee while doing this, as some of these modules has
dependencies of each other and won’t import before the dependencies has
completed the import process
Creating the
Runbook
Now that the prereqs are in place, it is time to author
the Runbook that will do the following:
Runbook
# Runbook for enabling DR on unprotected Hyper-V VMs
$Admin = Get-AutomationPSCredential -Name
AzureAdmin
$Subscription = Get-AutomationVariable
-Name SubscriptionID
Login-AzureRmAccount -Credential $Admin
Select-AzureRmSubscription -SubscriptionId $Subscription
# Fetching some variables for this particular setup
$RG = "DRGroup"
$Location = "west europe"
$vault = "HyperV"
$sitename = "KNHVSite"
$vault = Get-AzureRmRecoveryServicesVault -Name $vault -ResourceGroupName $rg
$storageaccount = "knasrdemo01"
# Creating a temp directory to store the VaultSettingsFile
$tempFileName = $env:TEMP
$FilePath = Get-AzureRmRecoveryServicesVaultSettingsFile -Vault $vault -Path $tempFileName
| % FilePath #
changed this to make sure we get the right filepath
Import-AzureRmSiteRecoveryVaultSettingsFile
-Path $FilePath
$SiteIdentifier = Get-AzureRmSiteRecoverySite
-Name $sitename
| Select -ExpandProperty SiteIdentifier
Get-AzureRmRecoveryServicesVaultSettingsFile
-Vault $vault
-SiteIdentifier $SiteIdentifier
-SiteFriendlyName $sitename
-Path $tempFileName
$protectionContainer = Get-AzureRmSiteRecoveryProtectionContainer
# Get Storage Account
$storageaccountID = Get-AzureRmStorageAccount
-Name $storageaccount
-ResourceGroupName $RG
| Select -ExpandProperty Id
# Get Recovery Policy
$policy = Get-AzureRmSiteRecoveryPolicy
# Enabling VMs for protection
$protectionEntity = Get-AzureRmSiteRecoveryProtectionEntity
-ProtectionContainer $protectionContainer
| Where-Object
{$_.FriendlyName
-like "*asr*"
-and $_.ProtectionStatus -eq
"Unprotected"}
foreach ($entity
in $protectionEntity)
{
if
($protectionEntity.ProtectionStatus
-eq "Unprotected")
{
Write-Output
"They aren't protected!"
Set-AzureRmSiteRecoveryProtectionEntity
-ProtectionEntity $entity
-Policy $Policy
-Protection Enable
-RecoveryAzureStorageAccountId $storageaccountID -OS
Windows -OSDiskName
$protectionEntity.Disks[0].Name -Verbose
}
else
{
Write-Output
"They were protected already :-)"
}
}
If you want to use this in your environment, ensure you are
changing the variables to meet your needs.
OMS
Assuming you already have the host registered to the
Workspace, you should do the following steps to be able to get the information you
are looking for to enable this scenario where you want to leverage the runbook
to protect newly created VMs.
Adding logs to OMS
In the OMS workspace, Click on ‘Settings’
Navigate to ‘Data’ and click on ‘Windows Event Logs’
Add the following log that will contain the information
about creation of virtual machines on the host
Ensure that Error, Warning and Information is selected
Note: OMS
doesn’t care about what has already happened, so only new events in this log
will appear in OMS.
Next, go back and drill into ‘Log Search’
I use the following search to pinpoint the specific
EventID and the Hyper-V host
EventID=13002 host11 | Select Computer, Activity,
TimeGenerated, Message, EventLog
I have saved the search and categorized it as ‘Hyper’V.
Now, I want to enable alerting on this search, so I click
on the search and then on the ‘Alert’ button
Assign a name to the alert and specify when the query
should run and when an alert should be generated.
I then specify the recipient of the alert and give it a
name.
As a last step, I connect the alert with the newly
created Runbook in Azure Automation and ensure that it will be executed by
using an Azure worker and click save.
Creating a new VM
to trigger the alert
Heading over to my Hyper-V host I created some new
virtual machines.
Coffee time
Since OMS will use this search query every 15 mins, I had
enough time to make myself some coffee while waiting for the e-mail to drop in
my inbox
Alert
Once the search query detect a new event, the alert is
triggered and an e-mail is fired away to my inbox.
Remediation
This should invoke the associated runbook for
remediation, and when I check into the Job view in Azure, I can see that it has
successfully been executed and also the output that tells me that replication
has now been enabled on the unprotected VMs.
Verifying the
remediation
Using PowerShell, I can access my Recovery Vault and
check to see which VMs are in the process of being protected in Azure by using
the following cmdlet:
Get-AzureRmSiteRecoveryProtectionEntity -ProtectionContainer $protectionContainer | select FriendlyName, ProtectionStateDescription
The output shows that several machines are already
protected while the newly created VM is in the process of doing an initial
replication
FriendlyName ProtectionStateDescription
------------ --------------------------
ASRGEN2-04
Protected
ASRGEN2-02
Protected
ASRGEN2-01 Initial
replication is in progress
ASRGEN2-03 Protected
Closing note
This was yet another blog post that shows some of the
capabilities of combining OMS, Azure and Azure Services together.
Moving forward, I will focus more on some of these
scenarios to show where we have some gaps today, and how you can overcome those
gaps with a little effort in engineering.
By the end of the day, we can more or less achieve
whatever we want as long as there’s an API for it, and that’s exactly what this
blog post was showing.
Thanks!