Monday, December 21, 2015

Azure Windows Server Container with IIS

A couple of months ago, Microsoft announced their plans for Azure and containers where they would provide you with a first class citizen resource provider in Azure so that you could build, run and manage scalable clusters of hosts machines onto which containerized applications would be running.

What you also probably have noticed is that Microsoft is having an open approach to container management. In fact, the container service is currently based and pre-configured with Docker and Apache Mesos, so any tools you would prefer for management “should just work”.
This is a new game for me to play so I am learning a lot. J

In the meantime, I am also working a lot with Windows Server Containers in Windows Server Technical Preview 4 – which is an image that is available in the Azure gallery.
However, I wanted to extend the experience a bit and decided to create my own ARM template that will ‘mimic’ some of the functionality in the Azure Container Resource Provider, to actually instantiate a new container running IIS Web-Server and be available for requests.

The template will deploy:

·         A Vnet
·         Network interface
·         Public IP address with DNS (the DNS will be based on the hostname.region.cloudapp.azure.com and provided as output once the deployment has completed)
·         Storage account
·         Network Security Group to allow RDP to the host – as well as http
·         Virtual machine (based on the TP4 image)
o   Custom Extension that will:
§  Spin up a new Windows Server Container based on the existing image (server core)
§  Install Web-Server within the newly created container
§  Stop the container – and create a new container image
§  Deploy a new container based on the newly created container image
§  Create a static NAT rule and a firewall rule to allow traffic on port 80 to the container from the host


This is a working experiment and I am planning to extend the template with more applicable tasks as we move forward.

The template can be explored and deployed from this GitHub repo: 

https://github.com/krnese/AzureDeploy/tree/master/AzureContainerWeb 


Thursday, December 3, 2015

Getting started with Containers in Azure

Recently, I had a presentation/workshop in Norway at a Docker conference (http://www.code-conf.com/day-of-docker-osl15/program/#knese )


This was quite a new audience for me and it was great to be the person who showed them what Microsoft is doing in the era of container technologies, using both Microsoft Azure and Windows Server 2016 Technical Preview 4.

The big picture 

One of the key things to point out is that containers are “just” a part of the big picture that we are seeing in these days.
The following graphic shows where we are coming from – and also where we’re heading.


Starting at the bottom, the early generation in this industry used to have a lot of physical machines to run their business. We all know that having workloads and applications on physical machines is not where we want to be today, because that is not flexible, scalable and for sure want do any good for our demand for utilization.

Above physical machines we can find machine virtualization. This should all be quite common now and we have been very good at virtualizing servers for quite some time. In fact, we are now not only virtualizing servers – but also the other infra components too, such as networks and storage.
Machine virtualization in this context is showing us that we are abstracting the compute resources from the underlying physical machine – which introduces us to the first stepping stones towards flexibility, scalability and increase the utilization.

Further, we have infrastructure hosting which can be seen as the early days of cloud, although the exact service model here is not defined. This means that “someone” would do the investment and ensure the required amount of capacity for you as a customer, and you can have your workloads and applications hosted in the hosting datacenter. This was machine virtualization at scale.

The next step is the more familiar service models we can consume from a cloud, such as Infrastructure as a Service, Platform as a Service and Software as a Service. Although these service models are different, they share the same set of attributes such as elasticity, self-servicing, broad network access, chargeback/usage and resource pooling. Especially elasticity and resource pooling is a way to describe the level of flexibility, scalability and utilization we can achieve. I expect you as the reader to be quite comfortable with cloud computing in general, so I won’t dive deeper into the definition at this point.

Next, we are now facing an era where containers are lit up – regardless whether you are a developer or IT-pro. Containers builds on many of the same principals as machine virtualization, where abstraction is key. A container can easily be lifted – and shifted to other deployment environments without having the same cost, luggage and complexity as a virtual machine – as a comparison.

In the Microsoft world we have two different runtimes for containers.
Windows Server Containers that are sharing the kernel with the container host which is ideal for scalability, performance and resource utilization.
Hyper-V Containers gives you the exact same experience, only that the kernel in this case isn’t shared among the containers. This is something you need to specify during deployment time. Hyper-V Containers will give you the level of isolation you require and is ideal when the containers aren’t trusting each other nor the container host.
Microsoft has also announced that they will come with their own Azure Container Service in the future, as a first-class citizen resource provider managed by ARM.

Last but not least, we have something called “microservices” on the top in this graphic. In the context of Microsoft we are talking about Service Fabric – which is currently a preview feature in Microsoft Azure today.
Service Fabric is a distributed system platform where you can build scalable, reliable and easily managed applications for the cloud. This is where we are really seeing that the redundancy, high-availability, resiliency and flexibility isn’t built into the infrastructure – but handled at the application level instead.
Service Fabric represents the next-generation middleware platform for building and managing these enterprise class, tier-1 cloud scale services.

From a Microsoft Azure standpoint it is also important that you know that “VM Scale Sets” (http://kristiannese.blogspot.no/2015/11/getting-started-with-vm-scale-sets-with.html ) is the IaaS that enables these PaaS services (Azure Container Service + Service Fabric).
Also, as part of Windows Server 2016 Technical Preview 4, we will be able to leverage Nano Server for containers too, so you can get the optimal experience for your born-in-the-cloud applications.

So, that was me trying to put things into context and why I spent some time that day to have a workshop on Containers using Azure.

Getting started with Containers in Microsoft Azure

The material I used for this workshop can be found in this public GitHub repo: https://github.com/krnese/AzureDeploy/tree/master/AzureContainer


I created an ARM template that will:

·         Create a new storage account
·         Create a new Network Security Group
o   Create a new vNet and associate the new subnet with the NSG
·         Create a new network interface
o   Associate the vNic with a public IP address
o   Associate the vNic with the vNet
·         Create a new virtual machine
o   Associate the VM with the storage account
o   Associate the VM with the network interface
o   Use Custom Script Extension that will create x amount of Windows Server Containers based on the parameter (count) input
 


















If you deploy this from GitHub and follow the ps1 examples you should be able to simulate the life-cycle of containers in Windows Server 2016 TP4.



Friday, November 20, 2015

Getting started with VM Scale Sets with Azure Resource Manager

Recently, Microsoft announced the public preview of ‘VM Scale Sets’ which is a new Azure Compute resource (Microsoft.Compute/virtualMachineScaleSets) that lets customers deploy and manage a set of virtual machines that are identical.

Sounds familiar?

Yes, but at the same time, this is new. Let me explain why.

Azure Compute, Network and Storage serves as the backend for many familiar Azure Services that we are using already today, such as Web Apps, Batch, Azure Automation and much more.

You have probably also heard about the newly announced public preview of Service Fabric – which is the ideal platform for microservices and containerized workloads to ensure business continuity and reliable applications, completely changing the way customers can develop applications at hyper-scale.
But did you know that the Service Fabric is also a service that runs on top of a virtual machine, requires network connectivity using a “normal” network in Azure and also can use some of the storage features in the backend?

That is also where the VM Scale Sets can come into play, serving as the perfect foundation for those kind of services and applications you want to build.
VM Scale Sets are designed to support autoscale and doesn’t require any pre-provisioning of the virtual machines.

Network and storage features are of course incorporated as you would expect, so that you can easily leverage VM Scale Sets as a first class citizen in Azure, following the Resource Group structure you prefer.

I encourage you to take a closer look at VM Scale Sets, which is a new resource type within the Microsoft.Compute namespace for the CRP.

I have already created an Azure Resource Manager template that let you deploy VM Scale Sets – using DSC VM Extension to configure Web-Server (IIS), as well as deploying a virtual machine you can use for management.
There’s also additional details such as load balancing, network security groups and more that you can explore.





Have fun – and happy scaling!



Thursday, October 29, 2015

Azure Resource Manager – Deployment options

Hi all,

This is just a quick blog post to demonstrate how to provision an IaaS environment in Azure with the VM DSC extension to instantiate a new Web Server (IIS).

Say what?

You have probably seen many examples of this already, so I won’t try to sell you something new here.
However, I want to point out the difference of using an Azure Resource Manager template (.json, declarative) compared to using PowerShell – in an imperative way.

The reason for this blog post is the newly released AzureRM PowerShell module which introduces us to a new set of cmdlets (the downside here is that I am now forced to update the whitepaper… https://gallery.technet.microsoft.com/Cloud-Consistency-with-0b79b775 ).

Where we are coming from

Previously with the Service Management API, we normally created our virtual machines in a similar way to this:

$image = Get-AzureVMImage -ImageName "a699494373c04fc0bc8f2bb1389d6106__Windows-Server-2012-R2-201412.01-en.us-127GB.vhd"
$vnet = Get-AzureVNetSite
$vm = New-AzureVMConfig -Name $VMName -InstanceSize "Basic_A2" -ImageName $image.ImageName

### Deploy a new domain joined VM

$vm = Add-AzureProvisioningConfig -VM $vm -AdminUsername $username -Password $pwd -WindowsDomain -JoinDomain "azure.systemcenter365.com" -Domain "azure" -DomainUserName "knadm" -DomainPassword "superPWD" | Set-AzureSubnet -SubnetNames $vnet.subnets.name | Set-AzureStaticVNetIP -IPAddress "10.0.40.52"

New-AzureVM -VM $vm -Location "North Europe" -VNetName $vnet.name -ServiceName $ServiceName -Verbose -WaitForBoot

Also, if I wanted to add DSC to my VM using the Service Management API, I would have to do something like this:

# Fire and forget some DSC

$dscvm = Get-AzureVM -ServiceName $ServiceName -Name $VMName

Set-AzureVMDSCExtension -VM $dscvm -ConfigurationArchive "azureDSCDemo.ps1.zip" -ConfigurationName "tester" | Update-AzureVM

This has drastically changed with Azure Resource Manager, which introduces us to a new world with a lot of more opportunities (someone would also say more complexity).

Where we are going

In order to show you where we are heading with this, I would like to point you to my GitHub repo where you can find some learning examples on how this looks like by using Azure Resource Manager templates – but also the new AzureRM PowerShell module

ARM Template with a single-button deployment + PowerShell cmdlet for deployment


PowerShell script using the new AzureRM Module to create IaaS environment with DSC


Here’s the example using PowerShell:

# Connect to your Azure subscription

Add-AzureRmAccount -Credential (get-credential)

# Add some variables that you will use as you move forward

# Global

$RGname = "KNRGTest01"
$Location = "west europe"

# Storage

$StorageName = "Knstor5050"
$StorageType = "Standard_LRS"

# Network

$vnicName = "vmvNic"
$Subnet1Name = "Subnet1"
$vNetName = "KNVnet01"
$vNetAddressPrefix = "192.168.0.0/16"
$vNetSubnetAddressPrefix = "192.168.0.0/24"

# Compute

$VMName = "KNVM01"
$ComputerName = $VMName
$VMSize = "Standard_A2"
$OSDiskName = $VMName + "osDisk"

# Create a new Azure Resource Grou

$RG = New-AzureRmResourceGroup -Name $RGname -Location $location -Verbose

# Create Storage

$StorageAccount = New-AzureRmStorageAccount -ResourceGroupName $RGname -Name knstor5050 -Type $StorageType -Location $Location

# Create Network

$PIP = New-AzureRmPublicIpAddress -Name $vnicName -ResourceGroupName $RGname -Location $Location -AllocationMethod Dynamic
$SubnetConfig = New-AzureRmVirtualNetworkSubnetConfig -Name $Subnet1Name -AddressPrefix $vNetSubnetAddressPrefix
$vNET = New-AzureRmVirtualNetwork -Name $vNetName -ResourceGroupName $RGname -Location $Location -AddressPrefix $vNetAddressPrefix -Subnet $SubnetConfig
$Interface = New-AzureRmNetworkInterface -Name $vnicName -ResourceGroupName $RGname -Location $Location -SubnetId $vnet.Subnets[0].Id -PublicIpAddressId $pip.Id

# Create Compute

# Setup local VM object

$Credential = Get-Credential
$VirtualMachine = New-AzureRmVMConfig -VMName $VMName -VMSize $VMSize
$VirtualMachine = Set-AzureRmVMOperatingSystem -VM $VirtualMachine -Windows -ComputerName $ComputerName -Credential $credential -ProvisionVMAgent -EnableAutoUpdate
$VirtualMachine = Set-AzureRmVMSourceImage -VM $VirtualMachine -PublisherName MicrosoftWindowsServer -Offer WindowsServer -Skus 2012-R2-Datacenter -Version "latest"
$VirtualMachine = Add-AzureRmVMNetworkInterface -VM $VirtualMachine -Id $interface.Id
$OSDiskUri = $StorageAccount.PrimaryEndpoints.Blob.ToString() + "vhds/" + $OSDiskName + ".vhd"
$VirtualMachine = Set-AzureRmVMOSDisk -VM $VirtualMachine -Name $OSDiskName -VhdUri $OSDiskUri -CreateOption fromImage

# Deploy the VM in Azure

New-AzureRmVM -ResourceGroupName $RGname -Location $Location -VM $VirtualMachine

# Publish DSC config to your newly created storage account

Publish-AzureRmVMDscConfiguration -ResourceGroupName $RGname -ConfigurationPath .\webdsc.ps1 -StorageAccountName knstor5050

# Add DSC Extension with config to the newly created VM

Set-AzureRmVMDscExtension -ResourceGroupName $RGname -VMName $virtualmachine.Name -ArchiveBlobName webdsc.ps1.zip -ArchiveStorageAccountName knstor5050 -ConfigurationName webdsc -Version 2.7 -Location $location

# Good night

Please have a look at these examples, and I encourage you to explore the new opportunities with the AzureRM module.

Happy ARMing!

Friday, October 2, 2015

Azure Resource Manager - Linking Templates

This summer, we wrote a whitepaper named «Cloud Consistency with Azure Resource Manager» that you can download from here: https://gallery.technet.microsoft.com/Cloud-Consistency-with-0b79b775

This whitepaper will soon be updated with new content, more examples and guidance around best practices for template authoring.

In the meantime I’ve been writing some templates that can be used by you to learn how you can link templates to have a nested deployment.

The basic example is available on GitHub - https://github.com/krnese/AzureDeploy/tree/master/Basic



You can explore all templates, but in essence I’m doing the following:

·         Have a dedicated template for storage that takes some input parameters and can be used separately
·         Have a dedicated template for virtual network that takes some input parameters and can be used separately
·         Have a master template that also contains compute, vNic and publicIP resource types that links to the storage and vnet templates

Again, this is a very easy example and I will provide you with a more advanced example in a couple of days where we split this up even further and are able to have a much more flexible and dynamic deployment scenario around IaaS/PaaS.

Pay attention to the resource section in the azuredeploy.json document, where we are using the API version “2015-01-01” and the resource type “Microsoft.Resources/deployments”.
Here I am linking to a public Uri for the template (hosted on my github) and specify the parameters I’d like to use in my configuration.



You can hit the “Deploy to Azure” link in order to explore the json structure in Azure and do an actual deployment.



If you want do deploy it through PowerShell, you can also see that the “Microsoft.Resources/deployments” resource types are being used.


Happy authoring – and see you next time.


Sunday, September 27, 2015

Explaining Windows Server Containers - Part Three

In the last blog post, I talked about the architecture of container images and how to use them in a similar way like our kids are using Lego.

Today, I want to shift focus a bit and talk more about management of container life-cycle using Docker in the Windows Server Technical Preview 3.

If you have any challenges or problems in your IT business today and ask me for advice, I would most likely point you to something that adds more abstraction.
Abstraction is key, and is how we have solved big and common challenges so far in this industry.
When we covered the architecture of containers in part 1, we compared it with server virtualization.
Both technologies are solving the same challenges. However, they are doing it at different abstraction layer.

With cloud computing we have had the IaaS service model for a long time already, helping organization to speed up their processes and development by leveraging this service model either in a private cloud, public cloud or both – in a hybrid cloud.
However, being able to spin up new virtual machines isn’t necessarily the answer to all the problems in the world.
Sure it makes you more agile and let you utilize your resources far better compared to physical machines, but it is still a machine. A machine requires management at the OS level, such as patching, backup, configuration and more.  Since you also have access at the OS level you might end up in a situation where you have to take actions that involves networking as well.

This is very often where it get complex for organizations with a lot of developers.
They need to focus, learn and adopt new skillsets, just to be capable of doing testing of their applications.

Wouldn’t it be nice if they didn’t have to care about this jungle of complexity at all, knowing nothing about the environment they will be shipping software into?
Given the fact that there’s different peoples involved when it comes to software development and managing the environment of the software, the challenges grows together with the organization itself and scale becomes a problem.

This is where containers comes to the rescue – or do they?

Containers has a good approach since all applications within a container look the same on the outside from the host environment perspective.
We can now wrap our software together within a container and ship the container image to a shared repository and don’t deal with any of the complexity that a managed OS normally require from us.

I have seen this in action, and here’s an example that normally trigger people’s interest:

1)      A developer create something new – or simply commit some changes to their version control system (GitHub, VSO etc).
2)      A new image (Docker in this case) is built with the application.
3)      The new Docker image goes through the entire testing and approval process.
4)      The image is committed to a shared repo.
5)      The new Docker image is deployed into production.

This seems like a well-known story we all have heard in the IaaS world, right?
Please note that no infrastructure was touched from the developer perspective during these steps.

This was just one example of real world organizations are using containers today, and I will cover more good use cases as we move forward in this blog series.
It is important that we’re honest and admit that new technologies that gives us more and more capabilities, features and possibilities, will at the same time introduce us for some new challenges as well.

With containers, we can easily end up in a scenario where the situation can remind us a bit about the movie called “Inception” ( https://en.wikipedia.org/wiki/Inception ). It might be hard to know exactly where you are when you are working - and have access to all the different abstraction layers.

In Technical Preview 3 of Windows Server 2016, Windows Server containers can be managed both with PowerShell and Docker.

What exactly is Docker?

Docker has been around for years and ensures automated deployments into containers by providing an additional layer of abstraction and automation of OS virtualization on Linux, MAC OS and Windows.
Just as with Windows Server containers, Docker provides resource isolation by using namespaces to allow independent containers to run within a single Linux instance, instead of having the overhead of running and maintaining virtual machines.
Although Linux containers wasn’t something new, it had been around for years already, Docker made those Linux containers become available for the general IT guy by simplifying the tooling and workflows.

In Windows Server 2016 TP3, containers can be deployed by both Docker APIs and the Docker client, and Windows Server Containers. Later, Hyper-V containers will be available too.
They important thing to note is that Linux containers will (always) require Linux APIs from the host kernel itself, and Windows Server Containers will require Windows APIs from the host Windows kernel. So although you can’t run Linux on Windows or vice versa, you can manage all of these containers with the same Docker client.

So getting back to the topic here – how to do management of containers?

Since Docker was first, this blog post will focus on the management experience by using Docker in TP3.

Note: In TP3, we are not able to see nor manage the containers if they are created outside of our preferred management solution. Meaning that containers that are created with Docker can only be managed by using Docker, and containers created with PowerShell can only be managed by using PowerShell.

During my testing on TP3, I have run into many issues/bugs when testing management of containers.
In the following recipe, I would like to point out that the following has been done:

1)      I downloaded the image from Microsoft that contains the Server Core image with the running container feature in addition to Docker
2)      I joined the container host to my AD domain
3)      I enabled the server for remote management and opened some required firewall ports
4)      I learned that everything I would like to test regarding Docker, should be performed on the container host itself, logged on through RDP

Once I’ve logged into the container host, I run the following cmdlet to see my images:

Docker images

This shows two images.

Next, I run the following cmdlet:

Docker ps

This will list all the containers on the system (note that Docker is only able to see containers created by Docker).



The next thing I’d like to show off, was how to pull an image from the Docker hub and then run it from my container host. First I get an overview of all the Images that’s compatible with my system:

Docker search server

I see that Microsoft/iis seems like a good option in my case, so I run the following cmdlet to download it:

Docker pull Microsoft/iis

This will first download the image and then extract it.
In the screen shot below, you can see all the steps I have taken so far and the output. Obviously the last part didn’t work as expected and I wasn’t able to pull the image down to my TP3 container host.



So heading back to the basics then and create a new container based on an existing image.

Docker run –it –name krnesedemo windowsservercore powershell

This will:

1)      Create a new container based on the Windows Server Core image
2)      Name the container “krnesedemo”
3)      Start an interactive PowerShell session since –it was specified. Note that this is one of the reasons why you have to run this locally on the container host. The cmdlet doesn’t work remotely

This will literally take seconds, and then my new container is ready with a PowerShell prompt.
Below you can see that I am running some basic cmdlets to verify that I am actually in a container context and not in the container host.
Also note the error I get after installing the Web-Server feature. This is a known error in TP3 that you have to run some cmdlets several times in order to get the right result. Executing it the second time shows that it went as planned.



After exiting the session (exit), I will be back at the container host’s cmdline session.
I run the following cmdlet to see all the running containers:

Docker ps –a

This shows that the newly created container “krnesedemo” is running PowerShell in an interactive session, when it was started and when I exited it.



Now, I want to commit the changes I did (installed Web-Server) and create a new image with the following cmdlet:

Docker commit krnesedemo demoimage

In my environment, this cmdlet takes a few minutes to complete. I also experienced some issues when the container was running prior to executing this command. So my advice would be to run “Docker stop “container name” “ prior to committing it.

After verifying that the image has been created (see picture below), I run the following cmdlet to create a new container based on the newly created image:

Docker run –it –name demo02 demoimage powershell



We have now successfully created a new container based on our newly created image, and through the interactive session we can also verify that the Web-Server is present.



Next time I will dive more into the PowerShell experience and see how you can leverage your existing skillset to create a good management platform for your Windows Containers.



Monday, September 7, 2015

Explaining Windows Server Containers – Part Two

In Part One, I covered the concept of Containers, compared to server virtualization in a Microsoft context.

Today, I want to highlight the architecture of container images and how you can use them as building blocks to speed up deployment.

Before we start

If you have a background in Server Virtualization, you are probably very familiar with VM templates.
A VM template is a sysprep’d image that is generalized and can be deployed over and over again. It is normally configure with its required components and applications and kept up to date with the latest patches.
A VM template contains the complete operating system (and eventually its associated data disk(s)) and has been used by administrators and developers for years when they want do rapidly be able to test and deploy their applications on top of those VMs.

With Containers, this is a bit different. In the previous blog post I explained that Containers are basically what we call “OS Virtualization” and with Windows Server Containers the kernel is shared between the container host and its containers.
So, a container image is not the same as a VM image.

Container Image

Think of a container image as a snapshot/checkpoint of a running container that can be re-deployed many times, isolated in its own user mode with namespace virtualization.
Since the kernel is shared, it is no need for the container image to contain the OS partition

When you have a running container, you can either stop and discard the container once you are done with it, or you can stop and capture the state and modifications you have made by transforming it into a container image.

We have two types of container images. A Container OS image is the first layer in potentially many image layers that make up a container. This image contains the OS environment and is also immutable – which means it cannot be modified.
A container image is stored in its local repository so that you can re-use the images as many times you’d like on the container host. It is also possible to store the images in a remote repository, making them available for multiple container hosts.

Let us see how the image creation process works with Windows Server Containers

Working with Container Images

In the current release, Windows Server Containers can be managed by Docker client and PowerShell.
This blog post will focus on the PowerShell experience and show which cmdlets you need to run in order to build images, just as easy as you would do by playing with Lego J

First, we will explore the properties of a Container Image. An Image contains a Name, Publisher and a version 



We are executing - and storing the following cmdlet in a variable: $conimage = Get-ContainerImage -Name "WinSrvCore" 


Next, we create a new container based on this image by executing - and storing the following cmdlet in a variable: $con = New-Container -Name "Demo" -ContainerImage $conimage -SwitchName "VM". 


Once the container is deployed, we will start it and invoke a command that installs the Web-Server role within this container ( Invoke-Command -ContainerId $con.ContainerId -RunAsAdministrator { Install-WindowsFeature -Name Web-Server } ). You can see that the picture below shows that the blue Lego block is now on top of the brown one (as in layers). 


As described earlier in this blog post, we can stop the running container and create an image if we want to keep the state. We are doing that by executing New-ContainerImage -Container $con -Name Web -Publisher KRN -Version 1.0


If we now executes Get-ContainerImage, we have two images. One that has only the ServerCore, and another one that has ServerCore and the Web-Server Role installed. 


We will repeat the process and create a new container based on the newly created Container Image.



In this container, we will install a web application too. The grey Lego block on top of the blue shows that this is an additional layer.


We are then stopping the running container again and creates another container image, containing the web application too.


In the local repository, we have now three different container images in a layered architecture.



Hopefully you found this useful, and I will soon be back with part three of this blog series.
Perhaps you will see more Lego as well .... :-) 

-kn