Monday, October 20, 2014

Understanding Windows Azure Pack and your service offerings

Understanding Windows Azure Pack and your service offerings

From time to time, I meet with customers (and also other system integrators) that is not fully aware of the definition of cloud computing.
I never expect people to know this to the very nasty details, but have an overview of the following:

·         Deployment models
·         Service models
·         Essential characteristics

What’s particular interesting when discussing Windows Azure Pack, is that the deployment model that’s relevant, is the private cloud. Yes, we are touching your own datacenter with these bits – the one you are in charge of.

For the service models, we are embracing Infrastructure as a Service (IaaS – using the VM Cloud Resource Provider), and Platform as a Service (PaaS – Using the Web Site Cloud Resource Provider).

The essential characteristics are also very important, as we’ll find elasticity, billing/chargeback, self-service, resource pooling and broad network access.

If you combine just self-service and IaaS, this tells us that we empower our users to deploy virtual machines on their own. Right?
So having the flexibility to provide such service, we also rely on the underlying architecture to support this. Due to scalability (elasticity), we need to ensure that these users constantly have access to the solution – no matter what device they are using (broad network access), we need to find out who is consuming what (billing/chargeback), and last but not least – be able to produce these services in an efficient way that makes it cost effective and profitable (resource pooling).

So, it starting to make sense.

There is a reason for what we are seeing and we are providing these services by abstracting the underlying resources into clouds, plans and subscriptions with the Cloud OS.

Implementing a complete IaaS solutions may bring some obstacles to the table.

Organizations tends to think that IaaS is something they have provided for years. Perhaps they have provided virtual machines, but not a complete IaaS solution.
The reason for that is that IaaS is relying on abstraction at every layer. This is not only about virtual compute (memory, CPU), but also about virtual storage and virtual networking.
This is when it gets interesting, using network virtualization.

Remember that self-service is an essential characteristic of the cloud, right?
So delivering IaaS would also mean that the user is able to do stuff with the networking aspect as well, with no interaction from the service provider/cloud administrator.
This is why Software-Defined Networking (NVGRE) is so essential to this service model, and hence we run into the following obstacles.

·         The customer (most often service provider) wants to continue to provide managed services, such as:
o   Backup (both crash consistent and app consistent)
o   Monitoring (above the operating system level, covering the application stack)

This is what they are doing today, with their infrastructure. But this also has a high cost to operate due to all the manual operations needed and involved to get the wheels moving.

Luckily, Windows Azure Pack is able to cover both scenarios, providing a consistent experience to users/tenants no matter if they are running resources in a “legacy” infrastructure, or a new modern IaaS infrastructure.

The following architecture shows that we are using two Virtual Machine Management Stamps.
Both of these are located behind the SPF endpoint – which present the capabilities, capacity and much more to the service management API in Azure Pack.

A cloud administrator then creates a Hosting Plan in the Admin Portal of Azure Pack, which is associated with the legacy cloud in the legacy VMM server. This plan is available for the users/tenants who are subscribing to managed services.

A new plan is created, associated with the IaaS cloud and the IaaS VMM server, available for the users/tenants that need IaaS, without the requirement of managed services. They are dealing with these themselves.

Hopefully this blog post gave you an overview of what’s possible to achieve using Azure Pack and combine both kind of services using a single solution.

(Want more info? – please join my TechEd session in Barcelona next week).

Tuesday, October 14, 2014

New TechEd session - Azure Site Recovery

New session at TechEd Europe

I have already announced that I will present at TechEd, Planning & Designing Management Stamps for Windows Azure Pack.

Another session is now available on the content catalog, where I will co-present together with Manoj Jain (PM for ASR) on the topic: “Microsoft Azure Site Recovery: Leveraging Azure as your Disaster Recovery Site”.

This will be fun and I really encourage you to join to see how you can extend your services, ensure business continuity and get a true Hybrid Cloud setup using the best from both clouds.

Thursday, October 9, 2014

The specific IP address is already allocated by the pool - SCVMM

Every now and then, I see fabric environments where the following have occurred:

·         They tried to create their Hyper-V Cluster in VMM
·         The process failed at some step (it can be many reasons for this, not necessary that’s VMM fault)
·         They go to one of the hosts and create the cluster from there
·         They refresh the nodes in VMM and the cluster appear

Now, that is quite common actually, and this works great.
One of the reasons why VMM is complaining a lot more than Failover Cluster Manager, is because VMM has high expectations regarding networking, storage etc.

So what happens when you create the cluster outside of VMM, and at the same time, is so rude and steal an IP address from the IP Pool in VMM?

You will see the following in the job view on a regular basis:

Frustrating. So imagine you have added Hyper-V Replica Broker to that cluster as well, stealing another IP from the pool in VMM? Then this can be noisy.


First thing first, find out what IP address VMM is referring to.


Find the right name in your environment. I will use “MGMT IP Pool Copenhagen” as I know this is a Hyper-V Cluster in that site.

Next, put that in a variable like this:

See which addresses you have registered:

Once you have detected the IP, it is time to remove it.

Get-SCIPAddress –IPAddress “” | Revoke-SCIPAddress

The only thing left, is to reserve this IP address in the VMM IP Pool so that VMM will ignore it in the future.
Once this is done, perform a refresh of the cluster object in VMM to verify that it is green and happy.

Sunday, October 5, 2014

Scratching the surface of Networking in vNext

The technical previews of both Windows Server and System Center is now available for download.
What’s really interesting to see, is that we are making huge progress when it comes to core infrastructure components such as compute (Hyper-V, Failover Clustering), storage and networking.

What I would like to talk a bit about in this blog post, is the new things in networking in the context of cloud computing.

Network Controller

As you already know, in vCurrent (Windows Server 2012 R2 and System Center 2012 R2), Virtual Machine Manager act as the network controller for your cloud infrastructure. The reasons for this have been obvious so far, but has also lead to some challenges regarding high availability, scalability and extensibility.
In the technical preview, we have a new role in Windows Server, “Network Controller”.

This is a highly available and scalable server role that provides the point of automation (REST API) that allows you to configure, monitor and troubleshoot the following aspects of a datacenter stamp or cluster:

·         Virtual networks
·         Network services
·         Physical networks
·         Network topology
·         IP Address Management

A management application – such as VMM vNext can manage the controller to perform configuration, monitoring, programming and troubleshooting on the network infrastructure under its control.
In addition, the network controller can expose infrastructure to network aware applications such as Lync and Skype.

GRE Tunneling in Windows Server

Working a lot with cloud computing (private and service provider clouds), we have now and then ran into challenges for very specific scenarios where the service providers want to provide their tenants with hybrid connectivity into the service provider infrastructure.

A typical example is that you have a tenant running VMs on NVGRE, but the same tenant also wants access to some shared services in the service provider fabric.
The workaround for this have never been pretty, but due to GRE tunneling in Windows Server, we have many new features that can leverage the lightweight tunneling protocol of GRE.

GRE tunnels are useful in many scenarios, such as:

·         High speed connectivity
This enables a scalable way to provide high speed connectivity from the tenant on premise network to their virtual network located in the service provider cloud network. A tenant connects via MPLS where a GRE tunnel is established between the hosting service provider’s edge router and the multitenant gateway to the tenant’s virtual network

·         Integration with VLAN based isolation
You can now integrate VLAN based isolation with NVGRE. A physical network on the service provider network contains a load balancer using VLAN-based isolation. A multitenant gateway establishes GRE tunnels between the load balancer on the physical network and the multitenant gateway on the virtual network.

·         Access from a tenant virtual networks to tenant physical networks
Finally, we can provide access from a tenant virtual network to tenant physical networks located in the service provider fabrics. A GRE tunnel endpoint is established on the multitenant gateway, the other GRE tunnel endpoint is established on a third-party device on the physical network. Layer-3 traffic is routed between the VMs in the virtual network and the third-party device on the physical network

No matter if you are an enterprise or a service provider, you will have plenty of new scenarios made available in the next release that will make you more flexible, agile and dynamic than ever before.
For hybrid connectivity – which is the essence of hybrid cloud, it is time to start investigate on how to make this work for you, your organization and customers.

Monday, September 29, 2014

Deploying Web Site Clouds for Windows Azure Pack

On this blog, you have mainly found (useful?) information around Hyper-V, VMM and Azure Pack when it comes to the private cloud area.
However, Azure Pack gives us a lot more than “just” the VM Cloud Resource Provider, that is the holy grail of Infrastructure as a Service offering in your private cloud.

I have attempted to cover the broader aspect of Azure Pack on this blog, like walking through the different portals, APIs and much more, especially related to the backend VM Cloud services.

Recently, I wrote about “Deploying Service Bus for Windows Azure Pack” - that shows the consistency related to PaaS between Microsoft Azure in the public cloud, and Azure Pack in the private cloud.

This blog post – which is focusing on the Web Site Cloud does also falls under the PaaS umbrella in Azure Pack.

Without going into the repeating story around “one consistent platform” and all of that, I assume you know this already, but as an introduction to Web Site Clouds in WAP, we can say the following:

The Web Site Clouds enables an on-premises, high-density, multi-tenant web hosting service for service providers and enterprise IT. The web sites provides an experience similar to Microsoft Azure Web Sites. It is scalable, shared, and secure web hosting platform that supports both template web applications and a broad range of programming languages like ASP.NET, PHP and Node.js.
In addition to a web site service, it includes a self-service management portal (tenant portal), uses both SQL and MySQL database servers, integrates with popular source control systems, and offers a customizable web application gallery of popular open source web applications.

Web Site Cloud

The Web Site Cloud consists of at least 6 server roles.

·         Controller – Provisions and manage the other web site roles. This is the first role you install and run the Web Site Cloud setup on
·         Management Server – This server exposes a REST endpoint that handles management traffic to the WAP Web Site Management API, and you connect the service management portal to this endpoint
·         Front End – Accepts web requests from clients, routes requests to web workers, and return web worker worker responses to clients. Front End servers are responsible for load balancing and SSL termination
·         Web Worker – These are web servers that process client web requests. Web workers are either shared or reserved to provide differentiated levels of service to customers. Reserved workers are categorized into small, medium and large sizes.
·         File Server – Provides file services for hosting web site content. The file server houses all of the application files for every web site that runds on the web site cloud.
·         Publisher – Provides content publishing to the web site farm for FTP clients, Visual Studio and WebMatrix through the Web Deploy and FTP protocols

And as everything else, a SQL server is required for the runtime database.
The roles are separated from, and in addition to, the servers that form the (express or distributed) installation of the service management API (Portals and APIs).

Before you start to install the Web Site Cloud, you must deploy and prepare some VM’s.

Obviously everyone is using VMM today, so here’s a short script that will go ahead and create 6 VMs to be used for the Web Site Cloud in your private cloud infrastructure:

$VMNames =@(1..5)
$Template = Get-SCVMTemplate -VMMServer vmm01 -ID "1ee6ba94-b5fc-49f1-8364-a3d1b2da3f40" | where {$_.Name -eq "GEN2 Template"}
Foreach ($VM in $VMNames)
$VM = "webroleblog"+$VM

New-SCVirtualMachine -Name $vm -VMTemplate $template -VMHost "hv03" -Path "c:\clusterstorage\csv01" -HighlyAvailable $true -RunAsynchronously


Once this is done, you can start by logging into the controller VM where you will install the Web Site Cloud to deploy and configure the other core roles for this resource provider.

Note: Although it is not a requirement I have found officially, I have experienced some issues if .NET 3.5 is not enabled within the OS’s before we start on this process. Through the Web Site Cloud installer, we will download, install and enable many web server features on each guest through an agent. This one seems to fail randomly if .NET 3.5 is left out)

On the controller, download and install Web Platform Installer version 5.0

Once this is done, start Web Platform installer and search for Web Site.
Add “Windows Azure Pack: Websites V2 Update 3” and install it on your server.

Post the installation, you will be prompted by the well-known configuration page of Azure Pack that let you connect to the SQL server, create the database, file server settings and a lot more.

The following screen shots will give you an understanding of what needs to be configured.

After the setup, we logon to our admin portal to add and configure the rest of the Web Site Cloud resource provider.

Click on Web Site Cloud and connect to your newly created environment.
Select a display name, connect to the management server (https://nameofmanagementVM), and type the credentials you specified during the setup that has access to the REST endpoint.

After the web site cloud is added, you can drill into the configuration and click on ‘Roles’.
This is where you will add the additional and required web site roles you need in order to deliver web sites to your customers.

We have already the following in place:

-          Controller
-          Management
-          File server

What we need to add now, is:

-          Publisher
-          Front End
-          Worker

You simply type the DNS or the IP address of the specific server(s) you want to add, and then Azure Pack will deploy and configure these roles for you, through the controller.

Note: The Web Site Cloud has a default domain that you specify during the setup. In our example, we are using  - which mean that every website that gets deployed, will have a suffix of
In order for this to work, you must also create some host records for the following roles:

·         Web Deploy DNS:
·         FTP Deploy DNS:
·         Front End: *

Also note that you are able to perform lifecycle management of your web site roles directly from the management portal:

Once the web site cloud is configured with the default settings, you can go ahead and add the web site cloud to an existing Hosting Plan.

You can configure the Web Site offering by clicking on the service within the plan, and edit the existing values. We will leave this alone for now, and head over to the tenant portal to see the interaction.

Now we have the Web Sites available in the tenant portal, since this tenant is subscribing to a hosting plan that contains this offering.
If we click on new, we can choose between quick create, custom create and from gallery.

Once the web site is deployed (regardless of the options listed), we can perform several operations post the deployment from the tenant portal.

From a developer perspective, it is very interesting to see that you can download the publish profile, so that you can leverage TFS to deploy your applications. This is a solid value add-on, in addition to the already exposed tenant public API (for more info, see: )

What I like the most about the web site cloud is the grade of integration with the other PaaS offerings in Azure Pack, just as you would find in Microsoft Azure.
Depending on the solution and application you create, you can link with SQL, MySQL and Service Bus which is basically everything you need in order to fulfill the PaaS solution you are working on.

Hopefully this gave you an overview of the Web Site Cloud in WAP.

Wednesday, September 24, 2014

Providing Disaster Recovery for Windows Azure Pack tenants

Providing Disaster Recovery for Windows Azure Pack tenants

In today’s datacenters where service providers are using Hyper-V, System Center and Windows Azure Pack, we will find a wide diversity of technologies and many moving parts that together will deliver one or more advanced sophisticated solutions.

To deliver cloud computing at scale, the most important thing is an understanding of the solutions as well as the preferred design to meet these goals.

In a nutshell, the holy grail of disaster recovery for tenants is something like this:

“A failover should occur with a minimum or none interaction from the tenants”

This one is hard to achieve to be honest, and the underlying design of the management stamp and the rest of the topology need to be designed in order for this to work.

Let us just look at some of the dependencies when a service provider is offering a complete IaaS solution to their tenants, leveraging the VM Cloud Resource Provider in Azure Pack.

·         VMM Management Stamp
o   The management stamp is the backend, the fabric controller of your VM Clouds in WAP
o   VMM manages the scale-units as well as the clouds abstracted in to WAP.
o   Networking is also a critical part of VMM once NVGRE is implemented. Since VMM act as a network controller in this context, we have in essence a boundary for the managed virtual machines
o   Virtualization Gateways is also managed by the stamp, within the single boundary
* Hyper-V Replica, managed through ASR and VMM
·         SPF
o   Service Provider Foundation exposes the IaaS capabilities in VMM to Azure Pack, and is the endpoint used by the VM Cloud Resource Provider in WAP
·         Active Directory
o   A service provider datacenter may contain several domain controllers and forests to manage their solutions, and hence a critical part of the offerings
o   ADFS for federation using multiple services can be considered as a key component
·         SQL
o   Almost everything in the Cloud OS stack on-premises has a dependency and requirement of SQL servers and databases
·         Windows Azure Pack
o   The API’s, the portals and the extensions are all there to deliver the solutions to the tenants, providing Azure similar services based on the service providers own fabric resources, such as IaaS and PaaS.
§  VM Cloud Resource Provider
·         Remote Console
·         VMM library with VM templates and VM Roles
·         Tenants
·         VMs
·         VM Roles
·         Virtual Networks
·         Usage
o   Other resource providers, such as Automation with SMA, Web Site Clouds, SQL Clouds and more, are all part of the solution too
·         Scale-units
o   Compute
o   Storage
o   Networking

In order to solve this, and address many of the challenges that you will be facing, it is really important to be aware of the structure of these components, how they scale, how they can survive and what it takes to bring them back online again.

We have been working on some designs that may fit the majority of the organizations out there, each and one of them with pros and cons.

Within the next weeks, I will publish our findings and this will give you a better understanding of what is possible and what may be the best solution to ensure business continuity for both the service provider and the tenants.

Wednesday, September 10, 2014

How Azure Pack is using Service Provider Foundation

How Azure Pack is using Service Provider Foundation

A while ago, I wrote several posts about the different APIs in Azure Pack.
As you may be aware of, Azure Pack consists of what we often refer to as “Service Management API”.
The API is similar to the one we will (not literally) find in Microsoft Azure, where the portal interacts with the APIs, that again aggregate all the wide diversity of resource providers available for us to consume.

A short summary

The Azure Pack Management Portal offers a familiar, self-service interface that every subscriber (tenant) uses to provision and manage services such as the web site offerings and the virtual machine with its virtual network capabilities.
We have portals for the admin (service provider) and the tenants.

Underlying the Management Portal is an OData Rest application programming interface (API) known as the Service Management API.
This API provides access to the underlying services and enables automation and replacement of the existing management portal.

Some of my API posts:

API summary:

Administrator API
REST APIs that are only available to Service Management for administrators. Default this Admin API is using port 30004, so the URI requests should reflect that.

Tenant API
REST APIs that are available for both administrators and tenants. Default the tenant API is using port 30005.

Public tenant API
Public REST APIs that support end-user subscription management for services presented by the service management API. Default the port is set to 30006.

Let us get back on track

When we are working with the VM Cloud Resource Provider in WAP, we are touching many many APIs on our journey, and one of the important ones (well, all of them are important for this to work) is the Service Provider Foundation (SPF).

SPF is provided with System Center 2012 R2 – Orchestrator (no, you don’t have to install Orchestrator, but the SPF setup is located in the Orchestrator setup/media).
SPF exposes an extensible OData web service that interacts with VMM. This enables service providers to design and implement multi-tenant self-service portals that integrate IaaS capabilities available in System Center 2012 R2 and Windows Server 2012 R2 – Hyper-V.

SPF contains several web services that has two locations to set credentials. On the server that has the SPF installed we use the application domain pool in IIS and the respective group in Computer Management. These groups (SPF_Admin, SPF_VMM, SPF_Usage and SPF_Provider) must contain a local credential (not a domain credential) that is also member of the Administrators group on the SPF server.

The SPF_VMM user must be added as an administrator to VMM in order to invoke actions from the WAP portal.

The Service Provider Foundation Web Services:

The admin web service is used to create and manage tenants, user roles, servers (like Remote Console), stamps (VMM), and other administrative objects.

The VMM web service invokes the VMM server to perform requested operations.
Examples of operations could be:

-          Creating virtual machines
-          Creating virtual networks
-          Creating user role definitions
-          Create cloud services and other fabric

Communication is bidirectional, so that actions triggered by a portal that’s using SPF (like WAP) as well as actions happening directly in VMM will be reflected on both sides.

An example:

You do something in VMM that affect one or more tenants, like adding a new VM to the tenant’s subscription. This will pop up in the tenant portal of WAP.

Another example is when a tenant makes changes to a virtual network in the portal, the jobs are triggered in VMM, aggregated by SPF and shows immediately.

Usage Web Service

SPF has also a Usage Web Service that can only be used by WAP, and uses data from Operations Manager’s data warehouse, which is integrated with VMM in order to collect information of the virtual machine metrics. You must use the spfcmdlets to register SCOM with SPF.

Provider Web Service

Resource providers for delivering infrastructure as a service (IaaS) uses this web service that provides a Microsoft ASP.NET web API. This one uses also the VMM and Admin web services but is not an Open Data (OData) service.

Registering SPF endpoint with Windows Azure Pack

As an administrator, you log on to the management portal and register the Service Provider Foundation endpoint. This will register a connection between the Service Management API and SPF.
Since SPF provides a programmatic interface to the stamps (VMM management servers), it enables service providers and enterprises to design and implement multi-tenant self-service portals that leverage IaaS capabilities provided by System Center and Windows Server.

After you have registered the SPF endpoint with the Service Management API:

·         All stamps that you have created directly in SPF will be listed in the management portal for administrators

·         All clouds created within the VMM stamp(s) will appear in the management portal for administrators

·         You can register stamps directly using the management portal for administrators

·         You can remove/change the association between stamp and service provider foundation