Problems with sharing data in China? Use Azure way to fix this

Table of Contents

• Introduction
• High-level Requirement
• Solution
• Some performance matrix from my environment

Introduction

The main problems while sharing data in china from rest of world is “The Great Firewall of China”

I’m sure that all of us are aware of the word so called” The Great firewall of china”

Let me put this into more technical term. Chinese government has its own network security/firewall so called “The great Firewall of China” to filter all the data traffic coming over the internet.

Well it’s good for cyber security but it causes many problems such as high latency and slowness of network speed, less privacy, limited reliability and so on.

There are many solutions available in the market to share files in China location. But I want to share my experience on using Azure way to do that.

How it works: Azure has one of the Data center in East Asia Hong-Kong.

Why we mentioned Hong-Kong? Read the below write up from Wiki

The special administrative regions (SAR) are one type of provincial-level administrative divisions of China directly under Central People's Government, which enjoys the highest degree of autonomy, and no or less interference by either Central Government or the Chinese Communist Party.

So let’s take an advantage of international law.

High-level Requirement

We had requirement where approximately 1 TB data to be moved to the China location from EU Data center (it was Sweden in my case)

Below are the high level business requirement from the customer.

1. Data should be moved to China DC location in a faster way
2. Data should travel in secure manner and encrypted way.
3. Solution should be cost effective.

My team tried to move the data from EU DC to China DC over the VPN/internet. But unfortunately it didn’t work due to the issues mentioned above. It took me about 25 hrs to move 200 gb of data and then I realized that data got corrupted.  My team tried multiple times with the same results.

Solution

Azure has datacentres in west Europe ( Netherlands ) and East Asia (Hong-Kong) and I used them for data movement.

Tools We used:

1. One Azure storage account in Azure DC in West Europe (Netherlands)
2. One Azure storage account in Azure DC in East Asia (Hong-Kong)

Azure tool(Azcopy): AzCopy is a Windows command-line utility designed for high-performance copying of data to and from Azure Storage.
This activity was done in 3 phases mentioned below.  

1. Customer had good connectivity between their EU DC and Azure DC(Netherlands) so in the first phase all data files were moved to Azure west Europe (Netherlands) using existing Express Router/ VPN connection. I used Azure storage account to store all the data in a one place.
2. Then data files were moved from Azure storage (Azure Europe) to one of the Azure storage account available in Hong-Kong region. (Microsoft has private tunnel for data to travel within Azure DC(s) worldwide so data which is being transferred from any datacentre to any datacentre will never pass-through the public internet. Hence data will be more secured and encrypted format while movement)
3. In the final phase I download the data from Azure storage (Hong-Kong) to the destination.

Some performance matrix from my environment

Around 1 TB data is moved with 100% accuracy rate and it took about 14 hrs for entire activity

EU DC File server à Azure Europe FS:  3 hrs

Azure Europe storage  à Azure HK storage FS: 7 hrs.

Azure HK FS àBranch site 4 hrs.

From 300 gb files

Hope the solution will be informative for you.

Br,

Digvijay

Windows Azure Pack: Subscription usages info through PowerShell

Introduction:

Windows Azure pack is a great tool for your Private cloud. It brings the ability to run Azure services inside your own environment. It gives ability to use subscription based access and control like Azure public cloud does.

Since WAP uses plan and subscription based concept for private cloud resource. It’s very important to keep a track of usages or each subscription.

 This Article will help you to generate a nice report about the usage of each subscription.  It’ll also give useful information’s such as co-admin details, subscription status, sync status, creation date etc.

So let’s go it into the detail and see how we do that.

 

Prereqs.

Import MgmtSvcAdmin PowerShell module. Follow this url if you don’t have module installed

(https://technet.microsoft.com/en-in/library/dn282110.aspx )

Access to the WAP admin API

Address of WAP Admin API (https://ServerName:30004)

At least 1 plan/subscription configured in WAP for trial purpose

PowerShell commends:

you can download contents from here.  

  $AdminUri = 'https://ServerName  :30004'

   $AuthSiteUri = 'https://ServerName  :30072'

   $ClientRealm = 'http://azureservices/AdminSite  '

  

  $token = Get-MgmtSvcToken -Type Windows -AuthenticationSite $AuthSiteUri -ClientRealm $ClientRealm -DisableCertificateValidation

   $allSubscription = Get-MgmtSvcSubscription -Token $token -AdminUri $AdminUri -DisableCertificateValidation

$a = "<style>"

$a = $a + "BODY{}"

$a = $a + "TABLE{border-collapse: collapse;width: 100%;}"

$a = $a + "TH{border-width: 1px;padding: 0px;border-style: solid;border-color: #ddd;padding: 10px;text-align: left;}"

$a = $a + "TD{border-width: 1px;padding: 0px;border-style: solid;border-color: #ddd;padding: 10px;text-align: left;}"

$a = $a + "table tr:nth-child(odd) {background-color: #f2f2f2;}"

$a = $a + "TABLE tr td table{width: 50%;}"

$a = $a + "TABLE tr td table th{border: 1px solid #ddd;}"

$a = $a + "TABLE tr td table td{border: 1px solid #ddd;}"

$a = $a + "</style>"

 

 

$body = "<H2>Windows Azure Pack Subscription Information</H2><table><colgroup><col/><col/><col/></colgroup><tr><th>Subscription Name</th><th>Subscription ID</th><th>Status</th><th>Created Date</th><th>Admin</th><th>Sync Status</th></tr>"

 

foreach($Subscription in $allSubscription){

    $SubUsage = Get-MgmtSvcSubscriptionUsage -SubscriptionId $Subscription.SubscriptionID -AdminUri $AdminURI -Token $token -DisableCertificateValidation

    $SubUsage.RetrievedSuccessfully

 

    $body = $body + "<tr><td>"+$Subscription.SubscriptionName+"</td><td>"+$Subscription.SubscriptionID+"</td><td>"+$Subscription.State+"</td><td>"+$Subscription.Created+"</td><td>"+$Subscription.CoAdminNames+"</td><td>"+$SubUsage.RetrievedSuccessfully+"</td></tr>"

    $body = $body + "<tr><td colspan=""6"">Usage Information<br><br><table><tr><th>Service Type</th><th>Usage</th><th>Total Limit</th><th>Unit</th></tr>"

 

    $SubUsageList =  $SubUsage.Usages

    foreach($Usage in $SubUsageList){

        $body = $body + "<tr><td>"+$Usage.DisplayName+"</td><td>"+$Usage.CurrentValue+"</td><td>"+$Usage.Limit+"</td><td>"+$Usage.UnitDisplayName+"</td></tr>"

    }

    $body = $body + "</table></td></tr>"

}

$body = $body + "</table>"

 

ConvertTo-HTML -head $a -Body $body | Out-File "$env:USerProfile\desktop\WAP-Sub-Usages.html"

  

 

Invoke-Expression "$env:USerProfile\desktop\WAP-Sub-Usages.html"










Output




I’ll now explain more into technical detail of it.

Addresses and URI of require components.

$AdminUri = 'https://AdminAPIServer  :30004'

$AuthSiteUri = 'https://WindowsAuthSiteServer  :30072'

$ClientRealm = 'http://azureservices/AdminSite  '

Note: You may have above components installed in different server in distributed environment.

You can use below command to see the address.

Get-MgmtSvcEndpoint | Format-Table -Property Name,Address

Generate a token:
The Get-MgmtSvcToken creates an identity token. Tokens are used by several of the Windows Azure Pack for Windows Server cmdlets. You can create a token and store it in a variable for use with other cmdlets. Find WAP subscription and take each for taking detailed information.

$token = Get-MgmtSvcToken -Type Windows -AuthenticationSite $AuthSiteUri -ClientRealm $ClientRealm -DisableCertificateValidation

I’m not expert in html code so I took help from my buddy "Tapan Gupta". However I am using html as an output for making it presentable. Hence you need listed codes.

You can modify it as per your requirement.

$a = "<style>"

$a = $a + "BODY{}"

$a = $a + "TABLE{border-collapse: collapse;width: 100%;}"

$a = $a + "TH{border-width: 1px;padding: 0px;border-style: solid;border-color: #ddd;padding: 10px;text-align: left;}"

$a = $a + "TD{border-width: 1px;padding: 0px;border-style: solid;border-color: #ddd;padding: 10px;text-align: left;}"

$a = $a + "table tr:nth-child(odd) {background-color: #f2f2f2;}"

$a = $a + "TABLE tr td table{width: 50%;}"

$a = $a + "TABLE tr td table th{border: 1px solid #ddd;}"

$a = $a + "TABLE tr td table td{border: 1px solid #ddd;}"

$a = $a + "</style>"

 

$body = "<H2>Subscription Information</H2><table><colgroup><col/><col/><col/></colgroup><tr><th>Subscription Name</th><th>Subscription ID</th><th>Status</th><th>Created Date</th><th>Admin</th><th>Sync Status</th></tr>"

Now it’s time for us to take each subscription one by one and have detailed information.

I’m using foreach loop to collect usages information’s.

foreach($Subscription in $allSubscription){

    $SubUsage = Get-MgmtSvcSubscriptionUsage -SubscriptionId $Subscription.SubscriptionID -AdminUri $AdminURI -Token $token -DisableCertificateValidation

    $SubAdmin = ""

    if($Subscription.CoAdminNames.Count -gt 0){

        $SubAdmin = $Subscription.CoAdminNames.Item(0)

    }

 

    $body = $body + "<tr><td>"+$Subscription.SubscriptionName+"</td><td>"+$Subscription.SubscriptionID+"</td><td>"+$Subscription.State+"</td><td>"+$Subscription.Created+"</td><td>"+$SubAdmin+"</td><td>"+$SubUsage.RetrievedSuccessfully+"</td></tr>"

    $body = $body + "<tr><td colspan=""6"">Usage Information<br><br><table><tr><th>Service Type</th><th>Usage</th><th>Total Limit</th><th>Unit</th></tr>"

 

    $SubUsageList =  $SubUsage.Usages

    foreach($Usage in $SubUsageList){

        $body = $body + "<tr><td>"+$Usage.DisplayName+"</td><td>"+$Usage.CurrentValue+"</td><td>"+$Usage.Limit+"</td><td>"+$Usage.UnitDisplayName+"</td></tr>"

    }

    $body = $body + "</table></td></tr>"

}

$body = $body + "</table>"

Final step is to convert output data and feed this informations to the html page which we created earlier.

ConvertTo-HTML -head $a -Body $body | Out-File "$env:USerProfile\desktop\WAP-Sub-Usages.html"

 Invoke-Expression "$env:USerProfile\desktop\WAP-Sub-Usages.html"

This script will list out all subscription and give the informations back in html format.


I hope this post will be informative.


Thanks

Digvijay Singh