Shulou(Shulou.com)06/03 Report--

AzCopy actually uses a lot of scenarios, which can also help us solve a lot of problems. This time, let's share how AzCopy helps us solve problems in two different scenarios.

First of all, let's take a look at what the two scenes are.

1. You need to download the blob stored on Azure Global

two。 Need to synchronize blob data between Azure Global and Azure China

Both of these are actual cases. First, let's look at the first one.

The background is that there is a 60T vhd file on Azure Global's storage, which is publicly shared by third-party vendors, and anyone can download it directly without authentication. What we need to do is to migrate this vhd file placed in Global to China, and then deploy it.

We tried a lot of ways.

1. Direct download-the speed is maintained at the kB level and is extremely unstable

two。 Use major network disks and download tools to download offline-it doesn't work at all.

3. Use indescribable tools to download science-the speed is objective at first, but it becomes zero after a period of time

4. Directly create a VM in Global Azure to download-the speed is at the MB level, and the stability is OK, but after waiting for a period of time, the download is interrupted

In the end, all of the above methods were found to be not feasible, and finally the problem was solved by using AzCopy.

First, we install azcopy, and then build a storage and container in China to store the copied data.

All these belong to the preparatory work.

Then let's take a look at how to copy the data. first of all, when both the source side and the destination side are azure blob, azcopy will have some restrictions.

You must append an SAS token to each source URL.

If you use Azure Active Directory (AD) to provide authorization credentials, you can only omit the SAS token from the target URL.

Therefore, our source side must be authenticated with SAS, and in our case, the source side is directly open, so we don't need SAS.

On the target side, we need to use Azure AD for authentication.

We have talked about Azure AD authentication many times. How to do it? azcopy v10 allows users to login, so this means that we can log in with our azure account, while azure users can now be directly granted read and write permissions to blob. There are mainly the following role

Storage Blob Data Reader

Storage Blob Data Owner

Storage Blob Data Contributor

The concept is a little similar to conventional role. We need to write files here, so we certainly can't use reader. We directly assign the permissions of an owner.

Then we start the migration, first logging in to the target environment

Azcopy login-- aad-endpoint https://login.partner.microsoftonline.cn

Visit the red url in the browser, and then type code. The whole process is the same as the powershell under cli and linux.

After logging in, we try the contents in the container under list, and we can see that the corresponding contents can be listed.

Azcopy list https://mxy.blob.core.chinacloudapi.cn/test

Then we directly use the copy command to copy the vhd file in Global to container

Azcopy copy https://xxxxx.blob.core.windows.net/release/xxxxx.vhd https://.blob.core.chinacloudapi.cn/sms/sms-connector.vhd

The whole process is completed in half an hour, and the average speed can reach 400-500Mbps.

The speed is so fast because AzCopy uses server-to-server api, so replicates data directly between storage servers. These replication operations do not use the computer's network bandwidth.

Let's look at the second scenario:

Synchronizing data between blob, because this kind of synchronization is probably not just an one-time synchronization, it is likely to be run regularly afterwards, so we use SAS for authentication on both the source side and the destination side.

With regard to SAS, we will not dwell on it because of the limited space here. We will directly generate a storage-level SAS.

Then use the azcopy sync command to synchronize

Azcopy sync 'https://mxxxc.blob.core.windows.net/$web?sv=2019-02-02&ss=bfqt&srt=sco&sp=rwdlacuxxxxxx&st=2020-02-10T03:42:25Z&spr=https,http&sigxxxxxxc%3D'' https://xxx.blob.core.chinacloudapi.cn/global?sv=2019-02-02&ss=bfqt&srt=sco&xxxxx5:59Z&st=2020-02-10T04:35:59Z&spr=https,http&sigxxxxxx3D'-recursive-s2s-preserve-access-tier=false

You can see that the data has come on the target side.

Then we upload a new file on the source side

Run the command again, and you can see that only one file is synchronized this time

Through this method, the data of gloabl can be synchronized to China quickly, and the security can be guaranteed.

These are the two cases used by azcopy

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.