Azure Storage – Undelete Soft-delete Objects

Posted by

This post has been republished via RSS; it originally appeared at: Microsoft Tech Community - Latest Blogs - .

Overview

 

Azure Storage supports soft delete to protect your data for Blob Storage and Azure Data Lake Storage Gen2. Soft delete allows you to recover data that has been deleted within a retention period. With public documentation Manage and restore soft-deleted blobs - Azure Storage | Microsoft Learn, we can recover a particular soft deleted file via portal, code and Powershell script. The documentation provides some Powershell examples to recover all soft-deleted objects. In some scenarios, you may want to undelete with Powershell scripts to undelete many blobs with certain conditions.

 

This article introduces 2 scripts that are used to undelete blobs with condition of path prefix and deletion time for both blob storage and Azure Data Lake Storage Gen2 respectively.

 

The scripts in this article are all tested with Powershell 7.2.6, Az.Storage 4.9.0. From documentation PowerShell Gallery | Az.Storage 4.9.0  ,if the Az.Storage module is less than 4.9.0, Az.Storage preview module is required. The Az CLI scripts are all tested with CLI version 2.40.0. For the version is less than 2.39.0, the extension storage-preview is required.

 

The Powershell version could be checked with $PSVersionTable.

jasoncao_0-1664530899978.png

Az.Storage module version could be checked with Get-InstalledModule.

jasoncao_1-1664530953498.png

Az CLI version could be checked with az version.

jasoncao_2-1664530988244.png

 

Azure Data Lake Storage Gen2

 

Powershell

  1. Below script is used to restore soft-deleted items that were deleted after a specific time. The sincedate could be specified with parameters $Year, $Month, $Day, $Hour, $Minute, $Second. Please note, the date relevant parameters are using current system environment time zone. If sincedate is not set, the script will restore all soft-deleted items.
    $storageAccountName = "xxx" 
    $StorageAccountKey = "xxxx" 
    #get context  
    $ctx = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $StorageAccountKey  
    $filesystemName= "xxx"  
    $dirName="xxx"
    $MaxReturn = 100    
    $Token = $Null   
    $Year = ''
    $Month=''
    $Day=''
    $Hour='0'
    $Minute='0'
    $Second = '0'
    If($Year -ne '' -and $Month -ne '' -and $Day -ne ''){
       $sinceDate = Get-Date -Year $Year -Month $Month -Day $Day -Hour $Hour -Minute $Minute -Second $Second -AsUTC -ErrorAction Stop
    }else{
       $sinceDate = (Get-Date -AsUTC).AddDays(-366)
    }
    write-host $sinceDate
    do   
    {   
       # list the deleted blobs in the path  
       $DeletedItems= Get-AzDataLakeGen2DeletedItem -Context $ctx -FileSystem $filesystemName -Path $dirName -MaxCount $MaxReturn -ContinuationToken $Token #get all deleted items
       write-host "========================================"   
       write-host "Soft deleted items: "   
       $DeletedItems.Path
       $Token = $DeletedItems[$DeletedItems.Count-1].ContinuationToken
       $DeletedItems| Where-Object {$_.DeletedOn -ge [DateTime] $sinceDate} |Restore-AzDataLakeGen2DeletedItem #restore the items that meet the criteria
    }while ($Token -ne '') 
    ​
  2. Below script is used to restore soft-deleted items that item path is in a specific pattern. $pathexpression is used to specify the pattern. It supports wildcard expression(*,?,[]). If the pathexpression is not set, it will restore all soft delete items.

    $storageAccountName = "xxx" 
    $StorageAccountKey = "xxx" 
    #get context  
    $ctx = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $StorageAccountKey  
    $filesystemName= "xxxx"  
    $dirName="xxx"
    $pathexpression= '' #pathexpression, a wildcard expression (containing *, ?, and [ ])like 'xxx*', if not setting, recover all soft delete blobs
    $MaxReturn = 100    
    $Token = $Null    
    if($pathexpression -eq ''){
       $pathexpression = '*'
    }
    write-host $pathexpression
    do   
    {   
       # list the deleted blobs in the path  
       $DeletedItems= Get-AzDataLakeGen2DeletedItem -Context $ctx -FileSystem $filesystemName -Path $dirName -MaxCount $MaxReturn -ContinuationToken $Token
       write-host "========================================"   
       write-host "Soft deleted items: "   
       $DeletedItems.Path
       $Token = $DeletedItems[$DeletedItems.Count-1].ContinuationToken
       $DeletedItems| Where-Object {$_.Path -like $pathexpression} |Restore-AzDataLakeGen2DeletedItem   
    }while ($Token -ne '') 
    

 

Az CLI

  1. Below script is used to restore soft-deleted items that were deleted after a specific time. The sincedate could be specified with parameters $Year, $Month, $Day, $Hour, $Minute, $Second. Please note, the date relevant parameters are using current system environment time zone. If sincedate is not set, the script will restore all soft-deleted items.
    $storageAccountName = "xxxx" 
    $StorageAccountKey = "xxxx" 
    $filesystemName= "xxx"  
    $dirName="xxx"
    $c=0
    $Year = ''
    $Month=''
    $Day=''
    $Hour='0'
    $Minute='0'
    $Second = '0'
    If($Year -ne '' -and $Month -ne '' -and $Day -ne ''){
       $sinceDate = Get-Date -Year $Year -Month $Month -Day $Day -Hour $Hour -Minute $Minute -Second $Second -AsUTC -ErrorAction Stop
    }else{
       $sinceDate = (Get-Date -AsUTC).AddDays(-366)
    }
    write-host $sinceDate
    If($dirName.Length -eq 0){
       $DeletedItems= az storage fs list-deleted-path -f $filesystemName --account-key $storageAccountKey --account-name $storageAccountName
    }else{
       $DeletedItems= az storage fs list-deleted-path -f $filesystemName --account-key $storageAccountKey --account-name $storageAccountName --path-prefix $dirName 
    }
    $DeletedItemsJson = "$DeletedItems" | ConvertFrom-Json
    write-host "========================================"   
    write-host "Soft deleted items: "  
    Foreach($item in $DeletedItemsJson) {
       if($item.deletedTime.ToUniversalTime() -ge [DateTime] $sinceDate) {
          $item.name + " - " +  $item.deletionId + " - " + $item.deletedTime.ToUniversalTime()
          az storage fs undelete-path -f $filesystemName --deleted-path-name $item.name --deletion-id $item.deletionId --account-key $storageAccountKey --account-name $storageAccountName
          $c++
       }
    } 
    write-host "A total of " $c " blobs were recovered"​
  2. Below script is used to restore soft-deleted items that item path is in a specific pattern. $pathexpression is used to specify the pattern. It supports wildcard expression(*,?,[]). If the pathexpression is not set, it will restore all soft delete items.
    $storageAccountName = "xxxx" 
    $StorageAccountKey = "xxxx" 
    $filesystemName= "xxxx"  
    $dirName="xxxx"
    $pathexpression= '' #pathexpression, a wildcard expression (containing *, ?, and [ ])like 'xxx*', if not setting, recover all soft delete blobs
    $c=0
    if($pathexpression -eq ''){
       $pathexpression = '*'
    }
    If($dirName.Length -eq 0){
       $DeletedItems= az storage fs list-deleted-path -f $filesystemName --account-key $storageAccountKey --account-name $storageAccountName
    }else{
       $DeletedItems= az storage fs list-deleted-path -f $filesystemName --account-key $storageAccountKey --account-name $storageAccountName --path-prefix $dirName 
    }
    $DeletedItemsJson = "$DeletedItems" | ConvertFrom-Json
    write-host "========================================"   
    write-host "Soft deleted items: "  
    Foreach($item in $DeletedItemsJson) {
       if($item.name -like $pathexpression) {
          $item.name + " - " +  $item.deletionId + " - " + $item.deletedTime.ToUniversalTime()
          az storage fs undelete-path -f $filesystemName --deleted-path-name $item.name --deletion-id $item.deletionId --account-key $storageAccountKey --account-name $storageAccountName
          $c++
       }
    } 
    write-host "A total of " $c " blobs were soft deleted and recovered" ​

 

Blob Storage (Below scripts are only work for no versioning enabled accounts, if you would like to restore soft-deleted items with versioning enabled, please check related documentation)

 

  1. Below script is used to restore soft-deleted items that were deleted after a specific time. The sincedate could be specified with parameters $Year, $Month, $Day, $Hour, $Minute, $Second. Please note, the date relevant parameters are using current system environment time zone. If sincedate is not set, the script will restore all soft-deleted items.
    $storageAccountName = "xxx" 
    $StorageAccountKey = "xxx" 
    $storageContainer = "xxx"
    #get context
    $ctx = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $StorageAccountKey  
    $MaxReturn = 100    
    $Token = $Null    
    $Year = ''
    $Month=''
    $Day=''
    $Hour='0'
    $Minute='0'
    $Second = '0'
    If($Year -ne '' -and $Month -ne '' -and $Day -ne ''){
        $sinceDate = Get-Date -Year $Year -Month $Month -Day $Day -Hour $Hour -Minute $Minute -Second $Second -AsUTC -ErrorAction Stop
    }else{
        $sinceDate = (Get-Date -AsUTC).AddDays(-366)
    }
    write-host $sinceDate
    do   
    {   
          # get a list of all of the blobs in the container  
        $Blobs = Get-AzStorageBlob -Container $storageContainer -Context $ctx -IncludeDeleted -MaxCount $MaxReturn  -ContinuationToken $Token 
        write-host "========================================"   
        write-host "All Blobs including soft deleted ones: "   
        $Blobs.Name   
        $DeletedBlobs = $Blobs | Where-Object { ($_.ICloudBlob.Properties.DeletedTime -ge $sinceDate) -and ($_.IsDeleted -eq $true)}
        if($DeletedBlobs.Count -gt 0){
            $DeletedBlobs.BlobBaseClient.Undelete()
        }
        $Token = $Blobs[$Blobs.Count -1].ContinuationToken;   
    }while ($Token -ne $Null) 
    ​
  2. Below script is used to restore soft-deleted items that item path is in a specific pattern. $pathexpression is used to specify the pattern. It supports wildcard expression(*,?,[]). If the pathexpression is not set, it will restore all soft delete items.
    $storageAccountName = "xxx" 
    $StorageAccountKey = "xxxx" 
    $storageContainer = "xxx"
    #get context
    $ctx = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $StorageAccountKey  
    $nameexpression= '' #nameexpression, a wildcard expression (containing *, ?, and [ ])like 'xxx*', if not setting, recover all soft delete blobs
    $MaxReturn = 100    
    $Token = $Null    
    if($nameexpression -eq ''){
        $nameexpression = '*'
    }
    do   
    {   
          # get a list of all of the blobs in the container  
        $Blobs = Get-AzStorageBlob -Container $storageContainer -Context $ctx -IncludeDeleted -MaxCount $MaxReturn  -ContinuationToken $Token 
        write-host "========================================"   
        write-host "All Blobs including soft deleted ones: "   
        $Blobs.Name   
        $DeletedBlobs = $Blobs | Where-Object { ($_.Name -Like $nameexpression) -and ($_.IsDeleted -eq $true)}
        if($DeletedBlobs.Count -gt 0){
            $DeletedBlobs.BlobBaseClient.Undelete()
        }
        $Token = $Blobs[$Blobs.Count -1].ContinuationToken;   
    }while ($Token -ne $Null) 
    ​

 

Related documentation

 

https://learn.microsoft.com/en-us/azure/storage/blobs/soft-delete-blob-overview

Azure Storage Explorer soft delete guide | Microsoft Learn

https://learn.microsoft.com/en-us/azure/storage/blobs/blob-powershell#restore-a-deleted-blob

Manage and restore soft-deleted blobs - Azure Storage | Microsoft Learn

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.