Filesystem backup utility for Azure Blob Storage

This Python-based tool uses tar to perform backups of a file system and upload the archive to Azure Blob Storage. In most cases, you should first evaluate Azure Backup which is a fully-managed backup service for virtual machines running both in Azure and on-premises. If for any reason Azure Backup is not applicable in your environment, this tool might be useful to you.

Current status: beta

Features and limitations

This tool requires Python 2.7 and should work on any Linux distribution. It was tested on SUSE Enterprise Linux 12.

Features:

• Perform full backups
• Can be automated via cron
• Configure file system to backup and files to exclude
• Storage account to use is determined by an instance tag
• Storage container to use is determined by configuration file
• Backup schedule is defined using instance tags

Requirements:

• Azure Virtual Machine: this tool is designed to run on an Azure virtual machine. It uses the Azure environment, like tags in the instance metadata, to determine how it should run. This allows you to control most of the behaviour of the tool through environment changes, instead of having to log on to the virtual machine to edit a configuration file.

• Azure Blob Storage: this tool is designed to upload the backup archives to an Azure Blob Storage account. It uses managed identities for authentication, which means that storage credentials don't need to be saved on the machine.

Installation using pip

Dependency to psutil

The tool has a dependency to the psutils Python library. This library only provides a source distribution, which means that it must be compiled on the target machine. If the target machine does not have development tools installed, the installation of the psutil library will fail. Please install the development tools before installing using the pip method.

Using virtualenv

The less intrusive way to install and test the tool is to use virtualenv. Here is how you can create and activate a virtual environment:

pip install --user virtualenv
~/.local/bin/virtualenv --python=python2.7 ~/azfilebak
source ~/azfilebak/bin/activate

Install the tool

You can install directly from a release URL:

pip install https://github.com/Azure/azfilebackup/releases/download/v1.0-alpha1/azfilebak-1.0b1.tar.gz

Or if you downloaded the distribution archive locally:

pip install azfilebak-1.0b1.tar.gz

Configuration

Storage account and managed identity

You will need an Azure Storage account to store the archives, and you will need to configure a system-assigned managed identity to authorize the virtual machine to access the storage account. You can follow this tutorial to set this up: Use a Linux VM system-assigned managed identity to access Azure Storage.

Tags

Tags are used to govern the backup schedule and configure certain parameters. This allows controlling the backup process without having to modify the configuration file on the machine. The files test-set-vm-tags-arm.json and test-set-vm-tags.sh how tags should be defined.

Usage

If the backup configuration file is not in the default location (/usr/sap/backup/backup.conf), use -c to specify an alternate location:

sudo azfilebak -c ./test_backup.conf -x

Run a full backup now, disregarding the schedule constraints:

sudo azfilebak --full --force

List existing backups:

sudo azfilebak --list

Download a backup archive (it will be saved to /tmp by default):

azfilebak --restore fs_test-backup_full_20181122_094011.tar.gz

Stream the contents of the archive to a local tar command, so you can dreictly list or extract files:

azfilebak --restore fs_test-backup_full_20181122_094011.tar.gz --stream | tar tvzf -

Development

The tool requires Python 2.7.

Install requirements:

pip install -r requirements.txt

Run tests:

make test

Tests

Some of the tests need to access a real Azure storage account. The name of the account to use can be changed in the file sample_instance_metadata.json:

"tags": "StorageAccount:sahec99az1backup0001;

The storage account key can be specified via an environment variable:

export STORAGE_KEY='xxx'

The tool requires the GNU version of the tar command. If you are developing and running the tests in a non-GNU environment (e.g. macOS), you can install GNU tar. On macOS for example, you can use Homebrew: brew install gnu-tar.

Packaging

The release directory contains instructions and a Dockerfile that are used to generate an RPM file suitabled for deployment on a system without impact on existing Python installations.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.