Backup Laravel site to Amazon S3 via laravel-backup package

Intro - Search for ideal package

Many things can go wrong in ordinary life of one web application, it can get hacked, database migrations or seeders could be wrongly configured and they could delete certain content, you can commit code with bugs, moderator can delete by mistake etc, I imagine you can think of a dozen of scenarios. 

In my web projects I usually write small bash scripts which are in charge of creating database dump files and then they are uploaded to third party services like Dropbox. Lately I became a bit lazy and started browsing through Laravel packages for backups.

First I bumped into backup-manager and it's Laravel implementation and then I noticed that latest commit was almost 9 months ago, so immediately I new it's lacking support for Laravel 5.3 and new notification mechanism.

After this I discovered post from Freek Van der Herten Taking care of backups with Laravel from September, which seemed like a good sign. Quick screening of the text discovered that Spatie's laravel-backup package is built for 5.3 and it posses notification support, search no more, that's it!

Idea

I want to setup daily cron which will backup the database and certain files to Amazon S3 bucket. After successful backup Laravel will send notification to my Slack channel. My plan is to protect my AWS account via dedicated IAM user and Inline Policy which will allow only limited actions on backup bucket.

Cause my KEY and SECRET for AWS user are stored in my web application there is a risk that someone could gain access to them and delete my backups from S3 bucket. This backup package needs Delete action on IAM user so it can clean old backups according to rules in the config file, so I plan to move clean action from Laravel application and create dedicated Lambda function which will handle that.

By doing this I can remove Delete action from IAM Policy associated with this AWS user, thus prevent anyone from deleting backup files if they gain access to AWS credentials.

It seems like ideal solution, so lets start.

Installing laravel-backup

I will pull this package from Packagist via Composer:

composer require spatie/laravel-backup

After this I need to register the service provider in app.php:

Spatie\Backup\BackupServiceProvider::class,

The only thing missing now is configuration file, I'll publish it with:

php artisan vendor:publish --provider="Spatie\Backup\BackupServiceProvider"

Configuration file defines a number of options, like application name, list of files for backup, database connections for backup, backup disks (local, s3, dropbox...), notification channels and cleanup rules.

I noticed one elegant thing with this package, in S3 bucket it creates separate folders for each stage of my app. Meaning when I execute backup command from my dev environment package created http---codingo.dev folder, and from live system it created https---codingo.me, pretty nice. It takes application name from APP_URL env variable, anyway I was surprised cause I am used to see packages without this separation. Imagine what could happen if you backup your dev database and forgot about it and then try to restore live database from that file, complete mess.

Now I'll check is backup command present in commands output with:

php artisan list

And output:

artisan list command output

Artisan list command output.

Everything works fine. I will now just comment-out section in config file where laravel-backup includes files:

//config/laravel-backup.php
// ...

'files' => [

                /*
                 * The list of directories and files that will be included in the backup.
                 */
                'include' => [
                    //base_path(),
                ],

                /*
                 * These directories and files will be excluded from the backup.
                 */
                'exclude' => [
                    base_path('vendor'),
                    base_path('node_modules'),
                ],

                /*
                 * Determines if symlinks should be followed.
                 */
                'followLinks' => false,
            ],
// ...

I am doing this cause I need only the database, I am using other mechanism for deploying the code. There you have an element which holds folders which you want to exclude, it is important not to upload vendor or node_modules folders, cause they can have 100MB+. 

S3 configuration

Laravel uses flysystem package for managing system disks. To be able to use S3 as disk I will need to pull dedicated S3 driver:

composer require league/flysystem-aws-s3-v3

 Flysystem configuration file holds details about my S3 bucket, I want to extract those details to my .env file, so I will modify s3 element to use env variables:

//config/flysystems.php
// ...

        's3' => [
            'driver' => 's3',
            'key'    => env('S3_KEY'),
            'secret' => env('S3_SECRET'),
            'region' => env('S3_REGION'),
            'bucket' => env('S3_BUCKET'),
        ],

// ...

Now I will create these keys in env file:

//.env
// ...

S3_BUCKET=backups
S3_KEY=
S3_SECRET=
S3_REGION=us-east-1

By default laravel-backup uses local disk, so I need to switch it with s3. I'll need to modify 2 elements in laravel-backup config file, one is backup.destination.disks and the other is monitorBackups.disks on both places I'll insert s3.

Creating S3 Bucket

You'll need to sign-in into Management Console and go to S3 section. There is a button Create Bucket click on it, it will open a modal with input field for bucket name, mine is backup-tuts and Us Standard region is fine. 

Create Bucket modal

Create Bucket modal.

That's all concerning bucket creation, now we'll need to create AWS user which will be able to take actions against this new bucket.

IAM User

As you guessed now we'll visit Identity and Access Management section of AWS Console and click on Users tab in left sidebar. It will open a list with all IAM users and at the top there will be a button Create New Users, lets click on it.

Now you see a form for new usernames, mine is laravel-backup-user.

Create IAM user page

Create IAM user form.

When you click Create AWS will create new IAM user and provide you with option to Download and see keys. Just click on Show User Security Credentials and you'll be able to see credentials.

Show IAM user Credentials

Show IAM user Credentials.

After you close that page, you will again see list of all IAM users, find your new user and click on its User Name. Go to Permissions tab and expand Inline Policies accordion. There you'll see link for creating new policy, click on it.

User Summary - Policies accordion

User Summary - Policies accordion

Now system will display Set Permissions page, there you will see Policy Generator option already checked out click Select.

For AWS Service choose Amazon S3 and for actions select all, this is only temporary in next section we will fine-tune policies for security reasons.

Amazon Resource Name is unique name of AWS resource for our new bucket it is arn:aws:s3:::backup-tuts , just change this bucket name to the name you used.

Edit Permissions Form

Edit Permissions Form

You can click on Add Statement button and after that click Next Step. Now Console will display policy in JSON format, first Validate policy and if it's valid click Apply. For reference policy looks like:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1478287316000",
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": [
                "arn:aws:s3:::backup-tuts"
            ]
        }
    ]
}

Policy is now generated and attached to IAM user for backups.

Testing backup to S3

You need to enter IAM user keys into .env file and execute:

php artisan backup:run
Backup command success

Backup command success

It seems backup command completed successfully, I'll go to my bucket to see were there any files uploaded:

Backup created new folder in the bucket

Backup created new folder in the bucket

As you can see backup command created new folder in my S3 bucket related to development site. I mentioned earlier that laravel-backup package is taking application name from .env file.

Backup zip file

Backup zip file

In the new folder system uploaded zip backup file. So everything is working fine.

Scheduling Cron Job

You should schedule at least backup creation command, there is also clean command available. In my case I am using both commands at least until I create Lambda function for cleaning backups.

//App\Console\Kernel
// ...

    /**
     * Define the application's command schedule.
     *
     * @param  \Illuminate\Console\Scheduling\Schedule  $schedule
     * @return void
     */
    protected function schedule(Schedule $schedule)
    {
        $schedule->command('backup:clean')->daily()->at('03:00');
        $schedule->command('backup:run')->daily()->at('03:30');
    }

In next section I will talk about connecting laravel-backup to Slack for notifications.

Trending

Newsletter

Subscribe to our newsletter for good news, sent out every month.

Tags