Restore and recovery

How to restore a backup file

If you have a backup file created by the DirectAdmin backup system, it can be restored from the DirectAdmin panel via the Admin Backup/Transfer page. The screen just initiates a restore process, which is next executed by the dataskq tool.

If you want to monitor a restore progress, you can now do so via the GUI in addition to via the command line. To do so via the command line, just call dataskq manually as usual:

/usr/local/directadmin/dataskq d80

Restore settings

Many global settings affecting backups/restores can be found on the Admin Backup/Transfer page and settings affecting a reseller's users' backups/restores can be found on the Reseller Manage User Backups page. Look for the "BACKUP/RESTORE SETTINGS" link on the top of each of these pages to access these settings. More details.

Restore Progress Monitoring

To monitor a restore process via the GUIopen in new window, initiate the restore and then navigate back to the Admin Backup/Transfer page. You should see an "In Progress" tab or a table showing the restore's progress.

You will see details regarding the restore as well as a progress bar displaying the restore process's percentage of completion.

This type of process tracking for restores is similar to the existing backup process trackeropen in new window.

The Level 1 percent bar is based on the User-data only backup settings, meaning there will be 3 phases per user (if applicable):

cpmove-to-DA conversion
decryption
user restore

Each phase will go from 0 to 100%. The larger Level 0 percent bar is as it is for backups, either 1 or 2 per User, depending on if there are FTP downloads involved.

So 10 Users will have 10% steps, unless downloading from FTP, then will be 5% steps.

The "Active Backups" table on CMD_ADMIN_BACKUP will have a new column after "Start Time" showing "Type", where each row will either be type "Backup" or "Restore".

The "Backup ID" column will always be blank for restores, since restores don't have crons.

Debugging the Restore Process

If you have small restores but wish to see the progress and need to slow it down, use debug level 2019 in the dataskq by issuing the restore through the GUI and quickly running:

./dataskq d2019

where each tracking "step" adds a 5 second sleep, allowing time to debug what the output is.

Restoring other User accounts with a Personal License (max 1 User)

The "Personal License" type only allows 1 account on the system.

By default, DirectAdmin installs the "admin" account.

If you're trying to import a domain from some other User account (e.g., from another server), let's say your backup is called:

user.admin.fred.tar.gz

where "fred" is the name of the User you're trying to merge into Admin, the simplest method is to copy the file to:

/home/admin/backups/user.admin.fred.tar.gz

and restore it from there.

Ensure the "backups" folder, and gz files are both chown'd to admin.

Restoring backups from the User Level is preferred in this case, for a few reasons:

It only restores things Users can control and will not affect admin's user.conf (we do not usually want "admin" to have any limits, which would be set if restored via other Levels).
The backups in this path always go to the restoring User (admin).

The alternative is to rename file user.admin.fred.tar.gz to be user.admin.admin.tar.gz, and restore it from ~/admin_backups, keeping in mind that the user.conf settings from the backup would be imposed upon the "admin" account, which is not usually desirable.

Restoring a system backup file

The sysbk program does not come with an automated restore tool (If you want automation, use the Reseller Level -> Manage User Backups, or Admin Level -> Admin Level Backup).

System Backups are stored into custom paths that the admin can specify.

An example: /backup/19-30-04

where the numbers represent the date of that backup under the /backup folder.

If (for example) you want to restore your /etc/virtual directory, you'd type:

cd /etc
tar xvzfp /backup/07-30-04/custom/etc/virtual.tar.gz

A list of files/directories used by DirectAdmin: https://directadmin.com/paths.htmlopen in new window

Be careful with the /etc/passwd, /etc/shadow, /etc/group, etc., files. Test out your login/logout of ssh with another window before logging out, after replacing those files, to ensure your login still works.

Admin restore: allow reseller creator override

For calls to the Admin Level Backup/Restore: CMD_ADMIN_BACKUP

When a restore is issued, an additional value: reseller_override=fred

can be passed. "fred" must be an Admin or a Reseller.

If reseller_override=fred is set, and fred is a Reseller, should any Resellers or Admins be selected for restore, "fred" will not be used as the creator, and will revert to the default (usually "admin") if reseller_override=admin is set (any Admin), then this will be applied regardless of the restore type.

This is useful for cpanel restores where the desired creator cannot be determined from the filename.

If the account being restored already exists, this setting will have no effect.

How to completely rebuild your /etc/virtual directory

Make a backup of the current /etc/virtual directory.

cd /etc
cp -Rp virtual virtual.backup

Ensure it exists:

mkdir -p /etc/virtual
chmod 755 /etc/virtual
chown mail:mail /etc/virtual

Create the /root/fix.sh script with code:

#!/bin/sh

echo -n '' > /etc/virtual/domains
echo -n '' > /etc/virtual/domainowners

mkdir -p /etc/virtual/`hostname`
chown mail:mail /etc/virtual/`hostname`
chmod 711 /etc/virtual/`hostname`
echo `hostname` >> /etc/virtual/domains

for u in `ls /usr/local/directadmin/data/users`; do
{
       for d in `cat /usr/local/directadmin/data/users/$u/domains.list`; do
       {
               echo "$d: $u" >> /etc/virtual/domainowners
               echo "$d" >> /etc/virtual/domains

               DMN=/etc/virtual/$d

               mkdir -p $DMN
               chmod 711 $DMN
               chown mail:mail $DMN

               touch $DMN/aliases
               if [ ! -s $DMN/aliases ]; then
                       echo "$u: $u" > $DMN/aliases
               fi
               touch $DMN/autoresponder.conf
               touch $DMN/filter
               touch $DMN/filter.conf
               touch $DMN/passwd
               touch $DMN/quota
               touch $DMN/vacation.conf
               chown mail:mail $DMN/*

               mkdir -p $DMN/majordomo
               chmod 751 $DMN/majordomo
               chown majordomo:daemon $DMN/majordomo

               mkdir -p $DMN/reply
               chmod 700 $DMN/reply
               chown mail:mail $DMN/reply

               for p in `cat /usr/local/directadmin/data/users/$u/domains/$d.pointers | cut -d= -f1 2>/dev/null`; do
               {
                       echo "$p: $u"  >> /etc/virtual/domainowners
                       echo "$p" >> /etc/virtual/domains
                       ln -s $d /etc/virtual/$p
               };
               done;
       }
       done;
}
done;

chown mail:mail /etc/virtual/domains
chown mail:mail /etc/virtual/domainowners
chmod 644 /etc/virtual/domainowners
chmod 644 /etc/virtual/domains

Make it executable then run it:

chmod 755 /root/fix.sh
/root/fix.sh

Create other related email files, and update your exim.conf.
Rebuild majordomo:

cd /usr/local/directadmin/custombuild
./build majordomo

How to rebuild /etc/virtual/domains file if it's missing

Create a /etc/virtual/fix_domains.sh script with the following contents:

#!/bin/sh

echo `hostname`;

for u in `ls /usr/local/directadmin/data/users`; do
{
          for d in `cat /usr/local/directadmin/data/users/$u/domains.list`; do
          {
                    echo "$d"

                    for p in `cat /usr/local/directadmin/data/users/$u/domains/$d.pointers 2>/dev/null | cut -d= -f1 2>/dev/null`; do
                    {
                              echo "$p"
                    }
                    done;
          }
          done;
}
done;

Make it executable then run and fix permissions:

cd /etc/
chmod 755 fix_domains.sh
./fix_domains.sh > domains
chmod 644 domains
chown mail:mail domains

How to rebuild /etc/virtual/domainowners file if it's missing

Create the script /etc/virtual/fix_domainowners.sh with content:

#!/bin/sh

for u in `ls /usr/local/directadmin/data/users`; do
{
          for d in `cat /usr/local/directadmin/data/users/$u/domains.list`; do
          {
                    echo "$d: $u"

                    for p in `cat /usr/local/directadmin/data/users/$u/domains/$d.pointers | cut -d= -f1 2>/dev/null`; do
                    {
                              echo "$p: $u"
                    }
                    done;
          }
          done;
}
done;

Fix permissions and run it:

cd /etc/virtual
chmod 755 fix_domainowners.sh
./fix_domainowners.sh > domainowners
chmod 644 domainowners
chown mail:mail domainowners

How to rebuild /etc/virtual/domain.com directory if it's empty

If, for whatever reason, you have your /etc/virtual/domain.com folder for User fred, but still have other valid data in /etc/virtual that you don't want to lose, then you can reset the default data with these commands (replace domain.com and fred with real data):

cd /etc/virtual/domain.com

echo "fred: fred" > aliases
touch autoresponder.conf
touch filter
touch filter.conf
touch passwd
touch quota
touch vacation.conf
echo "*: :fail:" >> aliases
chown mail:mail *

mkdir -p majordomo
chmod 751 majordomo
chown majordomo:daemon majordomo

mkdir -p reply
chmod 700 reply
chown mail:mail reply

chown mail:mail .

Then you can start the process of re-adding all User accounts through DA, which are listed here

ls -la /home/fred/imap/domain.com

Let's say that you've just transferred all of your data to a new disk e.g., http://www.directadmin.com/paths.htmlopen in new window

If the ownerships and permissions are not correct, then you can get the set_permissions.sh script to correct them for you:

cd /usr/local/directadmin/scripts
./set_permissions.sh all

Note that this script will not touch files uploaded by your users to public_html/private_html folders as it doesn't know what their permissions should be.

If you are not worried about potential permission problems and just want a very basic level of operation for all of the files your user have uploaded (e.g., if all of their files are owned by root and you need to change them), then you can run the following for a basic reset of the permissions (change domain.com to a real domain name):

./set_permissions.sh domaindir domain.com

Parallel restores: useradd /etc/passwd locking issues

If you're trying to restore many Users in parallel using multiple dataskq calls, you may run into useradd /etc/passwd locking issues.

This guide provides a script which creates a DirectAdmin lock to ensure the useradd binary is not being called twice at the same time.

Create the file (and user_create_pre directory) /usr/local/directadmin/scripts/custom/user_create_pre/lock.sh and add code:

#!/bin/bash
LOCK=/etc/passwd.da_lock
ATTEMPTS=100
set -C   # Sets > to fail if exists
while [ $ATTEMPTS -gt 0 ]; do
       ATTEMPTS=$(( ATTEMPTS-1 ))
       2>/dev/null >$LOCK
       RET=$?
       if [ "$RET" = "0" ]; then
               break;
       fi
       sleep 0.2
done

if  [ $ATTEMPTS -eq 0 ]; then
       if [ -e /etc/passwd.lock ]; then
               echo "Unable to get lock on $LOCK and /etc/passwd.lock still exists.  User $username not created"
               exit 1
       fi

       #could not get a lock, but we are not seeing the useradd lock.. proceeding anyway
       echo "Unable to get lock on $LOCK but /etc/passwd.lock does not exist. Proceeding anyway"
       rm -f $LOCK
fi

exit 0;

And make it executable:

chmod 755 /usr/local/directadmin/scripts/custom/user_create_pre/lock.sh

We will need a cleanup file to remove the lock, create /usr/local/directadmin/scripts/custom/user_create_post/lock.sh with the code below:

#!/bin/bash
LOCK=/etc/passwd.da_lock
rm -f $LOCK
exit 0;

And make it executable:

chmod 755 /usr/local/directadmin/scripts/custom/user_create_post/lock.sh

The effect will be, the /etc/passwd.da_lock will be created just before DA creates a User and removed just after the creation. It will loop 100 times (adjust as needed) to get the lock for itself. If all 100 attempts fail, if there is not /etc/passwd.lock file (created by /usr/sbin/useradd), then it will continue normally as if it did get the lock. If the passwd.lock does exist, it will fail and report in the Message System.

# Restore and recovery

# How to restore a backup file

# Restore settings

# Restore Progress Monitoring

# Debugging the Restore Process

# Restoring other User accounts with a Personal License (max 1 User)

# Restoring a system backup file

# Admin restore: allow reseller creator override

# How to completely rebuild your /etc/virtual directory

# How to rebuild /etc/virtual/domains file if it's missing

# How to rebuild /etc/virtual/domainowners file if it's missing

# How to rebuild /etc/virtual/domain.com directory if it's empty

# How to reset permissions and ownership on all DA related system files

# Parallel restores: useradd /etc/passwd locking issues