Novell Cool Solutions

Low Cost NetWare Backup to Disk Using FreeNAS – Part II



By:

May 7, 2010 5:13 pm

Reads:19,484

Comments:0

Score:Unrated

Print/PDF

In an article a few months ago I demonstrated how to use the built in RSYNC features of FreeNAS to provide a low cost backup to disk solution for NetWare. While investigating adding several features to the backup scripts and procedures in that article, I came across rdiff-backup at http://rdiff-backup.nongnu.org/ and was intrigued by the possibilities of smaller “diff” type backups (specifically for my GroupWise server).

Using FreeNAS and a few free/open source tools you can build a very low cost Netware backup-to-disc solution using rdiff-backup that has many of the features of high priced backup solutions, and open the possibilities to doing much more.

With the price of Terabyte drives in the sub $100.00 range, and 2 Terabyte drives becoming available, disc backup is the only way to go, and FreeNAS is perfect for this task.

This rather long article is divided into two parts, a “Basic” section and an “Advanced” section.

The Basic section will cover the setup of a basic backup solution including:
–How to setup NFS on Netware 6.5sp8
–Create a mount point on FreeNAS for the NFS share
–Run a rdiff-backup of the Netware volume(s).

The Advanced section will cover the setup of a more advanced solution including:
–How to add MySQL, PHP, and phpMyAdmin to FreeNAS
–Setting up automated rdiff-backups with Rdiff-Backup-Web
–Installing rdiffweb to browse backups to restore files and folders.

The first thing you will need to do is gather some tools.

I will be using a Windows workstation to perform administration tasks and will be using the following programs:

Get the latest version of FreeNAS http://www.freenas.org/
(As of this writing “0.7.1.5127” is the latest “Stable” version, and is the version I use in this document.)

I will be using the following setup.
Netware server IP = 10.1.1.8
FreeNAS server IP = 10.1.1.21

THE BASIC SECTION
Step 1: PREPARE THE NETWARE SERVER

Set up NFS on your NetWare server as per: https://www.novell.com/documentation/oes2/stor_nss_lx_nw/?page=/documentation/oes2/stor_nss_lx_nw/data/bswonku.html

From iMgr, select File Protocols, and then NFS(NetWare)
Start NFS (if it isn’t already)

Create the Export(s) and allow only specific hosts. You will need to add the IP address of your FreeNAS box, and probably your administrative workstation as well.

**Important**
Add an entry for FreeNAS to the Netware servers “hosts” file. The host file is in the SYS:\etc\ directory. A simple entry like:

     10.1.1.21<tab> freenas

added to the end of the hosts file is all you need.

That’s it, you NetWare server should be ready.

Step 2: INSTALL FREENAS

Install FreeNAS as per: https://developer.novell.com/wiki/index.php/HOWTO:_Install_FreeNAS

The newer versions of FreeNAS have a different looking Web GUI than the above “how to” but the functions and menus are similar and fairly intuitive.

**Important **
You MUST do a “full install to hard drive”. An “embedded” install will not work as a backup server as outlined in these procedures.

*TIP: if you are completely unfamiliar with FreeNAS, take some time to run through an install or two and spend some time getting to know the product a bit. It’s not hard to use, however, a little familiarly will go a long way. Pay special attention to the FAQ, and make sure you understand the differences between the “embedded” install and the “full” install.

I recommend making the following changes/settings to the FreeNAS installation.

During FreeNAS install change the partition/slice size from 128 (default) to at least 1024. This is the size of the partition available to the FreeNAS system.

set LAN IP from the console.
(in my case it’s 10.1.1.21/16 )

From web GUI, make sure the following sections are set and/or filled out correctly.

SYSTEM:
–general –> time, hostname, DNS
(time should be from the same time source as your NetWare server if possible)

DISKS:
–add disks
–add mount points
–Format as necessary
(For this document I am only using 1 disk, and have created a single mount point called “DATA” on the 2nd slice. My “real” backup servers have more disks, and do not use the FreeNAS installation disk as storage. I am only using this setup as an example).

SERVICES:
–cifs/smb –> enable CIFS/SMB and setup shares
(Change authentication, if desired….probably a good idea or your backups will be open to anyone that wants to see them. For this document I only created 1 share, and named it “DATA”).
–ssh –> enable, and allow “superuser (root) directly”.

ACCESS:
–Add a group called “mysql”
–Add a user called “mysql”
(make the “mysql user” a member of the “mysql group”)

DIAGNOSTICS:
–Ping an internet host to ensure conductivity

Reboot and make sure all setting have stayed.

Use Putty to access freenas shell:
login as root
(roots default password is “freenas”, you will probably want to change this at some time in the near future. By default he root username will be “root” the password will be whatever the “admin” users password is. ie: by default the admin user is “admin” with a password of “freenas”, if you do not change the password the root user will be “root” with a password of “freenas”. If you change the “admin” password, the “root” password will change as well.)

use winscp to connect to FreeNAS file system
login as root.

Step 3: CONFIGURE FREENAS

Most of the things that need to be done to the FreeNAS box will be done form the command line via ssh (Putty).

*Note* I will preface any command line commands with the “freenas:/#” prompt, the commands themselves will be after the prompt. I will try to give a brief explanation of the command line on the next line.

Using either winscp or the command line create mount points (directories) under the /mnt/ directory of the FreeNAS box.
For ease of use name them appropriately:
freenas:/# mkdir /mnt/SYS-MNT
freenas:/# mkdir /mnt/VOL1-MNT
(Create one mount point for each volume you wish to backup. Think of a mount point as a pointer or shortcut to the actual resource.)

Mount the Netware Volumes:
freenas:/# mount 10.1.1.8:/SYS /mnt/SYS-MNT
freenas:/# mount 10.1.1.8:/VOL1 /mnt/VOL1-MNT
(From the FreeNAS shell, try to mount the Netware volumes: where “10.1.1.8” is the Netware Servers IP address, “:/SYS” describes the SYS volume “export”, and “/mnt/SYS-MNT” is the mount point created in the previous step.)

freenas:/# df
(Use the “df” command at the shell to see the mount points.)

Use winscp (or the “ls” commands at the command prompt) to browse the Netware volumes via the mount points.
Make sure they are visible, and you can browse them.

Install the rdiff-backup package:
freenas:/# setenv PACKAGESITE ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7-stable/All/
(this command changes the path that FreeNAS will look for packages in. FreeNAS will default back to the original upon reboot)

freenas:/# pkg_add -r rdiff-backup-1.2.8,1.tbz
(this installs the rdiff-backup package and other requirements (the –r). like Python)

The rdiff-backup files are installed to /usr/local/bin/

You should reboot the system at this point, and take some time to read the rdiff man pages
http://rdiff-backup.nongnu.org/

Connect with winscp and putty again.
Mount the Netware volumes again

Create directories to save your backup to.
freenas:/# mkdir /mnt/DATA/SYSBACKUP
freenas:/# mkdir /mnt/DATA/VOL1BACKUP

Step 4: TEST FREENAS BASIC RDIFF-BACKUP

Test rdiff-backup from the FreeNAS shell to ensure that it backs up data from the correct location(s) to the correct location(s)

freenas:/# rdiff-backup /mnt/SYS-MNT /mnt/DATA/SYSBACKUP
(This will mirror the sys volume as defined by the SYS-MNT mount point to the /mnt/DATA/SYSBACKUP directory.)

If everything worked correctly, rdiff-backup should be performing it’s first “mirror”.
(This could take quite some time…a lot depends on hardware and the pipe between the boxes)

I occasionally get these errors (server not responding), but they seem harmless, in that the backup finishes fine and without errors, and no files are missing and/or corrupted.

It could just be an issue between my Netware server and my FreeNAS test box…..(test network does not have the best equipment on it :-)

You can see the backup by simply browsing to \\<FreeNASIP>\DATA\SYSBACKUP with windows explorer, or logging in with winscp and browsing to the /mnt/DATA/SYSBACKUP directory.

At this point, you could just add the rdiff-backup commands to a cron job, and call it good.

You would have to manage all rdiff operations via the shell, but, it is basically working at this point.

For a complete list of options for rdiff-backup, you should read the manual pages.

THE ADVANCED SECTION
Step 1: THE PACKAGES

Now that the basic system is tested, let’s add some useful (and way cool) features.

By adding a couple of packages, you can gain a web gui, and some very nice functionality to the basic rdiff-backup package.

The two packages are:

rdiffweb http://www.rdiffweb.org/wiki/index.php?title=Main_Page
“rdiffWeb is a web interface for browsing and restoring from rdiff-backup repositories. It is written in Python and is distributed under the GPL license.”

and

rdiff-backup-web http://rdiffbackupweb.sourceforge.net/
“The main web interface of rdiff-backup-web is written in php with a mysql backend. The backups are actually run using a command line python script that can be set up as a cron job. The application was originally written to do a specific task at the company I work for, but it turns out that it might be handy to other people.”

Similar names, but different products. Each product does a specific task very well.

Rdiff-backup-web has a great system for making rdiff-backups, while rdiffweb has a much better/nicer/faster system for browsing files/folder inside the backups.

Step 2: ADD MYSQL AND PHP TO FREENAS

This article:
http://www.howtoforge.com/installing-mysql-and-phpmyadmin-on-freenas

describes how to add MySQL and PHP to FreeNAS. For older versions of FreeNAS follow it to the letter, and you will have a working install of MySQL and PHP. The article is a bit dated and misses one important step for newer versions of FreeNAS.

I’ve included all of the commands here, just for cut/paste ease of use, the only real difference is the copy and paste of the libxml2.so.5 file. This is required for newer versions of FreeNAS.

freenas:/# setenv PACKAGESITE
ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7.2-release/Latest/
(this changes the path that FreeNAS will use to look for packages back to the default environment.)

“After a successfull installation and configuration of FreeNAS, install MySQL on the box using the following steps.

Create the group and user ‘mysql’ using the FreeNAS WebGUI, it is important because FreeNAS will forget about the users created on command line after reboot.”
(you should have already done this……if you followed the earlier Web GUI directions)

“Create a directory to store MySQL data files, in my case I create a sub-directory in /mnt/DATA/db to store database files, the reason to create this folder is to utilize the space on the data partition, as the data files grow it is a good place to store your growing database files.”

freenas:/# mkdir /mnt/DATA/db
freenas:/# mkdir /mnt/DATA/db/mysql
(makes the directory for the mysql databases)

freenas:~# ln -s /mnt/DATA/db/mysql /var/db/mysql
( links to default location to the newly created directory)

freenas:~# chown -R mysql:mysql /mnt/DATA/db/mysql
(sets ownership)

freenas:~# chmod 777 /tmp
freenas:~# chmod 777 /var/tmp
(changes file system modes on the /tmp directories)

freenas:~# setenv PACKAGESITE ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7.2-release/Latest/
(makes sure the PACKAGESITE environment is correct)

freenas:~# pkg_add -r mysql50-server
(add MySQL 50 Server and any requirements)

freenas:~# /usr/local/bin/mysql_install_db
(installs the test database(s))

freenas:~# chown -R mysql /mnt/DATA/db/mysql
freenas:~# chgrp -R mysql /mnt/DATA/db/mysql
(changes ownership and group file permissions)

freenas:~# /usr/local/bin/mysqld_safe -user=mysql &
(loads mysql, mysql should now be running…..you may have to hit <return> after this command completes to get back to a prompt)

freenas:~# ps ax
(to show processes)

Near the bottom of the screen you should see:

11628  p0  S      0:00.02 /bin/sh /usr/local/bin/mysqld_safe -user=mysql
11643  p0  S      0:00.37 /usr/local/libexec/mysqld --basedir=/usr/local --data
11738  p0  R+     0:00.00 ps ax

If you do, you’re good-to-go. MySQL loaded correctly.

Set the MySQL “root” user password:
freenas:~# /usr/local/bin/mysqladmin -u root password root
freenas:~# /usr/local/bin/mysqladmin -u root -h freenas.local password root
(Setting the mysql “root” users password to “root” is a very bad idea. Don’t ever do it, use a stronger/real password. I am only using it as an example)

To automatically start mysql after a reboot, add mysql_enable=”YES” to rc.conf. Under the FreeNAS web GUI select: System -> Advance -> rc.conf: Add (the + sign), and fill in the blanks.

REBOOT to make sure MySQL settings stick…..and that MySQL starts correctly.

After a reboot log back on with putty and enter the following:
freenas:~# ps ax
(Shows running processes)

Somewhere near the bottom you should see lines similar to:

 1467 con- I      0:00.01 /bin/sh /usr/local/bin/mysqld_safe --defaults-extra-f
 1509 con- I      0:00.70 /usr/local/libexec/mysqld --defaults-extra-file=/var/
 
 

If so, MySQL loaded automatically at start up.

Login to MySQL and make sure it’s working as expected:
freenas:~# /usr/local/bin//mysql -u root -p
(login to my sql as “root”)

mysql>use mysql;
(change to the mysql database)

mysql>show tables;
(you should see a list of tables…if so..mysql seems to be set up and running, and the test databases are loaded)

mysql>exit

MySQL is now installed.

Next we need to set up PHP. FreeNAS comes with a very basic PHP install, but it is very limited.

Read this entire section before you continue. Installing PHP with this process will break your FreeNAS web GUI, and you need to understand how to fix it.

*****IMPORTANT STEP******

Copy “/usr/local/lib/libxml2.so.5″ to your desktop with winscp before you continue

freenas:~# setenv PACKAGESITE
ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7.2-release/Latest/
(Path that FreeNAS will look for packages in)

freenas:~# pkg_add -r php5
freenas:~# pkg_add -r php5-extensions
freenas:~# pkg_add -r php5-xmlrpc
freenas:~# pkg_add -r php5-gettext
freenas:~# pkg_add -r php5-mcrypt
freenas:~# pkg_add -r php5-mysql
freenas:~# pkg_add -r php5-mbstring

(you will get a lot of warnings about: package ‘phpXXXXX’ requires ‘libiconv-1.11_1′, but ‘libiconv-1.13.1_1′ is installed….you should be able to ignore them…)

These commands install php5 and some basic packages onto the FreeNAS system, after the installation you will not be able to access FreeNAS webGUI interface because of a problem, don’t panic let’s correct it by issuing the following commands:

*****IMPORTANT STEP******
freenas:~# mv /usr/local/bin/php /usr/local/bin/php-cli
freenas:~# cp /usr/local/bin/php-cgi /usr/local/bin/php
(moves php to php-cli, and copies the contents of php-cgi to php)

*****IMPORTANT STEP******
copy “libxml2.so.5″ from your desktop to “/usr/local/lib/” with winscp.
(Overwrite any existing copies, and make sure the web GUI works)

The Web GUI should be back in business! The installation of the MySQL server and PHP has been completed.
(Another reboot at this point is a good idea….just to make sure everything is working)

At this point you’re done installing MySQL and PHP support on FreeNAS, you might want to install phpMyAdmin, just to get an idea of how you will need to install the other packages.

It’s fairly easy to install and is a handy utility to have for managing MySQL databases.

Step 3: ADD phpMyAdmin TO FREENAS

Installation of phpMyAdmin:
Create a directory to hold the applications, might as well put it on the DATA volume.

freenas:~# mkdir /mnt/DATA/apps

Download phpMyAdmin from http://sourceforge.net/projects/phpmyadmin/files/ in my case I’ve downloaded the “phpMyAdmin-3.2.5-english.7z” file.

I then used 7zip to extract the archive to my desktop, renamed the folder to just “phpMyAdmin”, and dragged the folder to the /mnt/DATA/apps directory via winscp.

Now create a link to run phpMyAdmin from the default location:
freenas:~# ln -s /mnt/DATA/apps/phpMyAdmin/ /usr/local/www/phpMyAdmin

if you look in the /usr/local/www/ directory with winscp, you should now see a link to the /mnt/DATA/apps/phpMyAdmin link.

Double click on the “phpMyAdmin” link in /usr/local/www/ (with winscp) and it should take you to the /mnt/DATA/apps/phpMyAdmin folder.

If it did…you’re good-to-go.

Point a browser at http://<freenasIP>/phpMyAdmin/index.php
Login as username : root
password: root

You should be in phpMyAdmin. If you are, the phpMyAdmin has been installed correctly.

Learn more about phpMyAdmin at http://www.phpmyadmin.net/home_page/index.php

Step 4: ADD rdiff-backup-web TO FREENAS

Download the 0.06 version from the sourceforge homepage at:
http://sourceforge.net/projects/rdiffbackupweb/files/

Use 7zip to extract the archive to your desktop, and rename the folder (if desired).

Copy the entire “rdiff-backup-web” folder to /mnt/DATA/apps (just like the phpMyAdmin folder.)

Read the “readme” included with the package. It’s a little confusing, but, it should give you a good idea of what some of the following commands/procedures are doing. The web site at http://rdiffbackupweb.sourceforge.net/ may also help (has a few screenshots)

Create a MySQL database for rdiffbackupweb.

freenas:/# /usr/local/bin//mysql -u root -p
(login as root to mysql)

mysql> create database rdiffbackupweb;
(create the database)

mysql> exit
(exit MySQL)

freenas:~# mysql -u root -p rdiffbackupweb < /mnt/DATA/apps/rdiff-backup-web/backup.sql
(populate the database from the backup.sql file provided with the rdiff-backup-web program)

freenas:/# /usr/local/bin//mysql -u root -p
(login as root)

mysql> grant all on rdiffbackupweb.* to backup_user identified by “password”;
(set privileges on the rdiffbackupweb database)

mysql> exit
(Exit MySQL)

Make a link (just like for phpMyAdmin):
ln -s /mnt/DATA/apps/rdiff-backup-web/ /usr/local/www/rdiff-backup-web

edit the “/mnt/DATA/apps/rdiff-backup-web/config.php” file with the built in winscp editor by double clicking on the file.

Change the following lines to read as:

########################### config.php ###########################################

$RDIFF_BACKUP = "/usr/local/bin/rdiff-backup -v2";
$BACKUP_STORE_DIR = "/usr/local/www/rdiff-backup-web/dstore/";
$MOUNT_TEMP_DIR   = "/usr/local/www/rdiff-backup-web/mnt/";
$DOWNLOAD_LOCATION = "/usr/local/www/rdiff-backup-web/tmp/";
$MOUNT = "/sbin/mount";
$UMOUNT = "/sbin/umount";
$RM = "/bin/rm";

$config_db      = "rdiffbackupweb";
$config_dbhost  = "localhost";
$config_dbuser  = "root";
$config_dbpass  = "root";
##################################################################

No other lines in the file need to be edited.

All you are doing here is telling rdiff-backup-web where to find some specific files, and where to store the backups at.

MAKE SURE you have NO EXTRA lines at the end of the config.php file. End the file at the “?>” bracket.

Then edit the “/mnt/DATA/apps/rdiff-backup-web/process_backups.py” file and make it point to your config file (the config.php file that was edited in the previous step)

Change the following lines to read as:

############################ process_backups.py ###################################
### CONFIG VARIABLES ###

##config_php = '/var/www/rdiff-backup-web/config.php'
config_php = '/mnt/DATA/apps/rdiff-backup-web/config.php'

############################################################

try running to Python script:
freenas:~# python /mnt/DATA/apps/rdiff-backup-web/process_backups.py
(You should get a “module not found” error relating to MySQLdb, install the following package)

freenas:~# setenv PACKAGESITE ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7-stable/All/
(set the download site ….again)

freenas:~# pkg_add -r py26-MySQLdb-1.2.2
(adds a required python mysql database module, it my give an error about the mysql client version….ignore it for now.)

try running the main python script again.
freenas:~# python /mnt/DATA/apps/rdiff-backup-web/process_backups.py

You will probably get errors like the following, about “libmysqlclient_r.so.15″.

##################################################
freenas:~# python /mnt/DATA/apps/rdiff-backup-web/process_backups.py
Traceback (most recent call last):
  File "/mnt/DATA/apps/rdiff-backup-web/process_backups.py", line 39, in <module>
    import MySQLdb
  File "/usr/local/lib/python2.6/site-packages/MySQLdb/__init__.py", line 19, in <module>
    import _mysql
ImportError: Shared object "libmysqlclient_r.so.15" not found, required by "_mysql.so"
freenas:~#
#################################################

To fix this…..install the mysql51 client

freenas:~# setenv PACKAGESITE
ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7.2-release/Latest/
(set the download site ….back to the default…..again)

freenas:~# pkg_add -r -f mysql51-client.tbz
(force add (the-f) the mysql 51 client)

Now try running the main python script:
freenas:~# python /mnt/DATA/apps/rdiff-backup-web/process_backups.py

you should get a good return:

################################################################
freenas:~# python /mnt/DATA/apps/rdiff-backup-web/process_backups.py
/usr/local/lib/python2.6/site-packages/MySQLdb/__init__.py:34: DeprecationWarning: the sets module is deprecated
  from sets import ImmutableSet
Config variable test [DOWNLOAD_LOCATION]: /usr/local/www/rdiff-backup-web/tmp/

ssh_backup_query returned 0 results

()
samba_backup_query returned 0 results

()
()
freenas:~#
###########################################################

If it ends without errors and drops you back at the prompt, it worked!!!!

But…the error will return….the “libmysqlclient_r.so.15″ not found error will return on rebooting the FreeNAS box.
So…fix that next.

Add search paths to the startup sequence. Use the Command Scripts section in the WebGUI
System-> Advanced -> Command Scripts->

Add
“The command to be executed.” ldconfig /usr/local/lib/mysql /usr/lib /usr/local/lib /lib
“Execute command pre or post system initialization (booting) or before system shutdown.” PostInit

Reboot the FreeNAS system and ssh in with putty.

Run the main python script again:
freenas:~# python /mnt/DATA/apps/rdiff-backup-web/process_backups.py

It should complete without any errors.
(you may see some complaining about some modules being deprecated, but it still should run fine.)

You can now browse to the rdiff-backup-web application at
http://<freenasIP>/rdiff-backup-web/

At this point you can login, (admin:admin) setup users, add backups etc….but…..you will run into an error later on if/when you try to restore a file.

To fix this issue before it happens:

edit the “/mnt/DATA/apps/rdiff-backup-web/functions/generic.php” file and delete the “$SUDO” string. (there is only one, just delete the entire string)
With FreeNAS there is no need for the SUDO command……as a matter of fact…there isn’t a SUDO command that comes standard with FreeNAS (or FreeBSD).

The “Download files from here” link should now work correctly.

From the rdiff-backup-web README:
******************************************************
5. Start using the system
The URL of the installation should be something like http://www.server.com/rdiff-backup-web/. Check that you get the login screen.
You need to set up some backups and users before you can log in. Login as username=admin,
password=admin and enter the Administration link . Here you need to set up users and passwords, addbackups, and then give your users access to individual backups
When you are adding a backup, you just need the following:
— Path to orig files: this is the server/share that you are connecting to, (eg10.1.1.8:/SYS)
— Path to backup:Just give a name. The system will create a directory in your datastore. Given the
above example, I would choose server_sys. If you use a leading slash, you can now use alternate
locations by giving an absolute path (/mnt/usb for a usb drive for example)
—include string: leave blank if you want everything. put in a list of directory names (case
sensitive!) if you only want to backup certain directories, with ;’s between them
—exclude string: similar to above. put in ** to exclude everything except what has been
specificially included.
— Time to keep for: the process_backups script clears out backups older than whatever you specify
here. Something like “2W” is 2 weeks. See the rdiff-backup docs for other options
— Connection Type: Drop down box to choose either NFS, samba, or SSH.
— Mount options: This is the samba mounting options you would normally use to mount the share
provided above. it will take the form “-t smbfs -o username=username,password=password,ro”. The ro is
not strictly necessary, but you might as well connect read only, because you won’t be writing
anything to the share. Leave this empty when using SSH.
— Active: Leave this as 1, switching to 0 will stop the backup being processed.
*******************************************************

Add users, add your backup jobs, and assign users to each job (otherwise they will not even be able to see the jobs in the “backups” link. You must add your users to “Access” the backup jobs….you even need to specifically add the “admin” user.)

When it’s all set, simply run the main Python script:

freenas:~# python /mnt/DATA/apps/rdiff-backup-web/process_backups.py
(The jobs will all be processed (in sequence). Nice!!)

Now setup a cron job to run the “process_backups.py” every day at your desired time(s). See the FreeNAS documentation on how to schedule cron jobs. (hint: Web GUI, System>Advanced)

Everytime the “process_backups.py” script is run, your rdiff backup jobs will be run in sequence.

As you can see, rdiff-backup-web is a great way to create and schedule your rdiff backups. You could also use it to back up other servers using samba and ssh connections. Sadly, it’s recovery of files leaves a lot to be desired. It is PAINFULLY slow at displaying the directory tree (for large trees) and feels slow and clunky.

To make file recovery faster and easier, and to add some notification features install the next product “rdiffweb”.

Step 5: ADD rdiffweb TO FREENAS

rdiffweb Version 0.5.1 is available at:
http://www.rdiffweb.org/wiki/index.php?title=Main_Page

download the rdiffweb package from
http://www.rdiffweb.org/wiki/index.php?title=Main_Page#Download
Download Stable release – 0.5.1 source

Uncompress the archive and copy it to a location on the FreeNAS box, (I used the same location that I used for “phpMyAdmin” and “rdiff-backup-web”…, you don’t have to copy it here…but, might as well be consistent.)

No need to create the link in the www directory, this product will run on it’s own port.
I renamed the folder to “rdiffWeb”.

Browsing the rdiffweb web page is probably a good idea since the “readme” file included with the package is blank. http://www.rdiffweb.org/wiki/index.php?title=Main_Page

The documentation is a bit confusing, and most of it seems to be aimed at the “newer” .60 version that uses sqlite rather than MySQL. However, there is still good information to be gained. Some of the procedures do not need to be followed with FreeNAS, so don’t think that you are missing anything by not doing every procedure as outlined on the web page.

Install a prerequsite package:
setenv PACKAGESITE ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7-stable/All/
(set the download site……)

pkg_add -r py26-cherrypy-old-2.3.0,1.tbz
(add the python cherrypy package )

run:
freenas:~# python /mnt/DATA/apps/rdiffWeb/rdiff-web-config

You should be prompted as follows:

*******************************************************
Enter 1 for file, or 2 for mysql [1]: 2 (we have mysql installed….might as well use it!)

MySQL backend chosen.

Enter the sql hostname: localhost
Enter the username: root
Enter the password: root
Enter the name of the database [rdiffweb]:

You must now add the first user account.  This user will have admin privileges, and therefore will have the ability to add/delete other users.
Enter the username: admin
Enter the password: admin

Each user has a user root directory.  This root directory acts much as the system root directory.  All backup repositories must be within this directory.
Examples: /var/backup/repos
Enter the root directory for 'admin': /mnt/DATA/apps/rdiff-backup-web/dstore
User added successfully.

Do you want me to automatically search the user's root directory for backup repositories? [yes]: yes
Repository search complete.
Configuration complete!
**********************************************************

Select the appropriate responses. For convenience, I am using the same default directory ( /mnt/DATA/apps/rdiff-backup-web/dstore) as the rdiff-backup-web program…..in a production machine I would probably change these or use soft links to point to a location with much more drive space.

Edit your /etc/rdiffweb/rdw.conf file and add the ServerName and ServerPort lines

********************/etc/rdiffweb/rdw.conf***************
# This is the name of the host that the server should bind to.
ServerName=0.0.0.0

# This is the port that the server should listen on
ServerPort=8080

UserDB=MySQL
sqlHost=localhost
sqlUsername=root
sqlPassword=root
sqlDatabase=rdiffweb
*****************************************************

To start the rdiffweb application:
freenas:/# python /mnt/DATA/apps/rdiffWeb/rdiff-web
(Ctrl+c will stop it…)

Now you can access rdiffWeb from your browser (at http://localhost:8080, if necessary replace localhost/port by the ip address/port of the server)
login:admin
password:admin

You can now browse backups, see the status of backups, and even subscribe to an RSS feed of your backup status (how cool is that!!)

******WARNING WARNING WARNING**********

DANGER ---- RDIFFWEB CAN CAUSE MAJOR PROBLEMS IF YOU ARE NOT CAREFUL!!!!!
RDIFFWEB USES THE "/TMP" DIRECTORY FOR A SOME OF IT'S ACTIVITY.
YOU NEED TO ENSURE THAT YOUR "/TMP" DIRECTORY HAS SUFFICIANT SPACE, OR RDIFFWEB CAN FILL IT UP, EFFECTIVELY FILLING THE FREENAS VOLUME, AND RENDERING THE FREENAS SYSTEM UNUSABLE!!!!!!!

Hopefully the above statement got your attention.....
Here is how to move the /tmp directory to another location with more space.

freenas:/# mkdir /mnt/DATA/tmp
(makes a directory on the DATA volume)

freenas:/# rmdir /tmp
(removes the original /tmp directory)

freenas:/# ln -s /mnt/DATA/tmp /tmp
(Creates a link named "tmp" to the "real /tmp" directory located on the DATA volume.)

freenas:/# chmod 777 /mnt/DATA/tmp
(changes the mode of the /tmp directory)

The original FreeNAS install only setup a 1Gig slice for FreeNAS....you will need a LOT of space (plus a little bit) to use the "Restore Folder" -> "Choose a backup date to restore as of:" parts of the rdiffweb application. This part of the application allows you to download directories that were backed up at a particular point in time. Unfortunately, it also requires enough space in the /tmp directory to recreate the entire backup structure for the folder you want to restore. So....if you have a large folder(s) you want to restore, you will need the appropriate space for the /tmp directory.

******END WARNING WARNING WARNING**********

To add the rdiffweb application to the startup sequence, use the Command Scripts
System-> Advanced -> Command Scripts->

Add
“The command to be executed.” python /mnt/DATA/apps/rdiffWeb/rdiff-web –background
“Execute command pre or post system initialization (booting) or before system shutdown.” PostInit

Reboot.

Access the rdiffweb application server at http://<freeNASIP>:8080
login: admin
password:admin

Handy disk space commands (just in case you ever need them):
freenas:/# du -d1 -h -x /
freenas:/# df -h

Notes, conclusion, and errata:

This solution doesn’t backup Netware file/directory rights, you will need to run a separate utility to backup the trustee rights. I recommend Novells trustee.nlm. Run trustee.nlm (from NetWares CRON) and back up the volume trustee rights an hour or two before the rdiff-backups. This way you

will always have a fairly recent trustee backup.

This solution doesn’t backup NDS. Use a replica and good NDS procedures.

This is probably not the best solution for disaster recovery, but seems to be very good for day-to-day backups.

By using rdiff-backup rather than RSYNC, you are trading space for time. Rdiff-backups take up less space than traditional RSYNC backups (with archiving of files), but it is slower and takes up more processing power.

Although written with Netware in mind, the rdiff-backps as described here would work just as well on any machine that rdiff-backup-web can connect to via SSH, Samba, or NFS.

With MySQL, PHP, and Python installed on FreeNAS, the options for installing packages could be wide open (Backula, Webmin, Amanda, BackupPC to name a few…). You could turn a FreeNAS box into a one-stop network backup device for almost every device on you network…….

0 votes, average: 0.00 out of 50 votes, average: 0.00 out of 50 votes, average: 0.00 out of 50 votes, average: 0.00 out of 50 votes, average: 0.00 out of 5 (0 votes, average: 0.00 out of 5)
You need to be a registered member to rate this post.
Loading...Loading...

Categories: Uncategorized

0

Disclaimer: This content is not supported by Novell. It was contributed by a community member and is published "as is." It seems to have worked for at least one person, and might work for you. But please be sure to test it thoroughly before using it in a production environment.

Comment

RSS