Novell Cool Solutions

Low Cost NetWare Backup to Disk Using FreeNAS



By:

September 29, 2009 4:50 pm

Reads:14,011

Comments:1

Score:5

Print/PDF

Two or three years ago, I started using the Novell RSYNC port from http://forge.novell.com/modules/xfmod/project/?rsync

Unfortunately, I never had the best experiences with RSYNC for NetWare, it always seemed just a little unstable and has left me wanting another couple of standard RSYNC features. It also seems to be a dead project in that no new versions have been produced for a few years.

A fellow NetWare administrator mentioned to me that he was having great success with using NCP mounts on a SUSE server, and using RSYNC from the SUSE box.

With that in mind I went back to one of my favorite free projects: FreeNAS.

Using the built in RSYNC functions of FreeNAS you can build a very low cost NetWare backup-to-disc solution, and open the possibilities of doing much more.

With the price of Terabyte drives entering the sub $100.00 range, disc backup is the only way to go, and FreeNAS seems to be almost perfect for this task.

This rather long article is divide into two parts, a “Basic” section and an “Advanced” section.

The Basic section will cover the setup of a basic backup solution including:

  • How to setup CIFS share on NetWare
  • Create a mount point on FreeNAS for the CIFS share
  • Schedule a daily RSYNC of the NetWare volume

The Advanced section will cover the setup of a more advanced solution including:

  • How to setup a CIFS share on NetWare
  • Create a mount point on FreeNAS for the CIFS share
  • Install Perl on FreeNAS
  • Use Perl to create a backup script
  • Schedule a daily RSYNC of the NetWare volume including daily file archives of deleted and changed files

The first thing you will need to do is gather some tools.

I will be using a Windows workstation to preform administration tasks and will be using the following programs:

Get the latest version of FreeNAS: http://www.freenas.org/

(As of this writing “.0.69.2″ is the latest “Stable” version, I am using the “nightly build” version 0.7RC2.4835 for this document…it has some iSCSI fixes I needed for other projects)

Install FreeNAS on your target machine as per:
https://developer.novell.com/wiki/index.php/HOWTO:_Install_FreeNAS

The newer versions of FreeNAS have a different looking Web GUI than the above “how to” but the functions and menus are similar and fairly intuitive.

*TIP: if you are completely unfamiliar with FreeNAS, take some time to run through an install or two and spend some time getting to know the product a bit. It’s not hard to use, however, a little familiarly will go a long way. Pay special attention to the FAQ, and make sure you understand the differences between the “embedded” install and the “full” install.

I will be using the following setup.

NetWare server IP = 10.1.171.200
FreeNAS server IP = 10.1.171.202

The Basic Solution

Step 1: Setup of the NetWare server

Two things need to be done on the NetWare server:

First, enable CIFS.
Second, setup a user that can access the NetWare volume with a simple password.

The documentation for your specific version of NetWare should cover the setup of CIFS (or native file access). The page located at
http://www.digitalissues.co.uk/html/os/netware/push-netware-smb.html
also gives directions for several different versions of NetWare.

I am using NetWare 6.5sp8, so I followed the instructions in section 9 of the above page:

*************************************************************
Type the NetWare IP address in a browser
Select Management Services
Select iManager (Provide admin account password and tree)
Select File Protocols
Select NetWare CIFS
Beside Server, select the magnifying lens and browse for your server object
Select Properties to adjust the following to taste

CIFS Virtual Server Name (VMSERVER_W)
Wins IP address (Wins if needed)
Comment (comments if needed)
Oplocks (enabled)
SMB Signature (Mandatory)
Authentication Mode (eDirectory)
Work Group / Domain Name: (WORKGROUP)
Primary Domain Controller (0.0.0.0)

There is no need to create any “shares” for the RSYNC backup to work.

Next create a user (or an alias) in the same context as your servers volume object. This user is the user we will be authenticating to the NetWare server as. This user should have at least Read and FileScan rights to the volume (or to the directories that need to be backed up). Alternatively, you could modify the CIFS config file and add the context to search for the user(s)….personally, I think just using an alias is easier.

After you create the user, you may have to reset the simple password even if you set it correctly during the user creation. For some reason I seem to have little luck setting the simple password at user creation.

If cifs is setup correctly, you should now be able to see the server in Network Neighborhood (under the correct workgroup). If you try and access the server from a workstation using the server IP (\\10.1.171.200), you should get a login/password prompt. Once you login, you should see all of the volumes available.

*TIP: Don’t forget to give your backup user appropriate access to the drive(s) and/or directories. If the user can’t Read the files, it definitely won’t be able to back them up. If your server name (VMSERVER_W in my case) doesn’t show up correctly in Network Neighborhood you can reload cifs with the “cifsstop” command to stop cifs, and then “cifs” command to reload it.

Make sure to read the last section at http://www.digitalissues.co.uk/html/os/netware/push-netware-smb.html regarding logging in from workstations that have the Novell Client installed.

Step 2 : FreeNAS Setup

Just a few things need to be done after the initial setup of the FreeNAS box. The Setup and Users Guide covers how to do all of the following and can be downloaded in PDF format from the FreeNAS web site.

First the appropriate drives need to be added, formatted and mounted under the “Disks” menu. This is covered in the Users Guide.

Next is to enable SSH and allow root access with the “Permit Root Login” selection.

The root username will be “root” the password will be whatever the “admin” users password is.
ie: by default the admin user is “admin” with a password of “freenas”, if you do not change the password the root user will be “root” with a password of “freenas”.

If you change the “admin” password, the “root” password will change as well.

But remember “admin” is not “root”, keep this in mind.

Enable CIFS/SMB and create the appropriate share(s). It may be a wise decision to name these shares for the server and/or volume they will be backing up.

If the CIFS/SMB share is done correctly, you should be able to browse to the share using the \\FreeNAS-IP with Windows Explorer (\\10.1.171.202 in my case).

If SSH is setup correctly you should able to attach to the server (as root) with both putty and WinSPC.

At this point the FreeNAS box should be pretty much ready.

Step 3 : Mounting the NetWare volume on the FreeNAS box.

To mount a NetWare volume on the FreeNAS box we first need to create a mount point (think of a mount point as a shortcut to the NetWare volume).

To make a mount point I prefer to use Winspc to browse to the /mnt directory then create a Directory.

You may want to adjust the permissions to your needs/taste, but I would start with 0777 until you become comfortable with the process and BSD permissions. Remember the name of your mount point. (VM-VOL1 in my case)

To mount the NetWare volume we will use the mount_smbfs command from the FreeNAS command line.

In order for this to be done without being prompted for a password we will need to create a file called “nsmb.conf”. The nsmb.conf file should contain the following:

[SERVER]
addr=10.1.171.200
[SERVER:BACKUP]
password=backup

where:

SERVER=NetWare Server CIFS share name (VMSERVER_W in my case)
addr= IP of NetWare server
BACKUP= user name created in section 1
password= Simple password of the user created in section 1

My final nsmb.conf file:

[VMSERVER_W]
addr=10.1.171.200
[VMSERVER_W:BACKUP]
password=backup

*TIP: The user name should be in all caps. I don’t know why….but it works for me.

You could also use a “~/.nsmbrc” file if you desire. (google is your friend).

The nsmb.conf file should be copied to the /etc directory of the FreeNAS server.

Once the nsmb.conf file is in place, connect to the FreeNAS box with putty, and type the following command (modify for your environment).

mount_smbfs -N -I 10.1.171.200 //BACKUP@VMSERVER_W/VOL1 /mnt/VM-VOL1

where:

-N specifies that we are using a config file (nsmb.conf ) look for the password in this file
-I host IP address (NetWare server IP address)
//user@server/source volume or CIFS share
/mnt/xxxxx the mount point we created in the previous step (VM-VOL1 in my case)

This command is case sensitive, type carefully.

If everything is set correctly, you should be back to a prompt without being asked for a password.

If you are prompted for a password, examine your nsmb.conf file for errors.

Type “df” at the prompt and you should be able to see your mount point!

You can “cd” to the /mnt/VM-VOL1 directory and use the “ls” command to view files on your NetWare server volume.

You can also browse to the /mnt/VM-VOL1 directory in WinSPC to browse the NetWare volume.

Step 4 : RSYNC the NetWare volume to the FreeNAS box.

If you are unfamiliar with RSYNC, read the RSYNC documentation!

If you are familiar with RSYNC, read the RSYNC documentation!

http://www.samba.org/ftp/rsync/rsync.html

Without delving into the syntax of RSYNC to much, here is the basic command structure:

rsync -OPTIONS /SOURCE /DESTINATION --SECONDARY OPTIONS

Notice only a space separates the options, source, destination, and secondary options.

Here is the RSYNC line that I will use to backup the NetWare volume:

rsync -arzv /mnt/VM-VOL1/ /mnt/MP1/VOL1/ --progress 

where:

rsync = the rsync command
-arzv = archive,recursive,compress,verbose
/mnt/VM-VOL1/ = source (the NetWare mount point)
/mnt/MP1/VOL1 = destination (the FreeNAS server mount point MP1 in a sub-directory VOL1)
–progress = secondary option to show progress on screen

*TIP: Since I am RSYNC’ing from volume to volume (a “local” RSYNC, and not using the RSYNC protocol) some of these options are probably not necessary, but I use them out of habit.
Pay special attention to the “/” following the source path.
Create a test directory for testing RSYNC commands before you try them on your production volumes, some RSYNC commands can be quite dangerous (the –delete option to be specific).

To test your RSYNC command, attach to the FreeNAS box with Putty, and type the command. If all goes well and your syntax is correct you should see RSYNC copy the files.

You can now browse to the FreeNAS server, and see your files.

At this point you will not be able to delete these files from Windows, this is OK, it’s working as expected.

We are backing up the files as the “root” user, and only the “root” user has full access to the files.

Using WinSPC you can see the rights to the directory and change them if you want to delete the directory.

Once your RSYNC command is working as expected, connect to the FreeNAS web GUI and select System > Advanced > CRON. Enable CRON if it isn’t already enabled.

Click the “+” to add a new CRON job, fill in the desired settings. Select the “Add” button, and then the “Apply Changes” button.

Wait.

Check that your job ran as expected.

*TIP: The “Run now” command will probably work, but if it doesn’t, don’t panic. Remember, you are running the job as “root”, but you are logged into the GUI as “admin”, it’s OK, the CRON job will run just fine.

That’s it! You now have a daily (or whatever) running RSYNC of your NetWare volume.

The first RSYNC of a volume may take quite some time, after that, RSYNC’s will be quite fast.

The Fine Print (and other notes):

A wise admin would add the mount_smbfs command to the etc/fstab. There are plenty of documents explaining this procedure much better than I can, so I won’t attempt to. This will automatically mount the NetWare volume every time the FreeNAS box is rebooted.

This solution doesn’t backup NetWare file/directory rights, you will need to run a separate utility to backup the trustee rights. I recommend Novells trustee.nlm. Run trustee.nlm (from NetWares CRON) and back up the volume trustee rights an hour or two before the RSYNC. This way you will always have a fairly recent trustee backup.

This solution doesn’t backup NDS. Use a replica and good NDS procedures.

In a single server environment, I suggest looking at eMBox.

Other options (from the Novell forums) could include:

—————————————————————————-

  1. -Variant *ONE*- (for vandal-admin only !!):
    -unload ds
    -load ‘cc.nlm’ (http://www.cnw.hu/ccnlm)
    -copy all files from _NETWARE directory in other directory ( for example NDS_COPY . Files of eDIR will be stored in it)
    -stop cc.nlm
    -load ds again

    After crach of eDIR you will delete all files in _NETWARE directory and
    will copy all files from NDS_COPY directory to _NETWARE directory

  2. -Variant *TWO*-: (for CNE/CNA-certify admin)
    Use very simple + very cheaply programm from ‘HBWARE.COM’
    (http://www.hbware.com/content/view/40/12/)

    After crach of eDIR you will restore eDIR from copy.

  3. -Variant *THREE*-:(for VeryLazy-admin)
    Simply do image of SYS-volume by StorManager from ‘PORTLOCK’
    (http://www.portlock.com/products/stormgr/index.htm)

    After crach of eDIR you will restore eDIR ( by restore all SYS-volume) from image

—————————————————–

The “Advanced” Section

Step 1: Do everything in the “Basic” section, and understand how it works.

Step 2: Add Perl to FreeNAS

Download the Perl port from:
ftp://ftp.freebsd.org/

I am using version “perl-5.8.9_3.tbz” from:
ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7-stable/perl5/

To Install Perl, login to the web GUI and select System > Packages, click the “+” to add a package, select “browse” and browse to the perl-5.8.9_3.tbz file, and click the “Install” button. If the package installs successfully you should see the install screen scrolling by. If you have problems with the “browse” button, use Firefox.

When it’s completed, you should see the Perl package installed.

Step 3: Use Perl to create a backup script

The reason for installing perl is to give the ability to create a more complicated script for running the RSYNC command. I am using Perl to create a bash script. The bash script will run the RSYNC command. The goal of this script is to run a RSYNC with some commands that will put all deleted and modified files into their own folder labeled with the days date in YearMonthDay format. I am not a Perl expert, and the script is very simple and certainly not great coding, but it works. I have tried to comment the script as much as possible.

----------------------code----------------------------
### sets the "$today" variable in YearMonthDay format
($mday, $mon, $year) = (localtime(time))[3..5];
        if ($mday < 10) { $mday = "0" . $mday; }
        $mon += 1;
        if ($mon < 10) { $mon = "0" . $mon; }
        $year += 1900;
$today = $year . $mon . $mday;
## sets up the rsync command **DANGER** be careful it contains the --delete option!!!
## make sure you know and understand the command line and options before running it.
$linept1 = 'rsync -arzv  /mnt/VM-VOL1/ /mnt/MP1/VOL1/ --delete --progress --backup --backup-dir=../';
##adds the date to the end of the rsync command line
$mystring = "$linept1$today\n";
## this is the output name and path of the bash file, edit the path and file name as necessary
$rsyncfile = '/mnt/MP1/scripts/MYrsync';
## opens MYrsync
open (CFG, ">$rsyncfile");
## copies the following bash lines to the rsyncfile file
## Several of these lines will need to be modified to work in your environment 
print CFG "#!/bin/bash\n";
print CFG "PATH=$PATH:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin\n";
print CFG "cd /\n";
print CFG "sleep 10\n";
print CFG "mount_smbfs -N -I 10.1.171.200 //BACKUP@VMSERVER_W/VOL1 /mnt/VM-VOL1\n";
print CFG "sleep 10\n";
print CFG "if df |grep -q '/mnt/VM-VOL1$\'\n";
print CFG "then\n";
print CFG "$mystring\n";
print CFG "sleep 20\n";
print CFG "umount /mnt/VM-VOL1\n";
print CFG "sleep 10\n";
print CFG "chmod -R 777 /mnt/VM-VOL1/\n";
print CFG "else\n";
print CFG "echo \"Aborted because the source disk is not mounted\"\n";
print CFG "exit -1\n";
print CFG "fi\n";
close (CFG);
## use chmod to make our file usable
chmod(0777, "/mnt/system/scripts/MYrsync");
------------------------------/code-------------------------------------------------

As a habit I create a “scripts” directory on the FreeNAS box, and save all my scripts to this directory.

In this example I am saving the bash script to “/mnt/MP1/scripts/” and naming the file “MYrsync”. I also save the Perl script in the same directory.

Here is the “MYrsync” file that the Perl script produces.

------------------------code---------------------------------------------
#!/bin/bash
PATH=:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin
cd /
sleep 10
mount_smbfs -N -I 10.1.171.200 //BACKUP/VOL1 /mnt/VM-VOL1
sleep 10
if df |grep -q '/mnt/VM-VOL1'
then
rsync -arzv  /mnt/VM-VOL1/ /mnt/MP1/VOL1/ --delete --progress --backup --backup-dir=../20090820

sleep 20
umount /mnt/VM-VOL1
sleep 10
chmod -R 777 /mnt/VM-VOL1/
else
echo "Aborted because the source disk is not mounted"
exit -1
fi
------------------------------/code----------------------------------- 

Lets go through the script:

#!/bin/bash
PATH=:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin
---sets the script interpreter (the shebang line)
---sets the search paths

cd /
sleep 10
---changes directory to the top of the directory tree
---pauses the script for 10 seconds

mount_smbfs -N -I 10.1.171.200 //BACKUP/VOL1 /mnt/VM-VOL1
sleep 10
---mounts the NetWare volume to a mount point, no need to add it to the fstab or mount it by hand.
---pauses the script for 10 seconds

if df |grep -q '/mnt/VM-VOL1'
then
rsync -arzv  /mnt/VM-VOL1/ /mnt/MP1/VOL1/ --delete --progress --backup --backup-dir=../20090820
---"if" checks the output of the "df" command (via grep) to make sure the volume is mounted
---"then" the RSYNC command
--- notice the RSYNC command line ends with "--backup-dir=../YearMonthDay", this combined with the "--delete" and "--backup" commands will save any deleted or changed files from the previous RSYNC to a directory named  "YearMonthDay". The YearMonthDay folder will be on the root of the destination volume.

sleep 20
umount /mnt/VM-VOL1
sleep 10
chmod -R 777 /mnt/VM-VOL1/
---pauses the script for 20 seconds
---unmounts the NetWare mount point
---pauses the script for 10 seconds
---changes permissions on the backup volume (so files/dirs can be modified)

else
echo "Aborted because the source disk is not mounted"
exit -1
fi
---the "else" part of the if-then-else statement, if the NetWare volume is not mounted, the script ends
---echo's the error
---exits the script
---end of the if-then-else statement

Run the Perl script from the prompt like: “perl /mnt/MP1/YourScript.pl”.

You must call Perl, and use the pl extension.

To schedule these scripts, simply use the FreeNAS GUI to add the scripts to the System > Advanced > CRON.

Schedule the Perl script to run 5 min or so before running the bash script.

The Perl script makes the bash script.

The bash script runs the RSYNC.

*TIP: The “Run now” command will probably not work with the Perl script, don’t panic. Remember, you are running the job as “root”, but you are logged into the GUI as “admin”, it’s OK, the CRON job will run just fine. See the FreeNAS forums for a better explanation of the differences between the GUI user and root.

The Fine Print (and other ramblings) Part Duex:

The key to this is getting the “RSYNC –delete –backup –backupdir” commands to work correctly. By creating a YearMonthDay directory every day, you are creating an archive directory for every day. This will allow you to recover files that were deleted or changed on previous days, and still have an up to date full RSYNC. Play with this with some test data until you get it exactly right, and you understand exactly what is happening. Once again, be careful with the –delete command.

Over time your backups and the daily archive folders can get quite large, plan accordingly (buy big drives!).

To create a backup solution for more than one server, you can overwrite the nsmb.conf file (if you need to change server IP’s, user names etc…) using the same Perl script you use to create the bash script. Add something like this to the end of it.

-----------------------------------code------------------------------------------------
## this is the output name and path of the autologin file for SMBFS, edit as necessary ##
$autolog = "/etc/nsmb.conf";
## opens nsmb.conf file
open (CFG2, ">$autolog");
## copys the following lines to the nsmb.conf file
print CFG2 "[SERVER2_W]\n";
print CFG2 "addr=10.1.1.203\n";
print CFG2 "[SERVER2_W:ROOT]\n";
print CFG2 "password=root\n";
close (CFG2);
## use the perl chmod to make our file usable
chmod(0777, "/mnt/system/scripts/MYrsync");
----------------------------------/code-----------------------------------------------

By installing Perl, you gain the ability to do many different things with FreeNAS. For example, you could use http://caspian.dotconf.net/menu/Software/SendEmail/ to e-mail daily RSYNC logs.

http://www.freenas.org/index.php?option=com_openwiki&Itemid=30&id=howto:adding_bacula_to_freenas

Looks like another great solution.

Combine the above with:
http://www.bacula.org/en/dev-manual/Automated_Disk_Backup.html

And you could have another great backup to disk solution.

Visit the FreeNAS forums for more ideas.

Using FreeNAS, you have the ability to move your backup device as far away from your actual server as bandwidth permits.

To restore a volume:

Read the RSYNC man pages!

Switch the RSYNC source and destination. DO NOT USE THE –DELETE COMMAND DURING A RESTORE. It would be a good idea to remove all secondary commands from the RSYNC during a restore, maybe leave just the –progress switch.

Once the files are back on the volume, run the trustee.nlm and restore trustee rights to the data volume(s).

Read the RSYNC man pages! And test test test!!!

This RSYNC solution is not the best solution for disaster recovery. It is a great solution for day to day backups.

You could always use the tried and true Novell solution of replacing a crashed server in case of a disaster.

For a disaster recovery solution, I went with an imaging solution.

I created an iSCSI target volume using FreeNAS (on the same volume that I save my RSYNC’s to), then use Portlock ( http://www.portlock.com/ ) and a small script to make an image my SYS volumes nightly (keeping 3 days worth of images) the “VeryLazy-admin” approach mentioned earlier. I can restore a SYS volume to the same hardware, different hardware, or a virtual machine. After the SYS volume is restored, I can restore the data volume(s) with RSYNC.

The one issue with this method, is that the iSCSI volume must be mounted to another NetWare server to get to the image of the SYS volume. This could be overcome by simply moving the SYS image to a more accessible place.

To RSYNC data volumes that contain live databases (GroupWise for example), use NSS snapshots to “snap” the volume, then RSYNC the snapshot. The daily archive of a GroupWise RSYNC will be quite large due to the massive amount of file changes, databases etc….

All in all, I think for significantly less then the cost of a reasonably sized tape drive, you could make a highly effective multi terabyte disk backup solution that covers both disaster recovery and daily file backups.

My hardware setup:

HP DC7600 — off lease/used $270.00
4 x 1 TB SATA HDD’s — $100.00/ea
SATA backplane drive cage — $100.00
Added RAM — $40.00
FreeNAS — Free

Even with the added cost of Portlock ($500 for business..less for Educational) , this solution is at least comparable or even better than most (all?) of the major NetWare backup solutions.

-Scott Owen

1 vote, average: 5.00 out of 51 vote, average: 5.00 out of 51 vote, average: 5.00 out of 51 vote, average: 5.00 out of 51 vote, average: 5.00 out of 5 (1 votes, average: 5.00 out of 5)
You need to be a registered member to rate this post.
Loading...Loading...

Categories: Uncategorized

1

Disclaimer: This content is not supported by Novell. It was contributed by a community member and is published "as is." It seems to have worked for at least one person, and might work for you. But please be sure to test it thoroughly before using it in a production environment.

1 Comment

  1. By:mbrady

    Great article Scott! I use FreeNAS here and there too, it’s a great tool.

Comment

RSS