Hello,


I have a few linux servers that i want to be able to restore in no time, so
i was thinking of dumping the entire disk contents to tape using dd.
sth. like dd if=/dev/sda of=/dev/st0.

the problem with this is: what if the disk to be restored on is different in
size.

Can i expect that if the disk is bigger, the restore would work without a
problem ?

The problem being that the servers are serviced by the manufacturer itself,
and who nows if they decide a year from now to only provide disks of 72GB
instead of currerntly 36GB. Would my 'dd' backup still be of any use ?

other question, could i restore these tape contents to an image file, and
mount this image file as a virtual disk ?

lastly,
are there other problems or better approaches for what i am trying to do ?
btw, the disk is a 36GB disk, and the tape drive would be a dds4 (20/40).
how do i get the entire disk contents to fit on one tape. is it safe to pipe
the data through gzip before writing them to tape ?


thx,

Tom.

tar it.
It's called tar for a reason :-) it is meant for this.

with tar : no problem.

Yes, but the bigger disk would only be used partially.

ditto.

Is it just one partition? If so yes. If not, I don't think so.

tar.

I wouldn't. Not sure if it will even help at all. Some tapedrives compress
as well, and assume a certain compression ratio can be achieved to reach
their "capacity".

Another reason for tar, you aren't storing irrelevant bits with tar.

Eric

Tom Van Overbeke wrote:
DDS4 has a 20GB native capacity. To get to 40GB, they assume 2:1
compression. IIRC, DDS4 drives can do the compression in hardware.
It's just a matter of enabling it. I stay clear of compression on my
backups so I can't tell you how to enable hardware compression.

It's not needed if your drive can do the compression in hardware. It
also depends on the type of data. If you're dumping a lot of already
compressed data (JPEGs, MPEGs, and the like) or uncompressable data,
gzip (or hardware compression) won't help at all.

Hi Eric,

thanks for your remarks.


However, as i want to restore 'in no time', i chose the 'dd' way because i
can then recreate the entire disk structure (MBR, partition table) on the
fly by simply restoring from the tape. using tar, i would need to create the
disk structure myself (including finding a way to put the MBR back), and
then doing the restore, is this correct ?

One fact i forgot to mention, the solution is supposed to be 'idiot proof'
as i will be shipping these servers to remote locations all over Europe and
I can not be sure of the technical capabilities of the local it staff.


Tom.



"Eric Moors" <scare.crow@oz.land> wrote in message
news:c1d2ks$3e6$1@news.surfnet.nl...

"Tom Van Overbeke" <tom.van.overbeke@pandora.be> writes:

Don't do it. First, it seems that /dev/sda hosts your root
partition. How long it would take to backup /dev/sda? Will
anything change during that time? You will have a corrupt
backup. If a file is changed, that's one thing. But the
filesystem metadata will also change, so your backup will be
problematic. You should use dd to copy a partition/disk only
if the partition is either not mounted or is mounted readonly.
Second, this thing will also backup unused space. Say, you had
a 300MB gzipped file a while ago but you already deleted. Then
dd, since it doesn't work on the filesystem level, will also
backup the still existing blocks of this long gone file. It will
inflate your backup, and it is also hard to compress. Third, if
your disk has surface errors, then that can also cause problems.

All in all, this is not a good way to backup your data.

No.

This is one reason why at work we don't want manufacturers to service our
machines... We buy the parts, we build it, and we are totally in control.

No, since you backup a whole disk and not a partition. You can mount
a partition image, but not a disk image (unless you make some real
messing around like extracting the partitions from the image).

Also, since you backed up a life rw filesystem, there *WILL BE*
errors on the backup. And some will be filesystem metadata which
means the whole image is unreliable. You might be able to mount it,
but it won't be what you are looking for. It will have some hidden
errors which will come back and haunt you in the long run.

If you are not easy writing your own backup solution
(and maintaining it), then consider buying a commercial backup
solution which handles these problems. Of course, there are
also free solutions. I don't know any of such since I wrote
our own backup program (perl and uses tar) to handle our needs.

If the tape does its own compression, then you don't have to worry
about compressing. But you might need to enable it. If it is not,
then you can run it through gzip. However, bzip2 would be a better
choice since it compresses better, and if for some reason the tape
becomes corrupted (i.e. one or two blocks become unreadable or
corrupt) you still can recover things behind it.

Vilmos

Vilmos makes a very good point, below:

All backups should be done on a quiescent device, i.e.
no writes going on, otherwise you cannot guarantee
that the backup is valid. This also applies to utilities
such as "tar" where the file may change out from
under you as it is being backed up.

The desirement (as opposed to requirement) for the
backup taking "no time" is also questionable.

The "dd" method will read all 36 GB from the disk
and write it to tape using a bit-for-bit image copy.
As Vilmos points out, this may be wasteful, if there
are large chunks on the disk which belong to no current files.
Just reading 36 GB of data from a disk may take on the
order of 10-20 minutes or more. Writing it to tape will
most probably take even longer so that the total time
to do the backup will be dominated by the tape.
Restoring will also take just as long.

Restoring from a "dd" will overwrite the partition
table with that data on the tape. Thus, your
hypothetical future 72 GB drive will become,
de facto, a 36 GB drive unless you do some
"smoke and mirrors" after the restore. (Don't
try this at home, kiddies!) But, if you're restoring
to another 36 GB drive, all should be OK, mostly.

"tar" will only back up the currently existing files,
which will save time if you don't have a full filesystems
on the disk. Restoring from a "tar", tho, will require
you to partition the new disk and mount (!) the partitions
for the restore to work.

Tradeoffs, always tradeoffs!

Either can be automated with a (most-probably) complex
shell script (you choose your own favorite shell).
However, the error checking necessary in that particular
script will probably be 75% of the code in it, and,
even then, it will not be "idiot proof."

(See signature line)

Vilmos Soti wrote:

--
Ñ
"It is impossible to make anything foolproof because fools are so
ingenious" - A. Bloch

On 2004-02-23, Tom Van Overbeke <tom.van.overbeke@pandora.be> wrote:

My advice: don't use dd for this. There are many more appropriate tools
available to you, ie: tar, cpio afio, dump/restore and many more.

Best scenario is that the protion of the drive beyond the restored data
will simply be unused. But remember, dd copies *everything* including
unused sectors, broken links, and so on. You don't need to plant that
kind of cruft on your nice new disk.

Only 36G worth in the best-case scenario. If you use one of the other
tools, it won't matter.

Probably.

Don't use dd. Use one of the other tools. You'll be glad you did.

If you use one of the other tools, you can specify a compression algorithm
on the command line. Your tape drive may support hardware compression
(the 20/40 you give suggests that it does); if so, don't bother with the
software compression, just let the drive compress it as it writes the
tape.

--

-John (JohnThompson@new.rr.com)

On 2004-02-23, John-Paul Stewart <jpstewart@sympatico.ca> wrote:

My experience with hardware compression (I have 3 DLT drives here) is that
it is quite robust. I don't think you gain anything by not compressing
the backup using the tape's hardware.

--

-John (JohnThompson@new.rr.com)

John Thompson wrote:
I've heard that hardware compression algorithms can be
manufacturer-specific (or even worse, device-specific) making it
impossible for a hardware-compressed tape created on one drive to be
read on another drive from a different manufacturer. I don't know if
that's true at all or only applies to certain classes of tape (i.e., it
applies to DDS but not DLT) or what. But I'd rather not learn that is
true when trying to recover from a disaster. Since I only have one
model of tape drive at the moment, I can't do any sort of compatability
testing either.

On Mon, 23 Feb 2004 15:05:47 +0100, "Tom Van Overbeke"
<tom.van.overbeke@pandora.be> wrote:


My $.02 worth.

Contrary to some beliefs here my shop required the same solution as
you are asking. I tried many methods and also tries a few
backup/restore products. By far the best of them IMHO was storix
from www.storix.com.

In fact, just last Friday we experienced a corrupted superblock on one
of our servers. The tools we had could not correct this. Our option
was to restore the disk. Fortunately we had created a storix backup
just the previous week and we had the volatile directory structure
backed up on a daily backup (also using storix).

We booted from the storix created diskettes (you could also use CDROM)
which prompted for the insertion of the backup tape. Upon examination
it displayed what it found and after we accepts all the defaults it
restored the entire data to our /dev/hda. Upon completion it
rebooted and the next thing we saw was a perfectly restored system.

We have also restored images made on disks with different geometry to
different disks with storix making all the adjustment for us.

The "personal edition" designed to work on a single system with an
attached tape drive is FREE. Even if you wish to get fancy and
network your backups, the server and client costs are quite
reasonable.

Give them a look - you shouldn't be disappointed.

Bob

On 2004-02-23, John-Paul Stewart <jpstewart@sympatico.ca> wrote:

DLT is probably immune to this, as all DLT drives are made by one
manufacturer (Quantum) and rebadged if necessary.

In any case, I have a DEC and two Compaq DLT drives and they no problem
reading each other's tapes.

--

-John (JohnThompson@new.rr.com)

Thx everyone for all the helpful advice.


meanwhile, i found mondo (www.mondorescue.org), an opensource software that
does exactly what i want (once i get it installed ).



regards,

Tom.



<R.Mariotti (at) FinancialDataCorp.com (Bob Mariotti)> wrote in message
news:403ab18e.1121259@news.cshore.com...

On Mon, 23 Feb 2004 14:39:02 -0500, the skies darkened, lightning flashed,
and the voice of John-Paul Stewart spake thus:

attempt to compress an already compressed file could result in a larger
result.

Well, the last place I worked at had a 40/80Gb DLT tape drive from HP, and
due to a sudden increase in backup requirements, it was decided to get a
HP 8 tape autoloader, also 40/80Gb DLT. Minor problem: the tapes writen
in the single drive wouldn't bet read by the 8 tape drive, and the 8 tape
drive couldn't blank or reuse tapes from it either, dispite both drives
being controlled by the same software (Veritas, Backup Exec v9). So it's
not just incompatibility between manufacturers, but between products from
the same manufacturer as well.

My new work is using Sony AIT tapes and drives, on 2 different machines.
Different sizes, on each. I might have to try the smaller tape in the
larger drive, see if it works.

ktabic
--
www.ktabic.co.uk
Many sysadmins won't give you the time of day.
Thats what NTP is for.

ktabic wrote:
compatability. My present machine has a VXA-1 on it and my new machine
(not yet even smoke tested, awaiting UPS and monitor) has a VXA-2. These
are pretty much the same drives, but the VXA-2 is twice as fast and
holds a lot more data (160GBytes, compressed). They are said to be
compatible (but a 160 GByte VXA-2 tape cannot be read by a VXA-1 drive,
nor can one be written by a VXA-1 drive).

--
.~. Jean-David Beyer Registered Linux User 85642.
/V\ Registered Machine 73926.
/( )\ Shrewsbury, New Jersey http://counter.li.org
^^-^^ 7:50am up 49 days, 19:11, 2 users, load average: 2.14, 2.14, 2.10

Jean-David Beyer wrote:
The VXA-2 drive writes data differently from the VXA-1 drive. The VXA-1
drive cannot read a VXA-2 tape as you stated, but it also cannot read a
VXA-1 tape that was written with the VXA-2 drive (IIRC). The VXA-2
drive *should* be able to read a VXA-1 tape regardless of where it was
written.

Because of the differences between the VXA-1 and VXA-2 drives, when
writing to a VXA-1 tape, the VXA-2 drive can put approximately 50% more
data onto the tape. (Talking native capacity always.) For example, the
V17 tape which will hold 33MB when written in a VXA-1 drive will hold
59MB when written in a VXA-2 drive. The capacities are summarized here:

http://www.exabyte.com/products/tape/tape_vxa2.cfm

On 2004-02-25, ktabic <ktabic@no-spam.ktabic.co.uk> wrote:

Most modern compression programs can be set to silently dump the
compressed output if it exceeds the size of the original, and write the
original uncompressed data instead.

I have a 20/40 DLT and two 15/30 DLTs. The 20/40 drive can read tapes
created in either of the 15/30 DLTS, and the 15/30s can read 15/30 tapes
created in the 20/40 drive.

--

-John (JohnThompson@new.rr.com)

John-Paul Stewart wrote:
A VXA-2 drive should be able to write a VXA-1 tape if you use a V-17 or
shorter tape. You must find a way to tell the VXA-2 to write a VXA-1
tape, though.

Exabyte say:

--
.~. Jean-David Beyer Registered Linux User 85642.
/V\ Registered Machine 73926.
/( )\ Shrewsbury, New Jersey http://counter.li.org
^^-^^ 4:30pm up 50 days, 3:51, 2 users, load average: 2.27, 2.17, 1.68