Skip to main content

Multiple generations of backup and hardlink

I suppose it a common way to preserve multiple generations of backup copy of your important data.
To preserve them as much as possible, archiving and compressing the backup copy is one way.
But there is another way to save the storage: hard-linking.

The basic idea is to share the identical files between different generations of backup copy with hard links.

If the target data consist of normal files, rsnapshot does most of the hard-linking management automatically.

But I recently encountered the problem with the database files managed by daemon processes.
Because they are supposed to be changed asynchronously,
 the consistency of th
eir copy taken with the cp command is not guaranteed.

Because this kind of problems is often aware in the design of the daemon processes,
special backup methods are provided to export the consistent backup copy.

For example, Subversion provides "svnadmin hotcopy" subcommand, and Trac provides  "trac-admin hotcopy" subcommand.

Unfortunately, rsnapshot does not seem to collaborate effectively with these hot backup copy.
Instead, I tried the "hardlink" command provided by the package "hardlink".

The "hardlink" investigates two directories given in its arguments,
comparing the files with the same relative paths.
If they are identical, "hardlink" replaces one of them with the other's hard-link.

For example, on 30 Apr 2012, if you take the subversion's hot copy of /srv/svn/myrepos/ to /srv/backup/20120430/myhrepos,
you give the following command:

 svnadmin hotcopy /srv/svn/myrepos/ /srv/backup/20120430/myrepos

Next, to share the identical files with the backup copy of the previous day,
you may give the following command:

 hardlink -t /srv/backup/20120429/myrepos /srv/backup/20120430/myrepos

As far as I use, hardlink saves much more disk space than bzip2 compression.

Comments

Popular posts from this blog

The final update to Debian 11 has come

Recently, the 11th point release of Debian 11 has been announced, and it will be the last one. For I am still maintaining one Debian 11 machine, I am starting to prepare its upgrade to Debian 12. Because it is a dual boot machine, I must not forget about the change of grub OS probing. Before I ugrade, I have to add  GRUB_DISABLE_OS_PROBER=false to / etc/default/grub,  or I lose grub menu for Windows OS.  Another thing that impacts me is the change of gnome-text-editor. In Debian 11, gnome-text-editor is a kind of alias of gedit controlled under the Debian alternative system with update-alternatives command. On the other hand, a new package gnome-text-editor appears in Debian 12, and it is a different thing from gedit. For I am an uim user, I was confused when I tried to use launch a Japanese input method via uim on gnome-text-editor in Debian 12, for it wouldn't work as in Debian 11. Yes, these two things are important reminder to me.

Data migration tool from Figaro's Password Manager 2

Figaro's Password Manager 2 was my favorite password management tool. It is GTK2-application to preserve and manage your pairs of ID and password secretly with cipher. To my regret, its Debian package was orphaned many years ago and I thought it was time to switch to another password management tool. But besides finding a good alternative choice, I had a big problem: migration. Figaro's Password Manager 2 does have data export function, but only in special XML format. If you want to move your data to a different application, you have to change the format of the data even if the application can import the data. As my next password management tool is KeePass2 or KeePassXC , I need a XML to CSV data converter. So, I have been developing the conversion tool very slowly and finally, completed. Today, I published it on GitHub , which I named fpm2_csv . The history of the code is quite embarrassing, but very few people in the world would be interested in it, that doesn't matte...

Epson MFP EP-977A3 and Linux box

Just a report of a hardware device usage for reminder.   EPSON EP-977A3 is a multi-functional peripheral for personal use once sold in Japanese market.   It has color printer, color flat-bed scanner, USB memory RW capability with USB and both wired and wireless LAN connectivity.  With the simple-scan software, whose package name is 'simple-scan' in Debian 10, I am able to scan documents from the flat-bed scanner of EPSON EP-977A3 connecting with my Debian box with an USB cable.   But when I tried to start scanning on the control panel of the scanner device, I got error with a message saying "The computer is not ready". I guessed if the scanned package for that purpose and installed it. Still, I got the same error message. Hmm... I need more research.