Skip to main content

Multiple generations of backup and hardlink

I suppose it a common way to preserve multiple generations of backup copy of your important data.
To preserve them as much as possible, archiving and compressing the backup copy is one way.
But there is another way to save the storage: hard-linking.

The basic idea is to share the identical files between different generations of backup copy with hard links.

If the target data consist of normal files, rsnapshot does most of the hard-linking management automatically.

But I recently encountered the problem with the database files managed by daemon processes.
Because they are supposed to be changed asynchronously,
 the consistency of th
eir copy taken with the cp command is not guaranteed.

Because this kind of problems is often aware in the design of the daemon processes,
special backup methods are provided to export the consistent backup copy.

For example, Subversion provides "svnadmin hotcopy" subcommand, and Trac provides  "trac-admin hotcopy" subcommand.

Unfortunately, rsnapshot does not seem to collaborate effectively with these hot backup copy.
Instead, I tried the "hardlink" command provided by the package "hardlink".

The "hardlink" investigates two directories given in its arguments,
comparing the files with the same relative paths.
If they are identical, "hardlink" replaces one of them with the other's hard-link.

For example, on 30 Apr 2012, if you take the subversion's hot copy of /srv/svn/myrepos/ to /srv/backup/20120430/myhrepos,
you give the following command:

 svnadmin hotcopy /srv/svn/myrepos/ /srv/backup/20120430/myrepos

Next, to share the identical files with the backup copy of the previous day,
you may give the following command:

 hardlink -t /srv/backup/20120429/myrepos /srv/backup/20120430/myrepos

As far as I use, hardlink saves much more disk space than bzip2 compression.

Comments

Popular posts from this blog

The final update to Debian 11 has come

Recently, the 11th point release of Debian 11 has been announced, and it will be the last one. For I am still maintaining one Debian 11 machine, I am starting to prepare its upgrade to Debian 12. Because it is a dual boot machine, I must not forget about the change of grub OS probing. Before I ugrade, I have to add  GRUB_DISABLE_OS_PROBER=false to / etc/default/grub,  or I lose grub menu for Windows OS.  Another thing that impacts me is the change of gnome-text-editor. In Debian 11, gnome-text-editor is a kind of alias of gedit controlled under the Debian alternative system with update-alternatives command. On the other hand, a new package gnome-text-editor appears in Debian 12, and it is a different thing from gedit. For I am an uim user, I was confused when I tried to use launch a Japanese input method via uim on gnome-text-editor in Debian 12, for it wouldn't work as in Debian 11. Yes, these two things are important reminder to me.

No planned electricity outage

The planned electricity outage for a peak overload has never taken place this summer in Japan. Although we were encouraged to cut ten percent peak load off and avoided the outage, we might lose a chance to learn enough how much we depend on uninterrupted electricity supply in the western part of Japan, which the huge earthquake didn't hit directly last year. It is still in question on the papers whether the two atomic power plants, which were re-started in July,  were necessary.

Electricity outage is planned this summer

You know, we're facing electricity shortage here in Kyoto.  All the nuclear electricity power plants are shut down periodically for the security inspection, but never to be brought back again after Fukushima nuclear power plants are damaged by the earthquake and the tsunami in March 11th last year.  Now, all the nuclear power plants are stopped in Japan. Electricity shortage is supposed to be inevitable. Especially, Kansai district including Kyoto is heavily dependent upon the nuclear power plants, electricity outage is planned this summer. The government says that Ooii power plants are going to be in service in six weeks, but it is far from enough.  I am planting creepers in the garden to block the summer sunshine, wishing that the climate will be moderate this summer.