An engineer's life around Kyoto

Several days ago, I wanted to make a backup of my home directory to an external HDD connected with USB, and decided to use the "rdup" utility. "Rdup" is a set of commands to manage multiple generations of backups of a certain directory.  What caught my eyes was its feature to make hard links between the generations automatically to save disk space. In order to accomplish a typical backup task, rdup's three commands should be combined appropriately.  But if your backup goes to a local storage straightforward daily, a shell script 'rdup-simple' gives a shortcut to the task like: # rdup-simple /home/mine /media/BACKUPHDD/mypc/ Because rdup is provided as a package in Debian, its installation was also very simple.

One of my friends said  several power supplier parts of the servers that he administrates has been damaged by a electric surge, caused by thunders in Kyoto the day before yesterday. Indeed, the climate around here was very bad. Streams were flooded with heavy rain, trains had been stopped for hours, and short-term electric outage occurred, I've heard.  I might be lucky that my computers were not damaged at all because I was out and pulled off their electric cables  from the outlets.

Here is another report on an incident in Japan. First Server is one of the major hosting providers of rental servers in Japan. On June 20th, when one of their engineers tried to apply a security fix to their customer's OS environment, he/she mistakenly erased more than five thousand disk images of their customers, as well as the corresponding backup disk images. After three days of recovery work they given up recovering all of them. Tragically the compensation will be at most the total price which the customer has been paid for the service, the spokesman of the company explained. This incident seems to be an alarm bell on the service management depending on the hosting service of third-parties. You can make a contract with them, but they do what they can do at most.

You know, we're facing electricity shortage here in Kyoto.  All the nuclear electricity power plants are shut down periodically for the security inspection, but never to be brought back again after Fukushima nuclear power plants are damaged by the earthquake and the tsunami in March 11th last year.  Now, all the nuclear power plants are stopped in Japan. Electricity shortage is supposed to be inevitable. Especially, Kansai district including Kyoto is heavily dependent upon the nuclear power plants, electricity outage is planned this summer. The government says that Ooii power plants are going to be in service in six weeks, but it is far from enough.  I am planting creepers in the garden to block the summer sunshine, wishing that the climate will be moderate this summer.

Ssh command can send back the data on the remote standard output stream to the local standard input like: ssh hoge@remote 'dd if=/dev/zero count=512' | dd of=/dev/null Of course, this does not make sense, but works  :-)  I often use this data transmission method for the backup purpose. Recently, I was asked by the network administrator to limit the band width of ssh data transmission. I didn't know how to archive his demand at first, but I found a good command-line tool, and worked it out. It was "cpipe". I could install very easily with: apt-get install cpipe and here is an example how to use cpipe: ssh hoge@remote 'dd if=/dev/zero count=512 | cpipe -s 1000' | dd of=/dev/null Have fun!

I suppose it a common way to preserve multiple generations of backup copy of your important data. To preserve them as much as possible, archiving and compressing the backup copy is one way. But there is another way to save the storage: hard-linking. The basic idea is to share the identical files between different generations of backup copy with hard links. If the target data consist of normal files, rsnapshot does most of the hard-linking management automatically. But I recently encountered the problem with the database files managed by daemon processes. Because they are supposed to be changed asynchronously,  the consistency of th オプション eir copy taken with the cp command is not guaranteed. Because this kind of problems is often aware in the design of the daemon processes, special backup methods are provided to export the consistent backup copy. For example, Subversion provides "svnadmin hotcopy" subcommand, and Trac provides  "trac-admin hotcopy...

If you are an owner of a laptop PC running Linux, you might be annoyed with the battery shortage. If so, try consulting the command 'powertop'. It tells you what kind of background tasks  are consuming the electricity a lot. Cheers!