de System Administration And Computer Usage Tips & Tricks

The following list is a collection of personal tips and tricks I carried together when I had the need and which I collected here as some kind of personal compendium. As such, most of these tips are rather terse and often are not much more than a collection of references to further reading. However, feel free to contact me directly if you encounter some specific problem when trying to follow one of these tips. Please also contact me whenever you know a better or more correct solution than the one I present below.

IMPORTANT: There's absolutely no guarantee that any of these tips actually works for you and won't eat your data (especially the tips concerning backup solutions and file conversions). Please make sure you backed up all of your important data until you verified that nothing bad will happen when following a given tip.

Managing Digital Fotos / Images

Over time I collected some tips and best practices for managing my digital fotos which I found by trial & error and Google research, which proved extremely useful for me and of which I hope will be useful to others as well:

  1. Automatic lossless rotation / re-orientation of digital images (Portrait vs. Landscape). There are serveral tools which can perform these operations, but only some will correctly adjust the hidden technical information embedded in every digital image.
  2. Invisibly storing text comments and descriptions directly in your digital images and process this information.
  3. Georeference your digital images such that you'll always know where exactly this holiday photo was taken - this way you will be able to browse the locations your images were take using the digital Google Earth globe.

fsvs - creating fully versioned backups

fsvs written and maintained by Philipp Marek (fsvs is an acronym for eg. File System VerSioning) is an extremely cool tool which allows you to create fully version backups of arbitrary directories, eg. of your whole system or your /home directory. This allows you to retrieve any previous version of any managed file. It uses the popular Subversion version control system (Debian-package: subversion) as its efficient backend storage, but in contrast to pure svn it also stores the file's ownership and access permission information and it does not keep local shadow copies of the backupped files, so fsvs-managed directories require much less disk space than a normal svn working copy directory.

I use fsvs to manage my whole /home directory.

Serverbackup - Scripts implementing a Live-Backup-mechanism

Here you'll find some tested backup scripts I'm using to keep life backups (unpacked backups) of some of my servers with a few days of history information. These backup scripts use rsync (Debian-package: rsync) so the backup process is rather quick and usually transfers not much data.

Scriptable PDF- and PostScript Processing

Often your task is to process or convert PDF- and PS-/PostScript documents, possibly even batch processing larger numbers of documents, splitting and (re-)combining the pages contained. This tip introduces tools like pdftk, pdfjam, pdftops and others which make these tasks a snap.

Subversion Performance

In most cases, if the version control system Subversion's performance is discussed, the topics revolve around the repository backends used (Berkeley DB (bdb) vs. FSFS) and tuning some server software or even hardware parameters.

It seems to be less known that the choice of the server variant used - the Apache Subversion mod_dav_svn module or the standalone svnserve server - have a great impact to measured and perceived subversion performance.

Usually svnserve is significantly faster than Apache mod_dav_svn

In a synthetic, non-representative benchmark test I performed using Subversion 1.4.5, Subversion 1.1.1 and Apache 2.0, mod_dav_svn's performance was 30% to 400% slower than svnserve's. svnserve's performance was close to local direct accesses to the repository using the svn command line tools.

The most significant performance penalty was measured during svn log and svn merge operations against the mod_dav_svn server - you'll notice worse svn log performance immediately if eg. using the Eclipse Subversion plugin Subclipse.

As svnserve supports virtually the same repository path-based access control features as mod_dav_svn from from subversion 1.3 on, it could prove to be a viable alternative to an Apache based svn server for many projects.

For reference, I publish a rather raw list of my subversion performance measurement results.

Upload Backups Using FTP

Many root-server providers also provide some backup space accessible via FTP. This imposes the question of how one can upload his encrypted backups in the most efficient way. (The FTP server is a foreign machine accessible via an insecure, unencrypted link from which both your data and your login credentials can be retrieved by unauthorized parties - so, you're encrypting your backups, right? ;)

duplicity (Debian-package: duplicity)
Duplicity creates encrypted incremental backups which it can optionally upload to an FTP server, so it sounds just like THE solution. However, as of May, 2007, Duplicity is still an unfinished product and the author does not yet recommend its usage in production critical environments. Additionally, the most recent development release is already more than a year old. It was not option for my, anyway, as I already created my encrypted backup files using custom scripts and needed a pure upload solution.
curlftpfs (Debian-Package: curlftpfs)

CURL-FTP-FS is a FUSE (Filesystem in UserSpacE) -based FTP filesystem implementation which allows simple mounting of remote FTP servers just like local volumes. In theory, this reduces the whole incremental upload problem to a simple cp -dRuv --preserve=timestamps /local/backupdirectory /mnt/ftp_server_mountpoint/ - in practice, I soon noticed that curlftpfs loads each file completely into RAM before uploading it to the remote site (This is true as of May, 2007, curlftpfs version 0.9). This can kill a whole machine and everything blows up even if the machine survives as soon as one of the files to be uploaded exceeds a size of 1GB.

However, curlftpfs may be the appropriate solution if you'd like to have read access to your backup space using standard UNIX tools. You can measure your backup space usage using du --summarize this way, for example.

mirrordir (Debian-package: mirrordir)

mirrordir mirrors - just as its name suggests - a directory tree to another location, as accurately as possible. Both the mirror source as well as the mirror destination may reside on local disks or at remote FTP servers. mirrordir does not seem to have major problems with huge files, provided that you have sufficient space in your TMPDIR, as mirrordir first created a temporary copy of a file before its uploaded. ;)

A command line to upload your backups may look as follows: TMPDIR=/var/tmp/ mirrordir /local/backupdirectory ftp://USERNAME@ftp.server.example.com/backup/. As usual, your FTP login credentials should be placed in your ~/.netrc file. However, you need to specify your FTP username within the URL anyway, for some strange reason.

Internally, mirrordir utilitzes GNU Midnight Commander's virtual file system layer for file access. I've currently settled on mirrordir for uploading my encrypted daily backups to the Hetzner backup FTP server.

Valid XHTML 1.0 StrictValid CSS!
-- /software/tipps_en.php#20090510-002246  [0] © Gunter Ohrner 2007-2013. Powered by CubbiCMS