Batch Rename Multiple Files with Linux

Saturday, 13th Dec 2008 at 00:07 (administration, linux)
Tags: , , , ,

Here is a list of ways how to rename multiple files with Linux. The easies and quickest way I found was to use ‘rename’.

Leave a Comment

One hack of a perfect (as in jack of all trades) backup solution for Ubuntu Linux (remote, flexible, instant restore, automated, reliable)

Tuesday, 14th Oct 2008 at 21:34 (administration, ideas, linux, security, ubuntu)
Tags: , ,

This is a work in progress (and most likely will always be so)!

Here is what I have been working on and looking for to aquip myself with. I wanted to keep working without any hassle on my daily stuff just as I ever had and with changes to come. But at the same time I needed to be sure for a situation where I needed older versions of my files — would it be due to a system or hard disk brake down or a file deleted erroneously or changes need to be undone — to just have them there by no more than one command away. To say in short I wanted a time machine for my files that just works™ — also in at least 5 years time. In the case of recreation of older versions I want to be able to focus on what to restore and not how. And also with previous backups I have had e.g. corrupted archive files or unreadable part files/CDs to many times (one is even to many) or I’ve had issues because of too old of a file format (mostly proprietary formats).

Here is what I’ve been looking for feature-wise generally:

  • no expenses money-wise
  • robust
  • using only small and freely available tools — the more system core utils the better
  • version control
  • snapshot system
  • remote storage
  • private, i.e. secure data transmission over network and reliably encrypted storage
  • suitable for mobility, independent of how I’m connected
  • simple yet flexible usage

for daily backups:

  • automation using cron
  • no need for interaction
  • easy and flexible declaration of files or folders to omit from backup

and for restoring data:

  • just works™ (see above)
  • fast and easy look up of what versions are available at best via a GUI like Timeline with filter options
  • at very best some sort of offline functionality, e.g. caching of most likely (whatever that means) required older versions

(partly) Alternative solutions I have come across on the run

  • Suns’ z file system (zfs): Haven’t had enough time to get it working with Ubuntu Linux (because of license issues not packaged, only working via FUSE so far). Need’s partition setup thus lavish. Not sure about networking/mobility demands, e.g. remote snapshot location nor ease of use.
  • subversion together with svk: Easy and flexible to use and automate, version control per se, distributed and offline operations (svk). Contra: Recovery relays on subversion software, i.e. no cp or mv. Basic idea is to work on a copy: checkout before you start) and have daily automated commits. Should need no interactions since I’m the only one working with my “backup projects”. See this lengthy description.
  • Coda file system: distributed file system with caching. Had not enough time to try out.
  • rsnapshot: Has remote feature (ssh, rsync), automation, rotation. Relies on file systems using hard links within backup folder hierarchy for “non-incremental files” and runs as root only (system wide conf file, ssh configure issue, ssh-key, …). Workaround could be to use a specific group.
  • sshfs: FUSE add on to use remote directories via ssh transparently.
  • croned bash backup script using tar and gzip; daily incremental and monthly save “snapshot” similar to logrotate.
  • grsync: gnome GUI for rsync optimized for (incremental) backups

Update 10/2009: A few weeks ago i stumbled upon Back In Time which has astonishingly many properties of what I expect from a perfect backup solution. It basis on flyback project and TimeVault. There is a — for some people maybe a little lengthy — video on blip.tv that shows how to install and use it and how straight forward the GUI is.

2 Comments

Ubuntu Linux backup utilities and links

Saturday, 20th Sep 2008 at 22:16 (administration, linux, software)
Tags: , , , , , ,

Resources:

1 Comment

Convert multiple Flash (flv) Videos to, for example, MPEG encoding

Tuesday, 15th Jul 2008 at 22:48 (linux, scripting)
Tags: , , , , , , ,

With ffmpeg and bash this is really easy. Also .flv and .mpeg are easily interchangeable by all other video codecs supported by ffmpeg:

#!/bin/bash
##convert all videos of $srctype type to $desttype type encoding
SRCCODEC="flv"
DESTCODEC="mpeg"
for i in *.$SRCCODEC; do
    echo
    echo -n "###################  "
    echo -n "will convert \"$i\""
    echo "  ###################"
    echo
    ffmpeg -i "${i}" -y "${i%%.${SRCCODEC}}.${DESTCODEC}" && rm -vf "$i"
done

Note that this includes removing the original video file only after a successful recode run overwriting existing destination files (-y option). The interesting part is this: ${i%%.${SRCCODEC}}. It removes the source’s postfix, i.e. file extension. You could save this as a text file, say convert_all_videos_in_pwd.sh and chmod +x filename. I have my own scripts go to ~/bin which I include in $PATH via ~/.bashrc. If you do so you would typically call this within the videos’ directory including logging all output and errors to convert.log, i.e. redirecting STDOUT (= file descriptor 1) and STDERR (= file descriptor 2) via &>:

convert_all_videos_in_pwd.sh &> convert_all.log &

If you want to see the output you could use tail -f logfile or read more on redirection and duplication. You could also do all that in a single command line:

for i in *.flv; do echo converting $i; ffmpeg -i "${i}" -y "${i%%.flv}.mpeg" && rm -vf "$i"; done

Leave a Comment

Opera: X Shared memory extension is not available. ZPixmap not supported

Saturday, 28th Jun 2008 at 23:23 (technical stuff)

It fell out of the skies today that Opera suddenly wouldn’t load any pages, not even feeds. I ran it from command line and got this not very verbose message:

Opera: X Shared memory extension is not available. ZPixmap not supported

So to find anything useful I had to start Firefox… 😉 I did find something not via google but rather right at the source at opera forums. That got me started. I tested if that was the problem by adding the suggested line to /etc/X11/xorg.conf (not without backing up the config file!) and restarted X before installing the brand new driver version as suggested.

X Shared memory extension seams to be available now — no more error message at start up. But still Opera doesn’t load any web content.

Than I moved the preferences folder to a backup location for testing. Now Opera seams to work fine. But what use if all my preferences are lost?!? Anyway, since I don’t have the time to dig in deeper and I cannot find anything on Opera’s forum on this issue I might as well just copy the important preference files over to the “fresh install” and change the rest of my settings manually. After all it’s always a good idea to start over once in a while…

Ressources:

6 Comments

Opera Mini Euro Cup – 2008

Saturday, 28th Jun 2008 at 21:51 (fun stuff)
Tags: , , , ,

Opera Mini Euro Cup – 2008

Leave a Comment

Opera Mini vs. iPhone

Saturday, 28th Jun 2008 at 21:38 (fun stuff)
Tags: , , , , ,

Here is a funny ad I saw the other day:

2 Comments

Emerge clone for Debian-based distributions like Ubuntu, or: Compile your own, dude!

Sunday, 15th Jun 2008 at 17:41 (administration, linux, open source)
Tags: , , , , , ,

There is a nice overview about apt-build, the package I’m talking about here. So I will not say much. Only so far as what to do to try it out. On my system gnomes system manager is fairly slow. So I gave it a try:

  1. install the bundle: 
    sudo aptitude --reinstall install apt-build
  2. configure your processor (dpkg-configure asks you about it)
  3. add deb-src to sources.list if you haven’t already
  4. run it on gnome-system-manager: 
    sudo apt-build install gnome-system-manager

And there you have it. You might want to copy the list of packages that apt-build installs via apt-get build-dep so you can mark them as auto installed using aptitude when done:

sudo aptitude markauto list_of_packages_you_copied_before

or, even easier, use apt-builds –remove-builddep option.

It really does make a difference!

If you’re really keen or you happen to have an older system just wasting away try this:

sudo apt-build world

and see what happens 😉

1 Comment

Compile Spicebird lastest trunk from Subversion (pre 0.7) on Ubuntu Linux (Hardy)

Thursday, 12th Jun 2008 at 21:41 (technical stuff)
Tags: , , , , ,

Spicebird blogs publishedWhat you need to do to get the cutting edge of Spicebird (I wanted to try out the upcoming “blog” feature) is, if you haven’t done any gimmi-the-source-luke-compile-yourself-ing yet, quite a bit. I’ll list it one by one:

  1. Get Spicebird’s sources via subversion (I used subclipse in Aptana but command line shouldn’t be any different in effect)
  2. Since Spicebird builds on mozilla code base just like Thunderbird one needs to get the compatible Mozilla source code. The table in that wiki tells you really what you need (the linked tar ball is the one). I was new to Mozilla coding/compiling so I got a little confused. Subversion told me I had gotten “revision 763” so that matches “702 and above” 🙂
  3. Get all Mozilla dependencies which are quite a bit since I like to use aptitude over apt-get but aptitude doesn’t know the build-dep command here is the full command line including with the list of (Ubuntu) packages I needed: 
    sudo aptitude install autoconf2.13 cdbs diffstat fdupes intltool libart-2.0-dev libatk1.0-dev libaudiofile-dev libavahi-client-dev libavahi-common-dev libavahi-glib-dev libbonobo2-dev libbonoboui2-dev libcairo2-dev libdbus-1-dev libdbus-glib-1-dev  libesd0-dev libexpat1-dev libfontconfig1-dev libfreetype6-dev libgail-dev libgconf2-dev libgcrypt11-dev libglade2-dev  libglib2.0-dev libgnome-keyring-dev libgnome2-dev libgnomecanvas2-dev libgnomeui-dev libgnomevfs2-dev libgnutls-dev  libgnutlsxx13 libgpg-error-dev libgtk2.0-dev libhunspell-dev libice-dev libidl-dev libjpeg62-dev liblzo2-dev libnspr4-dev  libnss3-dev libopencdk10-dev liborbit2-dev libpango1.0-dev libpixman-1-dev libpng12-dev libpopt-dev libpthread-stubs0  libpthread-stubs0-dev libselinux1-dev libsepol1-dev libsm-dev libtasn1-3-dev libx11-dev libxau-dev libxcb-xlib0-dev  libxcb1-dev libxcomposite-dev libxcursor-dev libxdamage-dev libxdmcp-dev libxext-dev libxfixes-dev libxft-dev libxi-dev  libxinerama-dev libxml-xpath-perl libxml2-dev libxrandr-dev libxrender-dev libxt-dev mozilla-devscripts patchutils quilt  sharutils x11proto-composite-dev x11proto-core-dev x11proto-damage-dev x11proto-fixes-dev x11proto-input-dev x11proto-kb-dev  x11proto-randr-dev x11proto-render-dev x11proto-xext-dev x11proto-xinerama-dev xtrans-dev xulrunner-1.9-dev zlib1g-dev libdbus-glib-1-dev curl libcurl4-openssl-dev automake make build-essential cvs

    To double-check run sudo apt-get build-deb firefox. If you would use aptitude normally you should cancel this command (Ctrl+c) after apt-get has shown you the necessary packages. Copy-paste the list (might need manual editing to remove line breaks) to sudo aptitude install … If you would like to keep this bunch of packages only to compile spicebird run the same command after you’re done compiling but with markauto in place of install. This will remove all packages that are only installed to satisfy this build-dependency.

  4. Now you should really have everything you need. Now move spicebird’s trunk folder as “collab” into the mozilla folder.
  5. Then you need to create your .mozconfig file in your home directory. In the suggested line 
    echo ". ${topsrcdir}/collab/config/mozconfig" > $HOME/.mozconfig

    ${topsrcdir} must be replaced by the absolute path to the newly created mozilla folder.

  6. Now everything should be set to actually compile the sources as suggested in Spicebird’s wiki. Be prepared to let it run for quite some time (> 1 hour depending on CPU speed and memory; that’s why I have time to write this post 😉 ); the code uses about 500 MB disk space.

If you are interested in building a deb package have a look at inkscape’s nice walk-through. Mind you it’s written for Inkscape. See also: how to compile deb packages by hand.

Ressources

2 Comments

« Older entries