Here is a list of ways how to rename multiple files with Linux. The easies and quickest way I found was to use ‘rename’.
Batch Rename Multiple Files with Linux
Saturday, 13th Dec 2008 at 00:07 (administration, linux)
Tags: batch, file, linux, multiple files, rename
Convert multiple Flash (flv) Videos to, for example, MPEG encoding
Tuesday, 15th Jul 2008 at 22:48 (linux, scripting)
Tags: bash, convert, for, for loop, linux, script, scripting, videos
With ffmpeg and bash this is really easy. Also .flv and .mpeg are easily interchangeable by all other video codecs supported by ffmpeg:
#!/bin/bash
##convert all videos of $srctype type to $desttype type encoding
SRCCODEC="flv"
DESTCODEC="mpeg"
for i in *.$SRCCODEC; do
echo
echo -n "################### "
echo -n "will convert \"$i\""
echo " ###################"
echo
ffmpeg -i "${i}" -y "${i%%.${SRCCODEC}}.${DESTCODEC}" && rm -vf "$i"
done
Note that this includes removing the original video file only after a successful recode run overwriting existing destination files (-y option). The interesting part is this: ${i%%.${SRCCODEC}}. It removes the source’s postfix, i.e. file extension. You could save this as a text file, say convert_all_videos_in_pwd.sh and chmod +x filename. I have my own scripts go to ~/bin which I include in $PATH via ~/.bashrc. If you do so you would typically call this within the videos’ directory including logging all output and errors to convert.log, i.e. redirecting STDOUT (= file descriptor 1) and STDERR (= file descriptor 2) via &>:
convert_all_videos_in_pwd.sh &> convert_all.log &
If you want to see the output you could use tail -f logfile or read more on redirection and duplication. You could also do all that in a single command line:
for i in *.flv; do echo converting $i; ffmpeg -i "${i}" -y "${i%%.flv}.mpeg" && rm -vf "$i"; done
How to Move VirtualBox’s Guest Hard Drives to Another (Physical) Location
Saturday, 26th Apr 2008 at 15:22 (administration, linux)
Tags: linux, vbox, vdi, virtual machine, virtualbox
If you ever wanted to move your (very large) file that contains a Virtualbox’s guest hard drive (file extension is .vdi) to another location and did so just like that Virtualbox wouldn’t find it the next time you wanted to boot into your guest system. The only thing Virtualbox needs to be pointed to is the new location of that file.
The standart location of VDI files is (for Linux) ~/.VirtualBox/VDI/ (see global settings, where you could also change this if that’s all you want). Remember the old file name and close Virtualbox, just in case it would overwrite settings when closed after the following changes. Now let’s move one file to, say, /media/USBDrive/virtualbox/. Now do:
- Use your favorite editor to open
~/.VirtualBox/Virtualbox.xml - Find the line that says something like:
<VirtualDiskImage filePath="VDI/old file name.vdi"/>
- Change it to point to your new location. So it should read something like:
<VirtualDiskImage filePath="/media/USBDrive/new file name.vdi"/>
- Done.
Making Apache server use SSL only
Tuesday, 8th Apr 2008 at 14:02 (administration, linux, open source)
Tags: apache, cms, https, linux, secure, ssl, web server
Here is a forum post on how to make all incoming connections to your apache web server redirect to https, i.e. take encripted connections via ssl by using apache’s rewrite module.
Recover Data and (deleted) Partition with Linux from Hard Drives, CD-ROMs or DVDs
Saturday, 5th Jan 2008 at 17:09 (linux)
Tags: chkdsk, file system, GNU ddrescue, hachoir, linux, lost partition, ntfs, partition recovery, partition table, restore data, system tools, testdisk
One important thing to know before doing anything else:
Every byte written to partitions in question will decrease chances to get data back since deletion of files and even partitions does not automatically mean actually deleting that data on disk. In most cases it only means deleting administrative entries in file or partition tables. But writing new files or partitions on the very same disk might actually overwrite data blocks containing pieces associated with files still intact! The second thing you might want to consider: If you mistakenly deleted single files or even a bunch of them look somewhere else but better not reboot until you really know why you have to do so as rebooting in those cases will sometimes also mess up chances to get back your data. Here I will only handle partition or hard drive errors.
So the very first step will be to get hold of extra storage (like USB drives), boot up some Live CD, e.g. your Ubuntu installation cd. If you have liability to paranoia you might even want to physically detach your hard drive and use some USB case to access it instead. First of all do something like
ddrescue -vr-1 /dev/sda recovered_data_blocks.img gddrescue.log
using gddrescue. Adjust parameters to your needs, especially /dev/sda (complete drive) or /dev/sda1 (partition in question). For Ubuntu you need to install gddrescue (= GNU ddrescue) package first. gddrescue will create an image trying hard to copy every data block still intact so you can, at any point, restore the drive stage where you first discovered you lost data. This, or course, can be used with any block-oriented medium such as CD-ROMs or DVDs. You might want to specify “-b 2048″ for CDs. Also use -n to get the most data on first run. For subsequent attempts you might want -rX where X is the number of retries for erroneous areas. I recommend to use -vr-1 for verbose infinite retries. gddrescue first reads as many as possible intact blocks and then splits error areas with each subsequent run. Note though for partitions or even complete hard drives it will take hours so be patient and make sure to use gddrescue’s log file option! With other tools I tried it took even more time which makes sense since gddrescue reads only blocks that really need to be read.
Second you should — just in case — save your partition geometries with fdisk (output similar):
$ sudo fdisk -ls Password: Disk /dev/hda: 160.0 GB, 160041885696 bytes 255 heads, 63 sectors/track, 19457 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/hda1 * 1 9145 73457181 b W95 FAT32 /dev/hda2 9146 19106 80011732+ 83 Linux /dev/hda3 19107 19457 2819407+ 5 Extended /dev/hda5 19107 19457 2819376 82 Linux swap / Solaris Disk /dev/hdb: 160.0 GB, 160041885696 bytes 255 heads, 63 sectors/track, 19457 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System Disk /dev/hdc: 20.8 GB, 20847697920 bytes 255 heads, 63 sectors/track, 2534 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/hdc1 * 1 2427 19494846 83 Linux /dev/hdc2 2428 2534 859477+ 5 Extended /dev/hdc5 2428 2534 859446 82 Linux swap / Solaris
After ddrescue has finished try TestDisk for lost partitions which, among others, looks for back up partition tables, inode tables, and the like. To restore plain data there are heaps of tools. Most notably foremost and PhotoRec. Note though that for lost partitions there is no way to really know what file type found blocks belong to or even the name of associated files’ paths, at least as far as I can imagine. File names really are — in all file systems I know — only attributes of a data segment called file (or inode) such as access date, ownership and such. Sometimes file names are even stored in special files (directories or folders) which means there data blocks need to be intact and it has to be known where they are! The scenario I ran into was I mistakenly installed Ubuntu on top of Windows XP (NTFS partition) instead of resizing the partition as intended. TestDisk even though listing a NTFS partition (from it’s back-up MFT located at the end of NTFS partitions) couldn’t however restore the partition as such. Running Microsoft’s chkdsk stored a bunch of files with their filenames several folders into one called found.000 (I ran it inMicrosoft’s recovery command line from CD after manually through fdisk changed partition type to NTFS/HPFS, e.g. x07). Using the above mentioned tools I could restore some more but without there original path or file names. All together I recovered ~40 GB out of probably 60 GB data (which included system files).
Resources
- Tincho’s Debian Package of the Day: gddrescue
- Pat Patterson’s article on Ubuntu hard drive recovery (stating John Gilmore to suggest GNU shred instead of /dev/urandom to securely shred data on dead hard drives)
- Find deleted or lost partitions using testdisk
- Ubuntu Wiki on Data Recovery
- On toad.com: “Disk drive recovery“
- Heaps of info on filesystems and mounting (especially section about FS checking) as part of a larger site about Ubuntu/Windows dual-boot set-ups; has a section on TestDisk with illustrated walk-throughs
- hachoir-subfile is a tool based on hachoir-parser to find subfiles in any binary stream.
- About damaged Superblocks (extfs based), what superblocks are and how to use backups of them stored throughout the partition
- From German Ubuntu User Wiki: “Notfall“, “Festplatten Problembehebung“
- From German Linuxwiki: “verlorene Partition finden“
- man pages from ntfsprogs in Linux-NTFS Wiki (ntfsck is not ready, yet)
Further reading:
- On a side note: Why *nix FS don’t need defragmentation and on Fragmentation and Unix file systems
- Another side note: Ubuntu Forum entry on the various mount/fstab options
- Easiest Way To Restore Deleted FAT and NTFS Logical Drives
Ubuntu System Panel (aka USP2 or USP3)
Tuesday, 30th Oct 2007 at 23:04 (ubuntu)
Tags: eye candy, linux, system panel, ubuntu
Like what you see?
Now?
Well, anyway, hit the Ubuntu forums or google code base of this project to just get it and try it. For a couple more snapshots there is another thread.
How to find out what occupies space on your Linux hard drive
Sunday, 30th Sep 2007 at 20:42 (administration, linux, ubuntu)
Tags: apt, bash, deb, linux, system, ubuntu
The other day I noticed that my settings directory (/etc) uses over 13 MB of my hard drive. So I wandered which package (I’m using a Debian based package managed system) makes the settings directory grow so large. After a couple of trails and errors I came up with the following sequence of commands:
$ du -h --max-depth=1 /etc 2> /dev/null | egrep '(^[5-9][0-9]{2}K)|M'
692K /etc/X11
672K /etc/acpi
712K /etc/xdg
2.1M /etc/brltty
500K /etc/ssl
528K /etc/mono
20K /etc/NetworkManager
13M /etc
$ dpkg -S '/etc/brltty'
brltty-x11, brltty: /etc/brltty
$ apt-cache show brltty | grep -A5 'Description'
Description: Access software for a blind person using a soft braille terminal
BRLTTY is a daemon which provides access to the Linux console (text mode)
for a blind person using a soft braille display. It drives the braille
terminal and provides complete screen review functionality.
The following display models are supported:
* Alva (ABT3xx/Delphi)
Fortunatelly, I’m not blind so I could remove brltty with aptitude which then suggested to remove dependencies, too.
References:
- Regex reference
- Resources for advanced Ubuntu topics, eg. how to remove …-desktop meta packages with apt-get (instead of aptitude), secure networking setup, etc.
Let Bash do the math: Doing calculations using that bash
Sunday, 26th Aug 2007 at 18:52 (linux)
Tags: arithmetics, bash, calc, calculator, cml, command line, linux, math, scripting
Have you ever wanted to “just do some math” without much fuss? I noticed a while ago (but haven’t really used it since) that bash (born again shell) has, amongst many other evaluations, an arithmetic one. Using the syntax $((expression)) you can evaluate the expression to arithmetically. Operations allowed are quite a few. From man bash:
ARITHMETIC EVALUATION
The shell allows arithmetic expressions to be evaluated, under certain circumstances (see the let and declare builtin commands and Arithmetic Expansion). Evaluation is done in fixed-width integers with no check for overflow, though division by 0 is trapped and flagged as an error. The operators and their precedence, asso ciativity, and values are the same as in the C language. The following list of operators is grouped into levels of equal-precedence operators. The levels are listed in order of decreasing precedence.id++ id-- variable post-increment and post-decrement ++id --id variable pre-increment and pre-decrement - + unary minus and plus ! ~ logical and bitwise negation ** exponentiation * / % multiplication, division, remainder + - addition, subtraction <> left and right bitwise shifts = comparison == != equality and inequality & bitwise AND ^ bitwise exclusive OR | bitwise OR && logical AND || logical OR expr?expr:expr conditional operator = *= /= %= += -= <>= &= ^= |= assignment expr1 , expr2 commaShell variables are allowed as operands; parameter expansion is performed before the expression is evaluated. Within an expression, shell variables may also be referenced by name without using the parameter expansion syntax. A shell variable that is null or unset evaluates to 0 when referenced by name without using the parameter expansion syntax. The value of a variable is evaluated as an arithmetic expression when it is ref erenced, or when a variable which has been given the integer attribute using declare -i is assigned a value. A null value evaluates to 0. A shell variable need not have its integer attribute turned on to be used in an expression.
Constants with a leading 0 are interpreted as octal numbers. A leading 0x or 0X denotes hexadecimal. Other wise, numbers take the form [base#]n, where base is a decimal number between 2 and 64 representing the arith metic base, and n is a number in that base. If base# is omitted, then base 10 is used. The digits greater than 9 are represented by the lowercase letters, the uppercase letters, @, and _, in that order. If base is less than or equal to 36, lowercase and uppercase letters may be used interchangeably to represent numbers between 10 and 35.
all posts
