Univ 1 recovery procedure

/univ_1 Recovery script

After some investigation we found a way to recover the "?"-files. This can (and we recommend it) be done by the user. The script can be found in /opt/hlrs/tools/recover_univ_1. Here we describe what the script will do to recover your data.

#!/bin/bash
user=$(whoami)

echo "Usage: recover_univ1 <DIRECTORY>"

if [ $# -eq 0 ]
then
     echo "Please type in the directory where you want to recover you data: /univ_1/ws1/ws/$user/...."
     read mynode
else
     mydir=$1
fi

#Creating the temp. file.
tempfile=`mktemp`
usertemp=`mktemp`
echo "That is the outputfile: $tempfile, you can run 'tail -f $tempfile' to see the progress."
read -p "Press [Enter] to run 'du -s' now! This can take a very long time (maybe even hours) depending on the number of files"

#Running the du -s command to find the corrupted files. Putting the output to a file.
du -s $mydir >> $tempfile 2>&1

#Correcting the output of du -s to remove not needed signs and blanks etc.
cat $tempfile | sed -e "s/du: cannot access//g" -e "s/': No such file or directory//g" -e 's/^[ \t]*//;s/[ \t]*$//' -e  's/^.//' -e '$d' > $usertemp

#Removing the temp. file
rm -rf $tempfile
echo "The output can be found in $usertemp"

#Reading the output of $usertemp to run the dd command
for i in $(cat $usertemp); do
        if [ ! -e "${i}_recover" ]; then
                dd if="$i" of="${i}_recover" bs=1
                if [ $? -ne 0 ]; then
                        echo "dd command failed please contact HLRS using the mailaddress: univ_1_recovery@hlrs.de<mailto:univ_1_recovery@hlrs.de>"
                        echo $i >> $failed
                        continue
                fi
                rm -f "${i}"
                mv "${i}_recover" "${i}"
        else echo "A file ${i}_recover already exists! Please rename it an start the script again"
        fi
done
mv $usertemp $HOME/recovered_files.$$
echo "Recover finished. Permissions of the recovered files has been set to default! Please check! The creation date of the files has also changed. 
You can find the list of recovered files in here: $HOME/recovered_files.$$. Files which were not recoverable are stored in $HOME/failed_files.$$"

The usage of the script is:

/opt/hlrs/tools/recover_univ_1 <DIRECTORY>

For example if you have broken files in /univ_1/perm/hpcander/ISV/tools/software/BZIP2/1.0.2/aix_4.3-xlc_4.3 you would use the script:
/opt/hlrs/tools/recover_univ_1 /univ_1/perm/hpcander/ISV/tools/software/BZIP2/1.0.2/aix_4.3-xlc_4.3

The script will run du -s through this directory and will look for "?"-files. The list will be stored in a temporary file. Once this is done, we start the actual recovery by using the command:

dd if=<damaged files from the list> of==<damaged files from the list_recover> bs=1

The most important option is bs=1. dd will then start to read from the first OST where the file was located and will go on until the file is back. Once this is done the script will remove the "?"-files and copy the *_recover file to the original name. A logfile will be stored in your home directory (~username/recovered_files.XX) where you can see which files have been restored.

Please be aware that the permissions are set to default, so you have to change the permissions back to whatever is good for you.

It is still possible, that some files cannot be recovered with that option. If you find file which cannot be recovered please use univ_1_recovery@hlrs.de and report us the files.