arranging files

kzo81 · 11-08-2016, 12:18 PM

Hi Folks,

I have an Axis video server that produces mkv files, but those mkv files are deeply scattered in subdirectories. There is a pattern in the filenames, each file begins with the date: 20111108_XYZA_121212.mkv , 20111109_XYDA_121A32.mkv etc.

I'd like to write a bash script that does 2 thing:

- makes directories: 20111108, 20111109
- moves files into the corresponding directory

I don't know the good approach for this job. Could you give me some ideas?

I've come across this so far to find out the directory names, but I don't know how to assign this into an array:

find . -type f -name "*.mkv" | grep -Eo '[0-9]{8}' | sort | uniq

This arranging function is buggy and not elegant:

https://github.com/kerzol81/Bash-and...nger_from_axis

Thanks,

Zoli

rtmistler · 11-08-2016, 12:35 PM

Some recommendations:

Please explain your current experience with BASH scripting for the benefit of those looking to assist you.
Please make an attempt at writing a BASH script to start with your problem and post where you are with this.
If you have no BASH scripting experience, there are several links in my signature which can help you to get started, go more advanced, and assist you with debugging BASH scripts.

Your problem definition is fairly clear.

Note which commands you would use in a terminal to accomplish what you want the script to do.

Follow my personal, golden BASH script rule:

Quote:

Anything commands you can type in a command line, you can write into a BASH script.

schneidz · 11-08-2016, 12:37 PM

mkdir, find -exec mv ...

kzo81 · 11-08-2016, 01:18 PM

basically this one does the job, but I think this is still buggy:

#!/bin/bash

LOCAL_FOLDER="$HOME/axis"

TO_FOLDER="$HOME/axis_arranged"

EXTENSION='mkv'

function arranging(){

mkdir -p $TO_FOLDER
cd $TO_FOLDER || exit 1

for i in `find $LOCAL_FOLDER -name *.$EXTENSION | grep -Eo '[0-9]{8}' | sort | uniq`;
do
if [ ! -d $i ]; then
mkdir -p $i
mv -u `find $LOCAL_FOLDER -name $i*.mkv` $i
else
mv -u `find $LOCAL_FOLDER -name $i*.mkv` $i
fi

done
}

function main(){

arranging
}

main

szboardstretcher · 11-08-2016, 01:21 PM

Seems overly complicated. I don't see a reason to 'sort' them.

I did not use 'mv' to MOVE the files, because I always thing it is best to COPY them first, then delete the old when you are done. So I used 'rsync' which gives you a status indicator.

This script will look in /working for any files with extension mkv, grab the 8-digit number from them, make the 8-digit number directory, then copy the file to that directory.

Feel free to modify it to MOVE the files, but beware.

Code:

#!/bin/bash

for f in $(find /working -type f -name *.mkv); do

newdir=$(echo "$f" | grep -Eo '[0-9]{8}')

if [ ! -d "/workingnew/$newdir" ]; then
 mkdir -p /workingnew/$newdir
fi

rsync -vah --progress $f /workingnew/$newdir/

done

kzo81 · 11-08-2016, 01:58 PM

Thank you. Rsync is great. Much better that what I came up with. This way if source file gets overwritten, rsync increments.
But this way I'll need double space :-) Anyway, thank you for your help.

Sefyir · 11-08-2016, 05:34 PM

You can break apart strings in bash. In this case, something like "${i%%_*}" will grab the first part of the file

eg

Code:

a="20111108_XYZA_121212.mkv"
echo "$a" "${a%%_*}" 
20111108_XYZA_121212.mkv 20111108

Allowing you to do something like this:

Code:

new_dir=foobar; while read i; do echo mkdir -p "$new_dir${i%%_*}"; echo cp "$i" "$new_dir""${i%%_*}$i"; done < <(find $PWD -type f -iname '*_*.mkv')

Formatted

Code:

new_dir=foobar
while read i
  do
    echo mkdir -p "$new_dir${i%%_*}"
    echo cp "$i" "$new_dir""${i%%_*}$i"
done < <(find $PWD -type f -iname '*_*.mkv')

After you verify the file/dirnames are correct, remove the echos

allend · 11-09-2016, 06:41 AM

Just for fun, an alternative bash solution.

Code:

#!/bin/bash

EXTENSION=".mkv"
LOCAL_FOLDER="$HOME/axis"
TO_FOLDER="$HOME/axis_arranged"

function arranging(){

mkdir -p $TO_FOLDER
cd $LOCAL_FOLDER || exit 1

shopt -s globstar
for i in **/*$EXTENSION; do
  [[ "$i" =~ ([[:digit:]]+)_.+$EXTENSION ]] \
    && [ ! -d "$TO_FOLDER/${BASH_REMATCH[1]}" ] \
    && mkdir -p "$TO_FOLDER/${BASH_REMATCH[1]}"
  cp -u  "$i"  "$TO_FOLDER/${BASH_REMATCH[1]}"
done
shopt -u globstar
}

arranging

kzo81 · 03-19-2017, 03:45 PM

thanks