LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 03-18-2017, 03:43 PM   #16
kzo81
Member
 
Registered: Aug 2014
Location: Hungary
Distribution: Debian, Linux Mint, CentOS
Posts: 174

Original Poster
Rep: Reputation: Disabled
Smile


Hi:-)

Now I have a working code:

Code:
#!/bin/sh

echo "[+] Creating symbolic links ..."

SD="/var/spool/storage/SD_DISK/"
ARRANGED_FOLDER="$SD/videos_by_day"

while read i
do

DAY=$(echo "$i" | cut -d'/' -f10 | cut -d'_' -f1 | sort | uniq)
	
	if [ ! -d "/$ARRANGED_FOLDER/$DAY" ]; then
		mkdir -p /"$ARRANGED_FOLDER"/"$DAY"
	fi
	
	cd "$ARRANGED_FOLDER"/"$DAY" || exit 1
	
	if [ `find "$ARRANGED_FOLDER" -samefile $i | wc -l` eq 0 ]; then 
		ln -sfn "$i"
	fi

done < $(find "$SD" -type f -name "*.mkv")
There is still some bug in it, since it complained for only one file:

/var/spool/storage/SD_DISK/20170215/09/20170215_093234_3E4A_ACCC8E565EC5/20170215_09/20170215_093234_29AF_ACCC8E565EC5.mkv: File name too long

It seems that now the script only creates symlinks of the newly arrived files.

Quote:
is that "suppose" to give you a file name? because it comes up empty when I run it. just run this simple version with a path you give it to search. any path will work it is not changing any data.
Yes, it gives me the day, which will be the folder to create later.
Thank you so much for the efforts you all make for me.
 
Old 03-19-2017, 06:19 AM   #17
kzo81
Member
 
Registered: Aug 2014
Location: Hungary
Distribution: Debian, Linux Mint, CentOS
Posts: 174

Original Poster
Rep: Reputation: Disabled
Guys, I did some tests on the video server. And still the for cycle version gives me the least error message.
The one that I created:
Code:
#!/bin/sh

SD="/var/spool/storage/SD_DISK/"
ARRANGED_FOLDER="/var/spool/storage/SD_DISK/videos_by_day"

for i in $(find "$SD" -type f -name "*.mkv"); do

DAY=$(echo "$i" | cut -d'/' -f10 | cut -d'_' -f1 | sort | uniq)
	
	if [ ! -d "/$ARRANGED_FOLDER/$DAY" ]; then
		mkdir -p /"$ARRANGED_FOLDER"/"$DAY"
	fi
	
	cd "$ARRANGED_FOLDER"/"$DAY"
	
	if ! ln -sfn "$i"; then
		echo "[-] Error: Could not create symbolic links"
	fi
done
The While version, this one:
Code:
#!/bin/sh


SD="/var/spool/storage/SD_DISK/"
ARRANGED_FOLDER="/var/spool/storage/SD_DISK/videos_by_day"


while read i; do

DAY=$(echo "$i" | cut -d'/' -f10 | cut -d'_' -f1 | sort | uniq)
	
	if [ ! -d "/$ARRANGED_FOLDER/$DAY" ]; then
		mkdir -p /"$ARRANGED_FOLDER"/"$DAY"
	fi
	
	cd "$ARRANGED_FOLDER"/"$DAY"
	
	if ! ln -sfn "$i"; then
		echo "[-] Error: Could not create symbolic links"
	fi
done < `find "$SD" -type f -name "*.mkv"`
Complains:
Quote:
+ DAY=�{ckא�A6�S������6�?i�x:�(�d�.QF�����1<�A�ޙ��'>1`z��IG����Z��*r�+!Juu����m)T��,�O�l�U6{9P
+ [ ! -d //var/spool/storage/SD_DISK/videos_by_day/�{ckא�A6�S������6�?i�x:�(�d�.QF�����1<�A�ޙ��'>1`z��IG����Z��*r�+!Juu����m)T��,�O�l�U6{9P ]
+ cd /var/spool/storage/SD_DISK/videos_by_day/�{ckא�A6�S������6�?i�x:�(�d�.QF�����1<�A�ޙ��'>1`z��IG����Z��*r�+!Juu����m)T��,�O�l�U6{9P
I know that shellcheck website, and You suggest me not using a for loop for, but as you can see I don't get any error message when I use the for loop version.

The only think that still bothers me, that I still couldn't implement a check that checks if an original file has a symlink already.
That command substitution is way complicated for me, I checked your suggested code, with the corrected if clause and it is doing terrible things, I mean this code:

Code:
SD="/var/spool/storage/SD_DISK/"
ARRANGED_FOLDER="/var/spool/storage/SD_DISK/videos_by_day"

while read FILENAME
do

 c=$FILENAME
 xpath=${c%/*} 
 xbase=${c##*/}
 
 xfext=${xbase##*.}
 xpref=${xbase%.*}
 
 path=${xpath}
 fname=${xpref}
 ext=${xfext}

ckfile="$fname"."$ext"

DAY=$(echo "$FILENAME" | cut -d'/' -f10 | cut -d'_' -f1 | sort | uniq)
	
	DailyFiles="$ARRANGED_FOLDER"/"$DAY"
        mkdir -p "$DailyFiles" 

        # check to see if in directory already / which it should not be?
[[ -h "$DailyFiles/$ckfile" ]] || ln -sfn $FILENAME "$DailyFiles/$ckfile"

done < $(find "$SD" -type f -name "*.mkv")
output:
Quote:
ln: /var/spool/storage/SD_DISK/videos_by_day/�h�c��t�e�c�W�뙁�@ø:�yw�� ig&�f�ȯ���J��.E��i�I�S4�`{�;�A
!kr����/�h�c��t�e�c�W�뙁�@ø:�yw�� ig&�f�ȯ���J��.E��i�I�S4�`{�;�A
�QI��O�2�e#f��L9�Hk���̀,�s���ȷ��[ !kr����_gg6�o-!���^������
-`���C�W���=H�����I�wo�9[�?(zT�b���� zK��Kr��?n��p��00Ѹ�������
��6�S%A�: File name too long

and if I put set -x in the script:

+ xbase=_B�X����Ѕ��I*�H�3ar�$��0�q���������^�Gߓ=GE��׹��![���(��?��Dx�E��=�7;���x�p���q�^Wa�ܟ����v�7n
2���x��+����.��sp;���516�sR��m:���
z���?������T)�eB�:�t��C������F�V#.�5 ��}�Һ��׾���ӵ|�=�(�u'����
g��{�
+ xfext=�5 ��}�Һ��׾���ӵ|�=�(�u'����
I really appreciate your help, but the problem of - to much SD_DISK write, when rerun the script - is still there.

I noticed that if I omit the -fn arguments, it gives me an error, but at least the script didn't recreate the link:

Code:
#!/bin/sh

SD="/var/spool/storage/SD_DISK/"
ARRANGED_FOLDER="/var/spool/storage/SD_DISK/videos_by_day"

for i in $(find "$SD" -type f -name "*.mkv"); do

DAY=$(echo "$i" | cut -d'/' -f10 | cut -d'_' -f1 | sort | uniq)
	
	if [ ! -d "/$ARRANGED_FOLDER/$DAY" ]; then
		mkdir -p /"$ARRANGED_FOLDER"/"$DAY"
	fi
	
	cd "$ARRANGED_FOLDER"/"$DAY"

	ln -s "$i" >/dev/null 2>&1
done

exit 0
So at least it's a workaround for now.

Last edited by kzo81; 03-19-2017 at 07:23 AM.
 
Old 03-19-2017, 10:08 AM   #18
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,090

Rep: Reputation: 7135Reputation: 7135Reputation: 7135Reputation: 7135Reputation: 7135Reputation: 7135Reputation: 7135Reputation: 7135Reputation: 7135Reputation: 7135Reputation: 7135
I do not really understand where those strange chars are coming from. Are they filenames or something else?
if the for cycle and while loop work differently that means the while is ok and the for loop makes something wrong.
if
Quote:
+ DAY=�{ckא�A6�S������6�?i�x:�(�d�.QF�����1<�A�ޙ��'>1`z��IG����Z��*r�+!Juu����m)T��,�O�l�U6{9P
that means calculating the DAY is not ok.
 
Old 03-19-2017, 02:22 PM   #19
BW-userx
LQ Guru
 
Registered: Sep 2013
Location: Somewhere in my head.
Distribution: Slackware (15 current), Slack15, Ubuntu studio, MX Linux, FreeBSD 13.1, WIn10
Posts: 10,342

Rep: Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242
I got a ask you do you know what a soft link basically is? It is just a fake copy of the same file pointing back to the original file. it looks feels and smells just like the original one. Q2: so how do you check to see if a file is present within a directory? apply the same methodology.

-L FILE
FILE exists and is a symbolic link (same as -h)

Code:
if [[ -L "$strFile" ]] && [[ -a "$strFile" ]];then 
  echo "'$strFile' link present"; 
fi
that bit of code I gave you that you didn't understand I. That slices up the complete path to a file and breaks it down to path to file - file name - extension.
 
1 members found this post helpful.
Old 03-19-2017, 02:41 PM   #20
BW-userx
LQ Guru
 
Registered: Sep 2013
Location: Somewhere in my head.
Distribution: Slackware (15 current), Slack15, Ubuntu studio, MX Linux, FreeBSD 13.1, WIn10
Posts: 10,342

Rep: Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242
you do not need all of this code
Code:
	if [ ! -d "/$ARRANGED_FOLDER/$DAY" ]; then
		mkdir -p /"$ARRANGED_FOLDER"/"$DAY"
	fi
Just this will work because mkdir does the check to see if a directory by that name is already there for you, all you need to do is add just this line.
Code:
mkdir -p /"$ARRANGED_FOLDER"/"$DAY"
what that leading / slash is for I do not know. I am surprised it works. Because if it is for a root dir on your SD Card then that should be within your variable name instead. just coding styles
Code:
ARRANGED_FOLDER="/root-new-whatever-dir"
example
Code:
userx@voider⚡️~⚡️$ newdir=/tmp
userx@voider⚡️~⚡️$ ls $newdir
cron.1FMpX9
userx@voider⚡️~⚡️$ ls /tmp
cron.1FMpX9

Last edited by BW-userx; 03-19-2017 at 02:49 PM.
 
Old 03-19-2017, 02:47 PM   #21
kzo81
Member
 
Registered: Aug 2014
Location: Hungary
Distribution: Debian, Linux Mint, CentOS
Posts: 174

Original Poster
Rep: Reputation: Disabled
Hi, I know what is a softlink. Though I had rarely used it until this problem emerged with this axis video server.
I also understand the test tool. As I mentioned, I achieved the effect what I wished by omiting arguments (ls -sfn).
I think I'll rework the code tomorrow to a WHILE cycle, I don't really understand why it doesn't work the same as the FOR version of the code.

This is wrong I, know, but it's working:
Code:
#!/bin/sh

SD="/var/spool/storage/SD_DISK/"
ARRANGED_FOLDER="/var/spool/storage/SD_DISK/videos_by_day"

for i in $(find "$SD" -type f -name "*.mkv"); do

DAY=$(echo "$i" | cut -d'/' -f10 | cut -d'_' -f1 | sort | uniq)
	
	if [ ! -d "/$ARRANGED_FOLDER/$DAY" ]; then
		mkdir -p /"$ARRANGED_FOLDER"/"$DAY"
	fi
	
	cd "$ARRANGED_FOLDER"/"$DAY"

	ln -s "$i" >/dev/null 2>&1
done

exit 0
But this doesn't work:
Code:
#!/bin/sh

set -x 

SD="/var/spool/storage/SD_DISK/"
ARRANGED_FOLDER="/var/spool/storage/SD_DISK/videos_by_day"

while read i; do

DAY=$(echo "$i" | cut -d'/' -f10 | cut -d'_' -f1 | sort | uniq)
	
	if [ ! -d "/$ARRANGED_FOLDER/$DAY" ]; then
		mkdir -p /"$ARRANGED_FOLDER"/"$DAY"
	fi
	
	cd "$ARRANGED_FOLDER"/"$DAY"
	
	ln -s "$i" >/dev/null 2>&1

done < `find "$SD" -type f -name "*.mkv"`
This gives an error:
Quote:
/var/spool/storage/SD_DISK/20170319/10/20170319_102601_27E2_ACCC8E565EC5/20170319_10/20170319_102601_2F6F_ACCC8E565EC5.mkv: no such file
The FOR version runs seamless. I don't know why. That file doesn't exist on the original place, only the While version of the code finds it...

Last edited by kzo81; 03-19-2017 at 04:07 PM.
 
Old 03-19-2017, 02:56 PM   #22
BW-userx
LQ Guru
 
Registered: Sep 2013
Location: Somewhere in my head.
Distribution: Slackware (15 current), Slack15, Ubuntu studio, MX Linux, FreeBSD 13.1, WIn10
Posts: 10,342

Rep: Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242
this
Code:
[[ -h "$DailyFiles/$ckfile" ]] || ln -sfn $FILENAME "$DailyFiles/$ckfile"
can be written like this too
if file link not there then create one.
Code:
[[ ! -h "$DailyFiles/$ckfile" ]] && ln -sfn $FILENAME "$DailyFiles/$ckfile"
-s, --symbolic
make symbolic links instead of hard links

is this really needed? you're logic is this - if link not there then make one else skip - right?
-f, --force
remove existing destination files


-n, --no-dereference
treat destination that is a symlink to a directory as if it were a normal file
 
Old 03-19-2017, 03:00 PM   #23
BW-userx
LQ Guru
 
Registered: Sep 2013
Location: Somewhere in my head.
Distribution: Slackware (15 current), Slack15, Ubuntu studio, MX Linux, FreeBSD 13.1, WIn10
Posts: 10,342

Rep: Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242
why are you going into the directory?
Code:
cd "$ARRANGED_FOLDER"/"$DAY"
all of your paths are already set to work outside of directory regardless if your inside it or not. so what is the point of being inside of it? just trying to understand your logic.
 
Old 03-19-2017, 04:02 PM   #24
kzo81
Member
 
Registered: Aug 2014
Location: Hungary
Distribution: Debian, Linux Mint, CentOS
Posts: 174

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by BW-userx View Post
why are you going into the directory?
Because if I dont, it'll create the symlinks in the directory where the script is. If you look at the man page of ln, see the 2nd form. It says that : "create a link to TARGET in the current directory".

If I only use ln -s, it wont recreate the symlink which is already created before although it throughs an error, which I redirect to the black-hole:-)

Last edited by kzo81; 03-19-2017 at 04:16 PM.
 
Old 03-19-2017, 04:45 PM   #25
BW-userx
LQ Guru
 
Registered: Sep 2013
Location: Somewhere in my head.
Distribution: Slackware (15 current), Slack15, Ubuntu studio, MX Linux, FreeBSD 13.1, WIn10
Posts: 10,342

Rep: Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242
Quote:
Originally Posted by kzo81 View Post
Because if I dont, it'll create the symlinks in the directory where the script is. If you look at the man page of ln, see the 2nd form. It says that : "create a link to TARGET in the current directory".
I do not specify remember if I've ever ran a ln -s like this outside of both directories before on cli - but I do think I have. I got a give that a try and see the results on that one.


Quote:
Originally Posted by kzo81 View Post
If I only use ln -s, it wont recreate the symlink which is already created before although it throughs an error, which I redirect to the black-hole:-)
I understand the -s it is the other two I was having issues with, specifically -f forcing it to link when there is no link to begin with. Just being cautious ??
 
Old 03-19-2017, 04:55 PM   #26
kzo81
Member
 
Registered: Aug 2014
Location: Hungary
Distribution: Debian, Linux Mint, CentOS
Posts: 174

Original Poster
Rep: Reputation: Disabled
You guys told me about string manipulation.
Do you know an elegant way of extracting the red part of this?

i="/var/spool/storage/SD_DISK/20170319/10/20170319_102601_27E2_ACCC8E565EC5/20170319_10/20170319_102601_2F6F_ACCC8E565EC5.mkv"

something like this: echo "${i%%_*.mkv}"
 
Old 03-19-2017, 05:00 PM   #27
kzo81
Member
 
Registered: Aug 2014
Location: Hungary
Distribution: Debian, Linux Mint, CentOS
Posts: 174

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by BW-userx View Post
it is the other two I was having issues with, specifically -f forcing it to link when there is no link to begin with. Just being cautious ??
first I thought that ln -sf will help me, but it didnt. Same goes with ln -sfn.
And interestingly many tools are not available on this videoserver. For instance there is no awk, but sed with no possibility of using arguments -oE... So I have to cook from that is available
 
Old 03-19-2017, 05:11 PM   #28
BW-userx
LQ Guru
 
Registered: Sep 2013
Location: Somewhere in my head.
Distribution: Slackware (15 current), Slack15, Ubuntu studio, MX Linux, FreeBSD 13.1, WIn10
Posts: 10,342

Rep: Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242
test it in your terminal to see what it does first like this
Code:
userx@slackwhere⚡️~⚡️$ newname=20170319_102601_2F6F_ACCC8E565EC5.mkv
userx@slackwhere⚡️~aname=${newname%%_*}
userx@slackwhere⚡️~⚡️$ echo $aname
20170319
WTH? go the other way ? using this one --> #

http://tldp.org/LDP/abs/html/string-manipulation.html

look in that and you will find it
Code:
userx@slackwhere⚡️~⚡️$ aname=${newname#*_}                                      
userx@slackwhere⚡️~⚡️$ echo $aname                                              
102601_2F6F_ACCC8E565EC5.mkv
 
1 members found this post helpful.
Old 03-19-2017, 05:28 PM   #29
kzo81
Member
 
Registered: Aug 2014
Location: Hungary
Distribution: Debian, Linux Mint, CentOS
Posts: 174

Original Poster
Rep: Reputation: Disabled
I tested it:

Code:
a="20111108_XYZA_121212.mkv"
echo $a "${a%%_*}"
20111108_XYZA_121212.mkv 20111108
But the string is much longer. I could eliminate the full path in front of the mkv file

Code:
i="/var/spool/storage/SD_DISK/20170319/10/20170319_102601_27E2_ACCC8E565EC5/20170319_10/20170319_102601_2F6F_ACCC8E565EC5.mkv"
echo "${i##/*/}"
20170319_102601_2F6F_ACCC8E565EC5.mkv

how could I go further and extract only the 8 characters, there is no similar example in that tutorial.

Last edited by kzo81; 03-19-2017 at 05:42 PM.
 
Old 03-19-2017, 06:33 PM   #30
BW-userx
LQ Guru
 
Registered: Sep 2013
Location: Somewhere in my head.
Distribution: Slackware (15 current), Slack15, Ubuntu studio, MX Linux, FreeBSD 13.1, WIn10
Posts: 10,342

Rep: Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242Reputation: 2242
Quote:
Originally Posted by kzo81 View Post
I tested it:

Code:
a="20111108_XYZA_121212.mkv"
echo $a "${a%%_*}"
20111108_XYZA_121212.mkv 20111108
But the string is much longer. I could eliminate the full path in front of the mkv file

Code:
i="/var/spool/storage/SD_DISK/20170319/10/20170319_102601_27E2_ACCC8E565EC5/20170319_10/20170319_102601_2F6F_ACCC8E565EC5.mkv"
echo "${i##/*/}"
20170319_102601_2F6F_ACCC8E565EC5.mkv

how could I go further and extract only the 8 characters, there is no similar example in that tutorial.
yes there is I think

${stringosition:length}

Extracts $length characters of substring from $string at $position.
Code:
 again=${newfile#[0-9]*}
using the [0-9] every number 0 thru 9 - first number

Last edited by BW-userx; 03-19-2017 at 06:34 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Is there any default numbering system for creating symlinks inside /etc/rc3.d/ krssarath Fedora 4 02-09-2013 03:38 PM
LXer: Creating Symlinks How and Why LXer Syndicated Linux News 2 08-03-2012 07:13 AM
[SOLVED] About creating symlinks in .SlackBuild and doinst.sh cigerma Slackware 7 12-31-2011 07:31 AM
Creating symlinks on a CIFS share? MasterC Linux - General 2 08-30-2007 05:34 PM
Creating symlinks bripage Linux - General 2 07-23-2002 08:33 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 04:57 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration