Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place! |
Notices |
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
Are you new to LinuxQuestions.org? Visit the following links:
Site Howto |
Site FAQ |
Sitemap |
Register Now
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
|
 |
|
03-18-2017, 02:43 PM
|
#16
|
Member
Registered: Aug 2014
Location: Hungary
Distribution: Debian, Linux Mint, CentOS
Posts: 207
Original Poster
Rep: 
|
Hi:-)
Now I have a working code:
Code:
#!/bin/sh
echo "[+] Creating symbolic links ..."
SD="/var/spool/storage/SD_DISK/"
ARRANGED_FOLDER="$SD/videos_by_day"
while read i
do
DAY=$(echo "$i" | cut -d'/' -f10 | cut -d'_' -f1 | sort | uniq)
if [ ! -d "/$ARRANGED_FOLDER/$DAY" ]; then
mkdir -p /"$ARRANGED_FOLDER"/"$DAY"
fi
cd "$ARRANGED_FOLDER"/"$DAY" || exit 1
if [ `find "$ARRANGED_FOLDER" -samefile $i | wc -l` eq 0 ]; then
ln -sfn "$i"
fi
done < $(find "$SD" -type f -name "*.mkv")
There is still some bug in it, since it complained for only one file:
/var/spool/storage/SD_DISK/20170215/09/20170215_093234_3E4A_ACCC8E565EC5/20170215_09/20170215_093234_29AF_ACCC8E565EC5.mkv: File name too long
It seems that now the script only creates symlinks of the newly arrived files.
Quote:
is that "suppose" to give you a file name? because it comes up empty when I run it. just run this simple version with a path you give it to search. any path will work it is not changing any data.
|
Yes, it gives me the day, which will be the folder to create later.
Thank you so much for the efforts you all make for me.
|
|
|
03-19-2017, 05:19 AM
|
#17
|
Member
Registered: Aug 2014
Location: Hungary
Distribution: Debian, Linux Mint, CentOS
Posts: 207
Original Poster
Rep: 
|
Guys, I did some tests on the video server. And still the for cycle version gives me the least error message.
The one that I created:
Code:
#!/bin/sh
SD="/var/spool/storage/SD_DISK/"
ARRANGED_FOLDER="/var/spool/storage/SD_DISK/videos_by_day"
for i in $(find "$SD" -type f -name "*.mkv"); do
DAY=$(echo "$i" | cut -d'/' -f10 | cut -d'_' -f1 | sort | uniq)
if [ ! -d "/$ARRANGED_FOLDER/$DAY" ]; then
mkdir -p /"$ARRANGED_FOLDER"/"$DAY"
fi
cd "$ARRANGED_FOLDER"/"$DAY"
if ! ln -sfn "$i"; then
echo "[-] Error: Could not create symbolic links"
fi
done
The While version, this one:
Code:
#!/bin/sh
SD="/var/spool/storage/SD_DISK/"
ARRANGED_FOLDER="/var/spool/storage/SD_DISK/videos_by_day"
while read i; do
DAY=$(echo "$i" | cut -d'/' -f10 | cut -d'_' -f1 | sort | uniq)
if [ ! -d "/$ARRANGED_FOLDER/$DAY" ]; then
mkdir -p /"$ARRANGED_FOLDER"/"$DAY"
fi
cd "$ARRANGED_FOLDER"/"$DAY"
if ! ln -sfn "$i"; then
echo "[-] Error: Could not create symbolic links"
fi
done < `find "$SD" -type f -name "*.mkv"`
Complains:
Quote:
+ DAY=�{ckא�A6�S������6�?i�x:�(�d�.QF�����1<�A�ޙ��'>1`z��IG����Z��*r�+!Juu����m)T��,�O�l�U6{9P
+ [ ! -d //var/spool/storage/SD_DISK/videos_by_day/�{ckא�A6�S������6�?i�x:�(�d�.QF�����1<�A�ޙ��'>1`z��IG����Z��*r�+!Juu����m)T��,�O�l�U6{9P ]
+ cd /var/spool/storage/SD_DISK/videos_by_day/�{ckא�A6�S������6�?i�x:�(�d�.QF�����1<�A�ޙ��'>1`z��IG����Z��*r�+!Juu����m)T��,�O�l�U6{9P
|
I know that shellcheck website, and You suggest me not using a for loop for, but as you can see I don't get any error message when I use the for loop version.
The only think that still bothers me, that I still couldn't implement a check that checks if an original file has a symlink already.
That command substitution is way complicated for me, I checked your suggested code, with the corrected if clause and it is doing terrible things, I mean this code:
Code:
SD="/var/spool/storage/SD_DISK/"
ARRANGED_FOLDER="/var/spool/storage/SD_DISK/videos_by_day"
while read FILENAME
do
c=$FILENAME
xpath=${c%/*}
xbase=${c##*/}
xfext=${xbase##*.}
xpref=${xbase%.*}
path=${xpath}
fname=${xpref}
ext=${xfext}
ckfile="$fname"."$ext"
DAY=$(echo "$FILENAME" | cut -d'/' -f10 | cut -d'_' -f1 | sort | uniq)
DailyFiles="$ARRANGED_FOLDER"/"$DAY"
mkdir -p "$DailyFiles"
# check to see if in directory already / which it should not be?
[[ -h "$DailyFiles/$ckfile" ]] || ln -sfn $FILENAME "$DailyFiles/$ckfile"
done < $(find "$SD" -type f -name "*.mkv")
output:
Quote:
ln: /var/spool/storage/SD_DISK/videos_by_day/�h�c��t�e�c�W�뙁�@ø:�yw�� ig&�f�ȯ���J��.E��i�I�S4�`{�;�A
!kr����/�h�c��t�e�c�W�뙁�@ø:�yw�� ig&�f�ȯ���J��.E��i�I�S4�`{�;�A
�QI��O�2�e#f��L9�Hk���̀,�s���ȷ��[ !kr����_gg6�o-!���^������
-`���C�W���=H�����I�wo�9[�?(zT�b���� zK��Kr��?n��p��00Ѹ�������
��6�S%A�: File name too long
and if I put set -x in the script:
+ xbase=_B�X����Ѕ��I*�H�3ar�$��0�q���������^�Gߓ=GE����![���(��?��Dx�E��=�7;���x�p���q�^Wa�ܟ����v�7n
2���x��+����.��sp;���516�sR��m:���
z���?������T)�eB�:�t��C������F�V#.�5 ��}�Һ�����ӵ|�=�(�u'����
g��{�
+ xfext=�5 ��}�Һ�����ӵ|�=�(�u'����
|
I really appreciate your help, but the problem of - to much SD_DISK write, when rerun the script - is still there.
I noticed that if I omit the -fn arguments, it gives me an error, but at least the script didn't recreate the link:
Code:
#!/bin/sh
SD="/var/spool/storage/SD_DISK/"
ARRANGED_FOLDER="/var/spool/storage/SD_DISK/videos_by_day"
for i in $(find "$SD" -type f -name "*.mkv"); do
DAY=$(echo "$i" | cut -d'/' -f10 | cut -d'_' -f1 | sort | uniq)
if [ ! -d "/$ARRANGED_FOLDER/$DAY" ]; then
mkdir -p /"$ARRANGED_FOLDER"/"$DAY"
fi
cd "$ARRANGED_FOLDER"/"$DAY"
ln -s "$i" >/dev/null 2>&1
done
exit 0
So at least it's a workaround for now.
Last edited by kzo81; 03-19-2017 at 06:23 AM.
|
|
|
03-19-2017, 09:08 AM
|
#18
|
LQ Addict
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 24,644
|
I do not really understand where those strange chars are coming from. Are they filenames or something else?
if the for cycle and while loop work differently that means the while is ok and the for loop makes something wrong.
if
Quote:
+ DAY=�{ckא�A6�S������6�?i�x:�(�d�.QF�����1<�A�ޙ��'>1`z��IG����Z��*r�+!Juu����m)T��,�O�l�U6{9P
|
that means calculating the DAY is not ok.
|
|
|
03-19-2017, 01:22 PM
|
#19
|
LQ Guru
Registered: Sep 2013
Location: Somewhere in my head.
Distribution: Slackware (15 current), Slack15, Ubuntu studio, MX Linux, FreeBSD 13.1, WIn10
Posts: 10,342
|
I got a ask you do you know what a soft link basically is? It is just a fake copy of the same file pointing back to the original file. it looks feels and smells just like the original one. Q2: so how do you check to see if a file is present within a directory? apply the same methodology.
-L FILE
FILE exists and is a symbolic link (same as -h)
Code:
if [[ -L "$strFile" ]] && [[ -a "$strFile" ]];then
echo "'$strFile' link present";
fi
that bit of code I gave you that you didn't understand I. That slices up the complete path to a file and breaks it down to path to file - file name - extension.
|
|
1 members found this post helpful.
|
03-19-2017, 01:41 PM
|
#20
|
LQ Guru
Registered: Sep 2013
Location: Somewhere in my head.
Distribution: Slackware (15 current), Slack15, Ubuntu studio, MX Linux, FreeBSD 13.1, WIn10
Posts: 10,342
|
you do not need all of this code
Code:
if [ ! -d "/$ARRANGED_FOLDER/$DAY" ]; then
mkdir -p /"$ARRANGED_FOLDER"/"$DAY"
fi
Just this will work because mkdir does the check to see if a directory by that name is already there for you, all you need to do is add just this line.
Code:
mkdir -p /"$ARRANGED_FOLDER"/"$DAY"
what that leading / slash is for I do not know. I am surprised it works. Because if it is for a root dir on your SD Card then that should be within your variable name instead. just coding styles
Code:
ARRANGED_FOLDER="/root-new-whatever-dir"
example
Code:
userx@voider⚡️~⚡️$ newdir=/tmp
userx@voider⚡️~⚡️$ ls $newdir
cron.1FMpX9
userx@voider⚡️~⚡️$ ls /tmp
cron.1FMpX9
Last edited by BW-userx; 03-19-2017 at 01:49 PM.
|
|
|
03-19-2017, 01:47 PM
|
#21
|
Member
Registered: Aug 2014
Location: Hungary
Distribution: Debian, Linux Mint, CentOS
Posts: 207
Original Poster
Rep: 
|
Hi, I know what is a softlink. Though I had rarely used it until this problem emerged with this axis video server.
I also understand the test tool. As I mentioned, I achieved the effect what I wished by omiting arguments (ls -sfn).
I think I'll rework the code tomorrow to a WHILE cycle, I don't really understand why it doesn't work the same as the FOR version of the code.
This is wrong I, know, but it's working:
Code:
#!/bin/sh
SD="/var/spool/storage/SD_DISK/"
ARRANGED_FOLDER="/var/spool/storage/SD_DISK/videos_by_day"
for i in $(find "$SD" -type f -name "*.mkv"); do
DAY=$(echo "$i" | cut -d'/' -f10 | cut -d'_' -f1 | sort | uniq)
if [ ! -d "/$ARRANGED_FOLDER/$DAY" ]; then
mkdir -p /"$ARRANGED_FOLDER"/"$DAY"
fi
cd "$ARRANGED_FOLDER"/"$DAY"
ln -s "$i" >/dev/null 2>&1
done
exit 0
But this doesn't work:
Code:
#!/bin/sh
set -x
SD="/var/spool/storage/SD_DISK/"
ARRANGED_FOLDER="/var/spool/storage/SD_DISK/videos_by_day"
while read i; do
DAY=$(echo "$i" | cut -d'/' -f10 | cut -d'_' -f1 | sort | uniq)
if [ ! -d "/$ARRANGED_FOLDER/$DAY" ]; then
mkdir -p /"$ARRANGED_FOLDER"/"$DAY"
fi
cd "$ARRANGED_FOLDER"/"$DAY"
ln -s "$i" >/dev/null 2>&1
done < `find "$SD" -type f -name "*.mkv"`
This gives an error:
Quote:
/var/spool/storage/SD_DISK/20170319/10/20170319_102601_27E2_ACCC8E565EC5/20170319_10/20170319_102601_2F6F_ACCC8E565EC5.mkv: no such file
|
The FOR version runs seamless. I don't know why. That file doesn't exist on the original place, only the While version of the code finds it...
Last edited by kzo81; 03-19-2017 at 03:07 PM.
|
|
|
03-19-2017, 01:56 PM
|
#22
|
LQ Guru
Registered: Sep 2013
Location: Somewhere in my head.
Distribution: Slackware (15 current), Slack15, Ubuntu studio, MX Linux, FreeBSD 13.1, WIn10
Posts: 10,342
|
this
Code:
[[ -h "$DailyFiles/$ckfile" ]] || ln -sfn $FILENAME "$DailyFiles/$ckfile"
can be written like this too
if file link not there then create one.
Code:
[[ ! -h "$DailyFiles/$ckfile" ]] && ln -sfn $FILENAME "$DailyFiles/$ckfile"
-s, --symbolic
make symbolic links instead of hard links
is this really needed? you're logic is this - if link not there then make one else skip - right?
-f, --force
remove existing destination files
-n, --no-dereference
treat destination that is a symlink to a directory as if it were a normal file
|
|
|
03-19-2017, 02:00 PM
|
#23
|
LQ Guru
Registered: Sep 2013
Location: Somewhere in my head.
Distribution: Slackware (15 current), Slack15, Ubuntu studio, MX Linux, FreeBSD 13.1, WIn10
Posts: 10,342
|
why are you going into the directory?
Code:
cd "$ARRANGED_FOLDER"/"$DAY"
all of your paths are already set to work outside of directory regardless if your inside it or not. so what is the point of being inside of it? just trying to understand your logic.
|
|
|
03-19-2017, 03:02 PM
|
#24
|
Member
Registered: Aug 2014
Location: Hungary
Distribution: Debian, Linux Mint, CentOS
Posts: 207
Original Poster
Rep: 
|
Quote:
Originally Posted by BW-userx
why are you going into the directory?
|
Because if I dont, it'll create the symlinks in the directory where the script is. If you look at the man page of ln, see the 2nd form. It says that : "create a link to TARGET in the current directory".
If I only use ln -s, it wont recreate the symlink which is already created before although it throughs an error, which I redirect to the black-hole:-)
Last edited by kzo81; 03-19-2017 at 03:16 PM.
|
|
|
03-19-2017, 03:45 PM
|
#25
|
LQ Guru
Registered: Sep 2013
Location: Somewhere in my head.
Distribution: Slackware (15 current), Slack15, Ubuntu studio, MX Linux, FreeBSD 13.1, WIn10
Posts: 10,342
|
Quote:
Originally Posted by kzo81
Because if I dont, it'll create the symlinks in the directory where the script is. If you look at the man page of ln, see the 2nd form. It says that : "create a link to TARGET in the current directory".
|
I do not specify remember if I've ever ran a ln -s like this outside of both directories before on cli - but I do think I have. I got a give that a try and see the results on that one.
Quote:
Originally Posted by kzo81
If I only use ln -s, it wont recreate the symlink which is already created before although it throughs an error, which I redirect to the black-hole:-)
|
I understand the -s it is the other two I was having issues with, specifically -f forcing it to link when there is no link to begin with. Just being cautious ??
|
|
|
03-19-2017, 03:55 PM
|
#26
|
Member
Registered: Aug 2014
Location: Hungary
Distribution: Debian, Linux Mint, CentOS
Posts: 207
Original Poster
Rep: 
|
You guys told me about string manipulation.
Do you know an elegant way of extracting the red part of this?
i="/var/spool/storage/SD_DISK/20170319/10/20170319_102601_27E2_ACCC8E565EC5/20170319_10/20170319_102601_2F6F_ACCC8E565EC5.mkv"
something like this: echo "${i%%_*.mkv}"
|
|
|
03-19-2017, 04:00 PM
|
#27
|
Member
Registered: Aug 2014
Location: Hungary
Distribution: Debian, Linux Mint, CentOS
Posts: 207
Original Poster
Rep: 
|
Quote:
Originally Posted by BW-userx
it is the other two I was having issues with, specifically -f forcing it to link when there is no link to begin with. Just being cautious ??
|
first I thought that ln -sf will help me, but it didnt. Same goes with ln -sfn.
And interestingly many tools are not available on this videoserver. For instance there is no awk, but sed with no possibility of using arguments -oE... So I have to cook from that is available
|
|
|
03-19-2017, 04:11 PM
|
#28
|
LQ Guru
Registered: Sep 2013
Location: Somewhere in my head.
Distribution: Slackware (15 current), Slack15, Ubuntu studio, MX Linux, FreeBSD 13.1, WIn10
Posts: 10,342
|
test it in your terminal to see what it does first like this
Code:
userx@slackwhere⚡️~⚡️$ newname=20170319_102601_2F6F_ACCC8E565EC5.mkv
userx@slackwhere⚡️~aname=${newname%%_*}
userx@slackwhere⚡️~⚡️$ echo $aname
20170319
WTH? go the other way ? using this one --> #
http://tldp.org/LDP/abs/html/string-manipulation.html
look in that and you will find it
Code:
userx@slackwhere⚡️~⚡️$ aname=${newname#*_}
userx@slackwhere⚡️~⚡️$ echo $aname
102601_2F6F_ACCC8E565EC5.mkv
|
|
1 members found this post helpful.
|
03-19-2017, 04:28 PM
|
#29
|
Member
Registered: Aug 2014
Location: Hungary
Distribution: Debian, Linux Mint, CentOS
Posts: 207
Original Poster
Rep: 
|
I tested it:
Code:
a="20111108_XYZA_121212.mkv"
echo $a "${a%%_*}"
20111108_XYZA_121212.mkv 20111108
But the string is much longer. I could eliminate the full path in front of the mkv file
Code:
i="/var/spool/storage/SD_DISK/20170319/10/20170319_102601_27E2_ACCC8E565EC5/20170319_10/20170319_102601_2F6F_ACCC8E565EC5.mkv"
echo "${i##/*/}"
20170319_102601_2F6F_ACCC8E565EC5.mkv
how could I go further and extract only the 8 characters, there is no similar example in that tutorial.
Last edited by kzo81; 03-19-2017 at 04:42 PM.
|
|
|
03-19-2017, 05:33 PM
|
#30
|
LQ Guru
Registered: Sep 2013
Location: Somewhere in my head.
Distribution: Slackware (15 current), Slack15, Ubuntu studio, MX Linux, FreeBSD 13.1, WIn10
Posts: 10,342
|
Quote:
Originally Posted by kzo81
I tested it:
Code:
a="20111108_XYZA_121212.mkv"
echo $a "${a%%_*}"
20111108_XYZA_121212.mkv 20111108
But the string is much longer. I could eliminate the full path in front of the mkv file
Code:
i="/var/spool/storage/SD_DISK/20170319/10/20170319_102601_27E2_ACCC8E565EC5/20170319_10/20170319_102601_2F6F_ACCC8E565EC5.mkv"
echo "${i##/*/}"
20170319_102601_2F6F_ACCC8E565EC5.mkv
how could I go further and extract only the 8 characters, there is no similar example in that tutorial.
|
yes there is I think
${string  osition:length}
Extracts $length characters of substring from $string at $position.
Code:
again=${newfile#[0-9]*}
using the [0-9] every number 0 thru 9 - first number
Last edited by BW-userx; 03-19-2017 at 05:34 PM.
|
|
|
All times are GMT -5. The time now is 02:37 PM.
|
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
|
Latest Threads
LQ News
|
|