bash scripting and arrays
 

While I'm not new to Linux, I'm lost at scripting. Feel free to move this to a more appropriate place. I learned BASIC many years ago, so I think I can lay it out in a logical fashion, but most definitely need instruction here, preferably with explanations.

I have a lot of video files, but the output volumes are all over the place. I'll have to crank the volume on one video, and the next one will be deafening so I have been attempting to write a bash script to "level out" the files using ffmpeg's af volumedetect. (No, I don't mean normalize)

Using ffmpeg -i *.mkv -af volumedetect -f null /dev/null on one particular file yielded the following output:

frame=120550 fps=2457 q=0.0 Lsize=N/A time=01:23:48.09 bitrate=N/A
video:11302kB audio:2828304kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[Parsed_volumedetect_0 @ 0x2532840] n_samples: 1448091648
[Parsed_volumedetect_0 @ 0x2532840] mean_volume: -25.2 dB
[Parsed_volumedetect_0 @ 0x2532840] max_volume: -1.3 dB
[Parsed_volumedetect_0 @ 0x2532840] histogram_1db: 138
[Parsed_volumedetect_0 @ 0x2532840] histogram_2db: 4022
[Parsed_volumedetect_0 @ 0x2532840] histogram_3db: 33737
[Parsed_volumedetect_0 @ 0x2532840] histogram_4db: 135497
[Parsed_volumedetect_0 @ 0x2532840] histogram_5db: 385067
[Parsed_volumedetect_0 @ 0x2532840] histogram_6db: 794773
[Parsed_volumedetect_0 @ 0x2532840] histogram_7db: 1318793

What I have been doing is running another instance of ffmpeg to encode it to mkv using, along with other parameters, taking the absolute value (a positive intger), in this case -vf "volume=1.3" (with quotes,) to bring the max_volume in line with that of the other files, but am at a loss how to automate this.

I have:

!# /bin/bash

mkdir ./encoded
for IFS in *.mkv; do
max_volume=`ffmpeg -i "${IFS}" -af volumedetect -f null /dev/null

As you can see I have been attempting to create an array thinking it was the way to go, if not, please advise me.

Now, I can understand "-f null" means a virtual file instead of an actual one, but from there....

I have tried to put the output through /dev/null but could not figure out how to obtain the max_volume value, change the negative to a positive or vice versa (*-1?) and then input that value into a second ffmpeg instance where the volume will be adjusted, video scaled, etc. I have the encoding parameters I want, the problem is inputting the value for "volume = "

Would it be better to output to a text file, or maybe stdout and then pick up the max_volume from stdin, and if so, how would I accomplish this?

I would appreciate some help resolving this. Thanks.

No need to go to a file, you should be able to pipe directly to something like sed. Using the above list, this should get you what you wantYou need to supply "some ffmpeg command" and the vertical bar symbol (pipe) pipes all the stdout into the sed. It is specific so as to ensure just the right data is picked up. Run it and see what it produces for you. You can assign that to a variable and use that in the following command.

excuse me if I seem a little dense, but I have no experience using sed, am I'm not sure exactly how to assign a variable to it. Here is what I have so far:

#! /bin/bash
mkdir ./encode
for IFS in *.mkv; do
ffmpeg -i "${IFS}" -af volumedetect | sed -nr '/max_volume:/ s/.*[[:space:]]-?([[:digit:]\.]+)[[:space:]]+dB$/-vf "volume=\1"/p'

This is where I get lost.

ffmpeg -i "$"IFS" bunch of encode stuff -af "volume=variable" ./encode/"$IFS";
done

Am I on the right track? I have the encode parameters I want, its a matter of inserting the correct volume setting taken fron max_volume above. I had one file, believe it or not, with a max_volume of -13.7 db. You can see why I would like to level off the volumes accross the different files.

Try it like this

Still don't seem to have it quite right. I ran:

#! /bin/bash
mkdir ./encode
for IFS in *.mkv; do
ffmpeg -i "${IFS}" -af volumedetect | sed -nr '/max_volume:/ s/.*[[:space:]]-?([[:digit:]\.]+)[[:space:]]+dB$/-vf "volume=\1"/p'
echo $vol;
done


And got:terry@TP500LA:~/Videos/test$ sh ./setvolume.sh
mkdir: cannot create directory ./encode: File exists
ffmpeg version 2.5.7-0ubuntu0.15.04.1 Copyright (c) 2000-2015 the FFmpeg developers
built with gcc 4.9.2 (Ubuntu 4.9.2-10ubuntu13)
configuration: --prefix=/usr --extra-version=0ubuntu0.15.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --shlibdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --enable-shared --disable-stripping --enable-avresample --enable-avisynth --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libshine --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libwavpack --enable-libwebp --enable-libxvid --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzvbi --enable-libzmq --enable-frei0r --enable-libvpx --enable-libx264 --enable-libsoxr --enable-gnutls --enable-openal --enable-libopencv --enable-librtmp --enable-libx265
libavutil 54. 15.100 / 54. 15.100
libavcodec 56. 13.100 / 56. 13.100
libavformat 56. 15.102 / 56. 15.102
libavdevice 56. 3.100 / 56. 3.100
libavfilter 5. 2.103 / 5. 2.103
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100
Trailing options were found on the commandline.
Input #0, matroska,webm, from 'xxxx.mkv':
Metadata:
COMPATIBLE_BRANDS: isomiso2avc1mp41
MAJOR_BRAND : isom
MINOR_VERSION : 512
ENCODER : Lavf56.1.0
Duration: 01:33:46.19, start: 0.000000, bitrate: 1360 kb/s
Stream #0:0(und): Video: h264 (High), yuv420p, 704x384 [SAR 32:33 DAR 16:9], SAR 31:32 DAR 341:192, 30.30 fps, 29.97 tbr, 1k tbn, 59.94 tbc (default)
Stream #0:1(und): Audio: aac (LC), 48000 Hz, stereo, fltp (default)
At least one output file must be specified

Should I put in the second instance now?

Sorry, ran:

#! /bin/bash
mkdir ./encode
for IFS in *.mkv; do
vfvol=$(ffmpeg -i "${IFS}" -af volumedetect | sed -nr '/max_volume:/ s/.*[[:space:]]-?([[:digit:]\.]+)[[:space:]]+dB$/-vf "volume=\1"/p')
echo ${vfvol}
done

All together, with the second ffmpeg instance included I just ran:

#! /bin/bash
mkdir ./encode
for IFS in *.mkv; do
vfvol=$(ffmpeg -i "${IFS}" -af volumedetect | sed -nr '/max_volume:/ s/.*[[:space:]]-?([[:digit:]\.]+)[[:space:]]+dB$/-vf "volume=\1"/p')
ffmpeg -i ""${IFS} -f matroska -maxrate 1M -b:v 1500k -r 29.97 -vcodec libxvid -filter:v scale=704:384 -aspect 16:9 -qmin 3 -qmax 5 -bufsize 4096 -mbd 2 -bf 2 -trellis 1 -flags +aic -cmp 2 -subcmp 2 -g 300 -acodec ac3 -strict -2 -b:a 128k -ac 6 -af "volume="${vfvol}"" ./encode/"${IFS}";
done


and the result was:

terry@TP500LA:~/Videos/test$ sh ./setvolume.sh
mkdir: cannot create directory ./encode: File exists
ffmpeg version 2.5.7-0ubuntu0.15.04.1 Copyright (c) 2000-2015 the FFmpeg developers
built with gcc 4.9.2 (Ubuntu 4.9.2-10ubuntu13)
configuration: --prefix=/usr --extra-version=0ubuntu0.15.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --shlibdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --enable-shared --disable-stripping --enable-avresample --enable-avisynth --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libshine --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libwavpack --enable-libwebp --enable-libxvid --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzvbi --enable-libzmq --enable-frei0r --enable-libvpx --enable-libx264 --enable-libsoxr --enable-gnutls --enable-openal --enable-libopencv --enable-librtmp --enable-libx265
libavutil 54. 15.100 / 54. 15.100
libavcodec 56. 13.100 / 56. 13.100
libavformat 56. 15.102 / 56. 15.102
libavdevice 56. 3.100 / 56. 3.100
libavfilter 5. 2.103 / 5. 2.103
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100
Trailing options were found on the commandline.
Input #0, matroska,webm, from 'xxxxx.mkv':
Metadata:
COMPATIBLE_BRANDS: isomiso2avc1mp41
MAJOR_BRAND : isom
MINOR_VERSION : 512
ENCODER : Lavf56.1.0
Duration: 01:33:46.19, start: 0.000000, bitrate: 1360 kb/s
Stream #0:0(und): Video: h264 (High), yuv420p, 704x384 [SAR 32:33 DAR 16:9], SAR 31:32 DAR 341:192, 30.30 fps, 29.97 tbr, 1k tbn, 59.94 tbc (default)
Stream #0:1(und): Audio: aac (LC), 48000 Hz, stereo, fltp (default)
At least one output file must be specified
ffmpeg version 2.5.7-0ubuntu0.15.04.1 Copyright (c) 2000-2015 the FFmpeg developers
built with gcc 4.9.2 (Ubuntu 4.9.2-10ubuntu13)
configuration: --prefix=/usr --extra-version=0ubuntu0.15.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --shlibdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --enable-shared --disable-stripping --enable-avresample --enable-avisynth --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libshine --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libwavpack --enable-libwebp --enable-libxvid --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzvbi --enable-libzmq --enable-frei0r --enable-libvpx --enable-libx264 --enable-libsoxr --enable-gnutls --enable-openal --enable-libopencv --enable-librtmp --enable-libx265
libavutil 54. 15.100 / 54. 15.100
libavcodec 56. 13.100 / 56. 13.100
libavformat 56. 15.102 / 56. 15.102
libavdevice 56. 3.100 / 56. 3.100
libavfilter 5. 2.103 / 5. 2.103
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100
: No such file or directory
terry@TP500LA:~/Videos/test$


Still willing to give it a go?

I just noticed a nasty syntax error that was entirely my fault. I used ""${IFS} when it should have been "${IFS}"

I have corrected my mistake and the script is running on two test files right now and will give you the results when it is finished.

I would like to thank you for all your valuable help. Of course should this be successful I will mark it as solved.

I would just like to point out that using IFS as the variable could be problematic. IFS is a builtin variable used by bash to determine word splitting and the like. I am not 100 sure it is causing problems here but I would change it nonetheless.

I saw you wish to mark this as solved, but you do seem to have some errors appearing in your output, such as ': No such file or directory'.


Did you check the output of the first script with the echo?
Because your output does not show the relevant output anywhere. To add to this, your second ffmpeg call would be incorrect:
The data stored in the variable vfvol would be :- -vf "volume=\1", where '\1' is replaced with what was gleaned from the output. So in your code once the variable was expanded it would look like:
As you can see, this is not what you were expecting.

I am afraid that my tiny, underdeveloped brain has reached its stress limit, and too much stress leaves me horizontal for hours, sometimes days. (I am unable to work due to major depression.)

Here is what I have:

#! /bin/bash
mkdir ./encode
for IFS in *.mkv; do
vfvol=$(ffmpeg -i "${IFS}" -af volumedetect | sed -nr '/max_volume:/ s/.*[[:space:]]-?([[:digit:]\.]+)[[:space:]]+dB$/-vf "volume=\1"/p')
ffmpeg -i "${IFS}" -f matroska -maxrate 1M -b:v 1500k -r 29.97 -vcodec libxvid -filter:v scale=704:384 -aspect 16:9 -qmin 3 -qmax 5 -bufsize 4096 -mbd 2 -bf 2 -trellis 1 -flags +aic -cmp 2 -subcmp 2 -g 300 -acodec ac3 -strict -2 -b:a 128k -ac 6 -af "volume="${vfvol}"" ./encode/"${IFS}";
done

and the resulting file has the same volume level as the original.

I must ask that if you wish to help that you give me a complete solution rather than just a section or hint. I may have to set this aside for a bit while my brain resets itself. Thank you for your help and patience. I'll try again in a few hours, maybe a day or two. I won't give up.

Grail: I was under the impression that IFS should be used whenever there was the chance of a <space> in the variable, but then I make a lot of mistakes.

Also, I find that I am experiencing a little too much stress trying to apply hints and suggestions, which can trigger yet another major depressive episode (and explains why I am so bad at scripting. Would it be possible to give me a more complete solution. Take your suggestion of using

[code]echo ${vfvol}[/code

as example. I tried adding it after the fitst part and commenting out the second, but it said something about needing to output to a file. My head is so muddled right now that I can't seem to get it right. Do I need quotes around the variable, or something else?
Would you be able to do a little copy and paste to give me the entire sequence to get the value to echo on the screen? This is all so frustrating for me. I used to have a working brain before the Depression set it, and now I have to struggle.

Thanks

syg00

Still have not succeeded, but thanks for the help so far.

syg00 I ran what you suggested and not get any output. Here is what it gave me:

terry@TP500LA:~/Videos/media$ sh ./setvolume2.sh
./setvolume2.sh: 1: ./setvolume2.sh: #!: not found
mkdir: cannot create directory ./encode: File exists
ffmpeg version 2.5.7-0ubuntu0.15.04.1 Copyright (c) 2000-2015 the FFmpeg developers
built with gcc 4.9.2 (Ubuntu 4.9.2-10ubuntu13)
configuration: --prefix=/usr --extra-version=0ubuntu0.15.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --shlibdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --enable-shared --disable-stripping --enable-avresample --enable-avisynth --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libshine --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libwavpack --enable-libwebp --enable-libxvid --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzvbi --enable-libzmq --enable-frei0r --enable-libvpx --enable-libx264 --enable-libsoxr --enable-gnutls --enable-openal --enable-libopencv --enable-librtmp --enable-libx265
libavutil 54. 15.100 / 54. 15.100
libavcodec 56. 13.100 / 56. 13.100
libavformat 56. 15.102 / 56. 15.102
libavdevice 56. 3.100 / 56. 3.100
libavfilter 5. 2.103 / 5. 2.103
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100
Trailing options were found on the commandline.
Input #0, matroska,webm, from 'xxxxx.mkv':
Metadata:
COMPATIBLE_BRANDS: isomiso2avc1mp41
MAJOR_BRAND : isom
MINOR_VERSION : 512
ENCODER : Lavf56.1.0
Duration: 02:00:01.23, start: 0.000000, bitrate: 344 kb/s
Stream #0:0(und): Video: h264 (High), yuv420p, 576x268 [SAR 335:336 DAR 15:7], SAR 254:255 DAR 12192:5695, 30.30 fps, 29.97 tbr, 1k tbn, 59.94 tbc (default)
Stream #0:1(und): Audio: aac (LC), 44100 Hz, stereo, fltp (default)
At least one output file must be specified

terry@TP500LA:~/Videos/media$


I waited for a while for the fog to lift from my brain, and have been doing some RTFM and trying many experiments that all failed in a most glorious fashion. I've even tried, armed with a sed tutorial to parse your sed command, but I am a mere Padewan and am having trouble interpretting it. I understand that it is going through space by space, but if there is a mistake there, as signified by the trailing options warning, I am unable to find it.

The regex I gave is for the specific data you listed.
Do not try to debug in a loop where you don't know the data being processed. Pick a file or two, and run your command against that. If the output includes a line like the "max_volume" in your original post, my regex should pull out the numbers you're looking for. When all that works on a select file or two, assign it to a variable, then echo it. Do it one command at a time by hand so you can see what is happening.
If your commands have errors, or not all ffmpeg commands produce that output, you will not get good results. Only you can resolve all those issues.

As for the regex, it is complex (maybe more so than needed), but was designed to make sure only the output line you offered was processed. Regex is notoriously difficult to get right. Here is a quick breakdown:

I've been parsing your sed in an attempt to learn about it and have slowly come to see the logic behind it and why you made it the way you did. I can match it up against the max_volume line and can appreciate your skill with scripting.

Some how I have mangled the second ffmpeg command so that it is not utilizing the generated $vfvol. Nor can I get echo to display the value on screen. Care to parse that for me? It is all complete, excepting, of course, the file names provided by $IFS.


#! /bin/bash
mkdir ./encode
for IFS in *.mkv; do
vfvol=$(ffmpeg -i "${IFS}" -af volumedetect | sed -nr '/max_volume:/ s/.*[[:space:]]-?([[:digit:]\.]+)[[:space:]]+dB$/-vf "volume=\1"/p') echo ${vfvol}
#ffmpeg -i "${IFS}" -f matroska -maxrate 1M -b:v 1500k -r 29.97 -vcodec libxvid -filter:v scale=704:384 -aspect 16:9 -qmin 3 -qmax 5 -bufsize 4096 -mbd 2 -bf 2 -trellis 1 -flags +aic -cmp 2 -subcmp 2 -g 300 -acodec ac3 -strict -2 -b:a 128k -ac 6 -af "volume="${vfvol}"" ./encode/"${IFS}";
done


I just had a thought. Is it at all possible that the audio is being encoded before it tries to apply the filter? What if I moved it to a spot before the codec. I know position doesn't usually matter, but there are times when it does.

Any thoughts?

You need to STOP. Read more of what syg00 has been telling you. Namely you are still using the for loop but have no idea if any of your code is working.

You need to go back to absolute basics and remove the for loop for the time being.

From some of your output I gather you have a file with the following name :- Terminator Genisys (2015).mkv

So using the above file as a test case, what do you get as output for:
Does this output contain the line 'max_volume'?

If the answer to above is yes, then try using the sed on the output:
Is the output from the above :- -vf "volume=N" .... here N would be replaced by the max_volume line you saw in the first test

If either of the above are not showing the desired output you will need to find where that problem is first. Once you have confirmed the above for at least 3 - 5 files, you can then assume you are now getting the
correct data and continue.

Your assumptions on IFS are incorrect, or at least flawed. For the time being, please an alternate variable name, ie. not IFS

The assigning to 'vfvol' is trivial once the above has been confirmed, however, it is important to note that using a variable as both a switch and value rarely works for many commands.
What I mean is, if vfvol looks like the following:
When your second call to ffmpeg uses this variable it will see it as a simple string but the switch part (-vf) will be interpreted as a string and not a switch and hence will more than
likely cause an error.

What I would suggest instead is trying to have vfvol only store the number required and then your second ffmpeg will look something like:
I used '...' as I didn't think typing in the whole command was needed.


See how you go.

I think you're have something there on the loop. When in (severe) doubt, dismantle and test each section as a seperate entity to make sure it is functioning as desired. I'll give that a try inputting each file name seperatly and checking the output.

I'll also read up on IFS. Because of the spaces in the file name I thought it was the right choice, but as I said, I'm not very experienced at scripting.

I'll play with it for a while and report any successes or failures.

(Guess I'm not so great at hints, thanks for giving it to me straight.)

* * *


Ok, I changed the movie name to "movie.mkv"
Started a new shell script and made it executable.

Contents of shell script:

#! /bin/bash

vfvol=$(ffmpeg -i movie.mkv -af volumedetect | sed -nr '/max_volume:/ s/.*[[:space:]]-?([[:digit:]\.]+)[[:space:]]+dB$/-vf "volume=\1"/p');
echo ${vfvol}


Ran script - results:

terry@TP500LA:~/Videos/media/volume$ sh ./test\ sed\ without\ loop.sh
./test sed without loop.sh: 1: ./test sed without loop.sh: #!: not found
ffmpeg version 2.5.8-0ubuntu0.15.04.1 Copyright (c) 2000-2015 the FFmpeg developers
built with gcc 4.9.2 (Ubuntu 4.9.2-10ubuntu13)
configuration: --prefix=/usr --extra-version=0ubuntu0.15.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --shlibdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --enable-shared --disable-stripping --enable-avresample --enable-avisynth --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libshine --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libwavpack --enable-libwebp --enable-libxvid --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzvbi --enable-libzmq --enable-frei0r --enable-libvpx --enable-libx264 --enable-libsoxr --enable-gnutls --enable-openal --enable-libopencv --enable-librtmp --enable-libx265
libavutil 54. 15.100 / 54. 15.100
libavcodec 56. 13.100 / 56. 13.100
libavformat 56. 15.102 / 56. 15.102
libavdevice 56. 3.100 / 56. 3.100
libavfilter 5. 2.103 / 5. 2.103
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100
Trailing options were found on the commandline.
[mpeg4 @ 0x20c8320] hmm, seems the headers are not complete, trying to guess time_increment_bits
[mpeg4 @ 0x20c8320] my guess is 12 bits ;)
[mpeg4 @ 0x20c8320] hmm, seems the headers are not complete, trying to guess time_increment_bits
[mpeg4 @ 0x20c8320] my guess is 12 bits ;)
[mpeg4 @ 0x20c8320] looks like this file was encoded with (divx4/(old)xvid/opendivx) -> forcing low_delay flag
[mpeg4 @ 0x20c8320] Error, header damaged or not MPEG4 header (f_code=0)
Last message repeated 3 times
Input #0, matroska,webm, from 'movie.mkv':
Metadata:
COMPATIBLE_BRANDS: isommp42
MAJOR_BRAND : mp42
MINOR_VERSION : 0
ENCODER : Lavf56.36.100
Duration: 02:07:28.24, start: 0.000000, bitrate: 1190 kb/s
Stream #0:0(und): Video: mpeg4, yuv420p, 704x384, SAR 65:67 DAR 715:402, 29.97 fps, 29.97 tbr, 1k tbn, 1k tbc (default)
Metadata:
LANGUAGE : und
HANDLER_NAME : VideoHandler
ENCODER : Lavc56.41.100 libxvid
Stream #0:1(und): Audio: ac3, 44100 Hz, 5.1(side), fltp, 128 kb/s (default)
Metadata:
CREATION_TIME : 2014-06-30 13:43:16
LANGUAGE : und
HANDLER_NAME : IsoMedia File Produced by Google, 5-11-2011
ENCODER : Lavc56.41.100 ac3
At least one output file must be specified

terry@TP500LA:~/Videos/media/volume$



Piece by piece as you suggested, grail. As you can see, I am not achieving the desired output.

BTW, thanks for being patient with me. I will now read up on IFS while I wait for a reply.

I believe you may have a space in your interpreter at the beginning and hence your first error:
So make sure the first line has no spaces anywhere in it:
As you are placing the interpreter in the file there is no need to call it with 'sh'. Simply make the file executable and run it from the command line:
Here I would also tend to make the file name something without spaces to help eliminate any calling issues.


Now personally I would still consider this not having broken it down to its simplest form. As in my previous post, try calling the commands at the command line, ie. not from the script.

1. So confirm the ffmpeg on its own returns the correct line
2. Call the same ffmpeg and pass it through sed and see what output you get


Lastly, please use [code][/code] tags around your code and data to preserve the format and make it easier to read.

Ok, grail, I removed the spaces as per you suggestion. Sometimes if there is no space after the ! all I get is an error message, but this time it did run.

ran skipping over the few thousand lines of the same message, we get

[data][null @ 0x21e3e00] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 229215 >= 229212
frame=183373 fps=2502 q=0.0 Lsize=N/A time=02:07:28.23 bitrate=N/A
video:17191kB audio:3952584kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[Parsed_volumedetect_0 @ 0x21a84e0] n_samples: 2023723008
[Parsed_volumedetect_0 @ 0x21a84e0] mean_volume: -35.8 dB
[Parsed_volumedetect_0 @ 0x21a84e0] max_volume: -5.0 dB
[Parsed_volumedetect_0 @ 0x21a84e0] histogram_5db: 16
[Parsed_volumedetect_0 @ 0x21a84e0] histogram_6db: 954
[Parsed_volumedetect_0 @ 0x21a84e0] histogram_7db: 6197
[Parsed_volumedetect_0 @ 0x21a84e0] histogram_8db: 18841
[Parsed_volumedetect_0 @ 0x21a84e0] histogram_9db: 37788
[Parsed_volumedetect_0 @ 0x21a84e0] histogram_10db: 70766
[Parsed_volumedetect_0 @ 0x21a84e0] histogram_11db: 131528
[Parsed_volumedetect_0 @ 0x21a84e0] histogram_12db: 233100
[Parsed_volumedetect_0 @ 0x21a84e0] histogram_13db: 387229
[Parsed_volumedetect_0 @ 0x21a84e0] histogram_14db: 608202
[Parsed_volumedetect_0 @ 0x21a84e0] histogram_15db: 914000
[/data]

I included the last line of the repeating message down to the end for clarity.

Next I copied and pasted

The results were


[data]
terry@TP500LA:~/Videos/media/volume$ vfvol=$(ffmpeg -i movie.mkv -af volumedetect | sed -nr '/max_volume:/ s/.*[[:space:]]-?([[:digit:]\.]+)[[:space:]]+dB$/-vf "volume=\1"/p'); echo ${vfvol}
ffmpeg version 2.5.8-0ubuntu0.15.04.1 Copyright (c) 2000-2015 the FFmpeg developers
built with gcc 4.9.2 (Ubuntu 4.9.2-10ubuntu13)
configuration: --prefix=/usr --extra-version=0ubuntu0.15.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --shlibdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --enable-shared --disable-stripping --enable-avresample --enable-avisynth --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libshine --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libwavpack --enable-libwebp --enable-libxvid --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzvbi --enable-libzmq --enable-frei0r --enable-libvpx --enable-libx264 --enable-libsoxr --enable-gnutls --enable-openal --enable-libopencv --enable-librtmp --enable-libx265
libavutil 54. 15.100 / 54. 15.100
libavcodec 56. 13.100 / 56. 13.100
libavformat 56. 15.102 / 56. 15.102
libavdevice 56. 3.100 / 56. 3.100
libavfilter 5. 2.103 / 5. 2.103
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100
Trailing options were found on the commandline.
[mpeg4 @ 0x1508320] hmm, seems the headers are not complete, trying to guess time_increment_bits
[mpeg4 @ 0x1508320] my guess is 12 bits ;)
[mpeg4 @ 0x1508320] hmm, seems the headers are not complete, trying to guess time_increment_bits
[mpeg4 @ 0x1508320] my guess is 12 bits ;)
[mpeg4 @ 0x1508320] looks like this file was encoded with (divx4/(old)xvid/opendivx) -> forcing low_delay flag
[mpeg4 @ 0x1508320] Error, header damaged or not MPEG4 header (f_code=0)
Last message repeated 3 times
Input #0, matroska,webm, from 'movie.mkv':
Metadata:
COMPATIBLE_BRANDS: isommp42
MAJOR_BRAND : mp42
MINOR_VERSION : 0
ENCODER : Lavf56.36.100
Duration: 02:07:28.24, start: 0.000000, bitrate: 1190 kb/s
Stream #0:0(und): Video: mpeg4, yuv420p, 704x384, SAR 65:67 DAR 715:402, 29.97 fps, 29.97 tbr, 1k tbn, 1k tbc (default)
Metadata:
LANGUAGE : und
HANDLER_NAME : VideoHandler
ENCODER : Lavc56.41.100 libxvid
Stream #0:1(und): Audio: ac3, 44100 Hz, 5.1(side), fltp, 128 kb/s (default)
Metadata:
CREATION_TIME : 2014-06-30 13:43:16
LANGUAGE : und
HANDLER_NAME : IsoMedia File Produced by Google, 5-11-2011
ENCODER : Lavc56.41.100 ac3
At least one output file must be specified

terry@TP500LA:~/Videos/media/volume$
[/data]


I hope this is better for you. I have a lot to learn about protocol. Is the [data] part correct?

No, somehow the data part is incorrect. Before I go on, it is not 'data' tags but 'code' tags you will need to use to have the data preserved correctly.

So somehow our data is not coming out correctly. If i take the extract you provided:
And place that in a file and then run the sed provided on the file, I get the following:
So when you are doing the echo, The only thing you should see after all the code is run is:
The fact that you are seeing all the output from ffmpeg would seem to imply one of 2 things:

1. Somehow you are not showing the correct code with the correct output (I doubt this as you have done it several times)
2. The output being displayed is the data being sent to stderr


My issue with option 2 is that even if that was all the data going to stderr we should still see the correct output to our problem at the end as you are using the echo command to output the value of the
variable.

Would you please try the output without assigning to a variable, ie. show me the outcome of the following pasted into your terminal:

Copied and pasted your code into a folder containing one video file named "movie.mkv", and hit enter. Made no changes whatsoever. Here, in entirety, is the result:

terry@TP500LA:~/Videos/media/volume$ ffmpeg -i movie.mkv -af volumedetect | sed -nr '/max_volume:/ s/.*[[:space:]]-?([[:digit:]\.]+)[[:space:]]+dB$/-vf "volume=\1"/p'
ffmpeg version 2.5.8-0ubuntu0.15.04.1 Copyright (c) 2000-2015 the FFmpeg developers
built with gcc 4.9.2 (Ubuntu 4.9.2-10ubuntu13)
configuration: --prefix=/usr --extra-version=0ubuntu0.15.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --shlibdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --enable-shared --disable-stripping --enable-avresample --enable-avisynth --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libshine --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libwavpack --enable-libwebp --enable-libxvid --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzvbi --enable-libzmq --enable-frei0r --enable-libvpx --enable-libx264 --enable-libsoxr --enable-gnutls --enable-openal --enable-libopencv --enable-librtmp --enable-libx265
libavutil 54. 15.100 / 54. 15.100
libavcodec 56. 13.100 / 56. 13.100
libavformat 56. 15.102 / 56. 15.102
libavdevice 56. 3.100 / 56. 3.100
libavfilter 5. 2.103 / 5. 2.103
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100
Trailing options were found on the commandline.
[mpeg4 @ 0x16ae320] hmm, seems the headers are not complete, trying to guess time_increment_bits
[mpeg4 @ 0x16ae320] my guess is 12 bits ;)
[mpeg4 @ 0x16ae320] hmm, seems the headers are not complete, trying to guess time_increment_bits
[mpeg4 @ 0x16ae320] my guess is 12 bits ;)
[mpeg4 @ 0x16ae320] looks like this file was encoded with (divx4/(old)xvid/opendivx) -> forcing low_delay flag
[mpeg4 @ 0x16ae320] Error, header damaged or not MPEG4 header (f_code=0)
Last message repeated 3 times
Input #0, matroska,webm, from 'movie.mkv':
Metadata:
COMPATIBLE_BRANDS: isommp42
MAJOR_BRAND : mp42
MINOR_VERSION : 0
ENCODER : Lavf56.36.100
Duration: 02:07:28.24, start: 0.000000, bitrate: 1190 kb/s
Stream #0:0(und): Video: mpeg4, yuv420p, 704x384, SAR 65:67 DAR 715:402, 29.97 fps, 29.97 tbr, 1k tbn, 1k tbc (default)
Metadata:
LANGUAGE : und
HANDLER_NAME : VideoHandler
ENCODER : Lavc56.41.100 libxvid
Stream #0:1(und): Audio: ac3, 44100 Hz, 5.1(side), fltp, 128 kb/s (default)
Metadata:
CREATION_TIME : 2014-06-30 13:43:16
LANGUAGE : und
HANDLER_NAME : IsoMedia File Produced by Google, 5-11-2011
ENCODER : Lavc56.41.100 ac3
At least one output file must be specified
terry@TP500LA:~/Videos/media/volume$

Ok ... I had to go and do some experimenting and I found that when no output file is supplied that the data displayed to screen is all sent to stderr.

There also seemed that i could not get ffmpeg to return your initial values, including max_volume, using your ffmpeg command submission. However, you seem to say it works for you so to get the
sed working we will simply need to send stderr through the pipe as well.

See how this goes:

Gosh darn it Grail, pardon my language, I thought we had it, but no such luck.

Copied your sed to terminal, and got the following:

terry@TP500LA:~/Videos/media/volume$ ffmpeg -i movie.mkv -af volumedetect |& sed -nr '/max_volume:/ s/.*[[:space:]]-?([[:digit:]\.]+)[[:space:]]+dB$/-vf "volume=\1"/p'
terry@TP500LA:~/Videos/media/volume$

I tried inserting a space between the pipe and the ampersand, like so:

result:

terry@TP500LA:~/Videos/media/New Folder2$ ffmpeg -i movie.mkv -af volumedetect | & sed -nr '/max_volume:/ s/.*[[:space:]]-?([[:digit:]\.]+)[[:space:]]+dB$/-vf "volume=\1"/p'
bash: syntax error near unexpected token `&'
terry@TP500LA:~/Videos/media/New Folder2$

then I tried removing the ampersand:

terry@TP500LA:~/Videos/media/volume$ ffmpeg -i movie.mkv -af volumedetect | sed -nr '/max_volume:/ s/.*[[:space:]]-?([[:digit:]\.]+)[[:space:]]+dB$/-vf "volume=\1"/p'
ffmpeg version 2.5.8-0ubuntu0.15.04.1 Copyright (c) 2000-2015 the FFmpeg developers
built with gcc 4.9.2 (Ubuntu 4.9.2-10ubuntu13)
configuration: --prefix=/usr --extra-version=0ubuntu0.15.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --shlibdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --enable-shared --disable-stripping --enable-avresample --enable-avisynth --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libshine --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libwavpack --enable-libwebp --enable-libxvid --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzvbi --enable-libzmq --enable-frei0r --enable-libvpx --enable-libx264 --enable-libsoxr --enable-gnutls --enable-openal --enable-libopencv --enable-librtmp --enable-libx265
libavutil 54. 15.100 / 54. 15.100
libavcodec 56. 13.100 / 56. 13.100
libavformat 56. 15.102 / 56. 15.102
libavdevice 56. 3.100 / 56. 3.100
libavfilter 5. 2.103 / 5. 2.103
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100
Trailing options were found on the commandline.
[mpeg4 @ 0x23d4400] hmm, seems the headers are not complete, trying to guess time_increment_bits
[mpeg4 @ 0x23d4400] my guess is 12 bits ;)
[mpeg4 @ 0x23d4400] hmm, seems the headers are not complete, trying to guess time_increment_bits
[mpeg4 @ 0x23d4400] my guess is 12 bits ;)
[mpeg4 @ 0x23d4400] looks like this file was encoded with (divx4/(old)xvid/opendivx) -> forcing low_delay flag
[mpeg4 @ 0x23d4400] Error, header damaged or not MPEG4 header (b_code=0)
[mpeg4 @ 0x23d4400] Error, header damaged or not MPEG4 header (f_code=0)
Last message repeated 1 times
Input #0, matroska,webm, from 'movie.mkv':
Metadata:
ENCODER : Lavf56.36.100
Duration: 02:19:11.42, start: 0.000000, bitrate: 1056 kb/s
Stream #0:0(eng): Video: mpeg4, yuv420p, 704x384, SAR 65:67 DAR 715:402, 29.97 fps, 29.97 tbr, 1k tbn, 1k tbc (default)
Metadata:
LANGUAGE : eng
ENCODER : Lavc56.41.100 libxvid
Stream #0:1(eng): Audio: ac3, 32000 Hz, 5.1(side), fltp, 128 kb/s (default)
Metadata:
LANGUAGE : eng
ENCODER : Lavc56.41.100 ac3
Stream #0:2(eng): Subtitle: ass (default)
Metadata:
LANGUAGE : eng
ENCODER : Lavc56.41.100 ass
At least one output file must be specified
terry@TP500LA:~/Videos/media/volume$

This has totally got me stumped, thanks for sticking with this.

Well as I stated, your original ffmpeg command does not yield the results you have mentioned previously:
The above for me and now for you, ignores the last part, ie. -af volumedetect

This means you are getting no good values from your command, which also means that the command you typed first has worked perfectly as there is no line in the data with "max_volume" in it.


Here is what i had to do to get the type of results you originally mentioned:
If I then apply the sed I get:
Yes the pipe and ampersand must remain together as this tell the previous command to pass both stdout & stderr to next command, which we want.

Well, grail, it seems rather strange that we get different outputs from the same sed. I got to thinking that, since distros tailor different software to suit their needs that maybe, just maybe that is what happened to bash in my Xubuntu 15.04 lappie.

I have my desktop, which has been giving me grief (not asking for help), and have determined that there is an issue with the SSD, so I loaded the OS on the HDD I used for /home, but Xubuntu 15.o4 freezes (apparently I'm not the only one), therefor I have decided to install a different distro altogether and try the sed in that environment. It can't hurt, the box is unusable as is, and I can learn a different OS. Sounds like a win to me. Of course, I will report my findings here.

I just wanted to thank you for all you have done to help. It can be very frustrating when all your hard work bears no fruit, so I should also thank you for your patience. I wouldn't blame you if all this was some jerk yanking your chain for a laugh. It isn't. If I can't get it working soon I will have to abandon the whole thing until I learn a lot more.

Regards.

At a stand still.
 

Thanks for all your help grail, but for now I can not go any further. I wanted to try this using a different OS, so I was determined to breathe life back into my desktop. Long story short, don't dd an SSD.

I installed Fedora to the drive I intended to use for storage. It kept hanging.Again, long story short, I did not believe the guy who told me that installing water cooling in a box could mess up your ram. Since running memtest86 is free and all you need is a live cd, I gave it a go. Ouch! I couldn't believe to number of errors!

I live in the capital city of my province. Checking online I could not find a retailer with a pair of 8Gb sticks of 240 pin ddr3 ram. You would think it so common they'd give it away free in cereal boxes, but I could place an order for delivery. Which I did through amazon.ca.

Just got an email from the supplier. Mui big SNAFU. Unable to fill order.

Ever get the feeling you should just curl up in a little ball and cry?

Canceled order from that supplier and reordered same from different source who should be able to fill order in under 1000 years.

The clerk who sold me the new SSD and told me about the ram also mentioned possible damage to the CPU, so I'm going to try it under some pretty nasty burn in.

Charlie Brown could not fly a kite because of the kite eating tree. I'm beginning to understand how he feels.

Thanks again for all you help. Now please forgive me whule I pinch myself to see if I can wake from this nightmare.

ok
 

Ok, maybe I'll try and install something on a stick, or give virtualbox a whirl, but for now I am really burned out.

A little farther along
 

After letting my overheated brain cool off for a few days I took another look at the line with the sed in it. I got to thinking about the error message "At least one output file must be specified."

No, it can't be that easy.

I changed the line ffmpeg -i movie.mkv -af volumedetect |& sed -nr '/max_volume:/ s/.*[[:space:]]-?([[:digit:]\.]+)[[:space:]]+dB$/-vf "volume=\1"/p'

to read

ffmpeg -i movie.mkv -af volumedetect -f null /dev/null |& sed -nr '/max_volume:/ s/.*[[:space:]]-?([[:digit:]\.]+)[[:space:]]+dB$/-vf "volume=\1"/p';
echo "${vfvol}"

Now, syg00's sed was perfectly fine, sorry for ever doubting you.

It immediately started chugging away happily chewing data and viola! out came the whole data, max_voulume included. I'm just having trouble putting it all together.

The next step is to take the value of $vfvol, apply af volume= and outputting the video file. Any help, hints or suggestions?

I have <code>#!/bin/bash

mkdir ./encoded1
for $movie in *.mkv ; do
vfvol=$(ffmpeg -i $movie -af volumedetect -f null /dev/null | sed -nr '/max_volume:/ s/.*[[:space:]]-?([[:digit:]\.]+)[[:space:]]+dB$/-vf "volume=\1"/p');
echo $vfvol
ffmpeg -i $movie -af volume=$vol ./encoded1/$movie;
done<code>

And what I get back is:
./adjust volume.sh: line 8: `$movie': not a valid identifier

I'm probably missing something small, but boy, do I have a lot to learn.

Well I am glad you are feeling better :) Unfortunately it is not helping you to read the previous suggestions correctly :( or at least not interpreting them correctly.

So i will start from top of post #29:

1. Please always use [code][/code] tags around ALL code. (you need to note the use of square brackets [] )

2. Whilst the file is not really needed it is neither here or there if it is included. What you need to pay attention to is the output:
Based on the above output, your new script would not work either as the line you have changed would now read as below assuming the same file was used and the shell has already interpreted your variable:
As you can see, the part in red is incorrect

3. Your new error is due to your for loop and the use of $movie, when it should be just movie as you want to store data in the variable and not read from it

4. So I believe your new script should look like the following (I have also corrected the incorrect variable names used):
Using the previous example I believe this would set the volume to now be 91.0

Didn't quite work out the way I expected grail.

Line 7 says Now, when I run the volumedetect on its own, the result is a negative number. In order to counter that I need a positive number, but can't see if it's being done. do I not have to multiply vfvol X -1?

Other than that my build insists the codec is experimental and to add .

Its so close I can almost taste it.

grail, we've been proceeding on the assumption that running
will give us an output similar to

[Parsed_volumedetect_0 @ 0x16c14a0] n_samples: 462422016
[Parsed_volumedetect_0 @ 0x16c14a0] mean_volume: -26.7 dB
[Parsed_volumedetect_0 @ 0x16c14a0] max_volume: -1.9 dB
[Parsed_volumedetect_0 @ 0x16c14a0] histogram_1db: 1
[Parsed_volumedetect_0 @ 0x16c14a0] histogram_2db: 274
[Parsed_volumedetect_0 @ 0x16c14a0] histogram_3db: 1734
[Parsed_volumedetect_0 @ 0x16c14a0] histogram_4db: 3879
[Parsed_volumedetect_0 @ 0x16c14a0] histogram_5db: 8453
[Parsed_volumedetect_0 @ 0x16c14a0] histogram_6db: 19589
[Parsed_volumedetect_0 @ 0x16c14a0] histogram_7db: 43251
[Parsed_volumedetect_0 @ 0x16c14a0] histogram_8db: 87977
[Parsed_volumedetect_0 @ 0x16c14a0] histogram_9db: 164030
[Parsed_volumedetect_0 @ 0x16c14a0] histogram_10db: 280869


and that we have picked up a quantifiable value for vfvol, but we have not Something seemed off, so I ran
[code]
#! /bin/bash
mkdir ./encode
for IFS in *.mp4; do
vfvol=$(ffmpeg -i "${IFS}" -af volumedetect -f null /dev/null| sed '/max_volume:/ s/\ \-[0-9]+\.[0-9]\ dBs/-vf "volume=\p')
echo "1"
echo ${vfvol}
done
[\code]
and the result was:
[Parsed_volumedetect_0 @ 0x16c14a0] n_samples: 462422016
[Parsed_volumedetect_0 @ 0x16c14a0] mean_volume: -26.7 dB
[Parsed_volumedetect_0 @ 0x16c14a0] max_volume: -1.9 dB
[Parsed_volumedetect_0 @ 0x16c14a0] histogram_1db: 1
[Parsed_volumedetect_0 @ 0x16c14a0] histogram_2db: 274
[Parsed_volumedetect_0 @ 0x16c14a0] histogram_3db: 1734
[Parsed_volumedetect_0 @ 0x16c14a0] histogram_4db: 3879
[Parsed_volumedetect_0 @ 0x16c14a0] histogram_5db: 8453
[Parsed_volumedetect_0 @ 0x16c14a0] histogram_6db: 19589
[Parsed_volumedetect_0 @ 0x16c14a0] histogram_7db: 43251
[Parsed_volumedetect_0 @ 0x16c14a0] histogram_8db: 87977
[Parsed_volumedetect_0 @ 0x16c14a0] histogram_9db: 164030
[Parsed_volumedetect_0 @ 0x16c14a0] histogram_10db: 280869
1

terry@TP500LA:~/shell scripts/1$


There, below the number one, we see that the value of vfvol is, for practical purposes, null. I would not want to accuse syg00 of making a mistake, in his sed, but either he did or I have screwed up somewhere along the way. Unless we can find a way to obtain and use the value of vfvol we are at a standstill. I don't know how good you are with sed, but I have proven that I do not make big bucks as a programmer.

I hate to give up, but I am in way over my head and can see it if you would prefer to spend your time more productively helping others. If, however, you want to continue I am open to suggestions. Either way I'd like to thank you for your time and patience, you've been a big help.

The above is completely wrong. We do not assume any such thing about the output.

I understand this is all new to you but you really need to read the information provided a lot more carefully.

So using the above I will start again:

1. First and foremost, do not use IFS as a variable

2. The output of the call to ffmpeg and the sed will be exactly as below (assuming the typos in your sed were a mistake):
3. Given the above is the exact output, this will error or give erroneous results if used in the following:
The reason the above will error or give wrong results is because the variable expands to what I have shown above.
So the once expanded it will look like:
As you can see, the part in red is all wrong. It should read:
You need to go back and re-read all the information supplied. syg00's sed works just fine as we had shown previously. I had also advised previously that his sed
will not provide the solution you want. You need to reduce it so it only returns the numbers you are interested in (which if you review post #30 you will see a solution)

grail, that code was from earlier on. I was using it to try and get an output for vfvol.

PLEASE FORGET ALL THAT!

I've been searching and cat napping and I just now discovered a work-around. Simply by adding
[code]
-report]
[\code]
to the line containing
[code]
-af volumedetect
[\code]

I was able to generate a file designated "ffmpeg-20150825-184828.log", which contained every line generated by volumedetect. Next, I tried
[code] grep -E "max_volume" [\code]

I won't post everything here, as I'm actually bouncing back and forth between two computers. I started off on my lappie but since I replaced the faulty ram, I started using Gargatubrain and a 50" tv, so the rest is over there. But long story made short, I have managed to isolate the line containing "max_volume."

My stress level is too high because I was obsessing over this. I knew there was an easy way to do it. The next thing to do is pump it through the sed, verify the output, and pop the variable into the last section where the volume adjustment is made and the final file generated. I'll also apply an if then to skip any file with a value of less than 1. I ran a file with a max_volume of 0.0db through volume=0.0 and the result was a disaster - a max_volume of -91db.

I really need a break now, but the end is in sight.

1) run volumedetect with -report in it.
2) grep -E "max_volume" on file "ffmpeg*.log"
3) run the sed for value of "max_volume"
4) if $ -lt 1 skip encoding. move to ~/Videos/Movies
5) apply to "volume=", move to Movies
6) delete log file.

That part was more for my benefit than anything else. I'm going to crash soon whether I want to or not, and it takes a bit for my brain to achieve optimal operating temperature. I'm not dumb, but this freaking depression really slows things down. If you want to write it, and you probably do just to see if it works, feel free.

Thanks for sticking with me through this, I never would have made it this far without you.

you know you can edit your posts, right?

you got the code tags right twice, but now it's worse than before:

[code]just some $CODE[\code]

this is wrong.
the backslash \ should be a slash /.

you know you can edit your posts, right?

and, sorry if i sound like a heartless a***hole, but if you're coding with the same level of attention as you're writing forum posts, no wonder it's difficult.
take it slow, most of all!
there's no hurry!
read it through, try it again, read it again, make some corrections...

so next time, post the whole script, and post the whole output of the script (incl. error messages).
how are we to know how far you've gotten along with improving your code?

btw, i hope you're using an editor with a monospaced font & syntax highlighting?

and please, use code tags.

ondoho, I think I've done pretty well, all things considered. I'm not sure what you know about depression, but that is what put me on disability. I still have a few years until I reach retirement age, but there is no way I could handle the stress. Stress worsens my depression which in turn further impairs my ability to think clearly. A few months ago my neighbor comes in telling me what he got me at the store as he began unloading the bags. When I asked him what this was all about as I really had no idea he told me that I had given him some money the day before to pick up a few things for me. If my mind can get so muddled because of my depression, which I did mention in a much earlier post, that I can't remember a conversation from the night before, what right have you to come down on me because I misremembered which way a slash goes?

I was so stressed and so depressed that when I awoke yesterday I laid there for nearly 90 minutes barely able to move. Yes, I am mentally ill, but I'm doing the best I can with what I've got.

Picture yourself with a brutal hangover trying to figure out how to accomplish a task you have little knowledge about in the first place. You know that persistant fog you just can't shake? That's what I have to contend with all day, every day of my life. Yes, you are being an A.....

I'll post my completed script in a moment. It is working right now. Then go back and see what others have contributed and what I, a scripting noob, have had to discover for myself. If you are fair and honest you just may be a little more tolerant in the future.

Success!
 

Edit: Not exactly right.
I wrote a little script to check the max_volume on the completed files and save the results to a log file. See post #43

I have been at this for, what, three days now and my script is finally working the way I want it too. It parses a single folder, calculates whay the volume level is for each file, and how far, if at all it is from the "ideal" level, designated 0.0 db. What got me started on this little project was having to constantly turn the volume up or down to a comfortable level. Say you just watched a movie with a max_volume of - 12.3 db, so you had to crank the volume way up. Later you play a video with a max_volume of 0.0 db and you're blasted out of your chair in a mad scramble for the remote. I learned that ffmpeg had an audio filter called "volumedetect" (-af volumedetect) that would give you a numerical value that you would then run a second instance of ffmpeg using the audio filter "volume: (-af volume=foo), changing a negative integer to a positive along the way.

For lack of a better term I call it volume leveling, making all video files have audio tracks at the same level. I seen music players that can apply the gain, the aforementioned "difference" across a series of audio tracks, but nothing for videos. Syg00 supplied the sed seen in the script, which worked perfectly to strip away unneeded characters to give me the value for gain once I figured out what to do with it. Special thanks to grail for support and encouragement while we were both crawling around in circles with our tongues nailed to the floor. Here is the final, hopefully, incarnation.

I didn't document it well, so I'll try to explain. Running volumedetect was straightforward, but how to take the value from the screen and put it in the script. Lots of mistakes were made, especially the idea that we could just push it through the sed. no such luck. What outputting to a file? More mistakes and false starts, lots and lots of crashing as my depression overwhelmed me and I pulled a Brian Wilson. (Open a tab and google the name, you might want to include "beach boys") Finally I stumbled upon " - report" which created a novel length file. More searching led me to the grep line.

I'm not going to try and explain grep, but found out that grep -Ev "foo123" would remove all lines with the words/letter/number/combination-of-these contained within the quotes, which was the opposite of what I wanted, but grep -E "foo123" would discard everything but that line. Hundreds, perhaps even thousands of syntax errors later I was able to isolate number-dot-number combination in the text file by pumping it through the sed so--- assign to variable. Since previous experience proved that using "0." for the volume yielded a file with "max_volume -91 db" a conditional loop -if/then/else brought on three separate, distinct eternities of inventing swear words as it seemed to bypass the loop and any file with "0.0" put out files with "-91 db" when tested.

More crashing, not caring if the world ended, more searching and "bash does not recognize floating point numbers" impinged on my consciousness, so, new variable of 10xvolume adjustment, fix if statement to reflect same and computer is now churning out files with the volumes adjusted.

Not bad for a scripting noob who keeps crashing, unable to move for hours.

Next project - automatically seek and unpacking packed bitstreams. Perhaps when I've had time to refine them I'll contact Winff ans see if they'll incorporate it in their GUI.

Well I am not sure you will want to read this, so you can ignore if you like, however, your script is riddled with problems that may not be obvious
but I would guess they do not help yield the results you are looking for.

I will take it from the top:

1. Upto the command, mkdir -p ./encoded ... all good

2. The variable 'fullfile' does not exist, so the 3 lines below serve no purpose
3. Your for loop re-uses the variable name that you attempted to set outside the loop. But as each iteration of the loop will populate the variable, anything previously set would be over-written

4. ffmpeg line works but I think we can use it straight with the sed instead of creating a file... will comeback to this

5. The grep line, apart from displaying on the screen that the 'max_volume' line exists, serves no purpose

6. Assuming the sed is now returning the correct data, it can just as easily be applied to the stream as the file: (I have used the below and confirm it works)
7. The above would of course replace your current setting of the 's' variable

8. I would suggest you place an echo in your script for the 't' variable. You can also do the following at the command prompt:
As you can see, 't' does not have what you expected. Also, bash does not perform arithmetic directly

9. As we are concerned with mainly round numbers, I would simply remove the decimal portion using your already learnt parameter substitution:
You will also notice the use of round brackets to perform arithmetic operations

10. Your current 'if' will always error and hence the 'else' portion will always run with any file you submit (see point 8 and try your test on the command line)

11. The 'if' has me a little confused (assuming it worked). I would think one of the commands is not required as you want to essentially leave the file with its current settings?? So i would play with your 'if' test and return only the one that needs to change the file (which would appear when it is true, ie. when you do not pass through -af volume)

12. I was also curious why your very first call to ffmpeg is the system one but when you alter the file you call a custom version. Why not just use the same one all the time so you get equivalent results for all parts of the program

13. Here I nearly said, why don't you place the 'mv' command after the 'if' so you only have to call it once, but then a few things caught my eye:

a. You are creating your newly encoded files under /home/terry/Videos/Movies/
b. You are moving a file of the format ... "Sfilename".mkv ... into the encoded directory, but as far as I can tell, this file does not exist anywhere.

An example for part 'b' above:

1. you currently have a file called movies.mp4
2. you encode this with the above script
3. this will create the following file: /home/terry/Videos/Movies/movies.mkv
4. you now attempt to 'mv' which will look like this: mv movies.mp4.mkv ./encoded - - - As you can see, there will never be a file with this name in this directory. So this will simply error.

14. You do not require a semi-colon (;) after every command. A new line in the script serves as the same purpose

Now i will have to leave the 'if' part to you as I am still not certain of the desired outcome there, however, the first portion could be as simple as;
As usual, hope some of that helps

Oh, I had no illusions that there was no room for improvement. What we have here is a bona fide Frankenstein's monster with bits and pieces literally stolen from many different sources, mainly forums. I knew those three lines were almost useless, but for some reason even though it was encoding in mkv, for some reason it wrote the extension as mp4. I verified the encoding method first, then copied that over to output the right extension. Your way of dealing with "s" is a lot simpler than the "comparable in length to War and Peace" method I started off with. I figured a new variable was a lot easier than trying to come up with a more elegant solution. I had taken a pretty nasty downturn taking nearly 90 minutes after waking one morning until I was able to get moving. Some people eat when they're depressed, I go horizontal, getting out of bed only when hydralic pressure forces me to make that trip down the hall.

For the moment it is doing what I intended and I'm crawling out of that hole I was in. I did learn from the experience and even knew the proper answers to some of the questions posed by others from my research. I will probably go through and refine it in the near future. I will likely need it again in the future, and learning a better way is worth the effort.

In short, I think of my script as an attempt at art by employing explosives as compared to, say, Rodin's sculpture, "The Thinker."

Thanks again for the help. I probably would have given up without the effort you were putting in to help someone you've never met.

Your -vol type stuff is over thinking it IMO.

cat $VAR_of_FILES | ffmpeg -i -

Although if your intent it to concatenate several movies into one long one. I tend to -f rawvideo and .raw the individual clips. Then "cat clips*.raw > newclip.raw" and run the encode on the newclip.raw file. You chew up a lot of temporary storage that way, but something that's possible in every version of ffmpeg for over a decade. Audio of course needs separate handling but basically the same approach.

You would not believe how widely they vary. Besides, I have 24/7/365 on my hands and not a whole hell of a lot to fill it with. While my depression does dull my thinking, I don't want my brain to atrophy too much and a gordian knot type of problem does tend to keep one's brain alive, even if I do have to tolerate those hours of oblivion I get when I crash.

Hey grail! I was just looking back on the script and realized you brought up a point that I did not explain properly.

you wrote:
An example for part 'b' above:
Allthough it was encoding in matroska it appended the file with "mp4", hence the crap up top I just copied and pasted. It strips away everything past "filename" allowing me to input mkv for the extenstion. Parsing the ffmpeg you are correct about the input "movie.mp4". It re-encodes the file with the corrected extension, giving me ./movie.mkv in the subfolder and halts. The semi-colon, of course indicates a new command to mv the file movie* to ./converted. Now we do not have two files in the folder, only one. mkv is in a child folder leaving only the original movie.mp4 to move.

Upon completion it goes back to the top, picks up a different file and goes through the loop again.

It confused the hell out of me 30 odd years back when I learned B.A.S.I.C., might look strange at first but there is never a point when two files with the exact same name or prefix are trying to co-exist in the same folder at the same time.

Regards.

Still not right
 

Having an issue with completed files having a max_volume of -91 db, which happens when you apply -af volume=foo. Working on it.

You need to go back and look at your code and what I have written as in this case I am not talking about 2 files with same name but the fact you are trying to move a file which does not exist at all.

Your latest issue also does not surprise me as ALL your files will undergo the same process as your 'if' was not working as intended

grail, if you're referring to that bit you posted about the if statement, I finally figured out that I forgotten to change that. As I am sure you know, one rule of diagnoses is to change ONE thing and see how that affects the outcome, which is what I did and forgot to go back to your stuff. I corrected that last night and everything seems to be fine. Here is the version I'm running now:

Figuring a little paranoia can be a good thing I altered the hell out of it, making it only do the volumedetect and appending it to a list, and running it on the destination directory. It seems that the odd one slips through, perhaps recoding the file interferes somehow, and make the adjustment manually.

And just for the hell of it, the command for the volume adjust by itself because when I was first learning Linux I'd be searching for answers and some of them were almost what I wanted, but lacked the skill to adapt it.

I usually just make a new folder named "1", put the new file in there, verify it, and if it is good cut and paste it up one level to overwrite the file there.

Finally, for those less experienced, the filter -af volume=foo automagically sets the video codec to copy so there is no need to add that to the command.

It is long past time to close this thread. Thank you again to those who offered their help. I never seem to learn anything from "Hello World" exercises, but let me dissect something where I've really got to use the old cocoanut and I get a far deeper understanding. My confidence level is way up. I have successfully written a little script to seek out avi files with packed bitstreams and unpack them. It isn't perfect as it places all the unpacked in the parent directory, but not as many people are using packed bitstreams these days and it beats the hell out of checking each one separately.

Now I just have close to 1000 files to work on.

Gosh darn! Heck! Spit! I thought I had it, but no joy. The hang-up is in reading the floating point number from ffmpegnew.log. Bash does not recognize floating point numbers so can't perform arithmatic functions on them. Someone with experience in sed or awk could remove the decimal point and use that number in the if statement. Python, Ruby, Perl and a bunch of other goodies can manipulate them but I can't find out how to do that while/before assigning to a variable. Then there's zsh.....

I'm going to
a) drink too much Kool-Aid
b) transcode all my files to mkv
c) run d) consult list.txt and make the adjustments manually.

I started on this so I wouldn't have to check each file by hand, and the little script above will save countless hous. Not exactly what I had envisioned, but half a loaf is better than none. This is not a failure as I learned from it. No venture where you walk away wiser is ever a total waste of time.

You can use "bc" to do floating point math in bash. But your comparisons will be basically string literals in bash. You can also multiply by 10/100/1000 to simulate floating point in whole numbers. Which gets a little cumbersome if the input is floating point numbers. Various means to an end. Any reason for not using ffprobe for getting the video details?

I tend to prefix my videos with the specs. p30r1280x720__some_video.webm . It makes using wild cards based on specs for playback a little simpler.

You didn't really think I was going to quit this close to the final solution, did you? Took some "Mental health" time.

Thanks, shadow7. I looked in to that but my brains were pretty well fried at the time and ended up exploring a different avenue. There is more than one possible route to a workable solution and I found one I'm happy with.

Point: Bash does not recognize floating point numbers. That is to say numbers with decimals. There are a number of ways around this. You can invoke a bit of code from python, or other language, as shadow7 pointed out, you can use bc as well to get your variable, say 8.3 for example, and adjust your volume level that way. I took a different route.

I installed zsh, with recommends, changed

to
reworked the formula for the variable to add spaces where needed, and, after banging my head for a while, discovered that the higher math functions were not enabled by default and how to invoke them. I was going for an all-in-one solution but occasional errors crept in, possibly due to transcoding the files.

Step 1 obtained a copy of unpackmp4.jar from
http://www.xmixdrix.com/tools/unpackmp4.html
and wrote a little script to search for files with packed bitstreams and unpacked them.

Make sure you put a copy of unpackmp4.jar in the parent folder and it will recurse through all folders in the branch, placing the unpacked file in the parent folder, appending the word "unpack" without the quotes before the extension. The are lots of renaming programs that are easy to use. I used "pyrenamer" to remove "unpack", then manually cut and paste them into their respective folders, overwriting the originals.

Step 2: Transcode your videos to the desired format. There are a myriad numbers of programs to do this for you and I don't feel it necessary to tell you how you should go about this. Having over 1700 files I moved them all into one folder and wrote a loop script with my parametersb (copied from a winff preset) and as I have way more disk space than I need, moved the original file to a new directory. That way if my machine went down for and reason (blown power transformer a couple of days ago, took them about three hours to restore power), I can simply restart my script and it picks up where it left off.

Step 3: THE FUN PART

I lied, but here is where we make the actual adjustment. The script runs -af volumedetect against each file in sequence, outputs what would normally be echoed on your screen to a log file, strips away every line that does not contain "max_volume", then the sed removes everything else, saving the value to a new log file, adjusts the volume level and on to the next file.

The file name and the adjustment value are appended to list.txt, the file with the adjusted volume goes in a subfolder for verinifation.

Note: copy both volumeadjust.sh and volumedetect.sh to working folder

volumeadjust.sh

Step 4:
cd ./encoded
Run "volumedetect.sh", which will run -af volumedetect against each file, appending to list.txt. If you are happy with the results move to final destination. Personally I move them up one directory, overwriting the original to avoid duplicated files with different volume levels.

volumedetect.sh


Step 5: POPCORN - you're on your own on this one.

Once again thanks for all the input. I'm not a programmer and this whole inelegant mess has been kluged together in what may seem to be a real tangle, but every line is there for a reason, if it doesn't do something, it fixes something.

I'm going to watch a video now. Good luck.