[SOLVED] ffmpeg to join mp4 files

GPGAgent · 01-20-2019, 11:03 AM

After upgrading from Mint 18 to Mint 19 (64bit) some of my ffmpeg scripts no longer work.
I've googled the error message and I cannot find a solution so here goes:

Files CROPPED-36.mp4 and CROPPED-37.mp4

The script:

Code:

ls CROPP*.mp4 | perl -ne 'print "file $_"' | ffmpeg -f concat -i - -c copy CROPP.mp4

The error message (after all the ffmpeg preamble):

Code:

[file @ 0x556f060a6000] Protocol 'file' not on whitelist 'crypto'!
[concat @ 0x556f060a3f40] Impossible to open 'CROPPED-PICT0036.mp4'
pipe:: Invalid argument

I have another method that works fine which is this:

Code:

[ -e list.txt ] && rm list.txt
for f in CROPP*.mp4
do
   echo "file $f" >> list.txt
done

ffmpeg -f concat -i list.txt -c copy  CROPP.mp4 && rm list.txt

I'm guessing the first method isn't generating the list of files to join, which points to the perl statement, bu I'm no expert with perl.

I'm sure someone can help so thanks in advance

hydrurga · 01-20-2019, 12:50 PM

I think it may have been due to this ffmpeg update:

https://ffmpeg.org/pipermail/ffmpeg-...ch/098694.html

See https://blog.yo1.dog/fix-for-ffmpeg-...rror-for-urls/ for a solution, namely adding:

Code:

-safe "0" -protocol_whitelist "file"

near the start of the command.

GPGAgent · 01-20-2019, 05:06 PM

Quote:

Originally Posted by hydrurga

I think it may have been due to this ffmpeg update:

https://ffmpeg.org/pipermail/ffmpeg-...ch/098694.html

See https://blog.yo1.dog/fix-for-ffmpeg-...rror-for-urls/ for a solution, namely adding:

Code:

-safe "0" -protocol_whitelist "file"

near the start of the command.

You're on the right track I think, but something else is missing, here's the command:

Code:

ls M*.mp4 | perl -ne 'print "file $_"' | ffmpeg -f concat -safe "0" -protocol_whitelist "file" -i - -c copy M-Joined.mp4

And the error message:

Code:

[pipe @ 0x56305401e9e0] Protocol 'pipe' not on whitelist 'file'!
pipe:: Invalid argument

Any ideas? Make sure you read the following post with a bit more detail.

GPGAgent · 01-20-2019, 05:24 PM

So just checking through my script I can't see why it's not working
I have these files:

Code:

ls MOV*.avi
MOVI0003.avi  MOVI0004.avi

And this code:

Code:

ls MOV*.avi | perl -ne 'print "file $_"'

Produces this:

Code:

file MOVI0003.avi
file MOVI0004.avi

Which gets piped to:

Code:

ffmpeg -f concat -safe "0" -protocol_whitelist "file" -i - -c copy M-Joined.avi

It should work, because it's effectively the same as what follows, but using a file - list.txt.

What does work is this:

Code:

ffmpeg -f concat -i list.txt -c copy  M-Joined.avi

Where list.txt contains this:

Code:

file MOVI0003.avi
file MOVI0004.avi

So any ideas folks.

hydrurga · 01-20-2019, 07:13 PM

You need to add pipe to the whitelist.

GPGAgent · 01-21-2019, 04:40 AM

Quote:

Originally Posted by hydrurga

You need to add pipe to the whitelist.

Cheers, that worked just fine, obvious really, so it now looks like this:

Code:

ffmpeg -f concat -safe "0" -protocol_whitelist "file,pipe" -i - -c copy "${FileName/

works fine for mp4 file but not for .mov, here's the error:

Code:

[mov @ 0x55db06663900] fatal error, input packet contains no samples
av_interleaved_write_frame(): Not yet implemented in FFmpeg, patches welcome
[mov @ 0x55db06663900] fatal error, input packet contains no samples
Error writing trailer of 04-Joined.MOV: Not yet implemented in FFmpeg, patches welcome
frame=    2 fps=0.0 q=-1.0 Lsize=     109kB time=00:00:00.06 bitrate=13997.3kbits/s speed=31.4x    
video:108kB audio:2kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Conversion failed!

Bit if I use my other method, echoing the file list into an actual file (see earlier post) like this:

Code:

ffmpeg -f concat -i list.txt -c copy  M-Joined.mov

where list.txt contains the list of files to join:

Code:

file MOVI0003.mov
file MOVI0004.mov

This works, it's using the same ffmpeg and switches?

Any ideas? Cheers

GPGAgent · 01-21-2019, 01:02 PM

Further investigation is a bit mystifying, basically the -f concat option works if you have an actual file. So can someone explain why one statement works and the other doesn't?

The folder contains:

Code:

-rwxrwxrwx 1 charlie charlie 22503233 Nov 15  2017 CRP8.mp4
-rwxrwxrwx 1 charlie charlie 23221581 Nov 15  2017 CRP9.mp4

This statement works:

Code:

ls CRP*.mp4 | perl -ne 'print "file $_"' > LLL.txt | ffmpeg -y -f concat  -i LLL.txt -c copy JOINED-CRP.mp4

This statement does not work:

Code:

ls CRP*.mp4 | perl -ne 'print "file $_"' | ffmpeg -y -f concat -i - -c copy JOINED-CRP.mp4

The error message is this:

Code:

  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
[file @ 0x560e19057c80] Protocol 'file' not on whitelist 'crypto'!
[concat @ 0x560e19055f40] Impossible to open 'CRP8.mp4'
pipe:: Invalid argument

Okay I know I can put the protocol and whitelist switches into the statement, but the statement that works, using a file to hold the list of files to join doesn't need these switches, that's the puzzle.

Looking forward to an explanation, over to you guys and gals.

ondoho · 01-21-2019, 02:23 PM

Quote:

Originally Posted by GPGAgent

basically the -f concat option works if you have an actual file.

that is my experience also.
maybe you can use a here string instead of an actual file?

GPGAgent · 01-21-2019, 04:43 PM

Quote:

Originally Posted by ondoho

that is my experience also.
maybe you can use a here string instead of an actual file?

My point is that the -f concat with a - did work just fine with Mint 18. Now I've upgraded to Mint 19 so has ffmpeg and my old script no longer works

Both of these scripts worked on Mint 18

Code:

ls CRP*.mp4 | perl -ne 'print "file $_"' | ffmpeg -y -f concat -i - -c copy JOINED-CRP.mp4
ls CRP*.mp4 | perl -ne 'print "file $_"' > LLL.txt | ffmpeg -y -f concat -i LLL.txt -c copy JOINED-CRP.mp4

but now only this one works:

Code:

ls CRP*.mp4 | perl -ne 'print "file $_"' > LLL.txt | ffmpeg -y -f concat -i LLL.txt -c copy JOINED-CRP.mp4

Read through the entire thread and you'll see how I got here.

Thanks for your time anyway.

ondoho · 01-22-2019, 12:52 AM

Quote:

Originally Posted by GPGAgent

My point is that the -f concat with a - did work just fine with Mint 18. Now I've upgraded to Mint 19 so has ffmpeg and my old script no longer works

so your point is not to find a solution for your problem, but to complain about ffmpeg's behavior having changed?

l0f4r0 · 01-22-2019, 04:45 AM

@GPGAgent: I don't have your answer but the last pipe in the following command seems useless to me:

Code:

ls CRP*.mp4 | perl -ne 'print "file $_"' > LLL.txt | ffmpeg -y -f concat -i LLL.txt -c copy JOINED-CRP.mp4

You should try:

Code:

ls CRP*.mp4 | perl -ne 'print "file $_"' > LLL.txt && ffmpeg -y -f concat -i LLL.txt -c copy JOINED-CRP.mp4

GPGAgent · 01-22-2019, 05:08 AM

Quote:

Originally Posted by ondoho

so your point is not to find a solution for your problem, but to complain about ffmpeg's behavior having changed?

I'm not complaining, I don't know how you come to that conclusion. Thanks anyway, everyday with linux is a school day, you never stop learning which is why this is great forum.

Some really knowledgeable people here and really helpful.

GPGAgent · 01-22-2019, 05:11 AM

Quote:

Originally Posted by l0f4r0

@GPGAgent: I don't have your answer but the last pipe in the following command seems useless to me:

Code:

ls CRP*.mp4 | perl -ne 'print "file $_"' > LLL.txt | ffmpeg -y -f concat -i LLL.txt -c copy JOINED-CRP.mp4

You should try:

Code:

ls CRP*.mp4 | perl -ne 'print "file $_"' > LLL.txt && ffmpeg -y -f concat -i LLL.txt -c copy JOINED-CRP.mp4

Okay, interesting point, both work as expected

Thanks

GPGAgent · 01-22-2019, 12:52 PM

Quote:

Originally Posted by hydrurga

I think it may have been due to this ffmpeg update:

https://ffmpeg.org/pipermail/ffmpeg-...ch/098694.html

See https://blog.yo1.dog/fix-for-ffmpeg-...rror-for-urls/ for a solution, namely adding:

Code:

-safe "0" -protocol_whitelist "file"

near the start of the command.

Well I think this is solved, I re-read those links, and missed how important the positioning of the -protocol_whitelist switch was.

So this now works fine:

Code:

ls 19*.mp4 | perl -ne 'print "file $_"' | ffmpeg -protocol_whitelist "file,pipe" -f concat -i - -c copy JOINED-FIL.mp4

I've dropped the -safe switch for now, but may need it problems occur.

Cheers and thanks for your help and pointer to the solution