ffmpeg - You can have Sound, or Video - but not both

astanton · 06-12-2011, 10:04 PM

I am unable to screencast with sound and video with ffmpeg

I can record sound. I can record video - but if I try to do both I only get video.

All I want to do is upload a few tuts to vimeo or youtube, mostly shell stuff in xterm, or some config/install demonstrations on the desktop.

I can do this with recordmydesktop no problem, except for the video quality.

This has apparently stumped the developers or those who are in #ffmpeg, I've been patiently waiting and asking there for a couple of days now.

And on the ffmpeg-users email list, the only response to date was by someone who suggested I leave out the -f which I was pretty sure would only result in an error condition - which is did.

Most everything regarding this problem that I've gone through so far is pasted at this link:

http://pastie.org/2059044

If you can help I would sure appreciate it, since this has stumped several folks who are much smarter than me, and I'm a bit concerned about that.

Maybe it's time (Or not) to submit a bug report?

Well, I' hoping that we can find a solution (or at least determine what's going wrong) here first.

The problem appears to be reproducible by a few people so far.

Once again, the details are at the link below, in order to not make people's eyes glaze over LOL

http://pastie.org/2059044

.

business_kid · 06-13-2011, 03:10 AM

I'd first try to record an audio mp3, or mp4. You're going to .wav. But when you gop to an AV format, does audio work?

teckk · 06-13-2011, 05:35 PM

http://www.linuxquestions.org/questi...torial-880858/

http://www.ffmpeg.org/ffmpeg-doc.html

Post the syntax you are using.

If alsa does not work, have you tried oss? Are you using pulse audio?

You'll also have to start ffmpeg first maybe, then start playing the sound.
Or ffmpeg will die with "sound device busy"

You can crop the file later with mencoder or ffmpeg.

Do you have the correct record volume up? Sometimes it's record, or igain, or main etc

teckk · 06-13-2011, 06:07 PM

Ok, reread your first post, I see the examples.

I'll think a little

teckk · 06-13-2011, 06:51 PM

Quote:

I get sound when I do this:
ffmpeg -f alsa -i plughw:0 alsaout.wav

Quote:

I get video but no sound when I do this:
ffmpeg -f alsa -i plughw:0,0 -f x11grab -r 25 -s 1440x900 -i :0.0 -acodec pcm_s16le -vcodec huffyuv -sameq screencast.avi

You let ffmpeg set the codec to use on the first example, just by specifying a file extension .wav

You told it to use pcm_s16 on the second. That sounds like it would be the same. Not necessarily. Neither one of those is a raw audio dump I don't think.

Have you tried other audio codecs? Play a little .flv or video that you have and capture it.
ffmpeg won't capture the audio if you don't launch it first, and then start using the sound device on some machines.

Code:

ffmpeg -f alsa -i plughw:0 -f x11grab -r 30 -s 400x600 -i :0.0 -acodec copy -vcodec mpeg4 -b 1000k -qscale 10 output.avi

Code:

ffmpeg -f alsa -i plughw:0 -f x11grab -r 30 -s 400x600 -i :0.0 -acodec libmp3lame -ab 128k -vcodec mpeg4 -b 1000k -qscale 10 output.avi

This should also work, tried and tested, specify your own sound device or use alsa.

Code:

ffmpeg -f oss -i /dev/dsp0.0 -f x11grab -r 30 -s 1024x768 -i :0.0 -acodec pcm_s32le -vcodec libx264 -vpre lossless_ultrafast output.avi

As for oss.
You don't say if you specified an oss device like,

Code:

ffmpeg -f oss -i /dev/dsp0........

The reason I say that is if you specify the correct device to use, you get the sound if the right mixer control is turned up.

Then there is also alsamixer. Depends on your sound card. Just because you can hear the sound playing does not mean ffmpeg will get it if the correct mixer is not turned up, or muted. I have an Arch box that I worked with for a couple of hours. Could not get ffmpeg to capture the sound....turned out it was a lack of enabling one of the items in alsamixer.

If your sound card will play your media, or you can hear yourself talking through your sound card on your mic, then ffmpeg should be able to grab it.

Or maybe you found some kind of bug with your version of ffmpeg, your sound module, alsamixer etc.

Good luck. If you solve this then do tell.

astanton · 06-16-2011, 04:01 AM

Quote:

Originally Posted by teckk

You told it to use pcm_s16 on the second. That sounds like it would be the same. Not necessarily. Neither one of those is a raw audio dump I don't think.

Okay we're getting somewhere now

I'll probably quote you more later, as I had put this on the back burner for a while.

But I came back and did a cut and past, your second and third pastes actually worked the third one, I had to change it out for libmp3lame.

Okay so here's what works so far (which gives me a starting point) and I'll follow it up with a discussion of what doesn't...

Code:

ffmpeg -f alsa -i plughw:0 -f x11grab -r 30 -s 400x600 -i :0.0 -acodec libmp3lame -ab 128k -vcodec mpeg4 -b 1000k -qscale 10 output.avi

and this doesn't:

Code:

ffmpeg -f alsa -i plughw:0 -f x11grab -r 30 -s $(xwininfo -root | grep 'geometry' | awk '{print $2;}') -i :0.0 -acodec libmp3lame -ab 128k -vcodec mpeg4 -b 1000k -qscale 10 output.avi

And this doesn't either...

Code:

ffmpeg -f alsa -i plughw:0 -f x11grab -r 30 -s 800x600 -i :0.0 -acodec libmp3lame -ab 128k -vcodec mpeg4 -b 1000k -qscale 10 output.avi

This works:

Code:

ffmpeg -f alsa -i plughw:0 -f x11grab -r 30 -s 400x600 -i :0.0 -acodec libmp3lame -vcodec libx264 -vpre lossless_ultrafast output.avi

This doesn't:

Code:

ffmpeg -f alsa -i plughw:0 -f x11grab -r 30 -s 800x600 -i :0.0 -acodec libmp3lame -vcodec libx264 -vpre lossless_ultrafast output.avi

Basically, the trend coming on here is, that anything above 400x600 has no audio - which is really bizarre to me. my resolution is 1440x900 too, which makes it a little more annoying, although I could just be interested in capturing a single xterm window for most of what i wanted to do along with my audio.

It's almost as if, I have a few straws too many for the camel, who goes on strike after a certain load.

To answer your questions, I don't have pulse, and oss and dsp come up with errors so I think it's strictly alsa and plughw I'm going to be working with on this laptop (plain old hw:0 doesn't give an error, but doesn't seem to work either, so plughw:0 does).

But at least it's a starting point, and I don't know if the *400* was a typo or not on your part, but therein lies at least part of the key to this problem.

For all your help so far, thank you so much

EDIT: I should note that no matter what, I ALWAYS get the message:

Code:

[alsa @ 0x63a4f0]ALSA buffer xrun.

And here's a copy of some of the data from the last session that worked with audio and video, if this helps much:

Code:

~/$ ffmpeg -f alsa -i plughw:0 -f x11grab -r 30 -s 400x600 -i :0.0 -acodec libmp3lame -vcodec libx264 -vpre lossless_ultrafast output.avi
FFmpeg version 0.6.1, Copyright (c) 2000-2010 the FFmpeg developers
  built on Feb  3 2011 00:17:04 with gcc 4.4.4
  configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --mandir=/usr/man --enable-libmp3lame --enable-libfaac --enable-nonfree --enable-gpl --enable-version3 --enable-postproc --enable-avfilter --enable-libdc1394 --enable-libfaad --enable-libgsm --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvpx --enable-libx264 --enable-runtime-cpudetect --enable-vaapi --disable-vdpau --enable-memalign-hack --enable-pthreads --enable-x11grab --enable-bzlib --enable-zlib --enable-shared --enable-static --disable-debug --arch=x86_64 --enable-pic --extra-cflags='-I/tmp/build/tmp-ffmpeg/ffmpegdeps/usr/include -DRUNTIME_CPUDETECT' --extra-ldflags='-L/tmp/build/tmp-ffmpeg/ffmpegdeps/usr/lib64 -lssl -lcrypto -lz -lusb'
  libavutil     50.15. 1 / 50.15. 1
  libavcodec    52.73. 2 / 52.73. 2
  libavformat   52.64. 2 / 52.64. 2
  libavdevice   52. 2. 0 / 52. 2. 0
  libavfilter    1.19. 0 /  1.19. 0
  libswscale     0.11. 0 /  0.11. 0
  libpostproc   51. 2. 0 / 51. 2. 0
[alsa @ 0x63a4f0]capture with some ALSA plugins, especially dsnoop, may hang.
[alsa @ 0x63a4f0]Estimating duration from bitrate, this may be inaccurate
Input #0, alsa, from 'plughw:0':
  Duration: N/A, start: 443109.438166, bitrate: N/A
    Stream #0.0: Audio: pcm_s16le, 44100 Hz, 1 channels, s16, 705 kb/s
[x11grab @ 0x65d0e0]device: :0.0 -> display: :0.0 x: 0 y: 0 width: 400 height: 600
[x11grab @ 0x65d0e0]shared memory extension  found
[x11grab @ 0x65d0e0]Estimating duration from bitrate, this may be inaccurate
Input #1, x11grab, from ':0.0':
  Duration: N/A, start: 1308215546.644068, bitrate: 230400 kb/s
    Stream #1.0: Video: rawvideo, bgra, 400x600, 230400 kb/s, 30 tbr, 1000k tbn, 30 tbc
File 'output.avi' already exists. Overwrite ? [y/N] y
[libx264 @ 0x66a6b0]using cpu capabilities: MMX2 SSE2Slow SlowCTZ
[libx264 @ 0x66a6b0]profile High 4:4:4 Predictive, level 3.0, bit depth 8
Output #0, avi, to 'output.avi':
  Metadata:
    ISFT            : Lavf52.64.2
    Stream #0.0: Video: libx264, yuv420p, 400x600, q=10-51, 200 kb/s, 30 tbn, 30 tbc
    Stream #0.1: Audio: libmp3lame, 44100 Hz, 1 channels, s16, 64 kb/s
Stream mapping:
  Stream #1.0 -> #0.0
  Stream #0.0 -> #0.1
Press [q] to stop encoding
[alsa @ 0x63a4f0]ALSA buffer xrun.
frame=   35 fps=  6 q=-1.0 Lsize=     115kB time=5.64 bitrate= 167.2kbits/s    
video:50kB audio:44kB global headers:0kB muxing overhead 22.449107%
[libx264 @ 0x66a6b0]frame I:1     Avg QP: 0.00  size: 29906
[libx264 @ 0x66a6b0]frame P:34    Avg QP: 0.00  size:   626
[libx264 @ 0x66a6b0]mb I  I16..4: 76.8%  0.0% 23.2%
[libx264 @ 0x66a6b0]mb P  I16..4: 53.4%  0.0%  0.0%  P16..4:  0.0%  0.0%  0.0%  0.0%  0.0%    skip:46.6%
[libx264 @ 0x66a6b0]coded y,uvDC,uvAC intra: 1.8% 2.7% 2.6% inter: 0.0% 0.0% 0.0%
[libx264 @ 0x66a6b0]i16 v,h,dc,p: 99%  1%  0%  0%
[libx264 @ 0x66a6b0]i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 40% 45%  5%  2%  1%  1%  1%  1%  2%
[libx264 @ 0x66a6b0]i8c dc,h,v,p: 93%  2%  5%  0%
[libx264 @ 0x66a6b0]kb/s:351.02
~/$

teckk · 06-16-2011, 12:56 PM

Do you ever have gkrellm or top running when you are capturing?

If you are pegging the CPU at 100% then that could explain why something gets lost. You should be able to capture 800x600 with sound on a modern machine though. You'll have to keep the cpu at les than 100% or you'll get jerky video or maybe lost sound.

You could lower the video bit rate, try changing the audio bit rate too.
Instead of video at 1000K lower it. Instead of audio at 128k, lower it.
If you are just trying to capture and xterm while you narrate what you are doing then 32kb audio and 300k video may suffice. I have also had better luck with a greater framerate than 30.

Code:

ffmpeg -f alsa -i plughw:0 -f x11grab -r 50 -s 640x480 -i :0.0 -acodec libmp3lame -ab 32k -vcodec mpeg4 -b 300k output.avi

Now that you know your hardware and software works you'll have to experiment a little