video conferencing
 

dear friends,

i write this with a lot of confusions. actually i am a new user to linux . but i am thrilled at the freedom this OS gives us. i am now studying linux programming , that gives us full access to the resources of this OS.

I am doing M C A . As part of our carriculam, we need to undergo a mini project and main project. now it is the time for mini project.

Even though it may be risky, we( three persons) have decided to take a project of video conferencing across a LAN and to implement it using Linux, C programming and Qt. we had gone through the TCP and UDP programming. so we thought, since we are aware of networking(even though little) ,we just need to get the video and audio and send to the network(LAN) accordingly.

we have gone through the system study, we have created a server and client according to our plan , (now this can act as a chat server and client or that can transfer file from one system to another).

---------------------------------------------------------------------------------------------
we captured sound using "/dev/dsp",that can be recorded and played. but there is a delay.we are not aware of the format of the file. this cannot be played in xmms.
---------------------------------------------------------------------------------------------

but now we are not aware of the device file by which we can capture the image using webcam.
--------------------------------------------------------------------------------------------
Our plan was that , if we get both sound and video ,we would be able to send as our "existing client and server".

But there one person told us we won't achieve synchronisation if we go along with this idea.

Now we are in a confusion. We don't know which should be the next step to be taken.

is there any device file like "/dev/dsp" that can capture sound and video simultaneously( that can handle microphone and webcam simultaneously.)

Another suggection was that we need to study the video streaming . Actually we are beginners in this field. But our motto was that if we do something, the result should be a perfect or "superb" software that is to be appreciated.

The problem we are facing is that there is nobody to help us. If we ask any doubt to somebody, they are trying to evade us. Nobody is here to help us , since our OS is linux.(only very few here knows into the depth).

That is why we come to you . Awaiting your valuable suggections and comments...........................

please post a reply,
with hope and love
vishak

You will need to slow down the capture rate, allow the program to process and sync it, then resend it back. You will have to find some way to synchronise and integrate the audio and visual streams, or depending on the sound card, separate capture streams.

You're all using the same card right? If not, you would need to design it to allow it to adapt to existing configurations as well...

I'm not the programmer, but that would be my plan.

Just a few ideas:

You can send it separately, but program it so they come within negligible milliseconds of each other.

What you should do is configure send and receive that when a person's camera program receives the audio and visual packet (separately), it will send an acknowledgement of successful transfer to the sender, which will echo a "play now" kind of thing, so after it has confirmed it has received both audio and visual packets it will play them both at once. The sender meanwhile, would have sent more packets, and after that more, so after the receiver plays the sound and visual, it will take the new packets. Of course these are all separate threads going on all at once. You may wish to use the famous Linux Arts server, it is a big help towards sound synchronisation.

Of course video conferencing is a combination of both send and receive on each computer, you have to keep in mind.


Also you should specifically set intervals of delay between sending of information, or have it customised (ie. send it faster if you know you can process, send it and the receiver process it within a regular rate counting bandwith) which can be set. This will not result in a smooth picture, however counting that if the processors of the computers are good and the LAN allows good transfers, you can set the delay to be quite short, so it can be quite smooth. However, there will be perhaps a few fractions of second in delay.

I have no idea how much time it will take to process the packets, and send and receive (you must bear in mind one PC both has to send AND receive, unless of course you set it for only one side), but you can play around with it. You should also make picture and sound quality customisable, etc.

My guess that processing time with 2.4 ghz and good memory (say 0.9 ghz for a receive, and another 0.9 ghz for send and the rest for using for other stuff) for and networking time will take about 150 milliseconds for each packet receive cycle over a LAN for synchronisation (and still have your computers respond) giving you a frame rate of about 6 frames per second. This is my estimation, although it will probably be somewhat different, just a guess. This is only for 1 to 1 though, I think unless you have supercomputers having mass video convos would be disastrous!

Packets mentioned are abstract here, they are simply units of They are not really socket packets, ie. you might choose to send sound and visuals in two hundred millisecond packets (this again should be customisable) which might actually be 50 different 2 kb real TCP/UDP packets being sent.

You can also choose two different options of synchronisation:

a. After sending a packet, sender will not send the next one till the "receive acknowledgement" signal arrives from the receiver.

b. Even when it has not received a successful acknowledgement of the reception (reminds me you should also provide a timeout for a resend) of a packet, it will continue sending packets so as not to waste time, with appropriate delays of course, ie. 100 milliseconds.

There are also variations, ie. send the next packet after it has received acknowledgement of playing the visuals or sound, rather than just acknowledge receive (good for PC's with good bandwith but low processing power or memory). I remind you that the packet transmission size, picture and sound quality should be customised as well as the delay to the performance level of the network or computer.

Mentioned above, A is good when the computers or network are not that good, however it might result in an erratic performance between frames (however sound and visuals will be synchronised) since idle processing time is wasted while waiting for the next packet to arrive. B is good for smooth quality on newer machines, but you have a danger of "flooding" the older machines who can't process all the information at once, as visual packets are quite large.

You should also provide some kind of notice or clearage should the video and sound capture end up storing so much information it hasn't sent that it uses too much memory, warnings should be provided ie. for senders to stop until the buildup decreases, or clear the memory buffers regularly.