Hello All,
I am a newbie to linux environment. I have to develop a data logging system in linux, that can collect data from multiple applications (executing in parallel) with least impact on performance.

I want to have a separate independent application that will continuously throw this data on UART Port to the host machine for analysis.

I am currently analyzing the shared memory as it provides the least performance impact but struggling to see how concurrency issues can be handled.

Any ideas in this are welcomed.

Regards,
Rishabh

Too slow.

The problem with shared memory for this is that you end up having to wait until the logging process copies the data out. Since it is coming from multiple sources you end up waiting to put the log information into the memory (synchronization), and waiting again if the segment fills up.

You do better with a socket where you can send the data... and the socket provides the buffer. This allows the logging process to copy data out continuously as long as there is data available, and the multiple sources don't have to wait.

For an example look at the rsyslog program.

1 members found this post helpful.

They sell a few real "real time" linux distro's just for such a use. I'd go for QNX myself.

I am not a fan of the near real time distro's of linux.

Start by looking for "prior art!"

No matter what you might be doing, the odds are frankly "near-zero" that you are somehow the first. Begin the project by canvassing the Internet for options – "Nagios," for example? – that might be pressed into service.

Also, be very reluctant to decide-upon specific implementation alternatives, simply because they're the first alternatives that came to you. (For instance, is "throwing the data on a UART port" the only or the necessary way to "send the data to the host machine for analysis?") And, even if it is, your project is fairly certain to be a "steal cabbage vs. build" decision.

It is absolutely natural for an engineer to be "depth-first." But, in this case and in these modern times, "breadth-first" is often the most-productive strategy. "Grab a solution from that bald guy over there" ... ... "and save precious hair-follicles for yourself!"

I tried message queues on desktop for message serialization. and observed 3usec time for each operation.
The use of Socket is a good idea. Will go through some of the socket examples to see how to get the thing working in multi application environment and its time impact.