LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 03-19-2017, 01:37 AM   #1
swmcl
LQ Newbie
 
Registered: Mar 2017
Posts: 5

Rep: Reputation: Disabled
Design of serial port logging program and then another to store in database


Hi,

I would like to pay someone to make for me a program to log a serial port and another program to then put relevant data into a database.

Can I ask what the hangups will be? Assume the serial logger will be on 24/7.

The initial logger should timestamp the serial port data.

Given that the program that does the filtering of the data and places it in the database will be slow it will not keep up with the logger and may lag behind by some time how do I ask for the solution?

Do the logger and the filtering programs share the same file?
Should the logger push its output into smaller size-limited files? Or perhaps change files within a short period of time? (say 100ms) The filtering program could then be using the older file ...

The filtering program needs to mark an entry into the logger file to record a successful database transaction I would think.

I'm concerned that the data to be logged should have unfettered access to its initial log file. If it is timestamped then the database stores the timestamp.

This is a design question.

I hope it makes some sense.

Cheers,
Steve
 
Old 03-19-2017, 11:34 AM   #2
michaelk
Moderator
 
Registered: Aug 2002
Posts: 21,589

Rep: Reputation: 4163Reputation: 4163Reputation: 4163Reputation: 4163Reputation: 4163Reputation: 4163Reputation: 4163Reputation: 4163Reputation: 4163Reputation: 4163Reputation: 4163
Many things to consider.

How many bytes are going to be sent from your device. What is the baud rate?
What is the expected recording rate as in samples per second?
How important is time latency?
Is the program going to request data or is going to be continuous?
What are you going to filter and why do you need a database?

Fixed sized files can be good because if the computer or logging program crashes you only loose the last opened file.
 
Old 03-19-2017, 05:33 PM   #3
swmcl
LQ Newbie
 
Registered: Mar 2017
Posts: 5

Original Poster
Rep: Reputation: Disabled
Hmmm.

The nature of the comms is that it is sporadic. The comms is initiated by both ends so comms being sent or received needs to be initially captured. The comms is in a range of 10 through 20 bytes at a time which is relatively small packets. It is important for the comms to be recorded quickly and that they be recorded with timestamping. Measures to ensure speed of recording and protective measures in case of some sort of failure (power, dislodged cable etc,.) are what I'm trying to reasonably define so that I can advertise on something like Freelancer.

Comms need to contain certain identifying features to help validate the data. So the filtering is removing the 'extraneous' information from th e comms frame to record only the actual data. A bit like taking the letter from the envelope. Not all comms seen on the port is valid. There will be broken frames and other 'noise'.

In terms of speed, I'd think say 20 interactions of 20 byte packages per second would be a solid start. So 400 bytes per second.

Comms in and out need to be monitored although some of the comms may not be recorded in the longer term. So if you can imagine a situation whereby comms goes out requesting something and expecting a certain reply. If that reply comes in then there would be little need to keep some of the data in the longer term. If the reply doesn't come in then a record of the attempt to communicate needs to be kept. All of this is further downstream of my immediate requirements though. For now, I need the ability to record all comms in a secure manner.

I hope this further clarifies my situation.

If I can be more precise in my initial job requirements this is the aim. Should I have a set of files through which comms is written? eg. say five(5) 1k files for the data sent and five(5) 1k files for data received?

Thanks for the reply,
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
data logging from serial port Daravon Linux - Software 3 05-02-2012 03:10 PM
[SOLVED] Database design using IDs as opposed as Candidate keys as Primary key (UML design) angel115 Programming 1 07-27-2011 08:58 AM
strugling with serial port interfacing program ravichandru Programming 1 08-25-2010 05:25 AM
Serial port monitor program pedjasns Linux - Newbie 2 03-19-2008 04:28 AM
C++ serial port program problem doar Programming 3 10-01-2004 10:46 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 11:33 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration