LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 08-26-2020, 08:59 PM   #1
bkelly13
LQ Newbie
 
Registered: Dec 2018
Posts: 4

Rep: Reputation: Disabled
fscanf(...) not completing the read


Edit, Just to put this at the top: I have followed the advice here and changed strategy to use strtok() and strtod().

The goal is to read a CSV file and put the values into variables. I have extracted the misbehaving code to a test program and it manifests the problem. The printf at line 79 looks good, but the next printf at line 79 shows only one character read. I am expecting a lot more. The same concepts applies within the for loop.
Conclusion: There is something about the fscanf(…) that I do not understand. Here is the code.

Edit: I simplified the code, shortened the input file to remove non-symptomatic lines, and am providing the output. In summary, the fscanf(...) is reading one field at a time, one one line. I checked errno immediately after the fscanf call and it is always zero.
After reading the post from GazL, I was certain that would resolve the problem. Alas, there is another defect that I am looking right at but cannot see.

Thank you for your time and patience.

Code:
// CentOS Linux release 7.7.1908 (Core)
// compile with c++ test.cpp
// run c++ --version to get, in part:
// (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39)
// run with     ./a.out

#include <stdio.h>
#include <string.h>
#include <errno.h>
int main( )
{
   FILE * multi_column_file;
   char   multi_column_name[] = "td_short.csv";
   int    field_count =    0;
   
   extern int errno;
   
   multi_column_file = fopen( multi_column_name, "r");

   size_t len = 0;
   ssize_t read = 0;;
      
   int row_number;
   float time, sine_1, sine_2, sine_3, sine_4, sine_5, composite = 0.0;
      
   char a[ 128 ] = "no";  char b[ 128 ] = "no"; char c[ 128 ] = "no";
   char d[ 128 ] = "no";  char e[ 128 ] = "no"; char f[ 128 ] = "no";
   char g[ 128 ] = "no";  char h[ 128 ] = "no";
      
      printf( "\n%4d errno at for entry         %d", __LINE__, errno );

      for( int i = 0; i < 5; i ++ )
      {
           // Simplified test, get the values as string
        field_count = fscanf( multi_column_file, "%s, %s, %s, %s, %s, %s, %s, %s\n",
            &a, &b, &c, &d, &e, &f, &g, &h );

        printf( "\n%4d after read strings errno %d", __LINE__, errno );
        printf( "\n%4d field_count %4d text   %s %s %s %s %s %s %s %s", 
            __LINE__, field_count, a, b, c, d, e, f, g, h ); 
        printf( "\n%4d length a %4d length b %4d\n", __LINE__, strlen( a ), strlen( b ) ); 
        

            
           // This is the goal
        field_count = fscanf( multi_column_file, "%d, %f, %f, %f, %f, %f, %f, %f\n",
            &row_number, &time, &sine_1, &sine_2, &sine_3, &sine_4, &sine_5, &composite );

        printf( "\n%4d after read numbers errno %d", __LINE__, errno );
        printf( "\n%4d field_count %4d text   %d %f %f %f %f %f %f %f", 
            __LINE__, field_count, &row_number, &time, &sine_1, 
            &sine_2, &sine_3, &sine_4, &sine_5, &composite );      

        printf( "\n%4d end loop .............\n",  __LINE__ );
      }

   fclose( multi_column_file );
   printf( "\n%4d Exit\n", __LINE__ );
   return 0;
}
Here is the shortened input data, removed the header.
Code:
    0,   0.0001,   0.0002,   0.0003,   0.0004,   0.0005,   0.0006,    0.0007
    1,   0.0008,   5.2264,   8.3165,   9.2705,   8.1347,  20.0000,   50.9481
    2,   0.0017,  10.3956,  16.2695,  17.6336,  14.8629,  34.6410,   93.8025
    3,   0.0025,  15.4508,  23.5114,  24.2705,  19.0211,  40.0000,  122.2539
    4,   0.0033,  20.3368,  29.7258,  28.5317,  19.8904,  34.6410,  133.1258
    5,   0.0042,  25.0000,  34.6410,  30.0000,  17.3205,  20.0000,  126.9615
    6,   0.0050,  29.3893,  38.0423,  28.5317,  11.7557,   0.0000,  107.7189
    7,   0.0058,  33.4565,  39.7809,  24.2705,   4.1582, -20.0000,   81.6661
    8,   0.0067,  37.1572,  39.7809,  17.6336,  -4.1582, -34.6410,   55.7724
    9,   0.0075,  40.4508,  38.0423,   9.2705, -11.7557, -40.0000,   36.0079
   10,   0.0083,  43.3013,  34.6410,   0.0000, -17.3205, -34.6410,   25.9808
   11,   0.0092,  45.6773,  29.7258,  -9.2705, -19.8904, -20.0000,   26.2421
   12,   0.0100,  47.5528,  23.5114, -17.6336, -19.0211,  -0.0000,   34.4095
   13,   0.0108,  48.9074,  16.2695, -24.2705, -14.8629,  20.0000,   46.0434
   14,   0.0117,  49.7261,   8.3165, -28.5317,  -8.1347,  34.6410,   56.0172
   15,   0.0125,  50.0000,   0.0000, -30.0000,  -0.0000,  40.0000,   60.0000
   16,   0.0133,  49.7261,  -8.3165, -28.5317,   8.1347,  34.6410,   55.6537
And here is the ouptut
Code:
[mcs@localhost sim]$ ./a.out

  30 errno at for entry         0
  38 after read strings errno 0
  40 field_count    1 text   0, no no no no no no no
  41 length a    2 length b    2

  49 after read numbers errno 0
  51 field_count    1 text   -888742996 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
  54 end loop .............

  38 after read strings errno 0
  40 field_count    1 text   .0001, no no no no no no no   /// added notice field 0.0001
  41 length a    6 length b    2

  49 after read numbers errno 0
  51 field_count    1 text   -888742996 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
  54 end loop .............

  38 after read strings errno 0
  40 field_count    1 text   .0002, no no no no no no no  /// compare with 0001 above, this is the next field on that row
  41 length a    6 length b    2

  49 after read numbers errno 0
  51 field_count    1 text   -888742996 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
  54 end loop .............

 lines omitted
  58 Exit

Last edited by bkelly13; 08-28-2020 at 04:39 PM. Reason: Clarify
 
Old 08-27-2020, 06:06 AM   #2
rtmistler
Moderator
 
Registered: Mar 2011
Location: USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 8,907
Blog Entries: 13

Rep: Reputation: 4040Reputation: 4040Reputation: 4040Reputation: 4040Reputation: 4040Reputation: 4040Reputation: 4040Reputation: 4040Reputation: 4040Reputation: 4040Reputation: 4040
DEBUG IT

You started to debug field_count in that compiled out code.

Debug it for the problem lines.

Perhaps the sample of data shown should show, lines before, on, and after line 79.

Use GDB and break at the fscanf() call, step over it and check the outcome.

Check ERRNO if field_count is negative.

Move the problem lines up to start at line 2 and see what happens.

Use fgets() and see what it thinks about those lines.

Read the whole file into a RAM buffer and examine that in the debugger.
 
Old 08-27-2020, 06:28 AM   #3
GazL
LQ Guru
 
Registered: May 2008
Distribution: CRUX
Posts: 5,547
Blog Entries: 14

Rep: Reputation: 3387Reputation: 3387Reputation: 3387Reputation: 3387Reputation: 3387Reputation: 3387Reputation: 3387Reputation: 3387Reputation: 3387Reputation: 3387Reputation: 3387
scanf() doesn't treat a comma as a field-separator. When your input contains commas you need to specify them in the format string something like scanf(" %d, %f, %f, ...", &args, ...) else you'll get a format mismatch when it hits the comma.

Oh, and use 'free()' not 'delete' in a C program.
 
1 members found this post helpful.
Old 08-27-2020, 08:10 AM   #4
rtmistler
Moderator
 
Registered: Mar 2011
Location: USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 8,907
Blog Entries: 13

Rep: Reputation: 4040Reputation: 4040Reputation: 4040Reputation: 4040Reputation: 4040Reputation: 4040Reputation: 4040Reputation: 4040Reputation: 4040Reputation: 4040Reputation: 4040
Figured I'd give that a small try, but your code has numerous compile errors.
 
Old 08-27-2020, 08:16 AM   #5
boughtonp
Member
 
Registered: Feb 2007
Location: UK
Distribution: Debian
Posts: 570

Rep: Reputation: 366Reputation: 366Reputation: 366Reputation: 366
Quote:
The goal is to read a CSV file and put the values into variables
I'm pretty sure there'll be an existing library that is tested and optimised to do that already. Why would you want to spend time re-writing a buggy version of it?

 
Old 08-27-2020, 09:48 AM   #6
bkelly13
LQ Newbie
 
Registered: Dec 2018
Posts: 4

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by rtmistler View Post
Figured I'd give that a small try, but your code has numerous compile errors.
That code did compile and run on my machine, Centos, compiles with command c++. I will look an ERRNO and check in the debugger.
Thanks for giving it a try.
 
Old 08-27-2020, 11:19 AM   #7
EdGr
Member
 
Registered: Dec 2010
Location: California, USA
Distribution: I run my own OS
Posts: 482

Rep: Reputation: 234Reputation: 234Reputation: 234
In addition to the delimiter problem that GazL pointed out, the code has a buffer overrun on the string variables. The safest method is to use getline to read the line and allocate a buffer, and then parse the line yourself.
Ed
 
Old 08-28-2020, 11:22 AM   #8
bkelly13
LQ Newbie
 
Registered: Dec 2018
Posts: 4

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by GazL View Post
scanf() doesn't treat a comma as a field-separator. When your input contains commas you need to specify them in the format string something like scanf(" %d, %f, %f, ...", &args, ...) else you'll get a format mismatch when it hits the comma.

Oh, and use 'free()' not 'delete' in a C program.
I was sure this would fix the problem, but no such luck. I edited the OP and shortened the code down to essentials, and showed the output.

foughtonp: Yes, there is probably such a library. But, once past the header, every line of data takes the exact same format.

In the meantime, I will be looking a sscanf(), or scans(), or something like that to read into a string the parse the string. Extra code but maybe I can make that work.

Thank you for your time and patience.
 
Old 08-28-2020, 11:50 AM   #9
rtmistler
Moderator
 
Registered: Mar 2011
Location: USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 8,907
Blog Entries: 13

Rep: Reputation: 4040Reputation: 4040Reputation: 4040Reputation: 4040Reputation: 4040Reputation: 4040Reputation: 4040Reputation: 4040Reputation: 4040Reputation: 4040Reputation: 4040
Probably better to skip using a fully inclusive library function like any of the scanf() variety and processing line by line in character arrays, looking for the comma delimiter.

More exhaustive code, but that's what loops are for.

Hey, ... done plenty decoding serial protocols, especially since you don't get the entire packet all the time. At least here you do not have to have a state machine which is re-entrant to remember your last parsed point, you just validate line by line from a file.
 
Old 08-28-2020, 04:04 PM   #10
GazL
LQ Guru
 
Registered: May 2008
Distribution: CRUX
Posts: 5,547
Blog Entries: 14

Rep: Reputation: 3387Reputation: 3387Reputation: 3387Reputation: 3387Reputation: 3387Reputation: 3387Reputation: 3387Reputation: 3387Reputation: 3387Reputation: 3387Reputation: 3387
%s will eat the comma in the input, so there won't be one to match the literal comma in the scanf format string. It should work with %d and %f however.

Here's a minimal example to show it works with your input file:
Code:
#include <stdio.h>

int main()
{

    int n;
    int d;
    double f[7];
        
    while ( ( n = scanf( " %d, %lf, %lf, %lf, %lf, %lf, %lf, %lf",  
                         &d , &f[0], &f[1], &f[2], &f[3], &f[4], &f[5], &f[6]) ) != EOF )
    {
        if ( n != 8 )
        {
            fprintf(stderr, "invalid input line, only %d fields.\n", n);
            break;
        }

        printf ("Line % 4d\t%12.6f %12.6f %12.6f %12.6f %12.6f %12.6f %12.6f\n", d, f[0], f[1], f[2], f[3], f[4], f[5], f[6] );
        
    }
    
    return 0;
}
P.S. This example wouldn't detect a line with too many input fields.

scanf() is notoriously poor at dealing with malformed input. If robustness matters you'll likely be better off using getline() with strtok() and strtod().

Last edited by GazL; 08-28-2020 at 04:23 PM.
 
1 members found this post helpful.
Old 08-28-2020, 04:37 PM   #11
bkelly13
LQ Newbie
 
Registered: Dec 2018
Posts: 4

Original Poster
Rep: Reputation: Disabled
Ok, I have changed my strategy and am going with the strtok and the strtod.
Wrote some test code and got it working.
Thank you to each of you for your time and patience.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
fscanf does not print on screen ddialani Linux - Newbie 7 01-07-2013 05:28 AM
fscanf - segfault - Address 0x0 is not stack'd, malloc'd or (recently) free'd jkobori Programming 12 11-20-2012 07:44 AM
use fscanf to read text from a file emp1953 Linux - Newbie 2 03-14-2008 03:08 PM
how to read optional characters %*[] with fscanf? phyx Programming 1 01-18-2007 08:50 PM
using fscanf function Linh Programming 2 07-14-2003 12:58 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 03:17 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration