[SOLVED] readdir on centos 7

dwhitney67 · 03-04-2015, 07:02 AM

I'm seeing an issue on a CentOS 7 x86-64 VM in which readdir() is reporting that each file that it finds is of an unknown type (DT_UNKNOWN).

I've verified that the path (and files within such) are all readable, but still no joy. Directories have permissions of 0755 and files of 0644.

If you have a similar system, can you please confirm whether this issue is endemic to only my system, or whether it is a general bug with this particular OS.

Similar tests on CentOS 6 and Ubuntu do not show a similar problem.

Thank you.

Btw, here's the sample C++ application I'm using where I pass the path to the directory to search (for XML files) containing the .xml extension:

Code:

#include <dirent.h>
#include <cstring>
#include <vector>
#include <string>
#include <iostream>

using namespace std;


typedef vector< string > StrVector;


StrVector getWildCardTokens( const string &wildcard )
{
  StrVector wildCardList;

  char *token = strtok( (char*) wildcard.c_str(), "*" );

  while ( token != NULL )
  {
    wildCardList.push_back( token );
    token = strtok( NULL, "*" );
  }

  return wildCardList;
}

void getAllFiles( StrVector &fileList, const string &dirName, const StrVector &wildCards )
{
  DIR *dirp = opendir( dirName.c_str() );

  if ( dirp )
  {
    struct dirent *dp = NULL;

    while ( (dp = readdir( dirp )) != NULL )
    {
        string file( dp->d_name );

        if ( file == "." || file == ".." )    // skip these
          continue;

        if ( dp->d_type & DT_DIR )
        {
          // found a directory; recurse into it.
          string filePath = dirName + "/" + file;

          getAllFiles( fileList, filePath, wildCards );
        }
        else if (dp->d_type == DT_UNKNOWN)
        {
            std::cerr << "Unknown file: " << dirName << "/" << file << std::endl;
        }
        else
        {
          bool matches = true;

          // found a file; check if all of the wildcard tokens exist in its name.
          // search linearly thru the file name (i.e. continue where the last token
          // is found).
          for ( size_t i = 0, pos = 0; i < wildCards.size(); ++i )
          {
            if ( (pos = file.find( wildCards[i], pos )) == string::npos )
            {
              matches = false;
            }
          }

          if ( matches )
          {
            fileList.push_back( dirName + "/" + file );
          }
        }
    }

    closedir( dirp );
  }
}

int main( int argc, char **argv )
{
  string dir      = argc > 1 ? argv[1] : "/tmp";
  string wildcard = argc > 2 ? argv[2] : "*.xml";

  StrVector wildCardTokens = getWildCardTokens( wildcard );

  StrVector fileList;
  getAllFiles( fileList, (argc > 1 ? argv[1] : dir), wildCardTokens );

  cout << "Found " << fileList.size() << " file(s)." << endl;
  for ( size_t i = 0; i < fileList.size(); ++i )
  {
    cout << "found file " << fileList[i] << endl;
  }
}

------------

Edit:

I wanted to add that on the CentOS 6 system, the files are reported to have the following type (using the 'file' command):

Code:

MyFile.xml: ASCII HTML document text

On CentOS 7, the result is:

Code:

MyFile.xml: exported SGML document, ASCII text

Why the difference?

NevemTeve · 03-04-2015, 07:10 AM

My manual says:

Code:

...
unsigned char  d_type;      /* type of file; not supported by all file system types */
...

so I think you have to call (l)stat(2) if d_type==DT_UNKNOWN

NevemTeve · 03-04-2015, 07:10 AM

Question#2: utility 'file' keeps improving

dwhitney67 · 03-04-2015, 07:56 AM

Quote:

Originally Posted by NevemTeve

My manual says:

Code:

...
unsigned char  d_type;      /* type of file; not supported by all file system types */
...

so I think you have to call (l)stat(2) if d_type==DT_UNKNOWN

I added this to my code to check whether d_type is supported, and it is.

Code:

...
#ifdef _DIRENT_HAVE_D_TYPE
int haveType = 1;
#else
int haveType = 0;
#endif

int main( int argc, char **argv )
{
  std::cout << "_DIRENT_HAVE_D_TYPE is " << haveType << std::endl;    // expected result is haveType == 1

  ...
}

I think the issue may somehow be related to the file system type in use on the CentOS 7 system. RHEL7 and CentoOS 7 now use XFS as the default file system, whereas with previous OS releases I believe EXT4 was the default.

NevemTeve · 03-04-2015, 08:01 AM

I think text 'not supported' in manual means 'is not filled with useful value'

dwhitney67 · 03-04-2015, 08:11 AM

Quote:

Originally Posted by NevemTeve

I think text 'not supported' in manual means 'is not filled with useful value'

I will accept your answer. After I re-read the man-page, it is quite apparent that I should not assume that the d_type is usable. From the man-page:

Quote:

Other than Linux, the d_type field is available mainly only on BSD systems. This field makes it possible to avoid the expense of calling lstat(2) if further actions depend on the type of the file.

However, further down (and this is why I think you are correct):

Quote:

Currently, only some file systems (among them: Btrfs, ext2, ext3, and ext4) have full support for returning the file type in d_type. All applications must properly handle a return of DT_UNKNOWN.

Since CentOS 7 is using the XFS file system, it is quite logical that d_type could very well be set to DT_UNKNOWN.

So, in conclusion, I will need to use lstat(2).

Thank you for your help... and for the enlightenment.