[SOLVED] libxml2 : xmlFreeTextReader crashes

suneelmekala · 02-22-2011, 01:27 AM

Dear all,

I am using libxml2 (libxml2-2.7.6-1.fc10.x86_64 )
I get crash when I call xmlFreeTextReader after the XMl file is processed.
Following is the sample code that I am using:

Code:

xmlTextPointer = xmlReaderForMemory(buffer, strlen(buffer), "somthing.xml", NULL, 0);
while(errorCode == 1)
  {
    printf("Name : %s\n",xmlTextReaderConstName(xmlTextPointer));
    errorCode = xmlTextReaderRead(xmlTextPointer);
  }
xmlFreeTextReader(xmlTextPointer);
xmlCleanupParser();

The buffer contains valid XML data, as it parses complete buffer and prints all the names (nodes)
and when xmlFreeTextReader is called it crashes

Following is the backtrace:

Code:

======= Backtrace: =========
/lib64/libc.so.6[0x3cfd477ec8]
/lib64/libc.so.6(cfree+0x76)[0x3cfd47a486]
/usr/lib64/libxml2.so.2(xmlDictFree+0xf8)[0x3d07906b48]
/usr/lib64/libxml2.so.2(xmlFreeParserCtxt+0x135)[0x3d07834335]
/usr/lib64/libxml2.so.2(xmlFreeTextReader+0x245)[0x3d078ee735]
/export/home/suneelm/Pro/xml/a.out[0x402de5]
/export/home/suneelm/Pro/xml/a.out[0x401ff8]
/export/home/suneelm/Pro/xml/a.out[0x40154d]
/lib64/libc.so.6(__libc_start_main+0xe6)[0x3cfd41e576]
/export/home/suneelm/Pro/xml/a.out[0x4010c9]

Any pointers? Thanks in advance.

-Sun

pgpython · 02-22-2011, 04:18 AM

Okay the clue here is with the cfree in your error. This is a big giveaway you are trying to free something which has not been allocated. The question is what it is. first of all if you look at the libxml2 api you will find xmlReaderForMemory returns NULL upon error so you need to check for that even if you know all the parameters you are passing in are valid that would necessarily guarantee you a valid pointer. Secondly if you also look at the documentation for xmlTextReaderConstName you will find it free's the resource associated with it. this is something you have to be careful about with libxml2 as some methods do that. so i would say that when you call xmlFreeTextReader you get a double free error

suneelmekala · 02-22-2011, 06:23 AM

Quote:

Originally Posted by pgpython

Okay the clue here is with the cfree in your error. This is a big giveaway you are trying to free something which has not been allocated. The question is what it is. first of all if you look at the libxml2 api you will find xmlReaderForMemory returns NULL upon error so you need to check for that even if you know all the parameters you are passing in are valid that would necessarily guarantee you a valid pointer. Secondly if you also look at the documentation for xmlTextReaderConstName you will find it free's the resource associated with it. this is something you have to be careful about with libxml2 as some methods do that. so i would say that when you call xmlFreeTextReader you get a double free error

Could you please point me to the documentation ?

Thank you.
-Sun

pgpython · 02-22-2011, 06:32 AM

The api documentation is on the libxml2 website, google it!

suneelmekala · 02-22-2011, 08:23 AM

I checked xmlReaderForMemory return value and it gives a valid pointer. Also it does not crash at xmlTextReaderConstName call. But it crashes when xmlFreeTextReader is called. I am checking the xmlTextPointer before calling xmlFreeTextReader.

Thanks in advance.

-Sun

ntubski · 02-22-2011, 10:43 PM

When you have problems like this it is really important to give whole program that reproduces the error:

Code:

#include <stdio.h>
#include <string.h>
#include <libxml/xmlreader.h>

int main()
{
    int errorCode = 1;
    static char buffer[] = "<something/>";
    xmlTextReaderPtr xmlTextPointer = xmlReaderForMemory(buffer, strlen(buffer),
        "somthing.xml", NULL, 0);
    while(errorCode == 1)
    {
        printf("Name : %s\n",xmlTextReaderConstName(xmlTextPointer));
        errorCode = xmlTextReaderRead(xmlTextPointer);
    }
    xmlFreeTextReader(xmlTextPointer);

    return 0;
}

Code:

~/src/xml/crash$ make crash
gcc -Wall -Wextra -g -O0 -I/usr/local/include/libxml2  -c -o crash.o crash.c
gcc   crash.o  -lxml2 -lm -o crash
~/src/xml/crash$ ./crash
Name : (null)
Name : something

As you can see, when I tried there was no crash, probably something in the part of the program you didn't post causes the problem.

Quote:

Originally Posted by pgpython

Secondly if you also look at the documentation for xmlTextReaderConstName you will find it free's the resource associated with it. this is something you have to be careful about with libxml2 as some methods do that. so i would say that when you call xmlFreeTextReader you get a double free error

Quote:

Function: xmlTextReaderConstName

const xmlChar * xmlTextReaderConstName (xmlTextReaderPtr reader)

The qualified name of the node, equal to Prefix :LocalName.
reader: the xmlTextReaderPtr used
Returns: the local name or NULL if not available, the string is deallocated with the reader.

That just warns against freeing the string returned by xmlTextReaderConstName(), which the code (at least what was posted) doesn't do.

suneelmekala · 02-23-2011, 03:55 AM

Yeah I also tried the code snippet which I posted and it did not crash. Following is the complete code which I am using which is crashing:

Code:

int parseDevDescMessage(char *buffer, DMXDevice *dmxLocal)
{
  //xmlDocPtr xmlDocPointer;
  xmlTextReaderPtr xmlTextPointer;
  int           errorCode=0;
  const xmlChar *name, *value;
  char          *xmlBuffer;
  xmlNode       *rootElement = NULL;
  xmlNode       *currentNode = NULL;

  devServices   *devServicesLocal=NULL;
  devServices   *devServicesLocalAdd=NULL;

  /*
   * This initialize the library and check potential ABI mismatches
   * between the version it was compiled for and the actual shared
   * library used.
   */
  LIBXML_TEST_VERSION

  xmlBuffer = (char *)malloc(strlen(buffer));
  strcpy(xmlBuffer, buffer, strlen(buffer));
  /*I am doing this string copy because the variable buffer is referred in the calling function. I am assuming that xmlFreeTextReader is freeing the buffer also

  xmlTextPointer = xmlReaderForMemory(xmlBuffer, strlen(xmlBuffer), "somthing.xml", NULL, 0);
  if(xmlTextPointer == NULL)
  {
    #ifdef CTL_MSG_PARSER_DEBUG_NEW
      printf("parseDevDescMessage : ERROR WHITE CREATING XML DOC FROM MEM\n");
    #endif
    errorCode = -1;
    goto error;
  }

  errorCode = xmlTextReaderRead(xmlTextPointer);  
  while(errorCode == 1)
  {
    if( ( (strcmp(xmlTextReaderConstName(xmlTextPointer),"serviceList")) == 0 ) &&
        ( (xmlTextReaderNodeType(xmlTextPointer)) == 1)
 )
    {
      errorCode = xmlTextReaderRead(xmlTextPointer);
      while(errorCode == 1)
      {
        if( ( (strcmp(xmlTextReaderConstName(xmlTextPointer),"service")) == 0) &&
            ( (xmlTextReaderNodeType(xmlTextPointer)) == 1)
          )
        {
          devServicesLocal =  dmxLocal->deviceServices;

          if(!devServicesLocal)
          {
            devServicesLocal = (devServices *)malloc(sizeof(devServices));
            memset(devServicesLocal, 0, sizeof(devServices));
            dmxLocal->deviceServices = devServicesLocal;
            //devServicesLocalAdd      = devServicesLocal;
          }
          else
          {
            while(devServicesLocal->nextService)
            {
              devServicesLocal = devServicesLocal->nextService;
            }
            devServicesLocal->nextService = (devServices *)malloc(sizeof(devServices));
            memset(devServicesLocal->nextService, 0, sizeof(devServices));
            devServicesLocal = devServicesLocal->nextService;
          }
          errorCode = xmlTextReaderRead(xmlTextPointer);
          errorCode = xmlTextReaderRead(xmlTextPointer);
          errorCode = xmlTextReaderRead(xmlTextPointer);

          value = xmlTextReaderConstValue(xmlTextPointer);
          devServicesLocal->serviceType = (char *)malloc(strlen(value));
          strcpy(devServicesLocal->serviceType, value);


          errorCode = xmlTextReaderRead(xmlTextPointer);
          errorCode = xmlTextReaderRead(xmlTextPointer);
          errorCode = xmlTextReaderRead(xmlTextPointer);
          errorCode = xmlTextReaderRead(xmlTextPointer);

          value = xmlTextReaderConstValue(xmlTextPointer);
          devServicesLocal->serviceId = (char *)malloc(strlen(value));
          strcpy(devServicesLocal->serviceId, value);

          errorCode = xmlTextReaderRead(xmlTextPointer);
          errorCode = xmlTextReaderRead(xmlTextPointer);
          errorCode = xmlTextReaderRead(xmlTextPointer);
          errorCode = xmlTextReaderRead(xmlTextPointer);

          value = xmlTextReaderConstValue(xmlTextPointer);
          devServicesLocal->SCPDURL = (char *)malloc(strlen(value));
          strcpy(devServicesLocal->SCPDURL, value);


          errorCode = xmlTextReaderRead(xmlTextPointer);
          errorCode = xmlTextReaderRead(xmlTextPointer);
          errorCode = xmlTextReaderRead(xmlTextPointer);
          errorCode = xmlTextReaderRead(xmlTextPointer);

          value = xmlTextReaderConstValue(xmlTextPointer);
          devServicesLocal->eventSubURL  = (char *)malloc(strlen(value));
          strcpy(devServicesLocal->eventSubURL, value);


          errorCode = xmlTextReaderRead(xmlTextPointer);
          errorCode = xmlTextReaderRead(xmlTextPointer);
          errorCode = xmlTextReaderRead(xmlTextPointer);
          errorCode = xmlTextReaderRead(xmlTextPointer);

          value = xmlTextReaderConstValue(xmlTextPointer);
          devServicesLocal->controlURL  = (char *)malloc(strlen(value));
          strcpy(devServicesLocal->controlURL, value);

        }
        else
        {
          errorCode = xmlTextReaderRead(xmlTextPointer);
        }
      }
    }
    else
    {
      errorCode = xmlTextReaderRead(xmlTextPointer);
    }
  }
  error:
  xmlMemoryDump();
  xmlFreeTextReader(xmlTextPointer);
  xmlCleanupParser();  
  return errorCode;
}

Thanks in advance,

-Sun.

pgpython · 02-23-2011, 06:41 AM

Its really difficult to see what is going on here for several reasons. there isn't enough comments to understand what is going on and secondly you have not provided enough data for the code to be understood and tested. such as what your passing into your function when it happens and the structure of dmxLocal and devServices as well. As a point of reference experience has taught me its a good idea in C to have a few as places as possible which call malloc,calloc,free and similar stuff so it may be an idea to create functionality for that part.

Might I also suggest you use valgrind and gdb. valgrind in particular is very good at detecting memory leaks and double free errors. While testing I would compile it with debugging symbols in the executable. It will make the program but it makes much easier to debug -ggdb in GCC

ntubski · 02-23-2011, 07:50 PM

Quote:

Originally Posted by suneelmekala

Following is the complete code which I am using which is crashing:

That is not a complete program.

EDIT:
may not be the actual problem, but I think buffer is one byte too small, remember strings end with a NUL byte

Code:

 xmlBuffer = (char *)malloc(strlen(buffer)+1);
  strcpy(xmlBuffer, buffer, strlen(buffer));

suneelmekala · 02-25-2011, 03:35 AM

Information about what is going on in the function.

This is a function which parses a buffer which contains a XML file. This buffer (xml file) is read from a server over the IP using sockets.

I want to parse this and save the required fields for future use.
Following structures are used to save this information :

Code:

typedef struct dev_services
{
  char                 *serviceType;
  char                 *serviceId;
  char                 *SCPDURL;
  char                 *eventSubURL;
  char                 *controlURL;
  struct dev_services  *nextService;
}devServices;


typedef struct dmx_device
{
  char               *cacheControl;
  char               *date;
  char               *location;
  char               *server;
  char               *deviceType;
  char               *uuid;
  char               *ipAddress;
  char               *port;
  char               *devDescXML;
  devServices        *deviceServices;
  struct dmx_device  *nextNode;
}DMXDevice;

Following is the code(bold line) from where it is called :

Code:

int  main()
{
  struct sockaddr_in address;
  int                socketFD=0;
  int                fd =0;
  int                addrLen=0;
  char               *buffer=NULL;
  char               msgReceiveBuffer[1500];
  char               xmlFileBuffer[5 * 1024];
  int                xmlFileSize=0;
  DMXDevice          *dmxLocal;
  fd_set             testFds, writeFds, readFds;
  int                retryForReceive=0;
  struct timeval     timeOut;
  int                errorCode=0;
  int                localErrorNo=0;

  dmxLocal      =    gDMXDevice;

  while(dmxLocal != NULL)
  {
    memset(xmlFileBuffer,0,sizeof(xmlFileBuffer));

    socketFD                = socket(AF_INET, SOCK_STREAM, 0);

    tempPort                = strtol(dmxLocal->port, &tempStr, 10);

    address.sin_addr.s_addr = inet_addr(dmxLocal->ipAddress);
    address.sin_port        = htons(tempPort);
    address.sin_family      = AF_INET;

      if (buffer != NULL)
      {
        free(buffer);
        buffer = (char *)malloc(51 +
                                strlen(dmxLocal->ipAddress) +
                                strlen(dmxLocal->devDescXML) +
                                sizeof(dmxLocal->port));
        memset(buffer,0, sizeof(buffer));
      }
      else
      {
        buffer = (char *)malloc(51 +
                                strlen(dmxLocal->ipAddress) +
                                strlen(dmxLocal->devDescXML) +
                                sizeof(dmxLocal->port));
        memset(buffer,0, sizeof(buffer));

      }
      strcpy(buffer, "GET /");
      strcat(buffer, dmxLocal->devDescXML);
      strcat(buffer, " HTTP/1.1\r\n");
      strcat(buffer, "HOST: ");
      strcat(buffer, dmxLocal->ipAddress);
      strcat(buffer, ":");
      strcat(buffer, dmxLocal->port);
      strcat(buffer, "\r\nCONTENT-LENGTH 0\r\n\r\n");

      errorCode               = connect(socketFD,
                                       (struct sockaddr*)&address,
                                       sizeof(address));

      errorCode = write(socketFD, buffer, strlen(buffer));
        if(errorCode < 0 || errorCode < strlen(buffer))
        {
          #ifdef CTL_DISCOVER_DEBUG_NEW
            printf("ctlGetDevDesc : ERROR WHILE WRITING TO %s[%s] errorCode : %d\n",
                   dmxLocal->ipAddress,
                   dmxLocal->devDescXML,
                   errorCode);
          #endif
        }
        while(1)
        {
          FD_ZERO(&readFds);
          FD_SET(socketFD, &readFds);

          testFds = readFds;

          timeOut.tv_sec  = 2;
          timeOut.tv_usec = 500000;

          errorCode = select(FD_SETSIZE,
                             &testFds,
                             (fd_set *)0,
                             (fd_set *)0,
                             &timeOut);
          if(errorCode < 0)
          {
            #ifdef CTL_DISCOVER_DEBUG_NEW
              printf("ctlGetDevDesc : ERROR WHILE SOCKET SELECT\n");
            #endif
            goto error;
          }
          else if(errorCode == 0)
          {
            #ifdef CTL_DISCOVER_DEBUG_NEW
              printf("ctlGetDevDesc : SOCKET TIMED OUT\n");
            #endif
            retryForReceive +=1;
            if(retryForReceive < GET_DEVICE_DESC_RETRY_COUNT)
            {
              continue;
            }
            else
            {
              #ifdef CTL_DISCOVER_DEBUG_NEW
                printf("ctlGetDevDesc : Re-TRIED 5 TIMES \n");
              #endif
              retryForReceive = 0;
              break;
            }
          }
          else
          {
            for(fd=0; fd < FD_SETSIZE; fd++)
            {
              if(FD_ISSET(fd,&testFds))
              {
                if (fd == socketFD)
                {
                  memset(msgReceiveBuffer,0,sizeof(msgReceiveBuffer));
                  errorCode = read(socketFD, msgReceiveBuffer, sizeof(msgReceiveBuffer));
                  if (errorCode < 0)
                  {
                    #ifdef CTL_DISCOVER_DEBUG_NEW
                      printf("ctlGetDevDesc : ERROR WHILE RECEIVING FROM  %s[%s]\n",
                             dmxLocal->ipAddress,
                             dmxLocal->devDescXML);
                    #endif
                  }
                  else
                  {
                    #ifdef CTL_DISCOVER_DEBUG
                      printf("ctlGetDevDesc : GET DEV DESC RECEIVED BYTES : %d MSG : %s\n",
                             errorCode,
                             msgReceiveBuffer);
                    #endif
                    if(if200OK(msgReceiveBuffer) != MSG_200_OK)
                    {
                       strcat(xmlFileBuffer, msgReceiveBuffer);
                    }
                  }
                }/*if (fd == socketFD)*/
              }/*if(FD_ISSET(fd,&testFds))*/
            }/*for(fd=0; fd < FD_SETSIZE; fd++)*/
          }/* else case of errorCode check for SELECT,select is success. got the read FD*/
        }/*while(1) CHECKING FOR ANY READ FD IS SET OR NOT*/
        xmlFileBuffer[strlen(xmlFileBuffer)] = '\0';
        errorCode = parseDevDescMessage(xmlFileBuffer , dmxLocal);
      }/*else case of if(errorCode == -1 && errno == EINPROGRESS)*/
      close(socketFD);
      dmxLocal = dmxLocal->nextNode;    
    }
    #ifdef CTL_DISCOVER_DEBUG_NEW
      printf("ALL DMX DEVICE'S DESCRIPTION IS RECEIVED \n");
    #endif
    displayInformation();
  }  /*while(1)*/
  error :
  printf("");
  //pthread_exit("receiveDiscoverResponsere : EXIT called for thread");
}

Please mention if you need more information.

Thanks in advance
-Sun

suneelmekala · 02-25-2011, 08:08 AM

Dear all,

Thanks a lot for the responses. Your responses really help me. I could solve the crash.
Here is what I did :

As suggested by pgpython I used VALGRIND and found a number of memory issues. I fixed one by one all the memory related issues and in the end this crash vanished.
Might be at some place because of the corruption this memory was held on because of which when this was released it was crashing.

Thanks,

-Sun