Segfault in a program using pthreads

rmsrox · 03-16-2010, 11:47 PM

Hi,

I am trying to write a peer to peer application in C++ using pthreads library. There are few types of nodes that run in the program. The 'beacon' nodes form the core of the network. The ordinary nodes 'join' the network by sending join requests to the beacon node.

So, my program runs till the join requests and responses are received. When I try to 'connect' using connect function from the socket library, I get a segfault. I am not sure if the segfault occurs for the connect function, because sometimes the program just runs to completion. It could be an issue related to threads or memory allocation.

When I run gdb, it shows me the following:

Code:

(gdb) run b2-n00.ini
Starting program: sv_node b2-n00.ini
servant:12600> [New LWP 1]
[New LWP 2]
[New LWP 3]
[New LWP 4]
[New LWP 5]
[New LWP 6]
Before Exiting[LWP 1 exited]
procfs: fetch_registers, get_gregs line 3768, /proc/23952/lwp/1: No such file or directory.

I am really not sure what's happening. Any pointers on what's going on?

ForzaItalia2006 · 03-17-2010, 02:33 PM

Hey,

that doesn't look like a typical segmentation fault (SIGSEGV, signal 11) reported by GDB ;-). I made a quick google search and found that the listed functions belong to the GDB code, so I would for now assume that this is a bug in some old GDB. Which version of GDB are you running (gdb --version). The newest version is 7.0.1. I think that - after upgrading to a newer version - you should then be able to see where the segmentation fault really happens and which parameter of connect(2) causes the crash.

For a short-term satisfaction you could also attach your source code (please DON'T post) and I'll try on my system ...

- Andi -

rmsrox · 03-17-2010, 10:29 PM

Hey,

I think I figured out the problem related to the 'thread not found', but now I have a segfault in my code. The gdb trace shows this:

Code:

Starting program: sv_node b2-n00.ini
servant:12600> [New LWP 1]
[New LWP 2]
[New LWP 3]
[New LWP 4]
[New LWP 5]
[New LWP 6]
[New LWP 7]
[New LWP 8]
12601 [LWP 3 exited]
[New LWP 3]
[LWP 4 exited]
[New LWP 4]
[LWP 5 exited]
[New LWP 5]
[New LWP 9]
[New LWP 10]
12592  12601 
Program received signal SIGSEGV, Segmentation fault.
[Switching to LWP 7]
0xfee56a60 in realfree () from /usr/lib/libc.so.1
(gdb) bt
#0  0xfee56a60 in realfree () from /usr/lib/libc.so.1
#1  0xfee57314 in cleanfree () from /usr/lib/libc.so.1
#2  0xfee5646c in _malloc_unlocked () from /usr/lib/libc.so.1
#3  0xfee5635c in malloc () from /usr/lib/libc.so.1
#4  0xff15dd10 in operator new (sz=424)
    at ../../.././libstdc++-v3/libsupc++/new_op.cc:57
#5  0x0002ae74 in __gnu_cxx::new_allocator<message>::allocate (this=0x474dc, 
    __n=1)
    at /auto/usc/gnu/gcc/4.2.1/bin/../lib/gcc/sparc-sun-solaris2.10/4.2.1/../../../../include/c++/4.2.1/ext/new_allocator.h:91
#6  0x0002aeb8 in std::_Deque_base<message, std::allocator<message> >::_M_allocate_node (this=0x474dc)
    at /auto/usc/gnu/gcc/4.2.1/bin/../lib/gcc/sparc-sun-solaris2.10/4.2.1/../../../../include/c++/4.2.1/bits/stl_deque.h:430
#7  0x0002af20 in std::deque<message, std::allocator<message> >::_M_push_back_aux (this=0x474dc, __t=@0xfe8fbb5c)
    at /auto/usc/gnu/gcc/4.2.1/bin/../lib/gcc/sparc-sun-solaris2.10/4.2.1/../../../../include/c++/4.2.1/bits/deque.tcc:309
#8  0x0002b008 in std::deque<message, std::allocator<message> >::push_back (
    this=0x474dc, __x=@0xfe8fbb5c)
    at /auto/usc/gnu/gcc/4.2.1/bin/../lib/gcc/sparc-sun-solaris2.10/4.2.1/../../../../include/c++/4.2.1/bits/stl_deque.h:1062
#9  0x0002b030 in std::queue<message, std::deque<message, std::allocator<messag---Type <return> to continue, or q <return> to quit---
e> > >::push (this=0x474dc, __x=@0xfe8fbb5c)
    at /auto/usc/gnu/gcc/4.2.1/bin/../lib/gcc/sparc-sun-solaris2.10/4.2.1/../../../../include/c++/4.2.1/bits/stl_queue.h:204
#10 0x00018830 in recvRoutine (recvarg=0xfedfbd04) at beacon.cc:851
#11 0xfeec88b4 in _lwp_start () from /usr/lib/libc.so.1
#12 0xfeec88b4 in _lwp_start () from /usr/lib/libc.so.1
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) frame 10
#10 0x00018830 in recvRoutine (recvarg=0xfedfbd04) at beacon.cc:851
851                     myqueue[qid].push(pktinfo);
(gdb) print pktinfo
$1 = {length = 0, location = 0, distance = 30193, 
  host = "nunki.usc.edu", '\0' <repeats 242 times>, 
  uoid = "\203R\016ñsI'\033+\2272Ó/Í\221¬\023\200^3", 
  recv_uoid = "]w\001ÇÃ\225Ñ]-Û#a;{\023@Úùçð", port = 12601, 
  type = 14 '\016', 
  msgtype = "join response forward\0009", '\0' <repeats 76 times>, 
  nsocket = 11, ttl = 30, dead = 0}
(gdb) print qid
$2 = 1

So, I guess now it shows a segfault on my recvRoutine (line 10 of bt) function.

Any pointers?

PS: I am new to gdb, so likely that I have missed out on few things.

ForzaItalia2006 · 03-18-2010, 10:03 AM

Hey,

ouch, that doesn't look good :-) From past experiences, a segmentation fault in one of the malloc/free functions really often indicates heap corruption, meaning that you possibly double free(3)ed a pointer or wrote past the allocated memory area and overwrote some malloc-specific meta-data.

The easiest way to figure out which part of your app causes the heap corruption, I would recommend using the 'valgrind' utility, especialy the memcheck tool of 'valgrind' should do it right.

Please check out these pages:

valgrind generic
http://valgrind.org/docs/manual/manual-core.html

mem-check specific
http://valgrind.org/docs/manual/mc-manual.html

Hope that helps,
- Andi -

rmsrox · 03-19-2010, 10:57 PM

Well, I tried to install the valgrind utility on the server that I am working on(My code is supposed to run on a particular server machine, so I am working on it directly, rather than on a local machine), but that machine did not let me do so.

So, I am still using gdb .. And I am still getting the segfault when I try to push a struct on the queue.

I am using STL queues, and I have an array of queues. I wonder if STL takes care of memory allocation by it's own.

I tried putting the struct in different queues(from the array of queues), but all of them return a segfault which makes me believe it is something relating to memory allocation/corruption.

Any help would be appreciated !
Thanks !

smeezekitty · 03-19-2010, 11:25 PM

Kan u post u code? No really.
Can you post your code?

neonsignal · 03-20-2010, 07:52 AM

Code:

I am using STL queues, and I have an array of queues. I wonder if STL takes care of memory allocation by it's own.

The STL does take care of memory allocation for most operations on STL data structures (eg push_back, etc).

One operation to watch is vector access using an index; this does not automatically resize the vector.