[SOLVED] Getting valgrind debug symbols to work properly in Debian

crazy-yiuf · 06-24-2016, 08:43 PM

Edit: You'll probably want to skip to post #7 /Edit

About 6 months ago I decided to make a little gtkmm application. I ran valgrind on the first tutorial example (here's a link, but the code is totally irrelevant to the question):
https://developer.gnome.org/gtkmm-tu...xample.html.en

and found that it "definitely lost" 16 kb, and much more with big GUIs. So I thought maybe I'd try to get to the bottom of it. But the valgrind output looks sketchy and starts with a bunch of errors:

Code:

--22792-- WARNING: Serious error when reading debug info
--22792-- When reading debug info from /lib/x86_64-linux-gnu/ld-2.22.so:
--22792-- Ignoring non-Dwarf2/3/4 block in .debug_info

It turned out this is a known issue they are working on. I finally checked back today, and it appears they have a patch for it:
https://bugs.debian.org/cgi-bin/bugr...cgi?bug=810295
https://bugs.kde.org/show_bug.cgi?id=303877

I'm running Debian sid and just updated (which puts me at valgrind 1:3.11.0-1+b1), and still get the errors. I'm not the best C programmer in the world, and can't really figure out what they're talking about in those threads. Is this a patch I can apply? Do I have to download the latest code and compile, or is there a more clever fix?

And is valgrind not as popular as I thought? Seems like people would be bellyaching about this a lot more. Thanks for any input, and sorry if I'm being dense about something.

crazy-yiuf · 06-24-2016, 08:52 PM

Ok, I read it better and it looks like I just need to compile it, bleh. Sorry for the dumb post, I'll mark it as solved until I inevitably can't get it to work, or just give up for another six months.

crazy-yiuf · 06-25-2016, 01:11 AM

It works! Here are the steps I took:

Code:

mkdir valsrc
cd valsrc

In this directory, save this page as thepatch:
https://bugs.debian.org/cgi-bin/bugr...d.patch;msg=29

Code:

apt-get source valgrind
patch -p0 < thepatch
cd valgrind-3.11.0/
dpkg-buildpackage -us -uc
apt-get install #any dependencies it complains about in the previous step
dpkg-buildpackage -us -uc
#this took about an hour on my old 1ghz processor
cd ..
dpkg -i valgrind_3.11.0-1.1_amd64.deb
dpkg -i valgrind-dbg_3.11.0-1.1_amd64.deb
dpkg -i valgrind-mpi_3.11.0-1.1_amd64.deb

And now I know how to patch and rebuild packages on Debian! Unfortunately, the output isn't all that much more useful, so I guess that's why it hasn't urgently been pushed into the repos.

crazy-yiuf · 06-25-2016, 11:36 PM

Ok, now I'm very confused. When I use those debs with valgrind -v I get a bunch of this stuff:

Code:

--8945-- Reading syms from /usr/lib/valgrind/vgpreload_core-amd64-linux.so
--8945--   Considering /usr/lib/valgrind/vgpreload_core-amd64-linux.so ..
--8945--   .. CRC mismatch (computed 594b1410 wanted 889d5cfd)
--8945--   Considering /usr/lib/debug/usr/lib/valgrind/vgpreload_core-amd64-linux.so ..
--8945--   .. CRC is valid
--8945-- Reading syms from /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so
--8945--   Considering /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so ..
--8945--   .. CRC mismatch (computed 046b3ed1 wanted aba53a2a)
--8945--   Considering /usr/lib/debug/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so ..
--8945--   .. CRC is valid

which on google seems to be related to mangled versions? Though I see these mismatches showing up a lot of other places too. So I uninstalled the three packages, and verified that "which valgrind" didn't find anything. Then I installed the manually built debs and did a stat /usr/bin/valgrind:

Code:

  File: '/usr/bin/valgrind'
  Size: 528       	Blocks: 8          IO Block: 4096   regular file
Device: 805h/2053d	Inode: 920229      Links: 1
Access: (0755/-rwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2016-06-25 22:02:11.000000000 -0600
Modify: 2015-09-25 05:41:20.000000000 -0600
Change: 2016-06-25 22:02:16.133937013 -0600
 Birth: -

Unmodified since 2015?? Where is it pulling this binary from? I guess it's possible that the makefile didn't deem it necessary to change the main binary, in which case, why the CRC mismatch?

crazy-yiuf · 06-26-2016, 05:02 PM

After some digging, I've found that buildpackage uses debhelper, which sets the mtime of all package files to the latest date in the change log. The latest date in the valgrind change log is Sept. 25, 2015

I'm satisfied with my valgrind now. Now to track down the rest of the debug symbols. Also for future googlers I feel I should say "modify time timestamp wrong binary buildpackage deb mtime"

norobro · 06-26-2016, 05:41 PM

Quote:

Originally Posted by crazy-yiuf

"modify time timestamp wrong binary buildpackage deb mtime"

valgrind is a wrapper script that the compiler doesn't modify:

Code:

$ cat /usr/bin/valgrind
#!/bin/sh -e
#
# Valgrind wrapper

# Default Debian debug libraries.
DBGPATH=/usr/lib/debug

# Use debug libraries if found.
if [ -z "$LD_LIBRARY_PATH" ]; then
        export LD_LIBRARY_PATH="$DBGPATH"
else
        export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$DBGPATH"
fi

# Force C++ STL to use malloc and to free memory by disabling 
# memory caching.
# For gcc < 3.4 versions
export GLIBCPP_FORCE_NEW=1

# For gcc >= 3.4 versions
export GLIBCXX_FORCE_NEW=1
                                                                                            
# Use 'exec' to avoid having another shell process hanging around.                          
exec $0.bin "$@"

Run stat on valgrind.bin :

Code:

$ stat /usr/bin/valgrind.bin
  File: '/usr/bin/valgrind.bin'
  Size: 18760           Blocks: 40         IO Block: 4096   regular file
Device: 811h/2065d      Inode: 813951      Links: 1
Access: (0755/-rwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2016-06-26 08:13:23.284944030 -0500
Modify: 2016-02-02 03:53:45.000000000 -0600
Change: 2016-05-21 13:53:36.869593939 -0500
 Birth: -

$ stat /usr/bin/valgrind
  File: '/usr/bin/valgrind'
  Size: 528             Blocks: 8          IO Block: 4096   regular file
Device: 811h/2065d      Inode: 813953      Links: 1
Access: (0755/-rwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2016-06-26 08:13:23.284944030 -0500
Modify: 2015-09-25 06:41:20.000000000 -0500
Change: 2016-05-21 13:53:36.869593939 -0500
 Birth: -

crazy-yiuf · 06-26-2016, 08:20 PM

Ah. Right you are. I really should have thought of that. Thanks!

I'm stuck again, though. I'm back to my gtkmm hello world program. I went through and installed all of the -dbg and -dbgsym packages I could find for these:

Code:

echo `pkg-config gtkmm-3.0 --cflags --libs`

And I still get::

Code:

valgrind -v --leak-check=full --track-origins=yes ./a.out
...
==14373== 16,384 bytes in 1 blocks are definitely lost in loss record 10,453 of 10,465
==14373==    at 0x4C29C0F: malloc (vg_replace_malloc.c:299)
==14373==    by 0x8671728: g_malloc (gmem.c:94)
==14373==    by 0x867B8CB: g_quark_init (gquark.c:62)
==14373==    by 0x400F2D9: call_init.part.0 (dl-init.c:72)
==14373==    by 0x400F3EA: call_init (dl-init.c:30)
==14373==    by 0x400F3EA: _dl_init (dl-init.c:120)
==14373==    by 0x4000CC9: ??? (in /lib/x86_64-linux-gnu/ld-2.22.so)
==14373== 
==14373== LEAK SUMMARY:
==14373==    definitely lost: 16,384 bytes in 1 blocks
==14373==    indirectly lost: 0 bytes in 0 blocks
==14373==      possibly lost: 8,040 bytes in 103 blocks
==14373==    still reachable: 1,891,323 bytes in 20,689 blocks
==14373==                       of which reachable via heuristic:
==14373==                         length64           : 35,320 bytes in 460 blocks
==14373==                         newarray           : 2,384 bytes in 69 blocks
==14373==         suppressed: 0 bytes in 0 blocks
==14373== Reachable blocks (those to which a pointer was found) are not shown.
==14373== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==14373== 
==14373== ERROR SUMMARY: 70 errors from 70 contexts (suppressed: 0 from 0)
==14373== ERROR SUMMARY: 70 errors from 70 contexts (suppressed: 0 from 0)

I'm new to valgrind, but here I'm assuming:
1. The ??? is what I really need to find the leak?
2. The ??? is caused by a library being compiled without debug information?
3. There's probably some other method for figuring out which executable it came from even without the debug symbols? Maybe gdb?

If I'm interpreting the valgrind -v correctly, I'm still missing the debug symbols for these:
libxkbcommon0
libepoxy0
libdbus-1-3
liblzma5
also valgrind-dbg apparently doesn't have a dynamic symbol table, but I doubt that's related.

crazy-yiuf · 06-27-2016, 12:55 AM

Edit: nevermind, deleted, stupid post.

ntubski · 06-27-2016, 10:05 AM

Quote:

Originally Posted by crazy-yiuf

Ah. Right you are. I really should have thought of that. Thanks!

I'm stuck again, though. I'm back to my gtkmm hello world program. I went through and installed all of the -dbg and -dbgsym packages I could find for these:

Code:

echo `pkg-config gtkmm-3.0 --cflags --libs`

And I still get::

Code:

valgrind -v --leak-check=full --track-origins=yes ./a.out
...
==14373== 16,384 bytes in 1 blocks are definitely lost in loss record 10,453 of 10,465
==14373==    at 0x4C29C0F: malloc (vg_replace_malloc.c:299)
==14373==    by 0x8671728: g_malloc (gmem.c:94)
==14373==    by 0x867B8CB: g_quark_init (gquark.c:62)
==14373==    by 0x400F2D9: call_init.part.0 (dl-init.c:72)
==14373==    by 0x400F3EA: call_init (dl-init.c:30)
==14373==    by 0x400F3EA: _dl_init (dl-init.c:120)
==14373==    by 0x4000CC9: ??? (in /lib/x86_64-linux-gnu/ld-2.22.so)
==14373== 
==14373== LEAK SUMMARY:
==14373==    definitely lost: 16,384 bytes in 1 blocks
==14373==    indirectly lost: 0 bytes in 0 blocks
==14373==      possibly lost: 8,040 bytes in 103 blocks
==14373==    still reachable: 1,891,323 bytes in 20,689 blocks
==14373==                       of which reachable via heuristic:
==14373==                         length64           : 35,320 bytes in 460 blocks
==14373==                         newarray           : 2,384 bytes in 69 blocks
==14373==         suppressed: 0 bytes in 0 blocks
==14373== Reachable blocks (those to which a pointer was found) are not shown.
==14373== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==14373== 
==14373== ERROR SUMMARY: 70 errors from 70 contexts (suppressed: 0 from 0)
==14373== ERROR SUMMARY: 70 errors from 70 contexts (suppressed: 0 from 0)

I'm new to valgrind, but here I'm assuming:
1. The ??? is what I really need to find the leak?
2. The ??? is caused by a library being compiled without debug information?
3. There's probably some other method for figuring out which executable it came from even without the debug symbols? Maybe gdb?

It's probably a false positive: http://valgrind.org/docs/manual/faq.html#faq.reports

Quote:

4.1. My program uses the C++ STL and string classes. Valgrind reports 'still reachable' memory leaks involving these classes at the exit of the program, but there should be none.

First of all: relax, it's probably not a bug, but a feature. Many implementations of the C++ standard libraries use their own memory pool allocators. Memory for quite a number of destructed objects is not immediately freed and given back to the OS, but kept in the pool(s) for later re-use. The fact that the pools are not freed at the exit of the program cause Valgrind to report this memory as still reachable. The behaviour not to free pools at the exit could be called a bug of the library though.

crazy-yiuf · 06-27-2016, 12:07 PM

Yeah, I'd ignore it if it was anything other than "definitely lost". There's a good chance you're right, but I'm going to try to prove it to myself as a learning exercise. Thanks though, any more thoughts are also appreciated.

I should have noted that I've been running with:
export G_SLICE=always-malloc
export G_DEBUG=gc-friendly
export NO_AT_BRIDGE=1

I don't have gnome installed, so the NO_AT_BRIDGE is to suppress this sort of error (which it manages to do about half the time):

Code:

** (synaptic:16706): WARNING **: Error retrieving accessibility bus address: org.freedesktop.DBus.Error.ServiceUnknown: The name org.a11y.Bus was not provided by any .service files

Also I'm compiling with:
g++ -gdwarf-4 `pkg-config gtkmm-3.0 --cflags --libs` main.cpp
Edit: export GLIBCXX_FORCE_NEW didn't make a difference.
Edit: Spent the day updating an old Kali installation, so I could try it in a full gnome environment. Exact same number - 16,384.