[SOLVED] 0x08063781 in LocalHost::LocalHost() ()

james000 · 02-12-2024, 01:36 PM

Hi,
I have two Centos 8.6 VMs. Both are created with same templates, so they are supposed to be identical, unless something was changed later which I can't figure. As per application team, same code is working on some set of server, but on another set of servers, so they are pointing, something is different or wrong on OS or library or config level.

appuser is supposed to run gdb on tcsid and then issue "run" command. It is suppsed to create a unique key for this host. Below are two example. I picked one server from working set of servers and one server from non-working set of servers. I do not know, how this application or code works, but trying to find if I can get some clue which may be different on OS level.

hostname, hostnamectl seems to be same and in same format.

Working server -->

Code:

[root@phl003 ~]# su - appuser
(18:09:41) PROD SITE=XA | phl003:appuser> cd /opt/tcs/corebaser-14.14.1/bin
(18:09:50) PROD SITE=XA | phl003:bin> gdb tcsid
GNU gdb (GDB) Red Hat Enterprise Linux 8.2-19.el8
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from tcsid...(no debugging symbols found)...done.
(gdb) run
Starting program: /opt/tcs/corebaser-14.14.1/bin/tcsid
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
79f1fd490796b5e2636478efdddc92521c10572f9c555a74582f8a7ebb9eess779555bf9e3c77dbff3566827c8044bf6
[Inferior 1 (process 1374472) exited normally]
Missing separate debuginfos, use: yum debuginfo-install tcs-corebaser-14.14.1-71557.202310081822.BC43o.i686
(gdb)

Non-working server -->

Quote:

[root@oyr003 ~]# su - appuser
(18:09:39) PROD SITE=YA | oyr003:appuser> cd /opt/tcs/corebaser-14.14.1/bin
(18:09:45) PROD SITE=YA | oyr003:bin> gdb tcsid
GNU gdb (GDB) Red Hat Enterprise Linux 8.2-19.el8
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from tcsid...(no debugging symbols found)...done.
(gdb) run
Starting program: /opt/tcs/corebaser-14.14.1/bin/tcsid
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

Program received signal SIGILL, Illegal instruction.
0x08063781 in LocalHost::LocalHost() ()
Missing separate debuginfos, use: yum debuginfo-install tcs-corebaser-14.14.1-71557.202310081822.BC43o.i686
(gdb)

Can someone help me with, what to look to do further troubleshooting ?

Thanks

pan64 · 02-12-2024, 01:46 PM

so you need to compare all the libs, one by one, need to find which one is different.
use ldd to find them. Also you need to check the binary itself and the config files.
you might try strace to find what's going on or what went wrong (probably helps).
Why do you need gdb to run it? Does it work without gdb?

james000 · 02-12-2024, 02:14 PM

Quote:

Originally Posted by pan64

so you need to compare all the libs, one by one, need to find which one is different.
use ldd to find them. Also you need to check the binary itself and the config files.
you might try strace to find what's going on or what went wrong (probably helps).
Why do you need gdb to run it? Does it work without gdb?

I tried to capture all rpms (rpm -qa > /var/tmp/all-rpms.txt) and compare, both have same and no difference.
/opt/tcs/corebaser-14.14.1/bin/tcsid checksum is same on working as well as non-working

I do not know, why they use gdb. It is their (apps team) standard procedure I guess. Maybe it was written in such way.

For strace, should I run it as "strace gdb tcsid". How there is way to do strace on "run" ? Maybe that can give some clue, but how to run it ?

pan64 · 02-12-2024, 02:25 PM

yes, exactly that way, but probably you need to add some flags:

Code:

strace -o output.txt -f <some command>

it should work exactly the same way, just it will be slower.
gdb is a debugger and run means the simple execution of that app, therefore gdb <app>; run is more or less equal to just run <app> without gdb. I don't really see the reason to use gdb. Anyway, you can try it without it to see at least if that will do the same.
But I'm not really sure if gdb and strace work together.
You can use ldd /opt/tcs/corebaser-14.14.1/bin/tcsid to find the required libs and you can also compare them.
You can also use the output of strace to see the files opened, so you will know what should be compared.

james000 · 02-15-2024, 11:11 AM

Libraries were all good. I found one difference between working and non-working server. There was avx flag in CPU, in working one.

Code:

[root@oyr003 ~]# grep ^flags /proc/cpuinfo |uniq |grep -i avx
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon nopl xtopology tsc_reliable nonstop_tsc cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced fsgsbase tsc_adjust bmi1 avx2 smep bmi2 invpcid avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves arat pku ospke avx512_vnni md_clear flush_l1d arch_capabilities
[root@oyr003 ~]#

After changing the feature on underlying blade and reboot of this VM (not reboot, but power off and power on), everything worked fine. Thanks for your efforts and help.