AIX relink warning

NevemTeve · 08-10-2015, 12:09 PM

I've just relinked Oracle Instant Client 12.1's libclntsh.so, it seems working, but I get annoying warnings I'd like not to get.

before

Code:

# dump -H -X64 libclntsh.so.orig
INDEX  PATH                          BASE                MEMBER
0      /ade/b/1630515813/oracle/lib:/usr/lib:/lib:/usr/lib:/lib
1                                    libclntshcore.so
2                                    libc.a              shr_64.o
3                                    libperfstat.a       shr_64.o
4                                    libpthreads.a       shr_xpg5_64.o
5      /                             unix
6                                    libc.a              aio_64.o
7                                    libdl.a             shr_64.o
8                                    libons.so

after

Code:

# dump -H -X64 libclntsh.so
INDEX  PATH                          BASE                MEMBER
0      /usr/lib:/lib
1      /orabin/instantclient/lib64   libons.so
2      /orabin/instantclient/lib64   libclntshcore.so
3      /usr/lib                      libperfstat.a       shr_64.o
4      /usr/lib                      libpthreads.a       shr_xpg5_64.o
5      /usr/lib                      libdl.a             shr_64.o
6      /usr/lib                      libc.a              shr_64.o
7      /usr/lib                      libc.a              aio_64.o
8      /                             unix

process:

Code:

# make libclntsh.so

ld -b64 -r -o libclntsh.o -bnso ../lib/libclntsh.so.orig

explist ../lib/libclntsh.so.orig >libclntsh.exp

ld -b64 -G -bernotok -bnoentry -brtl -bbigtoc -bE:libclntsh.exp -o libclntsh.so \
libclntsh.o \
/orabin/instantclient/lib64/libons.so \
/orabin/instantclient/lib64/libclntshcore.so    \
/usr/lib/libperfstat.a          \
/usr/lib/libpthreads.a          \
/usr/lib/libdl.a                \
/usr/lib/libc.a                 \
-bI:/usr/lib/iocp.exp
ld: 0711-224 WARNING: Duplicate symbol: .bcopy
ld: 0711-224 WARNING: Duplicate symbol: .fsav
ld: 0711-224 WARNING: Duplicate symbol: .fres
ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more information.
ld: 0711-773 WARNING: Object libclntsh.o, imported symbol slts_runmode
        Symbol was expected to be local. Extra instructions
        are being generated to reference the symbol.
ld: 0711-773 WARNING: Object libclntsh.o, imported symbol lxdobl
        Symbol was expected to be local. Extra instructions
        are being generated to reference the symbol.
ld: 0711-773 WARNING: Object libclntsh.o, imported symbol slfitcl
        Symbol was expected to be local. Extra instructions
        are being generated to reference the symbol.
ld: 0711-773 WARNING: Object libclntsh.o, imported symbol slfitlb
        Symbol was expected to be local. Extra instructions
        are being generated to reference the symbol.
ld: 0711-773 WARNING: Object libclntsh.o, imported symbol timezone
        Symbol was expected to be local. Extra instructions
        are being generated to reference the symbol.

echo 'If you are brave enough, you might copy libclntsh.so into ../lib'
If you are brave enough, you might copy libclntsh.so into ../lib

The 0711-773 warnings bug me; google proves than many people got those, but I couldn't find the solution.

pan64 · 08-12-2015, 04:26 AM

since there was no answer:
version mismatch problem, among libs/headers. Probably one (a few) of those libs were compiled with different flags/headers..

NevemTeve · 02-23-2017, 06:39 AM

I've just found out, that my instantclient!libclntsh.so contains BIO_* entries that should exist in openssl!libcrypto.so
It's quite disturbing.

Here is what's happened to poor PHP:

Code:

$ grep BIO libphp7.so.imports 
[375]   0x00000000    undef      IMP     DS EXTref /opt/lib64/libclntsh.so.12 BIO_printf
[376]   0x00000000    undef      IMP     DS EXTref /opt/lib64/libclntsh.so.12 BIO_free
[377]   0x00000000    undef      IMP     DS EXTref /opt/lib64/libclntsh.so.12 BIO_new
[378]   0x00000000    undef      IMP     DS EXTref /opt/lib64/libclntsh.so.12 BIO_write
[379]   0x00000000    undef      IMP     DS EXTref /opt/lib64/libclntsh.so.12 BIO_ctrl
[380]   0x00000000    undef      IMP     DS EXTref /opt/lib64/libclntsh.so.12 BIO_free_all
[484]   0x00000000    undef      IMP     DS EXTref /usr/local/lib64/libcrypto.so.1.0.2 BIO_puts
[485]   0x00000000    undef      IMP     DS EXTref /usr/local/lib64/libcrypto.so.1.0.2 BIO_s_mem
[486]   0x00000000    undef      IMP     DS EXTref /usr/local/lib64/libcrypto.so.1.0.2 BIO_new_mem_buf
[487]   0x00000000    undef      IMP     DS EXTref /usr/local/lib64/libcrypto.so.1.0.2 BIO_new_file

NevemTeve · 02-23-2017, 08:23 AM

A possible fix (part of the Makefile):

Code:

DONT_EXPORT_THESE := ^(BIO_|ERR_|SSLv23_|SSLv3_|TLSv1_|ssl23_|ssl3_|ssl_|tls1_|BZ2_|inflate|deflate)

%.exp_64: ../lib64/%.so.orig
        explist $< >$@.to_prune
        egrep -v '${DONT_EXPORT_THESE}' $@.to_prune >$@

%.exp_32: ../lib32/%.so.orig
        explist $< >$@.to_prune
        egrep -v '${DONT_EXPORT_THESE}' $@.to_prune >$@

NevemTeve · 02-25-2017, 10:19 AM

This problem might be related with this: aix6 / 64-bit / Apache / libphp5 / dlopen problem

NevemTeve · 04-26-2017, 04:30 AM

I think I should create the export-list this-way:

Code:

dump -Tv -X64 libclntsh.so.orig |
grep ' EXP ' |
while read _ _ _ _ _ _ _ S;
do
    echo "$S"
done |
LC_CYTPE=C sort

pan64 · 04-26-2017, 04:42 AM

that is better with an awk:

Code:

dump | awk '/ EXP / { print $6 }' | sort

Laserbeak · 04-27-2017, 09:25 PM

I worked at a company than ran AIX for about a year and I had never-ending problems trying to get things to compile correctly on it. The fact that I didn't have root access and the people that did worked in an office like 2,000 miles away didn't help either!

Michael AM · 06-01-2017, 09:40 AM

relinking - better - dealing with two types of warnings:

ld: 0711-224 WARNING: Duplicate symbol: .fres

ld: 0711-773 WARNING: Object libclntsh.o, imported symbol slts_runmode
Symbol was expected to be local. Extra instructions
are being generated to reference the symbol.

And first - I highly recommend using the hint given to understand what may be the root cause of the warning:

ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more information.

First of all, "Duplicate symbol" means that there are multiple objects (shared/static) that could resolve a symbol. If you know the code is equal (same code but in multiple archives aka .a files), not pretty, but harmless. If I recall correctly (irrc), -bloadmap will tell you which one is actually expected to be used.
Personally, I see these messages as a way to track down embedded library code that I actually want to not be embedded. So, as long as you are aware this is not an AIX problem - this is a packaging or build problem because the files offered as "resolvers" have multiple definitions and/or the expectation is that the symbol is supposed to be local (or static) but has been set to exported.

In any case, to get an idea of what your target "libclntsh.so" use the command

dump -Tv libclntsh.so

There is more that I am going to research - and I shall add, later, roughly, an explanation of what the commands are doing, and why, additional research, imho, is a necessity when warnings such as these appear.

Quick question: where did you get the files (source files) to create the "object" libclntsh.o ?

Michael AM · 06-01-2017, 10:41 AM

Anyway, seeing that the original poster was from 2015 - modifying how I move forward.

Basically, my guess that the "relink" was needed was because the directory

Quote:

/ade/b/1630515813/oracle/lib

did not exist, or was not generally available AND the poster wanted to have the libraries

libclntsh.so, libons.so and libclntshcore.so in the directory

Quote:

/orabin/instantclient/lib64

The warning messages are not being caused by the AIX files in /usr/lib or /unix. The warnings are coming from information in the files: libclntsh.so, libons.so and libclntshcore.so

NevemTeve · 06-01-2017, 02:18 PM

Well, the core of the problems is that AIX is an exotic platform with many unique features. Anyways, I tried to write something about these questions: http://web.axelero.hu/lzsiga/aix-linking.html

Michael AM · 06-02-2017, 02:13 AM

Well, I see you have done some homework. I'll remember it when I try to explain what it means to support the GNU ecosystem on your own (perhaps the main reason I use xlc rather than gcc, and when I cannot use gcc - I pass).

PHP is special. No idea why, but it has required a lot of manual intervention since the first time I worked on a build for AIX with php-4.0.4.

re: the loader flags - many of them that you quote are automatic aka default. Several of the ones you mention are extremely useful when you are doing something non-standard. These days, imho, anything in /usr/local is, read: should be treated as, an exception.

And to get back to exotic - good exotic or bad exotic (as in (poorly translated?) Dutch saying, the farmer does not eat what he does not know/grow).

I like to think of AIX as forgiving. When you are doing something dangerous: it warns and goes on if it can, and gives an error when it cannot. I prefer the warnings to no message (as in, he I got it done - not going to mention any potential conflicts) - ah, yes, there is the non-default -q flag for gcc to report all errors (-qerror=all, or something like that. I see it hardwired in many Makefiles.)

Every platform has it's idiosyncrasies - it is what makes them identifiable. Linux is fine, GNU is fine - but you have to learn to "think": Linix, GNU, POSIX, UNIX, HP-UX, Solaris, Tru64 and even AIX. AIX is the "one of those" I know best - but I try to respect all of them. "Smarter people than I" made decisions on all of these. That I do not agree all the time (or understand them immediately) does not make the wrong.

I'll re-read your post and maybe do a re-write more to my style. I normally do not refer to my portal - but you can see how I write at http://www.rootvg.net

Thanks for the conversation! You are helping me to improve on what I know! I appreciate that.

NevemTeve · 04-09-2018, 03:18 PM

Since then, I improved the export-list creating command to cover EXP, wEXP, ImpExp and wImpExp: http://web.axelero.hu/lzsiga/explist_so

NevemTeve · 03-18-2019, 07:32 AM

Moved to:
http://lzsiga.users.sourceforge.net/aix-linking.html
http://lzsiga.users.sourceforge.net/explist_so

Michael AM · 04-15-2019, 08:11 AM

Thanks for the update. Always a hassle when links "move".