readline problem (bug)

bartgymnast · 10-03-2013, 07:07 PM

I came across a problem with readline

I tried compiling some packages with system readline support, however more than half those packages could not find readline.

after a bunch of testing I made some changes in the readline.SlackBuild and everything was able to compile with readline.

the changes I made:

---libdir=/usr/lib${LIBDIRSUFFIX} \
+--libdir=/lib${LIBDIRSUFFIX} \

-make -j4 static shared || exit 1
+make -j4 SHLIB_LIBS=-lncurses static shared || exit 1

+mkdir -p $PKG/usr/lib${LIBDIRSUFFIX}
+mv -v $PKG/lib${LIBDIRSUFFIX}/lib{readline,history}.a $PKG/usr/lib${LIBDIRSUFFIX}

+rm -v $PKG/lib${LIBDIRSUFFIX}/lib{readline,history}.so
+ln -sfv ../../lib${LIBDIRSUFFIX}/libreadline.so.5 $PKG/usr/lib${LIBDIRSUFFIX}/libreadline.so
+ln -sfv ../../lib${LIBDIRSUFFIX}/libhistory.so.5 $PKG/usr/lib${LIBDIRSUFFIX}/libhistory.so

-chmod 755 $PKG/usr/lib${LIBDIRSUFFIX}/lib*.so.*
+chmod 755 $PKG/lib${LIBDIRSUFFIX}/lib*.so.*

Some thanks go to LFS

GazL · 10-04-2013, 03:46 AM

Whenever I've used readline I've just linked-in the curses library when I'm building the thing that uses it:

Code:

gazl@ws1:/tmp$ cat testreadline.c
#include <stdio.h>
#include <readline/readline.h>

int main(void)
{
   printf("You entered: %s\n", readline("input: "));
   return 0;
}
gazl@ws1:/tmp$ cc -o testreadline -lreadline testreadline.c
/usr/lib64/gcc/x86_64-slackware-linux/4.8.1/../../../../lib64/libreadline.so: undefined reference to `tputs'
/usr/lib64/gcc/x86_64-slackware-linux/4.8.1/../../../../lib64/libreadline.so: undefined reference to `tgoto'
/usr/lib64/gcc/x86_64-slackware-linux/4.8.1/../../../../lib64/libreadline.so: undefined reference to `tgetflag'
/usr/lib64/gcc/x86_64-slackware-linux/4.8.1/../../../../lib64/libreadline.so: undefined reference to `UP'
/usr/lib64/gcc/x86_64-slackware-linux/4.8.1/../../../../lib64/libreadline.so: undefined reference to `tgetent'
/usr/lib64/gcc/x86_64-slackware-linux/4.8.1/../../../../lib64/libreadline.so: undefined reference to `tgetnum'
/usr/lib64/gcc/x86_64-slackware-linux/4.8.1/../../../../lib64/libreadline.so: undefined reference to `PC'
/usr/lib64/gcc/x86_64-slackware-linux/4.8.1/../../../../lib64/libreadline.so: undefined reference to `tgetstr'
/usr/lib64/gcc/x86_64-slackware-linux/4.8.1/../../../../lib64/libreadline.so: undefined reference to `BC'
collect2: error: ld returned 1 exit status
gazl@ws1:/tmp$ cc -o testreadline -lreadline -lcursesw testreadline.c
gazl@ws1:/tmp$ ./testreadline 
input: wibble wobble froop
You entered: wibble wobble froop
gazl@ws1:/tmp$

Maybe it's the things you're trying to build that are at fault for not doing so?

bartgymnast · 10-04-2013, 06:49 AM

GazL,

You are saying yourself that you are compiling with linked-in the curses library.
and yes, the packages that I try to build are the faulty ones here.

* "bash" | with config option: --with-installed-readline
will not work.
* "bluez" 5.x | will error if not supplied by sed -i -e 's|-lreadline|\0 -lncursesw|g' Makefile.{in,tools}
* mozjs17.0.0 | will error on configure itself if supplied with the option --with-system-readline

this is a small selection.

programs these days expect that readline is already build against ncursesw.
The problem here is readline and not the other packages.

even tho, slackwares readline is build with the option --with-curses
it would mean that your supplied option of -lcursesw is not needed.

GazL · 10-04-2013, 07:23 AM

If upstream intended libreadline.so to be linked to curses when it is built then I would have expected them to have included the option to link it in the library's makefile and no modification would be necessary. Instead what they seem to have done is to include hooks (when --with-curses is used) that will allow the linking to be deferred until the executables that will use the library are built (requiring that both libraries be specified on the linker options). I'm not going to try and guess why they chose to do this, but as it's what they chose to do it seems to me that distros should respect that choice. The fact that other distros have muddied the waters by going against the intent of upstream and modifying the build: setting an expectation that '-lcurses' is not needed anymore on the programs that use it, is beside the point.

However, if I'm wrong on this and you are right that the way it is done now is indeed a bug, then it's a bug in the readline build process that should be addressed in upstream's makefile and should be reported upstream. IMO it's not the distro's job to second guess the intents of upstream: that way leads to fatal mistakes like the debian ssl debacle.

bartgymnast · 10-04-2013, 07:58 AM

I checked the readline Documentation.

And it appears slackware builds it wrong, or it is intended wrong:

The readline `configure' recognizes a single `--with-PACKAGE' option:

`--with-curses'
This tells readline that it can find the termcap library functions
(tgetent, et al.) in the curses library, rather than a separate
termcap library. Readline uses the termcap functions, but does not
link with the termcap or curses library itself, allowing applications
which link with readline the to choose an appropriate library.
This option tells readline to link the example programs with the
curses library rather than libtermcap.

Than the SHLIB_LIBS

In the stanza for your operating system-compiler pair, you will need to
define several variables. They are:

SHLIB_LIBS Any additional libraries that shared libraries should be
linked against when they are created.

GazL · 10-04-2013, 08:44 AM

Quote:

Originally Posted by bartgymnast

I checked the readline Documentation.

And it appears slackware builds it wrong, or it is intended wrong:

The readline `configure' recognizes a single `--with-PACKAGE' option:

`--with-curses'
This tells readline that it can find the termcap library functions
(tgetent, et al.) in the curses library, rather than a separate
termcap library. Readline uses the termcap functions, but does not
link with the termcap or curses library itself, allowing applications
which link with readline the to choose an appropriate library.
This option tells readline to link the example programs with the
curses library rather than libtermcap.

Than the SHLIB_LIBS

In the stanza for your operating system-compiler pair, you will need to
define several variables. They are:

SHLIB_LIBS Any additional libraries that shared libraries should be
linked against when they are created.

Which is exactly what slackware currently does. so I don't see how slackware is doing it wrongly.

bartgymnast · 10-04-2013, 09:30 AM

Thats why I said, maybe it is intended.

readline basically links against either libtermcap (standard) or libncurses
now comes the funny part, libncurses provides the libtermcap API

so thats why most programs expect readline to be built against ncurses.

Debian started building against ncurses in 1997, RedHat built it with their rhel 6.
so I think its becoming standard that programs expect readline to be build against ncurses.

GazL · 10-04-2013, 10:58 AM

I'd rather leave it as upstream intended and deal with the occasional build breakage than just follow the crowd, but that's just me.

saulgoode · 10-04-2013, 11:00 AM

I don't think licensing terms permit ncurses to be distributed when linked with the GPLed readline.

bartgymnast · 10-04-2013, 12:23 PM

The license is not against linking readline with ncurses.

pcretest example: --enable-pcretest-libreadline

If this is done, when pcretest's input is from a terminal, it reads it using
the readline() function. This provides line-editing and history facilities.
Note that libreadline is GPL-licenced, so if you distribute a binary of
pcretest linked in this way, there may be licensing issues.

Setting this option causes the -lreadline option to be added to the pcretest
build. In many operating environments with a sytem-installed readline
library this is sufficient. However, in some environments (e.g. if an
unmodified distribution version of readline is in use), it may be necessary
to specify something like LIBS="-lncurses" as well. This is because, to quote
the readline INSTALL, "Readline uses the termcap functions, but does not link
with the termcap or curses library itself, allowing applications which link
with readline the to choose an appropriate library." If you get error
messages about missing functions tgetstr, tgetent, tputs, tgetflag, or tgoto,
this is the problem, and linking with the ncurses library should fix it.

what is not allowed basically is, that binary programs can not use the below statement.

Summary: readline needs -lcurses if ncurses is installed

if we supply readline with -lcurses it is not a problem.