Unable to build KDE Application Manuals search index
 

I'm getting an error whenever I try to create the seach index for the KDE Application Manuals in the KDE Help Utility. Because of this I can't search for anything in KDE help.

Here is the error I get.

INDEXDIR: /home/erik/.kde/share/apps/khelpcenter/index/
Creating index for 'kde_application_manuals'
FINDCMD: find /opt/kde/share/doc/HTML/en/ -name index.docbook
htdig failed

I checked and "htdig" is installed.

I also noticed that some of the index files are created in "/home/erik/.kde/share/apps/khelpcenter/index/".
In that directory I have a file called "kde_application_manuals.conf" with the following contents:

# htdig configuration for doc 'kde_application_manuals'
#
# This file has been automatically created by KHelpcenter
common_dir: /srv/www/htdig/common/
locale: en
database_dir: /home/erik/.kde/share/apps/khelpcenter/index/
database_base: ${database_dir}/kde_application_manuals
local_urls: help://=/opt/kde/share/doc/HTML/en/ file://=/
local_urls_only: true
limit_urls_to: file:// help:/
ignore_noindex: true
max_hop_count: 4
robotstxt_name: kdedig
compression_level: 6
template_map: Long long /opt/kde/share/apps/khelpcenter/searchhandlers/htdig/htdig_long.html
search_algorithm: exact:1 prefix:0.8
maximum_pages: 1
matches_per_page: 10
start_url: file:///home/erik/.kde/share/apps/khelpcenter/index//kde_application_manuals.tmp/index.html
external_parsers: text/docbook /home/erik/.kde/share/apps/khelpcenter/index//kde_application_manuals.tmp/docbookparser
valid_extensions: .docbook .html
mime_types: /home/erik/.kde/share/apps/khelpcenter/index//kde_application_manuals.tmp/htdig_mime

The interesting thing is I don't have a "/srv/www/htdig/common/" directory. I'm wondering if that's the problem, but how to I correct the problem?

There are a few other files created in the directory "/home/erik/.kde/share/apps/khelpcenter/index/kde_application_manuals.tmp/":

docbookparser
htdig.mime
index.html

If I look at "index.html" it appears to contain references to all of the "index.docbook" files that were found.

I would appreciate any suggestions. Since I'm new to KDE, not being able to search the help is a major handicap for me.

khelpcenter is part of the kdebase package..........you might re-install it using pkgtool and see if the problem goes away..........

I tried reinstalling the kdebase package, and also the htdig package, but it didn't solve the problem.

I also tried creating the directory " /srv/www/htdig/common" with read write permissions for everyone. That also did not correct the problem. I looked for a directory with a similar name (anything ending in "htdig/common") but found no directories like that.

Does anyone have some other ideas? Do any of you know a way for me to get more information about what error occurred. The "htdig failed" message doesn't tell me much.

Well, I've about given up on this one.

I tried completely reinstalling Slackware 10.2 from scratch, and ended up with exactly the same problem. Any time I try to create the seach index for the KDE documentation, I get the error "htdig failed".

I also tried creating a symbolic link for "/srv/www/htdig" to "/opt/www/htdig", which is actually the correct location for "htdig" on Slackware. I tried giving all users permission to write to the various directories under "htdig". I also tried to run "htdig" manually using the "-c" option to specify the configuration file that was left over from the partially completed indexing operation.

There is a bug report from someone else (Bug #112518) on the KDE site about the same problem. It's about a month old, and has a status of "Unconfirmed" and severity "Normal". It has no other information. Does that mean I am just going to have to live with the problem until someone takes a look at the bug?

I would appreciate suggestions or help from anyone.

For what it's worth, I can confirm the behavior. I just installed Slackware 10.2 yesterday and see exactly the same thing and tried the same fixes as you did to no avail. I guess the /srv/ reference is a bit of a red herring. I found where that was generated, and I changed it to point to /opt/ to no effect. (generated in /opt/kde/bin/khc_docbookdig.pl and also appears in a couple other scripts there) Let me know if you find anything!

Thanks, your post helped a lot, since you found the PERL script.
I'll take a look at the problem more when I have some time.
If I do find a solution, I'm make sure to post it here.

khelpcenter - build index
 

Hi. I've been searching and probing also with no luck. I have printed the files that I will list below and will look through them over the weekend.

/usr/share/apps/khelpcenter/searchhandlers/docbook.desktop
/usr/share/apps/khelpcenter/searchhandlers/htdig.desktop
/usr/share/apps/khelpcenter/searchhandlers/man.desktop

~/.kde/share/apps/khelpcenter/index/kde_application_manuals.tmp/docbookparser
~/.kde/share/apps/khelpcenter/index/kde_application_manuals.tmp/htdig_mime
~/.kde/share/apps/khelpcenter/index/kde_application_manuals.tmp/index.html (when i found it non-empty)

And obviously, I've glanced at the other files in that index folder.


/usr/bin/khc_docbookdig.pl
/usr/bin/khc_htdig.pl
/usr/bin/khc_htsearch.pl

/usr/share/apps/khelpcenter/khelpcenterui.rc

I think I found something in the rc file. I don't know if I can find where it leads, as I don't know exactly how to read XML code, though I have programmed in several languages. (I'm gonna learn quick though!!) On line 36 of my khelpcenterui.rc, I see:
<Action name="build_index" append="save_merge"/>
I assume this is where the help center instructs some other program/file to build the index, but I've tried to find that file with no luck. I forget the name of it now, but there was one file I found (I think .so file) that I suspect may hold something useful. I was unable to read the file because I don't have anything installed that will allow me to view/edit it.

I have customized htdig.conf and put it in my home directory to run:
htdig -c ~/htdig.conf
Really, it's just there so i can change the database directory from index to index5 (since I've tried several approaches.

I have installed htdig and all the docbook stuff. I even installed the newer htdig. I have apache running at startup. I have installed Knoppix 3.9 (May 2005 version) to my harddrive SEVERAL times. I am using khelpcenter version 3.4 (using KDE 3.4.0). Each time, I have this problem. I probably won't reinstall again, since that has not helped, even with changing the way I've done things.

I do wonder if you guys also find that the help for things you have recently installed (I used apt-get) do actually look and function as they should. For instance, I have help available for kcpuload but not for ksysv.

I really believe that I did get it working but it didn't stick after reboot. Maybe I'm just hopeful (or crazy - too many late nights and my eyes are badly strained), but I genuinely believe I got it to work but don't remember how - even if I did, it didn't stick so it's not exactly useful. But it does give me hope that we can fix this somehow. I've been looking everywhere for an answer and keep coming back to Erik_FL's postings.

Unfortunately, I may not be able to be online much this weekend (re: family), but I can still try stuff without internet and check daily for any insight you guys can provide. I'll let you know if I find anything at all.

I found another hint at the following link:

http://lists.kde.org/?l=kde-usabilit...6344302854&w=2

It's a long discussion of various things, but the last paragraph alludes to a problem that might be this one, and the comment appears to be from someone who ought to know.

This looks like a possibility.

http://mirrors.atn.ro/htdig/htdig-pa...mmandParsing.0

Patch to allow spaces to be separators for htdig command parsing.
Apparently it only accepts tab as a separator character.

Sounds like it's worth a try!

I got a little bit further, but I am still getting an error.

To get past the "htdig failed" error, I had to edit the file "/opt/kde/bin/khc_docbookdig.pl".

I changed the following lines.
The correct information is as follows.
After doing that I now get an error from "htmerge", which I have not been able to solve.

Here is the error.

The error doesn't make sense, since I do see the document database files, and they appear to have valid information.

Here is the "kde_application_manuals.conf" file being used by "htdig" and "htmerge".

I noticed that "limit_urls_to" has only one slash after "help". I tried changing that to "help://".
That had no effect.
I also tried getting rid of the double slash after "index//".
That had no effect.

I am still trying to get "htmerge" to work.
I fixed the "Warning: unknown locale!" problem by making this change in the file "/opt/kde/bin/khc_docbookdig.pl".

Old script.
New script.
I was hoping that would solve the "htmerge" problem, but it didn't.

Ubuntu and KDE 3.5 have the same problem
 

This is just a bit of an update about the problem.
I installed Ubuntu Linux and found that it has the same problem building the KDE Application help index.
I also compiled the KDE 3.5 release from sources, and had the same problem.
Someone did report this bug to the KDE developers, but I wonder if anyone has looked into the bug.

Slackware 11.0 still can't build help index
 

I just recently tested this with Slackware 11.0 and discovered that it is still a problem.
Also I tested with the version 2.6.17 kernel and had the same results.
If anyone has information on a way to correct this problem please post it.

KDE bug report.
 

You can find the KDE bugzilla entry here.

------- Additional Comment #5 From giuseppe 2007-01-14 09:02 -------
This bug is solved on slackware
you need to edit
2 files /opt/kde/bin/khc_htdig.pl
and /opt/kde/bin/khc_docbookdig.pl
and edit the variables,should be like this

my $htdigdata = "/opt/www/htdig/common/";
my $htdigbin = "/opt/www/htdig/bin";
my $kdeprefix = "/opt/kde";


------- Additional Comment #6 From giuseppe 2007-01-19 20:17 -------
I forgot..
and edit the file /opt/kde/bin/khc_docbookdig.pl

my $htsearchpath="/opt/www/htdig/cgi-bin/htsearch";

Htdig MUST BE the latest 3.20b6

Thanks for the information. I tried that and I get the "htmerge failed" error. That's basically as far as I got on my own. It also appears that "Comment #6" should refer to the file "khc_htsearch.pl" instead of "khc_docbookdig.pl" since the variable referred to only exists in "hkc_htsearch.pl". I tried adding or changing it in both files with the same "htmerge failed" error. It seems the whole kde help indexing is dependant on the distro. I guess I'm going to have to wait until Slackware corrects the problem.

A few people have fixed problems with building the KDE help index, but none of them seem to "fix" all the problems in the Slackware distro. I can't easily use another distro because none of them will install on my RAID controller (or at least I can't figure out how to do that).

I got it to work using the method I posted.

In /opt/kde/bin/khc_htdig.pl I have the following variables:

my $htdigdata = "/srv/www/htdig/common/";
my $htdigbin = "/usr/bin";
my $kdeprefix = "/opt/kde";

In /opt/kde/bin/khc_docbookdig.pl I have the following variables:

my $htdigdata = "/srv/www/htdig/common/";
my $htdigbin = "/usr/bin";
my $kdeprefix = "/opt/kde";

In /opt/kde/bin/hkc_htsearch.pl I have the following variables:

my $htsearchpath="/opt/www/cgi-bin/htsearch";

I have to change $htdigbin to /usr/bin because that is where I installed the htdig binary file when I compiled the latest htdig sources.

Hope this helps you.

Regards,

Greg

Thanks Greg. Your suggestion helped me get a bit further. As it turns out, when I used "make install" for htdig it copied the binary files to "/opt/www/bin" instead of "/opt/www/htdig/bin"

I now get an indication of "success" building the index, but the search index is apparently empty. Attempts to search for anything obviously in the application manuals return no results.

There is still an error reported in the details.

I can't find the variable my $htsearchpath in that file. Slack 11 here. Any ideas?

IMPORTANT! There are two errors in the information that has been posted previously.
The file name and the direcotry name for "$htsearchpath" were both incorrect.

The file "/opt/kde/bin/khc_htsearch.pl" is where this change should be made.

The above search path is different than the ones previously posted.
If "$htsearchpath" is incorrect any attempts to search the help index will yeild no results (apparently reporting no error).

I had to make two more changes to resolve all the errors.

Rename the following directory.

/opt/kde/share/doc/HTML/en/KRegExpEditor

To this

/opt/kde/share/doc/HTML/en/kregexpeditor

The error that I got was because the file name matching is case sensitive.

I also had to add this at line 80 of /opt/kde/bin/khc_docbookdig.pl
The entire section of code around line 80 now looks like this.

After all of these changes I am now able to successfully search the help index and obtain results. Thanks to gbowden for providing the information to get me looking in the correct places.

Complete solution
 

To avoid confusion, here are ALL the steps that I did to correct the problems with the KDE help index.

First, it was necessary to compile and install the most recent "htdig" source.
I downloaded "htdig-3.2.0b6.tar.gz" and extracted the files to "/usr/src/htdig-3.2.0b6".
Although this step might not be necessary, it does affect the rest of the instructions for correcting the problem.
If this step is NOT done then substitute "/opt/www/htdig/bin" instead of "/opt/www/bin" in subsequent instructions.
I have only tested with the latest "htdig" so I highy recommend using that.

To compile and install "htdig" use these commands

Rename the following directory.

/opt/kde/share/doc/HTML/en/KRegExpEditor

To this

/opt/kde/share/doc/HTML/en/kregexpeditor

That corrects an error due to case sensitive file names.

Edit the following files as indicated.

File "/opt/kde/bin/khc_docbookdig.pl" change these variables to the values indicated.

File "/opt/kde/bin/khc_docbookdig.pl" change this (around line 79).

to this

File "/opt/kde/bin/khc_htdig.pl" change these variables to the values indicated.

File "/opt/kde/bin/khc_htdig.pl" change this variable to the value indicated.

NOTE: The above change isn't required to build the indexes, but is required for search to find any results when searching the indexes.

The "correct" location of the "htdig" binary files is not clear to me.
The "htdig" that comes with Slackware is located in "/opt/www/htdig/bin".
When I recompile and install "htdig" it is installed to "/opt/www/bin".
In any case, the variable "$htdigbin" in the scripts must specify the correct location.

I am not quite sure what to make of the "locale" problem. It may be due to some other problem and my "fix" may just be working around the other problem.

The other variables "$htdigdata" and "$kdeprefix" are dependent on the installation and distro in question.
These instructions should work for Slackware 11.0 but will probably not work with some other distros.
Substituting the correct directory names may work around the problem in other distros.