LinuxQuestions.org
Page 1 of 2 1 2
Show 50 post(s) from this thread on one page

LinuxQuestions.org (/questions/)
-   Slackware (https://www.linuxquestions.org/questions/slackware-14/)
-   -   Useful custom commands for Slackware (https://www.linuxquestions.org/questions/slackware-14/useful-custom-commands-for-slackware-174647/)

thegeekster 04-26-2004 06:08 AM
Useful custom commands for Slackware
 
Here are three commands I use which I find quite useful. They all deal with finding info on the Slackware packages currently installed on the computer and are a great addition to the other package commands in Slackware (such as installpkg, removepkg, upgradepkg, etc.). These are my own homegrown commands, not copied from anywhere, which I give freely to the public domain without restriction. They are script files which can be placed in /usr/local/bin and used like any other command.

Each has a short help message to help with usage, although I will provide a quick rundown on their usage. They require the utils 'less' (less pkg) and 'egrep' (grep pkg), which may or may not be installed by default.

PATTERN MATCHING <- This section has been edited. ->
All the scripts now emulate the wildcard pattern matching ( * and ? ) used for filename matches. These scripts do not use true globbing, as it is sometimes called, but merely substitutes the wildcards * and ? with a regular expression equivalent, which is only part of the picture (see 'man glob' for details for the rest of the story on wildcard pattern matching). This is sufficient for what is needed for these scripts. Using more complicated searching, such as with the square brackets ( [ ] ), curly braces ( { } ), and others used in globbing, will not work as expected as the rules are different than how they are applied in regular expressions (used by the 'egrep' command). I suppose I could do more substitution to make it closer to globbing, but I really don't see the need to do so at this time. NOTE: If your familiar with the extended regexp syntax, you could probably do more complicated searches with these scripts, if you remember to take into account the substition of * for .*, ? for .?, and . for \. ............


NOTE: Any changes or updates in code will be reflected here and explained in a following post below...........In other words, the code for these three scripts will always have the most recent changes in this post with an explanation in another post below.

lspkg
This command simply lists the packages currently installed on your machine.

UPDATE: See post #53 below for more info.
Code:
#!/bin/sh
#*******************************************************************************
# Name: lspkg
#
# A Slackware tool which simply lists the currently installed packages.
#
# Originally written by CTWaley (a.k.a., thegeekster) and originally posted in
# the Slackware forum at LinuxQuestions.org,
# <http://www.linuxquestions.org/questions/showthread.php?postid=899459#post899459>
#
# This program is given to the Public Domain and is free software. You can
# redistribute it and/or modify it without restriction of any kind.
#
# It distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
# without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
# PARTICULAR PURPOSE.
#*******************************************************************************

case $1 in
  -h | --help )  # If -h (--help) is given as an argument, show the usage.
    cat << __EOF__

Usage: lspkg [<package_name> ... ]

Lists the Slackware packages currently installed. If no argument is
given, it will output all packages currently installed. If an argument
is given on the command line, it will look for the package name matching
the pattern(s) supplied. More than one package name or pattern may be
given on the command line.

Use the wildcards * and ? when looking for a package name containing the
characters supplied on the command line. For example, to find a package
containing the word "lib", but does not necessarily begin with "lib",
use one of the patterns "*lib", "?lib", "??lib" or even "??lib*". To see
if you have both "bzip" and "bzip2" installed on your system, you can
use the search term "bzip?".  You may also enclose the search pattern in
single quotes (') or double quotes (") to avoid the shell interpeter from
interfering with the search term when using the wildcard characters.

__EOF__
    ;;
  * )
    [ -z $1 ] && 'ls' /var/log/packages/ | less \
    || {
      while [ ! -z $1 ]
      do
        pkg=`echo $1 | sed 's/\./\\./g; s/\*/.*/; s/\?/.?/g'`
        'ls' /var/log/packages/ | egrep -i "^$pkg\>"
        shift
      done | more
    }
esac

pkginfo
This command lists the package's name, size of the package, the package description, and a list of the files installed by the package.

UPDATE: See post #53 below for more info.
Code:
#!/bin/sh
#*******************************************************************************
# Name: pkginfo
#
# A Slackware tool to find the inforamtion about any installed package.
#
# Originally written by CTWaley (a.k.a., thegeekster) and originally posted in
# the Slackware forum at LinuxQuestions.org,
# <http://www.linuxquestions.org/questions/showthread.php?postid=899459#post899459>
#
# This program is given to the Public Domain and is free software. You can
# redistribute it and/or modify it without restriction of any kind.
#
# It distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
# without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
# PARTICULAR PURPOSE.
#*******************************************************************************

count_args() {
  args=$#
}

present_list() {
  cat << __EOF1__ | more

-------------------------------------------------------------------------------
 More than one package was found matching the name, "$ARG".
 Choose a package from the following list (Hint: Use the mouse to copy-n-paste
 the desired package name)...
-------------------------------------------------------------------------------

`echo "$PKGS" | egrep -i "\<$1\>" | while read i ; do basename $i ; done`

-------------------------------------------------------------------------------
__EOF1__

  echo -n " Enter package name here: " ; read pkg
  count_args `echo "$PKGS" | egrep -i "\<$pkg\>"`
  { [ $args -gt 1 ] && present_list $pkg
  } || {
    [ $args -lt 1 ] \
      && echo "No package was found matching the name, \"${pkg}\"." \
      && exit 1
  }
}

view_info() {
  cat << __EOF2__ | less
`cat \`echo "$PKGS" | egrep -i "\<$1\>"\``

`echo "INSTALLATION SCRIPT:"`
`cat \`echo "$SCRIPTS" | egrep -i "\<$1\>"\``
__EOF2__
}

case $1 in
  # If no argument or -h (--help) is given as an argument, show the help message.
  "" | "-h" | "--help" )
    cat << "__EOF3__"

Usage: pkginfo <package_name>

Lists the contents of the requested package. You can give any part of
the package name (such as bash, bash-2.05b, 2.05b, bash-2.05b-i486,
etc.). The search is case insensitive (upper- or lowercase is okay).
You may use the wildcards "*" or "?" for pattern matching. If more
than one match is found, you will be presented with a list of package
names to choose from (like when searching for the package xfree86).

TIP: If you know the first few letters of the package you are looking
    for, add the asterisk (*) at the end of the search term, such as
    "lib*". Or to find packages which end with the search term entered
    on the command line, add the "*" wildcard in front of the term
    (ie., "*lib"). You will then be presented with a list of package
    names to choose from. You may also enclose the search pattern in
    single quotes (' ') or double quotes (" ") to avoid the shell
    interpeter from interfering with the search term while using the
    wildcard characters.

__EOF3__
    ;;
  * )
    count=0
    ARG="$1"
    F="`echo \"$1\" | sed 's/\./\\./g; s/\*/.*/; s/\?/.?/g'`"
    PKGS=`find /var/log/packages/* -type f -prune`
    SCRIPTS=`find /var/log/scripts/* -type f -prune`
    count_args `echo "$PKGS" | egrep -i "\<$F\>"`
    case $args in
      0 )
        echo "No package was found matching the name, \"$ARG\"."        ;;
      1 )
        [ -z `echo "$SCRIPTS" | egrep -i "\<$F"` ] \
          && cat "`echo \"$PKGS\" | egrep -i \"\<$F\"`" | less \
          || view_info "$F"
        ;;
      * )
        present_list $F
        [ -z `echo "$SCRIPTS" | egrep -i "\<$pkg\>"` ] \
          && cat "`echo \"$PKGS\" | egrep -i \"\<$pkg\>\"`" | less \
          || view_info "$pkg"
        ;;
    esac
esac

whichpkg
Ever wonder which package a file came from? That's the purpose of this command, to find the package a file comes from.

UPDATE: See post #53 below for more info.
Code:
#!/bin/sh
#*******************************************************************************
# Name: whichpkg
#
# A Slackware tool to find which package a file comes from.
#
# Originally written by CTWaley (a.k.a., thegeekster) and originally posted in
# the Slackware forum at LinuxQuestions.org,
# <http://www.linuxquestions.org/questions/showthread.php?postid=899459#post899459>
#
# This program is given to the Public Domain and is free software. You can
# redistribute it and/or modify it without restriction of any kind.
#
# It distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
# without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
# PARTICULAR PURPOSE.
#*******************************************************************************

search_pkgs() {
  name1="/${1}\>"

  find /var/log/packages/* -type f -prune | while read file
  do
    # This makes certain assumptions about where to look, such as
    # ignoring certain directories and files which are easy to figure
    # out where the files came from:
    cat $file | egrep -A5000 'FILE\ LIST:' | egrep -i "${name1}\$"\
      | egrep -v '(FILE\ LIST:|^\./$|\<install/|\<install-sh|\<install\.sh)' \
      | egrep -v '(usr/src/linux|doc/|share/$1)' \
      | while read line
    do
      [ ! -d $file ] && echo "`basename $file`:/$line"
    done
  done

  # Due to a problem of a broken pipe when trying to find any symlinks
  # associated with the file "ln", I had to find a workaround.
  name2="\<${1}\>"
  [ "$1" = "ln" ] && name2="\<ln\>\ [^-rf]"
  [ "$1" = "rm" ] && name2="\<rm\>\ [^-rf]"

  find /var/log/scripts/* -type f -prune | while read file
  do
    cat $file | egrep -i "$name2" \
      | egrep -v '(^#|:| \<rm\ -r\>|\<if\ |\<else\ |>>|\$|\<for\ .*\ in\>|doinst\.sh)' \
      | while read line
    do
      [ ! -z "`echo $line | egrep '\<ln\ -sf' |egrep '\(\ cd'`" ] \
        && echo "`basename $file`:/`echo $line \
        | awk '{ print $3\"/\"$8\"  --linked-to->  \"$7 }'`"
      [ ! -z "`echo $line | egrep '\<mv\ '`" ] \
        && echo "`basename $file`:/`echo $line | sed 's/^.*\(\<mv\ \)/\1/' \
        | awk '{ print $2\"  --renamed-to->  \"$3 }'`"
    done
  done
}

case $1 in
  "" | -h | --help )  # If no argument is given, or -h (--help) is given,
                      # as an argument then show the usage.
    cat << "__EOF__"  | less

Usage: whichpkg FILE1 [ FILE2 ... ]

Finds which Slackware package contains the file supplied on the command
line. A filename must be supplied and more than one filenam may be used
on the command line. For pattern matching, you may use the wildcards *
and ?. For example, to find the files "bzip" and "bzip2", you may use
the pattern "bzip?" to look for both files. The search is case
insensitive (upper- or lowercase is okay). You may also enclose the
search pattern in single quotes (') or double quotes (") to avoid the
shell interpeter from interfering with the search term when using the
wildcard characters.

The output was made to be as easy as possible to read and the output
for each search term is easily identifiable. Each matching line will
look something link this:

    <package_name>:<matched_path>

For symlink matches, the output will look something like this:

    <package_name>:<matched_symlink>  -- linked to->  <real_file>

If a file was renamed when installed, the output will look something
like this:

    <package_name>:<old_filename>  -- renamed to->  <new_filename>

To give you an idea, here's a sample output for the search for the
multiple terms "sh" and "bash":

$ whichpkg sh bash
------------------------------------------------------------------------
Searching for "sh"...

aaa_base-9.1.0-noarch-1:/bin/sh  --linked to->  bash

------------------------------------------------------------------------
------------------------------------------------------------------------
Searching for "bash"...

aaa_base-9.1.0-noarch-1:/bin/sh  --linked to->  bash
bash-2.05b-i486-3:/bin/bash  --renamed to->  bin/bash.old
bash-2.05b-i486-3:/bin/bash2.new  --renamed to->  bin/bash
bash-2.05b-i486-3:/usr/bin/bash  --linked to->  /bin/bash

------------------------------------------------------------------------


NOTE: The searches will take a couple of moments for each term being
      searched and some searches may take longer than others, depending
      on the keyword used. Usually it depends on how many occurrences of
      the keyword is found, which then must be analyzed and filtered.

__EOF__
    ;;
  * )
    while [ ! -z "$1" ]
    do
      i="`basename \`echo \"$1\" | sed 's/\./\\./g; s/\*/.*/; s/\?/.?/g'\``"
      echo "------------------------------------------------------------------------"
      echo -e "Searching for \"${1}\"...\n"
      search_pkgs "$i" | sort
      echo -e "\n------------------------------------------------------------------------"
      shift
      done        ;;
esac

I hope you find these as useful as I have. While you can get the same information using a graphical package manager, they are much quicker to use from the command line.

Enjoy :)
---thegeekster

Ninja Cow 04-26-2004 07:24 AM
You rock, sir.

I especially like lspkg. Nice and handy. Thank you very, very much.

thegeekster 04-26-2004 02:48 PM
Thanx :)

But I did notice one little item in the code for whichpkg...............I forgot to add the -i switch in the grep command. It's this switch that makes the search case insensitive...............................LOL...which means I've been using it all this time as a case sensitive search...... :eek:

I've already made the changes in the code above..............................but for those who may have copied it already, at the bottom of the code for 'whichpkg' change the two grep lines to read "grep -i ..." (simply insert -i after the word grep)........

thegeekster 05-17-2004 07:18 AM
*** lspkg UPDATE ***
 
NOTE: This update is superseded by the changes posted here..... :)

the who 05-17-2004 09:02 AM
Thanks, been using these scripts since you first posted them. :)

slakmagik 05-17-2004 01:02 PM
I don't think I've ever posted one of my scripts before. I figure mine suck. As far as this, I've just been using this thing I wrote called 'sp' (Slack packages).
Code:
script=`basename $0`
pkgdir=/var/log/packages/*
all=/home/j/var/manifest

usage () { printf "$script: Usage: $script OPTION SEARCHTERM
-a Search the entire manifest
-f Search for a filename within installed packages only
-s Search for a string within installed packages only
-v View the description file of a filename search (like -f)
"
}

if [ $# -ne 2 ]; then
        usage
        exit
fi

case $1 in
        -a) less +/$2 $all
                ;;
        -f) grep -i \/$2$ $pkgdir | less
                ;;
        -s) grep -iR $2 $pkgdir | less
                ;;
        -v) less `grep -i \/$2$ $pkgdir | cut -d: -f 1`
                ;;
        *) usage
                ;;
esac
Mine's real simple-like and lame, I guess. But it does interrogate 'manifest' in a half-assed way which will help you know if Slack provides something at all, whether you installed it or not, so I post it up as something somebody might want to improve and use. I'm sure you could create an actually useful script for MANIFEST.bz2 (I uncompressed it and and copied it to a sysjunk directory) to add to the suite.

But how every Slack user doesn't come up with *something*, I dunno. It's too common an activity and pkgtool is just too slow. ;) Good stuff, thegeekster.

Oh, and my less is 'export LESS="-eFMRXj12"' - using the default less the way I do might be kind of annoying.

Oh, and
alias cdp='cd /var/log/packages'
is essential for manual inspection. Due to filenames, typing 'cd /v<tab>log/p<tab>' sucks.

thegeekster 05-17-2004 07:03 PM
Hey, digiot, don't sell yourself short..................this is what open source is about, posting what you have for the benefit of others and if someone knows of a better or more efficient way of doing it, or spots a bug, then it gets fixed...........in other words, community involvement...............I'm sure mine can be improved upon, too.........

This is also why I post these scripts, making it useful for others and hoping others will come forward and help contribute their bit to the community.............and this allows users to have more choices, instead of limiting choices........... ;)

I see your code is basically what I have, but all in one program, with one added ability..................I thought of making it all one script, but then I decided to follow the Unix programming philosophy of making a program do just one thing, and do it well.... :)

I must say, I never thought of using the manifest file for searching if a file is part of the distribution as a whole, whether the pkg is installed or not...................gives me an idea of updating the 'whichpkg' script to first search locally for installed programs, and if not found, then search the manifest to see if it's even part of the Slackware distribution..................otherwise, it must be a third-party app and not officially part of Slackware.......... :)

Also, you might be interested in knowing that you can 'cat' a zipped file (reading the file to the screen), whether compressed with gzip or bzip2, since they only compress a single file. The regular zip utility doesn't do this AFAIK, probably due to the fact it is not limited to compressing only a single file...................In other words, instead of unzipping the manifest file, just 'cat' the file in it's compressed form...............this way, you can save some disk space and still be able to read the file and pipe it through grep to filter the output

The commands for these formats is zcat filename.gz for gzipped files and bzcat filename.bz2 for bzip2 files.......this will output the contents of the compressed file to the screen........so for your 'all' variable and '-a' switch you can do something like this:
Code:
all=/home/j/var/MANIFEST.bz2

...

 <EDIT>  -a) bzcat $all | grep -i $2  ;; </EDIT>
You get the idea.......... :)

And anyone else.................feel free to step in and post someting you may have come up with..............if you know of a useful timesaver, then post it.... :)

slakmagik 05-17-2004 11:06 PM
Quote:
Originally posted by thegeekster
Hey, digiot, don't sell yourself short..................this is what open source is about, posting what you have for the benefit of others and if someone knows of a better or more efficient way of doing it, or spots a bug, then it gets fixed...........in other words, community involvement...............I'm sure mine can be improved upon, too.........

Thanks - yeah, I agree - I wish more people would toss shell scripts back and forth but I guess I'm part of the problem there. ;)

Also, you might be interested in knowing that you can 'cat' a zipped file...
Yeah, but I'd still have to less it to position the file on the search argument and less gets lost when it's reading from stdin - reading a file off disk I can get

/home/j/var/manifest lines 21792-21811/217164 8%

instead of

lines 1-22

(or whatever) in the status line which I figure might be useful for something. And with disk sizes these days I don't compress much.

I second that, though - I look forward to more folks' posts. :)

(OT - anyone else feel claustrophobic. ;) )

thegeekster 05-18-2004 02:05 AM
Quote:
Originally posted by digiot
Yeah, but I'd still have to less it to position the file on the search argument and less gets lost when it's reading from stdin - reading a file off disk I can get...
I just discovered that less will read a compressed bz2 file just as easily as if it was not compressed:
Code:
all=/home/j/var/MANIFEST.bz2

...

    -a) less +/$2 $all
Try it and see......

And I'm not that familiar with the 'less' command, other than piping something to less.................so I learned something new and gives me something to play around with...... :)

slakmagik 05-18-2004 02:41 AM
:o

Yeah. lesspipe.sh. I forgot that. :( Well, if you're not familiar with less, check out the man page - it's a great tool that can be a lot more effective than I think a lot of people realize. I obviously need to go read it again myself ;)

(Still like 'em uncompressed, though.)

thegeekster 06-06-2004 12:31 AM
*** chngcd ***
 
Here's a handy little script for changing CDs easily..................This is especially useful for you gamers whom oftentimes have to change a CD during gameplay........

To use, it's quite simple - open a console terminal and enter chngcd..........The CDROM tray will pop out automatically..........Then place a CD in the tray -OR- replace the current CD to continue -OR- remove the CD to quit, and then press Enter................You don't even have to close the CD tray, it will do it automaticallly................Simple and effective... ;)

To make it easy to create the script, just copy-n-paste _all_ of the following code at once into a console terminal, and the script will be created automatically, not to mention being executable as well.......... :)

If you prefer to do it the harder way, such as using a text editor, then copy-n-paste everything _except_ the first and last lines into the text editor...........Do not copy the lines containing __EOF__

NOTE: If you need to change the path to the CDROM mount point for any reason, after creating the script just open it up and change the CDROM variable to the desired path..........

Code:
cat << "__EOF__" > /usr/local/bin/chngcd && chmod 755 /usr/local/bin/chngcd
#!/bin/sh
#*******************************************************************************
# Name: chngcd

# Mount point for CDROM device
CDROM=/mnt/cdrom

umount -l $CDROM 2> /dev/null
eject $CDROM

echo -en "\n After replacing or removing the CD, simply press [Enter]..." \
  && read ans && echo
mount $CDROM 2> /dev/null
__EOF__
Oh yeah, this is a script I made up myself and didn't copy it from anyone. This script is also donated to the public domain, and free to use without restriction........ :)

gnashley 06-06-2004 02:47 AM
Since a few bash scripters are gathered here, I'll ask here.
How can parse root=/dev/????? out of /proc/cmdline?

thegeekster 06-06-2004 03:11 AM
Quote:
Originally posted by gnashley
...How can parse root=/dev/????? out of /proc/cmdline?
Try this:
Code:
cat /proc/cmdline | grep 'root=' | cut -d' ' -f3
:)

gnashley 06-06-2004 04:21 AM
It has to be more explicit to handle any cmdline order.
I've been greping the small 't' and then '='. This works as long as there is no other small t, for instance in the kernel name.
I thought sed might help but I don't know how to use it.
using grep -d '=' also gets confused.
can sed use a multi character delimiter like 'root=' ?

thegeekster 06-06-2004 04:38 AM
Hmmm...........then see if this is what you want:
Quote:
for i in `cat /proc/cmdline` ; do if [ ! -z `echo $i | grep 'root='` ] ; then echo $i ; fi ;done
:)

gnashley 06-06-2004 05:46 AM
Yeah great! i just add |cut -f2 -d '=' to that and it gets me what I want.
Does -z do like -v except it eliminates all FIELDS that don't match, or ??

thegeekster 06-06-2004 06:02 AM
-z is for testing empty strings.................in this case, the test is for NOT an empty string [ ! -z ... ], and by using the 'for' statement, it breaks up the input from the 'cat' command whenever 'for' parses any whitespace (blank spaces, tabs) or newlines, thereby letting you test each space-delimited field separately.......... :)

ivanatora 06-06-2004 07:14 AM
Re: *** chngcd ***
 
Quote:
Originally posted by thegeekster
Code:
cat << "__EOF__" > /usr/local/bin/chngcd && chmod 755 /usr/local/bin/chngcd
#!/bin/sh
#*******************************************************************************
# Name: chngcd

# Mount point for CDROM device
CDROM=/mnt/cdrom

umount -l $CDROM 2> /dev/null
eject $CDROM

echo -en "\n After replacing or removing the CD, simply press [Enter]..." \
  && read ans && echo
mount $CDROM 2> /dev/null
__EOF__
It doesn't work because the mount, umount and eject commands are available only for the root user. I prefer sudoing it:
/usr/local/bin/cdrom:
### beginning ###
case "$1" in
'mount')
sudo /sbin/mount /mnt/cdrom/ ;;
'umount')
sudo /sbin/umount /mnt/cdrom/ ;;
*)
echo "Usage $0 mount|umount" ;;
esac
### end ###
And following in my
/etc/sudoers
##begin##
ivanatora ALL=NOPASSWD: /sbin/mount /mnt/cdrom/,/sbin/umount /mnt/cdrom/
##end##
Of course 'ivanatora' is my username, so if you want this to work you should change this. It can be easily modified to eject the cdrom, but I can't see the point.

thegeekster 06-06-2004 07:39 AM
Re: Re: *** chngcd ***
 
Quote:
Originally posted by ivanatora
It doesn't work because the mount, umount and eject commands are available only for the root user...
Not on my machine...............those commands are in bin directories, not sbin directories................if you have problems, then it's probably a matter of how you're permissions are set for the CDROM device.............usually, the CDROM is set for anyone to mount and unmount..............otherwise CDROMs wouldn't be very useful for users.................. :)

thegeekster 06-06-2004 09:07 AM
If anyone is having problems, there are two ways to overcome it....................either add the user to the same group as the block device ("disk" for Slackware), or change the permissions on the block device to make it readable for "others", ie., brw-r--r--

If you add the user to the same group as the block device for the CDROM, you might have to logout completely or reboot the machine............... :)

usercsr 06-06-2004 11:05 AM
thanks guys, those are really useful scripts.

akshunj 06-06-2004 01:02 PM
Is there a page anywhere out there archiving all the useful bash scripts???

--Akshun J

thegeekster 06-06-2004 04:26 PM
Quote:
Originally posted by akshunj
Is there a page anywhere out there archiving all the useful bash scripts???

--Akshun J
By "out there", if you mean on the web, I started this thread a while ago with this mind:

Shell scripts on the web

:)

akshunj 06-06-2004 05:27 PM
Nice. That will get my feet wet. Teaches me to come in at the end of a thread...

--Akshun J

Neoslak 06-06-2004 06:33 PM
Here is a useful, if totally simplistic script, that is handy to see what Slackware packages in a given directory are *not* installed. For instance, I keep a local mirror of Slackware-current and I want to know if there are any packages in my slackware-current/slackware/gnome directory that are not already installed. I put a copy of this script in /usr/local/bin and change to /home/ftp/pub/slackware-current/slackware/gnome and run packcheck.sh. If anybody wants to improve on this, post your improvements here.

--------------------
#!/bin/sh

for pack in $(ls *.tgz); do
pack=$(basename $pack .tgz)
if [ ! -f /var/log/packages/$pack ]; then
echo "No Package $pack found"
fi
done
------------------

Chuck

thegeekster 06-14-2004 01:56 PM
*** whichpkg UPDATE ***
 
NOTE: This update is superseded by the changes posted here..... :)

cassiusclay 06-14-2004 05:36 PM
moving my post

Bebo 06-15-2004 12:30 PM
I just wanted to say a big THANKS to you geekster! I've been using your scripts a bit and find them very useful :)

thegeekster 07-07-2004 04:31 AM
*** FINAL UPDATES for lspkg, whichpkg, and pkginfo ***
 
Okay, I think I've gone about as far as I can with these three scripts, lspkg, whichpkg, and pkginfo................They are all converted over from using regular expression syntax to wildcard matching (globbing). While not true globbing, they are close enough for practical purposes here...........If you're interested knowing what the difference is, read the manpages for globbing and regular expressions, 'man 7 glob' and 'man 7 regexp'. (Note - For KDE users, you can view the manpages in a html format by entering these urls in Konqueror, 'man:/glob(7)' and 'man:/regexp(7)'. This only works in Konqueror, not any other browser in KDE.).......Be sure to include the number 7 in both............

I did a major overhaul for the whichpkg script, making it easier to read the output, filtering out more of the unwanted garbage, and allowing for multiple search terms. It ended up being a bit of a challenge, more than I had anticipated....... :)..........but I think I've got it right (it sure did sharpen my regexp skills.......LOL).............A few search terms were quite unpredictable, most notably the searches for "ln", "file", "install" and "sh"...........If there are others you come across which seem not quite right, be sure to post them here and I'll fix it. otherwise this is pretty much the final update.

The other two were mostly to convert to using wildcard searching (pkginfo) or a minor change in the usage of the wildcards in the search term (lspkg)..........Again, I don't see where they can go much farther, so these will also be the final updates.

HOWEVER................I'm going to create another Slackware specific script to deal with orphaned files by looking at all the system files and matching them against the list of files found in the /var/log/packages/ and /var/log/scripts/ directories..............Then you will be presented with a report of all the files that aren't accounted for from the installed packages..............Some of these may be from various scripts and files you may have added manually, such as these scripts, but some may be leftovers from old installations no longer around.................Either way, you will have a list to look over and decide for yourself whether they belong there or not..............The reason for this script is from some of the posts I've seen recently asking about what files are safe to remove, or a way to determine which files are not wanted or needed...........I do love a challenge............. :D

Okay, enought talk, the changes have already been made in the top post (post #1) above....



EDIT: Changed line number 53 in "pkginfo" script by removing the path "/var/log/packages/" (The third line from the end, above the two "esac" lines). Thanks goes to carboncopy for pointing out this error. :)

EDIT2: More changes needed in the "pkginfo" script as noted below.

carboncopy 07-07-2004 04:53 AM
Hi! I tested out all three new one but pkginfo seems to be not working properly for me. This is what happens:

Code:
bash-2.05b# ./pkginfo gnome

-------------------------------------------------------------------------------
 More than one package was found matching the name, "gnome".
 Choose a package from the following list (Hint: Use the mouse to copy-n-paste
 the desired package name)...
-------------------------------------------------------------------------------

/var/log/packages/gnome-applets-2.6.1-i486-1
/var/log/packages/gnome-audio-2.0.0-noarch-1
/var/log/packages/gnome-desktop-2.6.2-i486-1
/var/log/packages/gnome-games-2.6.1-i486-3
/var/log/packages/gnome-icon-theme-1.2.3-noarch-1
/var/log/packages/gnome-keyring-0.2.1-i486-1
/var/log/packages/gnome-media-2.6.2-i486-1
/var/log/packages/gnome-mime-data-2.4.1-noarch-1
/var/log/packages/gnome-netstatus-2.6.1-i486-1
/var/log/packages/gnome-panel-2.6.2-i486-1
/var/log/packages/gnome-session-2.6.2-i486-1
/var/log/packages/gnome-speech-0.3.3-i486-1
/var/log/packages/gnome-system-monitor-2.6.0-i486-1
/var/log/packages/gnome-terminal-2.6.1-i486-1
/var/log/packages/gnome-themes-2.6.2-i486-1
/var/log/packages/gnome-themes-extras-0.7-i486-1
/var/log/packages/gnome-utils-2.6.2-i486-1
/var/log/packages/gnome-vfs-2.6.1.1-i486-2
/var/log/packages/libgail-gnome-1.0.4-i486-1

-------------------------------------------------------------------------------
 Enter package name here: gnome-audio
cat: /var/log/packages//var/log/packages/gnome-audio-2.0.0-noarch-1: No such file or directory

gbonvehi 07-07-2004 05:09 AM
Try changing the line:
cat /var/log/packages/$( echo "$list" | grep -i "$pkg" ) | less ;;

to

cat /var/log/packages$( echo "$list" | grep -i "$pkg" ) | less ;;

thegeekster 07-07-2004 01:45 PM
carboncopy

Thanx for pointing out this error. gbonvehi has the right line that needs changing, which is line number 53 (the third from the last line, above the last two "esac" lines).................However, to correct it simply remove the path "/var/log/packages/" that follows the word "cat"...........it should read:

cat $( echo "$list" | grep -i "$pkg" ) | less ;;

OR you can use:

cat `echo "$list" | grep -i "$pkg"` | less ;;

NOTE: In bash scripting, both lines above are identical. The use of backquotes (not to be confused with the apostrophe) has the same meaning as using parenthesis with the dollar symbol in front of it.......I've changed my style to use the backquotes, ` ` , instead of the dollar sign/parentheses construction, $( ), because it's easier for me to type (On US keyboards, the backquote key is the one above the Tab key, to the left of the number 1 key)....... :D

thegeekster 07-07-2004 08:21 PM
Okay, let's try this again.............for the "pkginfo" script, there needs to be another change.............I know what happened.........I changed the lookup from 'ls /var/log/packages/*' to "find /var/log./packages/* -prune -type f" which changed the output to include the full path....................

So _all_ the necessary change will be as follows (changes made according to the line number):

Line 37
Code:
Change:
          cat /var/log/packages/$list | less        ;;

to this:
          cat $list | less        ;;
Line 47
Code:
Change:
`echo -e "$list\n" | more`

to this:
`echo -e "$list\n" | while read i ; do basename $i ; done | more`
Line 53 (this is the change I mentioned above)
Code:
Change:
        cat /var/log/packages/$( echo "$list" | grep -i "$pkg" ) | less ;;
 
to this:
        cat `echo "$list" | grep -i "$pkg"` | less ;;
NOTE: i've already made the appropriate changes in the top post above......Sorry about this, was in too much of a hurry.... :)

carboncopy 07-07-2004 10:08 PM
Thanks for the Great Code. save a tonne of time (pkgtool --> view pkg).

Bebo 07-08-2004 03:05 AM
OK, I haven't tried your new scripts yet (I will after finishing this post though :)), but I just wanted to say that if you can make the orphan script work I'd be very interested to see and use it. I tried to write exactly that kind of script but I must have given it too little thought 'cause it just was far too slow and memory consuming. In my script I also looked for "runaways", i.e. files belonging to an installed package that just weren't on the computer - perhaps something for your script too?

thegeekster 07-08-2004 04:22 AM
Bebo

It will probably take a bit of time to check the system files as it is, I don't really see any way around that, so I was only going to target the normal directories most apps install programs to...........And try to avoid any data directories or partitions.............I made a backup script a while back that is able to read /etc/fstab and /etc/mtab to find what system directories are mounted (such as /boot, /home /usr, etc.), mount the /boot directory if it isn't mounted {and unmount afterwards when done}...........And to exclude any mounted directories that are not the system directories (usually being the data directories)............

And yes, I was going to verify that what a program installed is still there........Basically, that would entail doing a check to see if the file exists by running through the lists of files in the /var/log/packages/ directory ( if [ -f FILE ] ; then ... ) and flagging the ones that are missing.............I was also going to use part of the code from my "whichpkg" script to verify the symlinks installed by programs as well................... :)


And carboncopy........thanx for the vote of confidence..... :D

Bebo 07-09-2004 05:21 AM
I couldn't help myself, so I rewrote my orphans script from scratch. I hope you don't mind if I post it here. It's rather long, so pardon me for clogging this thread :)

Note: This script might still be subject to updates, which will then replace the script here in this very post together with update info in a new post. Info on most recent update is posted below.

BEWARE: the script can of course NOT be used blindly - you HAVE to double-check the results! By this I mean that you have to make sure that the files reported aren't used/needed by some program. Although the script checks in both /var/adm/packages/* as well as /var/adm/scripts/* there will be false-positives, such as files generated by programs. These often contain important configuration, and other, data. For instance, some (most?) of the files in /etc that are reported as orphans might be extremely important.

Anyways, here it is.
Code:
#!/bin/sh

DEFAULTDIRS=( /bin /etc /opt/kde /sbin /usr )
ORPHANSKIPDIRS=( /dev /proc /sys /tmp /usr/tmp /usr/src /var /lib )
RENEGADESKIPDIRS=( /dev /lib /usr/src )


LOGCOMMAND=$(echo "$(basename $0) $@                          " | cut -c1-26 | tr -d '\n')
LOGFILE=$HOME/$(basename $0).log

ORPHANFILES=$PWD/orphaned-files
ORPHANTREE=$PWD/orphaned-dirs
RENEGADELIST=$PWD/renegades

TMPDIR=$(mktemp -d /tmp/$(basename $0)-XXXXXX) || exit 2

SLACKFILES=$TMPDIR/slackfiles
SLACKDIRS=$TMPDIR/slackdirs

DIRTREE=$TMPDIR/dirtree
ORPHANDIRS=$TMPDIR/orphandirs

PACKAGEFILES=$TMPDIR/packagefiles
PACKAGEDIRS=$TMPDIR/packagedirs

PKGFILES=$TMPDIR/pkgfiles
DOTNEWFILES=$TMPDIR/dotnewfiles
TEMPFILE=$TMPDIR/tempfile



renice 20 -p $$ > /dev/null

trap cleanup SIGINT

IFS=$'\n'



#======================================================================
# DEFINE FUNCTIONS
#----------------------------------------------------------------------
usage ()
{
cat << EOF

NAME
  $(basename $0) - script for locating orphaned and identify renegade files.

SYNOPSIS
  $(basename $0) [ACTIONS] [OPTIONS] [DIRECTORY...]

DESCRIPTION
  $(basename $0) will search for both orphaned files and directories as well
  as renegade files. An orphan is a file/directory on the file system that do
  not belong to any package as listed in /var/adm/packages. Renegade files are
  files that belong to installed packages but can not be found at the correct
  location.

  Directories subject to a search can be given as arguments on the command
  line. Some directories should usually be omitted in a search; these are
  given in the arrays ORPHANSKIPDIRS and RENEGADESKIPDIRS at the top of the
  script. These are currently:
      ORPHANSKIPDIRS:  ${ORPHANSKIPDIRS[@]}
      RENEGADESKIPDIRS: ${RENEGADESKIPDIRS[@]}

OUTPUT
  The results of an orphan scan are two files containing respectively one list
  of files and one list with a collapsed directory tree. The list files are
  defined by the variables ORPHANTREE and ORPHANFILES. These are currently:
      ORPHANTREE:  $ORPHANTREE
      ORPHANFILES: $ORPHANFILES

  A renegade search will put the results in RENEGADELIST, currently set to
      RENEGADELIST: $RENEGADELIST

DEFAULT
  If no action is given on the command line, searches for both orphans and
  renegades are conducted.

  If no directories are given as arguments on the command line, the search
  will be conducted in a number of default directories, given as the array
  DEFAULTDIRS at the top of the script. These are currently:
      DEFAULTDIRS: ${DEFAULTDIRS[@]}

ACTIONS
  --orphans | -o
      Search for orphaned files.

  --renegades | -r
      Search for renegade files.

OPTIONS
  --prune | -p
      Do not descend below the given directory.

  --quiet | -q
      Disable logging to logfile.

EOF
}



cleanup ()
{
    [ x$1 == x ] && echo -e "\n\n  SIGINT caught. Cleaning up.\n"
   
    case "$1" in
        orp* ) rm "$ORPHANFILES" "$ORPHANTREE" 2> /dev/null ;;
        ren* ) rm "$RENEGADELIST" 2> /dev/null ;;
        *    ) TEMP="$(echo $TMPDIR | sed 's|/tmp||' | sed 's|^/||')" && [ "x$TEMP" != x ] && ( cd /tmp && rm -rf "$TEMP" ) ;;
              # ^^ MAKE SURE WE DON'T DO SOMETHING REALLY STUPID ^^
    esac

    [ x$1 == x ] && exit 1
}



report ()
{
    REPORT="$1"
    ACTION="$2"
    STARTDATE="$3"
    ELAPSED="$4"

    case "$REPORT" in
        screen )
            case "$ACTION" in
                orp* )
                    printf "  %4d and %4d orphaned directories and files found and listed in %s and %s.\n" \
                        "$(cat $ORPHANTREE 2> /dev/null | grep '^/' | grep -c .)" "$(cat $ORPHANFILES 2> /dev/null | grep -c .)" "$ORPHANTREE" "$ORPHANFILES"
                    ;;
                ren* )
                    printf "  %4d renagade files detected and listed in %s.\n" \
                        "$(cat $RENEGADELIST 2> /dev/null | grep -c .)" "$RENEGADELIST"
                    ;;
            esac
            ;;
        file )
            case "$2" in
                orp* )
                    LOGLINE=$(printf "%s ORPHAN  search %s; %4d/%4d orphaned directories/files found in %8.3f seconds.\n" \
                        "$LOGCOMMAND" "$STARTDATE" "$(cat $ORPHANTREE 2> /dev/null | grep '^/' | grep -c .)" \
                        "$(cat $ORPHANFILES 2> /dev/null | grep -c .)" "$ELAPSED")
                    ;;
                ren* )
                    LOGLINE=$(printf "%s RENAGADE search %s;      %4d renagades detected in              %8.3f seconds.\n" \
                        "$LOGCOMMAND" "$STARTDATE" "$(cat $RENEGADELIST 2> /dev/null | grep '^/' | grep -c .)" "$ELAPSED")
                    ;;
            esac
            [ "x$LOGFILE" != x ] && echo "$LOGLINE" >> "$LOGFILE"
            ;;
    esac
}
#======================================================================



#======================================================================
# PARSE COMMAND LINE
#----------------------------------------------------------------------
FINDORPHANS=false
FINDRENEGADES=false
PRUNE=false

while [ "x$1" != x ] ; do
    case "$1" in
        "--orp*" )
            FINDORPHANS=true
            ;;
        "--ren*" )
            FINDRENEGADES=true
            ;;
        "--pru*" )
            PRUNE=true
            ;;
        "--qui*" )
            LOGFILE=""
            ;;
        "--hel*" | "-h" )
            usage
            exit 0
            ;;
        $(echo "$1" | grep '^\-[[:alnum:]]') )
              echo "$1" | grep -q o && FINDORPHANS=true
            echo "$1" | grep -q r && FINDRENEGADES=true
              echo "$1" | grep -q p && PRUNE=true
              echo "$1" | grep -q q && LOGFILE=""
            if echo "${1:1}" | grep -q '[^orpq]' ; then
                echo -e "\n  Unknown option: $(echo $1 | grep '[^orpq]' | sed 's|[orpq]||g')\n"
                exit 3
            fi
            ;;
        * ) unset DEFAULTDIRS
            [ -d "$1" ] && TOPDIRS[${#TOPDIRS[*]}]=$(cd "$1" ; pwd) || ERRORDIRS[${#ERRORDIRS[*]}]="$1"
            ;;
    esac
    shift
done

if ! $FINDORPHANS && ! $FINDRENEGADES ; then
    FINDORPHANS=true
    FINDRENEGADES=true
fi

[ "x$DEFAULTDIRS" != x ] && TOPDIRS=( "${DEFAULTDIRS[*]}" )

if [ "x$TOPDIRS" == x ] ; then
    echo -e "\n  No directories to search.\n"
    cleanup after
    exit 4
fi

if [ "x$ERRORDIRS" != x ] ; then
    [ ${#ERRORDIRS[*]} -eq 1 ] && echo -e "\n  Skipping nonexistent directory "${ERRORDIRS[*]}"."
    [ ${#ERRORDIRS[*]} -gt 1 ] && echo -e "\n  Skipping nonexistent directories "${ERRORDIRS[*]}"."
fi

TOPDIRS=( $(echo "${TOPDIRS[*]}" | sed 's|\([^/]\)$|\1/|') )
#======================================================================



echo ""

if $FINDORPHANS ; then
    cleanup orphans

    STARTDATE=$(date +'%Y-%m-%d %T')
    START=$(date +%s.%N)

    SKIPDIRS=( $(echo "${ORPHANSKIPDIRS[*]}" | sed 's|^|\^|' | sed 's|\([^/]\)$|\1/|') )

    if $PRUNE ; then
        BASEDIRS=( $(echo "${TOPDIRS[*]}" | grep -v "${SKIPDIRS[*]}" | sed 's|^|\^|') )
    else
        BASEDIRS=( $(find ${TOPDIRS[*]} -type d -maxdepth 1 -printf '%p/\n' | sed 's|//$|/|' | grep -v "${SKIPDIRS[*]}" | sed 's|^|\^|') )
    fi

    if [ ${#BASEDIRS[*]} -eq 0 ] ; then
        echo "  No directories to search for orphans."
    else
        tput sc ; echo -n "  ---> Setting up package structure..."

        cat /var/adm/packages/* | egrep '^[[:alnum:]]{3,4}\/' | sed 's|^|\/|' | grep "${BASEDIRS[*]}" | grep -v "${SKIPDIRS[*]}" > "$TEMPFILE"
        cat /var/adm/packages/* | egrep '^etc-incoming/' | sed 's|etc-incoming|etc|' | sed 's|^|\/|' | grep "${BASEDIRS[*]}" | grep -v "${SKIPDIRS[*]}" >> "$TEMPFILE"
        cat /var/adm/scripts/* | egrep '^config' | tr -s ' ' | cut -f2 -d' ' | egrep -v '[{(]' | sed 's|^|\/|' | grep "${BASEDIRS[*]}" | grep -v "${SKIPDIRS[*]}" >> "$TEMPFILE"

        cat "$TEMPFILE" | egrep '^*[^/]$' | sort -u > "$SLACKFILES"
        cat "$TEMPFILE" | sed 's|/[^/]\+$|/|' | sort -u > "$SLACKDIRS"

        rm "$TEMPFILE"

        tput rc ; tput el ; echo -n "  ---> Orphan search in " ; tput sc

        for BASEDIR in $(echo "${BASEDIRS[*]}") ; do
            echo -n "${BASEDIR:1}: building search tree"

            echo "${TOPDIRS[*]}" | grep -q "$BASEDIR" && TOPDIR=true || TOPDIR=false

            if $TOPDIR ; then
                grep -qx "$BASEDIR" "$SLACKDIRS" && echo "${BASEDIR:1}" > "$PACKAGEDIRS" || echo "${BASEDIR:1}" > "$ORPHANDIRS"
            else
                grep "$BASEDIR" "$SLACKDIRS" > "$PACKAGEDIRS"
                find "${BASEDIR:1}" -type d -printf '%p/\n' | sed 's|//$|/|' | grep -v "${SKIPDIRS[*]}" | sort -u > "$DIRTREE"
                comm -23 "$DIRTREE" "$PACKAGEDIRS" > "$ORPHANDIRS"
            fi
           
            if [ -s "$ORPHANDIRS" ] ; then
                tput rc ; tput el ; echo -n "${BASEDIR:1}: collapsing orphaned directory tree"
               
                while [ -s "$ORPHANDIRS" ] ; do
                    DIR=$(head -1 "$ORPHANDIRS")
                    echo "$DIR" >> "$ORPHANTREE"
                    grep -v "^$DIR" "$ORPHANDIRS" > "$TEMPFILE"
                    # ^^ NOT egrep HERE; egrep CRAPS OUT ON META CHARACTERS IN DIR NAMES, LIKE IN gtk+-2.2.4
                    mv "$TEMPFILE" "$ORPHANDIRS"
                done

                rm "$ORPHANDIRS" 2> /dev/null
            fi

            if [ -s "$PACKAGEDIRS" ] ; then
                tput rc ; tput el ; echo -n "${BASEDIR:1}: searching for orphaned files"
               
                if $TOPDIR ; then
                    grep "${BASEDIR}[^/]\+$" "$SLACKFILES" > "$PACKAGEFILES"
                    echo "${BASEDIR:1}" > "$DIRTREE"
                else
                    grep "$BASEDIR" "$SLACKFILES" > "$PACKAGEFILES"
                    #look "${BASEDIR:1}" "$SLACKFILES" > "$PACKAGEFILES" 
                    # look SEEMS SLOWER THAN grep... ODD, IT SHOULD TAKE ADVANTAGE OF THE SORTED FILES.
                fi

                for DIR in $(comm -12 "$PACKAGEDIRS" "$DIRTREE") ; do
                    grep "^${DIR}[^/]\+$" "$PACKAGEFILES" > "$PKGFILES"
                    if [ -s "$PKGFILES" ] ; then
                        find "$DIR" -type f -maxdepth 1 | sort > "$TEMPFILE"
                        comm -23 "$TEMPFILE" "$PKGFILES" >> "$ORPHANFILES"
                    else
                        find "$DIR" -type f -maxdepth 1 >> "$ORPHANFILES"
                    fi
                done
            fi

            tput rc ; tput el
        done

        if [ -s "$ORPHANTREE" ] ; then
            rm "$TEMPFILE" 2> /dev/null
            echo -n "the install scripts for the $(grep -c . $ORPHANTREE) apparently orphaned directories"
            for DIR in $(< "$ORPHANTREE") ; do
                SDIR=$(echo "$DIR" | sed 's|^/||' | sed 's|/$||')
                grep -q "$SDIR" /var/adm/scripts/* || echo "$DIR" >> "$TEMPFILE"
            done
            [ -s "$TEMPFILE" ] && mv "$TEMPFILE" "$ORPHANTREE" || rm "$TEMPFILE"
            tput rc ; tput el
        fi

        if [ -s "$ORPHANFILES" ] ; then
            egrep '.new$' "$SLACKFILES" | sed 's|\.new$||' | sort -u > $DOTNEWFILES
            sort -u "$ORPHANFILES" > "$TEMPFILE"
            mv "$TEMPFILE" "$ORPHANFILES"

            echo -n "the install scripts for the $(grep -c . $ORPHANFILES) apparently orphaned files"
            for FILE in $(comm -23 "$ORPHANFILES" "$DOTNEWFILES") ; do
                BASENAME="$(basename $FILE)"
                egrep -q "($BASENAME|${BASENAME}.new)" /var/adm/scripts/* && grep -q "$(dirname $FILE | sed 's|^/||')" /var/adm/scripts/* \
                    || egrep -q "($FILE|${FILE:1}.new)" /var/adm/scripts/* \
                    || echo "$FILE" >> "$TEMPFILE"
            done
            [ -s "$TEMPFILE" ] && mv "$TEMPFILE" "$ORPHANFILES" || rm "$TEMPFILE"

            tput rc ; tput el

            echo -n "taking care of a special case"
            for FILE in $(< "$ORPHANFILES") ; do
                [ "$(echo -e $FILE)" == $(echo -e "/usr/doc/kbd-1.12/utf/â\231ªâ\231¬") ] && fgrep -q "usr/doc/kbd-1.12/utf/â\231ªâ\231¬" /var/adm/packages/kbd-* || echo "$FILE" >> "$TEMPFILE"
            done
            [ -s "$TEMPFILE" ] && mv "$TEMPFILE" "$ORPHANFILES" || rm "$TEMPFILE"

            tput rc ; tput el
        fi

        FINISH=$(date +%s.%N)
        ELAPSED=$(echo "scale=3; $FINISH - $START" | bc -l)

        printf "\b\b\b\b finished in %8.3f seconds.\n" "$ELAPSED"

        report file orphans "$STARTDATE" "$ELAPSED"
    fi
fi



if $FINDRENEGADES ; then
    cleanup renagades

    STARTDATE=$(date +'%Y-%m-%d %T')
    START=$(date +%s.%N)

    SKIPDIRS=( $(echo "${RENEGADESKIPDIRS[*]}" | sed 's|^|\^|' | sed 's|\([^/]\)$|\1/|') )

    if $PRUNE ; then
        BASEDIRS=( $(echo "${TOPDIRS[*]}" | grep -v "${SKIPDIRS[*]}" | sed 's|^|\^|') )
    else
        BASEDIRS=( $(find ${TOPDIRS[*]} -type d -maxdepth 1 -printf '%p/\n' | sed 's|//$|/|' | grep -v "${SKIPDIRS[*]}" | sed 's|^|\^|') )
    fi

    if [ ${#BASEDIRS[*]} -eq 0 ] ; then
        echo  "  No directories to search for renegades."
    else
        echo -n "  ---> Renegade search: "
        tput sc
       
        for PKG in /var/adm/packages/* ; do
            echo -n "${PKG##*/}"
            for FILE in $(cat "$PKG" | egrep '^[[:alnum:]]{3,4}\/[[:print:]]*[^/]$' | sed 's|^|\/|' | grep "${BASEDIRS[*]}" | grep -v "${SKIPDIRS[*]}" | sed 's|\.new$||') ; do
                [ -f "$FILE" ] || [ -f "${FILE}.new" ] || echo "${PKG##*/}: $FILE" >> "$RENEGADELIST"
            done
           
            tput rc ; tput el
        done
       
        if [ -s "$RENEGADELIST" ] ; then
            echo -n "checking the few special cases..."
            for FILE in $(cat "$RENEGADELIST" | cut -f2 -d' ') ; do
                [ "$FILE" == "/bin/bash2" -a -f /bin/bash ] \
                    || [ "$(echo -e $FILE)" == $(echo -e "/usr/doc/kbd-1.12/utf/â\231ªâ\231¬") -a -f $(echo -e "/usr/doc/kbd-1.12/utf/â\231ªâ\231¬") ] \
                    || echo "$FILE" >> "$TEMPFILE"
            done
            [ -s "$TEMPFILE" ] && mv "$TEMPFILE" "$RENEGADELIST" || rm "$TEMPFILE"
            tput rc ; tput el
        fi

        FINISH=$(date +%s.%N)
        ELAPSED=$(echo "scale=3; $FINISH - $START" | bc -l)

        printf "\b\b finished in %8.3f seconds.\n" "$ELAPSED"

        report file renegades "$STARTDATE" "$ELAPSED"
    fi
fi



cleanup after

echo -e "\n\n  SEARCH RESULTS\n"
$FINDORPHANS && report screen orphans
$FINDRENEGADES && report screen renegades
echo ""

exit 0
Cheers

thegeekster 07-09-2004 12:28 PM
Quote:
Originally posted by Bebo
I couldn't help myself, so I rewrote my orphans script from scratch. I hope you don't mind if I post it here....
No problem about posting it here........Although if you noticed, I made some changes in my posts in this thread so any updated code goes in the original post for the code, with an explanation in a separate post...............I figure the admin or webmaster might appreciate it a little, since this will help reduce the "clutter" (and the amount of data saved in the forum's databse) and also make it easier to find any updated code........... :)

One question...........How long does it take to execute your script?......................I noticed that you pushed the script's priority way back to the minimum priority..............Myself, I would just use the ampersand to run the script in the background ( <scriptname> & )

If you want to time it to the nearest second, here's a little script, called "elapsed", I whipped up for just that purpose.............To time a script just add the variable START=`date +%s at the top, and at the bottom of the script add this line (assuming you save the "elapsed" script in /usr/local/bin): /usr/local/bin/elapsed $START `date +%s` . This will give you the elapsed time to the nearest second.
Code:
#!/bin/bash
#*******************************************************************************
# Name: elapsed

if [ $# -eq 2 ]
then
  DURATION=$(( $2 - $1 ))

  hh="$(( $DURATION / 3600 ))" && hh="$(( $hh / 24 )) + $(( $hh - ( $(( $hh / 24 )) * 24 )))"
  mm="$((( $DURATION % 3600 ) / 60 ))" && [ $mm -lt 10 ] && mm=0$mm
  ss="$((( $DURATION % 3600 ) % 60 ))" && [ $ss -lt 10 ] && ss=0$ss

  echo -e "\nElapsed time is ${hh}:${mm}:$ss\n"
else
  cat << __EOF__

 Usage: $0 <start_time> <stop_time>

Shows elapsed time in 1-second intervals. The output is in the format of

  d + h:mm:ss

where "d" is the number of days elapsed, "h" is the number of hours,
"mm" is a 2-digit display of minutes. and "ss" is a 2-digit display of
seconds.

NOTE: Start and stop times MUST be in seconds only, such as from the
      output of 'date +%s'

__EOF__

fi
NOTE: I've modified this code for the elapsed time by calculating the total number of seconds using the input from the 'date +%s' command..........Thanks to Bebo for this suggestion.........It removes the 24 hour limitation and another bug when trying to calulate the separated values of the "hh:mm:ss" input when using the input from the 'date +%T' command. ( The bug is when trying to caluclate using the digits 08 and 09 in the 2-digit format.)


Another thing is why not try to do the check on the fly, if you can.................it would save some time by not having to first create lists then having to read them.........As for the arrays, I haven't used arrays in bash scripting (yet) because I haven't found the need to.......*shrugs*

To show you what I mean, here's a sneak preview of the script I'm creating. This is a function that checks for missing files (what you call renegades, I think) on-the-fly, using 'egrep' for the filtering..................I'm still working on this but it's functional as it is:
Code:
#!/bin/sh
#*******************************************************************************
# Name:
START1=`date +%T`

# The kernel-source and kernel-ide pkgs are excluded by default when searching
# for missing package files. To exclude other pkgs, merely add it inside the
# parentheses, separating each package name by a pipe symbol (the vertical
# symbol, | ), such as "(kernel-source|mypackage)". There must not be any spaces
# before or after the pipe symbol. If for some reason the package name contains
# a space, try escaping the space by placing a backslash _before_ the space,
# like so: "(kernel-source|my\ package)".
EXCLUDEPKG="(kernel-source|kernel-ide)"

# Excluded list of files and directories that were removed by installation plus
# the first two lines of each package list in the /var/log/packages/ directory:
EXCLUDELIST="(FILE\ LIST:|^\./$|^install|var/lib/rpm/tmp)"

# These variables are for the final list of files either missing or orphaned:
MISSING="/root/missing-files"
ORPHANS="/root/orphaned-files"

find_missing() {
  find /var/log/packages/* | egrep -v "$EXCLUDEPKG" | while read list
  do
    egrep -A5000 'FILE\ LIST:' $list | egrep -v "$EXCLUDELIST" | while read file
    do
      case $file in
        # Some manpages may have been gzipped after they were already installed:
        `echo "$file" | egrep -i '/man/'` )
          if [ ! -e `echo "${file}.gz"` -a ! -e "$file" ]
          then
            echo "`basename $list` : /$file"
          fi        ;;
        # Check for "*.new" files which were renamed when installed:
        `echo "$file" | egrep -i '\.new$' | egrep -v 'bash2\.new'` )
          if [ ! -e `echo "$file" | sed 's/\.new$//'` -a ! -e "$file" ]
          then
            echo "`basename $list` : `echo \"/$file\" | sed 's/\.new$//'`"
          fi        ;;
        # This is for the "glibc" package which installs the libs in a tmp
        # folder called "/lib/incoming", then moves the libs to the parent
        # directory /lib:
        `echo "$file" | egrep '/incoming/'` )
          if [ ! -e `echo "$file" | sed 's,/incoming/,/,'` -a ! -e "$file" ]
          then
            echo "`basename $list` : `echo \"$file\" | sed 's,/incoming/,/,'`"
          fi        ;;
        # For the /bin/bash file which needs a separate filter all it's own:
        `echo "$file" | egrep 'bash2\.new'` )
          if [ ! -e `echo "$file" | sed 's/2\.new$//'` -a ! -e "$file" ]
          then
            echo "`basename $list` : `echo \"/$file\" | sed 's/2\.new$//'`"
          fi        ;;
        # This is for a localization file installed by the "kbd" package:
        `echo "$file" | egrep '/utf/342231252342231254'` )
          if [ ! -e `echo "$file" | sed 's,/utf/342231252342231254,/utf/♪♬,'` \
            -a ! -e "$file" ]
          then
            echo "`basename $list` : `echo \"/$file\" \
              | sed 's,/utf/342231252342231254,/utf/♪♬,'`"
          fi        ;;
        # And for the rest of the list:
        * )
          if [ ! -e "$file" ] ; then echo "`basename $list` : /$file" ; fi        ;;
      esac
    done
  done
}


# Check for missing files installed by the packages:
echo "Looking for missing files..."
( cd / ; find_missing ) > $MISSING
# If the missing-files report is empty (0-byte), then...
if [ ! -s $MISSING ] ; then echo "*** No missing files. ***" > $MISSING ; fi

echo "First run: `/usr/local/bin/elapsed $START1 \`date +%T\``"
I'm using the "case" scenario because it is slightly faster than "if...then...elif", and it's easy to add or remove the filtering for special handling.........And at last count, this took a little under 50 minutes to run in the foreground while also doing other things on the computer (AMD Athlon 900 CPU with 256 MB SDRAM)...... :)

Bebo 07-09-2004 09:03 PM
Hello,

I've just updated my script posted above.

Edit: Obsolete update info removed.

thegeekster,

The reason for renice'ing the script is just to stop it from taking all CPU. I don't want to ampersand it - or whatever the proper expression is - since I want to see that something is happening. Which of course also makes it slower; there are loads of echo'ing and tput'ing going on. Maybe I should give it a non-verbose option?

As for the timing... [And here I've deleted a bunch of crap about the fastness of my script that I wrote but of which I have no idea whatsoever where it came from.]

So case is faster than if? How come? BTW, if you want to calculate the elapsed time from seconds only, try date +%s :)

Ah yes, I forgot. The reason to chop the directory tree in small pieces and search one bit at a time is not only to have something nice (well... :)) to look at. I also wanted the list to grep in to be as short as possible. It takes a very long time to do a grep AFILE -f ANOTHERFILE when the lists contain tens of thousands of words.

Well, cheers :)

Bebo 07-11-2004 05:15 PM
Here's a second update to my script above. Main points:

(1) The use of arrays is resumed. It is actually faster to search through an array than a file - if the arrays aren't too large. If I put the package files in an array instead of a file, my script will be a lot slower.

(2) The verbosity is decreased, since it gives worse performance when the info on the terminal is updated quickly.

(3) The --quick option is gone. Turns out it wasn't so quick after all. Has to do with counting the number of elements in an array methinks.

(4) New option: --prune. Only search in the given directory - do not descend in directories below.

(5) The issue with /usr/bin/[ is fixed using the -F grep option.

(6) Some logging is done, timing for instance. In nanoseconds.

[Again, I have deleted some stuff about the script's fastness. What's the point of keeping track of the performance of an erroneous script?]

This is fun :D

thegeekster 07-11-2004 09:50 PM
Quote:
Originally posted by Bebo
...This is fun :D
Took the words right out of my mouth. :D ...................Looks like I can take it slower with the script I'm working on, such as relaxing and playing a few games, etc..... ;)

Anyway, I did another upgrade to the pkginfo script above to include any info from the installation script (doinst.sh) if one is found (not all packages have, or use, a "doinst.sh" installation script). This will be added at the bottom of the output to the screen, under the heading of "INSTALLATION FILE:" (just scroll down until you come to it)...............I thought this would be good to look at, too, for any additional information on what was done besides the adding of files from the package itself during the installation process (Remember the line that says Executing install script for <package_name>... when you install a package with the 'installpkg' command?)........... :)

I also did some minor code changes to the "whichpkg" script as well. There are no added features, tho', so it's not absolutely necessary to upgrade it........

Enjoy :)


PS: Read the section on PATTERN MATCHING in the original post above for important info on the "hybrid" pattern matching used by these scripts.

gnashley 07-12-2004 03:03 AM
Does one of you know the command to find a missing file in Manifest. i saw this once, but then couldn't find it. I've seen some suggestion using zless, but what I want is something like: gunzip Manifest.gz |grep "somestring" (maybe there was a 'cat' in there too), that works with it without leaving the file uncompressed.

rotvogel 07-12-2004 03:07 AM
Quote:
Originally posted by gnashley
Does one of you know the command to find a missing file in Manifest. i saw this once, but then couldn't find it. I've seen some suggestion using zless, but what I want is something like: gunzip Manifest.gz |grep "somestring" (maybe there was a 'cat' in there too), that works with it without leaving the file uncompressed.
Code:
zcat
zcat: compressed data not read from a terminal. Use -f to force decompression.
For help, type: zcat -h
Is that where you were looking for ?

thegeekster 07-12-2004 04:55 AM
Quote:
Originally posted by gnashley
Does one of you know the command to find a missing file in Manifest. i saw this once, but then couldn't find it. I've seen some suggestion using zless, but what I want is something like: gunzip Manifest.gz |grep "somestring" (maybe there was a 'cat' in there too), that works with it without leaving the file uncompressed.
Is this what you were referring to? http://www.linuxquestions.org/questi...16#post1039216

You can replace the less command with the grep command...........

:)

gnashley 07-12-2004 09:48 AM
Yeh, that's a start, but I need it to get the package name too, in case the file has a name that's not part of the package name. perhaps by patching your whichpkg...
I haven't tried yours and bebos scripts yet, waiting till you both are satisfied.
But finding the package name for a missing file from the Manifest seems to be the most accurate way to track a dependency for any standard Slackware package, at least.

thegeekster 07-12-2004 01:30 PM
Okay............If I understand you correctly, you're looking for something that will give you a dependency check of sorts..............plug in a filename name and it'll tell you which Slackware package is needed........

However, this method won't work when plugging in a name of a symlink (or a renamed file) as the MANIFEST file gives you an exploded list of the files stored in each package, whether source or slackpack..........but slackpacks don't store the symlinks in the package, they create symlinks after installing the stored files using the doinst.sh script, as well as renaming files...........

Basically, you would have to not only look at the filenames inside the packages, but read and analyze any doinst.sh scripts inside each package (if the package has one)..............and to do that, you would need to have a local mirror of the slackware tree....................it could probably be done using the first and second installation CDs.................Unless another list was created containing the symlinks and renamed files for every Slackware package............

Hmmm........this is beginning to sound interesting....... :)

thegeekster 07-12-2004 02:32 PM
Okay, here's a quick and dirty solution for searching through the MANIFEST files to find the file you're looking for and the package it will be found in.......

First grab the MANIFEST.bz2 files from a Slack mirror..........for each Slack version there should be a MANIFEST file in the directories extra. pasture, patches, slackware, source, testing. (NOTE: The Slack-current tree won't have a patches directory.).........Then run the following command on whichever MANIFEST file you choose, replacing FILENAME with the file you're looking for:
Code:
bzcat MANIFEST.bz2 | egrep '(Package:|\<FILENAME$)' | egrep -B1 '\<FILENAME$'
The '\<' in front of the search term is an extended regular expression metacharater that means what follows is the beginning of a word, and the '$' after the search term means to look at the end of the line..............this will help filter out the majority of false positives.....and you must use 'egrep' to use the '\<' metacharacter (plus 'egrep' is supposed to be a little faster than 'grep')..................

For example, here's the ouput when searching bzip2 in the slackware-10.0/slackware/MANIFEST.bz2 file:
Code:
$ bzcat MANIFEST.bz2 | egrep '(Package:|\<bzip2$)' | egrep -B1 '\<bzip2$'

||  Package:  ./a/bzip2-1.0.2-i486-5.tgz
-rwxr-xr-x root/bin      28580 2004-02-22 16:22:02 bin/bzip2
:)

gnashley 07-12-2004 04:21 PM
That seems interesting. I'll have to look at that tomorrow, though, as it's getting late here in Deutschland. Does that work if FILENAME$ is not part of the package name? Is that case sensitive or any other caveats?
maybe an 'incomplete' search (without looking in the doinstall.sh) would still be useful if the search string was general, since some/most symlinks have a similar name. However, an exhaustive search, seems like the only dependable way to resolve a dependency if it can be done with a Slackware package.

thegeekster 07-12-2004 06:55 PM
Quote:
...Does that work if FILENAME$ is not part of the package name?...
I'm not too sure what you mean by not being part of the package name. This will only find the strings in the MANIFEST file, which is merely a listing of what files can be found in a package............If you wondering about whether you need to know part of the package name to make it work, the answer is No.........

The search is for the filename, but how it works is the first 'egrep' command will list all the lines with the word "Package:" along with any lines that may include the search term (filename) you are looking for, listing each in the same order it finds it............while the second 'egrep' command will again look for the desired filename, but also include the line above it from the output of the first 'egrep' command, which _should_ be the name of the package.......

Remember that the search is for any lines containing the search term at the end of each line, which will exlcude the names of packages..............It's the second 'egrep' command which will include any line above the matching line from the output of the first 'egrep' command, which will hopefully be the name of the package

Quote:
...Is that case sensitive or any other caveats?...
Yes, it's case sensitive.........If you want a case insensitive search, just add the switch '-i' right after each 'egrep' command and before the search term..................As for any other limitations, you will need to use the whole filename for the search term, or you can use extended regular expression pattern matching (not wildcard globbing) to expand the search.............Also, see my post above (post #46) for a quick insight on what will be missing in the search..........and there may be other unforseen complications........I only came up with this on the spur of the moment without doing any real testing of search terms..............like I said at the beginning, a "quick and dirty solution"............ :)

Bebo 07-19-2004 02:06 PM
The orphan/renegade search script is updated. It has taken some time to get this finished, mostly due to some bug-mud I fell in concerning the declare -a that for instance the Advanced Bash-Scripting Guide says should speed up the use of arrays. It does speed it up, but it also makes the arrays unable to contain elements with spaces and parenthesis and stuff. Pretty odd, I'd say. The script should be fairly ready for use by now, but it can of course not be used blindly - you have to check the results! There are mainly two classes of false-positive orphaned files: (1) files generated by other programs, often containing configuration, and other, data, and (2) files in /var. The /var directory should perhaps be completely excluded from the search. Also, /bin/bash is of course a false-positive, since it is renamed from the package name /bin/bash2. Other possible false-positives are the man pages, since they sometimes are gzipped, sometimes not. On my system they do not show in a search, though, so it seems ok.

The most important update is the search for orphaned directories. This will first locate the directories on the file system that are not explicitly seen in /var/adm/packages/*. The created list of orphaned directories is collapsed to be as short as possible. The search for orphaned files is then adapted to exclude the orphaned directories. This means that the orphaned files in an orphaned directory will not be listed since they are included in the orphaned dir. This gives a performance penalty by a factor of about 2 compared to the former version, but it is a nice feature that I didn't want to leave out. It searches the 4 gigs of system files (/usr /bin /sbin /opt/kde and so on) on my computer in 10 minutes which is still bareable.

If you look carefully at the code, you might object (to a lot of things, but perhaps the thing I'm after here) that it should loop over the contents in $PACKAGEDIRS instead of $(grep -xf "$PACKAGEDIRS" "$DIRTREE") in the search for orphaned files. However, then it will follow symlinks, and in some cases erroneously classify the files seen under the symlinked directories as orphans.

There is also a new --quiet option which will disable the logfile. It could also be disabled by just setting LOGFILE="".


Now to something completely different. You know all those .new files that you find in for instance /etc after an update? It can be quite boring to locate all these and rename them, so I whipped up a script (oh, really? :)) to help me in this. It also backs up the older scripts, appending a .old, up to a maximum of two old versions. Here it is:
Code:
#!/bin/sh

LOG=$HOME/chknew.log
rm $LOG 2> /dev/null


DIR=/etc
[ x`echo $1` != x ] && DIR=$1


IFS=$'\n'


echo "Checking for .new files in $DIR... "

for NEWFILE in `find $DIR -xdev -type f -name '*.new'` ; do
  OLDFILE=${NEWFILE%%.new}

  if [ -e "$NEWFILE" -a -e "$OLDFILE" ] ; then
      AGE=older
      [ `find "$NEWFILE" -cnewer "$OLDFILE" | grep -c .new` -gt 0 ] && AGE=newer

      echo -en "\n$NEWFILE is $AGE than $OLDFILE. Wanna diff them? (Y/n)"
      read ANSWER

      if [ x$ANSWER != xn ] ; then
        diff -bB "$NEWFILE" "$OLDFILE"
        echo ""
      fi

  else
      echo -en "\n$NEWFILE has no $OLDFILE counterpart. "
  fi

  echo -n "Wanna move $NEWFILE to $OLDFILE? (y/N)"
  read ANSWER

  if [ x$ANSWER == xy ] ; then
      [ -e "${OLDFILE}.old" ] && mv -v "${OLDFILE}.old" "${OLDFILE}.old.old" && chmod a-x "${OLDFILE}.old.old"
      [ -e "${OLDFILE}" ] && mv -iv "${OLDFILE}" "${OLDFILE}.old" && chmod a-x "${OLDFILE}.old"

      mv -iv "$NEWFILE" "$OLDFILE"

      echo "$OLDFILE" >> $LOG
  else
      echo -n "Wanna remove $NEWFILE? (y/N)"
      read REPLY
      [ x$REPLY == xy ] && rm -i $NEWFILE
  fi
done

if [ -s $LOG ] ; then
    echo -e "\n========================================================="
    echo -e "\nYou've changed these files:\n"
    cat $LOG
    echo -e "\nRemember to make any new scripts executable with chmod!"
    echo -e "\n---------------------------------------------------------\n"
fi
Cheers


All times are GMT -5. The time now is 03:13 PM.
Page 1 of 2 1 2
Show 50 post(s) from this thread on one page