slfcli: slakfinder client

zerouno · 04-18-2022, 01:34 PM

Hi.

I'm working to an integration of slackpkg+ with slakfinder.org

I wrote a webservices to query slakfinder from commandline.
It helps to detect if a package exists in some repository that you have not configured in slackpkg+.

You can query it with curl or with a slakfinder client (see below)

Here some example:

Code:

# curl https://slakfinder.org/slfcli.php?pkg=slackpkg
name    version arch    reponame        repoid  url     repobrief       location        filename        id
slackpkg        15.0.10 noarch  Slackware64-15.0        178     https://mirrors.slackware.com/slackware/slackware64-15.0/       Official        ./slackware64/ap        slackpkg-15.0.10-noarch-1.txz   1144611
slackpkg        15.0.10 noarch  slackware64-current     105     https://mirrors.slackware.com/slackware/slackware64-current/    Official        ./slackware64/ap        slackpkg-15.0.10-noarch-1.txz   1549006
slackpkg        15.0.10 noarch  Slackware-15.0  171     https://mirrors.slackware.com/slackware/slackware-15.0/ Official        ./slackware/ap  slackpkg-15.0.10-noarch-1.txz   1350681
slackpkg        15.0.10 noarch  slackware-current       97      https://mirrors.slackware.com/slackware/slackware-current/      Official        ./slackware/ap  slackpkg-15.0.10-noarch-1.txz   1547003
slackpkg        2.82.1  noarch  Slackware64-14.2        108     https://mirrors.slackware.com/slackware/slackware64-14.2/       Official        ./slackware64/ap        slackpkg-2.82.1-noarch-3.txz    1141193
slackpkg        2.82.1  noarch  Slackware-14.2  111     https://mirrors.slackware.com/slackware/slackware-14.2/ Official        ./slackware/ap  slackpkg-2.82.1-noarch-3.txz    1349107
slackpkg        2.82.0  noarch  Slackware64-14.1        103     https://mirrors.slackware.com/slackware/slackware64-14.1/       Official        ./slackware64/ap        slackpkg-2.82.0-noarch-12.tgz   1358531
slackpkg        2.82.0  noarch  Slackware64-14.0        100     https://mirrors.slackware.com/slackware/slackware64-14.0/       Official        ./slackware64/ap        slackpkg-2.82.0-noarch-8.tgz    1355700
slackpkg        2.82.0  noarch  Slackware-14.1  95      https://mirrors.slackware.com/slackware/slackware-14.1/ Official        ./slackware/ap  slackpkg-2.82.0-noarch-12.tgz   1133348
slackpkg        2.82.0  noarch  Slackware-14.0  92      https://mirrors.slackware.com/slackware/slackware-14.0/ Official        ./slackware/ap  slackpkg-2.82.0-noarch-8.tgz    1347781

Use -d to send data in POST method so to avoid the server storing the query for your privacy.
Query the packageid (last column in the previous output) to query a specific package (note: it will change after slakfinder download newest metadata from repository, once a day currently)
Query a different format other than tabular (json is machine readable; available other format)

Code:

# curl -d "pkg=1144611&format=json" https://slakfinder.org/slfcli.php
[{"name":"slackpkg","version":"15.0.10","arch":"noarch","reponame":"Slackware64-15.0","repoid":"178","url":"https:\/\/mirrors.slackware.com\/slackware\/slackware64-15.0\/","repobrief":"Official","location":".\/slackware64\/ap","filename":"slackpkg-15.0.10-noarch-1.txz","id":"1144611"}]

look in slak-desc instead package name, limit to 1 results (max 50), print ALL package data available, print in a human readable (useful when you print all data), print metadata (count is the number of results)

Code:

# curl -d "desc=browser&max=1&format=array&metadata=true&fields=total" https://slakfinder.org/slfcli.php
Array
(
    [metadata] => Array
        (
            [count] => 1119
            [slfver] => 0.1.0
            [cliver] => curl/7.81.0
            [query] => [desc=browser&max=1&format=array&metadata=true&fields=total]
        )

    [0] => Array
        (
            [id] => 1143829
            [repository] => 178
            [filename] => kcachegrind-21.12.1-x86_64-1.txz
            [name] => kcachegrind
            [version] => 21.12.1
            [arch] => x86_64
            [build] => 1
            [compression] => txz
            [location] => ./slackware64/kde
            [comprsize] => 840 K
            [uncomprsize] => 3230 K
            [required] => 
            [conflicts] => 
            [suggests] => 
            [description] => kcachegrind (GUI to profilers such as Valgrind)

kcachegrind is a graphical browser for data produced by profiling
tools such as Valgrind.


            [reponame] => Slackware64-15.0
            [repodesc] => Slackware 64bit Official Distribution - 15.0 Version
            [repover] => 15.0
            [repobrief] => Official
            [url] => https://mirrors.slackware.com/slackware/slackware64-15.0/
            [repoid] => 178
            [repoclass] => 64150
            [repoarch] => x86_64
            [rank] => 15
        )

)

send information about the client, for example if you write a your custom application to query it (you will find it in metadata; also you can use it to masquerade curl version)
use jolly character aaa*
limit results to 64bit repositories
sort results for slackware release (dist=from current to 14.0, distr=from 14.0 to current); by default it will sort for relevance
print only some fields

Code:

# curl -A "myapp/1.0" -d "pkg=aaa*&repo=x86_64&order=distr&fields=name,version,repoclass,filename" https://slakfinder.org/slfcli.php
name    version repoclass       filename
aaa_base        14.0    64140   aaa_base-14.0-x86_64-5.txz
aaa_elflibs     14.0    64140   aaa_elflibs-14.0-x86_64-4.txz
aaa_terminfo    5.8     64140   aaa_terminfo-5.8-x86_64-1.txz
aaa_elflibs-compat32    14.0    64140   aaa_elflibs-compat32-14.0-x86_64-4compat32.txz
aaa_base        14.1    64141   aaa_base-14.1-x86_64-1.txz
aaa_elflibs     14.1    64141   aaa_elflibs-14.1-x86_64-3.txz
aaa_terminfo    5.8     64141   aaa_terminfo-5.8-x86_64-1.txz
aaa_elflibs-compat32    14.1    64141   aaa_elflibs-compat32-14.1-x86_64-3compat32.txz
aaa_base        14.2    64142   aaa_base-14.2-x86_64-2.txz
aaa_elflibs     14.2    64142   aaa_elflibs-14.2-x86_64-23.txz
aaa_terminfo    5.9     64142   aaa_terminfo-5.9-x86_64-1.txz
aaa_elflibs-compat32    14.2    64142   aaa_elflibs-compat32-14.2-x86_64-23compat32.txz
aaa_base        15.0    64150   aaa_base-15.0-x86_64-3.txz
aaa_terminfo    6.3     64150   aaa_terminfo-6.3-x86_64-1.txz
aaa_libraries   15.0    64150   aaa_libraries-15.0-x86_64-19.txz
aaa_glibc-solibs        2.33    64150   aaa_glibc-solibs-2.33-x86_64-5.txz
aaa_base        15.0    64150   aaa_base-15.0-x86_64-4_slack15.0.txz
aaa_glibc-solibs        2.33_multilib   64150   aaa_glibc-solibs-2.33_multilib-x86_64-5alien.txz
aaa_libraries-compat32  15.0    64150   aaa_libraries-compat32-15.0-x86_64-19compat32.txz
aaa_base        15.1    64cur   aaa_base-15.1-x86_64-2.txz
aaa_terminfo    6.3     64cur   aaa_terminfo-6.3-x86_64-1.txz
aaa_libraries   15.1    64cur   aaa_libraries-15.1-x86_64-3.txz
aaa_glibc-solibs        2.35    64cur   aaa_glibc-solibs-2.35-x86_64-2.txz
aaa_glibc-solibs        2.35_multilib   64cur   aaa_glibc-solibs-2.35_multilib-x86_64-2alien.txz
aaa_libraries-compat32  15.1    64cur   aaa_libraries-compat32-15.1-x86_64-2compat32.txz

Get the full help querying it without parameters:

Code:

# curl https://slakfinder.org/slfcli.php
slfcli webservices 0.1.0

Usage: slfcli.php?param1=value1&param2=value2

   you can query params as a GET request or as a POST request
   also you may send an user-agent 

   curl -A 'myapplication/0.1' https://slakfinder.org/slfcli.php?pkg=slackpkg
   curl -A 's/0.1' -d 'pkg=slackpkg' https://slakfinder.org/slfcli.php

   you can also download the commandline slakfinder client at 
        https://slakfinder.org/slakfinder.sh

params:

   the 'pkg' or 'desc' params is mandatory.

   pkg=<package>
             package to find; by default it does an exact match of <package>
             if <package> is a number it is the package id
             it accept '*' as jolly character (*firefox or mozilla*)
             it accept '/' for regular expression (/firefox$)
             it accept '%' for whole word (%firefox% or fox% or %fire)

   desc=<description>
             search into package description in slack-desc

   format=<outformats>
             outformats is csv,array,json,serial (see available output formats below). csv is the default

   fields=<outfields>
             comma separated list of fields to return (see available output fields below) or 'all' to select all fields
             excluding 'description' or 'total' to select all including 'description'.
             default: name,version,arch,reponame,repoid,url,repobrief,location,filename,id

   repo=<repository>
             limit results to specific repositories. You can select repository by ID or release (15.0, current, ...) or
             architecture (x86_64,i386,noarch,mixed) or repository class (see below)
             All available repository: https://slakfinder.org/showrepo.php

   order=<field>[r]
             sort output by field. Add 'r' for reverse order. Available values:
                   rank (default)
                   pkg    package name
                   ver    package version
                   arch   package architecture
                   dist   slackware release
                   repo   repository name
                   loc    location (path) of package

   start=<numstart>
             retrieve paginated results starting from <numstart>

   max=<nummax>
             retrieve no more than <nummax> results. By default '50'. You can't specify more than 50.
             '0' means no results (just print headers and metadata)

   metadata=true
       show metadata info. Currently available 
                     'count': show the number of results; query for 'max=0' to only know the result number;
                     useful to paginate results or for a faster output

results:

   available output formats are
      csv <tab> separated
      array in print_r() format
      json as list of dictionary in python format with some characters backquoted (as /)
      storable (machine readable) with serialize php() function

   available output fields are
      Package info:
        id             1175608
                       You can use this field to lookup package on slakfinder:
                       https://slakfinder.org/show.php?pkg=1175608
        repository     152
                       You can use this field to lookup repository on slakfinder:
                       https://slakfinder.org/showrepo.php?repo=152
  
        filename       slackpkg+-1.8.0-noarch-2mt.txz
        name           slackpkg+
        version        1.8.0
        arch           noarch
        build          2mt
        compression    txz
        location       ./pkg
        comprsize      80 K
        uncomprsize    280 K
        required       null
        conflicts      null
        suggests       null
        description    <slack-desc content>
  
        Repository info:
        reponame       slackpkgplus15.0
        repodesc       Slackpkg+ (stable for 15.0) is an extention to slackpkg
        repover        15.0
        repobrief      Slackpkgplus
        url            https://slakfinder.org/slackpkg+15/
        repoid         152
        repoclass      mi150
                       5 digits that define the target packages in the repository:
                          'mi'=mixed, '32'=i386, '64'=x86_64
                          'cur'=current, '150'=15.0, '142'=14.2, '141'=14.1, '140'=14.0
        repoarch       noarch
        rank           4.1389
                       Calculated rank to order results

PRIVACY INFORMATION
    The search parameters sent in the POST method are not stored
    in any database on the server and therefore not used for any further purpose
    to that envisaged for the delivery of results. If the data is sent in
    GET method stores the http request.
    In NO case the data are used to profile the service or any
    other activity outside those statistics after anonymisation of the
    sensitive data.

DISCLAIMER
    The accuracy of the information is not guaranteed. Make a use
    aware of the service. Don't abuse the service.

it is work in progress so I may add/modify features in any moment.

You can download latest slakfinder client here.

for now it is a lot limited but it is a way to not use manual http queries.

Code:

# ./slakfinder.sh libreoffice
name    version arch    reponame        repoid  url     repobrief       location        filename        id
libreoffice     5.4.0   i586    slackonly-14.2  116     https://packages.slackonly.com/pub/packages/14.2-x86/   SlackOnly       ./office/libreoffice    libreoffice-5.4.0-i586-1_slonly.txz     1113249
libreoffice     5.1.2   x86_64  slackonly64-14.1        11      https://packages.slackonly.com/pub/packages/14.1-x86_64/        SlackOnly       ./office/libreoffice    libreoffice-5.1.2-x86_64-1_slack.txz    1495351
libreoffice     7.3.2   i586    alien-current   72      https://slackware.nl/people/alien/sbrepos/current/x86/  Alien Bob       ./libreoffice   libreoffice-7.3.2-i586-1alien.txz       1553764
libreoffice     7.3.2   x86_64  alien64-current 71      https://slackware.nl/people/alien/sbrepos/current/x86_64/       Alien Bob       ./libreoffice   libreoffice-7.3.2-x86_64-1alien.txz     1553313
libreoffice     7.3.2   x86_64  alien64-15.9    135     https://slackware.nl/people/alien/sbrepos/15.0/x86_64/  Alien Bob       ./libreoffice   libreoffice-7.3.2-x86_64-1alien.txz     1552413
libreoffice     7.3.2   i586    alien-15.0      136     https://slackware.nl/people/alien/sbrepos/15.0/x86/     Alien Bob       ./libreoffice   libreoffice-7.3.2-i586-1alien.txz       1552862
libreoffice     7.0.4   x86_64  alien64-14.2    132     https://slackware.nl/people/alien/sbrepos/14.2/x86_64/  Alien Bob       ./libreoffice   libreoffice-7.0.4-x86_64-1alien.txz     1551697
libreoffice     7.0.4   i586    alien-14.2      133     https://slackware.nl/people/alien/sbrepos/14.2/x86/     Alien Bob       ./libreoffice   libreoffice-7.0.4-i586-1alien.txz       1552031
libreoffice     5.1.4   x86_64  salix64-14.2    114     https://download.salixos.org/x86_64/14.2/       Salix64 ./salix/xap     libreoffice-5.1.4-x86_64-1dj.txz        1536272
libreoffice     5.1.4   i686    salix-14.2      213     https://download.salixos.org/i486/14.2/ Salix   ./salix/xap     libreoffice-5.1.4-i686-1dj.txz  1535517
libreoffice     5.0.1   x86_64  salix64-14.1    4       https://download.salixos.org/x86_64/14.1/       Salix64 ./salix/xap     libreoffice-5.0.1-x86_64-1dj.txz        1352276
libreoffice     5.1.6.2 x86_64  schoepfer.info-x86_64-14.2      122     http://slackware.schoepfer.info/14.2_64/        Johannes Schopfer       ./slackware64/xap_jonix libreoffice-5.1.6.2-x86_64-1jsc.txz     1123435
libreoffice     5.2.6   x86_64  mled64-14.2     76      https://slackware.uk/microlinux/desktop-14.2-64bit/     kikinovak       ./slackware64/xap       libreoffice-5.2.6-x86_64-1_microlinux.txz       1128603
libreoffice     5.0.1   i686    salix-14.1      2       https://download.salixos.org/i486/14.1/ Salix   ./salix/xap     libreoffice-5.0.1-i686-1dj.txz  1371833
libreoffice     4.2.5   x86_64  salix64-14.0    3       https://download.salixos.org/x86_64/14.0/       Salix64 ./salix/xap     libreoffice-4.2.5-x86_64-1dj.txz        1082656
libreoffice     5.2.6   i586    mled-14.2       75      https://slackware.uk/microlinux/desktop-14.2-32bit/     kikinovak       ./slackware/xap libreoffice-5.2.6-i586-1_microlinux.txz 1128381
libreoffice     4.2.5   i686    salix-14.0      1       https://download.salixos.org/i486/14.0/ Salix   ./salix/xap     libreoffice-4.2.5-i686-1dj.txz  1080852
libreoffice     4.4.7.2 x86_64  schoepfer.info-x86_64-14.1      32      http://slackware.schoepfer.info/14.1_64/        Johannes Schopfer       ./slackware64/xap_jonix libreoffice-4.4.7.2-x86_64-1jsc.txz     1123009
libreoffice     5.4.4   x86_64  rlworkman-for-14.2      121     https://rlworkman.net/pkgs/14.2/        Robby Workman   ./x86_64        libreoffice-5.4.4-x86_64-1_rlw.txz      1122233
libreoffice     5.4.4   i586    rlworkman-for-14.2      121     https://rlworkman.net/pkgs/14.2/        Robby Workman   ./i486  libreoffice-5.4.4-i586-1_rlw.txz        1122193
libreoffice     5.2.6   x86_64  mled64-14.1     74      https://slackware.uk/microlinux/desktop-14.1-64bit/     kikinovak       ./slackware64/xap       libreoffice-5.2.6-x86_64-1_microlinux.txz       1390799
libreoffice     4.3.7.2 i686    schoepfer.info-i386-14.1        34      http://slackware.schoepfer.info/14.1/   Johannes Schopfer       ./slackware/xap_jonix   libreoffice-4.3.7.2-i686-1jsc.txz       1123281
libreoffice     5.2.6   i586    mled-14.1       73      https://slackware.uk/microlinux/desktop-14.1-32bit/     kikinovak       ./slackware/xap libreoffice-5.2.6-i586-1_microlinux.txz 1127866
libreoffice     5.0.3   x86_64  rlworkman-for-14.1      27      https://rlworkman.net/pkgs/14.1/        Robby Workman   ./x86_64        libreoffice-5.0.3-x86_64-1_rlw.txz      1388407
libreoffice     5.0.3   i586    rlworkman-for-14.1      27      https://rlworkman.net/pkgs/14.1/        Robby Workman   ./i486  libreoffice-5.0.3-i586-1_rlw.txz        1388276
libreoffice     4.0.3   x86_64  rlworkman-for-14.0      26      https://rlworkman.net/pkgs/14.0/        Robby Workman   ./x86_64        libreoffice-4.0.3-x86_64-1_rlw.tgz      1121968
libreoffice     4.0.3   i586    rlworkman-for-14.0      26      https://rlworkman.net/pkgs/14.0/        Robby Workman   ./i486  libreoffice-4.0.3-i586-1_rlw.tgz        1121933
libreoffice     7.2.4.1 i586    salix-15.0      159     https://download.salixos.org/i486/15.0/ Salix   ./salix/xap     libreoffice-7.2.4.1-i586-1gv.txz        1550161
libreoffice     7.2.4.1 x86_64  salix64-15.0    160     https://download.salixos.org/x86_64/15.0/       Salix64 ./salix/xap     libreoffice-7.2.4.1-x86_64-1gv.txz      1550602

Code:

# ./slakfinder.sh -h

    Usage: ./slakfinder.sh [ -h | -H | -V ] [ -w | -f | -r | -s ] [ -D ] [ -l <field>,<field>,... | -L | -A | -c ] [ -p <from>,<num>,<max> ] [ -P ]  { <string> | <packageid> }

      -h    show this help
      -H    show webservices help
      -V    show client and server version

      -c    only returns count of packages
      -l    field list to output (see -H for details)
      -L    list all package informations, except package description
      -A    list all package informations, including package description
                note that description field may generate problems in the output

      -P    paginated output with interactive mode
      -p    paginated query; default: 1,20,50
            <from> from which result number to start
            <num>  max results for query (for page if -P selected); no more than 50
            <max>  max total results (unlimited if -P selected)

      -D    search in package description

      -f    match the exact full string (default)
      -w    match a whole word          (equivalent to %string%) (default if '-D' selected)
      -r    match a regular expression  (equivalent to /string)
      -s    match a sub string          (equivalent to *string*)


      <string> by default match entire string (equivalent to -f):
                    slackpkg  : it does match slackpkg; it does not match slackpkg+
            it may contains special characters if no search methods is specified:
            '*'  Jolly character that means 'string'
                    *firefox  : ends with 'firefox'
                    mozilla*  : starts with 'mozilla'
                    *firefox* : contains firefox (equivalent to '-s')
            '%'  start or end of a word
                    lib%   : it does match glib-networking; it does not match libzip
                    %edit  : it does match dconf-editor; it does not match xedit
                    %font% : it match entire word as hack-font-ttf (equivalent to '-w')
            '/'  specify a regular expression (equivalent to -r)
                    /^libev[de] : it does match libevdev, libevent
                    /kmi.e      : it does match kmime, kmines
                    /fonts?$    : it does match xlsfonts, showfont
                    /^x.*x$     : it does match any string that start and end with x
                    /x\{2\}     : it does match any string containing x two consecutive time
                    /^[^0-9]*$  : it does match any string not containing numbers
                    ....

    DISCLAIMER:          
        This tool uses slakfinder.org service.  The accuracy of the information
        is not guaranteed. Make a use aware of the service. Don't abuse the service.

    PRIVACY INFORMATION: slakfinder.org does not store your queries.

Tonus · 04-18-2022, 04:38 PM

Sounds great.
Just curious about "alien64-15.9" repo : typo ? On wich side ?

The usecases I can imagine would involve searching for specific slackware version, or excluding it. I believe I would use regular expression match for that.
Maybe a hint for excluding x86_64 in examples ?

zerouno · 04-18-2022, 06:39 PM

Quote:

Originally Posted by Tonus

Sounds great.
Just curious about "alien64-15.9" repo : typo ? On wich side ?

typo server side.
fixed thanks.

Quote:

The usecases I can imagine would involve searching for specific slackware version, or excluding it. I believe I would use regular expression match for that.
Maybe a hint for excluding x86_64 in examples ?

you can look for specific version, not excluding it

Code:

# curl -d 'pkg=slackpkg&repo=i386' https://slakfinder.org/slfcli.php         => look in repository containing i386 and noarch
# curl -d 'pkg=slackpkg&repo=15.0' https://slakfinder.org/slfcli.php         => look in slackware 15.0 repository
# curl -d 'pkg=slackpkg&repo=noarch' https://slakfinder.org/slfcli.php       => look in repository with only noarch packages
# curl -d 'pkg=slackpkg&repo=no150' https://slakfinder.org/slfcli.php        => look in repository noarch for slackware 15.0
# curl -d 'pkg=slackpkg&repo=mi142' https://slakfinder.org/slfcli.php        => look in repository containing mixed 32 and 64 bit packages for 14.2
# curl -d 'pkg=slackpkg&repo=113' https://slakfinder.org/slfcli.php          => look in repository number 113 (https://slakfinder.org/showrepo.php?repo=113)

note that repo= look in repository category, not in package architecture.
go to slakfinder.org and click on "show all repositories"

regular expressions only works in package name (note, package name is the name of the package excluding version arch build)

or you can mix curl and grep

Code:

# curl -d 'pkg=slackpkg' https://slakfinder.org/slfcli.php|grep -v x86_64

but be sure about which fields you are grepping.