Common interface to slackbuilds resources
 

Hi folks,

slackbuilds resources are ftp://, http(s):// git:// file:// the problem is to create engine to query resource - say to search for some particular slackbuild. It would be much easier to use the same interface with one of those as backend offering some natural actions. Say I have bash script it resolves dependencies - but at this moment it works only with http(s) if web page provides search engine. Here are people having troubles with sbo scripts on -current. For -current there are only two http(s) resources and both are not of much use due to how search engines work. I don't know much about git - how to use it, for me simplest is to use file:// from archived scripts. But is completely unsatisfactory - it would be much easier to have interface with say plugin to serve particular resource. Someone interested in app to maintain/use/whatever slackbuilds would just rely on this interface. System of data maintain allowing searching and other things. Real action is done via plugin depending on what resource is accessible.

If I understand correctly, you're suggesting someone make a service which takes a package name and version, and returns the upstream's sources as a tarball, so that fetching the tarball uses a uniform interface for all packages?

The simplest implementation would be a static cache, exposed via http, with the appropriate organization. That would be useful for when the tarball disappears from upstream as well.

That doesn't seem hard (a full cache of 14.2 upstream source tarballs is only about 82GB), but if it became popular it could have nontrivial bandwidth requirements.

Before I get ahead of myself, though, please verify I am interpreting your post correctly.

There is already an archive similar to this idea - sbosrcarch. Which houses all the sources linked from SBo .info files (barring a handful of files we can't obtain) in a structure that matches the SBo tree, or by the MD5 sum your source may have.

Check it out: https://slackware.uk/sbosrcarch/

I thought about fetching information about script(s). For example you can use slackbuilds web page and search engine on that web page. Or you can download tarball containing all scripts extract it on hard drive an eg. to use find utility. But problem is these are not the same. For web in my script I used lynx now I use gawkinet, for files say find or something more sophisticated. My script serves well if I use web page but is useless to work with files on drive. I would have to put additional function inside the script to work with files. Something like that: if uri=http then use gawkinet if uri=file use find. But there are other places to look for ftp, git, say someone would create data base - say sql - so I would have to add to my script separate search function for each resource (if searchable). Now I came to solution to have unique service and query this only service - kind of protocol. So I imagined this as client - server configuration. Server backend is one or more resources. It sounds little crazy - why to use server to find file on hard drive - but server should offer little more than that. eg. it should be capable to return full dependency resolution - using standard data serialization format for example. Server essentially works via plugins - why plugins? They are written by different people - the same resource may have several plugins to work with. Some are with files - other with sqls. Some people know git very well. Plugins can be as simple as 100 line of code. It is about functionality not complexity. It is not one in all app. Say request can be send to inetd - or more complicated - standalone daemon. If protected well-enough can be queried remotely. I mean people around are willing to share their own scripts. But without submitting to slackbuilds. Plugins cause people have own preferences about programming language: perl, python whatever. One plugin is only used with the same type resource. Better keep that. Plugin for file://, another for git://, another for sql:// (if any), another for ftp://, http:// etc. I don't know did I make myself clear enough? Say session starts from querying sever capabilities. It depends on resource and plugins. It can be really sophisticated but is up to plugin developer. And of course limited by resource. Server reports what kind of information offers. Then you can query about scripts - find script, dependencies and whatever server offers. You can export resource. Use own-defined. Does not matter as long as there is plugin. It is simpler than sounds.

We already have reverse dependency checks here: https://slackbuilds.org/advsearch.php and for dependency checking, we have other tools to help, such as sbopkg.

Yeah I see now - it is under Advanced search. But I think it should be rather more exposed. For me advanced search is rather search wlong with some additional fields, keywords. But why it is limited up to depth level 10? There is no need for such limit. The another limit one need to provide exact slackbuild name. I tried gis - it returned nothing, gisto - returned 0 dependencies. Also it would be nice to add serialization - to establish correct build order. But perhaps you will see now my point. Do you mind to post php script which is used? I may have look at it - maybe I will able to improve it a little. To remove depth limitation and to provide data serialization.

Oh, I see because you provide graphic tree representation depth limit is necessary. I have a look for haskell-ansi-terminal. That's quite a lot of dependencies. But still they all can be put in serial file - 300 lines?

My second thought about dependency information about slackbuilds.org web page. I propose to add this engine to pages containing information about script. 1-first level dependencies are already listed on web page containing information about script. what is not listed are deepest dependencies. I think it would be more convenient to visitor to have dependency search for script just on script web page as function to be called - say by clicking button - there is no need to provide script name. For example
on web page https://www.slackbuilds.org/reposito...AtomicParsley/ - advsearch.php script is called with AtomicParsley - but to tell the truth - for scripts with small number of dependencies they as well can be listed directly without action of visitor. Say if there are few 2,3 level dependencies - then as well you can list them directly on web page. Without doing any search. I mean during page is opening it runs script - gathers information and send this to browser. It would improve functionality in my opinion.

please note that it's reverse dependency, not N-level dependency for a package

What I was thinking that information seen on the first picture as output of Advanced Search can be seen directly on script page - the second picture sat at bottom line or indirectly as output of visitor request. But no as search but say something like show dependencies. Cheat is that yo can essentially rerun your script for all packages and keep this information statically - as content of web page.

no, the rule for SBo is clear, only showing first level dependency
there are lots of third party tool that help you with solving dependencies

some scripts have LONG list of dependencies, like letsencrypt which has 44 deps. You want them to be listed in a single page?

now you will see perhaps may point. Every developer needs today somehow by itself solve the problem of dealing with scripts. Finding information, establishing build order. It would be much easier to use common protocol to query for such things. Protocol can be easily extended adding new features. While rewriting program is just frustrating. Also having protocol would allow people to export say data contained in scripts into sql data base - say as BSD ports.
Of course not. Only reasonable part. But there can be information eg. how deep is build tree. Also having serialization engine it should be possible to send on visitor request information about all dependencies. Without displaying on web page.

I wonder when you'll actually start writing this program instead of just arguing about it.
My take? All the information about nested dependencies is already contained in the .info files from slackbuilds.org. Several 3rd-party tools are already available that do smart things with this information.

There are already tools to list the dependencies of a random SBo package. Am I missing something?

Yeah, you right. Need go back to my script and use something less artificial than bashing prototype. However it has some advantages. It require to run only: gawkinet, curl, tar and common utilities like grep. You can run it on very thin installation. For python or perl script you need one or both installed. Happy to be removed from ignore list.

It almost sounds like you're looking for an API to query for SBo. Like you throw the program name at it and it outputs the downloads, dependencies, etc in a parseable output.

I don't know if SBo would consider planning something like this, since almost all tools just work with the git repo itself and they can write tools to do all that work for them. This would be writing code on the chance that someone (maybe you) writes tools that would interact with that API.