Problem using piped data as input
Neither of the following commands are working and I don't know why. Any ideas, anybody?
Code:
stuart@stuart:~/$cat showthread.php.html | grep -o "http://www.megaupload.com/[^\"< ]*" | uniq | head -n3 | plowdown - Code:
stuart@stuart:~/$ awk NR==$x Musiclist.txt | cut -d/ -f6 | basename $1 .mp3 Code:
stuart@stuart:~$ apt-cache showpkg bash |
The second command looks wrong, in a couple ways. Post some sample lines from Musiclist.txt, and explain what you want the command to do, and we can help you :)
For the first one, I don't know what "plowdown" is but I suspect it doesn't understand "-" to mean standard input. Try redirecting the output from the rest of the pipeline into a file, then running plowdown on that. i.e., Code:
long | list | of | stuff | head -n 3 > /tmp/somefile.txt |
Thanks for the quick reply.
Plowdown is an app that invokes wget to download files from filehosting sites like Rapidshare, and the like. http://www.webupd8.org/2009/09/downl...egaupload.html In both examples my primary concern is to solve the problem of using pp's which I cannot get to work. The following command - though I don't like creating files where piping will do - I had to break out of with ^C. Code:
cat showthread.php.html | grep -o "http://www.megaupload.com/[^\"< ]*" | uniq | head -n3 > downloads.txt; plowdown downloads.txt Code:
/mnt/sda1/Music/Belly/06 - The Bees.mp3 |
For your second command (though I'm not exactly sure what you were doing with the awk command you had), try this:
Code:
awk '{gsub(".*/|\\.mp3$","");print}' file Code:
/mnt/sda1/Music/Belly/06 - The Bees.mp3 Also note that I'm not sure how in your first post, the two commands you're running are supposed to be tied together... I'm thinking you want this awk command (in a pipeline, without the filename) to come after the `head -n3` but before the `plowdown`, so my apology if this isn't right - and if not, please explain some more. |
The $1 isn't doing what you want because it needs to run from inside a script. Normally you'd use it when you're putting your bash into a separate file with a "#!/bin/bash" line at the top, but if you really want to use it on the command line you can do things like this:
Code:
$ bash -c 'echo $2' unused a b c Code:
awk NR==$x Musiclist.txt | cut -d/ -f6 | basename "`cat`" .mp3 GrapefruiTgirl: The awk command was to get a specific line from the file. It's equivalent to: Code:
awk "NR==$x { print }" Musiclist.txt |
GrapefruiTgirl
The two commands are linked in my mind, not because i wish to use them in conjunction with one another, but because in both cases I want use the piped output on the last command in the chain. So: ...head -n3 | plowdown {piped data} ...cut -d/ -f6 | basename {piped data} .mp3 It's not a case of getting the commands to work by any means, but in this particular way. I hope this makes sense. |
OK, I'm still a bit fuddled, but maybe `xargs` can help?
Code:
...head -n3 | xargs -I{} plowdown {} Code:
...cut -d/ -f6 | xargs -I{} basename '{}' .mp3 |
Quote:
Is this plain erroneous then? Code:
curl http://some-website.com/page.html | \ And yes, GrapefruiTgirl, you're solution is far more elegant than mine! |
Quote:
|
Using the -0 option to xargs can solve some (or all - it works for me) of those punctuation problems, but maybe not extreme cases unless the input is "cleaned", and you don't want to do that because then the filenames may be wrong. Here's an example of "still works for me":
Code:
root@reactor: echo "/file/path/he'-l?<href:\!lo.mp3" | xargs -0 -I{} basename "{}" .mp3 It seems you have all the necessary pieces here to do what you need, it's a matter of putting them together the right way. At this point in time, I don't know precisely what command(s) you are running or what exact problem(s) remain, i.e. have you come up with a new command based on some of the suggestions in here, or are you still wondering what to do next? Show us the current command you are using, and explain what problem(s) are still occurring. Basically, bring us up to speed again on the current state of the problem: Have you modified your commands any, or are we still at post #1 and looking for a solution? :) |
Quote:
Using xargs as you do maybe worth exploring as a means to circumventing aluser's solution, which calls for further inventiveness. I find that the -0 option fails in other instances, however -see: Code:
stuart@stuart:~$ cat Musiclist.txt | xargs -0 -I{} basename "{}" .mp3 |
I don't have a solution for either question, per se. The reason I'm posting though is to let you know that the plowdown program is a straight Perl script.
I followed the link provided, downloaded the tarball and examined the source code. I wanted to verify whether plowdown could read download URLs directly from stdin. I'm not a Perl coder (unless I have to be), but it does not appear to support that. Though, I'm sure someone with Perl experience (or someone who wants to learn) could add that ability. The plowdown command is actually just the "download.sh" file located in the src directory of the tarball. So, in short, the "advanced" command given as an example on the linked page above is "too advanced" for plowdown--unless someone adds that capability. |
Thanks Dark_Helmet for that information. If the program does not support input on stdin, OP needs to make either a loop, and run plowdown on each file, OR string together a list (a sequence) of files and give them all at once on the plowdown commandline.
I'd also like to suggest further: based on what understanding of the thread so far, this is not an issue of positional parameters, but more of an issue of some data not coming out the end of a pipeline in the usable format we want; I'm still not 100% clear on how the two commands in the OP actually relate to each other (but no matter), but maybe a thread title (click Edit -> Advanced on post #1) edit would be appropriate to better describe the issues. If I were to change the thread title, even though there seem to be two similar but distinct (and related) problems here, I'd use maybe something like: "problem piping grepped URLs into plowdown (a perl downloader)" Plus, here's a take on this command: Code:
curl http://some-website.com/page.html | \ |
I realised that plowdown made use of wget, but not that was a perl script, so thankyou Dark_Helmet.
I have got both commands to work now, using various solutions posted here. I quite like aluser's suggestion of using the cat command ("`cat`"), as it deals with my ostensible issue of parsing piped data to a command. And yes GrapefruiTgirl, I can see that using two examples, one using pp's, the other not, confused this one problem. These solutions worked fine: Code:
stuart@stuart:~/$ awk NR==$x Musiclist.txt | cut -d/ -f6 | basename "`cat`" .mp3 I do have another issue but that's for another post. Thankyou to all. |
All times are GMT -5. The time now is 02:40 PM. |