bash scripting for html generation

daveoily · 10-08-2006, 01:16 PM

okies, I'm totally new to the world of scripting, bash is one that seems popular so I thought i'd try and use it to acomplish a fairly simple automated task. Just one problem... I don't have a clue where to start.

Anyhoo...

the idea is as follows, i point a script at a file containing a list of mp3's and the script runs through the list of what's in the file and write's an index of html files relating to said files. I don't want it to point to a.mp3 and b.mp3 rather i want it to point to a.htm and b.htm,

-----example of index.htm-----

<html>

<body>

<center>

<ul>

<li><a href="a.htm">a</a>
<li><a href="b.htm">b</a>

</ul>

</center>

</body>

</html>

-------example ends-----------

and the code in each of those pages should be
generated in such a way that the file is linked to and played automatically upon loading...

------example of a.htm------

<html>

<body>

<embed src="a.mp3" autostart="true">

</body>

</html>

------example ends----------

it shouldn't be THAT difficult to do if I knew what I was doing, but i don't. Any pointers to good guides to acomplish this sort of thing would be greatly appreciated, I may even be barking up the wrong tree with bash and there could be an easier way,

opinions please

MasterC · 10-08-2006, 08:13 PM

Sounds complicated. But, to make it somewhat less complex, you don't technically need the <html> or <body> code with that simple of an html page.

And, you would only be changing 'a' in the embed string, assuming you don't have aa, or aaa... it might also be much less complicated. Something like a variable, definitely won't work, but here's where I'd start:

Code:

cat file.list |xargs -n1 echo $1 > $1.htm

Again, definitely not going to work (at least I don't think it would), but that would be what I'd start out playing with.

I think a better way to ask your question (if variations can exist) would be to actually tell us what your final goal is, maybe something already exists.

Cool

daveoily · 10-13-2006, 11:47 AM

thanks for the reply... dunno what it means, but i'll type it in and see what happens sometime later in the week...

my example is something like this...

http://dr3553dtokill.port5.com/music/music.htm

bits of it keep crashing out, that's what you get for using free servers i guess...

i wanted to make my music collection streamable so i could listen to it from any webbed up terminal in the world, once went to greece and on a day where i was sunburnt i went to a cyber cafe, got hammered and listened to my songs

.

Anyway it occoured to me that it'd all fit together much better if i put it all on one server held at my place (that free webhost only does 15mb chunks

)and if i have it all at my place it saves me uploading and i can tweak the format.

Obviously doing my entire collection by hand would take an eternity, so i'm looking for a way to automate the process.

the songs on there are mostly in wma, but i use the lame encoder now and so i think mp3's would possibly be better suited to this project.

Always thought i needed the html and body tags, that would be helpul if i don't need to put that in

MasterC · 10-13-2006, 02:25 PM

So would:
http://www.gnu.org/software/gnump3d/index.html

Fit the bill of your request?

My above script does the following:
cat file.list |xargs -n1 echo $1 > $1.htm

cat file.list
Will send all the output of file.list (which is a list of your music files) to stdout. If you type just that command into a terminal (assuming file.list exists) it would read the contents of that file back to you, scrolling it quite fast across the terminal. But we are piping it (with the | key) to the next command.

xargs -n1
This will take the previous commands output and feed it, one line at a time, to the next command.

echo $1 > $1.htm
And this is where I think the breakdown of my script begins. What this will potentially (but probably not, it seems like I'm forgetting something here...) do is echo the output from xargs (remember it's only 1 line at a time, so it's a single file each time) into the $1.htm file. The $1 is a variable, and my assumption is that using echo to create that variable will work. So, echo $1 will end up being "echo filename.mp3" and then > $1.htm means to feed it into a file with the name of filename.mp3.htm (again assuming the variable works right). Not a perfect solution, but if it even works up to that point, it wouldn't be too hard to then trim off the last part of the filename using something like sed.

Hopefully gnump3d will work out

Cool

Tinkster · 10-13-2006, 03:33 PM

Create the lot - save as to_html.awk, run like
awk -f to_html.awk <file_list>

Code:

BEGIN{
  print "<html>"
  print "<body>"
  print "<center>"
  print "<ul>"
}
{
  print "<li><a href=\""$1"\">"gensub(/\.mp3/ , "" , g )"</a>"
  # and the other files ... 
  target = gensub(/\.mp3/ , ".html" , g )
  print "<html>" > target
  print "<body>" > target
  print "<embed src=\""$1"\" autostart=\"true\">" target
  print "</ul>" > target
  print "</center>" > target
  print "</body>" > target
  print "</html>" > target
  close( target  )
}
END{
  print "</ul>"
  print "</center>"
  print "</body>"
  print "</html>"
}

Cheers,
Tink

daveoily · 10-18-2006, 12:46 PM

Right, I've popped up just to mention on my further reading, a program called icecast it's similar in concept, but in my early stages of looking at it, doesn't seem to catalogue the stuff, it just makes the files available, which in and of itself isn't bad, it's just i'd have to remember the exact url of the song i wanted.

I've been playing with my windows boxes of late (overclocking old boards and trying not to make them burn up! :S), but I'll get on with a bit of what you've suggested tonight, this for me is more about the learning of scripting than anything else, and if i'm going to learn it, i might as well do something useful that i'm likely to enjoy, thanks for your helpful thoughts.

Tinkster · 10-18-2006, 05:57 PM

Quote:

Originally Posted by daveoily

Right, I've popped up just to mention on my further reading, a program called icecast it's similar in concept, but in my early stages of looking at it, doesn't seem to catalogue the stuff, it just makes the files available, which in and of itself isn't bad, it's just i'd have to remember the exact url of the song i wanted.

I've been playing with my windows boxes of late (overclocking old boards and trying not to make them burn up! :S), but I'll get on with a bit of what you've suggested tonight, this for me is more about the learning of scripting than anything else, and if i'm going to learn it, i might as well do something useful that i'm likely to enjoy, thanks for your helpful thoughts.

If you have any questions as to what which bit awk in the awk script
does, fire away. Or look in awks man-page :}

Cheers,
Tink

daveoily · 10-19-2006, 11:15 AM

If i were to ask you every question on my mind about the command line/linux/bash/python and whatever else is on my mind, you'd get very annoyed very quickly

, thanks for the offer, but i'll try the man page first, if I still don't get it, then I'll ask

daveoily · 10-19-2006, 11:37 AM

This is how dumb I am....

I was looking at an online manual about the basic awk stuff, realising how I'd started this thread, it sounded like I already had a list of the files. I didn't by the way, but it isn't going to be a problem as I realised some stuff, one of the bits in the manual said this...

ls −l | awk '{ print $5 $9 }'

would print out the stuff on lines 5 and 9, so I omitted the $5 and the $9 and just got a list, why didn't I think earlier? I can just type ls and cut and paste the result to get a list of the files in the directory?!?

It would be nice to pipe that stuff automatically to a file, but with my specialist windoze training, I seem to have handled that bit :P

So it looks like awk is another scripting language, and there's gawk, "gnu awk"? there sure are a lot of programs in the linux world, I guess that's a good thing, it does get a little confusing sometimes though.

Tinkster · 10-19-2006, 11:37 AM

I was referring to the way the script I posted works :) ... not offering
the answers to all your questions ;}

Cheers,
Tink

daveoily · 10-19-2006, 12:20 PM

fair enough,

having a little trouble now, doubtless I'm being thick but anyway...

I put in your code, and saved it as to_html.awk, went to the command line, and executed the command...

"awk -f to_html.awk <file_list>"

that is what you intended isn't it Tinkster?

now I get an error message...

dave@ubuntu:~/Desktop/disc1$ awk -f to_html.awk <file_list>
bash: syntax error near unexpected token `newline'

Tinkster · 10-19-2006, 12:25 PM

Ok, my bad. <file_list> is a generic parameter. Don't type the <>
and just pass the list of your mp3 files to it....

Cheers,
Tink

daveoily · 10-19-2006, 01:23 PM

Thanks very much Tinkster, I'm pretty damn impressed

.

Can I give you my impression of what you're up to? Then you can correct me when I'm wrong, and answer my questions, I'll stick to awk, I promise

-------------------

BEGIN{
print "<html>"
print "<body>"
print "<center>"
print "<ul>"
}

-------------------

pretty self explanitory, does the usual stuff for writing the html, getting all the stuff centered, and creating an unordered list (although it turns out alphabetical anyway as it happens because ls gives me an alphabetical by default I guess)

--------------------

this is where I get some stuff, and not other bits...

--------------------
{
print "<li><a href=\""$1"\">"gensub(/\.mp3/ , "" , g )"</a>"

--------------------

so it prints out <li> and the <a href" puts in line 1 ($1) closes the tag with a ">" so this bit ... "gensub(/\.mp3/ , "" , g ) ... I guess somehow types the name of the mp3, while removing the .mp3 at the end yeah? then it closes the tag

--------------------

this I think is where it doesn't quite do what I want it to, I'm no less impressed, it looks to me like you've got the idea of what I want it to do, i.e. generate lots of other html files with the mp3 autostarting within each file...

--------------------

# and the other files ...
target = gensub(/\.mp3/ , ".html" , g )

--------------------

the bit above here generates the filename by removing the .mp3 bit and replacing it with ".html" yes?

--------------------

the following bit looks like it's attempting to create a whole lot of html files presumably with the file name, is that what the target bit is about?

it doesn't, I get a lot of <li>'s followed by the "<embed src=" bits, quite a big file

--------------------

print "<html>" > target
print "<body>" > target
print "<embed src=\""$1"\" autostart=\"true\">" target
print "</ul>" > target
print "</center>" > target
print "</body>" > target
print "</html>" > target

--------------

What confuses me is that in amongst the code, once it's written, I don't get a lot of..

<html>
<body>

</body>
or
</html>

tags written in there, that's what I would have expected to see, but I didn't!

---------------

close( target )
}

--------------

guessing close( target ) fetches the next filename from the list?

--------------

and I guess the bit below closes out the previous file

--------------

END{
print "</ul>"
print "</center>"
print "</body>"
print "</html>"
}

I'm sure there's more to it than that, I've just written about what I think I get, for instance I have no idea what the relevance of "g" is...

So it generates one big file, complete with download links directly to the file, and autostarts the file right next to it, (I'm pretty glad my browser is missing a plugin or I could have been listening to all of my music collection at once!)

One other thing, the script seems to get confused by capital letters, one of my files, "As Long As We're Talking Shelf Life (Kennedy).mp3" comes up in the download link generated as "As", no "Long As We're Talking Shelf Life (Kennedy).mp3" is generated (however, the filename "As Long As We're Talking Shelf Life (Kennedy).html" is generated correctly

).

There's probably others, but I felt I had to mention it, in case there's an easy solution (another script to de-capitalise (if that's even a word!) everything in my list perhaps?

Thanks again it's a great start.

MasterC, I'll be looking at gnump3d tonight, I think I'll get the time at work

Tinkster · 10-19-2006, 01:36 PM

DOH. My bad. In the code I posted there's ONE > missing.

Code:

print "<embed src=\""$1"\" autostart=\"true\">" > target

Try that.

It worked here ;}

Cheers,
Tink

daveoily · 10-22-2006, 03:01 PM

I found that bit

, and the html it produced

I wasn'tlooking in the right place! :S, it did contain though only the following....

<html>
<body>
</ul>
</center>
</body>
</html>

in all of the html files, I've had a bit of a play with the code, apologies if it appears a bit hacked together, but that's exactly what it is! :S

it now reads...

BEGIN{
print "<html>"
print "<body>"
print "<center>"
print "<ul>"
}
{
print "<li><a href=\""$1"\">"gensub(/\.mp3/ , "" , g )"</a>"
# and the other files ...
target = gensub(/\.mp3/ , ".html" , g )
print "<html>" > target
print "<body>" > target
print "" > target
print "<center>" > target
print "" > target
print "<a href=\""$1"\">"gensub(/\.mp3/ , "" , g )"</a>" > target
print "" > target
print "" > target
print "<embed src=\""$1"\" autostart=\"true\">" > target
print "</ul>" > target
print "</center>" > target
print "</body>" > target
print "</html>" > target
close( target )
}
END{
print "</ul>"
print "</center>"
print "</body>"
print "</html>"
}

this makes the html files come out a little more like I want them to, they now read for a file called "_ Allegro.mp3" in a file called "_ Allegro.html"...

<html>
<body>

<center>

<a href="_">_ Allegro</a>


<embed src="_" autostart="true">
</ul>
</center>
</body>
</html>

I think I made a mistake, the script isn't having problems with capital letters, it has problems when it encounters spaces! I have no idea how to correct that, also, in the produced html files I would like it to link to the mp3 file, as you can see, above, it just says <a href="_"> not <a href="_.mp3"> which still wouldn't link to anything, but it'd be a step in the right direction

So I just thought about that bit, it appears that everything works fine, as long as there are no space in the file name, for instance, for a file called "10___ORANGES_AND_LEMONS_AGA.mp3"

the html reads...

<html>
<body>

<center>

<a href="10___ORANGES_AND_LEMONS_AGA.mp3">10___ORANGES_AND_LEMONS_AGA</a>


<embed src="10___ORANGES_AND_LEMONS_AGA.mp3" autostart="true">
</ul>
</center>
</body>
</html>

which is just what I was looking for.

The only other thing that needs tweaking now would be that the file produced in the terminal when the script runs, would idealy link to the produced html file as opposed to the mp3.

any ideas?