[SOLVED] Please write me a shell script!

alberich · 04-05-2016, 07:10 PM

Hello, I'm Alberich from Europe/Germany and I'm new to this forum.

I need to write me a bash script in OpenSuse, to convert/sort me some sequential data into an xml/gpx-file including a few needed tags.

Though I would like to be a powerful shell wizard unfortunately I'm not. Although I'm willing - to a certain (very limited) extent - to try hard.

As my primary intention is to convert the data I am asking you to kindly share your skills with me. If you like.

The raw data is present in two lines in a text file and goes like this:
[46.46480,8.02372],[46.46490,8.02383], ...
3026.00,3025.00, ...

The output is supposed to be like this:

<trkpt lat="46.46480" lon="8.02372">
<ele>3026.00</ele>
</trkpt>
<trkpt lat="46.46490" lon="8.02383">
<ele>3025.00</ele>
</trkpt>
...

So I guess the script needs to load the text file, or the two lines as seperate strings.

Then it should from the first line drop the brackets, read two numbers between commas into variables, put the values in an output file including the neccessary tagging.
Then read from the second line or string one number, and append it to the output with the neccessary tagging.
Then iterate from the points where it stopped reading in the lines last time.

I did some basic programming in Basic and Turbo Pascal, but even there I couldn't reproduce the necessary code in a fair amount of hours, I guess.

The reading from the two lines, as well as the output and tagging cold probably be done in two countered for loops (for processing each line: coordinate (possibly containing another sub-loop for: latitude, longitude), elevation) which would be nested in one primary loop, that refered to a complete "set" of output data (lat, lon; ele).

Is anybody willing to generate the code for me? Thank you.

alberich · 04-05-2016, 07:50 PM

I will in the first place try to generate the script myself.

TB0ne · 04-05-2016, 07:53 PM

Quote:

Originally Posted by alberich

Hello, I'm Alberich from Europe/Germany and I'm new to this forum.

I need to write me a bash script in OpenSuse, to convert/sort me some sequential data into an xml/gpx-file including a few needed tags.

Though I would like to be a powerful shell wizard unfortunately I'm not. Although I'm willing - to a certain (very limited) extent - to try hard.

As my primary intention is to convert the data I am asking you to kindly share your skills with me. If you like.

The raw data is present in two lines in a text file and goes like this:
[46.4648070,8.0237290],[46.4649040,8.0238320], ...
3026.00,3025.00, ...

The output is supposed to be like this:

<trkpt lat="46.4648070" lon="8.0237290">
<ele>3026.00</ele>
</trkpt>
<trkpt lat="46.4649040" lon="8.0238320">
<ele>3025.00</ele>
</trkpt>
...

So I guess the script needs to load the text file, or the two lines as seperate strings.

Then it should from the first line drop the brackets, read two numbers between commas into variables, put the values in an output file including the neccessary tagging.
Then read from the second line or string one number, and append it to the output with the neccessary tagging.
Then iterate from the points where it stopped reading in the lines last time.

I did some basic programming in Basic and Turbo Pascal, but even there I couldn't reproduce the necessary code in a fair amount of hours, I guess.

The reading from the two lines, as well as the output and tagging cold probably be done in two countered for loops (for processing each line: coordinate (possibly containing another sub-loop for: latitude, longitude), elevation) which would be nested in one primary loop, that refered to a complete "set" of output data (lat, lon; ele).

Is anybody willing to generate the code for me? Thank you.

No.

Read the Question Guidelines link. We will HELP you, but will NOT do your work/homework for you. Especially after saying that you're will to try to a 'limited extent' to try. Asking for handouts isn't a good thing, especially after saying you're not even willing to try.

Keith Hedger · 04-05-2016, 07:53 PM

No one is going to write a script for you, try yourself if you get stuck on a particular bit post and ask a apecific question and show what you have tried and where you are failing.

To get you started try getting hold of the advanced bash scripting guide ( abs ) it should be in your distros repo or you can do a quick google search, don't be put off by the 'advanced' in the title it has basic conceots as well.

Keith Hedger · 04-05-2016, 07:54 PM

Quote:

Originally Posted by TB0ne

No.

Read the Question Guidelines link. We will HELP you, but will NOT do your work/homework for you. Especially after saying that you're will to try to a 'limited extent' to try. Asking for handouts isn't a good thing, especially after saying you're not even willing to try.

You type faster than me!

grail · 04-05-2016, 08:23 PM

I do love how the OP went to the trouble of saying that he pretty much won't be trying ... gotta love how some people think.

alberich · 04-05-2016, 09:13 PM

I am active in a lot of forums, and I give handouts when I feel like it. Because I like to talk of, and apply the knowledge of the things that I am passionate about.

That's why I didn't mind to ask the people who know.

If someone seems unwilling to try that generally doesn't help, yes.

I didn't mean to in the first place... Anyway I know that I can probably do this coding on my own by the help of code snippets from google, it will be a nasty piece of code.

Now I'm set for it anyway.

Maybe it won't work out, then I'll ask you for your help again.

alberich · 04-05-2016, 11:11 PM

Well I already got closer as I thought I could in the beginning. The tagging structure comes out correct.

I still cannot get it to work though. To me it seems as it can't get the arrays defined correctly.

In each possible spot it fills in just values from the second line of data (second array, elevation data).

Can someone make anything of this error?

What I tried:
- Add a "declare -a" or "declare -A" to the definition of the array. That rather seems to worsen everything.
- Delete the elevation data line from the input file: Strangely the script fills every possible spot out with position data.

That I cannot understand, because it is my opinion that I programmed it to put specific data in very specific places. But it doesn't seem to mind at all.

(The brackets from the data input I deleted manually)

Code:

#!/bin/bash
FILE=data
k=0
while read line;do
array=(`echo $line | sed 's/,/\n/g'`)
array2=(`echo $line | sed 's/,/\n/g'`)       
done < $FILE
for (( start = 0; start <= 500; start++ ))
do
        if [[ $((start % 2)) -eq 0 ]]; 
   then
        printf '<trkpt lat="' >> file.gpx
        printf ${array[start]} >> file.gpx
        printf '" ' >> file.gpx
       
    else       
        printf 'lon="' >> file.gpx
        printf ${array[start]} >> file.gpx
        printf '"> %s\n <ele>' >> file.gpx
        printf ${aray2[k]} >> file.gpx
        printf '</ele> %s\n </trkpt>  %s\n' >> file.gpx
        ((k++))
        fi      
done

dugan · 04-06-2016, 12:41 AM

Do you really need to do it in BASH and not in, uh, almost anything else?

grail · 04-06-2016, 12:45 AM

Good job on having a go

Please use [code][/code] tags around your code / data in the future to maintain formatting.

Personally I am not sure I understand the input data file structure? Are you saying there are only 2 lines in the file or multiple?

Looking at your current script:

1. Your while loop will fill both array and array2 with identical data

2. In addition to the above, only the last line read from the input file will be in both arrays

3. Deleting the elevation data and the square brackets will completely change the problem, the biggest issue I see is now you will not know which long / lat go together

4. Why the 500 limit in the for loop? What do you know about the input that you have not shared that would indicate this number?

5. How does the variable 'k' factor in to the solution??

6. Investigate here documents ... this will remove the need for multiple printf / echo statements, although I am not convinced all the data could not be in a single printf / echo

7. Instead of redirecting to the same file several times, either use the aforementioned here document or redirect the output from the for loop itself

Hope some of that helps

alberich · 04-06-2016, 07:33 AM

Hello dugan,

I think maybe LibreOffice Calc and some text search & replace with geany would do. But I think shell scripting is a good option and it's a basic competence that would be good to know. My aim would be to easily repeat this processing from time to time.

Hello grail,

thanks for the detailed assitance.

The data is always present in two text lines:
[lat1],[lon1],[lat2],[lon2], ...
[ele1],[ele2], ...

I already had some doubt about 1. (/2.) and the function of the

Code:

while read line; do

I think that's the crux. I think I know how to solve it.

3. I deleted the elevation only for testing! It's supposed to be there.

latitude/longitude data is only recognised by the fact, that there is always a sequential pair of data! The code has no further robustness.

Manual deletion of the brackets is more efficient for me than finding out how to do in code. The same goes for the 4. manual iteration switch which I can manually fit to the maximum size of the raw GPX-Track data. I can find that later out if care.

5. One complete set of data consists of [lat1],[lon1],[ele1]
Variable "k" in the procedure serves to adress the second array to retrieve the according elevation data that belongs to a lat/lon pair (adressed in the first array by variable "start" distinct by even/odd number).

Thanks also for hints 6 and 7 concerning the quality/efficiency.

alberich · 04-06-2016, 10:33 AM

I could get the script written, and it works. Which is very nice!

Thanks a lot for your help. In wide parts I copied snippets. Many things I do not understand, like why (()), [[]] and ++, let alone RegExp and so forth.

It was not possible for me, to make the code much more elegant. I could not find out, how to concatenate strings and variables for use with printf. Also how to use the here document in that context. After over an hour of research I leave it be. Because it does what it should.

Code:

#!/bin/bash
FILE=data
k=0
l=0

# Add GPX file structure #
cat <<-EOF > file.gpx
<?xml version="1.0" encoding="UTF-8" standalone="no" ?>
<gpx xmlns="http://www.topografix.com/GPX/1/1">
<trk>
<name>gps-track</name>

<trkseg>
EOF
###


# Read data #
while read line;do
	((l++))
	if [ $l = 1 ]; then
		array=(`echo $line | sed 's/,/\n/g'`)
	fi
	if [ $l = 2 ]; then
		array2=(`echo $line | sed 's/,/\n/g'`)
	fi       
done < $FILE

# Write latitude #
for (( start = 0; start < ${#array[@]}; start++ ))
do
        if [[ $((start % 2)) -eq 0 ]]; 
   then
		printf '<trkpt lat="' >> file.gpx
		printf ${array[start]} >> file.gpx
		printf '" ' >> file.gpx

# Write longitude and elevation #    
    else       
        printf 'lon="' >> file.gpx
        printf ${array[start]} >> file.gpx
        printf '"> %s\n   <ele>' >> file.gpx
        printf ${array2[k]} >> file.gpx
        printf '</ele> %s\n</trkpt>  %s\n' >> file.gpx
        ((k++))
        fi      
done

# Add GPX file structure
cat <<-EOF2 >> file.gpx
    </trkseg>
  </trk>
</gpx>
EOF2
###

grail · 04-06-2016, 11:19 AM

Good job on completing it

I do agree with dugan that there are probably more appropriate languages, but this would be my crack at a bash one:

Code:

#!/usr/bin/env bash

c=0

cat > file.gpx <<EOF 
<?xml version="1.0" encoding="UTF-8" standalone="no" ?>
<gpx xmlns="http://www.topografix.com/GPX/1/1">
	<trk>
		<name>gps-track</name>

		<trkseg>
EOF

while IFS="[]," read -a line
do
	(( c++ == 0 )) && arr_ll=(${line[@]}) || arr_e=(${line[@]})
done< data

for (( i = 0; i < ${#arr_e[*]}; i++ ))
do
	(( f = i * 2 ))
	cat <<-EOF
	      <trkpt lat="${arr_ll[f]}" lon="${arr_ll[f+1]}">
	        <ele>${arr_e[i]}</ele>
	      </trkpt>
	EOF
done>>file.gpx

cat >> file.gpx <<EOF2
		</trkseg>
	</trk>
</gpx>
EOF2

Mine assumes the data is still in the original format in the file, ie. [] and , are in use

dugan · 04-06-2016, 01:15 PM

Quote:

Originally Posted by alberich

Hello dugan,

I think maybe LibreOffice Calc and some text search & replace with geany would do.

I wasn't thinking of using a scriptable text editor or spreadsheet, although that's a good idea. I was thinking of an actual programming language with JSON and XML libraries, as you're transforming (what looks like) JSON to XML.

alberich · 04-06-2016, 01:48 PM

@grail:
Thank you for this very lean and condensed approach. It works.

Unfortunately I cannot read the code ...completely. So my next script will be more like this one i wrote

Well, one won't learn proper coding en passant. Like anything else.

@dugan:
Thats interesting to know, I was going to ask anyway. Anyway I don't know any other language or IDE.

Yes, the data is Javascript generated. The GPS track is projected over a map. In the page source there is some jQuery containing that gpx-track data.