LinuxQuestions.org
Visit Jeremy's Blog.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 01-22-2020, 09:20 AM   #1
taylorkh
Senior Member
 
Registered: Jul 2006
Location: North Carolina
Distribution: CentOS 6, CentOS 7 (with Mate), Ubuntu 16.04 Mate
Posts: 2,127

Rep: Reputation: 174Reputation: 174
Simulate SQL inner join in bash?


I am using a Raspberry Pi as a gateway, router, DHCP and VPN connection sharing box for my home LAN. I have a systemd job which connects the VPN, monitors it and reconnects if it drops. Recently I upgraded my script so that I could control the operation of the VPN by posting flag files to a directory on the Pi. I then wrote a script which I run on my workstation which will post the appropriate flag files to the Pi using ssh. Works great. For my next trick I want my workstation script to interrogate the Pi to see which flag file is present and report what more the VPN is in before I make a change.

The flag files are:

(none) - default, connect to "fastest" VPN server using ProtonVPN tool
vpnoff - disconnect the VPN and leave it off
manual - allow me to specify the VPN server to connect to
good - connect to a random server from a list of "good" servers (which do not offend ebay, amazon etc.)

I can determine the flag file present, if any, by
Code:
=( $(ssh magic ls /etc/protonvpn/) )
If the return code is 1 I can assume that no file is present and I am in default mode. If not I need to compare the name of the file present against my list of vpnoff, manual, good.

Granted I only need to compare name of the file which I have found against three possible values. I can do this with an if elif fi structure with some greps but I am wondering if there is a cooler way Sort of like an inner join in SQL. Just a learning experience now but it might come in handy if I have to look against a larger list some day.

Any suggestions?

TIA.

Ken
 
Old 01-22-2020, 01:02 PM   #2
individual
Member
 
Registered: Jul 2018
Posts: 315
Blog Entries: 1

Rep: Reputation: 233Reputation: 233Reputation: 233
Quote:
Originally Posted by taylorkh View Post
I am using a Raspberry Pi as a gateway, router, DHCP and VPN connection sharing box for my home LAN. I have a systemd job which connects the VPN, monitors it and reconnects if it drops. Recently I upgraded my script so that I could control the operation of the VPN by posting flag files to a directory on the Pi. I then wrote a script which I run on my workstation which will post the appropriate flag files to the Pi using ssh. Works great. For my next trick I want my workstation script to interrogate the Pi to see which flag file is present and report what more the VPN is in before I make a change.

The flag files are:

(none) - default, connect to "fastest" VPN server using ProtonVPN tool
vpnoff - disconnect the VPN and leave it off
manual - allow me to specify the VPN server to connect to
good - connect to a random server from a list of "good" servers (which do not offend ebay, amazon etc.)

I can determine the flag file present, if any, by
Code:
=( $(ssh magic ls /etc/protonvpn/) )
If the return code is 1 I can assume that no file is present and I am in default mode. If not I need to compare the name of the file present against my list of vpnoff, manual, good.

Granted I only need to compare name of the file which I have found against three possible values. I can do this with an if elif fi structure with some greps but I am wondering if there is a cooler way Sort of like an inner join in SQL. Just a learning experience now but it might come in handy if I have to look against a larger list some day.

Any suggestions?

TIA.

Ken
I don't think "inner join in SQL" means what you think it means. You're just "querying" the protonvpn "table", e.g. "select mode from protonvpn." At any rate, you could also use a case statement.
 
Old 01-23-2020, 07:16 AM   #3
taylorkh
Senior Member
 
Registered: Jul 2006
Location: North Carolina
Distribution: CentOS 6, CentOS 7 (with Mate), Ubuntu 16.04 Mate
Posts: 2,127

Original Poster
Rep: Reputation: 174Reputation: 174
Thanks individual,

If I have two tables, one with the flag file name and the preference

table a:

Code:
[flag           preference
----           ----------
vpnoff         VPN Disconnected
manual         Specify desired server
good           Connect to "Good" server
and a second containing the name of the flag file which is present

table b:

Code:
flag
----
manual
select a.preference from a, b where a.flag = b.flag

would return the value "Specify desired server" which is what I wish to display. I guess what I really want is a lookup functionality. I did what I needed with an if elsif structure
Code:
function current_pref {
# server alias is magic, ssh keys facilitate connection

flagfile=( $(ssh magic ls /etc/protonvpn/flags/) )

# -z checks if variable is empty
if [ -z $flagfile ]; then
    pref='Connect to the "Fastest" server'
elif [ $flagfile = "vpnoff" ]; then 
    pref="VPN Disconnected"
elif [ $flagfile = "manual" ]; then 
    vpnserver=`ssh magic cat /etc/protonvpn/server`
    pref="Specify the desired server = "${vpnserver}
elif [ $flagfile = "good"  ]; then 
    pref='Connect to a "Good" server'
fi

echo "Current preference = " $pref
}
This works fine for my purpose. I guess I need to do some searching for lookup functionality. Something in the back of my dusty brain is nagging me that there is a more efficient way to do this which would be useful in the event of a larger population to search or the need to do the deed repetitively. Something like loading the "table" of combinations into an array... Oh well.

Thanks again,

Ken
 
Old 01-23-2020, 07:32 AM   #4
berndbausch
LQ Addict
 
Registered: Nov 2013
Location: Tokyo
Distribution: Mostly Ubuntu and Centos
Posts: 6,316

Rep: Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002
There is a join command.
 
Old 01-23-2020, 03:59 PM   #5
taylorkh
Senior Member
 
Registered: Jul 2006
Location: North Carolina
Distribution: CentOS 6, CentOS 7 (with Mate), Ubuntu 16.04 Mate
Posts: 2,127

Original Poster
Rep: Reputation: 174Reputation: 174
Very interesting. Thanks berndbausch.
 
Old 01-23-2020, 10:47 PM   #6
individual
Member
 
Registered: Jul 2018
Posts: 315
Blog Entries: 1

Rep: Reputation: 233Reputation: 233Reputation: 233
Quote:
Originally Posted by taylorkh View Post
Thanks individual,

If I have two tables, one with the flag file name and the preference

table a:

Code:
[flag           preference
----           ----------
vpnoff         VPN Disconnected
manual         Specify desired server
good           Connect to "Good" server
and a second containing the name of the flag file which is present

table b:

Code:
flag
----
manual
select a.preference from a, b where a.flag = b.flag

would return the value "Specify desired server" which is what I wish to display. I guess what I really want is a lookup functionality. I did what I needed with an if elsif structure
Code:
function current_pref {
# server alias is magic, ssh keys facilitate connection

flagfile=( $(ssh magic ls /etc/protonvpn/flags/) )

# -z checks if variable is empty
if [ -z $flagfile ]; then
    pref='Connect to the "Fastest" server'
elif [ $flagfile = "vpnoff" ]; then 
    pref="VPN Disconnected"
elif [ $flagfile = "manual" ]; then 
    vpnserver=`ssh magic cat /etc/protonvpn/server`
    pref="Specify the desired server = "${vpnserver}
elif [ $flagfile = "good"  ]; then 
    pref='Connect to a "Good" server'
fi

echo "Current preference = " $pref
}
This works fine for my purpose. I guess I need to do some searching for lookup functionality. Something in the back of my dusty brain is nagging me that there is a more efficient way to do this which would be useful in the event of a larger population to search or the need to do the deed repetitively. Something like loading the "table" of combinations into an array... Oh well.

Thanks again,

Ken
Another way is using an associative array that maps file names to actions. Something like this:
Code:
#!/bin/bash

flagfile="$(ssh magic ls /etc/protonvpn/flags/)"

declare -A options
options[vpnoff]='VPN Disconnected'
options[manual]="Specify the desired server = $(ssh magic cat /etc/protonvpn/server)"
options[good]='Connect to a "good" server'

[ -n "$flagfile" ] && echo ${options["$flagfile"]} || echo 'Connect to the "Fastest" server'
 
1 members found this post helpful.
Old 01-24-2020, 07:43 AM   #7
taylorkh
Senior Member
 
Registered: Jul 2006
Location: North Carolina
Distribution: CentOS 6, CentOS 7 (with Mate), Ubuntu 16.04 Mate
Posts: 2,127

Original Poster
Rep: Reputation: 174Reputation: 174
Thanks again individual,

Quote:
associative array
I think that is the term I was looking for. I have no idea if this would be more efficient in my case of picking from 4 possibilities as opposed to an if elif structure or a case statement. Perhaps I ran it in a loop for 100 million iterations and timed it On the other hand I can see the advantage of loading some associations into an array ONCE and then referring to it multiple times in a script, perhaps for various purposes.

My script is working and now I have another tool in my bash toolbox.

Thanks again,

Ken
 
Old 01-24-2020, 08:08 AM   #8
boughtonp
Senior Member
 
Registered: Feb 2007
Location: UK
Distribution: Debian
Posts: 3,645

Rep: Reputation: 2564Reputation: 2564Reputation: 2564Reputation: 2564Reputation: 2564Reputation: 2564Reputation: 2564Reputation: 2564Reputation: 2564Reputation: 2564Reputation: 2564
Quote:
Originally Posted by taylorkh View Post
I have no idea if this would be more efficient in my case of picking from 4 possibilities as opposed to an if elif structure or a case statement.
Focus on readability unless you've got a specific proven performance issue. (You haven't.)


Quote:
Perhaps I ran it in a loop for 100 million iterations and timed it
That's a common thing inexperienced programmers do, but unless it's a real-world application of the code, it's a meaningless test that is unlikely to reflect the performance in the wild.


(In situations where performance matters, you probably want to be looking at compiled languages, which have dedicated performance profiling tools to monitor whilst the software runs and compiles a list of the slowest parts, so the code that has the greatest impact can be optimised.)

Last edited by boughtonp; 01-24-2020 at 08:11 AM.
 
Old 01-24-2020, 08:40 AM   #9
taylorkh
Senior Member
 
Registered: Jul 2006
Location: North Carolina
Distribution: CentOS 6, CentOS 7 (with Mate), Ubuntu 16.04 Mate
Posts: 2,127

Original Poster
Rep: Reputation: 174Reputation: 174
Thanks for the reply boughtonp,

I DO like readability and my scripts and other programming files are generally VERBOSE with comments. An if elif or case construct with the data hard coded into the logic is certainly readable and appropriate for the purpose at hand. However, hard coding the data into the code... not very reusable. Of course I could (and often do) grab the useful chunk of code, copy, paste and modify it.

As to performance... It has been a while since I worked on anything in which performance was a concern. Years ago I did a lot of data conversion work. Pulling the data for 180k pieces of nuclear plant equipment from Datacom, making a bunch of transformations to it and writing data load files for a DB2 on MVS based application suite - that required a focus on efficiency. AND Visual FoxPro.

The project manager decided that he wanted the conversions done in Visual Basic (which we told him was a dumb idea.). He got the company's top VB programmer and assigned him to write the conversion for a smaller chunk of data. I don't recall what it was. After some weeks of development and testing the conversion routine was ready to go. On the conversion weekend he pulled the production data for the plant we were converting. (It was a small amount of data compared to the equipment data which I was converting.) Still, the conversion process ran for 3 or 4 hours. While it was running the client, who had some dBase III experience and one of my colleagues, out of boredom wrote the conversion routines in Visual FoxPro, tested, debugged and verified the converted data. All while the "official" VB based conversion was still running. We did not use Visual Basic any more after that

Another interesting project involved taking data from an ancient mainframe based records indexing system - look up key words and find that the document of interest is on microfilm cartridge 3817 frames 28-36 - and converted it to static web pages, one for each document. These were dumped on an old server and indexed. Presto an Intranet document search facility. I generated about 5 million web pages. My program - about a half page of code - generated the pages faster than Windoze could create files. I had to put a slowdown routing in it to keep Windoze from tripping over itself.

Back to my script - it is fast enough.

Thanks again,

Ken
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Is possible to simulate Load balancing algorithms using NS-2?for example i am trying to simulate Least Connection algorithm. mariio Linux - Networking 3 05-05-2016 08:06 AM
inner join in sed /awk sam@ Linux - Newbie 11 03-11-2014 10:01 PM
Java: what's the difference between an inner class and a static inner class? irey Programming 6 01-28-2009 03:34 AM
MySQL: Help with using join to simulate subquery MicahCarrick Programming 1 09-30-2006 07:17 AM
SELECT and INNER JOIN Gerardoj Programming 1 06-07-2004 03:09 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 01:53 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration