LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 05-04-2017, 02:42 PM   #1
usao
Member
 
Registered: Dec 2011
Location: Chandler, AZ
Posts: 286

Rep: Reputation: Disabled
Need help converting BASH script to Python


Ive been tasked to convert a bash script to python.
So far, the only way I know how to do that is to wrap every statement in the bash script with a call tothe system to execute. However, I found that Python is not persisting things as BASH does, for example, I have lines in the bash script which set variables, but when I execute those inside python as calls to the OS, they are not kept but lost.
So, im confused about how to proceed.
How does one execute shell commands from Python correctly?
 
Old 05-04-2017, 03:21 PM   #2
hydrurga
LQ Guru
 
Registered: Nov 2008
Location: Pictland
Distribution: Linux Mint 20 MATE
Posts: 8,048
Blog Entries: 5

Rep: Reputation: 2916Reputation: 2916Reputation: 2916Reputation: 2916Reputation: 2916Reputation: 2916Reputation: 2916Reputation: 2916Reputation: 2916Reputation: 2916Reputation: 2916
Personally, I would completely rewrite the logic in Python. Rewriting each Bash line as a Python statement in the way which you intend to do will not only be ponderous but produce very inefficient non-Pythonesque code.

The good thing about Python is that there is loads of help out there, with great examples, much of it on stackoverflow.

For example, Googling python execute shell command answers your query: http://stackoverflow.com/questions/8...mand-in-python (use the subprocess module).
 
Old 05-05-2017, 12:44 AM   #3
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 17,237
Blog Entries: 10

Rep: Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160
Quote:
Originally Posted by usao View Post
So far, the only way I know how to do that is to wrap every statement in the bash script with a call tothe system to execute.
this is illogical.

- why do you need python istead of bash? presumably because the target system cannot use bash
- the way you are writing it now, you will need BOTH bash AND python on the target system.
- so you could've left it as a bash script just as well.
 
Old 05-05-2017, 02:31 AM   #4
!!!
Member
 
Registered: Jan 2017
Location: Fremont, CA, USA
Distribution: Trying any&ALL on old/minimal
Posts: 748

Rep: Reputation: 323Reputation: 323Reputation: 323Reputation: 323
Can you just python "system" the WHOLE .sh file? Each system invocation is a seperate shell/process which knows NOTHING of the prior! ( it )? More... Oh: bash2py (from googling the thread title )

Last edited by !!!; 05-05-2017 at 02:51 AM.
 
Old 05-05-2017, 10:32 AM   #5
usao
Member
 
Registered: Dec 2011
Location: Chandler, AZ
Posts: 286

Original Poster
Rep: Reputation: Disabled
Never said it's logical, just that my boss is requiring that we convert to Python. This is on CentOS system and BASH is fully supported.
Ive made some progress. Have a few thousand lines of code and so-far managed to convert 10 lines to Python.
 
Old 05-05-2017, 10:35 AM   #6
hydrurga
LQ Guru
 
Registered: Nov 2008
Location: Pictland
Distribution: Linux Mint 20 MATE
Posts: 8,048
Blog Entries: 5

Rep: Reputation: 2916Reputation: 2916Reputation: 2916Reputation: 2916Reputation: 2916Reputation: 2916Reputation: 2916Reputation: 2916Reputation: 2916Reputation: 2916Reputation: 2916
Quote:
Originally Posted by usao View Post
Never said it's logical, just that my boss is requiring that we convert to Python. This is on CentOS system and BASH is fully supported.
Ive made some progress. Have a few thousand lines of code and so-far managed to convert 10 lines to Python.
If you translate something from French to English, you don't translate each single word into its English equivalent. You translate the meaning.

It's the same for Bash->Python. I don't think your boss is going to be very happy with the finished result, nor are the people who have to support your Python "code" in the future. In my opinion of course.
 
Old 05-05-2017, 10:38 AM   #7
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 16,487

Rep: Reputation: 5530Reputation: 5530Reputation: 5530Reputation: 5530Reputation: 5530Reputation: 5530Reputation: 5530Reputation: 5530Reputation: 5530Reputation: 5530Reputation: 5530
do you know why is it important? Probably we can help you to solve this issue somehow, but not by automagically convert thousands of lines. That is actually impossible I think (and also I think you won't post your scripts therefore we have no any information about their complexity).
 
Old 05-05-2017, 10:47 AM   #8
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 17,237
Blog Entries: 10

Rep: Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160
translating a script from one language to another is extremely painful and time consuming work; it would be quicker to actually write a new script. especially so if you're unfamiliar with one language.
with all due respect, asking my employees to translate 1000s of lines of code like that, is a waste of resources.
i really, really wonder where this requirement comes from. it feels like Arbeitsbeschaffungsmaßnahme...

but hey, i guess you can't post the actual code here, so in the end it amounts to "good python tutorial"...
https://docs.python.org/2/tutorial/
https://docs.python.org/3/tutorial/
 
Old 05-05-2017, 10:49 AM   #9
usao
Member
 
Registered: Dec 2011
Location: Chandler, AZ
Posts: 286

Original Poster
Rep: Reputation: Disabled
I can certainly post portions of my scripts where im having problems. For example, this is a portion of code ive managed to convert so-far.
As for the french/english comment, im not sure what you are getting at. All my boss wants is to hvae the code convertd to Python. He doesn't even know what Python looks like. He's just passing down direction from higher ups.

BASH:
#!/bin/bash

. ~tomcat/etl/.profileforETL

export STAGEDIR=/stage/inbound
export FEEDTAG=Medispan

export WORKDIR=$STAGEDIR/$FEEDTAG
export ARCHDIR=$STAGEDIR/archive/$FEEDTAG

export STARTTIME=`date +%Y%m%d-%H%M%S`
export DATEPART=`date +%Y%m`

export FEEDLOG=$WORKDIR/${FEEDTAG}_feedfiles.log
export LCK=$WORKDIR/$FEEDTAG.lck
export LOG=$WORKDIR/log/${FEEDTAG}_$DATEPART.log
export SQL=$WORKDIR/$FEEDTAG.sql
export OUT=$WORKDIR/$FEEDTAG.out

Python:
import datetime

STAGEDIR='/stage/inbound'
FEEDTAG='Medispan'

WORKDIR=STAGEDIR+'/'+FEEDTAG
ARCHDIR=STAGEDIR+'/archive/'+FEEDTAG

STARTTIME=datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
DATEPART=datetime.date.today().strftime("%Y%m")

FEEDLOG=WORKDIR+'/'+FEEDTAG+'_feedfiles.log'
LCK=WORKDIR+'/'+FEEDTAG+'.lck'
LOG=WORKDIR+'/log/'+FEEDTAG+'_'+DATEPART+'.log'
SQL=WORKDIR+'/'+FEEDTAG+'.sql'
OUT=WORKDIR+'/'+FEEDTAG+'.out'
 
Old 05-05-2017, 11:07 AM   #10
usao
Member
 
Registered: Dec 2011
Location: Chandler, AZ
Posts: 286

Original Poster
Rep: Reputation: Disabled
What I need to do is find a way to manage lockfiles. Currently, we have a bash function which creates an atomic lockfile, and also managed a TRAP to remove the lockfile on any exit.

lock ()
{
typeset -i w=0 # wait mode (0 = Don't wait)

# Check for command-line arguments.
while getopts w z; do
case $z in
w) w=1 ;; # Set wait mode.
esac
done
shift `expr $OPTIND - 1`

ln -s $$ $1 > /dev/null 2>&1 # Try to get a lock.
until [ $? -eq 0 ]; do
[ $w -eq 0 ] && return 1 # Failed and non-blocking.
sleep 10
ln -s $$ $1 > /dev/null 2>&1 # Try again.
done
trap "rm -f $1" ABRT EXIT FPE HUP ILL INT KILL PIPE QUIT SEGV TERM
return 0
} # lock ().
 
Old 05-05-2017, 11:08 AM   #11
usao
Member
 
Registered: Dec 2011
Location: Chandler, AZ
Posts: 286

Original Poster
Rep: Reputation: Disabled
Also, need to find a way to loop over files...

for TRIG in `ls -1 med-file*zip.done 2> /dev/null`; do
...
done
 
Old 05-05-2017, 12:04 PM   #12
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 16,487

Rep: Reputation: 5530Reputation: 5530Reputation: 5530Reputation: 5530Reputation: 5530Reputation: 5530Reputation: 5530Reputation: 5530Reputation: 5530Reputation: 5530Reputation: 5530
looks like you need to be familiar with python modules. You can see for example here: https://docs.python.org/2/library/fcntl.html
you also need to rewrite trap, see signal handling https://docs.python.org/2/library/signal.html
ls -1 will be os.walk: http://www.tutorialspoint.com/python/os_walk.htm
 
Old 05-10-2017, 11:10 PM   #13
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 7.7 (?), Centos 8.1
Posts: 17,860

Rep: Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596
As per eg hydrurga & others above, you don't try to convert word by word or even line by line or fn by fn.
Instead, you have extract the requirements ie derive the specification by analysing the current program, then rewrite the entire thing in the new lang from scratch.
This is known as 'porting' a program ...

PS: There may be some parts that call an external binary/program; I'm sure there is a python method for doing that.

Last edited by chrism01; 05-10-2017 at 11:11 PM.
 
Old 05-11-2017, 08:41 AM   #14
usao
Member
 
Registered: Dec 2011
Location: Chandler, AZ
Posts: 286

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by chrism01 View Post
As per eg hydrurga & others above, you don't try to convert word by word or even line by line or fn by fn.
Instead, you have extract the requirements ie derive the specification by analysing the current program, then rewrite the entire thing in the new lang from scratch.
This is known as 'porting' a program ...

PS: There may be some parts that call an external binary/program; I'm sure there is a python method for doing that.
Understood, and I know the requirements as I wrote the bash script. However, it's how to translate the shell functionality into Python that I don't know how to do, or at least do effectivly. I was hoping to find a converter routine which would translate the bash commands into Python commands, but no luck so far.
For example, the bash script has to go into a directory, get a listing of the files matching a RE pattern, and then loop over them.
I don't know how to do any of these operations in Python.
Im sure that many folks have had to do this before (Bash->Python), but im not finding any specific examples of how to do any of the basic shell commands in bash.
 
Old 05-11-2017, 11:34 AM   #15
scasey
LQ Veteran
 
Registered: Feb 2013
Location: Tucson, AZ, USA
Distribution: CentOS 7.8.2003
Posts: 5,420

Rep: Reputation: 2044Reputation: 2044Reputation: 2044Reputation: 2044Reputation: 2044Reputation: 2044Reputation: 2044Reputation: 2044Reputation: 2044Reputation: 2044Reputation: 2044
Quote:
Originally Posted by !!! View Post
Oh: bash2py (from googling the thread title )
usao,
Did you miss this on #4? Google bash2py.

Disclaimer: I know nothing about python.

Thought: Were I in your position, I would spend more time working to convince my boss that ANY effort to convert a process "just because" was a complete waste than I would attempting to do the conversion. Just sayin'
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
python script converting date to iso format vikatakavi Programming 3 10-16-2013 04:01 PM
[SOLVED] Converting Script from Linux (GNU) Bash 4 to Solaris Bash 2.05 - Any cheat sheet? oly_r Solaris / OpenSolaris 6 05-03-2013 08:25 AM
[SOLVED] Python TypeError when calling Python program from a Bash script noppeli Programming 2 01-15-2013 08:06 AM
Converting perl script to python!! Urgent need it soon. tbilisidavid Programming 3 07-24-2011 10:39 PM
[SOLVED] Converting a Shell script to a Python script TheIndependentAquarius Programming 4 01-29-2010 12:27 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 10:22 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration