LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Software (https://www.linuxquestions.org/questions/linux-software-2/)
-   -   Need help converting BASH script to Python (https://www.linuxquestions.org/questions/linux-software-2/need-help-converting-bash-script-to-python-4175605267/)

usao 05-04-2017 02:42 PM

Need help converting BASH script to Python
 
Ive been tasked to convert a bash script to python.
So far, the only way I know how to do that is to wrap every statement in the bash script with a call tothe system to execute. However, I found that Python is not persisting things as BASH does, for example, I have lines in the bash script which set variables, but when I execute those inside python as calls to the OS, they are not kept but lost.
So, im confused about how to proceed.
How does one execute shell commands from Python correctly?

hydrurga 05-04-2017 03:21 PM

Personally, I would completely rewrite the logic in Python. Rewriting each Bash line as a Python statement in the way which you intend to do will not only be ponderous but produce very inefficient non-Pythonesque code.

The good thing about Python is that there is loads of help out there, with great examples, much of it on stackoverflow.

For example, Googling python execute shell command answers your query: http://stackoverflow.com/questions/8...mand-in-python (use the subprocess module).

ondoho 05-05-2017 12:44 AM

Quote:

Originally Posted by usao (Post 5706298)
So far, the only way I know how to do that is to wrap every statement in the bash script with a call tothe system to execute.

this is illogical.

- why do you need python istead of bash? presumably because the target system cannot use bash
- the way you are writing it now, you will need BOTH bash AND python on the target system.
- so you could've left it as a bash script just as well.

!!! 05-05-2017 02:31 AM

Can you just python "system" the WHOLE .sh file? Each system invocation is a seperate shell/process which knows NOTHING of the prior! ( it )? :D More... Oh: bash2py (from googling the thread title :doh:)

usao 05-05-2017 10:32 AM

Never said it's logical, just that my boss is requiring that we convert to Python. This is on CentOS system and BASH is fully supported.
Ive made some progress. Have a few thousand lines of code and so-far managed to convert 10 lines to Python.

hydrurga 05-05-2017 10:35 AM

Quote:

Originally Posted by usao (Post 5706610)
Never said it's logical, just that my boss is requiring that we convert to Python. This is on CentOS system and BASH is fully supported.
Ive made some progress. Have a few thousand lines of code and so-far managed to convert 10 lines to Python.

If you translate something from French to English, you don't translate each single word into its English equivalent. You translate the meaning.

It's the same for Bash->Python. I don't think your boss is going to be very happy with the finished result, nor are the people who have to support your Python "code" in the future. In my opinion of course.

pan64 05-05-2017 10:38 AM

do you know why is it important? Probably we can help you to solve this issue somehow, but not by automagically convert thousands of lines. That is actually impossible I think (and also I think you won't post your scripts therefore we have no any information about their complexity).

ondoho 05-05-2017 10:47 AM

translating a script from one language to another is extremely painful and time consuming work; it would be quicker to actually write a new script. especially so if you're unfamiliar with one language.
with all due respect, asking my employees to translate 1000s of lines of code like that, is a waste of resources.
i really, really wonder where this requirement comes from. it feels like Arbeitsbeschaffungsmaßnahme...

but hey, i guess you can't post the actual code here, so in the end it amounts to "good python tutorial"...
https://docs.python.org/2/tutorial/
https://docs.python.org/3/tutorial/

usao 05-05-2017 10:49 AM

I can certainly post portions of my scripts where im having problems. For example, this is a portion of code ive managed to convert so-far.
As for the french/english comment, im not sure what you are getting at. All my boss wants is to hvae the code convertd to Python. He doesn't even know what Python looks like. He's just passing down direction from higher ups.

BASH:
#!/bin/bash

. ~tomcat/etl/.profileforETL

export STAGEDIR=/stage/inbound
export FEEDTAG=Medispan

export WORKDIR=$STAGEDIR/$FEEDTAG
export ARCHDIR=$STAGEDIR/archive/$FEEDTAG

export STARTTIME=`date +%Y%m%d-%H%M%S`
export DATEPART=`date +%Y%m`

export FEEDLOG=$WORKDIR/${FEEDTAG}_feedfiles.log
export LCK=$WORKDIR/$FEEDTAG.lck
export LOG=$WORKDIR/log/${FEEDTAG}_$DATEPART.log
export SQL=$WORKDIR/$FEEDTAG.sql
export OUT=$WORKDIR/$FEEDTAG.out

Python:
import datetime

STAGEDIR='/stage/inbound'
FEEDTAG='Medispan'

WORKDIR=STAGEDIR+'/'+FEEDTAG
ARCHDIR=STAGEDIR+'/archive/'+FEEDTAG

STARTTIME=datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
DATEPART=datetime.date.today().strftime("%Y%m")

FEEDLOG=WORKDIR+'/'+FEEDTAG+'_feedfiles.log'
LCK=WORKDIR+'/'+FEEDTAG+'.lck'
LOG=WORKDIR+'/log/'+FEEDTAG+'_'+DATEPART+'.log'
SQL=WORKDIR+'/'+FEEDTAG+'.sql'
OUT=WORKDIR+'/'+FEEDTAG+'.out'

usao 05-05-2017 11:07 AM

What I need to do is find a way to manage lockfiles. Currently, we have a bash function which creates an atomic lockfile, and also managed a TRAP to remove the lockfile on any exit.

lock ()
{
typeset -i w=0 # wait mode (0 = Don't wait)

# Check for command-line arguments.
while getopts w z; do
case $z in
w) w=1 ;; # Set wait mode.
esac
done
shift `expr $OPTIND - 1`

ln -s $$ $1 > /dev/null 2>&1 # Try to get a lock.
until [ $? -eq 0 ]; do
[ $w -eq 0 ] && return 1 # Failed and non-blocking.
sleep 10
ln -s $$ $1 > /dev/null 2>&1 # Try again.
done
trap "rm -f $1" ABRT EXIT FPE HUP ILL INT KILL PIPE QUIT SEGV TERM
return 0
} # lock ().

usao 05-05-2017 11:08 AM

Also, need to find a way to loop over files...

for TRIG in `ls -1 med-file*zip.done 2> /dev/null`; do
...
done

pan64 05-05-2017 12:04 PM

looks like you need to be familiar with python modules. You can see for example here: https://docs.python.org/2/library/fcntl.html
you also need to rewrite trap, see signal handling https://docs.python.org/2/library/signal.html
ls -1 will be os.walk: http://www.tutorialspoint.com/python/os_walk.htm

chrism01 05-10-2017 11:10 PM

As per eg hydrurga & others above, you don't try to convert word by word or even line by line or fn by fn.
Instead, you have extract the requirements ie derive the specification by analysing the current program, then rewrite the entire thing in the new lang from scratch.
This is known as 'porting' a program ...

PS: There may be some parts that call an external binary/program; I'm sure there is a python method for doing that.

usao 05-11-2017 08:41 AM

Quote:

Originally Posted by chrism01 (Post 5708928)
As per eg hydrurga & others above, you don't try to convert word by word or even line by line or fn by fn.
Instead, you have extract the requirements ie derive the specification by analysing the current program, then rewrite the entire thing in the new lang from scratch.
This is known as 'porting' a program ...

PS: There may be some parts that call an external binary/program; I'm sure there is a python method for doing that.

Understood, and I know the requirements as I wrote the bash script. However, it's how to translate the shell functionality into Python that I don't know how to do, or at least do effectivly. I was hoping to find a converter routine which would translate the bash commands into Python commands, but no luck so far.
For example, the bash script has to go into a directory, get a listing of the files matching a RE pattern, and then loop over them.
I don't know how to do any of these operations in Python.
Im sure that many folks have had to do this before (Bash->Python), but im not finding any specific examples of how to do any of the basic shell commands in bash.

scasey 05-11-2017 11:34 AM

Quote:

Originally Posted by !!! (Post 5706477)
Oh: bash2py (from googling the thread title :doh:)

usao,
Did you miss this on #4? Google bash2py.

Disclaimer: I know nothing about python.

Thought: Were I in your position, I would spend more time working to convince my boss that ANY effort to convert a process "just because" was a complete waste than I would attempting to do the conversion. Just sayin' :rolleyes:


All times are GMT -5. The time now is 03:20 AM.