[Python] - Script to show down nodes with log information per node

jonnybinthemix · 02-07-2017, 02:14 PM

Good Evening All,

So, I was planning to use this thread (please admin tell if I shouldn't) as a landing page to a project that I've decided to work on in order to learn Python. As I mentioned in a previous post, I've used bash shell scripts for everything for so long, it's about time I learned Python.

The script I'm planning would probably take me no more than an hour in a regular bash script, but I see it as a good opportunity to put in a little more effort and learn something new

So.. my plan is to create a script that I can run every morning to tell me which nodes in a cluster are down compared to the day before. The problem is this;

The script will be ran from a management node. The down node information is only available from a login node, so in bash I guess I'd just SSH to the login node, run the command and export the data, then store it in a tmp file and then use diff to tell me whats different and then get to working on the additional nodes which are down.

So my questions are twofold:

1.) Is there a module or something which would make the SSH section of this simple? Or would I just use Python to send shell commands, with subversion or something?

2.) The next step in the process would be to look for common strings in log files associated to the node names the first step reports. So again, in a regular shell script, I would just load the node name into a variable, or list and loop through them or something outputting log lines to the screen. I know with Python we can parse xml files with a module called xml.etree.ElementTree, is there something similar to help me parse just straight log files?

3.) The last step (ish) would be to run some specific commands against the nodes to check the MCE Banks for Memory Errors and CATERRS, I think I know the answer to this, which is just use something like subversion to run the tool and capture the output, so I guess my question is with regards to capturing the output... I suppose this question will develop as I move on.

So I know what you're probably thinking... he's asking all these questions, but where's his script. We're not going to do it for him!!! And I agree, I'm wanting to learn Python and so there is no script at the moment, but I'm really just looking for pointers, modules you know of which I can go away and read about, and as the days move on I'll post the script as it develops. I hope this is acceptable?

Thanks
Jon

jonnybinthemix · 02-07-2017, 02:52 PM

A quick search and this is what I've got now; however I'm not 100% sure I understand all of it.

Code:

#!/usr/bin/python
#
#
import subversion

ssh = subprocess.Popen("ssh root@boot nodestatus | egrep -v "(up|standby1|standby2)" | awk '{ print $2 }'")
    stdout = subprocess.PIPE, stderr=subprocess.STDOUT

ssh,stderr = ssh.communicate()
status = ssh.poll()

print output

So this would (I guess) print the output of the nodestatus command on the boot node to the management node, but I need to add the output as a list in a variable for me to do something with. I was hoping that using subversion in this way would not be best practice and that there may be an ssh module which I can use in a more efficient way, but I don't seem to see much information out there about such things.

~Jon

astrogeek · 02-07-2017, 02:59 PM

Quote:

Originally Posted by jonnybinthemix

So, I was planning to use this thread (please admin tell if I shouldn't) as a landing page to a project that I've decided to work on in order to learn Python. As I mentioned in a previous post, I've used bash shell scripts for everything for so long, it's about time I learned Python.

That would not be a good use of the forum on multiple levels. It would be difficult for others to follow the thread as it would meander among multiple topics, and it would be confusing for those seeking help with their own problems who land here following search results.

I would encourage you to use your blog space here on LQ as the landing page for your projects, and use the forum to get help for specific questions which may arise during the course of your project.

Please review the Site FAQ and Site Rules for guidance on acceptable forum use.

You can find access to your blog space in your profile page, or via the links under My LQ in the right sidebar of any page.

Thanks