LinuxQuestions.org - every hour run a bash function inside a bash shell

- Programming (https://www.linuxquestions.org/questions/programming-9/)

- - every hour run a bash function inside a bash shell (https://www.linuxquestions.org/questions/programming-9/every-hour-run-a-bash-function-inside-a-bash-shell-4175675887/)

Quote:

Originally Posted by Skaperen (Post 6129192)

... since it requires some specific action in the shell to add environment variables, either there is a bash bug affecting only one user, or something has infected a bash function. i'm trying to narrow down when it happens and see if it's any command i'm doing.

It is a bit hard to "remote-debug" your script without knowing what it looks like. However, I do not see how printing the environment variables every hour is going to help narrowing down the problem. You still will not know what part of the script actually triggered the change. Have you considered to simply use

Code:

set -xv

at the beginning of your script, redirect stderr to a logfile and then grep those variables in the logfile? That would give you a hint when/where the change happened. The actions that preceded the change would also have been logged and available for further analysis.

Again, without any further information it is a bit hard to recommend the best course of action. Consider posting a redacted version of your script.

Quote:

Originally Posted by Skaperen (Post 6129192)

if you had actually read and understood the posts then you would know that it is not one specific environment variable, but random, often gibberish names that frequently have portions of personal or confidential data in them. since it requires some specific action in the shell to add environment variables, either there is a bash bug affecting only one user, or something has infected a bash function. i'm trying to narrow down when it happens and see if it's any command i'm doing.

how would a subshell do this? setting an environment variable in a subshell does not change it i the main shell process. try it.

You still not understand.
So if you have a variable (at least one), you can check it (existence, content, whatever). If you have no any name to check you can still check the whole environment by adding a check function to your prompt.
If you have a function which produces this strange thing you can put it into a subshell and it will not modify your current environment any more.

Quote:

Originally Posted by crts (Post 6129195)

I do not see how printing the environment variables every hour is going to help narrowing down the problem. You still will not know what part of the script actually triggered the change.

My understanding is that the OP is still trying to find the script causing the environment variables to be set - logging the variables on a regular basis gives the boundaries of when the script ran - i.e. before the file where those variables appear, but after the previous one. (An hour is a wide period though, and the useful part of pan64's most recent post points out the logging can be run every time the prompt appears instead of every hour.)

A potential solution appears to be in post 26, but with security concerns raised about logging sensitive data - except it doesn't appear to be necessary to log any variables (only to know when they are set).

In which case, just have the logging system remove all known/expected environment variables and save a count of what remains (not the actual variables). Then it's a case of checking when the result doesn't contain zero and notifying in some way.

Quote:

Originally Posted by pan64 (Post 6129229)

If you have a function which produces this strange thing you can put it into a subshell and it will not modify your current environment any more.

That's a potential workaround if - once the culprit is found - it's not possible to figure out what's going on and stop it using more direct methods (i.e. by fixing whatever bug is causing the issue).

The OP is probably not understanding because they're still trying to find the cause, and isn't yet looking for a workaround to a problem that, once identified, may well have a real fix.

Quote:

Originally Posted by pan64 (Post 6129229)

i don't know which variable names it will create next, so i need to save the whole environment. because there are variables i do intentionally change, a simple diff is not enough. i plan to remove those variables as part of doing the comparison.

the functions i do have, which may have bugs, do some changes i need to have continue to work. so isolating them is not an option. determining which is the culprit is the intent. narrowing down which i was run when the odd changes happen is the goal.

Quote:

Originally Posted by crts (Post 6129195)

Again, without any further information it is a bit hard to recommend the best course of action. Consider posting a redacted version of your script.

there is no script until i know of a way to run one. i am dabbling around with trap and signals as i have time. the many functions that i have in my shell are too much to clean up and post right now.

all i am asking is how any script could do this. make up some small script and see if you can get that to be run every hour on your interactive shell, even when a long running program is running across that time.

Quote:

Originally Posted by boughtonp (Post 6129268)

The OP is probably not understanding because they're still trying to find the cause, and isn't yet looking for a workaround to a problem that, once identified, may well have a real fix.

right, if there is a bug causing this, it should be identified and fixed.

Quote:

Originally Posted by boughtonp (Post 6129268)

My understanding is that the OP is still trying to find the script causing the environment variables to be set ...

Thanks, I was under the impression that there was one long running script that modified its variables and then started misbehaving.

Quote:

Originally Posted by Skaperen (Post 6129393)

@OP Sorry for the confusion. Looks like trap might be the only tool that you have here to solve this.
I am not sure if you know this already or if it was mentioned before but you can edit Bash's history format (DISCLAIMER: I have not tried this) so you can know when you executed a certain command.

Alternatively, if also you want to log the output of the commands you entered, you may want to try to use screen to record your interactive shell session:

Code:

$ screen -L

The screen command will launch a screen session and store all output in screenlog.0 (-L option). In newer versions you can choose the location of the logfile but the version I have will create the logfile in the current directory from which the command was launched.

Setup your trap inside your new screen session and get its pid:

Code:

$ trap envout SIGUSR2

$ envout(){ env >/home/<username>/$(exec date +%F_%T.env) } # or wherever the logs are to be stored

$ echo $$

Now you can emit the signal from another terminal (or cron job), just like before

Code:

$ kill -SIGUSR2 <pid> # <pid> from previous echo $$

If you change your PS1 prompt inside your working screen session to contain a timestamp, e.g.,

Code:

$ export PS1="\$(date +%F_%T)$ "

then you can also log the time of what you were doing in screenlog.0. Now you have timestamped commands and their output.

PS:

Quote:

Originally Posted by Skaperen (Post 6128856)

... that process that sent the signal can wait a minute then check the files with variables and compare the last 2. if it sees the relevant difference it would then output to the terminal.

I have noticed that a trapped signal will only be processed after <Enter> is hit in the interactive shell. This means, if you are not doing anything when the signal is emitted, the logfile may not have been created when the check is performed. You might want to consider running the checks from yet another, independent process periodically.

PPS (only if you go with screen):
In case you run into problems, here is a useful link about environment variables in screen and the documentation for screen.
The file screenlog.0 logs all your keystrokes, i.e., if you use the cursor keys they will be logged as well. Therefore the file's contents look like "gibberish". I have found the best way to look at screenlog.0 is to use

Code:

$ more /home/<username>/env_monitor/screenlog.0

This will interpret the escape sequences. I have also tried 'less -r' but the cursor keys are still escaped.

Quote:

Originally Posted by crts (Post 6129412)

Thanks, I was under the impression that there was one long running script that modified its variables and then started misbehaving.

just the interactive bash, itself, under this user.

Quote:

Originally Posted by crts (Post 6129412)

i didn't know it. that does look interesting for many things.

Quote:

Originally Posted by crts (Post 6129412)

Alternatively, if also you want to log the output of the commands you entered, you may want to try to use screen to record your interactive shell session:

Code:

$ screen -L

i already have a logging tool, but screen might help. i didn't think of looking into screen, yet. and i do so much screen stuff.

Quote:

Originally Posted by crts (Post 6129412)

Code:

$ trap envout SIGUSR2

$ envout(){ env >/home/<username>/$(exec date +%F_%T.env) } # or wherever the logs are to be stored

$ echo $$

Now you can emit the signal from another terminal (or cron job), just like before

Code:

$ kill -SIGUSR2 <pid> # <pid> from previous echo $$

If you change your PS1 prompt inside your working screen session to contain a timestamp, e.g.,

Code:

$ export PS1="\$(date +%F_%T)$ "

then you can also log the time of what you were doing in screenlog.0. Now you have timestamped commands and their output.

i'm thinking about some ways to store the environment at each command.

Quote:

Originally Posted by crts (Post 6129412)

PS:

I have noticed that a trapped signal will only be processed after <Enter> is hit in the interactive shell. This means, if you are not doing anything when the signal is emitted, the logfile may not have been created when the check is performed. You might want to consider running the checks from yet another, independent process periodically.

or i could just launch a checker into the background when i store the environment. i'd also have the checker dispose of "identical" environment files.

Quote:

Originally Posted by crts (Post 6129412)

PPS (only if you go with screen):
In case you run into problems, here is a useful link about environment variables in screen and the documentation for screen.
The file screenlog.0 logs all your keystrokes, i.e., if you use the cursor keys they will be logged as well. Therefore the file's contents look like "gibberish". I have found the best way to look at screenlog.0 is to use

Code:

$ more /home/<username>/env_monitor/screenlog.0

This will interpret the escape sequences. I have also tried 'less -r' but the cursor keys are still escaped.

less -r has worked well enough for me, except when moving up. but another up then back down tends to fix things.

lots of useful help here, thanks much!!

Quote:

Originally Posted by Skaperen (Post 6129449)

... or i could just launch a checker into the background when i store the environment. i'd also have the checker dispose of "identical" environment files.

Have a look into incron, it might suit your needs quite well here. It would launch the checker on demand, i.e., whenever a file is created in your dedicated directory where you store all the logs. Have a look at this post as a quickstart guide on how to setup incron. I have not used incron in a few years now but I do not think that much has changed. Let me know if you get stuck with incron (if you want to use it, that is).

Quote:

Originally Posted by Skaperen (Post 6129449)

i'm thinking about some ways to store the environment at each command.

Well then, from 'man 1 bash'

Quote:

...
PROMPT_COMMAND
If set, the value is executed as a command prior to issuing each
primary prompt.
...

Which leads us to

Code:

$ mkdir /home/<username>/env_monitor/ # dedicated directory for (a lot of) logs

$ export PROMPT_COMMAND="env > /home/<username>/env_monitor/\$(date '+%F_%T,%3N').env"

This eliminates previous issues with delayed handling of a trapped signal.

Hope this helps.

do we have now any progress (from the beginning?)

there is progress. i don't know if it's from the beginning.

Ubuntu seems to no longer have a package named "incrond".

The package is named incron.

Quote:

Originally Posted by Skaperen (Post 6129527)

there is progress. i don't know if it's from the beginning.

Ubuntu seems to no longer have a package named "incrond".

The package name is incron, as shruggy posted, incrond is the daemon that needs to be started. I thought that searching for incrond would also find the correct package. I changed this in my previous post.

Is the setting the PROMPT_COMMAND variable doing what you had in mind?

> Is the setting the PROMPT_COMMAND variable doing what you had in mind?

i haven't gotten that far, yet.