Running a script with |tee myLogfile.log causes problems

DexterMorgan · 05-29-2017, 03:22 PM

Hi,

So I have a custom script for installation of my MySQL database that I'm running with tee to be able to go back and study it in case of something happens.

When the database is installed and the script is executing:

sudo /usr/bin/mysql_secure_installation

it asks my for my password, and then nothing happens. I know that I'm supposed to get questions like:

Remove anonymous users?
Disallow root login remotely?
Remove test database and access to it?
Reload privilege tables now?

But I'm left with nothing. If I press enter a bunch of times the I suddenly see a screen with all these questions already answered - when I pressed enter a couple of times I must've given it the default answer.

I'm executing the script as root:
./myScript.sh 2>errors.log | tee installation_history.log

How do I solve this so I get to answer the questions and save history to the logfile "installation_history.log"

norobro · 05-29-2017, 06:36 PM

Will "script" do what you want? http://man7.org/linux/man-pages/man1/script.1.html

DexterMorgan · 05-29-2017, 07:05 PM

Quote:

Originally Posted by norobro

Will "script" do what you want? http://man7.org/linux/man-pages/man1/script.1.html

I'm not following, can you elaborate on this? an example?

norobro · 05-29-2017, 07:18 PM

Example usage:

Code:

$ cat myscript.sh
#!/bin/bash

script -c mysql_secure_installation output.log

Run myscript (./myscipt.sh) and answer questions.

Code:

$ cat output.log
Script started on Mon 29 May 2017 07:15:11 PM CDT

Securing the MySQL server deployment.


VALIDATE PASSWORD PLUGIN can be used to test passwords
and improve security. It checks the strength of password
and allows the users to set only those passwords which are
secure enough. Would you like to setup VALIDATE PASSWORD plugin?

Press y|Y for Yes, any other key for No: n
Using existing password for user.
Change the password for user ? ((Press y|Y for Yes, any other key for No) : n

 ... skipping.
By default, a MySQL installation has an anonymous user,
allowing anyone to log into MySQL without having to have
a user account created for them. This is intended only for
testing, and to make the installation go a bit smoother.
You should remove them before moving into a production
environment.

Remove anonymous users? (Press y|Y for Yes, any other key for No) : n

 ... skipping.


Normally, root should only be allowed to connect from
'localhost'. This ensures that someone cannot guess at
the root password from the network.

Disallow root login remotely? (Press y|Y for Yes, any other key for No) : n

 ... skipping.
By default, MySQL comes with a database named 'test' that
anyone can access. This is also intended only for testing,
and should be removed before moving into a production
environment.


Remove test database and access to it? (Press y|Y for Yes, any other key for No) : n

 ... skipping.
Reloading the privilege tables will ensure that all changes
made so far will take effect immediately.

Reload privilege tables now? (Press y|Y for Yes, any other key for No) : n

 ... skipping.
All done! 

Script done on Mon 29 May 2017 07:15:15 PM CDT

r.stiltskin · 05-29-2017, 08:37 PM

If mysql_secure_installation isn't writing anything to stdout, there isn't anything for tee (or script) to send to a file. So you'll have to look at your script itself to see why it isn't doing what you expected. Maybe running it step by step (with the bash -x option) will help, e.g.

Code:

sudo bash -x /usr/bin/mysql_secure_installation

Shadow_7 · 05-29-2017, 10:00 PM

Maybe all of the prompts are being written to stderr, so sending 2 aka stderr aka /dev/stderr to >error.log leaves nothing for tee to get on stdout aka 1 aka /dev/stdout. Just a guess, using 2>&1 | tee might better suit, or at least one thing to try in the absence of other things.

DexterMorgan · 06-04-2017, 12:20 AM

So, I've tested out what you guys suggested, but neither was working the way I was expecting, so I'm back to square one,

so the pipe | tee mySavedHistory.log is causing all problems, but I still want the oppertunity to go back and check the log. Any more ideas?

And why do you think the pipe is cauasing this problem?
what does the pipe do exaktly?

and about 2>error.log
nothing of interest is showing up here, so I haven't the foggiest idea where the missing questions from /usr/bin/mysql_secure_installation is ending up, right now a black hole it seems.

Shadow_7 · 06-04-2017, 12:23 PM

The pipe basically takes the output of one thing and sends it to the input of other thing. Using tee lets you copy the flow of data to a file while sending it along. There's a caveat that the thing receiving the pipe actually takes input. Specifically from stdout.

Some basics:

0 stdin /dev/stdin
1 stdout /dev/stdout
2 stderr /dev/stderr

A practical demonstration:

FILE: bashtest.sh

Code:

#!/bin/bash

echo "some thing to stdout"
echo "some thing to stderr" >&2
echo "some thing else to stdout"
echo "some thing else to stderr" >&2

exit 0

$ bash bashtest.sh
some thing to stdout
some thing to stderr
some thing else to stdout
some thing else to stderr

$ bash bashtest.sh 2> tempSTDERR.txt
some thing to stdout
some thing else to stdout
$ cat tempSTDERR.txt
some thing to stderr
some thing else to stderr

$ bash bashtest.sh > tempSTDOUT.txt
some thing to stderr
some thing else to stderr
$ cat tempSTDOUT.txt
some thing to stdout
some thing else to stdout

$ bash bashtest.sh 2>&1 | > tempALL.txt
$ cat tempALL.txt

$ bash bashtest.sh 2>&1 | cat - > tempALL.txt
$ cat tempALL.txt
some thing to stdout
some thing to stderr
some thing else to stdout
some thing else to stderr

$ bash bashtest.sh 2>&1 | tee tempALL.txt
some thing to stdout
some thing to stderr
some thing else to stdout
some thing else to stderr
$ cat tempALL.txt
some thing to stdout
some thing to stderr
some thing else to stdout
some thing else to stderr

$ rm tempSTDERR.txt tempSTDOUT.txt tempALL.txt

$ bash bashtest.sh 2> tempSTDERR.txt > tempSTDOUT.txt
$ cat tempSTDERR.txt
some thing to stderr
some thing else to stderr
$ cat tempSTDOUT.txt
some thing to stdout
some thing else to stdout
$ cat tempALL.txt
cat: tempALL.txt: No such file or directory

Sometimes it's simpler to make a stupid test case like these to figure it out before modding/debugging the more complex monstrosity.

Habitual · 06-05-2017, 07:28 AM

Quote:

Originally Posted by DexterMorgan

I'm executing the script as root:
./myScript.sh 2>errors.log | tee installation_history.log

Then why is sudo necessary in it?

r.stiltskin · 06-06-2017, 07:10 AM

It would be helpful if you could post a simple (e.g. 1 or 2 questions/inputs) script that exhibits the same problem.

norobro · 06-06-2017, 10:22 AM

Give this page a read: http://www.pixelbeat.org/programming/stdio_buffering/

The section "stdio buffer control", near the bottom of the page, describes how to use the command "stdbuff".

Putting "stdbuf -oL mysql_secure_installation" in the script or putting "stdbuf -oL" first on the command line worked on my machine.

Possibly of interest:
https://bugs.mysql.com/bug.php?id=53796
http://bertvv.github.io/notes-to-sel..._installation/