Why is the memory usage sharply decreased after restarting the service?

Matthew Wai · 11-23-2023, 11:24 PM

Quote:

Originally Posted by pan64

did you read the post I sent?

I just did the following test:

Code:

Matthew_Wai@My-PC:~$ free -h
              total        used        free      shared  buff/cache   available
Mem:          5.7Gi       880Mi       2.6Gi        59Mi       2.3Gi       4.5Gi
Swap:         2.0Gi          0B       2.0Gi
Matthew_Wai@My-PC:~$ sudo systemctl stop HDD_temp && sudo systemctl start HDD_temp
[sudo] password for Matthew_Wai: 
Verification successful
Matthew_Wai@My-PC:~$ free -h
              total        used        free      shared  buff/cache   available
Mem:          5.7Gi       888Mi       2.4Gi        59Mi       2.4Gi       4.5Gi
Swap:         2.0Gi          0B       2.0Gi
Matthew_Wai@My-PC:~$

The above output shows the following:
(1) "buff/cache" was "2.3Gi" originally.
(2) The service was restarted.
(3) "buff/cache" was "2.4Gi" after the restart.

Restarting the service did not clear the cache.

pan64 · 11-23-2023, 11:30 PM

are you sure you can manage the memory better than the kernel itself (which was developed to do this). see post #5.
But there is another question here too: are you sure kernel won't free it up by itself if that memory will be needed for something more important?

Actually it is not a simple shell script, it invokes at least 5-6 additional tools, like sudo, date, service, grep, hddtemp.

Quote:

Originally Posted by Matthew Wai

Restarting the service did not clear the cache.

Obviously, it won't do that, just tells the kernel that the memory/cache used by this service is not needed any more.
As you can see you have a lot of memory available, why don't you allow the kernel to use it for something useful (speed up something a bit)?

Matthew Wai · 11-24-2023, 03:06 AM

Both the service and the script were written by me.
Both work fine after the service is restarted.
I am not smart enough to figure out why the service should not be restarted.

MadeInGermany · 11-24-2023, 04:30 AM

I wonder if your shell script will ever log "Failure".?
But I have no explanation why it should consume MBs of memory.

The following variant has a loop with max 200 cycles of service starts, and a 1s sleep in between.

Code:

#!/bin/bash
S="Failure"
sleep 10
for ((N=0; N < 200; N++)) ; do
  if service fancontrol status|grep running; then
    S="Success"; break
  fi
  sudo service fancontrol start
  sleep 1
done
File=/@/Fancontrol/Log
echo "$(date) ${S}" >> "${File}"

while : ; do
  T=$(sudo hddtemp -n /dev/sda)
  echo $((T * 1000)) > "/@/Fancontrol/Hddtemp"
  sleep 30
done

pan64 · 11-24-2023, 04:57 AM

Quote:

Originally Posted by Matthew Wai

Both the service and the script were written by me.
Both work fine after the service is restarted.
I am not smart enough to figure out why the service should not be restarted.

I'm not really convinced that restarting the service will help anything. You need to make at least more measurements and more precise measurements to be sure.

Matthew Wai · 11-24-2023, 05:36 AM

Quote:

Originally Posted by MadeInGermany

I wonder if your shell script will ever log "Failure".?

The variable "${S}" is empty by default. “Failure” will appear as long as "${S}" remains empty.
I have never seen “Failure” in the “Log” file, as the following command never fails repeatedly.
sudo service fancontrol start

Quote:

Originally Posted by MadeInGermany

But I have no explanation why it should consume MBs of memory.

After restarting the service, it uses 496 KB, which is reasonable. See below:

Code:

Matthew_Wai@My-PC:~$ service HDD_temp status|grep "Memory"
   Memory: 496.0K
Matthew_Wai@My-PC:~$

MadeInGermany · 11-24-2023, 08:00 AM

Quote:

“Failure” will appear as long as "${S}" remains empty.

But it won't leave your loop if $S is empty.

Code:

while [ -z "${S}" ]

Perhaps you want to log "Failure" within the loop not after it?

Matthew Wai · 11-24-2023, 10:14 PM

Quote:

Originally Posted by MadeInGermany

But it won't leave your loop if $S is empty.

Code:

while [ -z "${S}" ]

The loop will end even if "${S}" is empty. The reason is shown below:

Code:

for ((N=0;N<20;N++))

The loop will not be an endless loop.

Matthew Wai · 11-24-2023, 10:22 PM

Quote:

Originally Posted by MadeInGermany

But I have no explanation why it should consume MBs of memory.

One possible explanation is that there is a bug in the kernel or the memory management code.

pan64 · 11-25-2023, 08:48 AM

Quote:

Originally Posted by Matthew Wai

One possible explanation is that there is a bug in the kernel or the memory management code.

Obviously, this cannot be ruled out. It's just almost impossible. The Linux kernel is currently used on countless hosts (millions or even billions of devices and VMs) and has been running for decades, not just years. Such an error should have been detected and corrected a long time ago. Otherwise, all of these systems would inevitably be unstable and unreliable

pan64 · 11-25-2023, 08:53 AM

Quote:

Originally Posted by Matthew Wai

The loop will end even if "${S}" is empty. The reason is shown below:

Code:

for ((N=0;N<20;N++))

The loop will not be an endless loop.

the for loop is not an endless loop, the while loop you wrote is an endless loop, if fancontrol cannot be started this loop will just run forever. If the "service | grep" command always fails, it never sets S to anything.

Matthew Wai · 11-26-2023, 05:46 AM

Quote:

Originally Posted by MadeInGermany

But it won't leave your loop if $S is empty.

Quote:

Originally Posted by pan64

the while loop you wrote is an endless loop

I have just broken the loop by adding two conditions as shown below:

Code:

while [ -z "${S}" ];do for ((N=0;N<20;N++));do sleep 5s 
service fancontrol status|grep running && S="OK" && N=20
if [ -z "${S}" ];then sudo service fancontrol start;fi;done
if [ "${N}" == "20" ]&&[ -z "${S}" ];then S="Failed";fi;done

Matthew Wai · 11-26-2023, 06:01 AM

Quote:

Originally Posted by pan64

Such an error should have been detected and corrected a long time ago. Otherwise, all of these systems would inevitably be unstable and unreliable

A waste of 200 MB of memory will not cause an error, instability, or unreliability.

pan64 · 11-26-2023, 09:05 AM

Quote:

Originally Posted by Matthew Wai

A waste of 200 MB of memory will not cause an error, instability, or unreliability.

That is wrong. Any [unpredictable] memory leak will lead to crash sooner or later. And actually wasting a single byte (not 200MB) under any circumstances is very dangerous, unacceptable.

Matthew Wai · 11-26-2023, 10:08 PM

Quote:

Originally Posted by pan64

That is wrong. Any [unpredictable] memory leak will lead to crash sooner or later. And actually wasting a single byte (not 200MB) under any circumstances is very dangerous, unacceptable.

I did not mean a memory leak. When I wrote "a waste of 200 MB", I meant the memory usage could safely be reduced.