LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware
User Name
Password
Slackware This Forum is for the discussion of Slackware Linux.

Notices


Reply
  Search this Thread
Old 06-01-2024, 03:39 AM   #46
Petri Kaukasoina
Senior Member
 
Registered: Mar 2007
Posts: 1,960

Rep: Reputation: 1571Reputation: 1571Reputation: 1571Reputation: 1571Reputation: 1571Reputation: 1571Reputation: 1571Reputation: 1571Reputation: 1571Reputation: 1571Reputation: 1571

Quote:
Originally Posted by rizitis View Post
I have attached one output with zram and one no_zram.

what i realized is that I have a on top output
Code:
 5017 root      20   0   43.6g   8.9g   7.1g S 863.6%  14.2   9:35.22 chat
which probably I understand it wrong because it exist in both zram and no_zram.
So to be sure I added an mpstat -P ALL output in my script and re-run script...
The output of mpstat -P ALL output dont look like a 863.6% of cpu...

So I m waiting to explain me what is what...
thanks.
I think your logs show that everything works fine. The memory need of your program is so small compared to how much memory you have that it doesn't use swap at all. ZRAM is not used at all, and there is no difference between your two logs when chat has used 10 minutes of cpu time.

no_zram_output.txt:
Code:
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
 5017 root      20   0   43.6g   8.9g   7.1g S 863.6  14.2   9:35.22 chat
zram_output.txt:
Code:
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
 4549 root      20   0   43.5g   8.9g   7.1g S 872.7  14.2  10:09.53 chat
%CPU of 872.7% and cpu time used 10:09.53 mean that the process has used 10 minutes of cpu time in 70 seconds of wall clock time. The percentage is calculated from the cumulative usage of cpu time for the whole lifetime of the process so far. 9 processors giving all their cpu time for your application in user level would give 900%.

If you try 'mpstat 1' you see the total cpu usage info once a second at that moment (not for the lifetime of your process). And for the CPU=all line 13.41% is calculated so that 100% would correspond to all the processors added together. So, multiply 13.41% by the number of cpus 20 to get 268.2% that you can compare to the top output.

So, the difference in numbers comes from that: top and 'ps auxw' give %CPU calculated for the lifetime of the process, and it is more than 100% because your program uses many threads. mpstat shows momentary values for the cpu, not for your process.

By the way, why are you running the program as root?

Last edited by Petri Kaukasoina; 06-01-2024 at 03:45 AM.
 
1 members found this post helpful.
Old 06-01-2024, 04:05 AM   #47
rizitis
Member
 
Registered: Mar 2009
Location: Greece,Crete
Distribution: Slackware64-current, Slint
Posts: 722
Blog Entries: 2

Rep: Reputation: 533Reputation: 533Reputation: 533Reputation: 533Reputation: 533Reputation: 533
Understood, thank you very much!

Quote:
Originally Posted by Petri Kaukasoina View Post

By the way, why are you running the program as root?
Only this time i had to be root, its not daily use or needed.
The reason for now it was that I wrote above script to execute all commands and > print them in /tmp/{no_zram.txt,zram.txt} else some commands fails without sudo...
Code:
#!/bin/bash



> /tmp/no_zram_output.txt

echo "starting with stock slackware zram setup off:"
date
echo ""
uname -a
echo "===="

echo ""
echo "df -h output:"
echo ""
df -h
echo "===="
echo ""



echo""
echo "inxi output:"
echo ""
inxi
echo ""
inxi -m
echo "===="
echo ""

echo""
echo "swapon output"
echo ""
swapon -s
echo "===="
echo ""

echo "free output:"
echo ""
free -h
echo "===="
echo""

echo "zramctl --output-all:"
echo ""
zramctl --output-all /dev/zram0
echo "===="
echo ""


echo "syslog output:"
echo ""
tail -n 50 /var/log/syslog
echo "===="
echo ""

sleep 60

echo ""
echo "LOADING APP FOR MODELS"
echo ""
cd /usr/local/bin/GPT4All/bin || exit 1
./chat &
echo "===="

echo ""
echo "top output:"
echo ""
top -b -n 1 | head -n 50
echo "===="
echo ""

echo "mpstat -P ALL output:"
echo ""
/usr/bin/mpstat -P ALL
echo ""
echo "===="



echo ""
echo "script will sleep 60..."
sleep 60
echo ""
echo "END OF SLEEP"
echo "===="

date 
echo "===="
echo ""

echo "TURN 2 after model load"
echo "df -h output:"
echo ""
df -h
echo "===="
echo ""



echo""
echo "inxi output:"
echo ""
inxi
echo ""
inxi -m
echo "===="
echo ""

echo""
echo "swapon output"
echo ""
swapon -s
echo "===="
echo ""

echo "free output:"
echo ""
free -h
echo "===="
echo""

echo "zramctl --output-all:"
echo ""
zramctl --output-all /dev/zram0
echo "===="
echo ""


echo "syslog output:"
echo ""
tail -n 50 /var/log/syslog
echo "===="
echo ""


echo ""
echo "top output:"
echo ""
top -b -n 1 | head -n 50
echo "===="
echo ""

echo "mpstat -P ALL output:"
echo ""
/usr/bin/mpstat -P ALL
echo ""
echo "===="

echo ""
echo "3rd round after sleep 60"
sleep 60
echo ""
echo "end of sleep"
echo ""
echo "===="

echo "TURN 3"
echo "df -h output:"
echo ""
df -h
echo "===="
echo ""



echo""
echo "inxi output:"
echo ""
inxi
echo ""
inxi -m
echo "===="
echo ""

echo""
echo "swapon output"
echo ""
swapon -s
echo "===="
echo ""

echo "free output:"
echo ""
free -h
echo "===="
echo""

echo "zramctl --output-all:"
echo ""
zramctl --output-all /dev/zram0
echo"===="
echo ""


echo "syslog output:"
echo ""
tail -n 50 /var/log/syslog
echo "===="
echo ""


echo ""
echo "top output:"
echo ""
top -b -n 1 | head -n 50
echo "===="
echo ""

echo "mpstat -P ALL output:"
echo ""
/usr/bin/mpstat -P ALL
echo ""
echo "===="
echo ""
echo "Game Over or TILT... "
sudo script.sh >> /tmp/no_zram_output.txt

Last edited by rizitis; 06-01-2024 at 05:00 AM.
 
Old 06-04-2024, 01:07 PM   #48
rizitis
Member
 
Registered: Mar 2009
Location: Greece,Crete
Distribution: Slackware64-current, Slint
Posts: 722
Blog Entries: 2

Rep: Reputation: 533Reputation: 533Reputation: 533Reputation: 533Reputation: 533Reputation: 533
I just want to say that I was wrong and zram is ok and works perfect in slackware. See attachment to understand... For 1 hour no problem...
swap is my zram as stock slackware provide and no original swap partition exist...

EDIT: I also found the reason for fans speed... its tracker-miner-f
I kill -9 pid and fans stop... I might uninstall it also, I m not sure if i need it at all.
Attached Thumbnails
Click image for larger version

Name:	it_needed.png
Views:	57
Size:	106.7 KB
ID:	42998  

Last edited by rizitis; 06-06-2024 at 12:52 PM.
 
3 members found this post helpful.
Old 06-09-2024, 01:50 AM   #49
BratPit
Member
 
Registered: Jan 2011
Posts: 253

Rep: Reputation: 100Reputation: 100
Hi

I tested zram on my machine 32G memory . I process a lot of scans on 8 cores paralell.
I noticed that default Slack compression (LZRLO or so) give me about 50% more working space 32G+16G before out of memory.
So decided to change compression to ZSTD .
But I had to recompile kernel because zswap and zram compiled togather in huge kernel cause that I could not change compression algorithm.
ZsWap option was unchecked.
After that ZSTD is for me the best option . I have now about 32+32G space .
Performance is OK. A couple percent better but not much than from swap on SSD.
zram and swap on my system not working together. After out of memory from zram process crash
an not cooperate with active swap partition.

Last edited by BratPit; 06-09-2024 at 02:02 AM.
 
Old 06-09-2024, 02:03 AM   #50
Thom1b
Member
 
Registered: Mar 2010
Location: France
Distribution: Slackware
Posts: 489

Rep: Reputation: 339Reputation: 339Reputation: 339Reputation: 339
Hi,

Do you suggest to use zswap+zram? I thought zram (as swap) was useless when using zswap.

Last edited by Thom1b; 06-09-2024 at 02:19 AM. Reason: typo
 
Old 06-09-2024, 02:28 AM   #51
BratPit
Member
 
Registered: Jan 2011
Posts: 253

Rep: Reputation: 100Reputation: 100
Nope.
I off zswap from kernel because I could not change compression algoritm.
I said that ordinary swap partition (is active) not working well when zram is present.

Last edited by BratPit; 06-09-2024 at 02:32 AM.
 
Old 06-09-2024, 02:32 AM   #52
Thom1b
Member
 
Registered: Mar 2010
Location: France
Distribution: Slackware
Posts: 489

Rep: Reputation: 339Reputation: 339Reputation: 339Reputation: 339
OK, I misunderstood. Thanks.
 
Old 06-09-2024, 03:53 AM   #53
Petri Kaukasoina
Senior Member
 
Registered: Mar 2007
Posts: 1,960

Rep: Reputation: 1571Reputation: 1571Reputation: 1571Reputation: 1571Reputation: 1571Reputation: 1571Reputation: 1571Reputation: 1571Reputation: 1571Reputation: 1571Reputation: 1571
Quote:
Originally Posted by BratPit View Post
I noticed that default Slack compression (LZRLO or so) give me about 50% more working space 32G+16G before out of memory.
So decided to change compression to ZSTD .
/etc/default/zram sets the defaults that /etc/rc.d/rc.S uses when setting it up. A few lines from /etc/default/zram:
Code:
# Set the compression algorithm.
# Use zstd for best results.
# Nothing else makes any sense.
ZRAMCOMPRESSION=zstd
So, zstd is already the default. Yes, there are lines CONFIG_ZRAM_DEF_COMP_LZORLE=y and CONFIG_ZRAM_DEF_COMP="lzo-rle" in the kernel config but /etc/rc.d/rc.S explicitly sets the algorithm to $ZRAMCOMPRESSION instead of the kernel default. Command 'zramctl' will prove it:
Code:
$ zramctl 
NAME       ALGORITHM DISKSIZE  DATA COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram0 zstd         15.5G 36.5M    0B    0B       4 [SWAP]
Quote:
Originally Posted by BratPit View Post
But I had to recompile kernel because zswap and zram compiled togather in huge kernel cause that I could not change compression algorithm.
ZsWap option was unchecked.
There is no connection between ZSWAP and ZRAM. You don't use ZSWAP but there is no reason to rebuild the kernel without it. (And ZSWAP has its own kernel default CONFIG_ZSWAP_COMPRESSOR_DEFAULT_LZO=y, CONFIG_ZSWAP_COMPRESSOR_DEFAULT="lzo")

(By the way, you told that you use the huge kernel. In -current the difference between huge and generic is that the huge kernel has drivers for SCSI devices built in. If you don't need them to boot, you could as well use generic. If you don't need initrd with huge, you won't need initrd with generic, either, unless your root filesystem is on a SCSI drive. There is also no difference between huge and generic kernels related to zram or zswap.)
Quote:
Originally Posted by BratPit View Post
zram and swap on my system not working together. After out of memory from zram process crash
an not cooperate with active swap partition.
What do you mean by 'out of memory from zram'? If you have (in addition to the swap in zram) another swap device with lower priority, it should be used next after the zram device is full. What does command 'swapon' report then? Are you saying that the zram swap is used up but the other swap is not used at all?

It is also possible to setup a backing device, where you can move pages from the zram swap. See https://www.linuxquestions.org/quest....php?p=6504971 and https://www.linuxquestions.org/quest....php?p=6505010

Last edited by Petri Kaukasoina; 06-09-2024 at 04:00 AM.
 
Old 06-09-2024, 08:18 AM   #54
BratPit
Member
 
Registered: Jan 2011
Posts: 253

Rep: Reputation: 100Reputation: 100
1. I use Slackware 15 not current.
2. I made zram about year ago , and do not remember details but in rc.S there was no about zram.
3 .So I init it separately like this:

Quote:
start() {
if lsblk -lno name,mountpoint|grep -q ^zram ; then
echo "A zram block device for swap is already active."
exit
fi
modprobe zram num_devices=1
# keep user/sytem defined swappiness
cat /proc/sys/vm/swappiness > /usr/local/lib/swappiness

KRAM=$(sed -n 1p /proc/meminfo|sed "s/[ ]\{1,\}/ /"|cut -d" " -f2)
SWAP_DEV=$(awk '/swap/,$0=$1' /etc/fstab)
COMPRESSION_ALGORITHM=${COMPRESSION_ALGORITHM:-zstd}
echo $COMPRESSION_ALGORITHM > /sys/block/zram0/comp_algorithm
PERCENTAGE=${PERCENTAGE:-200}
echo $(($KRAM * $PERCENTAGE / 100))K >/sys/block/zram0/disksize

PRIORITY=${PRIORITY:-100}
mkswap /dev/zram0 >/dev/null
swapon -p $PRIORITY /dev/zram0
SWAPPINESS=${SWAPPINESS:-200}
echo $SWAPPINESS > /proc/sys/vm/swappiness
#done
zramctl -n
}
4. There was no option in huge kernel set it up to change to zstd without recompile kernel.
It's a piece o cake .I made it for 20 years.Never used initrd.

What do you mean by 'out of memory from zram'? If you have (in addition to the swap in zram) another swap device with lower priority, it should be used next after the zram device is full.

When I convert scans in parallel on 8 cores when zram (and ram) is full and need more space on secondary swap partition I got Segmentation faults on processes that need additional memory and go to swap.

Nevermind . I am glad using it.

Last edited by BratPit; 06-09-2024 at 08:21 AM.
 
Old 06-09-2024, 08:50 AM   #55
r1w1s1
Member
 
Registered: Mar 2004
Location: São Paulo - Brazil
Distribution: Slackware
Posts: 76
Blog Entries: 1

Rep: Reputation: 50
Quote:
Originally Posted by rizitis View Post
I just want to say that I was wrong and zram is ok and works perfect in slackware. See attachment to understand... For 1 hour no problem...
swap is my zram as stock slackware provide and no original swap partition exist...

EDIT: I also found the reason for fans speed... its tracker-miner-f
I kill -9 pid and fans stop... I might uninstall it also, I m not sure if i need it at all.
I did a fresh install in a VM without swap partition, automatically uses zram as swap.
 
Old 06-09-2024, 10:13 AM   #56
Petri Kaukasoina
Senior Member
 
Registered: Mar 2007
Posts: 1,960

Rep: Reputation: 1571Reputation: 1571Reputation: 1571Reputation: 1571Reputation: 1571Reputation: 1571Reputation: 1571Reputation: 1571Reputation: 1571Reputation: 1571Reputation: 1571
Quote:
Originally Posted by BratPit View Post
1. I use Slackware 15 not current.

4. There was no option in huge kernel set it up to change to zstd without recompile kernel.
I see. With Slackware 15 and huge kernel this is the error message in dmesg:
Code:
zstd_compress: exports duplicate symbol ZSTD_CCtxWorkspaceBound (owned by kernel)
I think it's again one of those problems when modules compiled for the generic kernel are used with the huge kernel. Patrick solved the problem in -current:
Code:
Thu Oct 26 19:55:16 UTC 2023
k/kernel-source-6.1.60-noarch-1.txz:  Upgraded.
  Hey folks, if you've been following LQ you know I've talked before about
  dropping the huge kernel and moving the distribution to use only the generic
  kernel plus an initrd. After mulling this over for a few months, I think I
  was looking at the problem in the wrong way. First of all, it's clear that
  some Slackware users have been using the huge kernel all along, without an
  initrd, and are (to say the least) unhappy about the prospect of a new
  requirement to start using one. I've been recommending the generic kernel for
  some time, and a major reason is that we've been using the same set of kernel
  modules with two slightly different kernels. Because of this, there have
  always been a few (generally seldom used) kernel modules that won't load into
  the huge kernel. These are things that aren't built into the huge kernel, but
  because of a difference in some kernel module dependency, they won't load.
  The conclusion that I've come to here is that rather than drop the huge
  kernel, or slap a LOCALVERSION on it and provide a whole duplicate tree of
  kernel modules especially for the huge kernel, it would be better to make the
  generic kernel more huge, and minimize the differences between the two kernel
  configs.
  That's what I've done here.
  Shown below are the differences between the previous generic kernel config
  and the one shipping in this update. You'll notice that most of the popular
  filesystems are built in. At this point the main difference it that the huge
  kernel has a couple of dozen SCSI drivers built into it. The modules for those
  drivers won't load into the huge kernel, but they're fully built in so that
  doesn't matter. If you find any other modules that will not load into the huge
  kernel, please make a note about it on LQ and I'll see what can be done.
  So, tl;dr - what does this change mean?
  Unless your root device is on SCSI, if you were able to use the huge kernel
  without an initrd previously, you should now be able to use the generic
  kernel without an initrd. The kernel is a bit bigger, but we probably have
  enough RAM these days that it won't make a difference.
  Enjoy! :-)
  -CIFS_SMB_DIRECT n
   9P_FS m -> y
   9P_FSCACHE n -> y
   BTRFS_FS m -> y
   CIFS m -> y
   CRYPTO_CMAC m -> y
   CRYPTO_CRC32 m -> y
   CRYPTO_XXHASH m -> y
   CRYPTO_ZSTD m -> y
   EFIVAR_FS m -> y
   EXFAT_FS m -> y
   EXT2_FS m -> y
   EXT3_FS m -> y
   EXT4_FS m -> y
   F2FS_FS m -> y
   FAILOVER m -> y
   FAT_FS m -> y
   FSCACHE m -> y
   FS_ENCRYPTION_ALGS m -> y
   FS_MBCACHE m -> y
   HW_RANDOM_VIRTIO m -> y
   ISO9660_FS m -> y
   JBD2 m -> y
   JFS_FS m -> y
   LZ4HC_COMPRESS m -> y
   LZ4_COMPRESS m -> y
   MSDOS_FS m -> y
   NETFS_SUPPORT m -> y
   NET_9P m -> y
   NET_9P_FD m -> y
   NET_9P_VIRTIO m -> y
   NET_FAILOVER m -> y
   NFSD m -> y
   NLS_CODEPAGE_437 m -> y
   NTFS3_FS m -> y
   NTFS_FS m -> y
   PSTORE_LZ4_COMPRESS n -> m
   PSTORE_LZO_COMPRESS n -> m
   PSTORE_ZSTD_COMPRESS n -> y
   QFMT_V2 m -> y
   QUOTA_TREE m -> y
   REISERFS_FS m -> y
   RPCSEC_GSS_KRB5 m -> y
   SMBFS m -> y
   SQUASHFS m -> y
   UDF_FS m -> y
   VFAT_FS m -> y
   VIRTIO_BALLOON m -> y
   VIRTIO_BLK m -> y
   VIRTIO_CONSOLE m -> y
   VIRTIO_INPUT m -> y
   VIRTIO_MMIO m -> y
   VIRTIO_NET m -> y
   VIRTIO_PCI m -> y
   VIRTIO_PCI_LIB m -> y
   VIRTIO_PCI_LIB_LEGACY m -> y
   VIRTIO_PMEM m -> y
   XFS_FS m -> y
   ZONEFS_FS n -> m
   ZSTD_COMPRESS m -> y
  +NFS_FSCACHE y
  +PSTORE_LZ4_COMPRESS_DEFAULT n
  +PSTORE_LZO_COMPRESS_DEFAULT n
  +PSTORE_ZSTD_COMPRESS_DEFAULT n
 
Old 06-10-2024, 08:39 AM   #57
Aeterna
Senior Member
 
Registered: Aug 2017
Location: Terra Mater
Distribution: VM Host: Slackware-current, VM Guests: Artix, Venom, antiX, Gentoo, FreeBSD, OpenBSD, OpenIndiana
Posts: 1,022

Rep: Reputation: Disabled
Quote:
Originally Posted by BratPit View Post
Hi

I tested zram on my machine 32G memory . I process a lot of scans on 8 cores paralell.
I noticed that default Slack compression (LZRLO or so) give me about 50% more working space 32G+16G before out of memory.
So decided to change compression to ZSTD .
But I had to recompile kernel because zswap and zram compiled togather in huge kernel cause that I could not change compression algorithm.
ZsWap option was unchecked.
After that ZSTD is for me the best option . I have now about 32+32G space .
Performance is OK. A couple percent better but not much than from swap on SSD.
zram and swap on my system not working together. After out of memory from zram process crash
an not cooperate with active swap partition.
You can have ZRAM and ZSWAP compiled together but you should use only one and whatever compression algorithm you like. Just apoend compression type to the kernel line. You can then adjust VM settings in sysctl.conf
 
1 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: ZRAM Finally Promoted Out Of Staging In Linux Kernel LXer Syndicated Linux News 0 12-19-2013 06:50 AM
LXer: zRAM Is Still Hoping For A Promotion LXer Syndicated Linux News 0 11-26-2013 02:41 AM
LXer: ZRAM Might Finally Be Moved Out Of Linux Staging LXer Syndicated Linux News 0 08-15-2013 08:10 AM
LXer: Ubuntu Linux Considers Greater Usage Of zRAM LXer Syndicated Linux News 0 12-08-2012 03:30 PM
LXer: Increased Performance In Linux With zRam (Virtual Swap Compressed in RAM) LXer Syndicated Linux News 8 10-04-2011 12:12 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware

All times are GMT -5. The time now is 04:31 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration