LinuxQuestions.org
Page 2 of 3 1 2 3
Show 50 post(s) from this thread on one page

LinuxQuestions.org (/questions/)
-   Linux - Desktop (https://www.linuxquestions.org/questions/linux-desktop-74/)
-   -   Swap Partition Ruining PC Performance? (https://www.linuxquestions.org/questions/linux-desktop-74/swap-partition-ruining-pc-performance-917353/)

H_TeXMeX_H 12-08-2011 08:38 AM
I missed that.

Some observations:

You are using a lot of apps that are leaky and eat up a lot of RAM. I would never run all of those at once. From what I see above, I think some part of XFCE may be leaking as well as Firefox (update to latest version). Maybe try another desktop environment other than XFCE.

FredJones 12-08-2011 08:59 AM
1 Attachment(s)
Regarding the RAM settings, here are the outputs that I know about, to see what's happening. If you could take a minute and confirm that everything matches. These are all taken within a few seconds of each other:

htop. What I find odd is that the bars for Mem shouldn't be so far across. 2566 is 78% of 3271. There are 30 bars and so 78% of those is 23. In the attached PNG you can see that they are ALL lit up. Here is the text version:

Code:
  1  [||                        4.6%]    Tasks: 114, 56 kthr; 1 running
  2  [||                        2.6%]    Load average: 0.06 0.03 0.05
  3  [|                        1.3%]    Uptime: 1 day, 19:42:14
  4  [||                        4.0%]
  Mem[|||||||||||||||||||2566/3271MB]
  Swp[|                  153/9091MB]

  PID USER      PRI  NI  VIRT  RES  SHR S CPU% MEM%  TIME+  Command
 1459 fred      20  0 1190M  737M 28224 S  0.0 22.5 49:41.26 firefox
 1420 fred      20  0  752M  343M 27160 S  0.0 10.5 13:06.78 /usr/lib/thunderb
  961 mysql      20  0  307M  167M  3144 S  0.0  5.1 15:40.78 /usr/bin/mysqld -
 4962 fred      20  0  369M  154M  7764 S  0.0  4.7  3:28.80 java -Xmx192M -ja
 1310 fred      20  0  134M  121M  6508 S  0.0  3.7  2:43.24 /usr/lib/xfce4/pa
 1635 fred      20  0  233M 99024 22416 S  0.0  3.0  1:16.90 /usr/lib/chromium
 1581 fred      20  0  396M 70236 27368 S  0.0  2.1  8:21.43 /usr/lib/chromium
 5833 fred      20  0  471M 58808 17188 S  0.0  1.8 30:51.45 /usr/lib/firefox/
10672 fred      20  0  233M 57940 32432 S  0.0  1.7  0:52.50 /usr/lib/libreoff
 1642 fred      25  5  198M 53700 15712 S  0.0  1.6  2:03.73 /usr/lib/chromium
 6389 fred      20  0 94824 52236  4504 S  0.0  1.6  7:05.66 skype
 7785 http      20  0 69784 50564  2176 S  0.0  1.5 18:45.72 /usr/sbin/httpd -
 1072 http      20  0 67064 50268  2176 S  0.0  1.5 21:18.12 /usr/sbin/httpd -
 1219 root      20  0  134M 49952 12092 S  2.0  1.5 37:08.20 /usr/bin/X -nolis
F1Help  F2Setup F3SearchF4FilterF5Tree  F6SortByF7Nice -F8Nice +F9Kill  F10Quit
Here is top:

Code:
top - 16:48:13 up 1 day, 19:42,  4 users,  load average: 0.05, 0.03, 0.05
Tasks: 170 total,  1 running, 169 sleeping,  0 stopped,  0 zombie
Cpu(s):  0.8%us,  0.8%sy,  0.0%ni, 98.3%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  3349920k total,  2919088k used,  430832k free,    4576k buffers
Swap:  9310012k total,  156856k used,  9153156k free,  287868k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND           
 1219 root      20  0  134m  48m  11m S    4  1.5  37:08.93 X                 
 6478 fred      20  0 70820  16m 9836 S    2  0.5  0:33.20 Terminal         
15442 fred      20  0  4916 1748 1156 S    1  0.1  0:00.51 htop             
 1699 fred      20  0  201m  33m  16m S    1  1.0  24:16.41 chromium         
 5833 fred      20  0  471m  57m  16m S    1  1.8  30:51.57 plugin-containe   
 1459 fred      20  0 1190m 737m  27m S    0 22.5  49:41.33 firefox           
 1581 fred      20  0  396m  68m  26m S    0  2.1  8:21.50 chromium         
 6389 fred      20  0 94824  51m 4504 S    0  1.6  7:05.70 skype             
    1 root      20  0  1972  520  492 S    0  0.0  0:01.96 init             
    2 root      20  0    0    0    0 S    0  0.0  0:00.01 kthreadd         
    3 root      20  0    0    0    0 S    0  0.0  0:03.57 ksoftirqd/0       
    5 root      20  0    0    0    0 S    0  0.0  0:00.00 kworker/u:0       
    6 root      RT  0    0    0    0 S    0  0.0  0:00.00 migration/0       
    7 root      RT  0    0    0    0 S    0  0.0  0:00.46 watchdog/0       
    8 root      RT  0    0    0    0 S    0  0.0  0:00.00 migration/1       
  10 root      20  0    0    0    0 S    0  0.0  0:04.62 ksoftirqd/1       
  12 root      RT  0    0    0    0 S    0  0.0  0:00.42 watchdog/1
free and /etc/rc.conf

Code:
[fred@arch ~]$ free
            total      used      free    shared    buffers    cached
Mem:      3349920    2919964    429956          0      4596    288204
-/+ buffers/cache:    2627164    722756
Swap:      9310012    156856    9153156
[fred@arch ~]$ free -m
            total      used      free    shared    buffers    cached
Mem:          3271      2851        419          0          4        281
-/+ buffers/cache:      2565        705
Swap:        9091        153      8938
[fred@arch ~]$ more /etc/rc.conf
#
# /etc/rc.conf - Main Configuration for Arch Linux
#

# -----------------------------------------------------------------------
# LOCALIZATION
# -----------------------------------------------------------------------
#
# LOCALE: available languages can be listed with the 'locale -a' command
# DAEMON_LOCALE: If set to 'yes', use $LOCALE as the locale during daemon
# startup and during the boot process. If set to 'no', the C locale is used.
# HARDWARECLOCK: set to "UTC" or "localtime", any other value will result
#  in the hardware clock being left untouched (useful for virtualization)
#  Note: Using "localtime" is discouraged.
# TIMEZONE: timezones are found in /usr/share/zoneinfo
# KEYMAP: keymaps are found in /usr/share/kbd/keymaps
# CONSOLEFONT: found in /usr/share/kbd/consolefonts (only needed for non-US)
# CONSOLEMAP: found in /usr/share/kbd/consoletrans
# USECOLOR: use ANSI color sequences in startup messages
# VERBOSE: Verbose level (from 1 to 8). man 3 syslog for level info
#
LOCALE="en_US.UTF-8"
DAEMON_LOCALE="no"
HARDWARECLOCK="UTC"
TIMEZONE="Asia/Jerusalem"
KEYMAP="us"
CONSOLEFONT=
CONSOLEMAP=
USECOLOR="yes"
VERBOSE="3"

# -----------------------------------------------------------------------
# HARDWARE
# -----------------------------------------------------------------------
#
# MODULES: Modules to load at boot-up. Blacklisting is no longer supported.
#  Replace every !module by an entry as on the following line in a file in
#  /etc/modprobe.d:
#    blacklist module
#  See "man modprobe.conf" for details.
#
MODULES=(vboxdrv)

# Udev settle timeout (default to 30)
UDEV_TIMEOUT=30

# Scan for FakeRAID (dmraid) Volumes at startup
USEDMRAID="no"

# Scan for BTRFS volumes at startup
USEBTRFS="no"

# Scan for LVM volume groups at startup, required if you use LVM
USELVM="no"

# -----------------------------------------------------------------------
# NETWORKING
# -----------------------------------------------------------------------
#
# HOSTNAME: Hostname of machine. Should also be put in /etc/hosts
#
HOSTNAME="arch"

# Use 'ip addr' or 'ls /sys/class/net/' to see all available interfaces.
#
# Wired network setup
#  - interface: name of device (required)
#  - address: IP address (leave blank for DHCP)
#  - netmask: subnet mask (ignored for DHCP)
#  - gateway: default route (ignored for DHCP)
#
# Static IP example
# interface=eth0
# address=192.168.0.2
# netmask=255.255.255.0
# gateway=192.168.0.1
#
# DHCP example
# interface=eth0
# address=
# netmask=
# gateway=

interface=eth0
address=
netmask=
gateway=

# Setting this to "yes" will skip network shutdown.
# This is required if your root device is on NFS.
NETWORK_PERSIST="no"

# Enable these netcfg profiles at boot-up. These are useful if you happen to
# need more advanced network features than the simple network service
# supports, such as multiple network configurations (ie, laptop users)
#  - set to 'menu' to present a menu during boot-up (dialog package required)
#  - prefix an entry with a ! to disable it
#
# Network profiles are found in /etc/network.d
#
# This requires the netcfg package
#
#NETWORKS=(main)

# -----------------------------------------------------------------------
# DAEMONS
# -----------------------------------------------------------------------
#
# Daemons to start at boot-up (in this order)
#  - prefix a daemon with a ! to disable it
#  - prefix a daemon with a @ to start it up in the background
#
# If something other takes care of your hardware clock (ntpd, dual-boot...)
# you should disable 'hwclock' here.
#
DAEMONS=(!syslog-ng @dbus @network @ntpd @netfs @crond @httpd @mysqld @oss @zramswap)
Here is ps_mem. It may not be 100% clear, but at least the relative numbers tell something:

Code:
[fred@arch ~]$ sudo ps_mem
Password:
 Private  +  Shared  =  RAM used        Program

  4.0 KiB +  21.5 KiB =  25.5 KiB        ck-launch-session
  4.0 KiB +  21.5 KiB =  25.5 KiB        xinit
  4.0 KiB +  22.0 KiB =  26.0 KiB        dbus-launch
 24.0 KiB +  29.5 KiB =  53.5 KiB        rtkit-daemon
 48.0 KiB +  12.5 KiB =  60.5 KiB        init
  4.0 KiB +  64.0 KiB =  68.0 KiB        login
  4.0 KiB +  76.5 KiB =  80.5 KiB        dconf-service
  4.0 KiB +  77.5 KiB =  81.5 KiB        sh
  4.0 KiB +  78.5 KiB =  82.5 KiB        startx
  4.0 KiB +  80.0 KiB =  84.0 KiB        mysqld_safe
 72.0 KiB +  15.0 KiB =  87.0 KiB        dhcpcd
 24.0 KiB +  70.5 KiB =  94.5 KiB        libreoffice
  4.0 KiB +  94.5 KiB =  98.5 KiB        gconf-helper
  4.0 KiB +  96.0 KiB = 100.0 KiB        at-spi-bus-launcher
 20.0 KiB +  80.0 KiB = 100.0 KiB        agetty (5)
 44.0 KiB +  75.5 KiB = 119.5 KiB        gvfsd-dnssd
104.0 KiB +  18.5 KiB = 122.5 KiB        gnome-pty-helper
112.0 KiB +  12.5 KiB = 124.5 KiB        crond
112.0 KiB +  19.0 KiB = 131.0 KiB        gpg-agent
 84.0 KiB +  60.5 KiB = 144.5 KiB        gvfsd-metadata
160.0 KiB +  64.0 KiB = 224.0 KiB        gvfsd
144.0 KiB +  80.5 KiB = 224.5 KiB        gvfs-fuse-daemon
180.0 KiB + 110.5 KiB = 290.5 KiB        gvfsd-network
140.0 KiB + 156.5 KiB = 296.5 KiB        oosplash.bin
224.0 KiB + 101.5 KiB = 325.5 KiB        at-spi2-registryd
260.0 KiB +  69.5 KiB = 329.5 KiB        xfconfd
284.0 KiB +  47.0 KiB = 331.0 KiB        ntpd
264.0 KiB +  71.5 KiB = 335.5 KiB        jeditbg
352.0 KiB +  39.0 KiB = 391.0 KiB        top
272.0 KiB + 141.0 KiB = 413.0 KiB        xfsettingsd
296.0 KiB + 170.0 KiB = 466.0 KiB        console-kit-daemon
380.0 KiB +  95.5 KiB = 475.5 KiB        plugplay.exe
416.0 KiB +  62.5 KiB = 478.5 KiB        gconfd-2
396.0 KiB + 113.5 KiB = 509.5 KiB        gvfsd-trash
272.0 KiB + 253.0 KiB = 525.0 KiB        udevd (3)
388.0 KiB + 144.0 KiB = 532.0 KiB        upowerd
  4.5 MiB + -3994.0 KiB = 626.0 KiB        winedevice.exe
452.0 KiB + 182.5 KiB = 634.5 KiB        pulseaudio
456.0 KiB + 181.0 KiB = 637.0 KiB        polkit-gnome-authentication-agent-1
604.0 KiB +  41.0 KiB = 645.0 KiB        sudo
556.0 KiB + 116.5 KiB = 672.5 KiB        services.exe
500.0 KiB + 205.0 KiB = 705.0 KiB        xfce4-settings-helper
624.0 KiB +  99.0 KiB = 723.0 KiB        gvfs-gdu-volume-monitor
544.0 KiB + 267.0 KiB = 811.0 KiB        panel-6-systray
712.0 KiB + 137.0 KiB = 849.0 KiB        gnome-keyring-daemon
808.0 KiB +  44.0 KiB = 852.0 KiB        htop
  4.7 MiB + -3862.5 KiB = 961.5 KiB        udisks-daemon (2)
688.0 KiB + 329.0 KiB =  1.0 MiB        xfce4-power-manager (2)
  4.7 MiB + -3784.0 KiB =  1.0 MiB        nm-applet
856.0 KiB + 184.0 KiB =  1.0 MiB        gnome-settings-daemon
  1.0 MiB + 106.5 KiB =  1.1 MiB        polkitd [updated]
928.0 KiB + 354.5 KiB =  1.3 MiB        panel-14-dateti
  1.2 MiB + 519.0 KiB =  1.7 MiB        bash (4)
  1.8 MiB + 259.0 KiB =  2.0 MiB        xfce4-session
  1.7 MiB + 374.5 KiB =  2.0 MiB        dbus-daemon (3)
  1.5 MiB + 653.5 KiB =  2.1 MiB        xfdesktop
  1.7 MiB + 548.0 KiB =  2.3 MiB        explorer.exe
  2.7 MiB +  20.5 KiB =  2.7 MiB        wineserver
  2.4 MiB + 819.0 KiB =  3.2 MiB        panel-9-xfce4-n
  2.9 MiB + 535.5 KiB =  3.4 MiB        xfce4-xkb-plugin
  5.1 MiB + 481.5 KiB =  5.5 MiB        panel-5-xfce4-c
  5.1 MiB + 689.0 KiB =  5.8 MiB        xfwm4
115.6 MiB + -110262.0 KiB =  8.0 MiB        xfce4-netload-plugin
  7.7 MiB + 867.5 KiB =  8.6 MiB        Terminal
  8.0 MiB + 847.0 KiB =  8.8 MiB        hamster-service
 13.0 MiB + -3075.0 KiB =  10.0 MiB        filezilla
 13.8 MiB + -2704.5 KiB =  11.1 MiB        hamster-tray
 16.4 MiB + -2810.0 KiB =  13.7 MiB        Thunar
 13.6 MiB + 826.5 KiB =  14.4 MiB        xfce4-panel
 14.5 MiB + 836.5 KiB =  15.4 MiB        SQLyogCommunity
 47.0 MiB + -16073.0 KiB =  31.3 MiB        skype
167.1 MiB + -135059.5 KiB =  35.2 MiB        mysqld
 38.5 MiB + 581.5 KiB =  39.0 MiB        Xorg
148.0 MiB + -108600.5 KiB =  42.0 MiB        java
 49.3 MiB + -7141.0 KiB =  42.3 MiB        soffice.bin
 43.7 MiB +  5.4 MiB =  49.2 MiB        plugin-container
399.7 MiB + -281018.5 KiB = 125.3 MiB        httpd (12)
217.6 MiB + -29451.0 KiB = 188.8 MiB        chromium (11)
330.7 MiB + -96333.5 KiB = 236.6 MiB        thunderbird-bin
732.5 MiB + -83329.5 KiB = 651.2 MiB        firefox
---------------------------------
                          1.5 GiB
=================================

 Private  +  Shared  =  RAM used        Program
Even if Firefox is a hog (as Thunderbird also is sometimes) it's 700 or 800 M shouldn't cause all the headaches I have. Oddly, I turned off my Xfce panel CPU monitor because it was occasionally coming to the top of htop as using a lot of RAM. No idea why.

FredJones 12-08-2011 09:03 AM
Quote:
Originally Posted by H_TeXMeX_H (Post 4545157)
You are using a lot of apps that are leaky and eat up a lot of RAM. I would never run all of those at once. From what I see above, I think some part of XFCE may be leaking as well as Firefox (update to latest version). Maybe try another desktop environment other than XFCE.
I need to run the apps to work. :)

I thought 4G (even 3.5) was a lot and it could handle all this. Maybe I am wrong. Your point about Xfce is interesting and pre-empts my previous post where I mentioned the CPU monitor (we posted simultaneously it seems) and I suppose that to try another DE is actually a very good idea. I could try and see what happens.

H_TeXMeX_H 12-08-2011 09:24 AM
Firefox is the main issue I think. Maybe try removing plugins or extensions that you don't need.

H_TeXMeX_H 12-12-2011 12:29 PM
Here's another thing to try:

Code:
echo 1 > /proc/sys/vm/dirty_background_ratio
I have noticed that when the disk / page cache gets full, the system starts to become unresponsive with short hangs. Using the above setting it helps a lot.

I'll experiment with some options to try and solve the issue. Probably a definitive solution is more RAM.

Also see:
http://www.westnet.com/~gsmith/conte...ux-pdflush.htm
http://kmaiti.blogspot.com/2011/09/c...cache-and.html

FredJones 12-28-2011 08:23 AM
I got a 500 G USB and put on it a swap partition. I tried adding it in /etc/fstab but I failed. So now I have no swap at all, temporarily.

I rebooted and when I just launch a few apps on my fresh Xfce desktop, I hear the disk drive grinding already. I opened up a terminal and htop said 400/3271MB and then 720/3271MB when all 6 apps were launched.

And this is without Firefox. :)

My point is that the disk is grinding and there's LOTS of free RAM. That's how it apears to me anyhow.

If I open Thunar and browse to /srv/http and then click the little icon to show all the directories in the left sidebar, I hear the disk grind for 3 seconds before it finishes and shows the directories. There are 44 directories plus 69 files in this /srv/http . Seems bizarre that it should grind, and this with 1450/3271MB Mem in htop.

This grinding has nothing to do with RAM nor swap, as far as I can see.

I ran

echo 1 > /proc/sys/vm/dirty_background_ratio

and then played with Thunar and it still grinds. So on the one hand, this seems unrelated. This setting is a RAM paging issue and I am testing disk read I guess. But perhaps the above demonstrates that just reading from the disk is NOT working as it should. Because the physical RAM is not even 1/3 in use.

Now I figured out how to manually add the swap partition so I have

Code:
$ free -m
            total      used      free    shared    buffers    cached
Mem:          3271      2444        827          0        45        644
-/+ buffers/cache:      1753      1517
Swap:        4999        51      4948
An hour after I took the above "free -m" I restarted Firefox now and the disk started to grind. I ran htop after a second or two and it said Mem 1600/3271MB and Swp 58/4999MB. I didn't time it but the grinding noise lasted like 15 seconds. Then I restarted it again, right away, before doing anything and it restarted (with 7 open tabs to load) in 5 seconds with no grinding.

I am confused and quite discouraged. Unless I am misunderstanding something, this is not a RAM problem, because I am seeing slowness and grinding with NO swap being used or with SWAP on a separate physical machine.

Here is another experiment, a somewhat scientific one. I made a bash script:

Code:
#!/bin/bash
rm -f lines.txt
date
for i in {1..5000000}
do
  echo "Line Number $i" >> lines.txt
done
echo "finished" >> lines.txt
date
grep "finished" lines.txt
date
ls lines.txt -l
This is what I see when I run it:

Code:
$ ./lines.sh
Wed Dec 28 16:11:19 IST 2011
Wed Dec 28 16:14:05 IST 2011
finished
Wed Dec 28 16:14:05 IST 2011
-rw-r--r-- 1 kirk kirk 98888905 Dec 28 16:14 lines.txt
Close to 3 minutes to make a 94 MB file. This is with me doing very little else on the machine during that time. Can anyone else also run this and see?

I unfortunately don't have any other ideas for clear tests to run.

I have only two ideas left:

1. Use PAE Kernel, but this will only gain me 0.5 G physical RAM and so I don't see much reason to think it will fix my other problems
2. Reinstall using 64 bit Kernel, perhaps on a different drive. Truth is this is already a replacement drive, but I used Clonezilla to copy from the old one and maybe there is something wrong with the setup or partitions. Don't know what, but maybe.

I could try a brand new install on a separate disk and then have both disks in the same PC and see if it helps...

H_TeXMeX_H 12-28-2011 08:46 AM
Maybe there are a lot of fragmented files. If you want, I wrote an experimental defrag script that you can try, it is on my site.
http://htexmexh.my3gb.com/linux/scripts.html

FredJones 12-28-2011 10:33 AM
Doesn't seem like it found much:


Code:
$ sudo ./localfinddefrag
Password:
find: `/home/kirk/#swap.txt#': No such file or directory
find: `/home/kirk/.gvfs': Permission denied
Ouput written to /home/kirk/.defrag/output
Sorted output written to /home/kirk/.defrag/sorted

Most fragmented files
1|/home/kirk/.wine/drive_c/windows/winsxs/x86_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.4148_none_deadbeef/msvcr90.dll
1|/home/kirk/.wine/drive_c/windows/winsxs/x86_microsoft.windows.gdiplus_6595b64144ccf1df_1.0.6000.16386_none_deadbeef/gdiplus.dll
1|/home/kirk/.wine/.update-timestamp
1|/home/kirk/.wine/userdef.reg
1|/home/kirk/.Xauthority
1|/home/kirk/.xchat2/keybindings.conf
1|/home/kirk/.xchat2/scrollback/FreeNode/seank.txt
1|/home/kirk/.xchat2/xchat.conf
1|/home/kirk/.xinitrc
1|/home/kirk/xx.sql
I do use one Windows app in Wine: SQLyog. It's the one app I haven't found a good replacement for, yet, in native Linux.

TobiSGD 12-28-2011 11:45 AM
Quote:
Originally Posted by FredJones (Post 4560033)
This is what I see when I run it:

Code:
$ ./lines.sh
Wed Dec 28 16:11:19 IST 2011
Wed Dec 28 16:14:05 IST 2011
finished
Wed Dec 28 16:14:05 IST 2011
-rw-r--r-- 1 kirk kirk 98888905 Dec 28 16:14 lines.txt
Close to 3 minutes to make a 94 MB file. This is with me doing very little else on the machine during that time. Can anyone else also run this and see?
Tried that script:
Code:
Wed Dec 28 18:41:34 CET 2011
Wed Dec 28 18:43:20 CET 2011
finished
Wed Dec 28 18:43:20 CET 2011
-rw-r--r-- 1 tobi users 98888905 Dec 28 18:43 lines.txt
This is on a six core machine with Slackware 13.37 64 bit on 8GB of RAM, the script was run on a RAID 0-array.

H_TeXMeX_H 12-28-2011 12:33 PM
It takes longer than 3 minutes on my netbook. I didn't wait for it, because it was taking too long.

johnsfine 12-28-2011 01:02 PM
I tried that on an old (hardware and software) Centos system that I happened to have a putty window open on and that was otherwise idle. It took 129 seconds. But then I realized my home directory there is an NFS mount, which makes exactly this kind of file I/O much slower than local (even though the LAN is 1Gb and the file server super fast RAID and heavily cached with ram and lightly loaded.)

So I switched to a local directory and tried it. That took 103 seconds.

I was curious how much of that time was disk I/O, so I switched to a directory on a tmpfs (with enough free ram that this test wouldn't cause tmpfs to use any SWAP) and the test took 88 seconds.

I don't really understand my middle result. I expect write behind caching to be very limited when using NFS, so the extra time there is reasonable for lots of waits for short messages across the LAN. But I expected writes to a local disk file (on a system with lots of free ram) would just go into "write behind caching" and be nearly as fast as ram. The actual disk writes should occur after the script thinks it is done.

FredJones 12-28-2011 01:13 PM
Quote:
Originally Posted by H_TeXMeX_H (Post 4560060)
Maybe there are a lot of fragmented files. If you want, I wrote an experimental defrag script that you can try, it is on my site.
http://htexmexh.my3gb.com/linux/scripts.html
I looked at .defrag/output now. Lines 194 through 1108 are in ~/.mozilla/firefox/

Then 1111 through 1901 are ~/.wine/drive_c/

2162 - 9835 are ~/.thumbnails/

Lots from my Trash, too.

In the end there are 357120 lines. The file is 34 MB. Perhaps that IS a lot actually. Most are "2 extents found, perfection would be 1 extent" A few are 3, 4 and 5, but just browsing through I don't see any bigger numbers than that.

H_TeXMeX_H 12-28-2011 02:18 PM
Fragmentation doesn't seem to be a problem. I never get 1, in fact I'm happy with 20-40 extents.

It must be something else.

Here, I ran the script:

Code:
bash-4.1$ ./test
Wed Dec 28 21:09:40 CET 2011
Wed Dec 28 21:16:06 CET 2011
finished
Wed Dec 28 21:16:06 CET 2011
-rw-r--r-- 1 demonslayer users 98888905 Dec 28 21:16 lines.txt
it took about 7 minutes.

H_TeXMeX_H 12-29-2011 08:50 AM
I would uses sysbench for more benchmarks:
http://sourceforge.net/projects/sysbench/

You may need to replace 'AC_PROG_LIBTOOL' with 'AC_PROG_RANLIB' in configure.ac and run 'autoconf' and './augen.sh' again for it to compile.

Your script uses a lot of CPU usage, so it may actually be testing how good your CPU is.

FredJones 12-29-2011 10:34 AM
I installed sysbench 0.4.12-1, a build for ArchLinux so I presume it matches my setup.

Using this bash script:

Code:
date
sysbench --num-threads=16 --test=fileio --file-total-size=3G --file-test-mode=rndrw prepare
date
sysbench --num-threads=16 --test=fileio --file-total-size=3G --file-test-mode=rndrw run
date
sysbench --num-threads=16 --test=fileio --file-total-size=3G --file-test-mode=seqrd run
date
sysbench --num-threads=16 --test=fileio --file-total-size=3G --file-test-mode=rndrw cleanup
date
I see this:

Code:
$ ./mysysb.sh
Thu Dec 29 18:20:02 IST 2011
sysbench 0.4.12:  multi-threaded system evaluation benchmark

128 files, 24576Kb each, 3072Mb total
Creating files for the test...
Thu Dec 29 18:20:42 IST 2011
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 16

Extra file open flags: 0
128 files, 24Mb each
3Gb total file size
Block size 16Kb
Number of random requests for random IO: 10000
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Threads started!
Done.

Operations performed:  6013 Read, 3998 Write, 12802 Other = 22813 Total
Read 93.953Mb  Written 62.469Mb  Total transferred 156.42Mb  (2.3925Mb/sec)
  153.12 Requests/sec executed

Test execution summary:
    total time:                          65.3801s
    total number of events:              10011
    total time taken by event execution: 605.9701
    per-request statistics:
        min:                                  0.00ms
        avg:                                60.53ms
        max:                              1031.56ms
        approx.  95 percentile:            379.01ms

Threads fairness:
    events (avg/stddev):          625.6875/67.78
    execution time (avg/stddev):  37.8731/0.85

Thu Dec 29 18:21:48 IST 2011
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 16

Extra file open flags: 0
128 files, 24Mb each
3Gb total file size
Block size 16Kb
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing sequential read test
Threads started!
Done.

Operations performed:  196608 Read, 0 Write, 0 Other = 196608 Total
Read 3Gb  Written 0b  Total transferred 3Gb  (103.59Mb/sec)
 6629.87 Requests/sec executed

Test execution summary:
    total time:                          29.6549s
    total number of events:              196608
    total time taken by event execution: 474.2813
    per-request statistics:
        min:                                  0.00ms
        avg:                                  2.41ms
        max:                                319.21ms
        approx.  95 percentile:              5.15ms

Threads fairness:
    events (avg/stddev):          12288.0000/382.08
    execution time (avg/stddev):  29.6426/0.00

Thu Dec 29 18:22:18 IST 2011
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Removing test files...
Thu Dec 29 18:22:18 IST 2011
and then I did this:

Code:
date
sysbench --test=memory --memory-block-size=128K --memory-scope=global --memory-total-size=100G --memory-oper=write --num-threads=1 run
sysbench --test=memory --memory-block-size=128K --memory-scope=global --memory-total-size=100G --memory-oper=write --num-threads=2 run
sysbench --test=memory --memory-block-size=128K --memory-scope=global --memory-total-size=100G --memory-oper=write --num-threads=4 run
sysbench --test=memory --memory-block-size=128K --memory-scope=global --memory-total-size=100G --memory-oper=write --num-threads=32 run
sysbench --test=memory --memory-block-size=128K --memory-scope=global --memory-total-size=100G --memory-oper=write --num-threads=64 run
sysbench --test=memory --memory-block-size=128K --memory-scope=global --memory-total-size=100G --memory-oper=write --num-threads=128 run
sysbench --test=memory --memory-block-size=16K --memory-scope=global --memory-total-size=100G --memory-oper=write --num-threads=1 run
sysbench --test=memory --memory-block-size=16K --memory-scope=global --memory-total-size=100G --memory-oper=write --num-threads=2 run
sysbench --test=memory --memory-block-size=16K --memory-scope=global --memory-total-size=100G --memory-oper=write --num-threads=4 run
sysbench --test=memory --memory-block-size=16K --memory-scope=global --memory-total-size=100G --memory-oper=write --num-threads=32 run
sysbench --test=memory --memory-block-size=16K --memory-scope=global --memory-total-size=100G --memory-oper=write --num-threads=64 run
sysbench --test=memory --memory-block-size=16K --memory-scope=global --memory-total-size=100G --memory-oper=write --num-threads=128 run
date
and get this:

Code:
$ ./mysysb.sh
Thu Dec 29 18:26:33 IST 2011
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Doing memory operations speed test
Memory block size: 128K

Memory transfer size: 0M

Memory operations type: write
Memory scope type: global
Threads started!
Done.

Operations performed: 0 (    0.00 ops/sec)

0.00 MB transferred (0.00 MB/sec)


Test execution summary:
    total time:                          0.0001s
    total number of events:              0
    total time taken by event execution: 0.0000
    per-request statistics:
        min:                            18446744073709.55ms
        avg:                                  0.00ms
        max:                                  0.00ms

Threads fairness:
    events (avg/stddev):          0.0000/0.00
    execution time (avg/stddev):  0.0000/0.00

sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 2

Doing memory operations speed test
Memory block size: 128K

Memory transfer size: 0M

Memory operations type: write
Memory scope type: global
Threads started!
Done.

Operations performed: 0 (    0.00 ops/sec)

0.00 MB transferred (0.00 MB/sec)


Test execution summary:
    total time:                          0.0001s
    total number of events:              0
    total time taken by event execution: 0.0000
    per-request statistics:
        min:                            18446744073709.55ms
        avg:                                  0.00ms
        max:                                  0.00ms

Threads fairness:
    events (avg/stddev):          0.0000/0.00
    execution time (avg/stddev):  0.0000/0.00

sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 4

Doing memory operations speed test
Memory block size: 128K

Memory transfer size: 0M

Memory operations type: write
Memory scope type: global
Threads started!
Done.

Operations performed: 0 (    0.00 ops/sec)

0.00 MB transferred (0.00 MB/sec)


Test execution summary:
    total time:                          0.0001s
    total number of events:              0
    total time taken by event execution: 0.0000
    per-request statistics:
        min:                            18446744073709.55ms
        avg:                                  0.00ms
        max:                                  0.00ms

Threads fairness:
    events (avg/stddev):          0.0000/0.00
    execution time (avg/stddev):  0.0000/0.00

sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 32

Doing memory operations speed test
Memory block size: 128K

Memory transfer size: 0M

Memory operations type: write
Memory scope type: global
Threads started!
Done.

Operations performed: 0 (    0.00 ops/sec)

0.00 MB transferred (0.00 MB/sec)


Test execution summary:
    total time:                          0.0008s
    total number of events:              0
    total time taken by event execution: 0.0000
    per-request statistics:
        min:                            18446744073709.55ms
        avg:                                  0.00ms
        max:                                  0.00ms

Threads fairness:
    events (avg/stddev):          0.0000/0.00
    execution time (avg/stddev):  0.0000/0.00

sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Doing memory operations speed test
Memory block size: 128K

Memory transfer size: 0M

Memory operations type: write
Memory scope type: global
Threads started!
Done.

Operations performed: 0 (    0.00 ops/sec)

0.00 MB transferred (0.00 MB/sec)


Test execution summary:
    total time:                          0.0012s
    total number of events:              0
    total time taken by event execution: 0.0000
    per-request statistics:
        min:                            18446744073709.55ms
        avg:                                  0.00ms
        max:                                  0.00ms

Threads fairness:
    events (avg/stddev):          0.0000/0.00
    execution time (avg/stddev):  0.0000/0.00

sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 128

Doing memory operations speed test
Memory block size: 128K

Memory transfer size: 0M

Memory operations type: write
Memory scope type: global
Threads started!
Done.

Operations performed: 0 (    0.00 ops/sec)

0.00 MB transferred (0.00 MB/sec)


Test execution summary:
    total time:                          0.0024s
    total number of events:              0
    total time taken by event execution: 0.0000
    per-request statistics:
        min:                            18446744073709.55ms
        avg:                                  0.00ms
        max:                                  0.00ms

Threads fairness:
    events (avg/stddev):          0.0000/0.00
    execution time (avg/stddev):  0.0000/0.00

sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Doing memory operations speed test
Memory block size: 16K

Memory transfer size: 0M

Memory operations type: write
Memory scope type: global
Threads started!
Done.

Operations performed: 0 (    0.00 ops/sec)

0.00 MB transferred (0.00 MB/sec)


Test execution summary:
    total time:                          0.0000s
    total number of events:              0
    total time taken by event execution: 0.0000
    per-request statistics:
        min:                            18446744073709.55ms
        avg:                                  0.00ms
        max:                                  0.00ms

Threads fairness:
    events (avg/stddev):          0.0000/0.00
    execution time (avg/stddev):  0.0000/0.00

sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 2

Doing memory operations speed test
Memory block size: 16K

Memory transfer size: 0M

Memory operations type: write
Memory scope type: global
Threads started!
Done.

Operations performed: 0 (    0.00 ops/sec)

0.00 MB transferred (0.00 MB/sec)


Test execution summary:
    total time:                          0.0000s
    total number of events:              0
    total time taken by event execution: 0.0000
    per-request statistics:
        min:                            18446744073709.55ms
        avg:                                  0.00ms
        max:                                  0.00ms

Threads fairness:
    events (avg/stddev):          0.0000/0.00
    execution time (avg/stddev):  0.0000/0.00

sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 4

Doing memory operations speed test
Memory block size: 16K

Memory transfer size: 0M

Memory operations type: write
Memory scope type: global
Threads started!
Done.

Operations performed: 0 (    0.00 ops/sec)

0.00 MB transferred (0.00 MB/sec)


Test execution summary:
    total time:                          0.0001s
    total number of events:              0
    total time taken by event execution: 0.0000
    per-request statistics:
        min:                            18446744073709.55ms
        avg:                                  0.00ms
        max:                                  0.00ms

Threads fairness:
    events (avg/stddev):          0.0000/0.00
    execution time (avg/stddev):  0.0000/0.00

sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 32

Doing memory operations speed test
Memory block size: 16K

Memory transfer size: 0M

Memory operations type: write
Memory scope type: global
Threads started!
Done.

Operations performed: 0 (    0.00 ops/sec)

0.00 MB transferred (0.00 MB/sec)


Test execution summary:
    total time:                          0.0006s
    total number of events:              0
    total time taken by event execution: 0.0000
    per-request statistics:
        min:                            18446744073709.55ms
        avg:                                  0.00ms
        max:                                  0.00ms

Threads fairness:
    events (avg/stddev):          0.0000/0.00
    execution time (avg/stddev):  0.0000/0.00

sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Doing memory operations speed test
Memory block size: 16K

Memory transfer size: 0M

Memory operations type: write
Memory scope type: global
Threads started!
Done.

Operations performed: 0 (    0.00 ops/sec)

0.00 MB transferred (0.00 MB/sec)


Test execution summary:
    total time:                          0.0012s
    total number of events:              0
    total time taken by event execution: 0.0000
    per-request statistics:
        min:                            18446744073709.55ms
        avg:                                  0.00ms
        max:                                  0.00ms

Threads fairness:
    events (avg/stddev):          0.0000/0.00
    execution time (avg/stddev):  0.0000/0.00

sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 128

Doing memory operations speed test
Memory block size: 16K

Memory transfer size: 0M

Memory operations type: write
Memory scope type: global
Threads started!
Done.

Operations performed: 0 (    0.00 ops/sec)

0.00 MB transferred (0.00 MB/sec)


Test execution summary:
    total time:                          0.0024s
    total number of events:              0
    total time taken by event execution: 0.0000
    per-request statistics:
        min:                            18446744073709.55ms
        avg:                                  0.00ms
        max:                                  0.00ms

Threads fairness:
    events (avg/stddev):          0.0000/0.00
    execution time (avg/stddev):  0.0000/0.00

Thu Dec 29 18:26:33 IST 2011
Not really sure what tests I should be running here, but anyhow this whole second set seems odd. No operations performed. I got the above command from http://milek.blogspot.com/2007/12/sy...n-niagara.html

Then I tried

Code:
$ sysbench --test=memory --memory-block-size=16K --memory-total-size=100G --memory-scope=global --memory-hugetlb=off --memory-oper=write --memory-access-mode=seq run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Doing memory operations speed test
Memory block size: 16K

Memory transfer size: 0M

Memory operations type: write
Memory scope type: global
Threads started!
Done.

Operations performed: 0 (    0.00 ops/sec)

0.00 MB transferred (0.00 MB/sec)


Test execution summary:
    total time:                          0.0002s
    total number of events:              0
    total time taken by event execution: 0.0000
    per-request statistics:
        min:                            18446744073709.55ms
        avg:                                  0.00ms
        max:                                  0.00ms

Threads fairness:
    events (avg/stddev):          0.0000/0.00
    execution time (avg/stddev):  0.0000/0.00
from http://pastebin.com/H66QTu55 (random page I found) and also I see "Operations performed: 0 ( 0.00 ops/sec)"

Now here http://forums.tabcrawler.com/index.php?topic=4933.0 (another random page) I did this:

Code:
$ sysbench --test=cpu --cpu-max-prime=20000 run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 20000


Test execution summary:
    total time:                          26.3220s
    total number of events:              10000
    total time taken by event execution: 26.3184
    per-request statistics:
        min:                                  2.59ms
        avg:                                  2.63ms
        max:                                  4.44ms
        approx.  95 percentile:              2.78ms

Threads fairness:
    events (avg/stddev):          10000.0000/0.00
    execution time (avg/stddev):  26.3184/0.00
At least it DID something. :)


All times are GMT -5. The time now is 05:52 AM.
Page 2 of 3 1 2 3
Show 50 post(s) from this thread on one page