Wierd: Cannot create a file as root

WantLinuxHelp · 01-29-2009, 02:33 PM

As root on a CentOS system made by Rocks cluster management, I cannot create any home directories in /home. I had done this previously in the same system, but rebuilt my system (it was the only way to install SGE). I can create files in / which is on the same mount, there is disk space, the drive is mounted rw, fstab looks OK to me (but I'm no expert here). I'm logged in as root. What could the problem be? Below are the commands I tried which show all these facts:

[root@fmhpcc-mn1 ~]# cd /home
[root@fmhpcc-mn1 home]# mkdir usera
mkdir: cannot create directory `usera': No such file or directory
[root@fmhpcc-mn1 home]# whoami
root
[root@fmhpcc-mn1 home]# ls -lart
total 4
drwxr-xr-x 2 root root 0 Jan 29 07:19 .
drwxr-xr-x 27 root root 4096 Jan 29 14:09 ..
[root@fmhpcc-mn1 home]# ls -lart / | grep home
drwxr-xr-x 2 root root 0 Jan 29 07:19 home
[root@fmhpcc-mn1 home]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 15872604 4439676 10613616 30% /
/dev/sda5 54869596 2723256 49314084 6% /state/partition1
/dev/sda2 3968124 206560 3556736 6% /var
tmpfs 4091160 0 4091160 0% /dev/shm
tmpfs 1997636 2304 1995332 1% /var/lib/ganglia/rrds
[root@fmhpcc-mn1 home]# mount
/dev/sda1 on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/sda5 on /state/partition1 type ext3 (rw)
/dev/sda2 on /var type ext3 (rw)
tmpfs on /dev/shm type tmpfs (rw)
tmpfs on /var/lib/ganglia/rrds type tmpfs (rw,size=2045580000,gid=99,uid=99)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
nfsd on /proc/fs/nfsd type nfsd (rw)
[root@fmhpcc-mn1 home]# cat /etc/fstab
LABEL=/ / ext3 defaults 1 1
LABEL=/state/partition /state/partition1 ext3 defaults 1 2
LABEL=/var /var ext3 defaults 1 2
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
LABEL=SWAP-sda3 swap swap defaults 0 0

# The ram-backed filesystem for ganglia RRD graph databases.
tmpfs /var/lib/ganglia/rrds tmpfs size=2045580000,gid=nobody,uid=nobody,defaults 1 0
[root@fmhpcc-mn1 home]# mkdir /testifthisworks
[root@fmhpcc-mn1 home]# ls -lart / | grep test
drwxr-xr-x 2 root root 4096 Jan 29 14:22 testifthisworks
[root@fmhpcc-mn1 home]#

WantLinuxHelp · 01-29-2009, 02:38 PM

I forgot one additional thing. I cannot delete the directory either:

[root@fmhpcc-mn1 home]# cd /
[root@fmhpcc-mn1 /]# ls -lart | grep home
drwxr-xr-x 2 root root 0 Jan 29 07:19 home
[root@fmhpcc-mn1 /]# ls -lart home
total 4
drwxr-xr-x 2 root root 0 Jan 29 07:19 .
drwxr-xr-x 28 root root 4096 Jan 29 14:22 ..
[root@fmhpcc-mn1 /]# rmdir home
rmdir: home: Device or resource busy
[root@fmhpcc-mn1 /]#

WantLinuxHelp · 01-29-2009, 04:04 PM

This continues to get more weird.

I booted up a CentOS install disk in "linux rescue" mode. when one does that, the drive is mounted in /mnt/sysimage, so I went to /mnt/sysimage/home and created the directory. It worked! So, I rebooted to the hard drive and went to /home to see if just rebooting cleared the problem by trying to create another directory there. Here is the weird thing: the directory I just created was GONE! I though, maybe in rescue mode I forgot to go to /mnt/sysimage/home and accidentally created it in /home, so I rebooted to the "linux rescue" and went to /mnt/sysimage/home. Guess what? The new sub-directory was STILL THERE!

OK, I just don't understand what is going on now. Can anyone help?

anomie · 01-29-2009, 05:37 PM

I'm not sure why you are trying to delete /home at this point, but back to your first post: I'd recommend booting into single-user mode and running a fsck on /dev/sda1. See if that corrects the problem.

If it does not, post the output of:
# lsattr -d /home

WantLinuxHelp · 01-29-2009, 06:26 PM

Thanks for your input, anomie

Quote:

Originally Posted by anomie

I'm not sure why you are trying to delete /home at this point,

Well, I normally wouldn't, but it seems to be unusable. I thought I might try to recreate it.

Quote:

Originally Posted by anomie

but back to your first post: I'd recommend booting into single-user mode and running a fsck on /dev/sda1. See if that corrects the problem.

It said "clean", and it didn't help. It's a brand new filesystem.

Quote:

Originally Posted by anomie

If it does not, post the output of:
# lsattr -d /home

[root@fmhpcc-mn1 ~]# lsattr -d /home
lsattr: Inappropriate ioctl for device While reading flags on /home
[root@fmhpcc-mn1 ~]# rocks list host

WantLinuxHelp · 01-29-2009, 07:07 PM

A private reply suggested that it could be SELinux. SELinux is on the system, but it is disabled. They suggested that when I reinstalled, if the drive was not reformatted, that possibly the dir or partition was somehow labeled under SELinux and the identifiers have changed.

My question about that: could such a thing occur even with SELinux disabled?

WantLinuxHelp · 01-29-2009, 07:47 PM

I enabled SELinux and put it in "permissive" mode. When I did that, it said it would have to relabel the entire device upon reboot, so I rebooted. After reboot, and even in permissive mode, I was unable to create the user home directory in /home, so my hypothesis is that SELinux is not the problem

anomie · 01-29-2009, 09:23 PM

It's strange -- a quick google on "lsattr: Inappropriate ioctl for device While reading flags" turned up this.

But the output you posted above indicates:

Code:

# mount
/dev/sda1 on / type ext3 (rw)

So /home should be ext3.

My only additional WAG at this point is that the "rocks" service (which I am not familiar with) is causing some interesting filesystem behavior. Would it be possible to temporarily disable it / unload it so that you can try to eliminate it as a possibility?

WantLinuxHelp · 01-30-2009, 07:35 AM

Quote:

Originally Posted by anomie

It's strange -- a quick google on "lsattr: Inappropriate ioctl for device While reading flags" turned up this.

But the output you posted above indicates:

Code:

# mount
/dev/sda1 on / type ext3 (rw)

So /home should be ext3.

My only additional WAG at this point is that the "rocks" service (which I am not familiar with) is causing some interesting filesystem behavior. Would it be possible to temporarily disable it / unload it so that you can try to eliminate it as a possibility?

Thanks again for the followup. Rocks isn't really a service. It's an installer from SDSC that installs compatible services and pushes distributions out to compute nodes in a cluster. Rocks helps manage the software on the cluster. So, really here, we're just talking about CentOS.

I've posted the question at the Rocks mailing list, but it's held up for moderator aproval.

I've looked at the link you sent (thanks for that). So far I am not understanding it, but I'll look some more when I get in to work this morning.

Thanks again!

WantLinuxHelp · 01-30-2009, 11:17 AM

I tried with strace. Does this make sense to anyone? What is coreutils.mo? Does that have anythign to do with the problem?

[root@fmhpcc-mn1 home]# strace mkdir usera
execve("/bin/mkdir", ["mkdir", "usera"], [/* 57 vars */]) = 0
brk(0) = 0x5cde000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aef6c7bb000
uname({sys="Linux", node="fmhpcc-mn1.fieldmuseum.org", ...}) = 0
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/opt/gridengine/lib/lx26-amd64/tls/x86_64/libselinux.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/opt/gridengine/lib/lx26-amd64/tls/x86_64", 0x7fff3e2ea7d0) = -1 ENOENT (No such file or directory)
open("/opt/gridengine/lib/lx26-amd64/tls/libselinux.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/opt/gridengine/lib/lx26-amd64/tls", 0x7fff3e2ea7d0) = -1 ENOENT (No such file or directory)
open("/opt/gridengine/lib/lx26-amd64/x86_64/libselinux.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/opt/gridengine/lib/lx26-amd64/x86_64", 0x7fff3e2ea7d0) = -1 ENOENT (No such file or directory)
open("/opt/gridengine/lib/lx26-amd64/libselinux.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/opt/gridengine/lib/lx26-amd64", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/etc/ld.so.cache", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=84607, ...}) = 0
mmap(NULL, 84607, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2aef6c7bc000
close(3) = 0
open("/lib64/libselinux.so.1", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@E`=0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=95464, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aef6c7d1000
mmap(0x303d600000, 2192784, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x303d600000
mprotect(0x303d615000, 2097152, PROT_NONE) = 0
mmap(0x303d815000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15000) = 0x303d815000
mmap(0x303d817000, 1424, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x303d817000
close(3) = 0
open("/opt/gridengine/lib/lx26-amd64/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/lib64/libc.so.6", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260\331!<0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1699880, ...}) = 0
mmap(0x303c200000, 3481848, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x303c200000
mprotect(0x303c34a000, 2093056, PROT_NONE) = 0
mmap(0x303c549000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x149000) = 0x303c549000
mmap(0x303c54e000, 16632, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x303c54e000
close(3) = 0
open("/opt/gridengine/lib/lx26-amd64/libdl.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/lib64/libdl.so.2", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\16\240<0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=23360, ...}) = 0
mmap(0x303ca00000, 2109696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x303ca00000
mprotect(0x303ca02000, 2097152, PROT_NONE) = 0
mmap(0x303cc02000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x303cc02000
close(3) = 0
open("/opt/gridengine/lib/lx26-amd64/libsepol.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/lib64/libsepol.so.1", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340<\340=0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=247528, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aef6c7d2000
mmap(0x303de00000, 2383168, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x303de00000
mprotect(0x303de3b000, 2097152, PROT_NONE) = 0
mmap(0x303e03b000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3b000) = 0x303e03b000
mmap(0x303e03c000, 40256, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x303e03c000
close(3) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aef6c7d3000
arch_prctl(ARCH_SET_FS, 0x2aef6c7d33d0) = 0
mprotect(0x303c549000, 16384, PROT_READ) = 0
mprotect(0x303cc02000, 4096, PROT_READ) = 0
mprotect(0x303c01a000, 4096, PROT_READ) = 0
munmap(0x2aef6c7bc000, 84607) = 0
access("/etc/selinux/", F_OK) = 0
brk(0) = 0x5cde000
brk(0x5cff000) = 0x5cff000
open("/etc/selinux/config", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=449, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aef6c7bc000
read(3, "# This file controls the state o"..., 4096) = 449
read(3, "", 4096) = 0
close(3) = 0
munmap(0x2aef6c7bc000, 4096) = 0
open("/proc/mounts", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aef6c7bc000
read(3, "rootfs / rootfs rw 0 0\n/dev/root"..., 4096) = 714
close(3) = 0
munmap(0x2aef6c7bc000, 4096) = 0
open("/selinux/mls", O_RDONLY) = 3
read(3, "1", 19) = 1
close(3) = 0
socket(PF_FILE, SOCK_STREAM, 0) = 3
connect(3, {sa_family=AF_FILE, path="/var/run/setrans/.setrans-unix"}, 110) = -1 ENOENT (No such file or directory)
close(3) = 0
open("/usr/lib/locale/locale-archive", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=56417728, ...}) = 0
mmap(NULL, 56417728, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2aef6c7d4000
close(3) = 0
mkdir("usera", 0777) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/locale.alias", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=2528, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aef6fda2000
read(3, "# Locale name alias data base.\n#"..., 4096) = 2528
read(3, "", 4096) = 0
close(3) = 0
munmap(0x2aef6fda2000, 4096) = 0
open("/usr/share/locale/en_US.iso885915/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_US/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en.iso885915/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
write(2, "mkdir: ", 7mkdir: ) = 7
write(2, "cannot create directory `usera\'", 31cannot create directory `usera') = 31
open("/usr/share/locale/en_US.iso885915/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_US/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en.iso885915/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
write(2, ": No such file or directory", 27: No such file or directory) = 27
write(2, "\n", 1
) = 1
close(1) = 0
exit_group(1) = ?
[root@fmhpcc-mn1 home]#

anomie · 01-30-2009, 11:56 AM

I captured some strace mkdir ... output as well (on a CentOS4 box), and the output is very similar right up until the actual mkdir call. Mine shows instead:

Code:

mkdir("foo", 0777)                      = 0
exit_group(0)                           = ?

Could you try something similar with an absolute path? i.e. # mkdir /home/foo and # touch /home/bar

At least while working with a relative path, your system seems to believe that /home does not exist.

WantLinuxHelp · 01-30-2009, 12:22 PM

Quote:

Originally Posted by anomie

I captured some strace mkdir ... output as well (on a CentOS4 box), and the output is very similar right up until the actual mkdir call. Mine shows instead:

Code:

mkdir("foo", 0777)                      = 0
exit_group(0)                           = ?

Could you try something similar with an absolute path? i.e. # mkdir /home/foo and # touch /home/bar

At least while working with a relative path, your system seems to believe that /home does not exist.

Thanks again, anomie. I really appreciate your advice and time.

I have completely disabled SELinux. Not only did I change the line in /etc/selinux/config to

Code:

     SELINUX=disabled

but I added

Code:

     selinux=0

to the end of my kernel line in /boot/grub/grub.conf

(I had disabled SELinux previously in the GUI, but that did not change the grub.conf line, only the line in /etc/selinux/config)

Here are the results of the commands with absolute paths as you requested:

Code:

[root@fmhpcc-mn1 home]# strace mkdir /home/usera
execve("/bin/mkdir", ["mkdir", "/home/usera"], [/* 57 vars */]) = 0
brk(0)                                  = 0x575e000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2ad1d0225000
uname({sys="Linux", node="fmhpcc-mn1.fieldmuseum.org", ...}) = 0
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/opt/gridengine/lib/lx26-amd64/tls/x86_64/libselinux.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/opt/gridengine/lib/lx26-amd64/tls/x86_64", 0x7fffda880d70) = -1 ENOENT (No such file or directory)
open("/opt/gridengine/lib/lx26-amd64/tls/libselinux.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/opt/gridengine/lib/lx26-amd64/tls", 0x7fffda880d70) = -1 ENOENT (No such file or directory)
open("/opt/gridengine/lib/lx26-amd64/x86_64/libselinux.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/opt/gridengine/lib/lx26-amd64/x86_64", 0x7fffda880d70) = -1 ENOENT (No such file or directory)
open("/opt/gridengine/lib/lx26-amd64/libselinux.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/opt/gridengine/lib/lx26-amd64", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/etc/ld.so.cache", O_RDONLY)      = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=84607, ...}) = 0
mmap(NULL, 84607, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2ad1d0226000
close(3)                                = 0
open("/lib64/libselinux.so.1", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@E`=0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=95464, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2ad1d023b000
mmap(0x303d600000, 2192784, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x303d600000
mprotect(0x303d615000, 2097152, PROT_NONE) = 0
mmap(0x303d815000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15000) = 0x303d815000
mmap(0x303d817000, 1424, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x303d817000
close(3)                                = 0
open("/opt/gridengine/lib/lx26-amd64/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/lib64/libc.so.6", O_RDONLY)      = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260\331!<0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1699880, ...}) = 0
mmap(0x303c200000, 3481848, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x303c200000
mprotect(0x303c34a000, 2093056, PROT_NONE) = 0
mmap(0x303c549000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x149000) = 0x303c549000
mmap(0x303c54e000, 16632, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x303c54e000
close(3)                                = 0
open("/opt/gridengine/lib/lx26-amd64/libdl.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/lib64/libdl.so.2", O_RDONLY)     = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\16\240<0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=23360, ...}) = 0
mmap(0x303ca00000, 2109696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x303ca00000
mprotect(0x303ca02000, 2097152, PROT_NONE) = 0
mmap(0x303cc02000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x303cc02000
close(3)                                = 0
open("/opt/gridengine/lib/lx26-amd64/libsepol.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/lib64/libsepol.so.1", O_RDONLY)  = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340<\340=0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=247528, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2ad1d023c000
mmap(0x303de00000, 2383168, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x303de00000
mprotect(0x303de3b000, 2097152, PROT_NONE) = 0
mmap(0x303e03b000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3b000) = 0x303e03b000
mmap(0x303e03c000, 40256, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x303e03c000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2ad1d023d000
arch_prctl(ARCH_SET_FS, 0x2ad1d023d3d0) = 0
mprotect(0x303c549000, 16384, PROT_READ) = 0
mprotect(0x303cc02000, 4096, PROT_READ) = 0
mprotect(0x303c01a000, 4096, PROT_READ) = 0
munmap(0x2ad1d0226000, 84607)           = 0
access("/etc/selinux/", F_OK)           = 0
brk(0)                                  = 0x575e000
brk(0x577f000)                          = 0x577f000
open("/etc/selinux/config", O_RDONLY)   = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=447, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2ad1d0226000
read(3, "# This file controls the state o"..., 4096) = 447
read(3, "", 4096)                       = 0
close(3)                                = 0
munmap(0x2ad1d0226000, 4096)            = 0
open("/proc/mounts", O_RDONLY)          = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2ad1d0226000
read(3, "rootfs / rootfs rw 0 0\n/dev/root"..., 4096) = 683
read(3, "", 4096)                       = 0
close(3)                                = 0
munmap(0x2ad1d0226000, 4096)            = 0
open("/usr/lib/locale/locale-archive", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=56417728, ...}) = 0
mmap(NULL, 56417728, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2ad1d023e000
close(3)                                = 0
mkdir("/home/usera", 0777)              = -1 ENOENT (No such file or directory)
open("/usr/share/locale/locale.alias", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=2528, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2ad1d380c000
read(3, "# Locale name alias data base.\n#"..., 4096) = 2528
read(3, "", 4096)                       = 0
close(3)                                = 0
munmap(0x2ad1d380c000, 4096)            = 0
open("/usr/share/locale/en_US.iso885915/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_US/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en.iso885915/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
write(2, "mkdir: ", 7mkdir: )                  = 7
write(2, "cannot create directory `/home/u"..., 37cannot create directory `/home/usera') = 37
open("/usr/share/locale/en_US.iso885915/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_US/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en.iso885915/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
write(2, ": No such file or directory", 27: No such file or directory) = 27
write(2, "\n", 1
)                       = 1
close(1)                                = 0
exit_group(1)                           = ?
[root@fmhpcc-mn1 home]#                    
[root@fmhpcc-mn1 home]# 
[root@fmhpcc-mn1 home]# 
[root@fmhpcc-mn1 home]# 
[root@fmhpcc-mn1 home]# 
[root@fmhpcc-mn1 home]# 
[root@fmhpcc-mn1 home]# strace touch /home/usera
execve("/bin/touch", ["touch", "/home/usera"], [/* 57 vars */]) = 0
brk(0)                                  = 0x1be5000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b9c498f3000
uname({sys="Linux", node="fmhpcc-mn1.fieldmuseum.org", ...}) = 0
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/opt/gridengine/lib/lx26-amd64/tls/x86_64/librt.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/opt/gridengine/lib/lx26-amd64/tls/x86_64", 0x7fff611b46a0) = -1 ENOENT (No such file or directory)
open("/opt/gridengine/lib/lx26-amd64/tls/librt.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/opt/gridengine/lib/lx26-amd64/tls", 0x7fff611b46a0) = -1 ENOENT (No such file or directory)
open("/opt/gridengine/lib/lx26-amd64/x86_64/librt.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/opt/gridengine/lib/lx26-amd64/x86_64", 0x7fff611b46a0) = -1 ENOENT (No such file or directory)
open("/opt/gridengine/lib/lx26-amd64/librt.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/opt/gridengine/lib/lx26-amd64", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/etc/ld.so.cache", O_RDONLY)      = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=84607, ...}) = 0
mmap(NULL, 84607, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2b9c498f4000
close(3)                                = 0
open("/lib64/librt.so.1", O_RDONLY)     = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 \"\240=0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=53448, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b9c49909000
mmap(0x303da00000, 2132936, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x303da00000
mprotect(0x303da07000, 2097152, PROT_NONE) = 0
mmap(0x303dc07000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x7000) = 0x303dc07000
close(3)                                = 0
open("/opt/gridengine/lib/lx26-amd64/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/lib64/libc.so.6", O_RDONLY)      = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260\331!<0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1699880, ...}) = 0
mmap(0x303c200000, 3481848, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x303c200000
mprotect(0x303c34a000, 2093056, PROT_NONE) = 0
mmap(0x303c549000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x149000) = 0x303c549000
mmap(0x303c54e000, 16632, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x303c54e000
close(3)                                = 0
open("/opt/gridengine/lib/lx26-amd64/libpthread.so.0", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/lib64/libpthread.so.0", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@W\340<0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=141456, ...}) = 0
mmap(0x303ce00000, 2200432, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x303ce00000
mprotect(0x303ce15000, 2093056, PROT_NONE) = 0
mmap(0x303d014000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x14000) = 0x303d014000
mmap(0x303d016000, 13168, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x303d016000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b9c4990a000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b9c4990b000
arch_prctl(ARCH_SET_FS, 0x2b9c4990af10) = 0
mprotect(0x303dc07000, 4096, PROT_READ) = 0
mprotect(0x303c549000, 16384, PROT_READ) = 0
mprotect(0x303d014000, 4096, PROT_READ) = 0
mprotect(0x303c01a000, 4096, PROT_READ) = 0
munmap(0x2b9c498f4000, 84607)           = 0
set_tid_address(0x2b9c4990afa0)         = 4386
set_robust_list(0x2b9c4990afb0, 0x18)   = 0
rt_sigaction(SIGRTMIN, {0x303ce05360, [], SA_RESTORER|SA_SIGINFO, 0x303ce0de80}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x303ce052b0, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x303ce0de80}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=10240*1024, rlim_max=RLIM_INFINITY}) = 0
brk(0)                                  = 0x1be5000
brk(0x1c06000)                          = 0x1c06000
open("/usr/lib/locale/locale-archive", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=56417728, ...}) = 0
mmap(NULL, 56417728, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2b9c4990c000
close(3)                                = 0
close(0)                                = 0
open("/home/usera", O_WRONLY|O_CREAT|O_NOCTTY|O_NONBLOCK, 0666) = -1 ENOENT (No such file or directory)
futimesat(AT_FDCWD, "/home/usera", NULL) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/locale.alias", O_RDONLY) = 0
fstat(0, {st_mode=S_IFREG|0644, st_size=2528, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b9c4ceda000
read(0, "# Locale name alias data base.\n#"..., 4096) = 2528
read(0, "", 4096)                       = 0
close(0)                                = 0
munmap(0x2b9c4ceda000, 4096)            = 0
open("/usr/share/locale/en_US.iso885915/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_US/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en.iso885915/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
write(2, "touch: ", 7touch: )                  = 7
write(2, "cannot touch `/home/usera\'", 26cannot touch `/home/usera') = 26
open("/usr/share/locale/en_US.iso885915/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_US/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en.iso885915/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
write(2, ": No such file or directory", 27: No such file or directory) = 27
write(2, "\n", 1
)                       = 1
close(1)                                = 0
exit_group(1)                           = ?
[root@fmhpcc-mn1 home]#

WantLinuxHelp · 01-30-2009, 12:55 PM

I think this is an automount/nfs issue. The machine is supposed to be the server. Rocks set up the nfs. I'm still looking and haven't solved it yet.

WantLinuxHelp · 02-02-2009, 12:11 PM

Quote:

Originally Posted by WantLinuxHelp

I think this is an automount/nfs issue. The machine is supposed to be the server. Rocks set up the nfs. I'm still looking and haven't solved it yet.

I have not been able to figure out anything with automount/nfs, nor have I solved this yet. I appreciate any and all suggestions. Thanks!

WantLinuxHelp · 07-19-2010, 09:01 AM

This was never solved. I eventually gave up and scrapped this system.