Solaris variants and ZFS for file server

madivad · 03-13-2008, 08:21 PM

I have created a new thread since I had drifted way off topic in my own thread on internet connectivity. A quick update, starting with my original post on this question:
______________________________________

Quote:

Originally Posted by jlliagre

They aren't exactly the same thing...

Now I'm only getting more confused. Maybe a very pointed question will help:

I am not very fluent in unix (I have been playing with solaris 10 for the last few days and pull my hair out over some of the simple things I just can't find or do - ie, things I can do easily enough in ubuntu, xandros, and others)... All I am after is an install that runs ZFS. I don't want any games, no open office, not even internet support. A simple file server for a few hard drives I have lying around here...

The Question: Which solaris version includes the shortest learning curve and will suit my needs?
______________________________________

Quote:

Originally Posted by kebabbert

OpenSolaris (aka Solaris Express) is very similar to a modern Linux like RedHat, SuSE, etc. S10 feels a bit antique. OpenSolaris autodetects most hardware, and is my main choice unless it is for enterprise purposes. I recommend strongly Opensolaris, just as guru Jilliagre.

______________________________________

Quote:

Originally Posted by jlliagre

As you now have a working Solaris 10 and you are looking for a platform that will be somewhat in "production", I would suggest you to stay with it for your ZFS file server.

On the other hand, if you want to stay closer to the bleeding edge, Solaris Express releases are better.

Feel free to ask here again about the simple or not so simple things you have trouble with ...

I have given it a go over the last couple of days, and the learning curve seems huge. I am pulling my hair out over simple things like trying to find the new hard drive that is installed and where the heck is my second network card. eg, network #1, there was nothing in the /dev/ regarding the network.

I did # ifconfig rge0 plumb or something similar and all of a sudden I am half way there to having it working. But where the heck did rge0 come from? How was I supposed to know that's where it would appear in /dev when it wasn't even in there to begin with? Now I am having the same problem with my new WD1TB HDD. (I found it in the top of dmesg, so the system knows about it)... I also am having trouble running pre-compiled apps (synergy) and I don't even know how to add packages like gcc yet...

I am not use to being too much of a noob and I hate it! But I have well and truly digressed again...

madivad · 03-13-2008, 08:22 PM

Quote:

Originally Posted by jlliagre

As you now have a working Solaris 10

Well, I confess, there seems to be much wrong with it, I am not sure if it is working with some errors, or is just scraping through by the skin of its teeth (or it is just me). But I think there is an issue with it (S10) just not recognising my hardware or something like that). I don't know that it has identified and works with my MB SATA controller

Before ditching it and trying OpenSolaris or Solaris Express, I have realised that half of my problem may have been as root I was using some java desktop and I could not find any administrative tools anywhere. When I created my user account, I went for the CDE? desktop and found it more "navigate-able" and found certain administration features that I was looking for (but don't all work).

Quote:

Originally Posted by jlliagre

Feel free to ask here again about the simple or not so simple things you have trouble with ...

I figured out how to use SMC as root in my user account, but when I go to Disks, BOTH are there (SATA controller must be working), but clicking on either one gives me a
Warning: No Solaris Fdisk Partition

This includes my new 1TB drive, but also includes the OS drive... The one that I am running the install on!

Breakfast may relieve the frustration

jlliagre · 03-13-2008, 08:47 PM

Quote:

Originally Posted by madivad

I am pulling my hair out over simple things like trying to find the new hard drive that is installed and where the heck is my second network card. eg, network #1, there was nothing in the /dev/ regarding the network.

Don't assume the way things works with Gnu/Linux will be the same with other OSes.

Quote:

I did # ifconfig rge0 plumb or something similar and all of a sudden I am half way there to having it working.

Better to run "ifconfig -a plumb" and have all network interfaces made available without knowing their names.

Quote:

But where the heck did rge0 come from?

Unlike with the Linux kernel where all interfaces are names ethxx, network interfaces are named according to the driver used to access them. rge is the Realtek Gigabit/Fast Ethernet Network Adapter driver. You can get some information about it with "man rge".

Quote:

How was I supposed to know that's where it would appear in /dev when it wasn't even in there to begin with?

There is actually no need to know this device exists in /dev.

Quote:

and I don't even know how to add packages like gcc yet...

You probably don't need to install gcc as it is already there. Just add /usr/sfw/bin to your PATH.

jlliagre · 03-13-2008, 08:52 PM

Quote:

Originally Posted by madivad

Before ditching it and trying OpenSolaris or Solaris Express, I have realised that half of my problem may have been as root I was using some java desktop and I could not find any administrative tools anywhere. When I created my user account, I went for the CDE? desktop and found it more "navigate-able" and found certain administration features that I was looking for (but don't all work).

Solaris administration is mostly done with the command line so using CDE or JDS doesn't make that much of a difference. These GUI you found are often missing functionalities. I almost never use them.

Quote:

I figured out how to use SMC as root in my user account, but when I go to Disks, BOTH are there (SATA controller must be working), but clicking on either one gives me a
Warning: No Solaris Fdisk Partition

This includes my new 1TB drive, but also includes the OS drive... The one that I am running the install on!

So what do you want to know/do with your disks ?

madivad · 03-14-2008, 12:08 AM

Quote:

Originally Posted by jlliagre

Solaris administration is mostly done with the command line so using CDE or JDS doesn't make that much of a difference. These GUI you found are often missing functionalities. I almost never use them.

yeah, i only went searching because I couldn't get the useradd to work. Turns out it didn't like me specifying /home/david as my home directory and I later learned it has to be /export/home/david instead.

Quote:

So what do you want to know/do with your disks ?

Well, I want to start getting this 1TB disk ready for ZFS. Do I have to have a minimum of 2 to get started? I'm picking up the next one next week.

I can't get fdisk to see it at all, it doesn't matter what I type as the /dev/dsk/xxx I get errors, namely:
1) xxx must be a raw device, or
2) cannot stat device

I can't even appear to do it for the disk that is already installed and running either.

I did find:

Code:

# prtvtoc /dev/dsk/c
c0d0p0     c0d0s12    c0d0s7     c0t1d0s1   c0t1d0s4   c1d0p3     c1d0s15
c0d0p1     c0d0s13    c0d0s8     c0t1d0s10  c0t1d0s5   c1d0p4     c1d0s2
c0d0p2     c0d0s14    c0d0s9     c0t1d0s11  c0t1d0s6   c1d0s0     c1d0s3
c0d0p3     c0d0s15    c0t1d0p0   c0t1d0s12  c0t1d0s7   c1d0s1     c1d0s4
c0d0p4     c0d0s2     c0t1d0p1   c0t1d0s13  c0t1d0s8   c1d0s10    c1d0s5
c0d0s0     c0d0s3     c0t1d0p2   c0t1d0s14  c0t1d0s9   c1d0s11    c1d0s6
c0d0s1     c0d0s4     c0t1d0p3   c0t1d0s15  c1d0p0     c1d0s12    c1d0s7
c0d0s10    c0d0s5     c0t1d0p4   c0t1d0s2   c1d0p1     c1d0s13    c1d0s8
c0d0s11    c0d0s6     c0t1d0s0   c0t1d0s3   c1d0p2     c1d0s14    c1d0s9
# prtvtoc /dev/dsk/c0d0s0
* /dev/dsk/c0d0s0 partition map
*
* Dimensions:
*     512 bytes/sector
*      63 sectors/track
*     255 tracks/cylinder
*   16065 sectors/cylinder
*   14592 cylinders
*   14590 accessible cylinders
*
* Flags:
*   1: unmountable
*  10: read-only
*
* Unallocated space:
*       First     Sector    Last
*       Sector     Count    Sector 
*   234372285     16065 234388349
*
*                          First     Sector    Last
* Partition  Tag  Flags    Sector     Count    Sector  Mount Directory
       0      2    00    1124550   8900010  10024559   /
       1      3    01      48195   1076355   1124549
       2      5    00          0 234388350 234388349
       7      8    00   10024560 224347725 234372284   /export/home
       8      1    01          0     16065     16064
       9      9    01      16065     32130     48194
# prtvtoc /dev/dsk/c1d0s0
* /dev/dsk/c1d0s0 partition map
*
* Dimensions:
*     512 bytes/sector
*     126 sectors/track
*     255 tracks/cylinder
*   32130 sectors/cylinder
*   60799 cylinders
*   60797 accessible cylinders
*
* Flags:
*   1: unmountable
*  10: read-only
*
* Unallocated space:
*       First     Sector    Last
*       Sector     Count    Sector 
*       96390 1953311220 1953407609
*
*                          First     Sector    Last
* Partition  Tag  Flags    Sector     Count    Sector  Mount Directory
       2      5    01          0 1953407610 1953407609
       8      1    01          0     32130     32129
       9      9    00      32130     64260     96389
#

If you couls just point me towards the tools I would need to use, I will google and man page them myself to do some research.

Thanks
dave

jlliagre · 03-14-2008, 03:26 AM

You can get information about your whole disk with this command:

Code:

fdisk -G /dev/rdsk/c0d0p0

and partition information with that one:

Code:

fdisk -v -W - /dev/rdsk/c0d0p0

You can use ZFS with a single disk, partition or slice. Two dedicated disks or more is better.

madivad · 03-14-2008, 04:18 AM

of course... that's what I was doing wrong. I needed to use /dev/rdsk/...

/me is banging head on desk

what do c, d, p, s and t stand for? how can I tell the raw device names of the drives connected to my system? Is there a table somewhere?

jlliagre · 03-14-2008, 05:15 AM

c: controller
d: disk
p: partition (primary fdisk)
s: slice (kind of extended partition inside the Solaris partition)
t: target (SCSI)

You can get the disks connected by using the format command. Just exit after the disks names have been shown.

Another useful command to get more information about the disks is "kstat -c device_error"

madivad · 03-14-2008, 05:46 AM

I have just installed another IDE drive, but it's not showing up anywhere.

The format command only gives me the two drives that were last on the system before I rebooted:

Code:

AVAILABLE DISK SELECTIONS:
       0. c0d0 <DEFAULT cyl 14590 alt 2 hd 255 sec 63>
          /pci@0,0/pci-ide@1f,1/ide@0/cmdk@0,0
       1. c1d0 <DEFAULT cyl 60797 alt 2 hd 255 sec 126>
          /pci@0,0/pci-ide@1f,2/ide@0/cmdk@0,0
Specify disk (enter its number): ^C

"kstat -c device_error" returned nothing

So how do I add the drive I just installed?

There are no new drive types indicated in /dev/rdsk (I know you told me not to worry about that anyway), so I am at a loss.

Here is my dmesg, you can see in there that there are three drives identified:
21:16:50 (ST3120022A) which registered at /dev/rdsk/c0d0p0
21:16:50 (ST3320620A) which does not get registered as a drive anywhere (that I know of)
21:17:12 (WDC WD10EACS-00Z) which registers as /dev/rdsk/c1d0p0

Code:

<dmesg log removed - old system>

There are a couple of other things I'd like to fix up in there, but let's start with the disk for the moment.

kebabbert · 03-14-2008, 06:44 AM

I still strongly urge you to try OpenSolaris. It is the beta for S11 and it is rock stable. Very much stable than any Linux for instance. It is far simpler than S10 too.

Ive had exactly the same experience as you. First Ive installed S10, and had a helluva problem setting it up. If you have SUN hardware, then S10 works right out of the box because it supports the SUN hardware. Now I have commodity P4@2.4GHz so there were no drivers included, etc. Then Jilliagre suggested OpenSolaris instead, and I had concerns of stability etc. It turns out that OpenSolaris kernel beta is far mor stable than any Linux kernel. Now I use Opensolaris and everything just works! It recognized my GeForce 6200 AGP, and the sound card, and my Nec 20WGX2M monitor correctly. etc. I strongly suggest that you try OpenSolaris. If youre used to a modern Linux, then you will be content with OpenSolaris.

madivad · 03-14-2008, 08:33 AM

OpenSolaris, I need to download Solaris Express Community Edition. Is that correct? I've just managed to corrupt one of the drives, so a full reinstall is easier, so it may as well happen now.

It can't be any harder than what Solaris 10 was, so hopefully when I get home tomorrow I can get it done very quickly.

jlliagre · 03-14-2008, 10:19 AM

My answer is a little late but you should have had the new disk to appear after running the "devfsadm -v" command.

And yes, download Solaris Express Community Edition for the latest and greatest features.

madivad · 03-14-2008, 02:52 PM

Quote:

Originally Posted by jlliagre

My answer is a little late but you should have had the new disk to appear after running the "devfsadm -v" command.

Not at all, thanks anyway, I just thought I'd have a look before going to work, and now there is a new drive LOL

What you might be able to help me with:

I was experimenting with ZFS and thrashing the harddrive, I was playing around with a couple of pools I had created. Copying files to and fro.

I went away for a moment and came back and the system had crashed. I rebooted, safe mode, did some disk admin that it wanted to do in failsafe and when it rebooted one of the pools were so corrupt that I can't do anything with it. In the web admin screen there is the top level pool and virtual device. I am unable to delete or add to it at all. Not even forcefully.

Any ideas?

The data is test data only and I would like to just start a-fresh (which is another reason I would like to just reinstall and try this opensolaris, but I would also like to be able to see what I can recover in the event of a real system crash anyway

edit: it has got to the stage now that I even tried deleting the partition on that disk and then tried re-accessing it in the webadmin and the system crashes and reboots.

jlliagre · 03-15-2008, 03:49 AM

You need to provide more information.

Why did the system crashed ? Solaris crashing is an unusual event. Any message in /var/log/messages ? Is there a crash dump in /var/crash/hostname/* ?

What return:

Code:

fdisk -v -W - /dev/rdsk/c0d0p0 # for each disk
zpool status -v
zpool list
zfs list

madivad · 03-15-2008, 04:18 PM

Quote:

Originally Posted by jlliagre

Why did the system crashed ? Solaris crashing is an unusual event.

That's what I thought. Admittedly, I was pushing the system, testing it out before going live. I was copying many files to dirs, and re copying them into nested groups and copying them recursively. Then I tried formatting another space and was bringing up the web gui when i went outside for a moment. When I returned, the PC had rebooted and stopped at the (ctrl+d to bypass root login) message.

I think it had to do with the way that i set up the zpool, I'm still not sure of that aspect 100% and feel that I was copying files into the pool and not the fs. But I'm not too sure... I know, but it is hard to find a good base setup for zpool/zfs. (explain later)

Quote:

Any message in /var/log/messages ? Is there a crash dump in /var/crash/hostname/* ?

I have already reinstalled, so that is gone. I tried the route of OpenSolaris and so far it has been worse. I have only given it a few hours and need to retry it when I get home this afternoon. (to break this up, I will elaborate next post)

regardong zpool and zfs, this is my understanding:

zpool = a group of (usually) drives together in a pool, eg 4x200GB drives = 1x800GB pool

zfs = the filesystem associated with a pool. It is here that you create a file system based on the redundency you need. ie, you could create 2x400GB fs in some RAID-Z config which would return 2x300GB usable redundant file system base, yes?

Then isn't there some sort of virtual setup underneath that? I think that is where i was really getting confused (not that I am not confused already). Looking at the web gui (which is not available to me right now), there were many aspects of the zpool able to be created and I just dont understand where each related to each other. I have been trying to find a nice graphical representation of where they meet and relate, do you have a link?