what is optimal install setup for OS; 2 SCSI, one is faster but has limited space?
 

I have finally solved my multi- OS booting woes and am ready to re-install Ubuntu Studio Hardy LTS. I plan on using it for multimedia purposes including video editing/encoding, using Wine and also want to web browse with the most speed possible. I have an HP workstation circa 2001/02 with SCSIs and adaptec controller at UltraWide 160 and one recent 500 GiG IDE drive (with /boot partition incl. "master" grub with chainloading, other linux OS and various data partitions). I have only 11 GiG free space on 1st SCSI (Fujitsu man3367MP, 36 GiGs) with Windows on the first partition, but I can squeeze in another 2 GiGs if necessary. My other SCSI (2cnd on the controller chain) is an 18 GiG Seagate Barracuda 36ES,LVD with slower rpm (7200 vs 10000) and the average access r,w(8.5/9.1 ms) is almost 2X slower than my Fujitsu drive.

I was thinking of setting up / on fastest SCSI and /home on slower. I already have 2 GIGS swap on both slow SCSI and IDE. I can select to put swap priority on the faster drive; in this case this would be the IDE- (with more heads, and, since I would be rendering the video files to /home- the drive not being written to can access swap faster).

I have an ext3 data partition on my IDE, which I would like to render or process my files to, but I noticed that some applications such as DVD ripper (forgot the exact name) force me to write/render/encode to my /home. Should I set my swap priority equal in case I can render to my IDE drive?

here is an excerpt from a swap HOW TO:


"Imagine replacing the entry for the swap partition with these three lines:

/dev/hda6 none swap sw,pri=3 0 0
/dev/hdb2 none swap sw,pri=2 0 0
/dev/hdc2 none swap sw,pri=1 0 0

This configuration would cause the kernel to use /dev/hda6 first. it has the highest priority assigned to it (pri=3). The maximum priority can be 32767 and the lowest 0. If that space were to max out, the kernel would start using /dev/hdb2, and on to /dev/hdc2 after that. Why such a configuration? Imagine that the newest (fastest) drives are given the highest priority. This will minimize speed loss as swap space usage grows.

It is possible to write to all three simulataneously. If each has the same priority, the kernel will write to them much like a RAID, with commensurate speed increases.

/dev/hda6 none swap sw,pri=3 0 0
/dev/hdb2 none swap sw,pri=3 0 0
/dev/hdc2 none swap sw,pri=3 0 0"



One last issue is the /temp files, as some of these video production apps. require a large /temp, but fast access speed is also desired! So do I make another partition just for /temp on the fast SCSI (is 2 GIGS enough for video editing?) or should I make a new (logical) partition on my IDE for /temp?

I am a little confused, since I am completely new to video editing; all I know is that it's better to write/render the files to another drive than the "system" drive. So I don't know if /temp is considered a "system file" and thus should not be on the same drive as where I might potentially render my video files to!

If I chance a certain setup now and discover ways of improving it later by moving folders and creating different partitions, is it just a matter of changing my entries in /etc/fstab or will there be many more steps?

Perhaps the difference in speeds are not worth a complex set-up, as a beginner and amateur anyways, I just want to optimize, considering my hardware limitations(I am stuck with an older but pretty decent nv video card, Quadro4 900XGL and 1 GIG RAM) and especially, I want to avoid dreaded system freeze/crashes!!!

So if anyone can set me straight on all this, it would be appreciated!

Your web browsing speed will be determined by your broadband speed. The fastest broadband is considerably slower than disk speed.

You probably will not do any swapping at all so treat swap as very inactive space. If you do happen to swap the slowdown will be so noticeable that you will immediately try to reconfigure your memory usage to eliminate swapping.

You are correct that the way to get the fastest performance from your IDE devices is to run them in parallel as much as possible. Since swap is rarely used, if ever, swap is not of concern. You do want to spread your most heavily used files across as many simultaneous pathways as possible.

Note that there is a problem with hda and hdb. Two IDE devices on the same cable cannot be accessed simultaneously. So with your setup hda and hdb cannot be run in parallel. I have 3 IDE disks and 1 IDE CD/DVD. In order to run them in parallel I bought a secondary IDE controller and placed each device on a seperate cable:

/dev/hda
/dev/hdc
/dev/hde
/dev/hdg


You may also have to change /boot/grub/menu.lst


-----------------------
Steve Stites

RE: what is optimal install setup for OS; 2 SCSI, one is faster but has limited space
 

What would be the best would be to use the IDE drive as the boot drive for both OS's and set up a RAID on the SCSI drives. But the way you are setup should work. You can tell DVDrip where to rip data to in the preferences. Got to Edit -> Edit Preferences and change the Default data base directory and the Default directory for .rip files to a directory on one of your storage partitions;

Ex. /media/Storage/dvdrip-data

You would have to create the dvdrip-data directory on the storage partition first.

I agree with the swap How to. Running swap files silmiltaneously would speed up swap usage when in use. You should keep the swap on the IDE drive just in case. If there is no swap and you run out of memory then CRASH! And all data currently being saved or written would be gone.

Move the /tmp directory to the storage partition;

Ex. mv /tmp /media/Storage/

and link it to /;

Ex.
cd /
ln -s /media/Storage/tmp .

Similarily, you could link your home directory aswell.

Ex. mv /home /media/Storage/

Ex.
cd /
ln -s /media/Storage/home .

You have limited OS space. Do not use the OS drive to store your data.

Keep in mind ripping DVD's, rendering and doing video editing / encoding takes up large amounts of system bandwith,memory, and data space.

You could even get a large capacity external firewire, USB 2.0, or e-SATA drive and link the /tmp and /home directory to it and the DVDrip data files. I would recommend getting at least another 512MB to 1Gig of memory aswell. Even with that amount of memory you may notice some sluggish behavior.

All my examples here are only examples, do not copy them exact, as your system is setup differently than these examples. These are my opinions and others may have better ideas.

RE: what is optimal install setup for OS; 2 SCSI, one is faster but has limited space
 

What would be the best would be to use the IDE drive as the boot drive for both OS's and set up a RAID on the SCSI drives. But the way you are setup should work. You can tell DVDrip where to rip data to in the preferences. Got to Edit -> Edit Preferences and change the Default data base directory and the Default directory for .rip files to a directory on one of your storage partitions;

Ex. /media/Storage/dvdrip-data

You would have to create the dvdrip-data directory on the storage partition first.

I agree with the swap How to. Running swap files silmiltaneously would speed up swap usage when in use. You should keep the swap on the IDE drive just in case. If there is no swap and you run out of memory then CRASH! And all data currently being saved or written would be gone.

Move the /tmp directory to the storage partition;

Ex. mv /tmp /media/Storage/

and link it to /;

Ex.
cd /
ln -s /media/Storage/tmp .

Similarily, you could link your home directory aswell.

Ex. mv /home /media/Storage/

Ex.
cd /
ln -s /media/Storage/home .

You have limited OS space. Do not use the OS drive to store your data.

Keep in mind ripping DVD's, rendering and doing video editing / encoding takes up large amounts of system bandwith,memory, and data space.

You could even get a large capacity external firewire, USB 2.0, or e-SATA drive and link the /tmp and /home directory to it and the DVDrip data files. I would recommend getting at least another 512MB to 1Gig of memory aswell. Even with that amount of memory you may notice some sluggish behavior.

All my examples here are only examples, do not copy them exact, as your system is setup differently than these examples. These are my opinions and others may have better ideas.

thnxs!!! for the prompt replies.
Jailbait, I hope you realize that the swap description you are refering to was an example pulled from a HOW TO and not my own setup (I only have 1 IDE drive)! I think will try the "raid-like" type of swap, in other words set their priorities at same number. Rjanjon7, I was actually just reading about RAID0 for video editing, its usefulness is contested by some, but I am trying to see if I can set it up for only one partition on each of my SCSIs, since I will not re-install windows, for example.

I hope that Jailbait is wrong about his take on swap, as I believe that my 1 GIG ram (too complicated and expensive to upgrade on this system) may be inefficient for some video apps so I hope that things won't get that slow!

I will try to change the DVDrip settings as you mentionned, Rjanjon7, but are you sure its ok to put temp. on the same drive as where I would write to (the IDE?), isn't it then possible to make 2 GIG partitions on each SCSI and create a 0 raid device with them and then install /temp on there? How much space did I need for temp. (video editing wise) anyways??

Lastly. Ryanjon7, you said my set up is ok, so for the simple set-up of / on fast SCSI and /home on slow SCSI, is 11 GiGs really enough space for my root and temp?

I understand that the howto explained how to maximize swap speed. I don't disagree with the howto. However maximizing swap speed is irrelevant if you don't swap and I don't think that you will do any swapping.

-----------------
Steve Stites

Ya, you could make the two 2 Gig partions and run them as a RAID. That would increase read and write access times to the /tmp folder. However, depending on how much rendering, ripping, and editing you r doing the two 2 gigs RAID ( Which would be 4 gigs of data space ) would probably not be enough, if large amounts of data will be stored in tmp. You have the right ideas . . . If you think about it 1 dvd is 4.7 Gigs of data. If you are editing one video at a time you need atleast that or a little more. Then when done you could delete the unessasary file from /tmp and start on the next edit. If you are rendering lets say animation files, you would definately need a way larger /tmp file. But ya, anything you want to read and write to faster, should be put on the SCSI drives, or RAID (if you create one) to improve performance. Or just put /tmp and /home on the fast SCSI drive and link as I previously specified, if space permits. Another option, would be to back up all your important data, and install windows on the IDE drive to dual boot linux and windows. Then setup the SCSI drives as a RAID 0 or RAID 1 depending what your system is capable of. Then you could use the RAID as fast data drives. You could put /home , /tmp, and swap on them to improve overall performance. But you are right, ya you want /tmp on the SCSI drivers anyhow.

Jailbait I think has a good point. You may very well not even swap, you don't want too. But swap is emergency memory if you run out of hardware mem. It's good to put it on the SCSI anyway incase you do swapp.

Once again, I thankyou both for the quick replies, yet I am still unsure of a few more details. Jailbait, thks for clarifying on swaps; I will keep my swaps for now even if they never/rarely get used, it can't hurt. I want to stick with the / on fastest drive.

But the raid0 "how to" still perplexes me; does hardy ubuntu come with all the software needed to install it or is there something else to add? Is it hell for a newbie to access files afterwards, and can my filesystem be ext3? I also read that the IDE can be included in the raid, which would be of great benefit for expanded space, and that partitions shared don't have to even be of exact same size. I still need Windows XP to access some system consuming apps., not found on linux, so it will stay on fast SCSI.

I am trying to grasp the "linking" concept. So you can have /home and /temp linked in subfolders on the data partition, while their "master" folder is still in another location/drive right? Sorry if I'm nooby on this!
Thus the video files will really be processed in the data partition (on my IDE) if assigned to do so, right?

Here is an excerpt from Cinelerra, pls. look at the section on storage. My video card does not even support open GL2.0, only GL1.0, so that means this will slow things down at the outset! (My CPUs are ok, memory just ok) Is cinelerra really referring to a temp file in the data target partition or does the file get processed straight to data partition? Finally if I am weary of creating my own links, can I just choose to install all of /temp -ubuntu offers this option- on separate partition within my raid array and then for each app. I use just select this /temp folder as the target for the files processed?
Then in this case, slow SCSI and IDE at 10 GIG each should be sufficient in size and hopefully in speed, for small simple projects! Ok, but last question forever, I hope, can a logical partition (from the IDE) be included in the raid, as it is all out of primary partitions!!!

"Cinelerra is demanding on all PC subsystems, as reading, decoding and playing video can be quite taxing. Thus, performance and usability of Cinelerra are directly proportional to the video format (SVCD/DV/HDV/HD/etc) used and the CPU and I/O bus speeds and video and memory bus architecture of your hardware. Therefore, it stands to reason that a less powerful system will be sufficient for users working with audio only or lower resolution video formats. However, that same system may slow down considerably when playing back a higher resolution format, such as DV video. Effects and several tracks of audio will compound these problems. Given these constraints, here are some suggestions for running Cinelerra:

* CPU speed
At least 500 MHz CPU speed, anything less would be useless. Dual-core and SMP processors greatly improve Cinelerra speed.
* Memory
When working with video, a large amount of free memory available can help speed up operations by avoiding unnecessary disk swaps and keeping material ready accessible. Have at least 256 Megabytes of memory. To really use Cinelerra for higher resolution video formats and larger projects, greater than 1 Gb memory space is suggested.
* Storage
Video editing can be quite I/O intensive. Storage requirements are based on your particular video editing needs. If you expect to produce long pieces in uncompressed or larger resolution formats, you should have large (>200 Gb) and fast (<10ms) disk drives. For example, DV uses about 3.5 Megs per second, or 12 Gigs per hour. For smaller projects you might get away with 1 Gb. RAID0 (stripe set), RAID1+0 (striped/mirrored) or RAID5 (stripe set with parity) will also speed playback
* Video adapters
Since version 2.1, Cinelerra benefits from OpenGL hardware acceleration. Make sure the video card you use supports OpenGL 2.0 in order to benefit from that acceleration. Nvidia series 7 (ie. 7600GS) are known to work well. Unfortunately, ATI's Linux drivers do not support a complete implementation of OpenGL 2.0. If you are going to send a composite signal directly to a TV or video recorder, make sure your video card supports it.
* Multiple monitors
You can use XFree86's Xinerama features to work on multiple monitor heads. This feature can be a very effective way of increasing productivity."

(My videocard enables 2 monitors at least!)

So the recap is: going raid0 and installing /temp on that partition/array- all during ubuntu install. Will a logical partition work, and will all/most of the video apps. really allow me to render to this partition without a hitch?
Thanks for bearing with me....