LinuxQuestions.org - FastPath and AIO

- AIX (https://www.linuxquestions.org/questions/aix-43/)

- - FastPath and AIO (https://www.linuxquestions.org/questions/aix-43/fastpath-and-aio-168383/)

FastPath and AIO

Hello All,

We are running Sybase ASE on a P670 16 cpu 24 GB Ram. Our question is about disk access. It seems that Raw devices are accessed through fastpath and FS is accessed through AIO. All of our databases are on RAW Devices. We are not sure if fastpath is Asynchronous or not. Does anyone know the answer to this? Our gut tell us to disable fastpath and use AIO through the OS. Any help would be appreciated.

-Carl

I don't know the answer, but I agree with your gut feeling. With AIX 5, the overhead offered by Raw devices over JFS2 is tiny, if it still exists at all. I know that Oracle/SAP have been recommending customers use LVM (i.e. FS and AIO) for some years, as the management benefits outweight the tiny performance hits.

If you do move over, remember that you can tune the number of aio servers on the fly. I think you get ten by default, but one of the manuals has good instructions on how and when to tune (probably one of the performance manuals or redbooks, but I don't remember).

I assume Fastpath is a Sybase utility as, by definition, raw lvs are accessed directly by the app without AIX getting involved. Maybe Sybase support can answer your question.

iainr thanks for the reply,

FastPath is the way that an app accesses disks without AIX getting involved. It is setup in smit in the aio section.... This is the entire basis for our question... Our gut is that if the app issues an io directly to the shark io subsystem how can it by async?

IBM has not been able to answer this question yet...

The Oracle 9i docs has some information that you mind find useful

Quote:

AIX versions 4 and higher support asynchronous I/O (AIO) for database files created both on file system partitions and on raw devices. AIO on raw devices is implemented fully into the AIX kernel, and does not require server processes to service the AIO requests. When using AIO on file systems, the kernel server processes (kproc) control each request from the time a request is taken off the queue until it completes. The kernel server processes are also used with I/O with virtual shared disks (VSDs) and HSDs with FastPath disabled. By default, FastPath is enabled. The number of kproc servers determines the number of AIO requests that can be executed in the system concurrently, so it is important to tune the number of kproc processes when using filesystems to store Oracle9i datafiles.

http://www.mamiyami.com/doc/oracle9i...7/appa_aix.htm

From this it sounds like the AIX kernel is managing the Async I/O, so although you bypass the LVM you still get this from the kernel.

iainr,

We actually did see the same doc. The part we do not follow is that the entire point of fastpath is to bypass the kernel. and send the requests directly to the shark.. (from our understanding).. Oracle is a bit different from sybase as sybase manages all connections internal.

I just can;t believe that IBM cannot answer this simple quesion.

There are lots of "simple" questions IBM can't easily answer - I could write an essay on it! Basically IBMs first support line are often not that technical - they look at IBM's worldwide database of problems (RETAIN) to find a solution. If it isn't on RETAIN, they can look at the same manuals we can. Only if that doesn't come up with a solution do they pass it onto the real techies behind the scenes.

I don't think that fastpath bypasses the kernel, only the LVM. Bypassing the kernel would suggest that Fastpath/sybase somehow maintains its own fibre adapter and ESS device drivers which I am certain is not the case.

iainr,

We agree.... FastPath disk access has to use the kernal aio.... But we are waiting for IBM to confirm. We think out larger issue may be the "Maximum number of requests" setting... Sybase is set for 65535 and aix is set for 4096. So in theory AIX is throttling the db. We want to raise to match the db 65535. We are just going to double check with IBM that this setting applies to both kernal and kproc aio. Again, Thanks for you help.

-Carl

Sybase is "write order dependent" ... file writes have to be flushed to disk in order, or you can corrupt your database (if the system crashes).

This is the reason for AIO; when Sybase gets confirmation a write is complete, the data is physically on disk.

In early versions of AIX, the asynchronous nature of AIO was implemented with aio daemons, who issue blocking read/write calls on behalf of the caller, and use a cross memory modification of the aiocb to indicate I/O is complete.

All of this was implemented prior to modern advances in unix:
- Sybase implemented their own user level threading code,
before there was a pthreads. Multiple concurrent threads are only possible if no kernel blocking calls are issued. pthreads now make this kind of heroic programming obsolete, especially with the introduction of hardware multithreading.

- Direct asynchronous IO calls were added to the filesystem API's, which now allow much more flexible filesystem space management and still achieve the goals of write-ordering that Sybase was looking for.

What about fast-path?

IBM added "fastpath" interfaces to AIO for raw devices. The kernel can perform the non-blocking read/write calls without any cross process synchronization overhead of AIO daemons. IBM considered too difficult to provide the same interface to filesystems, due to the complexity of space management.

By disabling fastpath in the AIO device driver, it means you revert to the old style AIO daemons, even when they are unnecessary for raw devices.

Summary: Use raw devices with Sybase and enable fastpath.