As of kernel 2.6.26
there is some sort of Memtest86 type of memory tester that you can build right into the kernel (see section 1.9 at the link). If you read about it, it claims to be "simpler and less capable" than the 'real' Memtest86 that you use standalone, but maybe better than nothing at all?
As for running Memtest86 (the standalone binary) from within
a running OS, it would not work, and if it *did* work, it would not be giving you the full story anyhow: based on what I know about how it works, it itself
is the only
thing loaded into memory when it is running, and it relocates itself from place to place so that it can test the memory it is occupying. Makes sense that trying to do a thorough memory test, *while* an OS is booted & running, is not ideal, and would either crash the OS or skip the occupied areas of memory -- neither or which is useful
Running the `file` command on my memtest86 binary, tells me that it is a "Linux x86 kernel" -- so it isn't like your average 'application' that you can just fire up.
Maybe enable the 'memtest' option in your kernel though, if possible (requires rebuilding the kernel probably, or upgrading it, so still perhaps not practical for you).