LinuxQuestions.org - Need help with 'dd'and 'mmap()'

- Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)

- - Need help with 'dd'and 'mmap()' (https://www.linuxquestions.org/questions/linux-newbie-8/need-help-with-ddand-mmap-553423/)

Agresvig

05-12-2007 03:28 PM

Need help with 'dd'and 'mmap()'

Hi

I am working on a project that involves creating a file recovery program in C++.
I need some help with the Unix commands to get started.

First off, here is what I want to do, in short:
1. Create an image file of a FAT32 thumbdrive.
2. In the C++ program, map chunks of this image file into memory, and process them chunk by chunk.

Now, for step 1, I am trying to use 'dd' to create an image of the thumbdrive. I have tried:

Code:

dd if=/media/RALLY of=tmp/tmpimg.img

where 'RALLY' is the name of the thumbdrive. This does not work, with the error "dd: reading '/media/RALLY': is a directory"

Whats the correct way to create the image?

When I have that sorted out, I would love to get some pointers on how to split the image into chunks to feed mmap().

Thanks for all help.

Daws	05-12-2007 03:33 PM

Quote:

dd if=/media/RALLY of=tmp/tmpimg.img

You want the device node. For a thumbdrive it's probably /dev/sda or /dev/sdb

Quote:

When I have that sorted out, I would love to get some pointers on how to split the image into chunks to feed mmap().

Hmm. Afraid I don't know what that is ... oh well

osor	05-12-2007 04:26 PM

Quote:

Originally Posted by Daws

You want the device node. For a thumbdrive it's probably /dev/sda or /dev/sdb

Yes. To figure out for sure, type “mount | grep /media/RALLY” or whatever. That gives you the block device mounted on /media/RALLY.

As for the second part, I’m not quite sure what you want. You can always use the “split” utility to split a file into multiple chunks. You can also mmap only part of a file at a time (vary the len and offset parameters).

For more info, see
man 1 split
man 2 mmap

Agresvig

05-12-2007 04:34 PM

Quote:

Originally Posted by Daws

You want the device node. For a thumbdrive it's probably /dev/sda or /dev/sdb

Perfect, /dev/sdb did the trick. Thanks a lot.

Now, I would love to get some suggestions on how to chop that image file into pieces.

About mmap():

Code:

void *mmap(void *start, size_t length, int prot, int flags,int fd, off_t offset);

If anyone has ideas on how to scan the image file using mmap(), I would love to hear them.

Thanks!

Agresvig

05-12-2007 04:45 PM

Quote:

Originally Posted by osor

sorry didnt see your psot until I posted mine.
But thanks, that useful, will check it out.

osor	05-12-2007 05:53 PM

In case you didn’t understand, sample implementation (in C) would go something like this:

Code:

#define filename "tmp/tmpimg.img" /* filename */

#define chunksize 1048576 /* size of a chunk in bytes = 1 megabyte */



int fd, numchunks, i;

struct stat buf;

void *map = 0;



if ((fd = open(filename, O_RDONLY )) == -1) {

        perror("Couldn not open " filename);

        exit(EXIT_FAILURE);

}

if (fstat(fd, &buf) == -1) {

        perror("Could not stat " filename);

        exit(EXIT_FAILURE);

}



numchunks = buf.st_size / chunksize;

if (numchunks % chunksize) // if the size is not a multiple of chunksize,

        numchunks++;      // we need an extra chunk



for (i = 0; i < numchunks; i++) {

        if((map = mmap(NULL, chunksize, PROT_READ, MAP_PRIVATE, fd, i * chunksize)) == MAP_FAILED) {

                perror("map failed");

                exit(EXIT_FAILURE);

        }



        process_chunk(map, chunksize);



        if(munmap(map, chunksize) == -1) {

                perror("unmap failed");

                exit(EXIT_FAILURE);

        }

}

The process_chunk function shall process a chunk at a time.

P.S., there might be an off by one error somewhere

Agresvig

05-14-2007 02:45 PM

Quote:

Originally Posted by osor

In case you didn’t understand, sample implementation (in C) would go something like this:

The process_chunk function shall process a chunk at a time.

P.S., there might be an off by one error somewhere

Thanks, that is of great help. Just one thing, could you explain your logic here a little more?:

Code:

numchunks = buf.st_size / chunksize;

if (numchunks % chunksize) // if the size is not a multiple of chunksize,

        numchunks++;      // we need an extra chunk

Thanks!

osor	05-14-2007 08:19 PM

Quote:

Originally Posted by Agresvig

Just one thing, could you explain your logic here a little more?

Code:

numchunks = buf.st_size / chunksize;

if (numchunks % chunksize) // if the size is not a multiple of chunksize,

        numchunks++;      // we need an extra chunk

First, I use fstat() to get file’s size. Then, I assign to the variable numchunks the integer quotient of the file’s size to the chunksize. Recall that integer division (at least in C — at least with ‘/’) always rounds “down” (toward zero). But that means if the file’s size is not a multiple of the chunksize, we will have a “fractional” chunk left over. Therefore, I test to see if there is a non-zero remainder when numchunks is divided by chunksize. If so, we know we have a fractional chunk left over, so we increment numchunks to account for this. The mmap() function will conveniently pad the excess memory with zeros (if this didn’t happen, we might instead request the exact remainder of the division of numchunks by chunksize and use it in one final call to mmap() after the loop).

All times are GMT -5. The time now is 12:15 AM.