LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 07-06-2011, 03:53 AM   #1
venu_s
LQ Newbie
 
Registered: Mar 2011
Posts: 5

Rep: Reputation: 0
Post memcpy fails to copy data, but byte by byte assignment work


Hi,
I got across a peculiar problem with memcpy.

History:
writing a code which do flash/Read functionality for SPI Parts.

Implementation:
1.memory map the Flash chip registers to userspace. - fine
2.Issue block read command
3.Just print the data I got, using the memory mapped address - fine I got the correct data.
4.Copy the data to a local buffer from memory mapped address using memcpy - FAIL

UPDATE:
on SUSE 11.3 -32 bit the above code worked perfectly, but on SUSE 11.4 - 64bit , Ubantu, Fedora it failed.
I am using gcc compiler obviously.
I am using i386/x86_64 OS. Processor: Intel core 2 duo, SNB

after step 4 , when I dump the data from local buffer every thing is FF.
But step 4, if i implement byte by byte copy using assignment operator, it worked.


is there any known bug with memcpy?
am I missing anything?

-Venu

Last edited by venu_s; 07-07-2011 at 01:39 AM.
 
Old 07-06-2011, 06:09 AM   #2
winning
Member
 
Registered: Apr 2011
Posts: 70

Rep: Reputation: 13
I'm guessing it all comes down to the actual ASM instructions that are generated by your particular compiler on your particular hardware. memcpy() might be optimized via some instructions which don't work in this particular scenario.
 
Old 07-06-2011, 08:20 AM   #3
Nominal Animal
Senior Member
 
Registered: Dec 2010
Location: Finland
Distribution: Xubuntu, CentOS, LFS
Posts: 1,723
Blog Entries: 3

Rep: Reputation: 943Reputation: 943Reputation: 943Reputation: 943Reputation: 943Reputation: 943Reputation: 943Reputation: 943
If local buffer overlapped with the memory map, memcpy() (as opposed to memmove()) could fill the target area with some of the first or last bytes of source. That does not seem to be the case here.

It could be your flash chip needs register addresses to be byte wide. memcpy() usually uses the widest possible moves to do the copy. It may also lock the bus or use string instructions in assembly, if your architecture has those; memory-mapped registers don't usually like that at all. A byte-by-byte copy does none of that, of course. Compilers like gcc prefer to use their own memcpy() and memmove() over those supplied by standard libraries, so compiler options may also have an effect on this.

Can you tell us which architecture you're using?
 
Old 07-06-2011, 01:54 PM   #4
ta0kira
Senior Member
 
Registered: Sep 2004
Distribution: FreeBSD 9.1, Kubuntu 12.10
Posts: 3,078

Rep: Reputation: Disabled
I hate to ask this sort of thing, but you're not taking the address of the variable storing the mmap pointer or using the size of that pointer in memcpy, are you?

I'd say it's more likely that such a bug would be in the implementation of mmap support at the driver level, not with memcpy itself; access should be transparent unless memcpy bypasses the driver, which isn't likely. I also wouldn't expect an overlap between the mmap and the buffer unless you specified the mapping address manually.
Kevin Barry

Last edited by ta0kira; 07-06-2011 at 01:59 PM.
 
Old 07-06-2011, 11:27 PM   #5
Nominal Animal
Senior Member
 
Registered: Dec 2010
Location: Finland
Distribution: Xubuntu, CentOS, LFS
Posts: 1,723
Blog Entries: 3

Rep: Reputation: 943Reputation: 943Reputation: 943Reputation: 943Reputation: 943Reputation: 943Reputation: 943Reputation: 943
In case I was unclear, I do believe memcpy()'s behaviour is correct. It is just that on some architectures, memory mapping I/O registers may pose additional constraints on how the mapped memory should be accessed. It may also depend on the device whose I/O registers are being mapped. Compilers often quite happily reorder accesses to the memory mapped regions; you may need to sprinkle volatile's and compiler memory barriers into the code to impose a specific order reliably. In some rare cases the CPU may do it too itself (if it supports out-of-order or speculative execution and cannot detect that the virtual memory address is actually an I/O register and not RAM).

Bus locking and string operations just refer to assembly instruction or instructions that will try to make sure the processor uses the entire bus bandwidth for the entire operation. String operations are just tight assembly loops. Locking will not allow any peripheral to interrupt that operation on the bus. These are available on some architectures, but not all. On an 80186 CPU, the construct would be e.g. lock rep movsw where the CPU should hold the bus for the entire operation. (It would also work on 8086 and 8088 CPUs, but they would lose the bus lock if the operation was interrupted).

All it means is that you should check what additional constraints your architecture has for memory-mapped I/O registers, and the assembly source code (gcc option -S) your compiler produces (where you use memcpy()) to see if the compiler-optimized memcpy() breaks those constraints. I thought the all-ones result points to this direction.

Oh, and always remember to compile your code with full warnings. Pedantic even, if you don't mind the occasional nitpick. (For GCC, I use -Wall -pedantic -std=c99.) It is surprising how easy it is to have an error in for example the parameter order. I think some memset(ptr, size, 0) calls have been found even in the Linux kernel. Perhaps you have used memcpy(sourceptr, destptr, size) by accident?
 
Old 07-07-2011, 02:17 AM   #6
venu_s
LQ Newbie
 
Registered: Mar 2011
Posts: 5

Original Poster
Rep: Reputation: 0
[QUOTE=ta0kira;4407024]I hate to ask this sort of thing, but you're not taking the address of the variable storing the mmap pointer or using the size of that pointer in memcpy, are you?
No, on SUSE11.3-32bit it worked. Most probably compiler might have done optimization on SUS11.4 - 64 bit or other distro.
Is there anyway to ask gcc not to optimize the memcpy in my case?
 
Old 07-07-2011, 03:59 AM   #7
Nominal Animal
Senior Member
 
Registered: Dec 2010
Location: Finland
Distribution: Xubuntu, CentOS, LFS
Posts: 1,723
Blog Entries: 3

Rep: Reputation: 943Reputation: 943Reputation: 943Reputation: 943Reputation: 943Reputation: 943Reputation: 943Reputation: 943
Do you have a reason why you don't use your own function instead of memcpy()?
Code:
static inline void *bytecopy(void *const dest, void const *const src, size_t bytes)
{
        while (bytes-->(size_t)0)
                ((unsigned char *)dest)[bytes] = ((unsigned char const *)src)[bytes];

        return dest;
}
If you supply -fno-builtin-memcpy to gcc, it will use the C library memcpy(), but that too will usually do wider-than-byte transfers.

Last edited by Nominal Animal; 07-07-2011 at 04:01 AM.
 
Old 07-08-2011, 04:29 AM   #8
venu_s
LQ Newbie
 
Registered: Mar 2011
Posts: 5

Original Poster
Rep: Reputation: 0
I have done the same,
I wrote my own byte by byte copy function and used it. But I was curious to know, why memcpy didn't work? are there any compilation flags which missed?
 
  


Reply

Tags
flashmemory


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Linux command to copy every other byte from file kdawgud Linux - General 16 06-27-2014 09:39 AM
Need to copy an entire disk byte-for-byte Pawprint Linux - Software 6 06-16-2011 12:01 PM
memcpy_toio transfers data in 4 byte chunks, but I need to transfer data in one lump. jbreaka4lyfe Linux - Embedded & Single-board computer 2 06-02-2008 12:25 PM
byte to byte remote comparison louisJ Linux - Newbie 3 09-21-2007 06:28 PM
backup byte-for-byte axion0917 Linux - Software 2 12-11-2003 06:01 PM


All times are GMT -5. The time now is 09:03 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration