Corrupted files on SD card - ext3 will solve?

Hi,

I'm working on a data logger project. My system is running over an ARM9 (mini2440), with a custom kernel (version 2.6.32.2) and the file system is yaffs.
The main program was written in c++ and uses a database (sqlite 3.7.8) to store all collected data.

I need to save everything on the database wich is placed on a sd card. The sdcard was formatted with FAT.
Sometimes the files inside it gets corrupted. So I need to run the "chkdsk" windows command to try to recover those files (sometimes I can't, and this is really boring).
Sometimes the equipment is suddenly powered down (I can't avoid it), and I think that's a possible (or the main) reason of the problem.

I decided to change from FAT to EXT3, because EXT3 have the journaling system (I read that journal keeps the information about file operations). I think that journal could help me, but I'm not sure. I'm very noob on linux.

So, I have 2 questions.

1 - Using the EXT3 filesystem, will really help to prevent those corruptions?

2 - Is there another way to prevent that?


Tutorials, books and any links are welcome


Anyway, I was trying to test the EXT3 file system, but i'm stuck on a problem: EXT3-fs: mounted filesystem with writeback data mode.

I used "gparted" to format my sdcard, and then I created an ext3 partition. On my PC (ubuntu 8.04) I can read and write files inside it. But when I try to mount it on ARM, i got the above error.

I'm using the following command line:

# mount -o async -o noatime -o rw -t ext3 /dev/mmcblk0p5 /sdcard
kjournald starting.  Commit interval 5 seconds
EXT3-fs: mounted filesystem with writeback data mode.

The device is 'correctly' mounted. I can cd to /sdcard and ls the files normally, but I can't write anything. It's mounted with read only mode.

# touch test.txt
touch: test.txt: Read-only file system

df command:

# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/root               256640     28132    228508  11% /
tmpfs                    29496         0     29496   0% /dev/shm
/dev/mmcblk0p5         1904536     35664   1772888   2% /sdcard

Can someone help me with this too?

Thanks in advance

That message is informational, not an error. Although I don't like writeback.
Did you try the mkfs on ARM ?.

Hi syg00,

I have mkfs on Ubuntu, but i don't have the compiled version for ARM. I'd tried to use the mkfs on PC, just to format the SD card, but I have no success. So I use gparted to format the sd.

I discovered that EXT3 have 3 operation modes (http://linux.die.net/man/8/mount):

journal - All data is committed into the journal prior to being written into the main filesystem. ordered (default) - This is the default mode. All data is forced directly out to the main file system prior to its metadata being committed to the journal. writeback - Data ordering is not preserved - data may be written into the main filesystem after its metadata has been committed to the journal. This is rumoured to be the highest-throughput option. It guarantees internal filesystem integrity, however it can allow old data to appear in files after a crash and journal recovery.

Journal mode seems to be the safest, but is slower.

About the read only mode, I don't figure out what is wrong, but I changed my command line to:

mount -o async,noatime,rw -t ext3 -o data=journal,rw /dev/sdcard /sdcard
mount -o remount,rw /sdcard

I think this will prevent any data loss when the supply is cut. There any other suggestions?

Moved: This thread is more suitable in the Linux Embedded forum and has been moved accordingly to help your thread/question get the exposure it deserves.

If your board is suddenly powered off as data is written to the disk, then corruption is unavoidable, unless your hardware could be changed to include a small battery to turn off the device if the mains are down.

I've had this problem in the past. You can try to minimize the problem by changing your software to decrease the number of writes if possible and flushing the cache (fsync) after each writes to make sure data is written as soon as possible and reduce the odds of corruption.

You could also run e2fsck or dosfsck (depending on FS used) to repair the disk at boot time.

With the FAT file system, the SD card often mounted as Read-Only, with ext2 there were also problems, but e2fsck seemed better than dosfsck to fix the issues.

Hello There,
SD cards are the foremost medium to save the data and this faces the corruption very often which lead the data to loss. SD card corruption have many reason behind it, this problem is faced by me many times and result of that i have lost files many times.
So after the loss i searched on internet and found the software which was very effective to do corrupted SDHC card recovery. The software can be used to recover the data from all sort of memory cards.

The download link is
[mod removed spam link]
Regards
Wilsong

Hi,

Welcome to LQ!

Originally Posted by wilsong Hello There, SD cards are the foremost medium to save the data and this faces the corruption very often which lead the data to loss. SD card corruption have many reason behind it, this problem is faced by me many times and result of that i have lost files many times. So after the loss i searched on internet and found the software which was very effective to do corrupted SDHC card recovery. The software can be used to recover the data from all sort of memory cards. The download link is [mod removed spam link] Regards Wilsong

You are correct for the power interruption causing issues for a 'SDHC' memory corruption. Sadly, the link you provided is for a MS & OSX recovery method using those propriety OS. Plus the linked software is crippleware.

OP(Original Post) stated: 'So I need to run the "chkdsk" windows command to try to recover those files (sometimes I can't, and this is really boring)'. so from that quote I see the OP needs a recovery method using Gnu/Linux not MS or OSX.

Please re-read the LQ Rules