LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 12-22-2009, 02:35 PM   #1
SirTristan
Member
 
Registered: Feb 2006
Distribution: GNU/Linux
Posts: 54

Rep: Reputation: 15
Editing extremely large files, too large for memory?


I'd like to edit the beginning and end of some very large files. These files are .sql dump files, many gigs in size (some 5GB-10GB), larger than even the RAM of the machine.

Currently I'm trying to use pico (via SSH), but that doesn't really work - either it loads the file after many minutes, with slow editing; or it never seems to load the file at all. It also takes up all my machine's resources.

How would I go about editing just the start and end of these files, without trying to load the whole file? Is there an editing command that would allow me to begin editing at the end?
 
Old 12-22-2009, 02:58 PM   #2
markush
Senior Member
 
Registered: Apr 2007
Location: Germany
Distribution: Slackware
Posts: 3,979

Rep: Reputation: 850Reputation: 850Reputation: 850Reputation: 850Reputation: 850Reputation: 850Reputation: 850
Hello SirTristan,

what do you mean with "editing"? If you only want to look at the beginning or the end of a file you may use the "head" and the "tail" commands. Look at the manpages.

Otherwise if you really want to make changes to the files, I'd suggest to use sed or a perlscript.

Markus
 
Old 12-22-2009, 03:06 PM   #3
Quakeboy02
Senior Member
 
Registered: Nov 2006
Distribution: Debian Squeeze 2.6.32.9 SMP AMD64
Posts: 3,281

Rep: Reputation: 126Reputation: 126
Try using the split command to split your file into smaller chunks. Edit the chunk with your data, then use the cat command to put it all back together.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Editing Large Text Files. truculentknight Linux - Newbie 7 11-08-2009 11:24 PM
ext3 performance -- very large number of files, large filesystems, etc. td3201 Linux - Server 5 11-25-2008 09:28 AM
Knoppix on CD, Upon starting I get large menu with large text & blank submenus samdaria Linux - Newbie 1 06-06-2008 09:59 PM
LXer: This week at LWN: Large pages, large blocks, and large problems LXer Syndicated Linux News 0 09-27-2007 11:40 AM
memory full when reading large files with fgetc() alshishtawy Programming 9 05-30-2004 07:45 AM


All times are GMT -5. The time now is 11:30 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration