LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 02-26-2005, 07:22 PM   #1
rabby
Member
 
Registered: Mar 2004
Distribution: gentoo, slackware
Posts: 41

Rep: Reputation: 15
SED has problems with Ram


hello!
i am really worry about sed: Couldn't re-allocate memory
it's task is to replace sth in a file of a size about 250 mb .
perhaps one of the 2 ram blocks inside the computer's hardware can be the reason of this. i remember myself that there have already been bluescreens some years ago when windows was running on this computer...
but during using linux there has no longer been a problem with the rams till today :-(
or is there another problem ?
is there a parameter to limit SED's power ? it would take longer but perhaps there was no error ..?
or can i use an alternative for sed to replace sth in a file of this size?
please help
thanks
 
Old 02-26-2005, 10:02 PM   #2
nilleso
Member
 
Registered: Nov 2004
Location: ON, CANADA
Distribution: ubuntu, RHAS, and other unmentionables
Posts: 372

Rep: Reputation: 31
I believe sed sets an arbitrary memory limitation. Use perl instead.
Code:
cat myfile | perl p e 's/old/new/g;' > newfile
or
Code:
perl i p e 's/old/new/g;' myfile
cheers
 
Old 02-27-2005, 06:14 AM   #3
rabby
Member
 
Registered: Mar 2004
Distribution: gentoo, slackware
Posts: 41

Original Poster
Rep: Reputation: 15
hello!
now i tried your tipp. unsuccessfully.
the reason may be that there is no \n inside the 250mb big file !
this was job of sed...
so what can i try now?

thanks
 
Old 02-27-2005, 07:19 AM   #4
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 677Reputation: 677Reputation: 677Reputation: 677Reputation: 677Reputation: 677
I can think of a few options now.

A) Provide more information so that a perl based program can work.
B) Perhaps changing the sed program so that it uses less memory
C) Breaking the file into manageable pieces.

You haven't supplied much particular information, about either the file you are trying to change, or the sed program you are using.
 
Old 02-27-2005, 08:39 AM   #5
rabby
Member
 
Registered: Mar 2004
Distribution: gentoo, slackware
Posts: 41

Original Poster
Rep: Reputation: 15
hello world!
now i wrote my own script in php:

Code:
<?
    $dateiname="./f1";
    $dateiname2="./f1_2";
    unlink($dateiname2);
    if(file_exists($dateiname))
        $xmlDoc=($dateiname);
        $xmlDoc2=($dateiname2);
    $xmlConn=fopen($xmlDoc,"r");
    $xmlConn2=fopen($xmlDoc2,"a");
    do
    {
        $data=fgets($xmlConn,1024);
        $xmlFeed=$data;
        $xmlFeed=str_replace('</tag> ',' </tag>'."\n", $xmlFeed);
        fputs($xmlConn2,$xmlFeed);
    }while($xmlFeed!='');
?>
have a nice day.
 
Old 02-28-2005, 07:46 PM   #6
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 677Reputation: 677Reputation: 677Reputation: 677Reputation: 677Reputation: 677
I'm not sure if sed preloads a file before it is worked on. Another possibility is if the file is compressed to eliminate whitespace and returns. Sed will match the longest matching pattern on a line, so if there are no returns, trying to find a match on a 256 MB line would overtax sed.

Anyway, I'm glad that you found a solution.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
ram problems userN Linux - Hardware 1 04-07-2005 01:40 AM
sed problems icisyd Linux - Software 2 11-19-2004 07:17 PM
Insert character into a line with sed? & variables in sed? jago25_98 Programming 5 03-11-2004 07:12 AM
weird sed problems C++freak Slackware 5 08-04-2003 05:00 PM
RAM problems philfighter General 1 08-01-2001 03:02 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 11:55 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration