LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 10-27-2016, 04:08 PM   #1
Rock26
LQ Newbie
 
Registered: Jul 2016
Posts: 8

Rep: Reputation: Disabled
Find some pattern from all files and remove certain characters from it.


Kindly help me solving following problem. Remove all "\n" from Test_Macro in all files. Please see below example:

Fil1.txt

Code:
    
Test_Macro(abc, def, "\n string1 string2 \n test string",
            "test string2 \n");
// Some code or text
     
Test_Macro(asdsadas, abc.str(), "test String1\n");
// Some code...
dir1/File2.txt

Code:
 
Test_Macro(abc, def, "\n string1 string2 \n test string",
           "test string2 \n",
            123456);
// Some code or text
    
Test_Macro(asdsadas, abc.str(), "test String1");
// Some code...
Expected Result:

File1.txt

Code:
 
Test_Macro(abc, def, " string1 string2 test string",
       "test string2 ");
// Some code or text
    
Test_Macro(asdsadas, abc.str(), "test String1");
// Some code...
dir1/File2.txt

Code:
 
Test_Macro(abc, def, " string1 string2  test string",
       "test string2 ",
        123456);
// Some code or text
     
Test_Macro(asdsadas, abc.str(), "test String1");
    // Some code...
Any help or suggestions are much appreciated. I am planning to write some script. Because I have many different types of files and many such macros. Thanks in advance!

Last edited by Rock26; 10-27-2016 at 04:14 PM. Reason: Don't know how to submit code in this forum.
 
Old 10-27-2016, 05:44 PM   #2
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,999

Rep: Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190
What effort have you made to this solution (apart from asking here of course)?

You mention writing a script ... what language will that be in?
 
Old 10-27-2016, 05:52 PM   #3
Rock26
LQ Newbie
 
Registered: Jul 2016
Posts: 8

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by grail View Post
What effort have you made to this solution (apart from asking here of course)?

You mention writing a script ... what language will that be in?
Hi Grail. I can write bash script if I find some ways to achieve this result. I tried some commands. But I can't achieve final result. "sed 's/\\n//g'", "sed ':a;N;$!ba;s/[^;]\n[ ]*/ /g;' file1.txt | grep Test_Macro" and etc. etc. But this remove all backslashes and n characters. I am also newbie. So any suggestions are welcome.
 
Old 10-27-2016, 06:06 PM   #4
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,103

Rep: Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117
NM - didn't read properly

Last edited by syg00; 10-27-2016 at 06:14 PM.
 
Old 10-27-2016, 06:20 PM   #5
Rock26
LQ Newbie
 
Registered: Jul 2016
Posts: 8

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by syg00 View Post
Presuming normally formatted text, the sed 's/\\n//g' should work. Show us full commands and output of those snippets you posted above.
Hi syg00,

This command will remove all "\n". I just want to remove from Test_Macro. I have approx. 5-8 directories with many files in each. Simple example for one file:

File1.c
Code:
 
Test_Macro(abc, def, "\n string1 string2 \n test string",
            "test string2 \n");
printf("Welcome to C world..\n");
// Some code or text
     
Test_Macro(asdsadas, abc.str(), "test String1\n");
Another_Macro("Done with this file...\n\n");
// Some code...
Expected result:

Code:
 
Test_Macro(abc, def, " string1 string2  test string",
            "test string2 ");
printf("Welcome to C world..\n");
// Some code or text
     
Test_Macro(asdsadas, abc.str(), "test String1");
Another_Macro("Done with this file...\n\n");
// Some code...
Actual output:

Code:
>sed 's/\\n//g' File1.c
Test_Macro(abc, def, " string1 string2  test string",
            "test string2 ");
printf("Welcome to C world..");             // removed "\n"
// Some code or text
     
Test_Macro(asdsadas, abc.str(), "test String1");
Another_Macro("Done with this file...");                  // removed "\n"
// Some code...

Last edited by Rock26; 10-27-2016 at 06:26 PM. Reason: Formatting..
 
Old 10-27-2016, 07:02 PM   #6
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,103

Rep: Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117
Yeah, sorry about that, I realised after I'd posted ...
You were pretty close, but you neeed to limit the accumulation of lines based on "Test" and the semi-colon. Try this instead
Code:
sed '/^Test/ {:a;/;/!{N;$!ba};s/\\n/ /g;n};' test.file
 
2 members found this post helpful.
Old 10-27-2016, 07:31 PM   #7
Rock26
LQ Newbie
 
Registered: Jul 2016
Posts: 8

Original Poster
Rep: Reputation: Disabled
Smile

Quote:
Originally Posted by syg00 View Post
Yeah, sorry about that, I realised after I'd posted ...
You were pretty close, but you neeed to limit the accumulation of lines based on "Test" and the semi-colon. Try this instead
Code:
sed '/^Test/ {:a;/;/!{N;$!ba};s/\\n/ /g;n};' test.file
Thanks a lot @syg00. This worked perfectly fine. Please suggest me some good "sed" and "regex" tutorial. Appreciate all your help and time.
 
Old 10-27-2016, 08:59 PM   #8
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,103

Rep: Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117
I'm probably the wrong person to ask. I tend to see something and think "hmmm, that's an interesting way to do it", and go and and find out more. Quite often posts here on LQ start the process.
For sed that means "info sed" - the gnu.org sed homepage has a downloadable html version if you prefer. Also have a look at the sed1liners at sf.
For regex, that is a whole 'nuther problem - different implementations (posix, basic, extended, PCRE, ...). Again I don't use tutorials, I just bumble along till I figure it out.

Hopefully others will have more sensible suggestions.
 
Old 10-27-2016, 11:04 PM   #9
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Rocky 9.2
Posts: 18,348

Rep: Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749
Re sed docs; you could start here http://www.grymoire.com/Unix/Sed.html
Collection of sed links etc http://sed.sourceforge.net/

As for regex generally, as syg00 pointed out, that's an entire down-the-rabbit-hole experience. I highly recommend http://regex.info/book.html
 
Old 10-28-2016, 02:09 AM   #10
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,999

Rep: Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190
Code:
awk '/Test/{gsub(/\\n/,"")}1' RS=";" ORS=";" file
 
Old 10-28-2016, 02:26 AM   #11
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,103

Rep: Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117
The reason I avoided that sort of thing grail was the possible case of plain text (not semi-colon terminated) lines containing "\n".
Edge case maybe, but we never seem to get the full specs .... :shrug:
 
Old 11-01-2016, 04:13 PM   #12
Rock26
LQ Newbie
 
Registered: Jul 2016
Posts: 8

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by Rock26 View Post
Thanks a lot @syg00. This worked perfectly fine. Please suggest me some good "sed" and "regex" tutorial. Appreciate all your help and time.
Hey,

This worked. But I just realized that it could not remove last "\n". For example,

Code:
cat foo.txt
TEST_MACRO(" test string\n",
                             "test string2\n",
                                     "test string\n);
//Some text or code..

>> sed '/^Test/ {:a;/;/!{N;$!ba};s/\\n//g;n};' foo.txt

Output:

TEST_MACRO(" test string",
                             "test string2",
                                     "test string\n");
 
Old 11-01-2016, 05:18 PM   #13
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,103

Rep: Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117
Should do - and does here. GNU sed 4.2.2
 
Old 11-01-2016, 05:45 PM   #14
Rock26
LQ Newbie
 
Registered: Jul 2016
Posts: 8

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by syg00 View Post
Should do - and does here. GNU sed 4.2.2
Ohh. I am using 4.2.1. I tried this for simple 5 lines. It only worked for first 3 lines. I think it doesn't accept ";" delimiter.


Code:
rock@test]$ sed --version
GNU sed version 4.2.1
Copyright (C) 2009 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE,
to the extent permitted by law.
See below output:

Code:
[rock@test]$ cat foo.txt 
TEST_MACRO(" test string\n",
               "test string2\n",
                   "test string3\n";
                      "test string4\n";
                        "test string5\n")
[rock@test]$ sed '/^TEST/ {:a;/;/!{N;$!ba};s/\\n/ /g;n};' foo.txt
TEST_MACRO(" test string ",
               "test string2 ",
                   "test string3 ";
                      "test string4\n";
                        "test string5\n");

Last edited by Rock26; 11-01-2016 at 05:49 PM.
 
Old 11-01-2016, 05:51 PM   #15
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,103

Rep: Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117
Quote:
Originally Posted by Rock26 View Post
I think it doesn't accept ";" delimiter.
That is a change in specification - you will need to re-work the script.
The small difference in version shouldn't matter.
 
  


Reply

Tags
awk, grep, perl, scripting, sed


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] How to remove certain pattern files except another certain pattern files from a list? Mike_Brown Linux - Newbie 4 04-23-2016 12:30 AM
[SOLVED] use sed to find string pattern and delete subsequent characters jigg_fly Programming 16 10-10-2013 08:34 AM
Need to remove the first three characters in the name of a ton of files... ooagentbender Linux - Newbie 17 10-04-2013 02:02 PM
find files with pattern, then copy those files to another directory ncsuapex Programming 4 08-13-2010 03:38 PM
unwanted characters returned with bash find - how to remove? babag Programming 5 06-10-2007 09:36 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 06:03 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration