[SOLVED] Find some pattern from all files and remove certain characters from it.

Rock26 · 10-27-2016, 05:08 PM

Kindly help me solving following problem. Remove all "\n" from Test_Macro in all files. Please see below example:

Fil1.txt

Code:

    
Test_Macro(abc, def, "\n string1 string2 \n test string",
            "test string2 \n");
// Some code or text
     
Test_Macro(asdsadas, abc.str(), "test String1\n");
// Some code...

dir1/File2.txt

Code:

 
Test_Macro(abc, def, "\n string1 string2 \n test string",
           "test string2 \n",
            123456);
// Some code or text
    
Test_Macro(asdsadas, abc.str(), "test String1");
// Some code...

Expected Result:

File1.txt

Code:

 
Test_Macro(abc, def, " string1 string2 test string",
       "test string2 ");
// Some code or text
    
Test_Macro(asdsadas, abc.str(), "test String1");
// Some code...

dir1/File2.txt

Code:

 
Test_Macro(abc, def, " string1 string2  test string",
       "test string2 ",
        123456);
// Some code or text
     
Test_Macro(asdsadas, abc.str(), "test String1");
    // Some code...

Any help or suggestions are much appreciated. I am planning to write some script. Because I have many different types of files and many such macros. Thanks in advance!

grail · 10-27-2016, 06:44 PM

What effort have you made to this solution (apart from asking here of course)?

You mention writing a script ... what language will that be in?

Rock26 · 10-27-2016, 06:52 PM

Quote:

Originally Posted by grail

What effort have you made to this solution (apart from asking here of course)?

You mention writing a script ... what language will that be in?

Hi Grail. I can write bash script if I find some ways to achieve this result. I tried some commands. But I can't achieve final result. "sed 's/\\n//g'", "sed ':a;N;$!ba;s/[^;]\n[ ]*/ /g;' file1.txt | grep Test_Macro" and etc. etc. But this remove all backslashes and n characters. I am also newbie. So any suggestions are welcome.

syg00 · 10-27-2016, 07:06 PM

NM - didn't read properly

Rock26 · 10-27-2016, 07:20 PM

Quote:

Originally Posted by syg00

Presuming normally formatted text, the sed 's/\\n//g' should work. Show us full commands and output of those snippets you posted above.

Hi syg00,

This command will remove all "\n". I just want to remove from Test_Macro. I have approx. 5-8 directories with many files in each. Simple example for one file:

File1.c

Code:

 
Test_Macro(abc, def, "\n string1 string2 \n test string",
            "test string2 \n");
printf("Welcome to C world..\n");
// Some code or text
     
Test_Macro(asdsadas, abc.str(), "test String1\n");
Another_Macro("Done with this file...\n\n");
// Some code...

Expected result:

Code:

 
Test_Macro(abc, def, " string1 string2  test string",
            "test string2 ");
printf("Welcome to C world..\n");
// Some code or text
     
Test_Macro(asdsadas, abc.str(), "test String1");
Another_Macro("Done with this file...\n\n");
// Some code...

Actual output:

Code:

>sed 's/\\n//g' File1.c
Test_Macro(abc, def, " string1 string2  test string",
            "test string2 ");
printf("Welcome to C world..");             // removed "\n"
// Some code or text
     
Test_Macro(asdsadas, abc.str(), "test String1");
Another_Macro("Done with this file...");                  // removed "\n"
// Some code...

syg00 · 10-27-2016, 08:02 PM

Yeah, sorry about that, I realised after I'd posted ...

You were pretty close, but you neeed to limit the accumulation of lines based on "Test" and the semi-colon. Try this instead

Code:

sed '/^Test/ {:a;/;/!{N;$!ba};s/\\n/ /g;n};' test.file

Rock26 · 10-27-2016, 08:31 PM

Quote:

Originally Posted by syg00

Yeah, sorry about that, I realised after I'd posted ...

You were pretty close, but you neeed to limit the accumulation of lines based on "Test" and the semi-colon. Try this instead

Code:

sed '/^Test/ {:a;/;/!{N;$!ba};s/\\n/ /g;n};' test.file

Thanks a lot @syg00. This worked perfectly fine. Please suggest me some good "sed" and "regex" tutorial. Appreciate all your help and time.

syg00 · 10-27-2016, 09:59 PM

I'm probably the wrong person to ask. I tend to see something and think "hmmm, that's an interesting way to do it", and go and and find out more. Quite often posts here on LQ start the process.
For sed that means "info sed" - the gnu.org sed homepage has a downloadable html version if you prefer. Also have a look at the sed1liners at sf.
For regex, that is a whole 'nuther problem - different implementations (posix, basic, extended, PCRE, ...). Again I don't use tutorials, I just bumble along till I figure it out.

Hopefully others will have more sensible suggestions.

chrism01 · 10-28-2016, 12:04 AM

Re sed docs; you could start here http://www.grymoire.com/Unix/Sed.html
Collection of sed links etc http://sed.sourceforge.net/

As for regex generally, as syg00 pointed out, that's an entire down-the-rabbit-hole experience. I highly recommend http://regex.info/book.html

grail · 10-28-2016, 03:09 AM

Code:

awk '/Test/{gsub(/\\n/,"")}1' RS=";" ORS=";" file

syg00 · 10-28-2016, 03:26 AM

The reason I avoided that sort of thing grail was the possible case of plain text (not semi-colon terminated) lines containing "\n".
Edge case maybe, but we never seem to get the full specs .... :shrug:

Rock26 · 11-01-2016, 05:13 PM

Quote:

Originally Posted by Rock26

Thanks a lot @syg00. This worked perfectly fine. Please suggest me some good "sed" and "regex" tutorial. Appreciate all your help and time.

Hey,

This worked. But I just realized that it could not remove last "\n". For example,

Code:

cat foo.txt
TEST_MACRO(" test string\n",
                             "test string2\n",
                                     "test string\n);
//Some text or code..

>> sed '/^Test/ {:a;/;/!{N;$!ba};s/\\n//g;n};' foo.txt

Output:

TEST_MACRO(" test string",
                             "test string2",
                                     "test string\n");

syg00 · 11-01-2016, 06:18 PM

Should do - and does here. GNU sed 4.2.2

Rock26 · 11-01-2016, 06:45 PM

Quote:

Originally Posted by syg00

Should do - and does here. GNU sed 4.2.2

Ohh. I am using 4.2.1. I tried this for simple 5 lines. It only worked for first 3 lines. I think it doesn't accept ";" delimiter.

Code:

rock@test]$ sed --version
GNU sed version 4.2.1
Copyright (C) 2009 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE,
to the extent permitted by law.

See below output:

Code:

[rock@test]$ cat foo.txt 
TEST_MACRO(" test string\n",
               "test string2\n",
                   "test string3\n";
                      "test string4\n";
                        "test string5\n")
[rock@test]$ sed '/^TEST/ {:a;/;/!{N;$!ba};s/\\n/ /g;n};' foo.txt
TEST_MACRO(" test string ",
               "test string2 ",
                   "test string3 ";
                      "test string4\n";
                        "test string5\n");

syg00 · 11-01-2016, 06:51 PM

Quote:

Originally Posted by Rock26

I think it doesn't accept ";" delimiter.

That is a change in specification - you will need to re-work the script.
The small difference in version shouldn't matter.