Search and delete a block using sed command in bash in ubuntu

AmirJamez · 09-21-2012, 08:55 PM

The thing I would like to perform is to
1- Find
2- delete the whole definition block of (__c64):

Code:

unsigned long long __c64(unsigned int llvm_cbe_hi, unsigned int llvm_cbe_lo) {
unsigned long long llvm_cbe_retval;    /* Address-exposed local */
unsigned long long llvm_cbe_retval1;



  *(&llvm_cbe_retval) = (((((unsigned long long )(unsigned int )llvm_cbe_hi)) << 32ul) |  (((unsigned          long long )(unsigned int )llvm_cbe_lo)));
  llvm_cbe_retval1 = *(&llvm_cbe_retval);
  return llvm_cbe_retval1;
}

in all but one .c files in a project. Since in the compilation phase of .o to the executable, I got the error "Multiple definition of __c64", I was thinking to delete all of those but one in order to solve the problem.

when I use :

Code:

":%s/unsigned long long __c64\(.\+\n\)\+\(\n\)\+\(.\+\n\)\+}//"

inside the vi editor for each file, everything is fine and it works but when I wanna embed it inside other bash script with sed -i :

Code:

sed -i 's/unsigned long long __c64\(.\+\n\)\+\(\n\)\+\(.\+\n\)\+}//' "$f"

It didn't change anything though :|

Any ideas?

Regards,

Amir

unSpawn · 09-21-2012, 09:35 PM

With "-e" it works on a stream but with "-i" sed expects a file to work on?

AmirJamez · 09-22-2012, 04:58 PM

Thanks unSpawn, but

Quote:

With "-e" it works on a stream but with "-i" sed expects a file to work on?

, I edited the question, I already pass the file name with "" at the end, it didn't work, I tried also with -e, the same results.

It is so strange that inside the vi editor it is perfectly working but not in bash with sed. got frustrated:|

Best,

Amir

ntubski · 09-22-2012, 05:38 PM

I'm not familiar with vi, but in sed \( starts a regexp group, if you want to match a ( just use a single (. Furthermore, sed works a line at a time, you have to jump through some hoops to match multiple lines. An easier transform to do with sed is make __c64 a static function, that way it won't conflict with definitions in other files:

Code:

sed -i 's/unsigned long long __c64(/static &/' *.c

AmirJamez · 09-22-2012, 06:22 PM

Thanks ntubski for the reply.

but it didn't work as I performed it. i got no errors, but when i search the file the block was still there, how can I ,lets say, debug the process ?

Btw, are you sure your command :

Quote:

sed -i 's/unsigned long long __c64(/static &/' *.c

was complete? I mean, I couldn't figure it out how sed could have understood the whole 6 lines and where to put an end to the match in order to delete it.

Thanks,

AMir

David the H. · 09-23-2012, 10:28 AM

First of all, ntubski is right about the backslashes. sed and grep have two forms of regex, basic, which is limited to a small subset of regex syntax, and extended, which allows most of the common features. In the gnu versions, when in basic mode, a backslash enables the extended meaning of some characters, and vice-versa when extended mode is enabled. See the grep man page for a detailed account.

But second, are you saying that you are trying to span multiple lines at once? Remember, in spite of some basic syntax similarity, sed and vi are really quite different.

sed operates on single lines only by default. While it's easy to use address ranges to apply your expressions to multiple lines, you can't directly span multiple lines in a single expression without the use of special flow commands; ones that append extra lines into the pattern buffer first.

The grymoire tutorial has good coverage of it.
http://www.grymoire.com/Unix/Sed.html

I'm not sure about straight vi, but I know that vim is able to be used in scripted mode. So you could possibly just use your current regex directly, with another command or two for saving, etc. I couldn't tell you the exact syntax off-hand, either, but I believe the -s option flag provides one way.

Edit: use -e -s and you can send EX mode commands into it from stdin. Or use -c to specify individual commands directly as options. And both options are apparently specified by posix, so vi should have them too.

I think something like this should work for you then:

Code:

printf '%s\n' '%s/unsigned long long __c64\(.\+\n\)\+\(\n\)\+\(.\+\n\)\+}//' 'w' | vim -e -s file.txt

Just stick it in a loop to process multiple files.

ntubski · 09-23-2012, 11:16 AM

Quote:

Originally Posted by AmirJamez

Btw, are you sure your command :

Quote:

sed -i 's/unsigned long long __c64(/static &/' *.c

was complete? I mean, I couldn't figure it out how sed could have understood the whole 6 lines and where to put an end to the match in order to delete it.

I was proposing a different transformation that doesn't have to match the whole 6 lines. Specifically:

Code:

// this line
unsigned long long __c64(unsigned int llvm_cbe_hi, unsigned int llvm_cbe_lo) {
// will be transformed into
static unsigned long long __c64(unsigned int llvm_cbe_hi, unsigned int llvm_cbe_lo) {

AmirJamez · 09-23-2012, 08:59 PM

Quote:

Originally Posted by David the H.

First of all, ntubski is right about the backslashes. sed and grep have two forms of regex, basic, which is limited to a small subset of regex syntax, and extended, which allows most of the common features. In the gnu versions, when in basic mode, a backslash enables the extended meaning of some characters, and vice-versa when extended mode is enabled. See the grep man page for a detailed account.

But second, are you saying that you are trying to span multiple lines at once? Remember, in spite of some basic syntax similarity, sed and vi are really quite different.

sed operates on single lines only by default. While it's easy to use address ranges to apply your expressions to multiple lines, you can't directly span multiple lines in a single expression without the use of special flow commands; ones that append extra lines into the pattern buffer first.

The grymoire tutorial has good coverage of it.
http://www.grymoire.com/Unix/Sed.html

I'm not sure about straight vi, but I know that vim is able to be used in scripted mode. So you could possibly just use your current regex directly, with another command or two for saving, etc. I couldn't tell you the exact syntax off-hand, either, but I believe the -s option flag provides one way.

Edit: use -e -s and you can send EX mode commands into it from stdin. Or use -c to specify individual commands directly as options. And both options are apparently specified by posix, so vi should have them too.

I think something like this should work for you then:

Code:

printf '%s\n' '%s/unsigned long long __c64\(.\+\n\)\+\(\n\)\+\(.\+\n\)\+}//' 'w' | vim -e -s file.txt

Just stick it in a loop to process multiple files.

Thanks for the reply.

Just after the submitting, i figured out that there are lots of these types and those were dynamic in-terms of lines. So I tried to use

Code:

sed '/^unsigned long long __c64/,/}/d'

and duplicate it for each and every data type i.e. __c64, __s64, etc (each have their own definition).

my solution didn't go quit well with __divrem64 as it was :

Code:

void __divrem64(unsigned long long llvm_cbe_a, unsigned long long llvm_cbe_b, unsigned long long *llvm_cbe_div, unsigned long long *llvm_cbe_rem) {
  unsigned long long llvm_cbe_a_addr;    /* Address-exposed local */
  unsigned long long llvm_cbe_b_addr;    /* Address-exposed local */
  unsigned long long *llvm_cbe_rem_addr;    /* Address-exposed local */
  unsigned long long llvm_cbe_tmp__44;
  unsigned long long llvm_cbe_tmp__45;
  unsigned long long *llvm_cbe_tmp__46;

  *(&llvm_cbe_a_addr) = llvm_cbe_a;
  *(&llvm_cbe_b_addr) = llvm_cbe_b;
  *(&llvm_cbe_rem_addr) = llvm_cbe_rem;
  *llvm_cbe_div = (((unsigned long long )(((unsigned long long )llvm_cbe_a) / ((unsigned long long )llvm_cbe_b))));
  llvm_cbe_tmp__44 = *(&llvm_cbe_a_addr);
  llvm_cbe_tmp__45 = *(&llvm_cbe_b_addr);
  llvm_cbe_tmp__46 = *(&llvm_cbe_rem_addr);
  *llvm_cbe_tmp__46 = (((unsigned long long )(((unsigned long long )llvm_cbe_tmp__44) % ((unsigned long long )llvm_cbe_tmp__45))));
  return;
}

Could you please explain why i can't use the same structure for this one?

best,

Amir

AmirJamez · 09-23-2012, 09:03 PM

Quote:

Originally Posted by ntubski

I was proposing a different transformation that doesn't have to match the whole 6 lines. Specifically:

Code:

// this line
unsigned long long __c64(unsigned int llvm_cbe_hi, unsigned int llvm_cbe_lo) {
// will be transformed into
static unsigned long long __c64(unsigned int llvm_cbe_hi, unsigned int llvm_cbe_lo) {

Thanks for your reply. I just got it and it did work as I tried to link the project. as you could see in the previous reply of mine, I tried another solution which worked on every data definition except __divrem64, I tried to make it static and it worked.

here is my question, first why it didn't work? Second, Can I use static for all those twenty something data types in order to avoid the deletion replace of sed??

Third, why I could get the binary at the end (with one static transformation and lots of find/replace with sed), but i couldn't have run it afterward??

best,

Amir

ntubski · 09-24-2012, 07:30 AM

Quote:

I just got it and it did work as I tried to link the project. as you could see in the previous reply of mine, I tried another solution which worked on every data definition except __divrem64, I tried to make it static and it worked.

here is my question, first why it didn't work

I'm confused about what you're saying works and doesn't work; please use examples. For instance, your sed works for __divrem64:

Code:

% cat divrem.c
void bar() {
    /* don't delete me */
}

void __divrem64(unsigned long long llvm_cbe_a, unsigned long long llvm_cbe_b, unsigned long long *llvm_cbe_div, unsigned long long *llvm_cbe_rem) {
  unsigned long long llvm_cbe_a_addr;    /* Address-exposed local */
  unsigned long long llvm_cbe_b_addr;    /* Address-exposed local */
  unsigned long long *llvm_cbe_rem_addr;    /* Address-exposed local */
  unsigned long long llvm_cbe_tmp__44;
  unsigned long long llvm_cbe_tmp__45;
  unsigned long long *llvm_cbe_tmp__46;

  *(&llvm_cbe_a_addr) = llvm_cbe_a;
  *(&llvm_cbe_b_addr) = llvm_cbe_b;
  *(&llvm_cbe_rem_addr) = llvm_cbe_rem;
  *llvm_cbe_div = (((unsigned long long )(((unsigned long long )llvm_cbe_a) / ((unsigned long long )llvm_cbe_b))));
  llvm_cbe_tmp__44 = *(&llvm_cbe_a_addr);
  llvm_cbe_tmp__45 = *(&llvm_cbe_b_addr);
  llvm_cbe_tmp__46 = *(&llvm_cbe_rem_addr);
  *llvm_cbe_tmp__46 = (((unsigned long long )(((unsigned long long )llvm_cbe_tmp__44) % ((unsigned long long )llvm_cbe_tmp__45))));
  return;
}

void foo() {
    /* don't delete me! */
}
% sed '/^void __divrem64(/,/}/d' divrem.c 
void bar() {
    /* don't delete me */
}


void foo() {
    /* don't delete me! */
}

Quote:

Second, Can I use static for all those twenty something data types in order to avoid the deletion replace of sed??

static functions of the same name in different compilation units (files) won't conflict, so yes.

Quote:

Third, why I could get the binary at the end (with one static transformation and lots of find/replace with sed), but i couldn't have run it afterward??

What do you mean "couldn't have run"? Paste any errors.

AmirJamez · 09-24-2012, 08:15 AM

Quote:

Originally Posted by ntubski

I'm confused about what you're saying works and doesn't work; please use examples. For instance, your sed works for __divrem64:

Code:

% cat divrem.c
void bar() {
    /* don't delete me */
}

void __divrem64(unsigned long long llvm_cbe_a, unsigned long long llvm_cbe_b, unsigned long long *llvm_cbe_div, unsigned long long *llvm_cbe_rem) {
  unsigned long long llvm_cbe_a_addr;    /* Address-exposed local */
  unsigned long long llvm_cbe_b_addr;    /* Address-exposed local */
  unsigned long long *llvm_cbe_rem_addr;    /* Address-exposed local */
  unsigned long long llvm_cbe_tmp__44;
  unsigned long long llvm_cbe_tmp__45;
  unsigned long long *llvm_cbe_tmp__46;

  *(&llvm_cbe_a_addr) = llvm_cbe_a;
  *(&llvm_cbe_b_addr) = llvm_cbe_b;
  *(&llvm_cbe_rem_addr) = llvm_cbe_rem;
  *llvm_cbe_div = (((unsigned long long )(((unsigned long long )llvm_cbe_a) / ((unsigned long long )llvm_cbe_b))));
  llvm_cbe_tmp__44 = *(&llvm_cbe_a_addr);
  llvm_cbe_tmp__45 = *(&llvm_cbe_b_addr);
  llvm_cbe_tmp__46 = *(&llvm_cbe_rem_addr);
  *llvm_cbe_tmp__46 = (((unsigned long long )(((unsigned long long )llvm_cbe_tmp__44) % ((unsigned long long )llvm_cbe_tmp__45))));
  return;
}

void foo() {
    /* don't delete me! */
}
% sed '/^void __divrem64(/,/}/d' divrem.c 
void bar() {
    /* don't delete me */
}


void foo() {
    /* don't delete me! */
}

static functions of the same name in different compilation units (files) won't conflict, so yes.

What do you mean "couldn't have run"? Paste any errors.

My question was lets say, why I couldn't use this find and delete for __divrem64 ?? , like this :

Code:

sed '/^void __divrem64(unsigned long long llvm_cbe_a, unsigned long long llvm_cbe_b, unsigned long long *llvm_cbe_div, unsigned long long *llvm_cbe_rem) {/,/}/d'

since i was deleting from the first line of the __divrem64 to "}" at the end of the definition (around 8-12 lines). i could use that structure for all other data type definitions(Like __c64,__xon64, etc). Does it have something to do with the void at the beginning ?

Second, by performing :

Code:

set -x 
/home/user/VEX/vex-3.43/bin/cc -o bin/run_inst *.o -lm

I got the executable lie this :

Code:

+ ./run_inst
user@VM-LARGE:~/gw_opt_order/workspace/h264_tests/most.node.0.0/1/bin$

and I could run the project with :

Code:

 ./run_inst

again with no errors, but still something is wrong when i check the subsequent log files i could get from VEX. I mean since i got no errors executing the ./run_inst , I can be 100% sure that the work is done right till here ? so the problem is from now. isn't it ?

best,

Amir

ntubski · 09-25-2012, 09:37 AM

Quote:

Originally Posted by AmirJamez

My question was lets say, why I couldn't use this find and delete for __divrem64 ?? , like this :

Code:

sed '/^void __divrem64(unsigned long long llvm_cbe_a, unsigned long long llvm_cbe_b, unsigned long long *llvm_cbe_div, unsigned long long *llvm_cbe_rem) {/,/}/d'

That won't work because you didn't escape the "*" characters.

Quote:

no errors, but still something is wrong when i check the subsequent log files i could get from VEX. I mean since i got no errors executing the ./run_inst , I can be 100% sure that the work is done right till here ? so the problem is from now. isn't it ?

So you're saying the program gave the wrong answers? Then you can 100% sure there is a bug in it. The bug could have been in there from before, or introduced by the transformations you did to it. An example: I believe you mentioned in an earlier thread that the VEX has 32 bit long longs, in which case the __divrem64 function will actually implement __divrem32. That may or may not cause a bug in the program.