change makefile for a VEX benchmark to use in LLVM-to-VEX chain

AmirJamez · 07-15-2012, 11:09 PM

I see.

I tried to run gsm.c without the sed options, and I had the errors :

Code:

"opt/llvm_transformed.c", line 239: warning: undefined escape sequence in literal
"opt/llvm_transformed.c", line 239: warning: undefined escape sequence in literal
"opt/llvm_transformed.c", line 239: warning: undefined escape sequence in literal
"opt/llvm_transformed.c", line 239: warning: undefined escape sequence in literal
"opt/llvm_transformed.c", line 243: warning: illegal combination of pointer and integer, op ==
"opt/llvm_transformed.c", line 243: warning: illegal combination of pointer and integer, op ==
"opt/llvm_transformed.c", line 244: error: redeclaration of formal parameter, X
"opt/llvm_transformed.c", line 244: error: redeclaration of formal parameter, Y
"opt/llvm_transformed.c", line 244: warning: illegal combination of pointer and integer, op !=
"opt/llvm_transformed.c", line 244: warning: illegal combination of pointer and integer, op !=
"opt/llvm_transformed.c", line 245: error: redeclaration of formal parameter, X
"opt/llvm_transformed.c", line 245: error: redeclaration of formal parameter, Y
"opt/llvm_transformed.c", line 245: warning: illegal combination of pointer and integer, op ==
"opt/llvm_transformed.c", line 246: error: redeclaration of formal parameter, X
"opt/llvm_transformed.c", line 246: error: redeclaration of formal parameter, Y
"opt/llvm_transformed.c", line 246: warning: illegal combination of pointer and integer, op !=
"opt/llvm_transformed.c", line 247: error: redeclaration of formal parameter, X
"opt/llvm_transformed.c", line 247: error: redeclaration of formal parameter, Y
"opt/llvm_transformed.c", line 247: warning: illegal combination of pointer and integer, op <
"opt/llvm_transformed.c", line 248: error: redeclaration of formal parameter, X
"opt/llvm_transformed.c", line 248: error: redeclaration of formal parameter, Y
"opt/llvm_transformed.c", line 248: warning: illegal combination of pointer and integer, op >
"opt/llvm_transformed.c", line 249: error: redeclaration of formal parameter, X
"opt/llvm_transformed.c", line 249: error: redeclaration of formal parameter, Y
"opt/llvm_transformed.c", line 249: warning: illegal combination of pointer and integer, op <=
"opt/llvm_transformed.c", line 250: error: redeclaration of formal parameter, X
"opt/llvm_transformed.c", line 250: error: redeclaration of formal parameter, Y
"opt/llvm_transformed.c", line 250: warning: illegal combination of pointer and integer, op >=
"opt/llvm_transformed.c", line 251: error: redeclaration of formal parameter, X
"opt/llvm_transformed.c", line 251: error: redeclaration of formal parameter, Y
"opt/llvm_transformed.c", line 251: warning: illegal combination of pointer and integer, op ==
"opt/llvm_transformed.c", line 252: error: redeclaration of formal parameter, X
"opt/llvm_transformed.c", line 252: error: redeclaration of formal parameter, Y
"opt/llvm_transformed.c", line 252: warning: illegal combination of pointer and integer, op !=
"opt/llvm_transformed.c", line 253: error: redeclaration of formal parameter, X
"opt/llvm_transformed.c", line 253: error: redeclaration of formal parameter, Y
"opt/llvm_transformed.c", line 253: warning: illegal combination of pointer and integer, op <
"opt/llvm_transformed.c", line 254: error: redeclaration of formal parameter, X
"opt/llvm_transformed.c", line 254: error: redeclaration of formal parameter, Y
"opt/llvm_transformed.c", line 254: warning: illegal combination of pointer and integer, op >
"opt/llvm_transformed.c", line 255: error: redeclaration of formal parameter, X
"opt/llvm_transformed.c", line 255: error: redeclaration of formal parameter, Y
"opt/llvm_transformed.c", line 255: warning: illegal combination of pointer and integer, op <=
"opt/llvm_transformed.c", line 256: error: redeclaration of formal parameter, X
"opt/llvm_transformed.c", line 256: error: redeclaration of formal parameter, Y
"opt/llvm_transformed.c", line 256: warning: illegal combination of pointer and integer, op >=
"opt/llvm_transformed.c", line 360: error: illegal indirection
"opt/llvm_transformed.c", line 362: error: pointer required
"opt/llvm_transformed.c", line 362: error: illegal indirection
"opt/llvm_transformed.c", line 363: error: pointer required
"opt/llvm_transformed.c", line 363: compiler error: too many errors
 --(MGW)-->  WARNING Execution Failed

So it seems that I really need to change the long long to unsigned long.

By the way, I checked the C-opt-C.sh :

Code:

#!/bin/bash
#set -x

function print_usage() {
  echo "Usage: $0 [OPTIONS]"
  echo "  -h               Print this help."
  echo "  -c               Configuration file"
  echo "  -s               Source files path"
  echo "  -w               Work files path"
  echo "  -o               Output files path"
  exit 0
}

configfile="#"
srcpath="#"
workpath="#"
outpath="#"

while getopts "hc:s:w:o:" flag; do
  case $flag in
    h) print_usage;;
    c) configfile=$OPTARG;;
    s) srcpath=$OPTARG;;
    w) workpath=$OPTARG;;
    o) outpath=$OPTARG;;
    ?)  echo "unknown";;
  esac
done

if [ ! -f $configfile ]; then
  echo "Configuration file missing. Use -help to see help"
#!/bin/bash
#set -x

function print_usage() {
  echo "Usage: $0 [OPTIONS]"
  echo "  -h               Print this help."
  echo "  -c               Configuration file"
  echo "  -s               Source files path"
  echo "  -w               Work files path"
  echo "  -o               Output files path"
  exit 0
}

configfile="#"
srcpath="#"
workpath="#"
outpath="#"

while getopts "hc:s:w:o:" flag; do
  case $flag in
    h) print_usage;;
    c) configfile=$OPTARG;;
    s) srcpath=$OPTARG;;
    w) workpath=$OPTARG;;
    o) outpath=$OPTARG;;
    ?)  echo "unknown";;
  esac
done

if [ ! -f $configfile ]; then
  echo "Configuration file missing. Use -help to see help"
  exit 1
fi;

if [ ! -d  $srcpath ]; then
  echo "Source path missing. Use -help to see help"
  exit 1
fi;

if [ $workpath = "#" ]; then
  echo "Work path missing. Use -help to see help"
  exit 1
fi;

if [ $outpath = "#" ]; then
  echo "Output path missing. Use -help to see help"
  exit 1
fi;

sources=""
for f in $(ls $srcpath/*.c); do
  sources="$sources $(basename $f)"
done

mkdir -p $workpath
mkdir -p $outpath

echo "Optimization options: $(cat $configfile)"
for f in $sources; do
  echo "  Processing $f"
  infile=$srcpath/$f
  noopt=$workpath/${f%%.c}.llnoopt
  opt=$workpath/${f%%.c}.llopt
  cout=$outpath/$f
  /home/user/LLVM_2_8/llvm-gcc-4.2-2.8-i686-linux/bin/llvm-gcc -S -emit-llvm $infile -o $noopt
  /home/user/LLVM_2_8/build/bin/opt $(cat $configfile) -S -o $opt $noopt
  /home/user/LLVM_2_8/build/bin/llc -march=c $opt -o $cout
done
echo "Done"

There is no explicit declaration of any 64 bits output.

Best,

Amir

AmirJamez · 07-16-2012, 08:24 AM

I asked my professor some of these issues, and here is the replies :

Quote:

Q1- So why when we use the makefile of the h264 the executable is 32 bit as I checked, also with the gsm.c we could use the same changing but here no??

Prof_Answer : 32-bit executable size is not the same as the size of the long long words (the latter is compiler dependent).

Quote:

2- our LLVM transformation results is 16bit or 64bit ? (I could guess 64 cuz we used the sed -i for changing the ull-to-ll)

Prof_Answer : As far as I understand, LLVM source code assumes LL as 64, but on vex it is 32. That's why it is important to convert the literal to 32 bit two's-complement.

Quote:

3- I don't understand, changing the literal if it's derived from negative? ( I have to change some part of the source code itself or use some scripts to change before the compilation ? )

Prof_Answer : You should change it with a script after the code has been generated. It is pretty easy to spot the places to change.

So as far as I understood, i have to write some scripts to change them? Could you please help me me pinpoint the problem ? (Do I have to open up a new thread regarding this issue ? "i.e. changing 64 bit longling code to 32 bit unsigned line in ubuntu... ")

Best,

Amir

ntubski · 07-16-2012, 03:30 PM

Quote:

Originally Posted by AmirJamez

Prof_Answer : As far as I understand, LLVM source code assumes LL as 64, but on vex it is 32. That's why it is important to convert the literal to 32 bit two's-complement.

I feel like it would be better to tell LLVM that long longs are 32bit, but apparently the LLVM C backend is no longer supported, so nasty hacks it is...

Quote:

So as far as I understood, i have to write some scripts to change them? Could you please help me me pinpoint the problem ? (Do I have to open up a new thread regarding this issue ? "i.e. changing 64 bit longling code to 32 bit unsigned line in ubuntu... ")

What are you having trouble with specifically? Do you understand 2's complement?

AmirJamez · 07-16-2012, 04:38 PM

Theoretically yes, but don't know how to convert it, as far as the Prof said, it seems that I can change the site with some scipts after the transformation of LLVM and before the input of VEX...

Amir

AmirJamez · 07-16-2012, 04:39 PM

and by the way, we use LLVM 2.8 without any further update to the newer ones...

ntubski · 07-16-2012, 09:06 PM

Quote:

Originally Posted by AmirJamez

Theoretically yes, but don't know how to convert it

Well, you'll need something that can handle 64bit integers, which rules out sed (no numbers) and awk (uses floating point ~ 53 bit integers I think). I would go with Python, but you can use whatever you feel comfortable with. You could even use sed/awk/shell and use dc/bc for the number handling part.

Quote:

and by the way, we use LLVM 2.8 without any further update to the newer ones...

Quote:

Originally Posted by Owen Anderson

As far back as the LLVM 2.8, the release notes have included this warning:
"The C backend has numerous problems and is not being actively maintained. Depending on it for anything
serious is not advised."

AmirJamez · 07-17-2012, 09:14 AM

I checked the LLVM and VEX compiler for the max ullong, and the results were 32 for vex and 64 for llvm, so for sure I need to convert these two.

About the mention solutions, as far as my compiler and C programming level is not that good, i would like to ask for the help.

P.S: just to be clear, your mentioned loop in the first thread :

Quote:

for cfile in opt/*.c ; do
llvm_cfile=opt/llvm_transformed_$(basename "$cfile")
mv "$cfile" "$llvm_cfile"

sed -i 's/$[0-9]$ull/\1ul/g' "$llvm_cfile"

echo "C to Vex obj"
/home/user/VEX/vex-3.43/bin/cc -ms -O1 -c99inline -fmm=MachConf.mm -I/usr/include/ -c "$llvm_cfile"
done
echo "Vex objs to Vex executable"
/home/user/VEX/vex-3.43/bin/cc -o bin/run_inst llvm_transformed_*.o -lm
bin/run_inst

seems to fail in the first iteration when it transformed the first .c file (usually llvm_transformed_annexb.c) which is i think the first file entering the loop in alphabetical order, i was thinking if on the first step we first transform all the .c files ( distinguished by i.e. numbers) and then input them in another loop ?? do you think it is better ? Since I thought maybe some inter-nested references and dependencies maybe get broken otherwise ? ( anyway, totally confused!! )

best,

Amir

ntubski · 07-18-2012, 11:58 AM

Quote:

Since I thought maybe some inter-nested references and dependencies maybe get broken otherwise ? ( anyway, totally confused!! )

No no, it's nothing that complicated.

The problem is the code has something like this:

Code:

(signed long long )(18446744073709551615ull)

The sed command changes the ull to ul but 18446744073709551615 doesn't fit in 32 bits, so it still won't compile. Now, since the number is being cast to signed we can tell that it really means -1 (because it's the 2's complement of -1). So we have to convert it to -1, or the 32 bit unsigned equivalent: 4294967295ul.

AmirJamez · 09-14-2012, 09:22 AM

Dear ntubski,

Its been a while

I was just rechecking the code which u mentioned sometimes ago (in the reply #2), which u abbreviated these :

Code:

sed -i 's/0ull/0ul/g' opt/llvm_transformed.c
sed -i 's/1ull/1ul/g' opt/llvm_transformed.c
sed -i 's/2ull/2ul/g' opt/llvm_transformed.c
sed -i 's/3ull/3ul/g' opt/llvm_transformed.c
sed -i 's/4ull/4ul/g' opt/llvm_transformed.c
sed -i 's/5ull/5ul/g' opt/llvm_transformed.c
sed -i 's/6ull/6ul/g' opt/llvm_transformed.c
sed -i 's/7ull/7ul/g' opt/llvm_transformed.c
sed -i 's/8ull/8ul/g' opt/llvm_transformed.c
sed -i 's/9ull/9ul/g' opt/llvm_transformed.c

to this :

Code:

sed -i 's/\([0-9]\)ull/\1ul/g' "$llvm_cfile"

I was wondering if the replace section could be 0-9 as well, cause in this line we are going to change all the patterns of 0-9 to just 1, isn't it ?

Best,

Amir

ntubski · 09-14-2012, 09:46 AM

Quote:

Originally Posted by AmirJamez

I was wondering if the replace section could be 0-9 as well, cause in this line we are going to change all the patterns of 0-9 to just 1, isn't it ?

No, \1 means the first parenthesized expression in the matching regex, in this case whatever [0-9] matched. So really, we are leaving the 0-9 pattern unchanged (by replacing it with itself).

AmirJamez · 09-14-2012, 11:13 AM

But I don't know why still I am having some problems in compiling like :

Code:

"llvm_transformed_annexb.c", line 1152: error: syntax error
"llvm_transformed_annexb.c", line 1152: error: ul undefined
"llvm_transformed_annexb.c", line 1152: error: syntax error
"llvm_transformed_annexb.c", line 1152: error: syntax error

where is stands for :

Code:

 *(&llvm_cbe_retval) = ((((((signed long long )llvm_cbe_x) > ((signed long long )-1ulul))) ? (llvm_cbe_x) : ((-(llvm_cbe_x)))));
  llvm_cbe_retval1 = *(&llvm_cbe_retval);
  return llvm_cbe_retval1;

Thanks for helping

Amir

ntubski · 09-14-2012, 04:17 PM

Code:

(signed long long )-1ulul

The doubled ul looks like the problem.

AmirJamez · 09-14-2012, 05:58 PM

I see. this was the outcome of changing the ull literals to ul "sed -i 's/$[0-9]$ull/\1ul/g' "$llvm_cfile"" as we discussed in the latter reply.

Since the second compiler was 32 bits, I had to change these. Therefore, what do you suggest?

Regards,

Amir

ntubski · 09-15-2012, 07:55 AM

Quote:

Originally Posted by AmirJamez

this was the outcome of changing the ull literals to ul "sed -i 's/$[0-9]$ull/\1ul/g' "$llvm_cfile""

In fact, it was not:

Code:

% echo '(signed long long )(18446744073709551615ull)' | sed 's/\([0-9]\)ull/\1ul/g'
(signed long long )(18446744073709551615ul)

As you can see that sed command puts only a single ul. You haven't posted the transformation that puts the -1, perhaps the doubled ulul was introduced then.

AmirJamez · 09-15-2012, 10:22 AM

Oh yes, you were right. I had to do like this(instead of -1ul) :

Code:

sed -i 's/18446744073709551615ull/-1/g' "$llvm_cfile"

Now the error of ulul left, the final error which I have to fix is that there are lots of HEX floating points which I have to convert to normal form just in bash. the C program that works can be :

Code:

#include <stdio.h>
int main (void) {
    double d = 0x0.3p10; //these type pf hex floating points
    printf ("%.f\n", d);
    return 0;
}

I tried sth like :

Code:

$ sed 's/0x[^p]\+p[+-]./esyscmd(printf "%f" &)/'

but it didn't work. could you please advise me on that ?

Regards,

Amir