Combining a project with multiple .c file into one .c

AmirJamez · 05-27-2012, 03:15 PM

Hello Everyone,

I am using VEX open-source compiler and some of its ready to use benchmarks like "gsm" and "h264" in ubuntu to run and test the effects of options in the performance results. The point is I really need just one .c file containing all the i.e. H264 project to integrate with my so-far scripts and the other working programs.

As I checked those benchmarks, they have their own makefile and compiler routine but I want them to be places inside my own scripts to run them several times in a looped chain.

P.S: for the reference, this is the dl link of VEX :
http://www.hpl.hp.com/downloads/vex/vex-3.43.i586.tgz

the benchmarks can be found on the folder ~/share/apps

Best,

Amir

tronayne · 05-28-2012, 04:58 PM

This could be done with a combination of find and cat; you would find all the *.c files in a given directory (~/share/apps/gsm or ~/share/apps/h264) and concatenate them into individual files named; e.g., gsm.c and another, h264.c. I use /tmp/gsm and /tmp/h264 as directory targets for copying (after the copy is done, simply move those directories wherever you want them to live).

So, to do that

Code:

mkdir /tmp/gsm
cat >> /tmp/gsm/gsm.c $(find ~/share/gsm -name '*.c')
mkdir /tmp/h264
cat >> /tmp/h264/h264.c $(find ~/share/gsm -name '*.c')

While you're at it you'll probably need any header files. You want to copy them separately:

Code:

cat >> /tmp/gsm/gsm.h $(find ~/share/gsm -name '*.h')
cat >> /tmp/h264/h264.h $(find ~/share/gsm -name '*.h')

That will get everything concatenated into separate header and source code files for you; it will not, however, deal with the Makefiles (and you don't want to simply copy them). You'll need to create a simple Makefile in those directories containing arguments handed to the compiler -- they'll be at the top of the existing Makefile (usually) and may look similar to this (only an example, yous will probably be quite complex):

Code:

#       use whichever one of these is for your system (don't add any additoinal flags, not needed)
#CFLAGS =       -xarch=native -xO3 -DANSI -I$(BASDIR)/include   # Solaris 32-bit
#CFLAGS =       -xarch=native64 -xO3 -DANSI -I$(BASDIR)/include # Solaris 64-bit
#CFLAGS =       -O2 -DANSI -I$(BASDIR)/include                  # Linux 32-bit
CFLAGS  =       -O2 -FPIC -m64 -DANSI -I$(BASDIR)/include       # Linux 64-bit

There may be a number of similar flags and compiler directives defined; you'll need to figure them out and include them in individual Makefiles.

This is not a trivial exercise, expect to have problems and be ready to deal with them.

Hope this helps some.

AmirJamez · 05-28-2012, 06:52 PM

Thanks a lot for the reply.

So If I got it right, at the end I'll have i.e. one h262.c and one h262.h ? As I mentioned in the question, there is no need to have a makefile for compiling, cuz I will use the .c file in my own automated scripts.

Is it a way to include the header files in the .c file as well ? all in one ?

Thanks,

Amir

tronayne · 05-29-2012, 07:25 AM

Close -- you'll end up with one (large) h264.c and h264.h (if there are any headers) in a directory h265, and one gsm.c and one hsm.h (ditto) in another directory gsm (if I understood the question correctly that is).

Yes, you can include the headers in one file at the begging of your .c, but you'd need to go through the entire .c file and remove all the #include directives for local header files -- the difference is

Code:

#include <stdio.h>     means it's in /include, /usr/include or similar
#incldue "file.h"      means it is the local directory tree

Lot's of screwing around to little effect (including a header file is trivial during the compilation process. Note, though, that you'll probably need to edit out all of the #include directives in any event from your giant .c; it's quite likely that header files will get included multiple times and you don't want that (it'll be messy at least).

What you can do is use, say, sed and uniq to create a little file of all the header directives, something like this

Code:

<after you've created the giant .c file>
sed -n '/#include/p' | uniq > file_name

That will give you a list of the include directives that you can insert (with a text editor) at the begging of the .c file -- after you've done this

Code:

sed '/#include/d' gsm.c > gsm_new.c

That will delete all the include directives, saving the result in gsm_new.c. Then just edit gsm_new.c and read the unique header list in at the top of it.

There are, of course, other ways to do that (regular expressions is one) but the above is quick and easy.

Again, this is not going to be trivial -- when you first try to compile this thing it's probably going to spit out a whole lot of errors and you're most likely going to be doing a whole lot of editing and moving things around before you'll get a clean compile. Make sure you keep back up copies before you edit things.

Hope this helps some.

AmirJamez · 05-29-2012, 03:51 PM

Thanks a lot buddy. I am on it. Hope the standard structure of the project doesn't change by doing this since I wanna publish some official results out of optimization of compilation with the whole automated system.

Will let you know.

regards,

Amir

AmirJamez · 07-08-2012, 09:59 PM

Dear tronayne,

Why in the :
<after you've created the giant .c file>
sed -n '/#include/p' | uniq > file_name

phase nothing wouldn't happen ? I mean just the curser standing in the next line without having anything on the output file_name. i have made the giant .c file, but this step forward i can't go through :|

Thanks,

Amir

tronayne · 07-09-2012, 06:32 AM

That's 'cause I missed putting the input file name in there -- duh!

Code:

<after you've created the giant .c file>
sed -n '/#include/p' gsm.c | uniq > file_name

That will give you a list of unique file names of the header files.

You could make "file_name" something like "gsm.h" then, after you do the next step, simply

Code:

#include "gsm.h"

in your "new" .c file (the one where the #include lines are deleted.

Sorry about the fumble-finger.

pan64 · 07-09-2012, 06:49 AM

you can use -E with gcc to preprocess only the sources. The result is one single file which can be simply compiled with gcc -o exe <preprocessed.c> -L ... -l .... I think this preprocessing stage can also be used with VEX sources because they looks compatible on this level.

AmirJamez · 07-09-2012, 10:28 PM

Thanks tronayne for the revision.

But again it didn't work, also in the h264.c (giant file), there were other includes in "# include" style (I don't know why in this style it has been written??) which I added afterward, but again no satisfactions. I can compile the whole project with 15-20 .c files with 15-20 .h files (which i placed them in one folder together) but neither with one .c and one .h nor with one giant .c and one .h (lists).

Dear pan64, Also I checked the "opt" C2C scripts where LLVM starts the first transformations, -E didn't help ( or maybe I used it wrong), the LLVM scripts are :

Code:

 /home/user/LLVM_2_8/llvm-gcc-4.2-2.8-i686-linux/bin/llvm-gcc -S -emit-llvm $infile -o $noopt
  /home/user/LLVM_2_8/build/bin/opt $(cat $configfile) -S -o $opt $noopt
  /home/user/LLVM_2_8/build/bin/llc -march=c $opt -o $cout

It is getting really annoying, the point is if I want to stick to the first solution ( many .c/many .h), I think I have to change a lot the remaining parts of the automated scripts. As you may see the "#mv opt/ldecod.c opt/llvm_transformed.c" which i added bellow. Here I pasted them :

Code:

#########################################################

echo "Compiling\n"
mkdir -p log bin opt opt/tmp
echo "C to C"

/home/user/gw_opt_order/tools/C2C/C-opt-C.sh -c opt.cfg -s $TARGET_FILE_DIR  -w opt/tmp -o opt > log/swat_opt.log 2>&1

mv opt/*.c opt/llvm_transformed.c
#mv opt/ldecod.c opt/llvm_transformed.c

#sed -i 's/signed long long/unsigned long/g' opt/llvm_transformed.c
sed -i 's/0ull/0ul/g' opt/llvm_transformed.c
sed -i 's/1ull/1ul/g' opt/llvm_transformed.c
sed -i 's/2ull/2ul/g' opt/llvm_transformed.c
sed -i 's/3ull/3ul/g' opt/llvm_transformed.c
sed -i 's/4ull/4ul/g' opt/llvm_transformed.c
sed -i 's/5ull/5ul/g' opt/llvm_transformed.c
sed -i 's/6ull/6ul/g' opt/llvm_transformed.c
sed -i 's/7ull/7ul/g' opt/llvm_transformed.c
sed -i 's/8ull/8ul/g' opt/llvm_transformed.c
sed -i 's/9ull/9ul/g' opt/llvm_transformed.c

echo "C to Vex"
#/home/user/VEX/vex-3.43/bin/cc -ms -O1 -c99inline -fmm=MachConf.mm -I/usr/include/ -c opt/llvm_transformed.c > log/vex.log 2>&1
/home/user/VEX/vex-3.43/bin/cc -ms -O1 -c99inline -fmm=MachConf.mm -I/usr/include/ -c opt/llvm_transformed.c
echo "Vex to obj"
/home/user/VEX/vex-3.43/bin/cc -o bin/run_inst llvm_transformed.o -lm
bin/run_inst

Thanks again for your time

Amir