LinuxQuestions.org
Go Job Hunting at the LQ Job Marketplace
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 08-24-2008, 03:01 PM   #1
tomva
Member
 
Registered: May 2006
Distribution: Fedora
Posts: 45

Rep: Reputation: 15
Basic C++ compiler/preprocessor question: how to include file as data?


Hello all-

I've been programming C++ for years but this one has stumped me. I'm guessing there is a simple answer but I haven't found it yet.

I'd like to do something like this in my .cpp file:

Code:
static const char * s_myLargeConstantString = <contents of local file>;
That is, I'd like to declare a static string constant, and have its value be the contents of a local file. Similar in spirit to a #include directive, but including the file as data.

This is obviously handy when you need large data such as copyright notices etc linked into the executable.

I understand there are good reasons NOT to do this, which is why I haven't needed this until now. But this is for some bootstrap code where an application needs to install a few local files. So now I find I actually need this functionality.

Is there a way to do this? Some clever preprocessor macro etc?

I'm hoping I don't have to hack up some pre-compilation scripts to create bogus source files to accomplish this.

-Thomas
 
Old 08-24-2008, 03:14 PM   #2
PatrickNew
Senior Member
 
Registered: Jan 2006
Location: Charleston, SC, USA
Distribution: Debian, Gentoo, Ubuntu, RHEL
Posts: 1,148
Blog Entries: 1

Rep: Reputation: 48
This is inelegant, but if you know that the file contains no double quotes, you could do this:
Code:
static const char* s_myLargeConstantString = "\
#include "filename.txt"
";
Which will sorta work, but append a newline to the end that wasn't there to start with.

Last edited by PatrickNew; 08-24-2008 at 03:14 PM. Reason: Haha, forgot semicolon
 
Old 08-24-2008, 03:59 PM   #3
ErV
Senior Member
 
Registered: Mar 2007
Location: Russia
Distribution: Slackware 12.2
Posts: 1,202
Blog Entries: 3

Rep: Reputation: 62
Quote:
Originally Posted by tomva View Post
That is, I'd like to declare a static string constant, and have its value be the contents of a local file. Similar in spirit to a #include directive, but including the file as data.

This is obviously handy when you need large data such as copyright notices etc linked into the executable.

I understand there are good reasons NOT to do this, which is why I haven't needed this until now. But this is for some bootstrap code where an application needs to install a few local files. So now I find I actually need this functionality.

Is there a way to do this? Some clever preprocessor macro etc?
The only reason you might avoid including file as text is that HUGE constant character strings aren't supported on all compilers. For example microsoft vc compiler has maximum string size as 65535. And don't forget about escape sequences.

Another solution for this is to write script which will read file and print is as C++ array or struct. This will be compatible with various compilers.

I've been using this python script to convert binary files into C++ array:
bin2c.py:
Code:
#!/usr/bin/python
import sys
import os
import os.path
import re
import array
 
USAGE = """bin2c - Embed binary file in a C header file
Usage: png2c [file ..] Convert all the file <file> to <file.h>"""
 
if len(sys.argv) < 2:
        print USAGE
        sys.exit(1)
 
r = re.compile("^([a-zA-Z._][a-zA-Z._0-9]*)[.]([a-zA-Z])+$")
 
for path in sys.argv[1:]:
        filename = os.path.basename(path)
        m = r.match(filename)
        # Allow only filenames that make sense
        # as C variable names
        if not(m):
                print "Skipped file (unsuitable filename): " + filename
                continue
 
        # Read PNG file as character array
        bytes = array.array('B', open(path, "rb").read())
        count = len(bytes)
 
        # Create the C header
#        text = "/* %s - %d bytes */\n" 
#		"static const unsigned char %s_png[] = {\n" % (filename, count, m.group(1))
	text = """struct {
long size;
	unsigned char data[]
} %s={\n
	%d,\n""" % (m.group(1), count)
 
        # Iterate the characters, we want
        # lines like:
        #   0x01, 0x02, .... (8 values per line maximum)
        i = 0
        count = len(bytes)
        for byte in bytes:
                # Every new line starts with two whitespaces
                if (i % 8) == 0:
                        text += "  "
                # Then the hex data (up to 8 values per line)
                text += "0x%02x" % (byte)
                # Separate all but the last values 
                if (i + 1) < count:
                        text += ", "
                if (i % 8) == 7:
                        text += '\n'
                i += 1
 
        # Now conclude the C source
        text += "};\n"
 
        open(path + ".h", 'w').write(text)
run this script like:
Code:
bin2c.py 1.bin
and it'll create 1.bin.h with structure containing file contents.
 
Old 08-24-2008, 07:16 PM   #4
ta0kira
Senior Member
 
Registered: Sep 2004
Distribution: FreeBSD 9.1, Kubuntu 12.10
Posts: 3,078

Rep: Reputation: Disabled
If you have no aversion to making the file a run-time dependency of the program, you can do it with mmap.
ta0kira
 
Old 08-24-2008, 07:54 PM   #5
ta0kira
Senior Member
 
Registered: Sep 2004
Distribution: FreeBSD 9.1, Kubuntu 12.10
Posts: 3,078

Rep: Reputation: Disabled
Quote:
Originally Posted by ErV View Post
I've been using this python script to convert binary files into C++ array:
Sorry to ruin your fun. It doesn't have to be that complicated:
Code:
echo \{ "$( hexdump -ve '1 1 "0x%.2x, "' file )"'0x00 };'
ta0kira
 
Old 08-24-2008, 08:24 PM   #6
Four
Member
 
Registered: Aug 2005
Posts: 298

Rep: Reputation: 30
Recently I made a "layer" to the c preprocessor because I wanted to precalculate complex stuff and put them into the in case statments

example:

Code:
case function(CONST_INPUT):
dosomething();
So I made a script which another way to do this more dynamicly and only rely on make. Save the script below as "precalc.pl"

Code:
# precalc.pl
use strict;

my $i;
my $file=0;
my $fh=undef;

# declare the functions which return something you would like
# to be put into C-code or include using "use" somewhere

# charNum is my example which takes 4 chars and combines them to make an int
sub charNum {
	my ($n) = @_;
	my @list;
	@list = split(//, $n);
	my $v;
	my $i;
	for($i = 0; $i < @list; $i++){
		$list[$i] = ord($list[$i]);
	}
	$v = $list[0] | ($list[1] << 8) | ($list[2] << 16) | ($list[3] << 24);
	return $v;
}

# is it an integer?
sub isInt {
	my ($v) = @_;
	return $v =~ /\d*/;
}


for($i = 0; $i < @ARGV; $i++){
	if($ARGV[$i] =~ /^-/){
		warn $ARGV[$i] . " Is not supported\n";
		exit(-1);
	} else {
		# take input file
		$file = $ARGV[$i];
	}
}

# if a file was not specified use stdin
if(!$file){
	$fh = "stdin";
} else {
	open $fh, "<$file" or die "Could not open $file\n";
}

while(<$fh>){
	my $e; # expression to evaluate
	my $r; # result of expresion
	# get whats in between 2 $ sign's
	if($_ =~ /\$(.*)\$/){
		$e = $1;
		# evaluate and store in $r
		eval "\$r = $e;";
		# if its an integer we would like to spit it out as hex
		# although it doesn't really matter.
		if(isInt($r)){
			$r = "0x" . sprintf("%X", $r);
		}
		# put result in between $ and $
		$_ =~ s/\$.*\$/$r/g;
	}
	# print out line
	print $_;
}
# close if we opened a file
if($file){
	close $fh;
}
Now in your makefile have

Code:
PRECALC=perl precalc.pl
.SUFFIXES: .c .cpp .o

.c.o: %.c
	$(CC) $< -E $(CFLAGS) | $(PRECALC) | $(CC) -x c  -c - -o $@
	
.cpp.o: %.c
	$(CXX) $< -E $(CXXFLAGS)| $(PRECALC)| $(CXX) -x c++ -c - -o $@
An example program:
Code:
#include <stdlib.h>
#include <stdio.h>

int main(){
printf("The windows bitmaps start with $charNum("BM")$\n");
return 0;
}
Now you can put perlcode in your C/C++ programs to be preprocessed and include data.

One thing though I wish when compiling it wouldn't say stuff like "<stdin>:33:error ...." instead to put the actual file the error was in.

Just another (more complicated) solution.

Edit: Modified the makefile so when making it will give the correct file & line numbers provided the perlscript doesn't generate extra lines.

Last edited by Four; 08-25-2008 at 10:09 PM.
 
Old 08-24-2008, 10:55 PM   #7
tomva
Member
 
Registered: May 2006
Distribution: Fedora
Posts: 45

Original Poster
Rep: Reputation: 15
Wow! Who would have guessed there was no facility for this in the C preprocessor?

It sounds like I'm going to have to do my own pre-compilation hacking to automate the inclusion of files. I already have a perl-based build system. I will probably add support for autogeneration of resource files as part of the build, something like what Erv and Four recommend.

Regarding the mmap approach: no, I can't take a run-time dependency on the file. Any necessary resources have to be embedded in the executable itself.

Thanks for the many responses!

-Thomas
 
Old 08-25-2008, 02:16 PM   #8
ta0kira
Senior Member
 
Registered: Sep 2004
Distribution: FreeBSD 9.1, Kubuntu 12.10
Posts: 3,078

Rep: Reputation: Disabled
Quote:
Originally Posted by tomva View Post
Wow! Who would have guessed there was no facility for this in the C preprocessor?
Remember that C comes from the Dark Ages, back when people were happy enough not to have to maintain Unix in assembly.
ta0kira
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
QT 4 include files produce compiler errors in wrappers on C YuriyRusinov Linux - General 0 10-29-2007 05:38 AM
Simple C Preprocessor Question ta0kira Programming 22 09-26-2006 06:33 PM
Is there a free compiler for BASIC Delano_Lucas Programming 2 07-24-2006 05:48 PM
basic compiler ryedunn Programming 5 09-26-2005 11:00 AM
is there a BASIC interpreter/compiler? needforspeed Programming 4 05-01-2005 03:47 PM


All times are GMT -5. The time now is 03:24 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration