[SOLVED] I want to create a dummy file of 1TB

Shozib · 07-23-2019, 02:54 AM

Hi, everyone
As listed in the subject basically I want to create a large file in linux.
Currently I am using this command:
dd if=/dev/zero of=file.txt count=1024 bs=1024

to do that, it can create a file in some GBs like 7, 8 etc but I want to create a large file like 500GB or 1TB.

When I give a large number to the parameters "Count" and "bs"
I am having this error:
DD: Memory Exhausted by Input Buffer of Size Bytes (9Gib)

Can anyone please guide me in this regard. Or suggest me to use any other suitable way. I would be very thankful to you.

Thanks
Shozib

syg00 · 07-23-2019, 03:13 AM

Depending on how you intend to use the file, "truncate" is probably best. Have a read of sparse_file.

mike acker · 07-23-2019, 06:08 AM

Code:

#include <stdio.h>
#include <string.h>

#define OUTPUT_SIZE_REQUEST 1000
#define OUTPUT_FILE_NAME cmd_line_arg[1]

	int main(int nof_args, char **cmd_line_arg);

	int main(int nof_args, char **cmd_line_arg)
	{
		FILE *text_out;
		char *test_text = "This has been a test. \n "
			"Had it been an actual alert you would not have received this message.\n";

		long int msg_size, total_output = 0;		puts(test_text);

		if (2==nof_args) ; else { puts ("Supply output file name on command line"); return(1); }
		
		text_out=fopen(OUTPUT_FILE_NAME,"w");

		if (NULL==text_out)
		{ printf("unable to open %s file for output\n", OUTPUT_FILE_NAME);
				return(2);	}
		else printf("%s file opened for output\n", OUTPUT_FILE_NAME);

		msg_size = strlen(test_text);

		do {	fputs(test_text,text_out);
		 	total_output+=msg_size; }
		while (total_output<OUTPUT_SIZE_REQUEST);

		printf("%d output characters written\n", total_output);

		fclose(text_out);
		return(0);

	}

Shozib · 07-23-2019, 06:20 AM

Quote:

Originally Posted by syg00

Depending on how you intend to use the file, "truncate" is probably best. Have a read of sparse_file.

Alright, Thankyou for your prompt reply.

BW-userx · 07-23-2019, 07:07 AM

Quote:

Originally Posted by mike acker

Code:

#include <stdio.h>
#include <string.h>

#define OUTPUT_SIZE_REQUEST 1000
#define OUTPUT_FILE_NAME cmd_line_arg[1]

	int main(int nof_args, char **cmd_line_arg); // not needed

	int main(int nof_args, char **cmd_line_arg)
	{
		FILE *text_out;
		char *test_text = "This has been a test. \n "
			"Had it been an actual alert you would not have received this message.\n";

		long int msg_size, total_output = 0;		puts(test_text);

		if (2==nof_args) ; else { puts ("Supply output file name on command line"); return(1); }
		
		text_out=fopen(OUTPUT_FILE_NAME,"w");

		if (NULL==text_out)
		{ printf("unable to open %s file for output\n", OUTPUT_FILE_NAME);
				return(2);	}
		else printf("%s file opened for output\n", OUTPUT_FILE_NAME);

		msg_size = strlen(test_text);

		do {	fputs(test_text,text_out);
		 	total_output+=msg_size; }
		while (total_output<OUTPUT_SIZE_REQUEST);

		printf("%d output characters written\n", total_output);

		fclose(text_out);
		return(0);

	}

why did you defind main? that is the only function that I know that does not need to be defined, but has to be included or a program will not work.

mike acker · 07-23-2019, 08:10 AM

Quote:

Originally Posted by BW-userx

why did you defind main? that is the only function that I know that does not need to be defined, but has to be included or a program will not work.

leftover habit, probably.
the program works without the function definition.

the need for this:

Code:

#define _GNU_SOURCE /* include in order to use fcloseall() */

kinda threw me; I'm running Debian.10 -- and I had not needed that earlier.

Shadow_7 · 07-26-2019, 04:28 AM

You could create the 1GB file and cat it to a new file >> append style 1000 times. Or 1024. A simple one liner bash script could help accomplish that.

$ for ((COUNT=0; COUNT<1000; COUNT++)); do echo $COUNT; $(cat FILE1GB.dat >> BIGFILE1TB.dat); done

Be sure the file is empty before start, or add code on COUNT=0 to > instead of >>. Many ways.

BW-userx · 07-26-2019, 06:49 AM

Wouldn't this too work?

Code:

sudo fallocate -l 1GB dummyfile

http://man7.org/linux/man-pages/man1/fallocate.1.html

which is used for swap files, and is not much different then dd to create the same. Just a crap file that then is assigned to be swap or whatever after being created.