How to read tar file from buffer in C

RohitB · 01-15-2019, 02:48 AM

Hi,

I created tar file inside buffer by using libarchive API.
archive_write_open_memory(file_handle->a,file_handle->buf,1024,&file_handle->used);
File will be created inside file_handle->buf
Now i want to read data from this buffer in place of file.
For example for read data from tar file we can call read system call by passing fd.
How we can achieve same reading through buffer?

rtmistler · 01-15-2019, 06:36 AM

A continued welcome to LQ,

Of course you can read data from memory, or a structure, or a file using the C language.

What's missing here is any code excepting that one line statement you've shown.

Have you reviewed the documentation for libarchive, https://libarchive.org/?

You should follow the suggestion you received in your other thread, Creating tar in buffer through libarchive and streaming it, and review information about how to use the Linux Questions site as well as how to ask effective questions.

Glad you feel you've succeeded with creating the buffer. Now you're saying you wish to read the buffer. Either this buffer is in memory, or it is a file. Do you know how to reference and read memory using the C language? Do you know how to open and read a file using the C language? Either case, these are the items you'd need to do to read information. It's rather hard to give you an exact statement how to do something when it is unclear what you've tried, and what your programming experience is.

RohitB · 01-15-2019, 07:23 AM

Hi,
Its not a simple buffer reading, the buffer will be containing tar data so how to do tar reading from buffer.
Buffer i am refering to inmemory not file
Simple buffer reading will not work since it is a tar stream.
Normal buffer reading will not work since it conatin tar stream.
But if we do read system call from file it works as expected but read need a fd and use for file reading.

pan64 · 01-15-2019, 07:30 AM

Did you try to read the documentation of libarchive (as it was mentioned)?
https://github.com/libarchive/libarchive/wiki/Examples

RohitB · 01-15-2019, 07:57 AM

Tried everything, there is no example in libarchieve of reading from buffer.I already read all example.

BW-userx · 01-15-2019, 08:04 AM

aside from what the api doing it, buffer is just a memory slot with data in it, like opening a file and putting it contents into a buffer then acting on it. I'd think this to be similar.

List contents of Archive stored in File

List contents of Archive stored in Memory, same page

RohitB · 01-15-2019, 08:08 AM

Hi ,
I have limitation of hardisk i cannot create tar in file.I need to do keep tar in buffer and stream it.

BW-userx · 01-15-2019, 08:30 AM

Quote:

Originally Posted by RohitB

Hi ,
I have limitation of hardisk i cannot create tar in file.I need to do keep tar in buffer and stream it.

List contents of Archive stored in Memory, same page

Code:

struct archive *a = archive_read_new();
archive_read_support_compression_gzip(a);
archive_read_support_format_tar(a);
r = archive_read_open_memory(a, buff, sizeof(buff));

RohitB · 01-17-2019, 08:49 AM

Hi,

archive_read_open_memory this will again create archieve after reading from buffer.I dont want this i want data to be copied to normal char * buffer that can flush.If you will archieve buffer as normal buffer it will give incalid result

NevemTeve · 01-17-2019, 10:37 AM

@OP: Do you happen to have a Minimal Complete, Verifiable Example (mcve) ?

RohitB · 01-18-2019, 03:10 AM

Its simple reading taken from example:-

Code:

char buff[1024];//temp buffer for reading from file.
char archive_buf[2048]//archive will be store in this buffer.
int used ;
a = archive_write_new();
archive_write_add_filter_gzip(a);
archive_write_set_format_pax_restricted(a); // Note 1
archive_write_open_memory(a,archive_buf,1024,&used);
stat(*filename, &st);
entry = archive_entry_new(); 
archive_entry_set_pathname(entry, "sample.xml");
archive_entry_set_size(entry, st.st_size); 
archive_entry_set_filetype(entry, AE_IFREG);
archive_entry_set_perm(entry, 0644);
archive_write_header(a, entry);
fd = open("sample.xml", O_RDONLY);//opening xml file and reading its content so that tar can be prepared.

len = read(fd, buff, sizeof(buff));
while ( len > 0 ) {
 len = read(fd, buff, sizeof(buff));
 archive_write_data(a, buff, len);
}

Creation of archive in buffer is done.
Now how to do reading from this buffer and send it as MIME to Server?

If I have tar as file below method is working as expected.
bytes_read = fread(buf_containing_read_bytes_from_tar_file, sizeof(char), 1024, (FILE*)file pointer to tar file);
Same thing how i can do when data is in buffer??//Note I cannot use buf which i used for creating archive.It is just a place holder.I need to read data from this buffer and write data into streaming buffer.

astrogeek · 01-18-2019, 02:15 PM

Please place your code snippets inside [CODE]...[/CODE] tags for better readability. You may type those yourself or click the "#" button in the edit controls.

I have fixed that in your previous post as an example.

NevemTeve · 01-18-2019, 09:19 PM

This isn't a complete program. Please read this: https://stackoverflow.com/help/mcve