Linux command to copy every other byte from file

kdawgud · 03-02-2009, 07:22 AM

I'm looking for a linux command to copy every other byte from a binary file. Another way to state this: I want to copy only the odd numbered bytes or only the even numbered bytes from a file.

I am familiar with other programs such as head and tail that let me trim the beginning or end of a file, but they don't seem like they will help me in this case.

I'm hoping some sort of built-in utility exists so I don't have to write my own C program to do this.

Thanks!

pixellany · 03-02-2009, 07:29 AM

Welcome to LQ!!

Why do you want to do this? Knowing the whole picture might help us give better answers.

I want to believe that "dd" will do this, but I don't have time to dig into it.

kdawgud · 03-02-2009, 07:37 AM

Quote:

Originally Posted by pixellany

Welcome to LQ!!

Why do you want to do this? Knowing the whole picture might help us give better answers.

I want to believe that "dd" will do this, but I don't have time to dig into it.

I'm dealing with a signal acquisition card that outputs interlaced data to file. I need to de-interlace the files in order to view the data properly. It happens in this case that the de-interlace process simply involves copying the odd bytes into one file and the even bytes into another file.

I looked at the dd man page and didn't see an applicable option.

pixellany · 03-02-2009, 07:42 AM

Try "man -k byte". This searches the man pages for the word "byte". There are a bazillion utilities, but most are not BASH---ie they are system calls.

Also try the Advanced Bash Scripting Guide----http://tldp.org

AuroraCA · 03-02-2009, 07:51 AM

I believe that you can do this with dd by setting ibs=2 and obs=1

allend · 03-02-2009, 08:07 AM

Perhaps something along these lines?

Quote:

od -w2 -t x1 -v temp.txt | awk ' { print $2 } '

This will print the odd bytes in hexadecimal as ASCII.
Change $2 to $3 for the even bytes.
If you want decimal representations, change the print format in the awk action.

kdawgud · 03-02-2009, 08:20 AM

Quote:

Originally Posted by allend

Perhaps something along these lines?

This will print the odd bytes in hexadecimal as ASCII.
Change $2 to $3 for the even bytes.
If you want decimal representations, change the print format in the awk action.

What if I need the data back in binary format? How do I convert the ASCII text output back into binary?

allend · 03-02-2009, 08:59 AM

Quote:

od -w2 -t u1 -v temp.txt | awk ' { printf("%c", strtonum($2)) } ' > temp3.txt

This takes the odd numbered bytes from file 'temp.txt' and dump them to 'temp3.txt'
Change $2 to $3 for the even numbered bytes.

allend · 03-02-2009, 08:59 AM

Sorry - Double post

H_TeXMeX_H · 03-02-2009, 11:46 AM

Quote:

Originally Posted by AuroraCA

I believe that you can do this with dd by setting ibs=2 and obs=1

Wow, very ingenious, I would have thought for sure a C program was in order ... but logically this should work.

AuroraCA · 03-02-2009, 12:10 PM

I've not actually tried the dd solution but dd is a very powerful command. You may need to use ibs="c 2" obs="c 1" .

alexander.stohr.h · 06-26-2014, 06:38 AM

dd is doing only block wise copys of data - and it can only skip leading block in input and/or output files.
the block sizes control how big the chunk of data for each read/write operation is. that is helpful for speeding things up. it might also have some impact when operating with char devices and streams. the above mentioned command line specification of "c 2" and similar seemingly is faulty and thus will be rejected by the tool
rationale: dd wont work for you.

od is a tool for printing binary data in a human readable fashion, thus using ASCII decoded formats as output in a line based fashion. to my best understanding it can not extract binary data following some pattern to some other binary data file.
rationale: od wont work for you.

alexander.stohr.h · 06-26-2014, 07:01 AM

how about xxd, see "man xxd".
create a to-ascii + to-binary pipeline and then add something like "cut -b 1-2" in the middle. :-)

Example for some 2 MB of input data with some read-offset that will result into some 1 MB of output data.

Quote:

$ xxd -p -c 2 -s 0xABCD -l 2048000 input.bin | cut -b 1-2 | xxd -r -p >output.bin

rtmistler · 06-27-2014, 07:58 AM

Write a C program to do this. You are doing signal analysis, a pretty technical thing and you have a highly specific channel separation requirement of a binary file.

What happens if/when you go to 4, 6, or 8 channels?

use open(), read() and process every byte in that file

Use argc and argv to allow for input/output names.

Write it so that you can extend it for the next time when the customized requirements change.

Very simple example, untested, but compiles in gcc:

Code:

#include <stdio.h>
#include <string.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>

int main(int argc, char *argv[])
{
    int rdHdl, wr1Hdl, wr2Hdl, ret, i;
    char buffer[1024];

    if(argc < 1) {
        printf("Usage: %s <input-filename>\n", argv[0]);
        return -1;
    }

    if((rdHdl = open(argv[1], O_RDONLY | O_NONBLOCK)) < 0) {
        printf("Error opening file %s.  Error %d:%s\n", argv[1], errno, strerror(errno));
        return -1;
    }

    memset(buffer, 0, sizeof(buffer));
    strncpy(buffer, argv[1], sizeof(buffer));
    strcat(buffer, ".1");

    if((wr1Hdl = open(buffer, O_WRONLY | O_NONBLOCK)) < 0) {
        // print error
        return -1;
    }

    memset(buffer, 0, sizeof(buffer));
    strncpy(buffer, argv[1], sizeof(buffer));
    strcat(buffer, ".2");

    if((wr2Hdl = open(buffer, O_WRONLY | O_NONBLOCK)) < 0) {
        // print error
        return -1;
    }

    /* Input file and two output files are now open */
    /* Read input and put into output */
    while((ret = read(rdHdl, &buffer[0], sizeof(buffer))) > 0) {
        for(i = 0; i < ret; i += 2) {
            write(wr1Hdl, buffer[i], 1);
            write(wr2Hdl, buffer[i+1], 1);
        }
    }

    close(rdHdl);
    close(wr1Hdl);
    close(wr2Hdl);
    // print success
    return 0;
}

schneidz · 06-27-2014, 08:08 AM

for shiggles, heres my credit card swiper parser. it reads every 8th character on the card:

Code:

#include "stdio.h"
 
main(int argc, char *argv[])
{
 int i, c;
 FILE * fstream;
 
 fstream = fopen(argv[1], "r");
 c = fgetc(fstream);
 
 for(i=0; i < 4059; i++)
   fgetc(fstream);

 for(i=0; i < 380; i++)
 {
  if((i - 1) % 8 == 0)
   printf("%c", c);
  c = fgetc(fstream);
 }
 fclose(fstream);
}