How can I un XOR encode a file?

abefroman · 03-18-2013, 04:17 PM

How can I un XOR encode a file?

When I try with C I get a file that can't be read by objdump:

Code:

        
        buf = malloc(size);

        fread(buf, 1, size, f);

        for (i = 0; i < size; i++) {
                buf[i] ^= 0x81;
        }

Code:

$ objdump -DR output2
objdump: output2: File format not recognized

I can still strings that file and get more info but I'm trying to view the assembly code.

jpollard · 03-18-2013, 04:29 PM

Just run xor again on the output from the first time.

xor is reflexive - doing it a second time reverses what was done the first time.

abefroman · 03-18-2013, 04:42 PM

Quote:

Originally Posted by jpollard

Just run xor again on the output from the first time.

xor is reflexive - doing it a second time reverses what was done the first time.

That's what I did in C, but it appears the whole program is not XOR, which I am assuming is why objdump can't read it.

This would be the decode routine to the compiler I'm assuming:

Code:

     
    xor    %eax,%eax
    lea    0x0(%esi,%eiz,1),%esi
    xorb   $0x81,0x138(%eax,%ebx,1)
  
    add    $0x1,%eax
    cmp    $0x1ed,%eax

dwhitney67 · 03-23-2013, 07:03 AM

Quote:

Originally Posted by abefroman

That's what I did in C, but it appears the whole program is not XOR, which I am assuming is why objdump can't read it.

You are incorrectly assuming that 'buf' always contains 'size' bytes, where in fact it only contains the number of bytes that have been reported to be read by fread() --- which you failed to check. I would recommend something similar to:

Code:

const size_t MAX_BUF_SZ = 1024;

char* buf = malloc(MAX_BUF_SZ);
size_t bytesRead = 0;

while ((bytesRead = fread(buf, 1, MAX_BUF_SZ, f)) > 0)
{
    for (size_t i = 0; i < bytesRead; ++i)
    {
        buf[i] ^= 0x81;
    }

    // ... (write bytesRead number of bytes from buf to other file)
}

As for objdump not recognizing your file, well there's good reason... the file should be pure data with no discernible format to disassemble.