Quickie: what is a file that starts with these bytes?

dedec0 · 09-07-2019, 04:23 PM

Now i have this binary file that should be some sort of patch (for a project file tree, not for a single file in it), but i do not know what to do with it. It is offered do download without extension - which lead me to test it with a few extract commands: bzip, xz, gzip, tar.

A few lines from my terminal should be informative:

Code:

$  head  patch-1  |xxd |head -n 2
00000000: 2558 445a 3030 3425 0000 0008 0015 0015  %XDZ004%........
00000010: 0000 0000 0000 0000 0000 0000 0000 0000  ................

Do you know what that file header is, or how to find it with that info?

(xz starting bytes have "7z" written... do they have the same format that 7zip uses?)

-------
Post "mark as solved" note: this thread is marked as solved, since its title question and problem have been answered. But there are other things still without answer. They are listed in the post #22.

rtmistler · 09-07-2019, 05:22 PM

Does the file(1) command give anything?

michaelk · 09-07-2019, 05:33 PM

I could not find %XDZ in any file signature list so I am guessing probably not.

dedec0 · 09-07-2019, 05:37 PM

Quote:

Originally Posted by rtmistler

Does the file(1) command give anything?

Nice command! I would never imagine it existed.

Code:

$  file patch-1 
patch-1:  XDelta binary patch file 1.1
$

michaelk · 09-07-2019, 05:40 PM

Well I guessed wrong.

Firerat · 09-07-2019, 05:40 PM

startpage got me to this

http://www-inf.it-sudparis.eu/cours/...l/TP/doc/magic

Code:

#------------------------------------------------------------------------------
# file(1) magic(5) data for xdelta  Josh MacDonald <jmacd@CS.Berkeley.EDU>
<snip>
0	string	%XDZ004%	XDelta binary patch file 1.1

damn, I got beat by file and its magic

dedec0 · 09-07-2019, 05:50 PM

Quote:

Originally Posted by dedec0

Nice command! I would never imagine it existed.

Code:

$  file patch-1 
patch-1:  XDelta binary patch file 1.1
$

After discovevring that (and searching and reading around a bit), i installed 'xdelta' command, which was done very quickly in the Debian i am using:

Code:

$ sudo -s
[...]
# apt-get install xdelta3
[...]
#
$

But i am not understanding, by reading its manpage, what i have to do with the patch file, and the project tree (or its compated file).

Hints? /-:

The manpage is:

Code:

XDELTA3(1)                  General Commands Manual                 XDELTA3(1)

NAME
       xdelta3 - VCDIFF (RFC 3284) binary diff tool

SYNOPSIS
       xdelta3 [command] [options] [input [output]]

DESCRIPTION
       xdelta3  is  a  binary diff tool that uses the VCDIFF (RFC 3284) format
       and compression.

COMMANDS
       config prints xdelta3 configuration

       decode decompress the input, also set by -d

       encode compress the input, also set by -e (default)

       test   run the builtin tests

       printdelta
              print information about the entire delta

       printhdr
              print information about the first window

       printhdrs
              print information about all windows

       recode encode with new application/secondary settings

OPTIONS
       standard options:

       -0 .. -9
              compression level

       -c     use stdout

       -d     decompress

       -e     compress

       -f     force overwrite

       -h     show help

       -q     be quiet

       -v     be verbose (max 2)

       -V     show version

       memory options:

       -B     bytes source window size

       -W     bytes input window size

       -P     size compression duplicates window

       -I     size instruction buffer size (0 = unlimited)

       compression options:

       -s     source source file to copy from (if any)

       -S [djw|fgk|lzma|none]
              enable/disable secondary compression

       -N     disable small string-matching compression

       -D     disable external decompression (encode/decode)

       -R     disable external recompression (decode)

       -n     disable checksum (encode/decode)

       -C     soft config (encode, undocumented)

       -A [apphead]
              disable/provide application header (encode)

       -J     disable output (check/compute only)

       -T     use alternate code table (test)

NOTES
       The XDELTA environment variable may contain extra args:

              XDELTA="-s source-x.y.tar.gz" \
              tar --use-compress-program=xdelta3 -cf \
              target-x.z.tar.gz.vcdiff target-x.y/

EXAMPLES
       Compress the differences between SOURCE and TARGET, yielding OUT, using
       "djw" secondary compression:

       xdelta3 -S djw -s SOURCE TARGET OUT

       Do the same, using standard input and output:

       xdelta3 -S djw -s SOURCE < TARGET > OUT

       To decompress OUT, using SOURCE, yielding TARGET:

       xdelta3 -d -s SOURCE OUT TARGET

AUTHOR
       xdelta3 was written by Josh MacDonald <josh.macdonald@gmail.com>.

       This    manual    page   was   written   by   Leo   'costela'   Antunes
       <costela@debian.org> for the Debian project (but may be  used  by  oth‐
       ers).

Xdelta3                           August 2009                       XDELTA3(1)

Firerat · 09-07-2019, 05:53 PM

from memory its is pretty much like how "normal" patch works
only it isn't text

edit
nah, but different

xdelta3 -S djw -s SOURCE TARGET OUT
xdelta3 -S djw -s yourpatch whatyouarefixing thefixedversion

dedec0 · 09-07-2019, 05:56 PM

Quote:

Originally Posted by dedec0

After discovevring that (and searching and reading around a bit), i installed 'xdelta' command, which was done very quickly in the Debian i am using:

Code:

$ sudo -s
[...]
# apt-get install xdelta3
[...]
#
$

But i am not understanding, by reading its manpage, what i have to do with the patch file, and the project tree (or its compated file).

Hints? /-:

The manpage is:

Code:

XDELTA3(1)                  General Commands Manual                 XDELTA3(1)

NAME
       xdelta3 - VCDIFF (RFC 3284) binary diff tool

SYNOPSIS
       xdelta3 [command] [options] [input [output]]

DESCRIPTION
       xdelta3  is  a  binary diff tool that uses the VCDIFF (RFC 3284) format
       and compression.

COMMANDS
       config prints xdelta3 configuration

       decode decompress the input, also set by -d

       encode compress the input, also set by -e (default)

       test   run the builtin tests

       printdelta
              print information about the entire delta

       printhdr
              print information about the first window

       printhdrs
              print information about all windows

       recode encode with new application/secondary settings

OPTIONS
       standard options:

       -0 .. -9
              compression level

       -c     use stdout

       -d     decompress

       -e     compress

       -f     force overwrite

       -h     show help

       -q     be quiet

       -v     be verbose (max 2)

       -V     show version

       memory options:

       -B     bytes source window size

       -W     bytes input window size

       -P     size compression duplicates window

       -I     size instruction buffer size (0 = unlimited)

       compression options:

       -s     source source file to copy from (if any)

       -S [djw|fgk|lzma|none]
              enable/disable secondary compression

       -N     disable small string-matching compression

       -D     disable external decompression (encode/decode)

       -R     disable external recompression (decode)

       -n     disable checksum (encode/decode)

       -C     soft config (encode, undocumented)

       -A [apphead]
              disable/provide application header (encode)

       -J     disable output (check/compute only)

       -T     use alternate code table (test)

NOTES
       The XDELTA environment variable may contain extra args:

              XDELTA="-s source-x.y.tar.gz" \
              tar --use-compress-program=xdelta3 -cf \
              target-x.z.tar.gz.vcdiff target-x.y/

EXAMPLES
       Compress the differences between SOURCE and TARGET, yielding OUT, using
       "djw" secondary compression:

       xdelta3 -S djw -s SOURCE TARGET OUT

       Do the same, using standard input and output:

       xdelta3 -S djw -s SOURCE < TARGET > OUT

       To decompress OUT, using SOURCE, yielding TARGET:

       xdelta3 -d -s SOURCE OUT TARGET

AUTHOR
       xdelta3 was written by Josh MacDonald <josh.macdonald@gmail.com>.

       This    manual    page   was   written   by   Leo   'costela'   Antunes
       <costela@debian.org> for the Debian project (but may be  used  by  oth‐
       ers).

Xdelta3                           August 2009                       XDELTA3(1)

Nonono... forget that, for a few minutes, at least... the examples in the end should contain one with exactly what i need. What was confusing me is the command line options only talk about compress and decompress, and i was looking for diffs and comparisons things...

Firerat · 09-07-2019, 06:01 PM

yeah, the very last one if no compression

xdelta3 -d -s SOURCE OUT TARGET

Edit:
Now that I think about it, I've gotten this confused with
bdiff and bpatch

dedec0 · 09-07-2019, 06:38 PM

Quote:

Originally Posted by Firerat

yeah, the very last one if no compression

xdelta3 -d -s SOURCE OUT TARGET

Edit:
Now that I think about it, I've gotten this confused with
bdiff and bpatch

I thought that was the right example too. But i am lost, and could not get it to work.

I have these files (with artificial comments i write only here):

Code:

$ ls
bacana-x11-1.2.tar.bz2         # the project source
bacana-x11-1.2                 # project folder containing everything in the above file
patch-bacana-x11-1.2           # the path file that lead me to start this thread

$

I tried these:

Code:

$ # xdelta3 -d -s [patch file] [proj folder name] [unexistent file/dir name]
xdelta3: not a VCDIFF input: XD3_INVALID_INPUT
xdelta3: normally this indicates that the source file is incorrect
xdelta3: please verify the source file with sha1sum or equivalent

$ # xdelta3 -d -s [proj folder name] [patch file] [unexistent file/dir name]
xdelta3: input read failed: qt-x11-free-3.3.8: Is a directory

$ # xdelta3 -d -s [patch file] [project bz2 file] [unexistent file/dir name]
xdelta3: externally compressed input: bzip2 -dc < [unexistent file/dir name]
xdelta3: WARNING: the encoder is automatically decompressing the input file;
xdelta3: WARNING: the decoder will automatically recompress the output file;
xdelta3: WARNING: this may result in different compressed data and checksums
xdelta3: WARNING: despite being identical data; if this is an issue, use -D
xdelta3: WARNING: to avoid decompression and/or use -R to avoid recompression
xdelta3: WARNING: and/or manually decompress the input file; if you know the
xdelta3: WARNING: compression settings that will produce identical output
xdelta3: WARNING: you may set those flags using the environment (e.g., GZIP=-9)
xdelta3: not a VCDIFF input: XD3_INVALID_INPUT
xdelta3: normally this indicates that the source file is incorrect
xdelta3: please verify the source file with sha1sum or equivalent

$

I am not in the right path, probably... /-: Should i use the tar file?

Firerat · 09-07-2019, 06:56 PM

Code:

SOURCE="bacana-x11-1.2.tar.bz2"
TARGET="patched_bacana-x11-1.2.tar.bz2"              
OUT="patch-bacana-x11-1.2"   

xdelta3 -d -s "${SOURCE}" "${OUT}" "${TARGET}"

if that didn't work I would drop the -d

if that didn't work I would try

Code:

bzcat "${SOURCE}" > "${SOURCE%.bz2}"
xdelta3 -d -s "${SOURCE%.bz2}" "${OUT}" "${TARGET%.bz2}"

and again drop the -d

I'm just guessing here

ondoho · 09-08-2019, 02:17 AM

I wonder what software you are trying to binary-patch there.
It just seems wrong to any self-respecting FLOSS user, and I immediately ask myself what the end goal is and if there aren't much better solutions to achive it...

dedec0 · 09-15-2019, 05:41 PM

Quote:

Originally Posted by ondoho

I wonder what software you are trying to binary-patch there.
It just seems wrong to any self-respecting FLOSS user, and I immediately ask myself what the end goal is and if there aren't much better solutions to achive it...

X11 Qt3 has an official patch i am trying to use.

In the folder https://download.qt.io/archive/qt/3/, there are these 3 files i downloaded:

patch-qt-x11-free-3.3.8
qt-x11-free-3.3.8b.tar.gz
qt-x11-free-3.3.8.tar.bz2

There is another file for 3.3.8 version, which i did not try anything:

qt-x11-free-3.3.8.tar.gz

(...)

smallpond · 09-15-2019, 05:51 PM

The Makefile may already know how to apply patches. Check the README file for instructions.