[SOLVED] Searching for a specific hex string or byte array in various images

plasma33 · 05-07-2016, 01:21 AM

David, I tried copying and pasting your given code against .jpg files. It works without any issues but for some weird reason it doesn't want to work on the .tif file format. When I try to use the codes given below it works without any issues and gives me what I want but without the String search count:

Code:

xxd -p /your/file | tr -d '\n' | grep -c '22081b00081f091d2733170d123f3114' WORKS FINE and returns 1 for a single .tif file. I tried on 400 .tif files too and it works without any issues and returns me 1 each time.

and,

awk '{/\x00\x02\x13\x12\x30\x44/ {print FILENAME ; exit}' $f WORKS FINE too and returns the full .tif file locations for all the available .tif files in that specific directory.

What might be possibly wrong? I am 100% certain that the string I am using is a genuine one and is found inside all the 400 .tif files.

Thanks.

Plasma33

plasma33 · 05-07-2016, 01:25 AM

Quote:

Originally Posted by syg00

I would simply pipe it to "wc -l".

Hi syg00,

Is this the right way to do it?

Code:

wc -l | awk '{/\x00\x02\x13\x12\x30\x44/ {print FILENAME ; exit}'

Thanks.

Plasma33

DavidPhillips · 05-07-2016, 01:34 AM

I'm not sure as I was using your example with xxd. You could substitute the whole command string from cat with your awk command.

file findhex

Code:

chmod 755 findhex

Code:

#!/bin/sh 
FTYPE=$1
STR=$2
echo
echo 'Total Files '`ls *"$FTYPE" | wc -l` 
echo 'Search string count '`cat *"$FTYPE" | xxd -p $f | tr -d '/n'|grep -c "$STR" | awk '{ SUM += $1}; END { print SUM }'`

david@david-Dell-System-XPS-L702X ~/pics $ ./findhex png 67984

Total Files 37
Search string count 33

[/code]

syg00 · 05-07-2016, 02:01 AM

Nope, other end - pipe the output of awk to wc

plasma33 · 05-07-2016, 03:52 AM

Thank you for that awesome innovative code. It worked without any issues but for some unknown reason its giving me a count of 1 instead of 400 again. Is there any chance of piping the output of xxd to wc as @syg00 is suggesting. Thank you @syg00 for that comment. Could you please demostrate by giving us a command for it, please.

Thanks.

Plasma33

plasma33 · 05-07-2016, 03:00 PM

Hi guys,

Thank you for all your support and help here. I managed to find this code through stack overflow website. Please find the working code below:

Code:

#!/bin/bash

files=/FILEPATH/*
for f in $files
do
  # Watch byte order. To grep for 22081b00:
  grep -m 1 -P '\x22\x08\x1b\x00' < "$f"
done | wc -l

Thanks, again for all your efforts here.

Plasma33

keefaz · 05-07-2016, 03:26 PM

Why not use -l (L lowercase) switch for grep, you save a loop

Code:

grep -l -P '\x22\x08\x1b\x00' /FILEPATH/* | wc -l

plasma33 · 05-08-2016, 01:31 AM

Quote:

Originally Posted by keefaz

Why not use -l (L lowercase) switch for grep, you save a loop

Code:

grep -l -P '\x22\x08\x1b\x00' /FILEPATH/* | wc -l

Thanks, keefaz for your code. It works without any issues. Any ideas for adding echo "Something Here" command inside a loop especially inside the code given below, please. I can't do it for some reason. Also, is it possible to add echo command after done and before the pipe?

Code:

#!/bin/bash

files=/FILEPATH/*
for f in $files
do
  # Watch byte order. To grep for 22081b00:
  grep -m 1 -P '\x22\x08\x1b\x00' < "$f"
done | wc -l

Thanks.

Plasma33

keefaz · 05-08-2016, 05:31 AM

If you add any echo line before the wc -l pipe, you'll end with wrong count at output, wc will count also your echo lines
Or maybe redirect echo output to standard error
As for putting a command between the done and the wc pipe, it won't work, you'll break link with output from the loop and wc pipe input.

Code:

for f in $files
do
  # Watch byte order. To grep for 22081b00:
  grep -m 1 -P '\x22\x08\x1b\x00' < "$f"
  echo "Something Here" >&2
done | wc -l

plasma33 · 05-08-2016, 06:57 PM

Thanks, keefaz. It works perfectly. Thanks for the explanation.

Plasma33