md5sum from coreutils 8.15 "--check" option

brodo · 02-09-2012, 03:41 PM

It seems that the latest md5sum from coreutils package (8.15)
behaves differently than it used to in previous versions.

Invoking md5sum with "--check" option to check directory files against a text file containing computed md5sums gives no valid result.

Restoring previous md5sum version from coreutils package 8.12 fixed
this problem.

Alien Bob · 02-09-2012, 04:39 PM

I can not reproduce this on slackware64-current with "coreutils-8.15-x86_64-1" installed.

I tested on a random file:

Code:

$ md5sum wine-1.3.37.tar.bz2 > MD5SUMS
$ md5sum test/wine-1.3.37.tar.bz2 >> MD5SUMS
$ md5sum --check MD5SUMS 
wine-1.3.37.tar.bz2: OK
test/wine-1.3.37.tar.bz2: OK

Eric

willysr · 02-09-2012, 05:24 PM

I'm having this problem on -Current (32 bit)

Code:

md5sum -c CHECKSUMS.md5 | less
md5sum:  ./ANNOUNCE.13_37: No such file or directory
md5sum:  ./BOOTING.TXT: No such file or directory

the file exists as it's a local mirror of Slackware-Current

Alien Bob · 02-09-2012, 06:26 PM

Seems like the newer md5sum has issues with a CHECKSUMS file containing more than just md5 checksums... if you delete the header lines from Slackware's CHECKSUMS.md5 and only leave the actual checksum lines, then the command "md5sum -c CHECKSUMS.md5" works.

Eric

willysr · 02-09-2012, 07:58 PM

Quote:

Originally Posted by Alien Bob

Seems like the newer md5sum has issues with a CHECKSUMS file containing more than just md5 checksums... if you delete the header lines from Slackware's CHECKSUMS.md5 and only leave the actual checksum lines, then the command "md5sum -c CHECKSUMS.md5" works.

Yes, i think you are right about this one
it's a bug upstream, but i still haven't found any discussion on the coreutils list so far

brodo · 02-10-2012, 04:18 AM

Quote:

Originally Posted by Alien Bob

Seems like the newer md5sum has issues with a CHECKSUMS file containing more than just md5 checksums... if you delete the header lines from Slackware's CHECKSUMS.md5 and only leave the actual checksum lines, then the command "md5sum -c CHECKSUMS.md5" works.

Eric

Yes, deleting header infos from the beginning of the CHECKSUMS.md5 file fixed this.
I am using 32-bit current Slack.

Thanks.

MadMaverick9 · 02-10-2012, 05:56 AM

This happens because "md5sum" now detects "bsd reversed" format:

Code:

2c2
<    Copyright (C) 1995-2011 Free Software Foundation, Inc.
---
>    Copyright (C) 1995-2012 Free Software Foundation, Inc.
102c102
<    + 2 /* blank and binary indicator */ \
---
>    + 1 /* blank */ \
124a125,131
> /* With --check, exit with a non-zero return code if any line is
>    improperly formatted. */
> static bool strict = false;
> 
> /* Whether a BSD reversed format checksum is detected.  */
> static int bsd_reversed = -1;
> 
130c137,138
<   QUIET_OPTION
---
>   QUIET_OPTION,
>   STRICT_OPTION
140a149
>   { "strict", no_argument, NULL, STRICT_OPTION },
189a199,201
>       fputs (_("\
>       --strict         with --check, exit non-zero for any invalid input\n\
> "), stdout);
196,197c208,209
< a line with checksum, a character indicating type (`*' for binary, ` ' for\n\
< text), and name for each FILE.\n"),
---
> a line with checksum, a character indicating input mode (`*' for binary,\n\
> space for text), and name for each FILE.\n"),
301,303c313,330
<   if (s[i] != ' ' && s[i] != '*')
<     return false;
<   *binary = (s[i++] == '*');
---
>   /* If "bsd reversed" format detected.  */
>   if ((s_len - i == 1) || (s[i] != ' ' && s[i] != '*'))
>     {
>       /* Don't allow mixing bsd and standard formats,
>          to minimize security issues with attackers
>          renaming files with leading spaces.
>          This assumes that with bsd format checksums
>          that the first file name does not have
>          a leading ' ' or '*'.  */
>       if (bsd_reversed == 0)
>         return false;
>       bsd_reversed = 1;
>     }
>   else if (bsd_reversed != 1)
>     {
>       bsd_reversed = 0;
>       *binary = (s[i++] == '*');
>     }
358c385
< static bool
---
> static bool _GL_ATTRIBUTE_PURE
436a464
>   uintmax_t n_improperly_formatted_lines = 0;
503a532,533
> 
>           ++n_improperly_formatted_lines;
606c636,637
<           && n_open_or_read_failures == 0);
---
>           && n_open_or_read_failures == 0
>           && (!strict || n_improperly_formatted_lines == 0));
659a691,693
>       case STRICT_OPTION:
>         strict = true;
>         break;
695a730,736
> 
>   if (strict & !do_check)
>    {
>      error (0, 0,
>         _("the --strict option is meaningful only when verifying checksums"));
>      usage (EXIT_FAILURE);
>    }

The header info in "CHECKSUMS.MD5" causes "md5sum" to set "bsd_reversed" to "1".

Quote:

Don't allow mixing bsd and standard formats, to minimize security issues with attackers renaming files with leading spaces.

From then on "md5sum" expects all following checksums to be in "bsd reversed" format, which they are not of course, and therefor all fails.

I verified this by changing all lines in "CHECKSUMS.MD5" to "bsd reversed" format and all works well.

Code:

sed 's/  / /' CHECKSUMS.md5 | md5sum --check

"bsd reversed" format has only one space between hash and filename.

MadMaverick9 · 02-10-2012, 07:11 AM

I found the following in the "md5sum" source code:

Code:

      /* Ignore comment lines, which begin with a '#' character.  */
      if (line[0] == '#')
	continue;

This is the source code from 2005 http://git.savannah.gnu.org/cgit/cor...e4d37a4cd#n424

And here the latest source code http://git.savannah.gnu.org/cgit/cor...3f21431da#n509

willysr · 02-10-2012, 09:19 AM

Here's the commit code http://git.savannah.gnu.org/gitweb/?...0e9a35fd89da3b