CRC32. Behind reliability.
by Ilya O. Levin

We're all do not like a forgery. This is the reason why we're using some verification methods while file exchanging to be sure about authenticity. The one of this method is a checksum calculation. This article is about weakness and misuse of CRC32. Supposes you know what does CRC32 means and how it works. If not then you better start with [1] before keep reading this one. I also drop some math to keep this text clear and readable.

The major misuse of CRC32 is confusing of its functional usage. There are lots of modern files authentication and validation tools using CRC32 as a MDC (Modification Detection Code) function. This is totally wrong because CRC32 is an error detection function by itself. CRC32 is completely helpless against non-accidental tamper because it easily can be faked in just 4 bytes. You may read all the necessary details in [2] - one of the best documents related to this problem. I would not pay attention on it here, just simple demonstrate it by example.

Take a look at two GIF files (you can download them as a single zip archive from http://www.nattyware.com/bin/crc32fake.zip):

Picture 1
File 1
Picture 2
File 2

As you can see these files are different. However both have same size and same CRC32. Take a Pkzip archiver for example and check this yourself. You'll see the following results:
PKZIP(R)  Version 2.50  FAST! Compression Utility for Windows 95/NT 4-15-1998
Copyright 1989-1998 PKWARE Inc.  All Rights Reserved. Shareware Version
PKZIP Reg. U.S. Pat. and Tm. Off.  Patent No. 5,051,745

 Viewing .ZIP: crc32fake.zip

  Length  Method    Size  Ratio    Date    Time    CRC-32   Attr    Name
  ------  ------    ----- -----    ----    ----   --------  ----    ----
    1428 DeflatN      734 48.6% 07.16.2001  2:21p 4a3c7d29 --w----  p001.gif
    1428 DeflatN     1098 23.2% 07.16.2001  2:20p 4a3c7d29 --w----  p002.gif
  ------           ------ -----                                     ----
    2856             1832 35.9%                                        2
This is not a bug in PkZip - you'll see the same result by using any other CRC32 calculation utility. It is a CRC32 feature. Bad feature. By using it you could easily be cheated by a hacker or a virus could affect your files.

Good news: the solution for reliable files authentication is simple. There is a checksum calculation method I call it XUM32. It's a combination of two 32-bits hash functions CRC32 and ELF with a file size confusion. To whom, who want know more details about XUM32 and how it works, I would like recommend to read [3]. XUM32 produces a 32-bits checksum value you can use instead of CRC32. Please note XUM32 value is not a unique file fingerprint (it's mostly impossible in 32 bits) but a control checksum allows detect any modification of file. For example, a result of XUM32 check for files listed above is:
XUM32 Version 2.0 GUI Edition for Win32/80x86
Copyright (c) Nattyware, 2000-2001 All rights reserved.
Please visit http://www.nattyware.com and buy this copy to keep use it

Param: C:\Documents\CRCFake\*.gif

File name                                             xum32     crc32
--------                                            --------  --------
p001.gif                                            4FA87DB0  4A3C7D29
p002.gif                                            4FA87C5B  4A3C7D29
--------                                            --------  --------
Total 2 file(s)                                     59A5343B  472D1578
As you can see, XUM32 values are different and this check avoids a lack of CRC32 one.

So if you need a truly reliable files authentication then look no further - XUM32 suits all your needs. Please visit http://www.nattyware.com and download an efficient and convenient utility named XUM32. With XUM32 utility, you can easily detect whether your files were tampered with or not. This is worth to do to keep your system safe.


References:
[1] "A painless guide to CRC error detection algorithm." Ross N. Williams
[2] "CRC and how to reverse it." Anarchriz/DREAD
[3] "XUM32 and its implementation." Ilya O. Levin
  Home
Products:
     Pixie
     WinJanitor
     XUM32
Support
Contacts

 Copyright © Nattyware, 2000-2004  All rights reserved.