six demon bag

Wind, fire, all that kind of thing!

2017-03-21

Verifying checksums on Windows systems - correctly

Posted here, since the Fedora people apparently can't be bothered to fix their documentation.

In January 2016 I came across this question on StackOverflow, asking about an OutOfMemory error when validating the SHA256 checksum of a Fedora ISO image. The Fedora documentation suggested reading the full file and then calculating the checksum from the bytes:

$sha256.ComputeHash([System.IO.File]::ReadAllBytes("$PWD\$image"))

Why anyone would even want to read an entire ISO image into memory for a checksum calculation is beyond me. The recommended way of doing this is to open the file as a stream and calculate the checksum on that stream:


$stream = (Get-Item $filename).OpenRead()
$hash = $sha256.ComputeHash($stream)
$stream.Close()

I submitted bug report #1303428, which quickly got closed as a duplicate of #1175759. Switching to that bug report I suggested the following PowerShell snippet:

$image = 'Fedora-Server-DVD-x86_64-21.iso'
$checksum_file = 'Fedora-Server-21-x86_64-CHECKSUM'

$sha256 = New-Object Security.Cryptography.SHA256Managed
$stream = (Get-Item "$PWD\$image").OpenRead()
$download_checksum = [BitConverter]::ToString($sha256.ComputeHash($stream)).ToLower() -replace '-'
$stream.Close()

$checksum_data = Get-Content $checksum_file
$expected_checksum = ($checksum_data -match [regex]::Escape($image) -split ' += +')[1].ToLower()

echo "Download Checksum: $download_checksum"
echo "Expected Checksum: $expected_checksum"
if ( $download_checksum -eq $expected_checksum ) {
  echo "Checksum test passed!"
} else {
  echo "Checksum test failed."
}

which resolves all problems of the [IO.File]::ReadAllBytes() approach: potential memory exhaustion on low memory systems, and a 2 GB limit on the ReadAllBytes() method (which Microsoft probably imposed to discourage people from trying to read gigabyte-sized files into memory).

With that said, to fully verify the integrity of a downloaded image you need not only verify the checksum of the image, but also verify that the reference checksum hasn't been tampered with (e.g. by a Man in the Middle during the download). The latter is usually done by verifying the PGP signature of the checksum file. Unfortunately PGP (or GnuPG) isn't commonly installed on Windows computers, and I'm not aware of PowerShell modules or .NET classes providing this functionality, so there's no out-of-the-box solution for verifying PGP signatures. You can use Chocolatey for installing something like Gpg4win, though. Run the following as an admin user:

Set-ExecutionPolicy RemoteSigned -Force
iex ((New-Object Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))

Restart PowerShell (on one system I had to reboot), then run (again as admin)

choco install gpg4win

Import the key for your Fedora release and verify its fingerprint (as a normal user):

$gpg = if ($env:PROCESSOR_ARCHITECTURE -eq 'x86') {
  "${env:ProgramFiles}\GNU\GnuPG\gpg2.exe"
} else {
  "${env:ProgramFiles(x86)}\GNU\GnuPG\gpg2.exe"
}
& $gpg --keyserver pgp.mit.edu --recv-key <key_id>
& $gpg --fingerprint <key_id>

Then verify the signature on the checksum file:

& $gpg --verify $checksum_file

The minimum requirements for both the checksum and the PGP signature verification are:

  • Windows XP SP3 or newer
  • PowerShell v2
  • .NET Framework 2.0

Posted 00:24 [permalink]