Re: Enabling Checksums

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Greg Smith <greg(at)2ndQuadrant(dot)com>
Cc: Ants Aasma <ants(at)cybertec(dot)at>, Heikki Linnakangas <hlinnakangas(at)vmware(dot)com>, Simon Riggs <simon(at)2ndQuadrant(dot)com>, Jeff Davis <pgsql(at)j-davis(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Enabling Checksums
Date: 2013-03-20 02:05:26
Message-ID: 17821.1363745126@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Greg Smith <greg(at)2ndQuadrant(dot)com> writes:
> While being a lazy researcher today instead of writing code, I
> discovered that the PNG file format includes a CRC-32 on its data
> chunks, and to support that there's a CRC32 function inside of zlib:
> http://www.zlib.net/zlib_tech.html

Hah, old sins keep coming back to haunt one ;-)

Keep in mind that PNG was designed in 1995, and that any speed
considerations in that spec were decided in the context of whether it
would take noticeably longer to view an image downloaded over analog
dialup. That design context also informed a greater interest in error
checking than has been seen in any other image file format before (or
since, I believe).

> And they've already put some work into optimizing its table-driven
> implementation. Seems possible to punt the whole problem of how to do
> this efficiently toward the zlib developers, let them drop into assembly
> to get the best possible Intel acceleration etc. one day.

I would not hold my breath waiting for any such work from either the
zlib or libpng developers; both of those projects are basically in
maintenance mode AFAIK. If we want hardware acceleration we're going
to have to deal with the portability issues ourselves.

FWIW, I would argue that any tradeoffs we make in this area must be made
on the assumption of no such acceleration. If we can later make things
better for Intel(TM) users, that's cool, but let's not screw those using
other CPUs.

regards, tom lane

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2013-03-20 02:51:05 Ignore invalid indexes in pg_dump
Previous Message Daniel Farina 2013-03-20 01:10:06 Re: postgres_fdw vs data formatting GUCs (was Re: [v9.3] writable foreign tables)