Re: Indexes for hashes

From: julyanto SUTANDANG <julyanto(at)equnix(dot)co(dot)id>
To: Ivan Voras <ivoras(at)gmail(dot)com>
Cc: Claudio Freire <klaussfreire(at)gmail(dot)com>, postgres performance list <pgsql-performance(at)postgresql(dot)org>
Subject: Re: Indexes for hashes
Date: 2016-06-17 08:48:56
Message-ID: CAGu3fERehXvsV25zNWx3durYw4qkJQgTa-qg9gffMxhChQKVpg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

Crc32 is great because it is supported by Intel Hardware, unfortunatelly
you have to code something like this:

http://stackoverflow.com/questions/31184201/how-to-implement-crc32-taking-advantage-of-intel-specific-instructions

int32_t sse42_crc32(const uint8_t *bytes, size_t len){
uint32_t hash = 0;
size_t i = 0;
for (i=0;i<len;i++) {
hash = _mm_crc32_u8(hash, bytes[i]);
}

return hash;}

It is supported by GCC and will implemented as hardware computing which
really fast.
you can use 64bit integer to have more precise hashing, so don't worry
about uniformity.

Btw: crc32 is not part of the cryptography, it is part of hashing or
signature.

Regards,

On Fri, Jun 17, 2016 at 3:32 PM, Ivan Voras <ivoras(at)gmail(dot)com> wrote:

> And in any case, there's no crc32 in the built-in pgcrypto module.
>
>
> On 17 June 2016 at 06:18, Claudio Freire <klaussfreire(at)gmail(dot)com> wrote:
>
>> On Fri, Jun 17, 2016 at 1:09 AM, julyanto SUTANDANG
>> <julyanto(at)equnix(dot)co(dot)id> wrote:
>> > This way is doing faster using crc32(data) than hashtext since crc32 is
>> > hardware accelerated in intel (and others perhaps)
>> > this way (crc32) is no way the same as hash, much way faster than
>> others...
>> >
>> > Regards,
>>
>> Sure, but I've had uniformity issues with crc32.
>>
>
>

--

Julyanto SUTANDANG

Equnix Business Solutions, PT
(An Open Source an Open Mind Company)

Pusat Niaga ITC Roxy Mas Blok C2/42. Jl. KH Hasyim Ashari 125, Jakarta
Pusat
T: +6221 22866662 F: +62216315281 M: +628164858028

Caution: The information enclosed in this email (and any attachments) may
be legally privileged and/or confidential and is intended only for the use
of the addressee(s). No addressee should forward, print, copy, or otherwise
reproduce this message in any manner that would allow it to be viewed by
any individual not originally listed as a recipient. If the reader of this
message is not the intended recipient, you are hereby notified that any
unauthorized disclosure, dissemination, distribution, copying or the taking
of any action in reliance on the information herein is strictly prohibited.
If you have received this communication in error, please immediately notify
the sender and delete this message.Unless it is made by the authorized
person, any views expressed in this message are those of the individual
sender and may not necessarily reflect the views of PT Equnix Business
Solutions.

In response to

Browse pgsql-performance by date

  From Date Subject
Next Message Adam Brusselback 2016-06-17 15:18:19 Re: 9.6 query slower than 9.5.3
Previous Message Ivan Voras 2016-06-17 08:32:09 Re: Indexes for hashes