Re: Acclerating INSERT/UPDATE using UPS

Joshua,

Thanks for your comments !

Right. As you pointed out, Sigres cannot recover if UPS fails.
Therefore, I think currently Sigres cannot be used for mission critical
applications.
Sigres keeps data integrity *only when* UPS works.

On the other hand, some users such as researchers of sensor networks
would like store
and analyze 10^6 Hz sensor data insertions in real-time, and obviously
research usages are not mission critical. Thus Sigres may be accepted
for researchers, but I have no prospect now since I have just started to
distributing Sigres for research institutes in Japan.

BTW, Joshua, could you please let me know or give me any pointers for
the reason why fsync=off option exists on PostgreSQL although PostgreSQL
developers does not allow sacrificing data integrity ?
If the reason is famous and clear in the community, I am sorry for
bothering you.

-- Hideyuki

Joshua D. Drake wrote:
> Hideyuki Kawashima wrote:
>
>> Hello PostgreSQL Hackers,
>>
>> I have made a modification of PostgreSQL which accelerates INSERT/UPDATE using UPS. The name of the software is "Sigres", and the philosophy is considering a battery supplied memory as a persistent device instead of a disk. You can download Sigres from http://sourceforge.jp/projects/sigres/ .
>>
>> In the maximum case, Sigres is 7 times faster than PostgreSQL default (fsync=on) in my environment (CoreDuo 2.66GHz, UDMA/133), and it is also 10% faster than PostgreSQL without fsync (fsync=off).
>>
>
> Interesting and what happens when the UPS fails? My main concern is that
> one of the purposes of PostgreSQL is data integrity. I am all for every
> performance enhancement we can achieve, that does *not* sacrifice that.
>
> Sincerely,
>
> Joshua D. Drake
>
>
>> The magic lies in usually skipping XLogWrite() and ignoring WALWriteLock. The exceptions are XLogWrite() calls from AdvanceXLInsertBuffer(). In addition, in XLogFileClose() issue_xlog_fsync() before close(). (In this point, Sigres is different from just simply setting fsync=off.)
>>
>> Although I think Sigres can be considered as one of the future directions of PostgreSQL, I do not know whether this software can be accepted or not. Could you please give me some comments ?
>>
>> Best Regards,
>>
>> -- Hideyuki Kawashima
>> Assistant Professor, University of Tsukuba
>>
>>
>>
>> ---------------------------(end of broadcast)---------------------------
>> TIP 1: if posting/reading through Usenet, please send an appropriate
>> subscribe-nomail command to majordomo(at)postgresql(dot)org so that your
>> message can get through to the mailing list cleanly
>>
>>
>
>
>