Re: backup postgresql with snapshot AWS

On 1/19/2018 10:23, Achilleas Mantzios wrote:
> On 19/01/2018 17:40, Rui DeSousa wrote:
>
>> If the snapshot is not atomic and/or using multiple filesystems; that
>> is fine as long as it’s issued and completed within the
>> pg_start_backup() and pg_stop_backup() calls.
> And as long as the needed wals belonging to that interval between
> start and stop are to be found in the archive when the problem arises.
> So the whole system must be well documented and periodically tested.
>>
The latter is the problem (e.g. when it says "complete" it really is
done and it really is safe to issue the pg_stop_backup() and grab the
WAL) and further you are relying on an assumption of atomicity within
the I/O subsystem when it is requested by the application within your VM
(in which it's reasonable to believe that when fsync() returns it really
HAS committed the data from the perspective of said VM) yet you're doing
some other operation *behind* the VM on which your application is running.

IMHO the more you intertwine things you cannot be certain of because
they operate outside of your particular VM's space with a process that
relies on them the more you are risking finding out that your
assumptions are wrong the hard way.  You *shouldn't* get bit by this in
a well-designed cloud or cluster system but that's not an acceptable
word to describe the expected results when it's important.

--
Karl Denninger
karl(at)denninger(dot)net <mailto:karl(at)denninger(dot)net>
/The Market Ticker/
/[S/MIME encrypted email preferred]/