Re: .ready and .done files considered harmful

On 8/24/21, 12:09 PM, "Robert Haas" <robertmhaas(at)gmail(dot)com> wrote:
> I can't quite decide whether the problems we're worrying about here
> are real issues or just kind of hypothetical. I mean, today, it seems
> to be possible that we fail to mark some file ready for archiving,
> emit a log message, and then a huge amount of time could go by before
> we try again to mark it ready for archiving. Are the problems we're
> talking about here objectively worse than that, or just different? Is
> it a problem in practice, or just in theory?

If a .ready file is created out of order, the directory scan logic
will pick it up about as soon as possible based on its priority. If
the archiver is keeping up relatively well, there's a good chance such
a file will have the highest archival priority and will be picked up
the next time the archiver looks for a file to archive. With the
patch proposed in this thread, an out-of-order .ready file has no such
guarantee. As long as the archiver never has to fall back to a
directory scan, it won't be archived. The proposed patch handles the
case where RemoveOldXlogFiles() creates missing .ready files by
forcing a directory scan, but I'm not sure this is enough. I think we
have to check the archiver state each time we create a .ready file to
see whether we're creating one out-of-order.

While this may be an extremely rare problem in practice, archiving
something after the next checkpoint completes seems better than never
archiving it at all. IMO this isn't an area where there is much space
to take risks.

> I really want to avoid getting backed into a corner where we decide
> that the status quo is the best we can do, because I'm pretty sure
> that has to be the wrong conclusion. If we think that
> get-a-bunch-of-files-per-readdir approach is better than the
> keep-trying-the-next-file approach, I mean that's OK with me; I just
> want to do something about this. I am not sure whether or not that's
> the right course of action.

I certainly think we can do better. The get-a-bunch-of-files-per-
readdir approach can help us cut down on the directory scans by one or
two orders of magnitude, which is still a huge win. Plus, such an
approach retains much of the resilience of the current implementation
(although there may be bit more delay for the special cases).

That being said, I still think the keep-trying-the-next-file approach
is worth exploring, but I think it's really important to consider that
there is no guarantee that a directory scan will happen anytime soon.

Nathan