Re: pg_background (and more parallelism infrastructure patches)

Robert,

* Robert Haas (robertmhaas(at)gmail(dot)com) wrote:
> On Wed, Oct 1, 2014 at 4:56 PM, Stephen Frost <sfrost(at)snowman(dot)net> wrote:
> > Was just thinking that we might be able to work out what needs to be
> > done without having to actually do it on a per-character basis. That
> > said, I'm not sure it's really worth the effort given that we're talking
> > about at most 8 bytes currently. I had originally been thinking that we
> > might increase the minimum size as it might make things more efficient,
> > but it's not clear that'd really end up being the case either and,
> > regardless, it's probably not worth worrying about at this point.
>
> I'm still not entirely sure we're on the same page. Most of the data
> movement for shm_mq is done via memcpy(), which I think is about as
> efficient as it gets.

Right- agreed. I had originally thought we were doing things on a
per-MAXIMUM_ALIGNOF-basis somewhere else, but that appears to be an
incorrect assumption (which I'm glad for).

> The detailed character-by-character handling
> only really comes up when shm_mq_send() is passed multiple chunks with
> lengths that are not a multiple of MAXIMUM_ALIGNOF. Then we have to
> fiddle a bit more. So making MAXIMUM_ALIGNOF bigger would actually
> cause us to do more fiddling, not less.

Sorry- those were two independent items. Regarding the per-character
work, I was thinking we could work out the number of characters which
need to be moved and then use memcpy directly rather than doing the
per-character work, but as noted, we shouldn't be going through that
loop very often or for very many iterations anyway, and we have to deal
with moving forward through the iovs, so we'd still have to have a loop
there regardless.

> When I originally designed this queue, I had the idea in mind that
> life would be simpler if the queue head and tail pointers always
> advanced in multiples of MAXIMUM_ALIGNOF. That didn't really work out
> as well as I'd hoped; maybe I would have been better off having the
> queue pack everything in tightly and ignore alignment. However, there
> is some possible efficiency advantage of the present system: when a
> message fits in the queue without wrapping, shm_mq_receive() returns a
> pointer to the message, and the caller can assume that message is
> properly aligned. If the queue didn't maintain alignment internally,
> you'd need to do a copy there before accessing anything inside the
> message that might be aligned. Granted, we don't have any code that
> takes advantage of that yet, at least not in core, but the potential
> is there. Still, I may have made the wrong call. But, I don't think
> it's the of this patch to rework the whole framework; we can do that
> in the future after this has a few more users and the pros and cons of
> various approaches are (hopefully) more clear. It's not a complex
> API, so substituting a different implementation later on probably
> wouldn't be too hard.

Agreed.

> Anyway, it sounds like you're on board with me committing the first
> patch of the series, so I'll do that next week absent objections.

Works for me.

Thanks!

Stephen