Re: Documentation improvements for partitioning

On Fri, Feb 24, 2017 at 9:53 AM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> The good news is that logical replication DOES work with partitioning,
> but only for a Publication with PUBLISH INSERT, pushing from a normal
> table to a partitioned one. Which is useful enough for first release.
>
> The work on having UPDATE work between partitions can be used to make
> updates and deletes work in logical replication. That might yet be
> made to work in this release, and I think we should look into it, but
> I think it can be left til next release if we try.

Are you speaking of the case where you want to replicate from an
unpartitioned table to a partitioned table? I would imagine that if
you are replicating between two identical partitioning hierarchies, it
would just work. (If not, that's probably a bug, though I don't know
whose bug.)

>> You don't get to show up more than two
>> months after the feature is committed and start complaining that it
>> doesn't include all the things you want. Which things ought to be
>> included in the initial patch was under active discussion about a year
>> ago this time.
>
> I've provided lots of input across many years, for example my
> explanation of how to do tuple routing is very clearly the basis of
> the design of the current feature. You *knew* I would have feedback
> and if it it hadn't appeared yet you knew it would come.

Not really. I can't always predict who will take an interest in a
patch. I'm not surprised that you have feedback on it, and if that
feedback were phrased as "let's try to do X, Y, and Z in future
releases" I'd have no problem with it. What I'm reacting against is
really the idea that any of this should be done in v10 (and also the
idea, which may be an overly defensive reading of your emails, that my
original commit was somehow not up to the mark).

> The "two months after the feature is committed" is a direct result of
> the lack of docs. Note that within hours of reading the docs I'd given
> feedback. How could I give feedback earlier? Well, I tried.

Quite a few other people managed to give feedback earlier, so it
evidently wasn't totally impossible.

> I've flown
> to Japan to ensure I could talk to Amit in person about this feature.
> Your absence at two consecutive developer meetings hasn't helped
> there. It is you that needs to show up more. I'm sure we both have
> family and work reasons not to attend them.

We used to have one developer meeting a year and I have attended it
every year I was invited. Then it went up to two per year and I kept
attending one of them per year. Now it seems to be three per year and
I plan to keep attending one of them per year, unless one of the
others happens to be scheduled together with an event I'm planning to
attend anyway. As you say, for both family and work reasons, it's
hard to commit to doing multiple such events per year.

> The need for design feedback is exactly why the docs for logical
> replication were published in June, six months before logical
> replication was committed.

Sure, I think that's great, but there's never been an obligation to
write the documentation before the code in the PostgreSQL community.
It's not a bad idea and I'm happy if people do it, but I'm not going
to start refusing to commit the patches of people who don't do it.
Probably 25% of patches have no documentation at all in the first
version and that gets added later once some consensus is reached and
the feature gets closer to commit; I think that's just fine. For
larger features, the documentation often gets expanded after the
initial commit; I think that's also fine. Unlike you, I actually
don't think it's very hard to follow the discussion about the design
of these features in most cases, and it speeds up development if every
change in the proposed design doesn't require an adjustment to both
the documentation and the code. I think the way logical replication
was done was reasonable, and I think the way that partitioning was
done was also reasonable.

> With repect, you are making a few mistakes. The first is to imagine
> that review comments are negative or complaints; with the right
> viewpoint they could easily be seen as helping people to see what has
> been missed, yet you repeatedly see them as personal attacks and throw
> words back. Oh sure, I've done that myself in earlier days.

Sure.

> The second
> is to imagine that discussing things on Hackers in multiple threads,
> spanning many months with long, detailed emails and rapid replies is
> something that anybody could have followed if they wished.

I don't really see how that's a mistake. It might take more time than
someone's willing to put in -- I have that problem too, on some
threads -- but if someone has the time and is willing to spend it,
then they can follow that discussion.

> And the
> third is to imagine I have no right to review; I will watch and see if
> you deploy this same "You don't get to show up.." argument against Tom
> or Noah when they point out holes in late reviews, though we already
> both know you won't. I see you using that yourself, objecting
> frequently against patches large and small if they do not meet your
> exacting standards, yet you have spoken multiple times against my
> right to do that.

I don't think this primarily is about how I react to you vs. how I
react to other people, although anybody will have noticed by now that
you and I don't agree as often as I agree with some other people, and
perhaps I'm being more negative than is deserved. That having been
said, Tom and Noah typically complain about bugs in what got committed
rather than complaining that the scope of the project was all wrong.
I have heard them complain that the scope of the project is all wrong,
but before things get committed, not after. What I hear you doing is
saying that there are features missing that ought to have been there
in the original commit or at least in the same release, and I wouldn't
agree with that argument no matter who made it. Nice to have? Yes.
Required before 10 goes out the door? Nope. It's not like the scope
of this project wasn't extensively discussed long before anything to
committed, and the only things we can reasonably do at this release
cycle are ship it more or less as-is, with some small fixes here and
there, or rip the whole thing out and try again later. I'm pretty
convicted that the first option is better, but obviously people can
disagree about that. What we can't do is insist on a whole bunch of
additional development in the time remaining before feature freeze;
the only way that can happen is if we move the whole release timetable
out.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company