Re: removing old ports and architectures

On Wed, Oct 16, 2013 at 3:26 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
>> I don't agree with that policy. Sure, 97% of our users are probably
>> running Linux, Windows, MacOS X, or one of the fairly-popular BSD
>> variants. But I think a part of the appeal of PostgreSQL is that it is
>> cross-platform, and doesn't require a lot of special hardware support,
>> and I'm loathe to give that up. It's one thing to say, gee, we don't
>> know whether this is actually going to compile and work on your
>> platform, because it's not tested. It's quite something else to say,
>> anything that's not on this short list of supported platforms has zero
>> chance of working without jumping through major hoops.
>
> Well, I am not advocating to break platforms just because they are not
> on the buildfarm, but I don't see how we can introduce new facilities
> that require platform support if we don't have any way to test them on
> some platforms.

Well, on that we agree. But my answer to that is - any facilities
that require additional platform support had better be optional. If
we can't do it without raising the bar for platform support, then we
don't do it.

> I think by using architecture independent, compiler provided intrinsics
> we take care of that for most non-mainstream platforms. Just about
> everything even remotely new that is/will be out there is/will be using gcc
> or something compatible to it (e.g. llvm). Those provide the __sync_*
> intrinsics we'd wrap.
> That's incidentally why I am advocating including pg_atomic_clear and
> pg_atomic_test_set in the set of supported atomics. With those porting
> to a new platform will often require exactly zero changes since
> spinlocks will just use them.

I grant that helps a great deal. The fact that gcc has added those
atomics makes this a lot more feasible to contemplate that it would be
otherwise. Now, given some of our other experiences with gcc, I am
slightly worried that there may be bugs. But if that turns out to be
the case we can figure out what to do about it. It's very nice to at
least have a place to start.

On the other hand, I'm not convinced that we don't need to give any
thought to UNIX vendors that are still pushing their proprietary
compilers. Many of the old players are dead, but IBM's ICC and HP's
aCC definitely aren't, and I wouldn't be surprised to find one or two
other big ones as well, plus maybe half-a-dozen others that are
clinging to life.

> Which? We only seem to disagree about M32R and m68k, right? I've
> recanted mips and superh days ago ;).
> If you find alpha important, that's fine, that's supported by gcc. I
> just doubt we'll get it even remotely right without tests...

As far as spinlock support goes, you proposed removing VAX, univel,
sinix, sun3, natsemi 32k, superH, ALPHA, m86k, M32R, mips non-GCC,
s390 non-GCC, and 32bit/<v9 sparc. I agreed unequivocally with 4 of
those (though I don't see the code you're talking about wrt/32bit<v9
sparc), and you withdrew 2 of those suggestions, so I think there are
6 that are still in doubt: vax, univel, ALPHA, m32r, m68k, and s390
non-GCC.

>> It's hard to say where to draw the line here. I don't want the
>> illusion of support for platforms that don't in fact have a prayer of
>> working to prevent us from making needed improvements. On the other
>> hand, I also don't want to blithely rip out support for architectures
>> that people may well still be using and where, in some cases, people
>> have gone out of their way to add that support. If we get to the
>> point where some relatively-obscure architecture is the only thing
>> standing between us and improvement X, then we can weigh those things
>> against each other and decide. But I don't really want to go rip out
>> support for half-a-dozen semi-supported architectures without some
>> clear goal in mind. That just doesn't seem friendly.
>
> Well, I only started to look at this somewhat seriously because more and
> more people in the last year or so, both onlist and towards 2ndq were
> complaining about massive spinlock contention (up to 97% spent in
> s_lock) on somewhat bigger machines. The primary offender in many
> workloads is the lwlock internal spinlock, quite often for LW_SHARED
> acquisition where we shouldn't need to block.
>
> That triggered developing the wait-free LW_SHARED patch
> http://www.postgresql.org/message-id/20130926225545.GB26663@awork2.anarazel.de
> which indeed shows quite some promise. Unfortunately it pretty
> fundamentally requires compare_swap and fetch_add. Now we could just
> implement lwlocks in two pretty much independent ways, but that seems to
> be pretty bad from a maintainability POV.
>
> The next big thing after that is getting rid of spinlocks around buffer
> pinning (and by that in buffer hdr locking). That would end up in quite
> some #ifdef's sprinkled around.

I do understand that it's going to be painful to carry multiple
implementations of some of these facilities. But at least from where
I sit I'm not sure we really have a choice. If we have a
--disable-atomics option, there's no reason we can't have regular
buildfarm coverage of that code; broken lwlocks tend to make things
die pretty horribly, so I'm relatively confident bugs will show up.

Obviously, #ifdefs scattered all over the backend is not going to be
good. We'll need to find ways to isolate the code that does
platform-specific things, but we have been relatively successful at
doing that in the past and I see no particular reason to be
pessimistic about the future.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company