Re: Postgres XA support

Ludovic Orban wrote:
>> >> From the comments I saw in the source, transaction interleaving, join
>> >> and suspend/resume are still not supported and forget is still
>> >> unimplemented. This means you cannot mix local and global
>> >> transactions, cannot support EJBs with REQUIRES_NEW CMT declaration
>> >> and can get into troubles during crash recovery.
>>
>> Can you clarify how that can get you into trouble during crash recovery?
>
> Simply because XAResource.forget() is not implemented. You won't
> exactly be into trouble because of this but extra manual work might be
> needed to check that an after-crash recovery succeeded.

Right, this comes down to the definition of heuristic commit/abort. The
way I read the specs, a heuristic commit/rollback means that the
resource adapter has independently decided to commit or rollback a
transaction. A typical implementation would be to roll back prepared
transactions after a timeout if the connection to the TM is lost.

The way you read the specs is that when the administrator manually
commits or aborts a transaction, that's also a heuristic commit/abort.

The JTA spec doesn't go into details, but the XA spec (Section 2.3.3)
says: "Some RMs may employ heuristic decision-making: an RM that has
prepared to commit a transaction branch may decide to commit or roll
back its work independently of the TM."

Section 7.3 explicitly lists "Heuristics" as an optional feature.

I don't think the XA spec supports the view that manual commit or
rollback is considered a heuristic decision. The administrator better
not mess with in-doubt transactions. If she does, the safe way to do it
is to first take the TM and all resource managers offline, manually
finish all the transactions in the TM, and then manually finish all the
transactions in the RMs.

BTW: If we wanted to report manual commits/aborts as heuristic
decisions, that would be possible with some minor changes to the
backend. We would need to add an option to the COMMIT/ROLLBACK PREPARED
commands saying it's a heuristic decision, and have those commands
insert a row to a table. forget would then delete the row. But I don't
think we should do that.

> You will have the same doubts when reconciliating the TM logs with the
> DB logs after a heuristic decision happened.

Heuristic decisions don't happen, because PostgreSQL doesn't do them.
(per my interpretation of the spec)

> Opening a new connection as you describe would probably work. It is
> then the responsibility of the EJB container to implement REQUIRES_NEW
> in this way instead of using the TM's suspend service but I don't know
> of any container that implements REQUIRES_NEW without calling suspend
> on the TM.

Oh, the container can call TM's suspend service. But that doesn't mean
that the TM has to call suspend on the resource managers. In fact, if
you have a non-distributed transaction, with a non-XA enabled JDBC
driver, there isn't a suspend-method to call.

To turn the question around: Do you know of a container/app.server that
*does* require suspend/resume support from the XADataSource
implementation to support REQUIRES_NEW?

>> I don't see much value in implementing the full XA spec. What we have
>> now is enough to implement distributed transactions reliably, and that's
>> what XA is all about.
>
> I disagree. There are some parts of the XA spec that might well be
> considered as optional but suspend/resume, ability to mix local/global
> transactions and recovery can't be. Yes you can run distributed
> transactions reliably but with so many restrictions that it's of no
> use.

Do you have a use case in mind that's not currently possible? I'd like
to hear about it so we can try to make it work.

Recovery definitely isn't optional, and it's required for reliable
operation. Suspend/resume is a required part of the spec, but it isn't
required for reliable operation and it's not required in practice
because app. servers can work around it.

What do you mean by mixing local/global transactions?

> Please don't get me wrong here. I'm not bashing Postgres nor your work
> in any way I'm just saying that it's still lacking features to make it
> really usable with XA transactions. Mysql and Ingres fall in exactly
> the same category: they provide incomplete (even worse, sometimes
> incorrect) XA support as well.

I believe there's three reasons most RM implementations don't implement
suspend/resume or transaction interleaving:

1. They don't let you to do anything you can't do without
2. It would be a very invasive change to add them to the underlaying
architecture and wire protocol of an existing DBMS
3. Because of 1, and the lack of support in other DBMS's, app servers
don't use them, and provide ways to work around them, making the
features optional, de facto.

An incorrect implementation is indeed much worse than an incomplete
implementation. Some DBMS vendors claim XA support even with incomplete
or broken implementations. We've tried to be very up-front of the
limitations of the PostgreSQL driver.

No offense taken :). It's important to discuss these things if there's
any doubts.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com