Re: DDL Damage Assessment

On Thu, Oct 2, 2014 at 6:19 PM, Stephen Frost <sfrost(at)snowman(dot)net> wrote:
> * Claudio Freire (klaussfreire(at)gmail(dot)com) wrote:
>> On Thu, Oct 2, 2014 at 6:03 PM, Stephen Frost <sfrost(at)snowman(dot)net> wrote:
>> >> That sounds extremely complex. You'd have to implement the fake
>> >> columns, foreign keys, indexes, etc on most execution nodes, the
>> >> planner, and even system views.
>> >
>> > Eh? We have MVCC catalog access.
>>
>> And that needs locks, especially if you modify the underlying filesystem layout.
>
> And we wouldn't be doing that, certainly. It's a dry-run.

...

> (...) We might also be able to simply get away with
> short-circuiting them and not actually writing anything (and the same
> for reading data..). What would probably be useful is to review actual
> migration scripts and see if this would really work. I know they'd work
> for at least a subset of the migration scripts that I've dealt with
> before, and also not all of them.

I believe, for it to be reasonably intrusive, you'd have to abort at
the first need to actually read/write data.

Most of my migration scripts, especially the ones that would benefit
from this, require some of that, but that's just my personal common
practice, not a general case.

>> >> IMO, dry-run per se, is a BEGIN; stuff; ROLLBACK. But that still needs
>> >> locks. I don't think you can simulate the side effects without locks,
>> >
>> > Why? If you know the transaction is going to roll back and you only add
>> > entries to the catalog which aren't visible to any other transactions
>> > than your own, and you make sure that nothing you do actually writes
>> > data out which is visible to other transactions..
>>
>> But that's not the scope. If you want a dry-run of table-rewriting
>> DDL, or DDL interspersed with DML like:
>>
>> alter table blargh add foo integer;
>> update blargh set foo = coalesce(bar, baz);
>>
>> You really cannot hope not to have to write data. The above is also
>> the case with defaulted columns btw.
>
> The point is to not write anything which is visible to other
> transactions, which means we'd have to put DML into some different
> 'mode' which doesn't actually write where other processes might be
> looking. I'm not saying it's trivial to do, but I don't think it's
> impossible either. (...)

No, I don't think it's impossible either. Just very, very
time-consuming. Both in developing the patch and in its eventual
maintenance.

TBH, a separate read-only transaction like explain alter would also be
quite difficult to keep in sync with actual alter logic, unless it's
handled by the same code (unlikely in that form).

>> >> so getting the local view of changes will be extremely difficult
>> >> unless you limit the scope considerably.
>> >
>> > I agree that there may be complexities, but I'm not sure this is really
>> > the issue..
>>
>> In essence, if you want MVCC catalog access without AEL, you're in for
>> a rough ride. I'm not as experienced with pg's core as you, so you
>> tell me, but I imagine it will be the case.
>
> It's not clear to me what you're getting at as the 'rough' part,
> exactly..

A lot of work, touching most of the codebase, and hard to maintain in the end.

Unless you limit the scope, as you said, just touching the catalog,
not data, could be doable. But it would act as an implicit
"norewrite".