Re: AW: posgres 12 bug (partitioned table)

From: Pavel Biryukov <79166341370(at)yandex(dot)ru>
To: Wilm Hoyer <w(dot)hoyer(at)dental-vision(dot)de>, Andres Freund <andres(at)anarazel(dot)de>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Robert Haas <robertmhaas(at)gmail(dot)com>, PostgreSQL mailing lists <pgsql-bugs(at)lists(dot)postgresql(dot)org>
Subject: Re: AW: posgres 12 bug (partitioned table)
Date: 2020-08-12 10:10:52
Message-ID: 95271597226022@mail.yandex.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs pgsql-hackers

<div>Wilm,</div><div> </div><div>Have a look, there is even an extension method for xmin (stable branch):</div><div> </div><div><a href="https://github.com/npgsql/efcore.pg/blob/stable/src/EFCore.PG/Extensions/NpgsqlEntityTypeBuilderExtensions.cs">https://github.com/npgsql/efcore.pg/blob/stable/src/EFCore.PG/Extensions/NpgsqlEntityTypeBuilderExtensions.cs</a></div><div> </div><div>EF Core is mostly independent from DB, but there usually are some "tweaks" for each database that you should know (like for SQL server the native concurrency token should be like byte[]:<ol style="box-sizing:border-box;color:rgb( 51 , 51 , 51 );font-family:'menlo' , 'monaco' , 'consolas' , 'courier new' , monospace;font-size:13px;font-style:normal;font-weight:400;margin-bottom:0px;margin-top:0px;padding-left:0px;text-decoration-style:initial;text-indent:0px;text-transform:none;white-space:pre;word-spacing:0px"><li style="box-sizing:border-box;list-style-type:none;margin:0px"><code style="background:rgb( 249 , 249 , 249 );border-radius:0px;box-sizing:border-box;color:inherit;font-family:'menlo' , 'monaco' , 'consolas' , 'courier new' , monospace;font-size:inherit;padding:0px;white-space:pre-wrap"><span style="box-sizing:border-box;color:#666600">[</span><span style="box-sizing:border-box;color:#555555">TimeStamp</span><span style="box-sizing:border-box;color:#666600">]</span></code></li><li style="background:rgb( 249 , 249 , 249 );box-sizing:border-box;list-style-type:none;margin:0px"><code style="background:rgb( 249 , 249 , 249 );border-radius:0px;box-sizing:border-box;color:inherit;font-family:'menlo' , 'monaco' , 'consolas' , 'courier new' , monospace;font-size:inherit;padding:0px;white-space:pre-wrap"><span style="box-sizing:border-box;color:#0000ff">public</span><span style="box-sizing:border-box;color:#000000"> </span><span style="box-sizing:border-box;color:#0000ff">byte</span><span style="box-sizing:border-box;color:#666600">[]</span><span style="box-sizing:border-box;color:#000000"> </span><span style="box-sizing:border-box;color:#555555">RowVersion</span><span style="box-sizing:border-box;color:#000000"> </span><span style="box-sizing:border-box;color:#666600">{<!-- --></span><span style="box-sizing:border-box;color:#000000"> </span><span style="box-sizing:border-box;color:#0000ff">get</span><span style="box-sizing:border-box;color:#666600">;</span><span style="box-sizing:border-box;color:#000000"> </span><span style="box-sizing:border-box;color:#0000ff">set</span><span style="box-sizing:border-box;color:#666600">;</span><span style="box-sizing:border-box;color:#000000"> </span><span style="box-sizing:border-box;color:#666600">}</span></code></li><li style="background:rgb( 249 , 249 , 249 );box-sizing:border-box;list-style-type:none;margin:0px"><code style="background:rgb( 249 , 249 , 249 );border-radius:0px;box-sizing:border-box;color:inherit;font-family:'menlo' , 'monaco' , 'consolas' , 'courier new' , monospace;font-size:inherit;padding:0px;white-space:pre-wrap"><span style="box-sizing:border-box;color:#666600">)</span></code></li></ol></div><div> </div><div>Additional "self defined column" for billion rows tables leads to additional space needed. We use partitioning when the tables are LARGE :)</div><div> </div><div>This works fine in 10, 11, it's strange it is broken in 12 (from db users point of view, I haven't examined the sources of PG for internals...)</div><div> </div><div> </div><div>-- <br />С уважением, Павел</div><div> </div><div> </div><div> </div><div>12.08.2020, 12:35, "Wilm Hoyer" &lt;w(dot)hoyer(at)dental-vision(dot)de&gt;:</div><blockquote><div lang="DE"><div><blockquote style="margin-bottom:5pt;margin-top:5pt"><p>Hi,</p><p>while the request for returning xmin of partitioned tables is still valid, i’d like to add some information and a possible workaround.</p><p><br />On 2020-08-11 21:31:52 +0300, Pavel Biryukov wrote:</p><blockquote style="margin-bottom:5pt;margin-top:5pt"><p style="font-family:'calibri' , sans-serif;font-size:11pt;margin:0cm"> Entity Framework is an ORM for .Net (and .Net Core). It has different providers<br /> for different databases (NpgSql for Postgres). It uses Optimistic concurrency.<br /> The common use case is to use xmin as "concurrency token".<br />  <br /> In code we make "var e = new Entity();", "dbContext.Add(e)" and<br /> "dbContext.SaveChanges()" (smth like that), and EF Core constructs sql for us,<br /> classical ORM;<br />  <br /> When new row is inserted, EF makes an insert with "RETURNING xmin" to keep it<br /> as concurrency token for further updates (update is made like "where id = [id]<br /> AND xmin=[xmin]" to be sure the row hasn't been updated by other clients).</p><p style="font-family:'calibri' , sans-serif;font-size:11pt;margin:0cm"> </p><p style="font-family:'calibri' , sans-serif;font-size:11pt;margin:0cm">Neither the Entity Framework, nor npgsql rely on the column xmin. Both don’t know about this column in their codebase.</p><p style="font-family:'calibri' , sans-serif;font-size:11pt;margin:0cm">In the case oft he EF i’m sure that this holds true for all versions, since it is designed as DBMS independant, and as such will never know anything about a PostgreSQL specific column.</p><p style="font-family:'calibri' , sans-serif;font-size:11pt;margin:0cm">Also you can use any ADO.Net provider to connect to a concrete DBMS – i for example use dotConnect for PostgreSQL because it provided more features and less bugs as Npgsql at the time of  decission.</p><p style="font-family:'calibri' , sans-serif;font-size:11pt;margin:0cm">As for Npgsql i have only checked that the current HEAD has no reference to xmin in its source code.</p><p style="font-family:'calibri' , sans-serif;font-size:11pt;margin:0cm"> </p><p style="font-family:'calibri' , sans-serif;font-size:11pt;margin:0cm">With that in mind, i assume the OP included the column xmin in his Entity Model by himself and set the ConcurrencyMode to fixed for that column.</p><p style="font-family:'calibri' , sans-serif;font-size:11pt;margin:0cm">As xmin is a system column that the EF should never try to update (PostgreSQL will reject this attempt, i think), i’d suggest using a self defined column (row_version for example) and either use triggers on update and insert to increment its value (works even with updates outside of EF) or let the EF do the increment.</p><p style="font-family:'calibri' , sans-serif;font-size:11pt;margin:0cm"> </p><p style="font-family:'calibri' , sans-serif;font-size:11pt;margin:0cm">Regards</p><p style="font-family:'calibri' , sans-serif;font-size:11pt;margin:0cm">Wilm Hoyer.</p><p style="font-family:'calibri' , sans-serif;font-size:11pt;margin:0cm"> </p><p style="font-family:'calibri' , sans-serif;font-size:11pt;margin:0cm"> </p><p style="font-family:'calibri' , sans-serif;font-size:11pt;margin:0cm"> </p></blockquote></blockquote></div></div></blockquote>

Attachment Content-Type Size
unknown_filename text/html 6.8 KB

In response to

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message Wilm Hoyer 2020-08-12 10:57:49 AW: AW: posgres 12 bug (partitioned table)
Previous Message Wilm Hoyer 2020-08-12 09:35:04 AW: posgres 12 bug (partitioned table)

Browse pgsql-hackers by date

  From Date Subject
Next Message Wilm Hoyer 2020-08-12 10:57:49 AW: AW: posgres 12 bug (partitioned table)
Previous Message Christoph Berg 2020-08-12 10:04:27 Re: Make contrib modules' installation scripts more secure.