Re: Fastest way to clone schema ~1000x

On 2/26/24 01:06, Emiel Mols wrote:
> On Mon, Feb 26, 2024 at 3:50 PM Daniel Gustafsson <daniel(at)yesql(dot)se
> <mailto:daniel(at)yesql(dot)se>> wrote:
>
> There is a measurable overhead in connections, regardless of if they
> are used
> or not.  If you are looking to squeeze out performance then doing
> more over
> already established connections, and reducing max_connections, is a
> good place
> to start.
>
>
> Clear, but with database-per-test (and our backend setup), it would have
> been *great* if we could have switched database on the same connection
> (similar to "USE xxx" in mysql). That would limit the connections to the
> amount of workers, not multiplied by tests.

That is because:

https://dev.mysql.com/doc/refman/8.3/en/glossary.html#glos_schema

"In MySQL, physically, a schema is synonymous with a database. You can
substitute the keyword SCHEMA instead of DATABASE in MySQL SQL syntax,
for example using CREATE SCHEMA instead of CREATE DATABASE. "

>
> Even with a pooler, we're still going to be maintaining 1000s of
> connections from the backend workers to the pooler. I would expect this
> to be rather efficient, but still unnecessary. Also, both
> pgbouncer/pgpool don't seem to support switching database in-connection
> (they could have implemented the aforementioned "USE" statement I
> think!). [Additionally we're using PHP that doesn't seem to have a good
> shared memory pool implementation -- pg_pconnect is pretty buggy].
>
> I'll continue with some more testing. Thanks for now!

--
Adrian Klaver
adrian(dot)klaver(at)aklaver(dot)com