Re: [SPAM] Re: WAL directory size calculation

From: Moreno Andreo <moreno(dot)andreo(at)evolu-s(dot)it>
To: pgsql-general(at)postgresql(dot)org
Subject: Re: [SPAM] Re: WAL directory size calculation
Date: 2016-07-29 09:19:03
Message-ID: fc342107-e37e-52da-7eda-b1fae106a2b7@evolu-s.it
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Il 29/07/2016 10:43, John R Pierce ha
scritto:<br>
</div>
<blockquote
cite="mid:5f27cf5e-666d-501e-f250-b4bae0a3f3a3(at)hogranch(dot)com"
type="cite">
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
<div class="moz-cite-prefix"><br>
</div>
<blockquote
cite="mid:5f43e1cd-9348-64bc-ff0c-1906db671277(at)evolu-s(dot)it"
type="cite">
<blockquote type="cite" style="color: #000000;">
<blockquote type="cite" style="color: #000000;">Aside of this,
I'm having 350 DBs that sum up a bit more than 1 TB, and
plan <br>
to use wal_level=archive because I plan to have a backup
server with barman. <br>
</blockquote>
<b class="moz-txt-star"><span class="moz-txt-tag"></span></b></blockquote>
</blockquote>
<br>
<p>With that many databases with that so many objects</p>
</blockquote>
350 DBs with about 130 tables and a bunch of sequences each, for the
sake of precision.<br>
With extensive use of BLOBs.<br>
<br>
<blockquote
cite="mid:5f27cf5e-666d-501e-f250-b4bae0a3f3a3(at)hogranch(dot)com"
type="cite">
<p>and undoubtable client connections, </p>
</blockquote>
Yes, that's another big problem... we run normally between 500 and
700 concurrent connections... I had to set max_connections=1000, the
whole thing grew up faster than we were prepared for...<br>
<br>
<blockquote
cite="mid:5f27cf5e-666d-501e-f250-b4bae0a3f3a3(at)hogranch(dot)com"
type="cite">
<p>I'd want to spread that across a cluster of smaller servers.</p>
</blockquote>
That will be step 2... while migration is running (and will run for
some months, we have to plan migration with users) I'll test putting
another one or two machines in cluster, make some test cases, and
when ready, databases will be migrated on other machines, too.<br>
I posted a question about this some months ago, and I was told that
one solution would be to set the servers to be master on some
databases and slave on others, so we can have a better load
balancing (instead of having all writes on the sole master, we split
among all masters depending on which database is getting the write
command, especially when having to write BLOBs that can be some
megabytes in size).<br>
I don't know to achieve this, but I will find a way somewhere.<br>
<br>
<blockquote
cite="mid:5f27cf5e-666d-501e-f250-b4bae0a3f3a3(at)hogranch(dot)com"
type="cite">
<p>just sayin...</p>
</blockquote>
ideas are always precious and welcome.<br>
<blockquote
cite="mid:5f27cf5e-666d-501e-f250-b4bae0a3f3a3(at)hogranch(dot)com"
type="cite">
<p><br>
</p>
<pre class="moz-signature" cols="72">--
john r pierce, recycling bits in santa cruz</pre>
</blockquote>
<p><br>
</p>
</body>
</html>

Attachment Content-Type Size
unknown_filename text/html 3.1 KB

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message FarjadFarid(ChkNet) 2016-07-29 09:44:09 Re: [SPAM] Re: WAL directory size calculation
Previous Message John R Pierce 2016-07-29 08:43:16 Re: [SPAM] Re: WAL directory size calculation