Re: TCP network cost

On Tue, Feb 17, 2009 at 12:20:02AM -0700, Rusty Conover wrote:
>
>
> Try running tests with ttcp to eliminate any PostgreSQL overhead and
> find out the real bandwidth between the two machines. If its results
> are also slow, you know the problem is TCP related and not PostgreSQL
> related.

I did in fact run a simple netcat client/server pair and verified that I
can transfer that file on 0.12 sec localhost (or hostname), 0.35 over the
net, so TCP stack and network are not to blame. This is purely inside
the postgresql code issue, I believe.

On Tue, Feb 17, 2009 at 10:13:40AM -0800, Aaron Turner wrote:
>
> TCP has additional overhead as well as going through the IP stack
> which for non-tuned Linux kernels is pretty limiting.

Right. Already tuned those so long ago, I failed to mention it. Note the
'bare' transfer times added above. Nothing to write home about
(~3Mb/sec) but another order of magnitude faster than the postgresql
transfer.

> long story short, there are things in /proc you can use to increase
> buffers and window sizes which will help with large TCP streams (like
> a 40MB file for example). There's a lot of documentation on the net
> for how to tune the Linux IP stack so I won't repeat it here.
>
> Now, having your DB box 5 hops away is going to add a lot of latency
> and any packet loss is going to kill TCP throughput- especially if you
> increase window sizes. I'd recommend something like "mtr" to map the
> network traffic (make sure you run it both ways in case you have an
> asymmetric routing situation) for a long period of time to look for
> hiccups.

The 5-hops in on campus, gigabit all the way, w/ reasonable routing -
and not the issue: I see the same times from another machine attaached
to the same switch (which is the real use-case, actually.)

Ross
--
Ross Reedstrom, Ph.D. reedstrm(at)rice(dot)edu
Systems Engineer & Admin, Research Scientist phone: 713-348-6166
The Connexions Project http://cnx.org fax: 713-348-3665
Rice University MS-375, Houston, TX 77005
GPG Key fingerprint = F023 82C8 9B0E 2CC6 0D8E F888 D3AE 810E 88F0 BEDE