Using AWS ephemeral SSD storage for production database workload?

From: Pritam Barhate <pritambarhate(at)gmail(dot)com>
To: pgsql-general(at)lists(dot)postgresql(dot)org
Subject: Using AWS ephemeral SSD storage for production database workload?
Date: 2018-01-29 16:41:34
Message-ID: CALpo98USPhKVAPYDLf=S92cE8tYtxd4Vi1Wh=AqjGe=wH3SuNQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hi everyone,

As you may know, EBS volumes though durable are very costly when you
need provisioned IOPS. As opposed to this AWS instance attached ephemeral SSD
is very fast but isn't durable.

I have come across some ideas on the Internet where people hinted at
running production PostgreSQL workloads on AWS ephemeral SSD storage.
Generally, this involves shipping WAL logs continuously to S3 and keeping
an async read replica in another AWS availability zone. Worst case scenario
in such deployment is data loss of a few seconds. But beyond this the
details are sketchy.

Have you come across such a deployment? What are some best practices that
need to be followed to pull this through without significant data loss?
Even though WAL logs are being shipped to S3, in case of loss of both the
instances, the restore time is going be quite a bit for databases of a few
hundred GBs.

Just to be clear, I am not planning anything like this, anytime soon :-)
But I am curious about trade-offs of such a deployment. Any concrete
information in this aspect is well appreciated.

Regards,

Pritam.

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Rakesh Kumar 2018-01-29 16:50:46 Re: PG Sharding
Previous Message Kumar, Virendra 2018-01-29 16:19:53 pgpool Connections Distributions Among Nodes