Software Catalogue - Clustering/replication

As the BiSoft team, which has been providing PostgreSQL support to many public and private institutions for many years, we attach importance to the spread of PostgreSQL both in our country and in the world and to increase its efficiency. With our ongoing PostgreSQL management experience, in line with the needs we see, we now meet the needs of advanced auto failover and auditing. We are excited to announce our software that we have developed to meet. Once a database environment has been installed, one of the first things that come to mind is the continuity and sustainability of data access (HA). In PostgreSQL HA is provided with built-in replication solutions. There is no multi-master architecture based on the shared storage logic. In the replication environment, if the master server is failed, the most appropriate of the standby servers should be designated as the new master and if any, other standby servers should be directed to the new master server. These operations require manual intervention or an external tool that can perform them automatically. BFM automates all of these in PostgreSQL and provides uninterrupted service during failover.

Multi-master and multi-slave asynchronous replication system for Postgres.

CloudNativePG is an open source operator designed to manage PostgreSQL workloads on any supported Kubernetes cluster running in private, public, hybrid, or multi-cloud environments. CloudNativePG adheres to DevOps principles and concepts such as declarative configuration and immutable infrastructure.

It defines a new Kubernetes resource called Cluster representing a PostgreSQL cluster made up of a single primary and an optional number of replicas that co-exist in a chosen Kubernetes namespace for High Availability and offloading of read-only queries.

Applications that reside in the same Kubernetes cluster can access the PostgreSQL database using a service which is solely managed by the operator, without having to worry about changes of the primary role following a failover or a switchover. Applications that reside outside the Kubernetes cluster, need to configure a Service or Ingress object to expose the Postgres via TCP. Web applications can take advantage of the native connection pooler based on PgBouncer.

CloudNativePG was originally built by EDB, then released open source under Apache License 2.0 and submitted for CNCF Sandbox in April 2022. The source code repository is in Github.

Tungsten Enterprise is a complete replication and data management solution for PostgreSQL. Tungsten Enterprise uses replication and distributed management to create virtualized databases using redundant data copies. Tungsten Enterprise helps customers build fault-tolerant clusters that ensure uninterrupted access to data, improve performance and lower costs. Best of all, Tungsten does not require DBMS upgrades, or application updates to use. It works transparently with existing applications and database versions.

Tungsten Enterprise builds on Tungsten Replicator, an open source, data replication engine for PostgreSQL. Tungsten Enterprise includes distributed management with simple operator commands to view status and perform maintenance, business rules to execute failover procedures, transparent SQL routing and load balancing, and integration with high-end storage systems like NetApp. Tungsten Enterprise runs equally well in clouds as well as on local hardware.

Tungsten Enterprise customers include large enterprises, SaaS businesses, social networking sites and ISPs. Applications that depend on Tungsten Enterprise range from in-house IT systems to multi-tenant applications that process hundreds of millions of transactions a day for thousands of customers.

EDB Postgres for Kubernetes is an operator designed, developed, and supported by EDB that covers the full lifecycle of a highly available Postgres database clusters with a primary/standby architecture, using native streaming replication. It is based on the open source CloudNativePG operator, and provides additional value such as compatibility with Oracle using EDB Postgres Advanced Server and additional supported platforms such as IBM Power and OpenShift.

Key features available include: - Kubernetes API integration for high availability - Self-healing through failover and automated recreation of replicas - Capacity management with scale up/down capabilities - Planned switchovers for scheduled maintenance - Read-only and read-write Kubernetes services definitions - Rolling updates for Postgres minor versions and operator upgrades - Continuous backup and point-in-time recovery - Connection Pooling with PgBouncer - Integrated metrics exporter out of the box - PostgreSQL replication across multiple Kubernetes clusters - Red Hat certified operator for OpenShift

The operator has been renamed from Cloud Native PostgreSQL. Existing users of Cloud Native PostgreSQL will not experience any change, as the underlying components and resources have not changed.

GridSQL is a shared-nothing clustered database system targeted at data warehousing and data mart applications. It includes intelligence to maximize parallelization over multiple servers, delivering faster query response times than can be achieved by single-node databases.

Kubegres is a Kubernetes operator allowing to deploy a cluster of PostgreSql instances with data replication enabled out-of-the box. It brings simplicity when using PostgreSql considering how complex managing stateful-set's life-cycle and data replication could be with Kubernetes.

Kubegres has the following features:

• It creates a cluster of PostgreSql servers with data replication enabled: it creates a Primary PostgreSql pod and a number of Replica PostgreSql pods and replicates primary's database in real-time to Replica pods.

• It manages fail-over: if a Primary PostgreSql crashes, it automatically promotes a Replica PostgreSql as a Primary.

• It has a data backup option allowing to dump PostgreSql data regularly in a given volume.

• It provides a very simple YAML with properties specialised for PostgreSql.

• It is resilient, has over 55 automatized tests cases and has been running in production.

• It works with the PostgreSql containers created by the Docker Official Images team: it does not ship nor require a custom Docker image to work.

Kubegres was developed by Reactive Tech Limited and Alex Arica as the lead developer.

It was developed with the framework Kubebuilder version 3, an SDK for building Kubernetes APIs using CRDs. Kubebuilder is maintained by the official Kubernetes API Machinery Special Interest Group (SIG).

You can find more details about how to install and use it in the Getting started page

Pervasync Database Synchronization Framework supports synchronization of local PostgreSQL databases with central PosrgreSQL databases.

The signature feature of the framework is that it allows you to sub-set the central server data using free form SQL queries with parameters. Each local database could sync shared as well as private data with central databases.

Synchronization could be two-way, meaning writes can be done on both local and central databases. Change capture, conflict detection and resolution are built in to the core of the sync framework.

pg_auto_failover is an extension and service for PostgreSQL that monitors and manages automated failover for a Postgres cluster. It is optimized for simplicity and correctness and supports Postgres 10 and newer.

pg_auto_failover supports several Postgres architectures and implements a safe automated failover for your Postgres service. It is possible to get started with only two data nodes which will be given the roles of primary and secondary by the monitor. Starting with pg_auto_failover version 1.4, it is possible to implement a production architecture with any number of Postgres nodes, for better data availability guarantees.

The pg auto failover documentation is available online at https://pg-auto-failover.readthedocs.io/ and includes a fully detailed tutorial to get started using Azure VMs.

Pg Chameleon is a MySQL to PostgreSQL replica system developed in python 3.3+. The system can replicate the data changes from MySQL to PostgreSQL in real time.

The library python-mysql-replication collects the data changes fromt the mysql replication protocol and use PostgreSQL as relay log and as replica destination.

Home page

Pypi package

Github page

Pgpool-II is a synchronous replication system for PostgreSQL 7.4 or later.It provides all desired functionalities to build a high performance/high availabilty cluster using PostgreSQL, including connection pooling and load balancing.

Pgpool can also be integrated with other replication systems such as Slony-I.

Slony-I is a "master to multiple slaves" replication system supporting cascading (e.g. - a node can feed another node which feeds another node...) and failover.

The big picture for the development of Slony-I is that it is a master-slave replication system that includes all features and capabilities needed to replicate large databases to a reasonably limited number of slave systems.

Slony-I is a system designed for use at data centers and backup sites, where the normal mode of operation is that all nodes are available.

SPQR is a query router for horizontal scaling of PostgreSQL via sharding.

Main features

• Sharding. If possible, the router determines a shard from the first statement of a transaction and keeps client bound to that shard until transaction ends. But you can explicitly specify a shard or a sharding key in a comment request.
• Transaction and session pooling. Just as in your favorite connection pooler (Odyssey or PgBouncer).
• Multiple routers for fault tolerance. The router stores the sharding rules only for cache purposes. Information about the entire installation is stored inside the QDB service, so the number of routers running simultaneously is unlimited.
• Liquid data migrations. Data migration between shards aims to balance the workload across shards proportionally. The main idea is to minimize any locking impact during these migrations, which is accomplished by reducing the size of the data ranges being transferred.
• Limited cross-shard queries. SPQR router supports limited subset of cross-shard queries. This is made from best-effort logic in a non-disruptive and non-consistent way and is used mainly for testing purposes. Please do not use this feature in your production, cross-shard snapshot will be inconsistent.
• Multiple servers and failover. In the router configuration, it is possible to specify multiple servers for one shard. Then the router will distribute read-only queries among the replicas. However, in addition to the automatic routing, you also have the option to explicitly define the destination for a specific query by using the target-session-attr parameter within the query.
• Minor overhead for query execution. See benchmarks here and here.

TCapture is a bidirectional multi master replication server based on a 'capture and apply' asynchronous replica engine

The actual TCapture engine is a Java application which runs as a separate program outside "PostgreSQL" , and which must be started explicitly. When TCapture is running, it will scan the transaction log of all primary databases and pick up transactions which must be replicated. Transactions which have been picked up are stored in the 'store database' , a "PostgreSQL" user database exclusively used by TCapture. In the store database, transaction is 'copied' to all store databases which have a subscription for this transaction. Transaction is then applied to the replicate databases

One of the attractions of TCapture is that it’s quite easy to set up and configure: starting from scratch, you can deploy a working replication system in less than 30 minutes. The setup procedure is described in the TCapture Guide. As always with replication, make sure you have a clear idea of the replication logic you want to implement before you start.

KEY FEATURE:

• Transactional. SQL are captured transactionally, can be coupled with surrounding business logic.
• Efficient. It capture transactions for replication from Write-Ahead Logs (WAL) instead of using triggers, eliminating overhead on master databases and significantly reduces latency
• Flexible. No limits on the number of producers or consumers, but complexity increases
• Reliable. Transactions are stored in "PostgreSQL" database – this adds the benefit of write ahead logging and crash recovery.
• Transparent. No impact on the applications already running on that database, since both engine and 'store database' can run separate from production databases.
• Easy to use. Simple to set up and configure, is an effective tool for data replication between different "PostgreSQL" versions
• Open Source. No licensing fees, but occasionally you'll have to get your hands dirty