Any experiences with de-duplicating backup of dumps?

Hi everyone,

I'm in the process of changing backups from simply storing
encrypted dumps (encrypted by the host) on some external NAS to using
the software Borg, a de-duplicating archiver.

> Deduplication is performed across multiple backups, both on data and
> metadata, using Chunks created by the chunker using the Buzhash
> algorithm.

https://borgbackup.readthedocs.io/en/0.30.0/internals.html

I've created dumps with highest compression available to make the
backups as small as possible in the past, to use the available storage
on the NAS as best as possible. Now I'm wondering if I should keep
compression before de-duplication (and again compression) of Borg or
if things might behave better with less or none compression at all. I
could e.g. use the plain text format, though, the custom one seems
more flexible and supports no compression as well.

What makes me additionally wonder is if Postgres processes things in
constant order, which de-duplication should benefit from most likely.
Especially if e.g. rows are mostly added instead of updated, which is
some of my use cases.

Do you have any test results and experiences already about that topic?
Which formats do you use with de-duplication? Or wasn't it worth it at
all?

Thanks!

Mit freundlichen Grüßen

Thorsten Schöning

--
AM-SoFT IT-Service - Bitstore Hameln GmbH
Mitglied der Bitstore Gruppe - Ihr Full-Service-Dienstleister für IT und TK

E-Mail: Thorsten(dot)Schoening(at)AM-SoFT(dot)de
Web: http://www.AM-SoFT.de/

Tel: 05151- 9468- 0
Tel: 05151- 9468-55
Fax: 05151- 9468-88
Mobil: 0178-8 9468-04

AM-SoFT IT-Service - Bitstore Hameln GmbH, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 221853 - Geschäftsführer: Janine Galonska