Re: storing zipped SQLite inside PG ?

Or, if you want to extend this theme, you can use a PostgreSQL-based
"SQLite file player" with
PostgreSQL + Python[sqlite3] extension.This way you can provide direct
access to SQLite files without duplicating data in PostgreSQL cluster
tables.
PS: It may seem that this will reduce performance. When I started my
project, I had some preconceptions about Python. But analyzing projects
like Patroni changed my mind.
--
Regards, Dmitry!

ср, 22 дек. 2021 г. в 10:24, David G. Johnston <david(dot)g(dot)johnston(at)gmail(dot)com>:

> On Tue, Dec 21, 2021 at 10:06 PM David Gauthier <davegauthierpg(at)gmail(dot)com>
> wrote:
>
>> I'll have to read more about sqlite_fdw. Thanks for that Steve !
>>
>> Each SQLite isn't that big (billions of records), more like 30K records
>> or so. But there are lots and lots of these SQLite DBs which add up over
>> time to perhaps billions of records.
>>
>> This is for a big corp with an IT dept. Maybe I can get them to upgrade
>> the DB itself.
>> Thank You too David !
>>
>>>
>>>
> So, more similar to the image storage question than I first thought, but
> still large enough where the specific usage patterns and needs end up being
> the deciding factor (keeping in mind you can pick multiple solutions - so
> that really old data, ideally on a partition, can be removed from the DB
> while still remaining accessible if just more slowly or laboriously).
>
> One possibility to consider - ditch the SQLite dependency and just store
> CSV (but maybe with a funky delimiter sequence). You can then us
> "string_to_table(...)" on that delimiter to materialize a table out of the
> data right in a query.
>
> David J.
>
>