From: | Joe Conway <mail(at)joeconway(dot)com> |
---|---|
To: | Carsten Klein <c(dot)klein(at)datagis(dot)com>, pgsql-general <pgsql-general(at)postgresql(dot)org> |
Subject: | Re: Generic File Access Function to read program output |
Date: | 2024-02-07 14:54:49 |
Message-ID: | 27439ea4-9a05-4057-becc-09e4efce742a@joeconway.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general |
On 2/7/24 05:28, Carsten Klein wrote:
> Hi there,
>
> on PostgreSQL 14, I'm using function pg_read_file to read a JSON file on
> the server. After that, the JSON file gets casted to jsonb and with
> function jsonb_array_elements I'm iterating over the "records", which I
> transform into a PostgreSQL ROWTYPE with jsonb_populate_record...
>
> Since the source files are actually XML files, these are turned into
> JSON files with Node JS and the fast-xml-parser module (processing JSON
> is much faster and more comfortable than processing XML in PostgreSQL).
>
> The command line of this conversion process is like this:
>
> # node /opt/my_node_apps/xml_to_json.js <path_to_xml_file>
>
> In order to do this without temporary JSON files (which need to be
> deleted at some time), it would be great to have a new Generic File
> Access Function
>
> pg_read_program_output(command)
>
>
> Although one could argue, that it's not a Generic *File* Access
> Function, that function would be a worthwhile addition and could use the
> same semantics and rules as with the
>
> COPY table_name FROM PROGRAM 'command'
>
> statement. Also the implementation (running a command with the shell and
> capture it's STDOUT) is nearly the same.
>
> In contrast to the other Generic File Access Functions, it will be
> almost impossible to restrict access to programs or commands within the
> database cluster directory (could be a complex shell command). Aside
> from that this makes no sense since, typically, there are no executable
> programs in those directories.
>
> Even worse, it's likely also not possible to restrict the source of the
> content read (the STDOUT) to be any of these directories, since the
> program could just dump anything to its STDOUT.
>
> AFAIT, that's not really an issue but only makes this new Generic File
> Access Function special, in that these restrictions and the meaning of
> role pg_read_server_files just do not apply for it.
>
> Do you know if there is already such a function, maybe provided by an
> extension I do not yet know?
Maybe write your own in plpython or plperlu?
--
Joe Conway
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
From | Date | Subject | |
---|---|---|---|
Next Message | Tom Lane | 2024-02-07 15:12:35 | Re: Error on create extension pgcrypto. Undefined symbol "EVP_bf_ecb" |
Previous Message | Marco Aurelio | 2024-02-07 14:40:21 | Re: Error on create extension pgcrypto. Undefined symbol "EVP_bf_ecb" |