From: | Bear Giles <bgiles(at)coyotesong(dot)com> |
---|---|
To: | pgsql-hackers(at)postgresql(dot)org |
Subject: | what would tar file FDW look like? |
Date: | 2015-08-17 14:14:18 |
Message-ID: | CALBNtw6C8XGWmO2g6V2pwkPyF=z=-toXCWeZBTj6P4WmRy2C6Q@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
I'm starting to work on a tar FDW as a proxy for a much more specific FDW.
(It's the 'faster to build two and toss the first away' approach - tar lets
me get the FDW stuff nailed down before attacking the more complex
container.) It could also be useful in its own right, or as the basis for a
zip file FDW.
I have figured out that in one mode the FDW mapping that would take the
name of the tarball as an option and produce a relation that has all of the
metadata for the contained files - filename, size, owner, timestamp, etc. I
can use the same approach I used for the /etc/passwd FDW for that.
(BTW the current version is at https://github.com/beargiles/passwd-fdw.
It's skimpy on automated tests until I can figure out how to handle the
user mapping but it works.)
The problem is the second mode where I pull a single file out of the FDW.
I've identified three approachs so far:
1. A FDW mapping specific to each file. It would take the name of the
tarfile and the embedded file. Cleanest in some ways but it would be a real
pain if you're reading a tarball dynamically.
2. A user-defined function that takes the name of the tarball and file and
returns a blob. This is the traditional approach but why bother with a FDW
then? It also brings up access control issues since it requires disclosure
of the tarball name to the user. A FDW could hide that.
3. A user-defined function that takes a tar FDW and the name of a file and
returns a blob. I think this is the best approach but I don't know if I can
specify a FDW as a parameter or how to access it.
I've skimmed the existing list of FDW but didn't find anything that can
serve as a model. The foreign DB are closest but, again, they aren't
designed for dynamic use where you want to do something with every file in
an archive / table in a foreign DB.
Is there an obvious approach? Or is it simply a bad match for FDW and
should be two standard UDF? (One returns the metadata, the second returns
the specific file.)
Thanks,
Bear
From | Date | Subject | |
---|---|---|---|
Next Message | Greg Stark | 2015-08-17 14:29:58 | Re: what would tar file FDW look like? |
Previous Message | Kouhei Kaigai | 2015-08-17 13:40:31 | Re: Our trial to TPC-DS but optimizer made unreasonable plan |