Re: BUG #15449: file_fdw using program cause exit code error when using LIMIT

=?utf-8?q?PG_Bug_reporting_form?= <noreply(at)postgresql(dot)org> writes:
> SELECT * FROM test_file_fdw_program_limit LIMIT 0;
> /*
> [38000] ERROR: program "echo "test"" failed Detail: child process exited
> with exit code 1
> */

> LIMIT 0 is the easiest way we found to reproduce the error, but we
> encountered same issue with LIMIT (< file lines) on a program doing an if
> exists + file output stream manipulation.

Yeah, I can reproduce this on macOS as well as Linux. Capturing stderr
shows something pretty unsurprising:

sh: line 1: echo: write error: Broken pipe

I get similar behavior from "cat"; didn't try anything else.

So the called program is behaving in a somewhat reasonable way: it's
detecting EPIPE on its stdout (after we close the pipe), reporting that,
and doing exit(1).

Unfortunately, it's not clear what we could do about that, short of
always reading the whole program output, which is likely to be a
cure worse than the disease. If the program were failing thanks to
SIGPIPE, we could recognize that as a case we can ignore ... but with
behavior like this, I don't see a reliable way to tell it apart from
a generic program failure, which surely we'd better report.

The only idea that comes to mind is to introduce a file_fdw option
specifying "read program input to end, even if query doesn't want
it all". Not quite sure which choice should be the default; you
could make an argument either way I think.

Curiously, if you do stuff like "cat longfile | head -10", you don't
get any visible error, though surely cat must be seeing the same
thing underneath ... I wonder how the shell is arranging that?

regards, tom lane