Re: New "raw" COPY format

On Tue, Oct 15, 2024, at 03:35, Joel Jacobson wrote:
> On Mon, Oct 14, 2024, at 21:59, Joel Jacobson wrote:
>> On Mon, Oct 14, 2024, at 10:51, Joel Jacobson wrote:
>>> On Mon, Oct 14, 2024, at 10:07, Joel Jacobson wrote:
>>>> Attached is a first draft implementation of the new proposed COPY "raw" format.
>>>>
>>>> The first two patches are just the bug fix in HEAD, reported separately:
>>>> https://commitfest.postgresql.org/50/5297/
...
> Sorry about the noise. I'm not running the full test suite,
> with tap and `meson test --num-processes 32`,
> so hopefully I won't cause cfbot failures as often any longer.

Ops, that should have said:
"Sorry about the noise. I'm *now* running the full test suite"

However, I see Windows still failed on copy2.sql,
and I think the reason could be the use of \qecho -n
to create files with inconsistent newline style, e.g.:

\o :filename
\qecho -n line1
\qecho -n '\n'
\qecho -n line2
\qecho -n '\r\n'
\o
COPY copy_raw_test_errors (col1) FROM :'filename' (FORMAT raw);

Maybe Windows automatically translates \n into \r\n, and vice versa?
If so, this would explain why this test failed on Windows.

Btw, anyone know if it's possible to download the "regression.diffs" file from a the Ci task?

I've downloaded all the crashlog, meason_log, testrun zip files from

https://cirrus-ci.com/task/4564405273231360

but none of these contained the "regression.diffs" mentioned here:

[02:09:42.431] # The differences that caused some tests to fail can be viewed in the file
"C:/cirrus/build/testrun/regress/regress/regression.diffs".

Anyhow, I think I've fixed the problem now, in a cross-platform safe way,
by shipping src/test/regress/data/newline*.data files:

newlines_cr.data
newlines_cr_lr.data
newlines_cr_lr_nolast.data
newlines_cr_nolast.data
newlines_lr.data
newlines_lr_nolast.data
newlines_mixed_1.data
newlines_mixed_2.data
newlines_mixed_3.data
newlines_mixed_4.data
newlines_mixed_5.data

These are then used in copy.sql and copy2.sql, e.g.:

copy.sql:

\set filename :abs_srcdir '/data/newlines_lr.data'
TRUNCATE copy_raw_test;
COPY copy_raw_test (col) FROM :'filename' (FORMAT raw);
SELECT col, col IS NULL FROM copy_raw_test ORDER BY id;

copy2.sql:

-- Test inconsistent newline style
\set filename :abs_srcdir '/data/newlines_mixed_1.data'
COPY copy_raw_test_errors (col1) FROM :'filename' (FORMAT raw);

Attaching new version. It's only patch 0016 that has been updated.

/Joel