Re: PL/Python patch for Universal Newline Support

On Mon, Mar 21, 2005 at 10:28:28AM +1100, Neil Conway wrote:
> Michael Fuhr wrote:
>
> >How should I submit regression tests?
>
> Yes, please.

The operative word there was "how" :-) I don't see anything testing
PL/{Python,Perl,Tcl} under src/test/regress -- should I put something
there? Can regression tests be run conditionally? If so, what's
the preferred way to do that? If regression tests for this kind
of patch need to be done another way, how should I submit them?

> Does this work for "\r\n" embedded in string literals?

Literal carriage returns (ASCII character 13) will be translated,
just as Python would do when reading a script from a file if Python
is built with Universal Newline Support (enabled by default, at
least in recent versions). Escape sequences won't be touched if
they're stored in prosrc as escape sequences (backslash-r) instead
of actual carriage returns. For example:

CREATE FUNCTION foo() RETURNS text AS $$
return "\r\n"
$$ LANGUAGE plpythonu IMMUTABLE;

SELECT prosrc FROM pg_proc WHERE proname = 'foo';
prosrc
-----------------

return "\r\n"

(1 row)

SELECT length(foo()), ascii(foo()), ascii(substr(foo(), 2));
length | ascii | ascii
--------+-------+-------
2 | 13 | 10
(1 row)

But the following fails (the function is in single quotes instead
of dollar quotes, so \r\n becomes an actual CRLF):

CREATE OR REPLACE FUNCTION foo() RETURNS text AS '
return "\r\n"
' LANGUAGE plpythonu IMMUTABLE;

SELECT prosrc FROM pg_proc WHERE proname = 'foo';
prosrc
---------------

return "
"

(1 row)

SELECT foo();
ERROR: plpython: could not compile function "foo"
DETAIL: exceptions.SyntaxError: EOL while scanning single-quoted string (line 3)

An ordinary Python script that looked like that would fail as well:

% cat -v foo.py
print "^M
"

% python foo.py
File "foo.py", line 1
print "
^
SyntaxError: EOL while scanning single-quoted string

Note Python's translation in this case:

% cat -v foo.py
print """a^M
b^M
c^M
"""

% python foo.py | od -tx1
0000000 61 0a 62 0a 63 0a 0a
0000007

(The extra newline is appended by "print".)

I was thinking that the patch could be applied to HEAD and hopefully
somebody could do some additional testing with Windows clients and
servers. If there are no problems, and especially if the patch
solves the problem it's intended to solve, then the patch could be
applied to REL8_0_STABLE so it would be in 8.0.2 whenever that comes
out. Tom, that sounded reasonable to you, didn't it?

http://archives.postgresql.org/pgsql-general/2005-03/msg00842.php

--
Michael Fuhr
http://www.fuhr.org/~mfuhr/