Re: Practical impediment to supporting multiple SSL libraries

* Tom Lane (tgl(at)sss(dot)pgh(dot)pa(dot)us) wrote:
> Please mention some specific examples. We need some examples as a
> reality check.

Just took a look through a couple of Debian packages which depend on
libpq4:

libpam-pgsql: pam_pgsql.c, line 473 it uses PQgetvalue() as one would
expect, but doesn't actually save the pointer anywhere, just uses it to
do comparisons against (all it stores, apparently, is a password in the
DB).

libnss-pgsql: src/backend.c, line 228:

sptr = PQgetvalue(res, row, colnum);
slen = strlen(sptr);
if(*buflen < slen+1) {
return NSS_STATUS_TRYAGAIN;
}
strncpy(*buffer, sptr, slen);
(*buffer)[slen] = '\0';

*valptr = *buffer;

*buffer += slen + 1;
*buflen -= slen + 1;

return NSS_STATUS_SUCCESS;

That really seems to be the classic example to me. Get the data from
PQresult, store it in something else, work on it.

mapserver: mappostgis.c, starting from line 1340:

shape->values = (char **) malloc(sizeof(char *) * layer->numitems);
for(t = 0; t < layer->numitems; t++) {
temp1= (char *) PQgetvalue(query_result, 0, t);
size = PQgetlength(query_result, 0, t);
temp2 = (char *) malloc(size + 1);
memcpy(temp2, temp1, size);
temp2[size] = 0; /* null terminate it */

shape->values[t] = temp2;

}

This same code repeats in another place (1139). They also
appear to forget to PQclear() in some cases. :( They don't appear to
ever save the pointer returned by PQresult() for anything.

postfix: src/global/dict_pgsql.c, starting from line 349:

numcols = PQnfields(query_res);

for (expansion = i = 0; i < numrows && dict_errno == 0; i++) {
for (j = 0; j < numcols; j++) {
r = PQgetvalue(query_res, i, j);
if (db_common_expand(dict_pgsql->ctx, dict_pgsql->result_format,
r, name, result, 0)
&& dict_pgsql->expansion_limit > 0
&& ++expansion > dict_pgsql->expansion_limit) {
msg_warn("%s: %s: Expansion limit exceeded for key: '%s'",
myname, dict_pgsql->parser->name, name);
dict_errno = DICT_ERR_RETRY;
break;
}
}
}
PQclear(query_res);
r = vstring_str(result);
return ((dict_errno == 0 && *r) ? r : 0);

exim does something similar to postfix too.

It really seems unlikely that anyone keeps PQresult's around for very
long and they all seem to want to stick it into their own memory
structure. I don't know how many people would move to a new API should
one be provided though. Callbacks can be kind of a pain in the butt to
code too which makes the amount of effort required to move to using them
a bit higher too. This all means double memory usage though and that
really makes me want some kind of API that can be used to process data
as it comes in.

Another thought along these lines: Perhaps a 'PQgettuple' which can be
used to process one tuple at a time. This would be used in an ASYNC
fashion and libpq just wouldn't read/accept more than a tuple's worth
each time, which it could do into a fixed area (in general, for a
variable-length field it could default to an initial size and then only
grow it when necessary, and grow it larger than the current request by
some amount to hopefully avoid more malloc/reallocs later).

Thanks,

Stephen