Re: Some encoding trouble via libpq

Ahhh, I get it! Thanks for pointing this out. It ended up not being the
problem I was running into, but eventually it would have snagged me
(solution to original problem below). I spent some time going over that bit
of code and what you said with a coworker, and we ended up taking care of
it. So many little gotchas when doing this stuff! Here's the snippet as of
now for anyone interested:

int size = STDIN_BLOCK + 1;
char *buffer = (char *) xmalloc(size);
int offset = 0;
int read = 1;

while ( (read > 0) && (offset <= STDIN_MAX) )
{
if (offset + STDIN_BLOCK >= size)
{
size += STDIN_BLOCK;
buffer = xrealloc (buffer, size);
}
syslog (LOG_DEBUG, "Reading a block...");
read = fread (buffer + offset, 1, STDIN_BLOCK, stdin);
offset += read;
} // while

// null terminate the string...
memset(buffer + offset + 1, '\0', 1);

Also, an update on the original problem: It turned out with some
experimenting that the problem I experienced had nothing to do with the
standard input reading routine, or postgresql itself, but rather how I was
re-inserting into one query data I had gotten from a previous query. And
this is maybe useful for others on the list:

I was doing a select on one table to get a foreign key id that I would use
subsequently in an insert statement, like thus:

result = PQexecParams(conn,
"SELECT id FROM events WHERE ping_id = $1 AND
serial = $2",
2,
NULL, // backend figures out type itself
paramValues,
NULL, // apparently we don't need param lengths
NULL, // all text params
0 // we don't want binary results, no
);
... a bunch of tuples checking, then...
event_id = PQgetvalue(result, 0, 0); // <-- BAD GUY!
PQclear(result); // <-- ACCOMPLICE!
... then the insert....
paramValues[0] = ping_id;
paramValues[1] = event_id;
paramValues[2] = message; // get from std input! how do we do that
again???
result = PQexecParams(conn,
"INSERT INTO event_changes (ping_id, event_id,
created_at, message) VALUES ($1, $2, NOW(), $3)",
3,
NULL, // backend figures out type itself
paramValues,
NULL, // apparently we don't need param lengths
NULL, // all text params
0 // we don't want binary results, no
);

This is what was causing our woes. event_id is merely a pointer to data in
result. When you PQclear the result, it's gone! Sort of... Mac OS X was
forgiving on some level in that the data we were pointing to was still
there. But since different operating systems manage their memory
differently, Linux wasn't having any of it, it was basically pointing to
garbage!

The solution was to not clear result until later, or to copy the data in
allocated memory, then clear the result, which we did thusly:

tmp_str = PQgetvalue(result, 0, 0);
tmp_str_len = strlen(tmp_str);
event_id = xmalloc(tmp_str_len + 1);
strncpy(event_id, tmp_str, tmp_str_len);

Until next time,
Billy

On 3/29/07, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>
> "William Gray" <billy(dot)zophar(at)gmail(dot)com> writes:
> > ... And in the case
> > that fread() pulls in less data than requested, that means the next call
> to
> > fread() should return zero, right?
>
> Wouldn't count on that, particularly not when reading from an
> interactive device. You are more likely to get a line per call.
>
> What's bothering me about your code is that it assumes there are
> exactly STDIN_BLOCK bytes available in the buffer when you call
> fread, and the code does nothing that guarantees that. Personally
> I'd have used "size - offset" as the fread length parameter and not
> had to worry.
>
> regards, tom lane
>