Ridvansirma/pgduck engine server side error logs - #648
Conversation
pgduck_server already maps DuckDB error types onto the ErrorResponse SQLSTATE, but the canned class derived from it was only logged by Postgres. Emit the same class from the engine too, so it is available in the engine's own log for failures that never reach a client cleanly. The line holds a class from a fixed literal set and nothing else. Every other line here may carry the DuckDB message, up to 500 characters of the statement, or the client address, so anything collecting these lines has to match the prefix rather than the process. --no_log_engine_errors turns the line off; the SQLSTATE and error text are unaffected either way. Startup announces the disabled state alongside the other non-default options, since an absent class line otherwise reads as no errors having occurred. That announcement names pgduck_engine_error without the trailing colon so it does not match a prefix-based collector itself. Signed-off-by: Ridvan Sirma <ridvan.sirma@snowflake.com> Co-authored-by: Cursor <cursoragent@cursor.com>
A genuine RAM OOM is the case a client often only sees as lost_connection. Assert the canned out_of_memory record is on stderr before exit(), and that the statement does not ride along. Signed-off-by: Ridvan Sirma <ridvan.sirma@snowflake.com> Co-authored-by: Cursor <cursoragent@cursor.com>
pgduck_server stderr is already a niche stream; an opt-out flag is extra surface without a use. The class line is always written. Signed-off-by: Ridvan Sirma <ridvan.sirma@snowflake.com> Co-authored-by: Cursor <cursoragent@cursor.com>
sfc-gh-abozkurt
left a comment
There was a problem hiding this comment.
try setting credentials in a pgduck_server session and make it throw some error. and lets verify if the credentials in the failing command show up in the log with the prefix. (we should not leak credential with this prefix)
In any failure that goes through this helper, the new line is only LOG pgduck_engine_error: . The class is the SQLSTATE mapping (out_of_memory / io_error / invalid_input / internal_error / other). No SQL, no DuckDB message, no credentials. I also added a test that fails CREATE SECRET with a KEY_ID, SECRET, and SESSION_TOKEN and checks that none of those values appear on that line. |
…n error class log Add a pytest that issues an invalid CREATE SECRET with distinctive KEY_ID/SECRET/SESSION_TOKEN values and asserts the classified engine error line carries the error class but none of the credential strings. Signed-off-by: Ridvan Sirma <ridvan.sirma@snowflake.com> Co-authored-by: Cursor <cursoragent@cursor.com>
8f40702 to
2acc2bf
Compare
Problem
Postgres already logs a canned class for engine failures (out_of_memory, io_error, invalid_input, internal_error, other). Failures that never reach a client cleanly (fatal OOM, lost connection) only show up in pgduck_server stderr, mixed with DuckDB messages, query snippets, and client addresses, so a collector cannot take those lines as-is.
Solution
Emit the same class from handle_pgsession_error_message, the choke point for reportable DuckDB statuses, including fatal ones that exit() right after. The line is only pgduck_engine_error: . Collectors have to match that prefix; every other stderr line here may still carry PII.
--no_log_engine_errors turns the class line off without changing the SQLSTATE or error text. Startup logs that the classification is off, naming pgduck_engine_error without a trailing colon so a prefix-based collector does not treat the announcement as an error.
Protocol stubs (named prepared statements, COPY, Close) and type-conversion statuses with a NULL SQLSTATE stay off this line or land in other, same as today.
Checklist
DCO Reminder (important)
This project uses the Developer Certificate of Origin (DCO).
DCO is a simple way for you to confirm that you wrote your code and that you have the right to contribute it.
If the DCO check fails, please sign off your commits.
How to sign off
For your last commit:
git commit --amend -s
git push --force
For multiple commits:
git rebase --signoff main
git push --force
More info: https://developercertificate.org/