Skip to content

Ridvansirma/pgduck engine server side error logs - #648

Merged
sfc-gh-rsirma merged 4 commits into
mainfrom
ridvansirma/pgduck_engine_server_side_error_logs
Sep 23, 2026
Merged

sfc-gh-rsirma merged 4 commits into
mainfrom
ridvansirma/pgduck_engine_server_side_error_logs

Conversation

@sfc-gh-rsirma

@sfc-gh-rsirma sfc-gh-rsirma commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Postgres already logs a canned class for engine failures (out_of_memory, io_error, invalid_input, internal_error, other). Failures that never reach a client cleanly (fatal OOM, lost connection) only show up in pgduck_server stderr, mixed with DuckDB messages, query snippets, and client addresses, so a collector cannot take those lines as-is.

Solution

Emit the same class from handle_pgsession_error_message, the choke point for reportable DuckDB statuses, including fatal ones that exit() right after. The line is only pgduck_engine_error: . Collectors have to match that prefix; every other stderr line here may still carry PII.

--no_log_engine_errors turns the class line off without changing the SQLSTATE or error text. Startup logs that the classification is off, naming pgduck_engine_error without a trailing colon so a prefix-based collector does not treat the announcement as an error.

Protocol stubs (named prepared statements, COPY, Close) and type-conversion statuses with a NULL SQLSTATE stay off this line or land in other, same as today.


Checklist

  • I have tested my changes and added tests if necessary
  • I updated documentation if needed
  • I confirm that all my commits are signed off (DCO)

DCO Reminder (important)

This project uses the Developer Certificate of Origin (DCO).
DCO is a simple way for you to confirm that you wrote your code and that you have the right to contribute it.

If the DCO check fails, please sign off your commits.

How to sign off

For your last commit:
git commit --amend -s
git push --force

For multiple commits:
git rebase --signoff main
git push --force

More info: https://developercertificate.org/

sfc-gh-rsirma and others added 2 commits September 21, 2026 14:05
pgduck_server already maps DuckDB error types onto the ErrorResponse
SQLSTATE, but the canned class derived from it was only logged by Postgres.
Emit the same class from the engine too, so it is available in the engine's
own log for failures that never reach a client cleanly.

The line holds a class from a fixed literal set and nothing else. Every
other line here may carry the DuckDB message, up to 500 characters of the
statement, or the client address, so anything collecting these lines has to
match the prefix rather than the process.

--no_log_engine_errors turns the line off; the SQLSTATE and error text are
unaffected either way. Startup announces the disabled state alongside the
other non-default options, since an absent class line otherwise reads as no
errors having occurred. That announcement names pgduck_engine_error without
the trailing colon so it does not match a prefix-based collector itself.

Signed-off-by: Ridvan Sirma <ridvan.sirma@snowflake.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
A genuine RAM OOM is the case a client often only sees as lost_connection.
Assert the canned out_of_memory record is on stderr before exit(), and that
the statement does not ride along.

Signed-off-by: Ridvan Sirma <ridvan.sirma@snowflake.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread pgduck_server/src/command_line/command_line.c Outdated
pgduck_server stderr is already a niche stream; an opt-out flag is extra
surface without a use. The class line is always written.

Signed-off-by: Ridvan Sirma <ridvan.sirma@snowflake.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread pgduck_server/src/pgsession/pgsession.c

@sfc-gh-abozkurt sfc-gh-abozkurt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

try setting credentials in a pgduck_server session and make it throw some error. and lets verify if the credentials in the failing command show up in the log with the prefix. (we should not leak credential with this prefix)

@sfc-gh-rsirma

Copy link
Copy Markdown
Contributor Author

try setting credentials in a pgduck_server session and make it throw some error. and lets verify if the credentials in the failing command show up in the log with the prefix. (we should not leak credential with this prefix)

In any failure that goes through this helper, the new line is only LOG pgduck_engine_error: . The class is the SQLSTATE mapping (out_of_memory / io_error / invalid_input / internal_error / other). No SQL, no DuckDB message, no credentials.

I also added a test that fails CREATE SECRET with a KEY_ID, SECRET, and SESSION_TOKEN and checks that none of those values appear on that line.

…n error class log

Add a pytest that issues an invalid CREATE SECRET with distinctive
KEY_ID/SECRET/SESSION_TOKEN values and asserts the classified engine
error line carries the error class but none of the credential strings.

Signed-off-by: Ridvan Sirma <ridvan.sirma@snowflake.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@sfc-gh-rsirma
sfc-gh-rsirma force-pushed the ridvansirma/pgduck_engine_server_side_error_logs branch from 8f40702 to 2acc2bf Compare September 23, 2026 16:38
@sfc-gh-rsirma
sfc-gh-rsirma merged commit bb4e064 into main Sep 23, 2026
88 checks passed
@sfc-gh-rsirma
sfc-gh-rsirma deleted the ridvansirma/pgduck_engine_server_side_error_logs branch September 23, 2026 16:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants