Skip to content

Fixed hub upgrade PG migration failing with Permission denied on old binaries - #2575

Open
nickanderson wants to merge 1 commit into
cfengine:masterfrom
nickanderson:ENT-14899/master
Open

nickanderson wants to merge 1 commit into
cfengine:masterfrom
nickanderson:ENT-14899/master

Conversation

@nickanderson

Copy link
Copy Markdown
Member

Copies bin into the PostgreSQL backup instead of hardlinking it, and checks/repairs that cfpostgres can execute the old binaries before migrating.

Ticket: ENT-14899

@nickanderson

Copy link
Copy Markdown
Member Author

@cf-bottom jenkins please

@cf-bottom

Copy link
Copy Markdown

@olehermanse

Copy link
Copy Markdown
Member

@cf-bottom jenkins, please

@cf-bottom

Copy link
Copy Markdown

@nickanderson

Copy link
Copy Markdown
Member Author

@craigcomstock what do you think about this one? I think the failure is a flake.

@craigcomstock craigcomstock left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's OK but I would prefer explicitly checking the state of pg binaries rather than just trying to execute pg_ctl, modifying pg_ctl that might be there and trying again.

Would be good to get explicit information out of the failure, like does pg_ctl exist? If cfpostgres can't run it, what are it's ownership, permissions, file attributes?

Comment thread packaging/common/cfengine-hub/postinstall.sh Outdated
Comment thread packaging/common/cfengine-hub/preinstall.sh
@nickanderson

Copy link
Copy Markdown
Member Author

Agreed, the retry was guesswork. 577b356 replaces it with explicit checks before the database is touched: does pg_ctl exist, can cfpostgres execute it, and does pg_ctl --version run. If any of them fails it stops and shows ls -ld of the backup dir, bin and pg_ctl, namei -l of the whole path, lsattr, and pg_ctl's own error output, so we can see what's wrong instead of trying to fix it.

On the masterfiles side, cfengine/masterfiles#3226 stops the CFE-951 perms promises from recursing into state/pg/backup. I left the copy-instead-of-hardlink and the mode normalization in preinstall, since an upgrade can run with an older MPF that doesn't have that exclusion yet.

@nickanderson
nickanderson dismissed craigcomstock’s stale review October 26, 2026 20:38

Addressed, now explicitly checking the state of pg executablity

Comment thread packaging/common/cfengine-hub/postinstall.sh Outdated
Comment thread packaging/common/cfengine-hub/postinstall.sh Outdated
@nickanderson
nickanderson dismissed craigcomstock’s stale review October 27, 2026 16:39

Now killing cf-agents still running after a wait

# things, like permissions under state/pg (ENT-14899). Gives running agents
# 1.5 times the average agent run time from promise_summary.log (150s if
# unknown) to finish, then kills them. Does nothing without pgrep.
command -v pgrep >/dev/null || return 0

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should fail not return 0

Comment thread packaging/common/script-templates/script-common.sh Outdated
Comment thread packaging/common/script-templates/script-common.sh
Comment thread packaging/common/script-templates/script-common.sh Outdated

@craigcomstock craigcomstock left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good! one optional nit.

safe_cp "$BACKUP_DIR" bin "$PREFIX"
safe_cp_nolink "$BACKUP_DIR" bin "$PREFIX"
# Ensure restored old binaries are executable by cfpostgres.
chmod 755 "$PREFIX/bin"/* 2>/dev/null || true

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are there files in bin which should not be 755?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the rpm sets everything in bin to 755 so I think it's OK but i could limit it to pg_ctl, pg_dumpall, postgres, and pg_controldata if you prefer

@nickanderson

Copy link
Copy Markdown
Member Author

@cf-bottom jenkins please

@cf-bottom

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

5 participants