Payments flow adaptation for lazy payments

[Tiago-Salles]

Context

When the user clicks Continuar on PayGate, PayGate redirects back to PayGateCallbackSuccessResponseView. Until now, this view called handle_payment_and_create_order synchronously, which in turn invoked PayGate.handle_processor_response. That method queries the PayGate BackOfficeSearchTransactions API expecting to find a completed transaction for the basket.

For asynchronous payment methods (MB references, MBWAY) the upstream payment is not yet confirmed at this exact moment — the user still has to pay at an ATM / home-banking / phone. PayGate correctly returns an empty list, handle_processor_response raises:

oscar.apps.payment.exceptions.GatewayError: PayGate couldn't double check if basket has been payed

…and the user is shown the misleading "You have not been charged." page even when, minutes later, PayGate will confirm the payment via the server-to-server callback.

What this PR changes

The success callback no longer attempts to fulfil the order synchronously when a Thank-You URL is configured. Instead, it records the callback response and redirects the user to a Thank-You page in the ecommerce micro-frontend, where the actual payment status is lazily resolved per basket by the new nau_extensions.BasketPaymentStatusView.

• processors.py — PayGate now exposes a new thank_you_url property that reads the optional thank_you_url payment-processor configuration entry. Returns None when not configured.
• views.py — PayGateCallbackSuccessResponseView.get:
  • records the PayGate callback as before;
  • if thank_you_url is configured → redirects to <thank_you_url>?order_number=<basket.order_number> without calling handle_payment_and_create_order and without calling PayGate BackOfficeSearchTransactions;
  • if thank_you_url is not configured → falls back to the previous behaviour (synchronous fulfillment + redirect to the receipt page). This preserves backwards compatibility for any deployment that has not yet rolled out the Thank-You page.

Configuration

Add thank_you_url to the paygate entry of PAYMENT_PROCESSOR_CONFIG to enable the new behaviour. Example:

paygate:
  access_token: PwdX_XXXX_YYYY
  merchant_code: NAU
  api_checkout_url: https://lab.optimistic.blue/paygateWS/api/CheckOut
  api_back_search_transactions: https://lab.optimistic.blue/paygateWS/api/BackOfficeSearchTransactions
  api_basic_auth_user: username
  api_basic_auth_pass: password
  payment_types: ["VISA", "MASTERCARD", "MBWAY", "REFMB", "DUC"]
  # NEW: when set, the success callback redirects here instead of running
  # handle_processor_response synchronously. The basket order_number is
  # appended as a query string parameter.
  thank_you_url: https://orders.nau.edu.pt/thank-you

Related PRs

• ecommerce-nau-extensions: Payments flow adaptation for lazy payments ecommerce-nau-extensions#37
• frontend-app-ecommerce: Payments flow adaptation for lazy payments frontend-app-ecommerce#3

Related to: https://github.com/fccn/nau-technical/issues/923

[Tiago-Salles]

Hey, @ManuelStarDo and @rguerra-fccn. You can start reviewing this PR, while I deal with the pipeline errors. Thanks.

[ManuelStarDo]

The following are concrete code-level items worth addressing before merge:

• paygate/pending_orders.py::confirm_pending_order — no guard against concurrent double-confirmation. If the lazy-resolution endpoint (in the nau_extensions PR) and the server-to-server callback both call confirm_pending_order for the same order at nearly the same time, both could pass the "still Pending" check before either commits, and both would call save_payment_details(order) / set_status(ORDER.OPEN), potentially double-recording a PaymentSource/PaymentEvent. There's no select_for_update() or a re-check of order.status after acquiring a lock.

  • Fix: add a lock/guard, e.g. order = Order.objects.select_for_update().get(pk=order.pk); if order.status != ORDER.PENDING: return order at the top of confirm_pending_order, run inside transaction.atomic().

• paygate/pending_orders.py::place_pending_order — basket.submit() is called without catching a possible double-submit if two requests race before either commits the new Order row (i.e., before the IntegrityError that the caller in views.py already handles). If basket.submit() itself raises (Oscar generally rejects submitting an already-submitted basket) rather than the Order.number uniqueness constraint, that exception is not IntegrityError and falls into the view's generic except Exception branch, which unconditionally redirects to the error page — even though the "losing" request's sibling actually succeeded.

  • Fix: in the view's generic except Exception handler (not just the IntegrityError branch), also attempt get_order(basket) before deciding to show the error page, mirroring the recovery logic already used for IntegrityError.

• paygate/utils.py — minor duplication between get_order() and order_exist()
  Risk: get_order(basket) (new in this PR) and order_exist(basket) (pre-existing) both answer the same underlying question — "is there an Order for this basket?" — via two separate Order.objects.filter(...) queries. order_exist() returns a bool and is still used in paygate/processors.py::retry_baskets_payed_in_paygate(); get_order() returns the object and is used in paygate/views.py, which needs the actual Order (e.g. to check existing_order.status). Neither is dead code, so nothing needs to be removed, but having two independent lookups for the same thing is a small maintainability smell — if the lookup logic ever needs to change (e.g. adding a filter), it's easy to update one and forget the other.
  Fix: Rewrite order_exist() to delegate to get_order() so there's a single source of truth:

def order_exist(basket: Basket) -> bool:
    """
    Utility method that check if there is an Order for the Basket
    """
    return get_order(basket) is not None

Possible Improvements / Suggestions

• paygate/views.py::run_post_order: the docstring explaining why it's not named handle_post_order (to avoid shadowing the mixin's method) is good self-documentation; consider moving that explanation into the class-level docstring so it's visible alongside the other helper methods (create_new_order, handle_payment_and_create_order) for a reader scanning the class body.
• The module docstring in pending_orders.py explicitly flags the "nothing expires a Pending order" limitation as accepted for now — worth linking to or creating a follow-up issue if one doesn't already exist, so it doesn't get lost.

[Tiago-Salles]

@ManuelStarDo, I have applied the changes you requested. Ready to review.

[rguerra-fccn]

Tested locally: 52/52 tests passing. The race handling (select_for_update), idempotency, and IntegrityError recovery in pending_orders.py/views.py are solid and well covered.

However, the success-callback fallback path (no thank_you_url configured) doesn't match the PR description's claim of 'falls back to the previous behaviour (synchronous fulfillment + redirect to the receipt page)'. In the current code, handle_payment_and_create_order is never called from PayGateCallbackSuccessResponseView.get() anymore, for any payment method, regardless of thank_you_url. It unconditionally places a Pending order (if none exists) and redirects — thank_you_url only changes the redirect destination, not whether synchronous fulfillment happens.

This is confirmed by your own test, test_success_callback_without_thank_you_url_uses_the_receipt_page:

self.assertEqual(Order.objects.get(number=basket.order_number).status, ORDER.PENDING)
mock__make_api_json_request.assert_not_called()  # BackOfficeSearchTransactions never called

This isn't just a transient window either: there's no nau-tutor-configs PR yet that sets thank_you_url in PAYMENT_PROCESSOR_CONFIG, so this will be the default production behaviour immediately after merge, for however long until a follow-up config PR lands. A paying card user could be redirected straight to the receipt page for an order that is still Pending at that exact moment, relying entirely on the server-to-server callback (which does correctly pick up and confirm the pending order) to land afterwards.

Could you either:

1. Gate the pending-order-only path behind thank_you_url being configured, so the old synchronous path is genuinely preserved until the flag is turned on, or
2. Confirm the receipt page tolerates a momentarily-Pending order gracefully, and update the PR description to state plainly that this is an intentional, unconditional change in reliance on the server-to-server callback.

Requesting changes pending that clarification — happy to re-review quickly once addressed.

[ManuelStarDo]

@Tiago-Salles
How do we handle payments that never get paid?
There should be a method to clean stale payments that never got paid.

Two points worth addressing:

[MEDIUM] paygate/views.py:364 — thank_you_url query-string append is naive string concatenation
Risk: If a misconfigured thank_you_url already contains a trailing ? or &, the redirect URL becomes malformed, potentially breaking the Thank-You page load.
Fix: Use urllib.parse.urlparse/urlencode (or at minimum .rstrip('?&') on thank_you_url before appending) for a more robust join.

[LOW / SUGGESTION] pending_orders.py — unresolved "Pending orders never expire" limitation
Risk: Over time, unpaid Multibanco baskets accumulate as permanently Pending orders with no automated cleanup, relying entirely on manual support intervention.
Fix: Confirm a tracking issue exists (referenced PR mentions nau-technical#923 — verify this specific limitation is captured there or file a dedicated follow-up) for a future management command to expire stale Pending orders using the already-scaffolded but commented-out REFMB_START_DATE/REFMB_END_DATE fields.

[Tiago-Salles]

@ManuelStarDo, applied to suggestion.

About the pending order, it's not a concern for this ticket, please don't mix subjects. This ticket (with the three PRs that implement it) has the intention of building the structure where payments are able to be handled in an asynchronous way, meaning the platform will not expect the confirmation right after it just created an order.

I am about to create an issue to handle pending payments management subejct. Any specific MBREF business logic, or payments management related topics have nothing to do with the ability of a person paying a course asynchronously.