Logout Accepts Empty Token in Modulith HA

[balhar-jakub]

ZaasClientIntegrationTest.java:334 — sends an empty string "" as the token to the logout endpoint and expects ZaasClientException. In modulith HA the logout returns 204 No Content instead.

How the Request Flows

Client side (ZaasJwtService.java:233): The empty token is sent as a cookie apimlAuthenticationToken= (or an empty Bearer header) with no client-side validation.

Security layer (NewSecurityConfiguration.java:148): The login and logout endpoints share one filter chain configured with .anyRequest().permitAll(). Spring Security does not enforce any token presence before the request reaches the logout handler.

Logout handler (JWTLogoutHandler.java:39): Calls getJwtTokenFromRequest(), which:

• Filters out cookies with empty values (!cookie.getValue().isEmpty())
• Returns Optional.empty() for an empty Bearer header

So with an empty token, token.isPresent() is false and the handler calls failure.onAuthenticationFailure(...) — this should produce an error response. This path works correctly in single-instance mode.

Why It Breaks in Modulith HA

The divergence is in what happens before reaching JWTLogoutHandler. In modulith HA there are multiple gateway instances. The request passes through the Gateway's filter chain on its way to ZAAS, and the key suspect is the token validation filter that runs on the Gateway side.

AuthenticationService.invalidateJwtToken() (AuthenticationService.java:188):
Application app = isModulithMode
    ? eurekaClient.getApplication(CoreService.GATEWAY.getServiceId())  // HA path
    : eurekaClient.getApplication(CoreService.ZAAS.getServiceId());

In HA mode, invalidateTokenOnAnotherInstance() (AuthenticationService.java:285) iterates over all peer instances and calls DELETE /invalidate/ on each. With an empty token, this call either:

• Returns a non-error response from the peer (the peer also has no validation on that endpoint)
• Has its exception caught and silently swallowed at line 301–303

If the distributed call "succeeds" (peer returns 200/204 for an empty token), doInvalidate() returns Boolean.TRUE and the success handler writes 204 No Content — the logout appears to succeed.

In single instance, application.getInstances() only contains the current instance which is skipped (it's self), so returnValue stays true and then parseJwtToken("") throws a BadCredentialsException. In single instance, isInvalidatedOnAnotherInstance is false, so the exception is re-thrown at line 238. In HA, if a peer call actually completes (even incorrectly), isInvalidatedOnAnotherInstance becomes true, and the same BadCredentialsException is suppressed at line 236–238.

Root Cause (Precise)

In AuthenticationService.doInvalidate() (AuthenticationService.java:233):

} catch (BadCredentialsException e) {
    if (!isInvalidatedOnAnotherInstance) {
        throw e;  // re-thrown in single instance
    }
    // silently swallowed in HA if a peer responded
}

When there is a peer in HA mode and the peer's /invalidate/ endpoint responds (even to an empty token), isInvalidatedOnAnotherInstance = true. The subsequent parseJwtToken("") throws BadCredentialsException, but the catch block suppresses it because isInvalidatedOnAnotherInstance is true. The method returns Boolean.TRUE, the success path runs, and the logout returns 204.

Key Files

│ File │ Line │ Issue │
│-------------------│-------│-----------------------------------------------------------│
│ NewSecurityConfiguration.java │ 148 │ .permitAll() — no auth enforcement at filter chain level │
│ JWTLogoutHandler.java │ 39–45 │ Correct for single-instance; never reached in HA bypass path │
│ AuthenticationService.java │ 233–238 │ Root cause — BadCredentialsException suppressed when peer responded │
│ AuthenticationService.java │ 285–309 │ Peer invalidation call succeeds for empty token, setting isInvalidatedOnAnotherInstance=true │
│ ZaasJwtService.java │ 233 │ No client-side empty-token guard │

Recommended Fix

The most targeted fix is in AuthenticationService.doInvalidate(): validate the token format before attempting peer distribution, not after. An empty or unparseable token should be rejected immediately, before isInvalidatedOnAnotherInstance is set:

private Boolean doInvalidate(String jwtToken, boolean distribute, Application app) {
    // Validate token format first — before any peer calls
    final QueryResponse queryResponse;
    try {
        queryResponse = parseJwtToken(jwtToken).getQueryResponse();
    } catch (BadCredentialsException | TokenFormatNotValidException e) {
        throw e; // always propagate — don't suppress based on HA state
    }

    boolean isInvalidatedOnAnotherInstance = false;
    if (distribute) {
        isInvalidatedOnAnotherInstance = invalidateTokenOnAnotherInstance(jwtToken, app);
        if (!isInvalidatedOnAnotherInstance) {
            return Boolean.FALSE;
        }
    }
    // ... rest of invalidation logic
}

A defense-in-depth fix would also add an explicit empty/null check at the top of JWTLogoutHandler.logout() and in invalidateJwtToken() before any processing begins.

[balhar-jakub]

Problem Analysis — Logout Accepts Empty Token in Modulith HA (#4754)

Refined Problem Statement

When the API Mediation Layer runs in modulith High Availability (HA) mode (multiple Gateway instances, combined Gateway+ZAAS), sending a logout request with an empty JWT token returns 204 No Content (success) instead of an authentication error. In single-instance mode, the same request correctly throws a BadCredentialsException.

Real-world scenario: A misconfigured client or an automated script sends apimlAuthenticationToken= (empty cookie) or Bearer (empty Authorization header) to the logout endpoint. In a production HA deployment, the logout appears to succeed silently — no error log, no failure response. This creates a false sense of security: the administrator may believe the session was terminated when it was not.

Current Architecture Touchpoints

The logout flow touches four layers:

1. Client transport (ZaasJwtService.java:233): No client-side empty-token validation — sends empty string as apimlAuthenticationToken= directly to the logout endpoint.

2. Spring Security filter chain (NewSecurityConfiguration.java:148): The login+logout filter chain uses .anyRequest().permitAll() — no token presence validation at the filter level. Spring Security delegates entirely to JWTLogoutHandler.

3. Logout handler (JWTLogoutHandler.java:39–45): Calls getJwtTokenFromRequest() which filters empty cookies (line 548: !cookie.getValue().isEmpty()). For an empty string, token.isPresent() returns false and the handler correctly calls failure.onAuthenticationFailure(). This path works correctly in both single-instance and HA modes.

4. Authentication service — distribution logic (AuthenticationService.java:188–241): The root cause lies here. In modulith HA:

   • invalidateJwtToken() targets GATEWAY instances (line 195) instead of ZAAS
   • invalidateTokenOnAnotherInstance() (line 285–309) distributes a DELETE /invalidate/<empty-token> to peer instances
   • The peer's AuthController.invalidate() endpoint processes the call — parseJwtToken("") throws BadCredentialsException, caught by the HttpClientErrorException handler at line 301–303, setting returnValue = false
   • But: on the ORIGINAL instance, isInvalidatedOnAnotherInstance is set to true if ANY peer responded (line 214–218). The original instance then calls parseJwtToken("") at line 221, which throws BadCredentialsException
   • The catch block at 234–238 suppresses the exception when isInvalidatedOnAnotherInstance is true ← THIS is the root cause

   In single-instance mode, there are no peer instances to distribute to, so isInvalidatedOnAnotherInstance stays false, and the BadCredentialsException correctly propagates.

Scope

In scope:

• Fix AuthenticationService.doInvalidate() so empty/unparseable tokens are rejected before peer distribution
• Add defense-in-depth empty/null check at the top of the invalidation path
• Verify the fix works in both single-instance and HA modes
• Update ZaasClientIntegrationTest expectations if behavior changes

Out of scope:

• Changes to the Spring Security filter chain (.permitAll() is intentional — the logout handler owns validation)
• Client-side validation in ZaasJwtService (separate concern; the server must protect itself regardless of client behavior)
• Changes to how cookies are parsed or the getTokenFromCookie filter

Acceptance Criteria

1. AC1: Sending a logout request with an empty token string ("") in modulith HA mode returns an authentication error (4xx), not 204 No Content.
2. AC2: Sending a logout request with a null token in modulith HA mode returns an authentication error (4xx).
3. AC3: The fix does not break normal logout with a valid token in HA mode (peer distribution still works correctly).
4. AC4: The fix does not change behavior in single-instance mode (already correct — regression gate).
5. AC5: ZaasClientIntegrationTest.givenInvalidTokenBut_thenExceptionIsThrown() passes in both HA and single-instance configurations.
6. AC6: All existing AuthenticationService unit tests pass without modification to test logic (test assertions should still be valid; test setup may need HA-mode mocking if new validation path is introduced).

Design Questions for the Architect

1. Token validation placement: Where should the empty/null token check live — in invalidateJwtToken() (before the HA distribution path), in doInvalidate() (before parseJwtToken), or in both for defense-in-depth? The issue suggests validating before peer distribution to avoid wasted network calls.

2. Exception type: What exception should be thrown for an empty/null token? TokenNotProvidedException (used by JWTLogoutHandler for missing tokens) or BadCredentialsException (thrown by parseJwtToken for unparseable input)? This affects the HTTP status code and error response format.

3. Error suppression logic: Should the catch (BadCredentialsException) block at AuthenticationService.java:234 be removed entirely, or should it be narrowed? The current logic exists so that if peer invalidation succeeded but local parsing failed (e.g., race condition with token expiry), the logout can still report success. Is this scenario realistic enough to keep?

4. Gateway vs ZAAS targeting: In modulith HA, invalidateJwtToken() targets GATEWAY instances. Is there a scenario where an empty token reaches invalidateJwtToken() directly (bypassing JWTLogoutHandler), perhaps via the /invalidate/<token> endpoint called by a peer?

5. Cache eviction: Currently doInvalidateAndUpdateCaches() only evicts the cache when result == TRUE. If we reject early on an empty token, the cache is never touched — is this correct, or should we also handle the edge case where an empty token somehow entered the cache?

6. Logging: Should an empty or null token at the AuthenticationService level be logged at WARN level (potential misconfiguration/attack probe) or at DEBUG (normal noise)?

Constraints

• Must not change the .permitAll() configuration in NewSecurityConfiguration.java — login/logout endpoints are intentionally open at the filter level
• Must not change the cookie parsing filter in getTokenFromCookie — empty-value cookie filtering is correct behavior for the request-extraction phase
• Must maintain backward compatibility with existing ZAAS client libraries
• Java 17, Spring Boot 3.x, Spring Security 6.x

[balhar-jakub]

Solution Design — Logout Accepts Empty Token in Modulith HA (#4754)

Problem Summary

In modulith HA mode, sending apimlAuthenticationToken= (empty cookie) or Bearer (empty header) to the logout endpoint returns 204 No Content instead of a 4xx authentication error. The root cause is that AuthenticationService.doInvalidate() distributes invalid tokens to peers BEFORE validating them, and the validation error is silently swallowed by logic intended for the legitimate cross-instance invalidation flow.

Root Cause (Code-Level Trace)

Entry point divergence: The Gateway's HttpUtils.getCookieValue() (line 75-78) does NOT filter empty cookie values:

// Gateway — HttpUtils.java:75-78
return Mono.justOrEmpty(exchange)
    .mapNotNull(ex -> ex.getRequest().getCookies().getFirst(cookieName))
    .map(HttpCookie::getValue);  // ← no empty-value filter!

Compare with ZAAS AuthenticationService.getTokenFromCookie() (line 548) which filters: .filter(cookie -> !cookie.getValue().isEmpty()). This allows the empty string "" to reach the invalidation logic on the Gateway side.

Silent Boolean.FALSE: In doInvalidate() (line 212-241):

1. invalidateTokenOnAnotherInstance("", app) is called first (line 215). In HA with peers:
   • Peer's /gateway/api/v1/auth/invalidate/ receives empty token
   • parseJwtToken("") → TokenNotValidException → peer returns 400 BAD_REQUEST
   • Caller gets HttpClientErrorException → returnValue = false
2. isInvalidatedOnAnotherInstance = false
3. Line 217: return Boolean.FALSE; — execution stops here
4. Back in the Gateway's LogoutHandler: no exception, Mono.empty() → Spring treats as successful logout → 204

In single-instance mode the bug does not occur because application.getInstances() only contains self (skipped), so returnValue stays true, isInvalidatedOnAnotherInstance is true, parseJwtToken("") throws TokenNotValidException which propagates to LogoutHandler where it's caught and translated to a proper error response.

Note on exception types: parseJwtToken("") throws TokenNotValidException (not BadCredentialsException as originally speculated). TokenNotValidException extends AuthenticationException. The catch (BadCredentialsException) at line 234 was originally intended for ZOSMF invalidation failures after successful token parsing — it does not catch TokenNotValidException, which propagates right past it.

Solution Design

Fix 1 (PRIMARY): Validate token BEFORE distribution in doInvalidate()

Move parseJwtToken() to the top of doInvalidate(), BEFORE the peer distribution call. This is the most targeted fix and directly addresses the root cause.

File: zaas-service/src/main/java/org/zowe/apiml/zaas/security/service/AuthenticationService.java
Method: doInvalidate() (lines 212–241)

Current code (simplified):

private Boolean doInvalidate(String jwtToken, boolean distribute, Application app) {
    boolean isInvalidatedOnAnotherInstance = false;
    if (distribute) {
        isInvalidatedOnAnotherInstance = invalidateTokenOnAnotherInstance(jwtToken, app);
        if (!isInvalidatedOnAnotherInstance) return Boolean.FALSE;
    }

    final QueryResponse queryResponse = parseJwtToken(jwtToken).getQueryResponse();
    // ... local invalidation ...
}

New code:

private Boolean doInvalidate(String jwtToken, boolean distribute, Application app) {
    // Validate token format FIRST — before any peer calls
    // If token is unparseable, throw immediately with descriptive exception
    final QueryResponse queryResponse;
    try {
        queryResponse = parseJwtToken(jwtToken).getQueryResponse();
    } catch (TokenNotValidException | TokenExpireException | TokenFormatNotValidException e) {
        throw e; // Always propagate — never suppress based on HA state
    }

    boolean isInvalidatedOnAnotherInstance = false;
    if (distribute) {
        isInvalidatedOnAnotherInstance = invalidateTokenOnAnotherInstance(jwtToken, app);
        if (!isInvalidatedOnAnotherInstance) {
            return Boolean.FALSE;
        }
    }

    // Local invalidation using already-parsed queryResponse...
    try {
        switch (queryResponse.getSource()) {
            case ZOWE:
                final String ltpaToken = getLtpaToken(jwtToken);
                if (ltpaToken != null) zosmfService.invalidate(LTPA, ltpaToken);
                break;
            case ZOSMF:
                zosmfService.invalidate(JWT, jwtToken);
                break;
            default:
                throw new TokenFormatNotValidException("Unknown token type.");
        }
    } catch (BadCredentialsException e) {
        // ZOSMF rejected the token. If a peer already invalidated it, the
        // distributed invalidation is sufficient — suppress this secondary failure.
        if (!isInvalidatedOnAnotherInstance) {
            throw e;
        }
    }

    return Boolean.TRUE;
}

Why this works: Invalid tokens are rejected before any peer distribution:

• TokenNotValidException (empty/malformed) → propagates to LogoutHandler → translated to 401
• Peer calls only happen for parseable tokens → no wasted network calls
• BadCredentialsException catch retained for legitimate case where ZOSMF rejects a locally-parseable token after successful peer distribution

Fix 2 (DEFENSE-IN-DEPTH): Null/empty token check in invalidateJwtToken()

Add an explicit guard at the entry point to reject null or blank tokens before any processing begins.

File: Same as Fix 1, method invalidateJwtToken() (line 188)
Add after log.debug(...) line:

public Boolean invalidateJwtToken(String jwtToken, boolean distribute) {
    log.debug("Invalidating JWT: ...{}", StringUtils.right(jwtToken, 15));
    if (jwtToken == null || jwtToken.isEmpty()) {
        throw new TokenNotProvidedException("No JWT token provided for invalidation");
    }
    // ... existing code ...
}

Same check should be added to invalidateJwtTokenGateway() (line 266) since it has a separate entry point used by the Gateway side.

Why: Defense-in-depth. Even if the main fix catches it in doInvalidate(), this provides an explicit, early rejection with the appropriate exception type (TokenNotProvidedException).

Fix 3 (DEFENSE-IN-DEPTH): Filter empty cookie values in Gateway HttpUtils

Make the Gateway's cookie extraction consistent with the ZAAS version by filtering empty values.

File: apiml/src/main/java/org/zowe/apiml/util/HttpUtils.java
Method: getCookieValue() (line 75-78)

Change from:

public Mono<String> getCookieValue(ServerWebExchange exchange, String cookieName) {
    return Mono.justOrEmpty(exchange)
        .mapNotNull(ex -> ex.getRequest().getCookies().getFirst(cookieName))
        .map(HttpCookie::getValue);
}

Change to:

public Mono<String> getCookieValue(ServerWebExchange exchange, String cookieName) {
    return Mono.justOrEmpty(exchange)
        .mapNotNull(ex -> ex.getRequest().getCookies().getFirst(cookieName))
        .map(HttpCookie::getValue)
        .filter(value -> !value.isEmpty());
}

Why: Prevents empty cookies from reaching the authentication service at the earliest possible point. Consistent with ZAAS getTokenFromCookie() behavior. Also handles the Bearer header case since getBearerTokenFromHeaderReactive() already filters .filter(token -> !token.isBlank()).

Design Questions Answered

1. Token validation placement: Primary validation in doInvalidate() before distribution (Fix 1), with defense-in-depth null/empty checks at both invalidateJwtToken() and invalidateJwtTokenGateway() entry points (Fix 2), plus Gateway cookie filtering (Fix 3).

2. Exception type: TokenNotProvidedException for null/empty at the entry points (matches JWTLogoutHandler behavior). TokenNotValidException from parseJwtToken() for unparseable tokens will naturally propagate. Both extend AuthenticationException, producing 401 responses.

3. Error suppression logic: The catch (BadCredentialsException) at line 234 is retained but is now safe: since we validate the token before distribution, parseJwtToken() already succeeded. The catch only handles the case where ZOSMF rejects a previously-valid token after peer distribution — a legitimate secondary-failure scenario. The catch at line 234 should NOT be removed; only its position in the method changes.

4. Gateway vs ZAAS targeting: Yes, an empty token reaches invalidateJwtToken() directly via the Gateway's /gateway/api/v1/auth/logout endpoint because HttpUtils.getCookieValue() doesn't filter empty values. This is the primary entry point in modulith HA. Fix 3 addresses this at the source. The /invalidate/<token> peer-to-peer endpoint is also affected but is addressed by Fix 1 (pre-distribution validation rejects the token before the peer call is made).

5. Cache eviction: Correct. When rejecting early on empty/null tokens, the cache is never touched. This is correct behavior — there is nothing to evict or mark as invalidated.

6. Logging: At the AuthenticationService level, empty/null token should be logged at DEBUG level. These can occur from misconfigured clients or normal browser behavior (browsers may send cookies with empty values on redirects). WARN would create noise. The TokenNotProvidedException thrown will produce a proper 401 response; the client error tells the story.

Independent Implementation Areas

1. AuthenticationService.doInvalidate() restructure — Move parseJwtToken() before distribution (Fix 1). This is the critical fix.
2. AuthenticationService entry-point guards — Add null/empty checks to invalidateJwtToken() and invalidateJwtTokenGateway() (Fix 2).
3. HttpUtils.getCookieValue() filter — Add empty-value filter to Gateway cookie extraction (Fix 3).

Note: All three areas are in the same codebase module (zaas-service and apiml) and can be implemented in a single engineering pass. Listed separately so the PM can decompose if the architect decides areas should be handled independently.

Edge Cases and Risks

Edge Case	Behavior After Fix
Empty string token ""	TokenNotProvidedException (null/empty guard) or TokenNotValidException (parseJwtToken) → 401 response
Null token	TokenNotProvidedException → 401 response
Whitespace-only token " "	TokenNotValidException from parseJwtToken() → 401 response
Valid token, HA mode	No change. Parse succeeds → distribute to peers → local invalidation → 204
Valid token, HA mode, ZOSMF rejects after peer success	Peer distribution succeeds. BadCredentialsException from ZOSMF is suppressed (distribution already happened). Logout reports success. Same as current behavior.
Valid token, single instance	No change. Already works correctly — regression gate.
ZaasClientIntegrationTest.givenInvalidTokenBut_thenExceptionIsThrown()	Will pass because TokenNotValidException now propagates before distribution.

Required Tests

• Unit: AuthenticationServiceTest — verify doInvalidate("") throws TokenNotValidException before distribution (mock parseJwtToken to throw, verify invalidateTokenOnAnotherInstance is never called)
• Unit: AuthenticationServiceTest — verify invalidateJwtToken("") throws TokenNotProvidedException
• Unit: AuthenticationServiceTest — verify doInvalidate(validToken) with HA mock still calls distribution first (regression)
• Unit: HttpUtilsTest — verify getCookieValue() returns Mono.empty() for empty cookie value
• Integration: ZaasClientIntegrationTest.givenInvalidTokenBut_thenExceptionIsThrown() — already exists, must pass

Files Changed

1. zaas-service/src/main/java/org/zowe/apiml/zaas/security/service/AuthenticationService.java — Fix 1 and Fix 2
2. apiml/src/main/java/org/zowe/apiml/util/HttpUtils.java — Fix 3
3. zaas-service/src/test/java/org/zowe/apiml/zaas/security/service/AuthenticationServiceTest.java — new unit tests (Fix 1 coverage)
4. apiml/src/test/java/org/zowe/apiml/util/HttpUtilsTest.java — new unit tests (Fix 3 coverage)

[balhar-jakub]

Decomposition — #4754

The architect's solution design identified three fixes that share the same codebase and can be implemented as a single engineering pass:

1. Fix 1 (PRIMARY): Restructure doInvalidate() — move parseJwtToken() before peer distribution
2. Fix 2 (DEFENSE-IN-DEPTH): Null/empty token guards at invalidateJwtToken() and invalidateJwtTokenGateway() entry points
3. Fix 3 (DEFENSE-IN-DEPTH): Filter empty cookie values in Gateway HttpUtils.getCookieValue()

Affected files:

• zaas-service/.../AuthenticationService.java (Fixes 1+2)
• apiml/.../HttpUtils.java (Fix 3)

Created 1 engineer task (chained): [Step 4a] Engineer — Implement token validation + entry-point guards + cookie filter for #4754

Pipeline: PM → Architect (done) → PM decompose (done) → Engineer → Architect PR/review → CI Gate → QA → Security → PM notify