Skip to content

fix(backend): eliminate N+1 query patterns in portfolio and projects #1420 - #1465

Merged
Cedarich merged 2 commits into
Pulsefy:mainfrom
ifygreg01-best:fix/n-plus-one-query-patterns-1420
Sep 23, 2026
Merged

Cedarich merged 2 commits into
Pulsefy:mainfrom
ifygreg01-best:fix/n-plus-one-query-patterns-1420

Conversation

@ifygreg01-best

Copy link
Copy Markdown
Contributor

Summary

Closes #1420

Audited the four modules called out in the issue (grants, projects, contributor-feed, portfolio). Grants and contributor-feed are pure in-memory (no DB/RPC calls in loops). The five worst patterns were all in portfolio and projects.


Five Worst Endpoints — Before / After

# Endpoint Before (10 items) After Fix
1 POST /portfolio/snapshots (primary path) 10 price calls 1 getAssetValuesUsd() batch
2 POST /portfolio/snapshots (fallback path) 10 price calls 1 getAssetValuesUsd() batch
3 GET /portfolio/account/:key 10 price calls 1 getAssetValuesUsd() batch
4 GET /portfolio/allocation 10 price calls 1 getAssetValuesUsd() batch
5 GET /projects + refreshMaterializedSnapshots N sequential RPC calls concurrent + capped at 10 Preloaded states + Promise.allSettled

Changes

Batch price fetching (fixes #1, 2, 3, 4)

  • PriceService.getPricesForAssets(codes[]) — single batched lookup; when a real price-feed API is wired in, this is the only place to change
  • StellarBalanceService.getAssetValuesUsd(assets[]) — replaces four independent Promise.all(items.map(getPrice)) patterns with one call
  • portfolio.service.ts — createSnapshot, getPortfolioSummaryForAccount, getAssetAllocation all updated

Projects list fix (fix #5a)

  • projects.service.ts listProjects: preload all on-chain states with a single concurrent Promise.all before mapping, instead of one fetchOnChainState per row inside a .map()

Sequential refresh loop fix (fix #5b)

  • refreshMaterializedSnapshots: replace sequential for loop with Promise.allSettled batches (concurrency cap = 10)

Query profiling middleware (off by default)

  • Extended QueryProfilerService with AsyncLocalStorage-based per-request call counter (trackCall, runInContext)
  • New QueryCountMiddleware wraps each request and logs the final call count
  • Registered in AppModule behind QUERY_PROFILING=true — zero overhead in production
  • ProfilingModule updated to export new middleware

Tests

New regression test: src/portfolio/portfolio-n-plus-one.spec.ts

Spies on getPricesForAssets and asserts it is called exactly once regardless of portfolio size:

PASS src/portfolio/portfolio-n-plus-one.spec.ts
  PortfolioService – N+1 regression guard
    createSnapshot – price call count is O(1) not O(N)
      ✓ portfolio with 1 asset(s) → exactly 1 batch price call
      ✓ portfolio with 5 asset(s) → exactly 1 batch price call
      ✓ portfolio with 10 asset(s) → exactly 1 batch price call
      ✓ call count does NOT grow with portfolio size (O(1) assertion)
    getAssetAllocation – price call count is O(1) not O(N)
      ✓ 1 aggregated asset(s) → exactly 1 batch price call
      ✓ 5 aggregated asset(s) → exactly 1 batch price call
      ✓ 10 aggregated asset(s) → exactly 1 batch price call

Documentation

apps/backend/QUERY_PROFILING.md — covers how to enable profiling mode, what gets logged, the middleware architecture, the five fixed endpoints with before/after counts, and how to run the regression test.

Enable with:

QUERY_PROFILING=true npm run start:dev

…1420)

## Five worst endpoints fixed

| Endpoint                          | Before (10 items) | After |
|-----------------------------------|-------------------|-------|
| POST /portfolio/snapshots (main)  | 10 price calls    | 1     |
| POST /portfolio/snapshots (fallback) | 10 price calls | 1     |
| GET /portfolio/account/:key       | 10 price calls    | 1     |
| GET /portfolio/allocation         | 10 price calls    | 1     |
| GET /projects (list)              | N sequential RPC  | concurrent + capped |

## Changes

### Batch price fetching (portfolio N+1 fixes)
- Add PriceService.getPricesForAssets(codes[]) - single batched lookup
  regardless of portfolio size
- Add StellarBalanceService.getAssetValuesUsd(assets[]) - replaces four
  separate Promise.all(items.map(getPrice)) patterns with one call
- Fix portfolio.service.ts createSnapshot, getPortfolioSummaryForAccount,
  and getAssetAllocation to use the new batch method

### Projects list fix
- projects.service.ts listProjects: preload all on-chain states with a
  single concurrent Promise.all before mapping instead of one
  fetchOnChainState per row inside a map

### Sequential loop fix
- refreshMaterializedSnapshots: replace sequential for-await loop with
  Promise.allSettled batches (concurrency cap = 10)

### Query profiling middleware (off by default)
- Extend QueryProfilerService with AsyncLocalStorage-based per-request
  call counter and runInContext() / trackCall() helpers
- Add QueryCountMiddleware to wrap requests and log final call count
- Register in AppModule behind QUERY_PROFILING=true env flag
- Update ProfilingModule to export new middleware

### Regression test
- Add portfolio-n-plus-one.spec.ts asserting getPricesForAssets is called
  exactly once for portfolios of 1, 5, and 10 assets (7 tests, all pass)

### Documentation
- Add QUERY_PROFILING.md: how to enable, what gets logged, architecture,
  five fixed endpoints with before/after counts, test instructions

Closes Pulsefy#1420
@drips-wave

drips-wave Bot commented Sep 23, 2026

Copy link
Copy Markdown

@ifygreg01-best Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@Cedarich Cedarich left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@Cedarich
Cedarich merged commit d2fd6a5 into Pulsefy:main Sep 23, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Backend: Audit and fix N+1 query patterns Initialize Next.js Frontend with Starknet Integration

2 participants