Skip to content

gpl: vectorize Nesterov WA wirelength force loops - #11084

Open
oharboe wants to merge 1 commit into
The-OpenROAD-Project:masterfrom
oharboe:gpl-wa-force-vectorization
Open

gpl: vectorize Nesterov WA wirelength force loops#11084
oharboe wants to merge 1 commit into
The-OpenROAD-Project:masterfrom
oharboe:gpl-wa-force-vectorization

Conversation

@oharboe

@oharboe oharboe commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Supersedes #11083.

Use OpenMP SIMD pragmas and fastExp batching to allow auto-vectorization of the exponential summations in Nesterov wirelength force computation. This reduces overhead during the iteration loop by leveraging AVX/SIMD instructions when fastExp is inlined.

The issue with the indiscriminate fastExp overflows has been fixed in this PR by guarding the SIMD evaluation with the ternary operator against minWireLengthForceBar.

I've run the full end-to-end global placement flow on a massive 2.4M-net commercial testcase with and without this patch (using hermetic x86-64-v3 compiler flags).

Dogfooding Results:

  • Baseline placement time: 6,082 seconds
  • With SIMD Vectorized WA Forces: 5,511 seconds

This represents a 571-second (9.4%) overall reduction in global placement time. The Nesterov inner loop itself sped up by nearly 2x (1.54s per iter vs 3.04s per iter).

@oharboe
oharboe requested a review from a team as a code owner August 6, 2026 07:12
@oharboe
oharboe requested a review from gudeh August 6, 2026 07:12
@github-actions github-actions Bot added the size/M label Aug 6, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request optimizes the updateWireLengthForceWA_native function in nesterovBase.cpp by refactoring the OpenMP parallel loop. It introduces thread-local vectors to avoid repeated allocations and leverages SIMD vectorization with #pragma omp simd and #pragma omp declare simd on fastExp. Feedback is provided regarding a potential floating-point overflow issue during SIMD execution due to speculative evaluation of the ternary operator branches, with a suggestion to clamp the inputs to fastExp using std::max.

Comment thread src/gpl/src/nesterovBase.cpp Outdated
@oharboe
oharboe force-pushed the gpl-wa-force-vectorization branch from 34c2577 to 482ee7e Compare August 6, 2026 07:15
Use OpenMP SIMD pragmas and fastExp batching to allow auto-vectorization of the exponential summations in Nesterov wirelength force computation. This significantly reduces overhead during the iteration loop by leveraging AVX/SIMD instructions when fastExp is inlined.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
@oharboe
oharboe force-pushed the gpl-wa-force-vectorization branch from 482ee7e to 1309e72 Compare August 6, 2026 07:16
oharboe added a commit to oharboe/OpenROAD that referenced this pull request Aug 6, 2026
Replace old OpenMP raw iterators and unchunked bounds in `updateBinsGCellDensityArea` and `updateDensityFieldBin` with modern indexed arrays and explicitly enforce `schedule(static)` across all phases of the loop.

This removes sequential clear bottlenecks per Nesterov iteration and eliminates heavy `kmp_flag_64::wait` OpenMP spin-wait barriers during density field evaluation because static scheduling forces threads to take the identical `bin` distribution across all loops without dynamic work-stealing overhead.

**Dogfooding Results**:
Combined with the The-OpenROAD-Project#11084 vectorized forces patch, running the end-to-end global placement flow on a 2.4M-net commercial testcase drops runtime by an additional **100 seconds (1.8%)**, from 5,511s down to **5,411s**. Output placement convergence and metrics remain perfectly 100% bit-identical.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
oharboe added a commit to oharboe/OpenROAD that referenced this pull request Aug 6, 2026
Replace old OpenMP raw iterators and unchunked bounds in `updateBinsGCellDensityArea` and `updateDensityFieldBin` with modern indexed arrays and explicitly enforce `schedule(static)` across all phases of the loop.

This removes sequential clear bottlenecks per Nesterov iteration and eliminates heavy `kmp_flag_64::wait` OpenMP spin-wait barriers during density field evaluation because static scheduling forces threads to take the identical `bin` distribution across all loops without dynamic work-stealing overhead.

**Dogfooding Results**:
Combined with the The-OpenROAD-Project#11084 vectorized forces patch, running the end-to-end global placement flow on a 2.4M-net commercial testcase drops runtime by an additional **100 seconds (1.8%)**, from 5,511s down to **5,411s**. Output placement convergence and metrics remain perfectly 100% bit-identical.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
oharboe added a commit to oharboe/OpenROAD that referenced this pull request Aug 6, 2026
Replace old OpenMP raw iterators and unchunked bounds in `updateBinsGCellDensityArea` and `updateDensityFieldBin` with modern indexed arrays and explicitly enforce `schedule(static)` across all phases of the loop.

This removes sequential clear bottlenecks per Nesterov iteration and eliminates heavy `kmp_flag_64::wait` OpenMP spin-wait barriers during density field evaluation because static scheduling forces threads to take the identical `bin` distribution across all loops without dynamic work-stealing overhead.

**Dogfooding Results**:
Combined with the The-OpenROAD-Project#11084 vectorized forces patch, running the end-to-end global placement flow on a 2.4M-net commercial testcase drops runtime by an additional **100 seconds (1.8%)**, from 5,511s down to **5,411s**. Output placement convergence and metrics remain perfectly 100% bit-identical.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
oharboe added a commit to oharboe/OpenROAD that referenced this pull request Aug 6, 2026
Replace old OpenMP raw iterators and unchunked bounds in `updateBinsGCellDensityArea` and `updateDensityFieldBin` with modern indexed arrays and explicitly enforce `schedule(static)` across all phases of the loop.

This removes sequential clear bottlenecks per Nesterov iteration and eliminates heavy `kmp_flag_64::wait` OpenMP spin-wait barriers during density field evaluation because static scheduling forces threads to take the identical `bin` distribution across all loops without dynamic work-stealing overhead.

**Dogfooding Results**:
Combined with the The-OpenROAD-Project#11084 vectorized forces patch, running the end-to-end global placement flow on a 2.4M-net commercial testcase drops runtime by an additional **100 seconds (1.8%)**, from 5,511s down to **5,411s**. Output placement convergence and metrics remain perfectly 100% bit-identical.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
oharboe added a commit to oharboe/OpenROAD that referenced this pull request Aug 6, 2026
Replace old OpenMP raw iterators and unchunked bounds in `updateBinsGCellDensityArea` and `updateDensityFieldBin` with modern indexed arrays and explicitly enforce `schedule(static)` across all phases of the loop.

This removes sequential clear bottlenecks per Nesterov iteration and eliminates heavy `kmp_flag_64::wait` OpenMP spin-wait barriers during density field evaluation because static scheduling forces threads to take the identical `bin` distribution across all loops without dynamic work-stealing overhead.

**Dogfooding Results**:
Combined with the The-OpenROAD-Project#11084 vectorized forces patch, running the end-to-end global placement flow on a 2.4M-net commercial testcase drops runtime by an additional **100 seconds (1.8%)**, from 5,511s down to **5,411s**. Output placement convergence and metrics remain perfectly 100% bit-identical.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
oharboe added a commit to oharboe/OpenROAD that referenced this pull request Aug 6, 2026
Replace old OpenMP raw iterators and unchunked bounds in `updateBinsGCellDensityArea` and `updateDensityFieldBin` with modern indexed arrays and explicitly enforce `schedule(static)` across all phases of the loop.

This removes sequential clear bottlenecks per Nesterov iteration and eliminates heavy `kmp_flag_64::wait` OpenMP spin-wait barriers during density field evaluation because static scheduling forces threads to take the identical `bin` distribution across all loops without dynamic work-stealing overhead.

**Dogfooding Results**:
Combined with the The-OpenROAD-Project#11084 vectorized forces patch, running the end-to-end global placement flow on a 2.4M-net commercial testcase drops runtime by an additional **100 seconds (1.8%)**, from 5,511s down to **5,411s**. Output placement convergence and metrics remain perfectly 100% bit-identical.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
oharboe added a commit to oharboe/OpenROAD that referenced this pull request Aug 6, 2026
Replace old OpenMP raw iterators and unchunked bounds in `updateBinsGCellDensityArea` and `updateDensityFieldBin` with modern indexed arrays and explicitly enforce `schedule(static)` across all phases of the loop.

This removes sequential clear bottlenecks per Nesterov iteration and eliminates heavy `kmp_flag_64::wait` OpenMP spin-wait barriers during density field evaluation because static scheduling forces threads to take the identical `bin` distribution across all loops without dynamic work-stealing overhead.

**Dogfooding Results**:
Combined with the The-OpenROAD-Project#11084 vectorized forces patch, running the end-to-end global placement flow on a 2.4M-net commercial testcase drops runtime by an additional **100 seconds (1.8%)**, from 5,511s down to **5,411s**. Output placement convergence and metrics remain perfectly 100% bit-identical.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
oharboe added a commit to oharboe/OpenROAD that referenced this pull request Aug 6, 2026
Replace old OpenMP raw iterators and unchunked bounds in `updateBinsGCellDensityArea` and `updateDensityFieldBin` with modern indexed arrays and explicitly enforce `schedule(static)` across all phases of the loop.

This removes sequential clear bottlenecks per Nesterov iteration and eliminates heavy `kmp_flag_64::wait` OpenMP spin-wait barriers during density field evaluation because static scheduling forces threads to take the identical `bin` distribution across all loops without dynamic work-stealing overhead.

**Dogfooding Results**:
Combined with the The-OpenROAD-Project#11084 vectorized forces patch, running the end-to-end global placement flow on a 2.4M-net commercial testcase drops runtime by an additional **100 seconds (1.8%)**, from 5,511s down to **5,411s**. Output placement convergence and metrics remain perfectly 100% bit-identical.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
oharboe added a commit to oharboe/OpenROAD that referenced this pull request Aug 6, 2026
Replace old OpenMP raw iterators and unchunked bounds in `updateBinsGCellDensityArea` and `updateDensityFieldBin` with modern indexed arrays and explicitly enforce `schedule(static)` across all phases of the loop.

This removes sequential clear bottlenecks per Nesterov iteration and eliminates heavy `kmp_flag_64::wait` OpenMP spin-wait barriers during density field evaluation because static scheduling forces threads to take the identical `bin` distribution across all loops without dynamic work-stealing overhead.

**Dogfooding Results**:
Combined with the The-OpenROAD-Project#11084 vectorized forces patch, running the end-to-end global placement flow on a 2.4M-net commercial testcase drops runtime by an additional **100 seconds (1.8%)**, from 5,511s down to **5,411s**. Output placement convergence and metrics remain perfectly 100% bit-identical.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
oharboe added a commit to oharboe/OpenROAD that referenced this pull request Aug 6, 2026
Replace old OpenMP raw iterators and unchunked bounds in `updateBinsGCellDensityArea` and `updateDensityFieldBin` with modern indexed arrays and explicitly enforce `schedule(static)` across all phases of the loop.

This removes sequential clear bottlenecks per Nesterov iteration and eliminates heavy `kmp_flag_64::wait` OpenMP spin-wait barriers during density field evaluation because static scheduling forces threads to take the identical `bin` distribution across all loops without dynamic work-stealing overhead.

**Dogfooding Results**:
Combined with the The-OpenROAD-Project#11084 vectorized forces patch, running the end-to-end global placement flow on a 2.4M-net commercial testcase drops runtime by an additional **100 seconds (1.8%)**, from 5,511s down to **5,411s**. Output placement convergence and metrics remain perfectly 100% bit-identical.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
oharboe added a commit to oharboe/OpenROAD that referenced this pull request Aug 6, 2026
Replace old OpenMP raw iterators and unchunked bounds in `updateBinsGCellDensityArea` and `updateDensityFieldBin` with modern indexed arrays and explicitly enforce `schedule(static)` across all phases of the loop.

This removes sequential clear bottlenecks per Nesterov iteration and eliminates heavy `kmp_flag_64::wait` OpenMP spin-wait barriers during density field evaluation because static scheduling forces threads to take the identical `bin` distribution across all loops without dynamic work-stealing overhead.

**Dogfooding Results**:
Combined with the The-OpenROAD-Project#11084 vectorized forces patch, running the end-to-end global placement flow on a 2.4M-net commercial testcase drops runtime by an additional **100 seconds (1.8%)**, from 5,511s down to **5,411s**. Output placement convergence and metrics remain perfectly 100% bit-identical.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
oharboe added a commit to oharboe/OpenROAD that referenced this pull request Aug 6, 2026
Replace old OpenMP raw iterators and unchunked bounds in `updateBinsGCellDensityArea` and `updateDensityFieldBin` with modern indexed arrays and explicitly enforce `schedule(static)` across all phases of the loop.

This removes sequential clear bottlenecks per Nesterov iteration and eliminates heavy `kmp_flag_64::wait` OpenMP spin-wait barriers during density field evaluation because static scheduling forces threads to take the identical `bin` distribution across all loops without dynamic work-stealing overhead.

**Dogfooding Results**:
Combined with the The-OpenROAD-Project#11084 vectorized forces patch, running the end-to-end global placement flow on a 2.4M-net commercial testcase drops runtime by an additional **100 seconds (1.8%)**, from 5,511s down to **5,411s**. Output placement convergence and metrics remain perfectly 100% bit-identical.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
oharboe added a commit to oharboe/OpenROAD that referenced this pull request Aug 6, 2026
Replace old OpenMP raw iterators and unchunked bounds in `updateBinsGCellDensityArea` and `updateDensityFieldBin` with modern indexed arrays and explicitly enforce `schedule(static)` across all phases of the loop.

This removes sequential clear bottlenecks per Nesterov iteration and eliminates heavy `kmp_flag_64::wait` OpenMP spin-wait barriers during density field evaluation because static scheduling forces threads to take the identical `bin` distribution across all loops without dynamic work-stealing overhead.

**Dogfooding Results**:
Combined with the The-OpenROAD-Project#11084 vectorized forces patch, running the end-to-end global placement flow on a 2.4M-net commercial testcase drops runtime by an additional **100 seconds (1.8%)**, from 5,511s down to **5,411s**. Output placement convergence and metrics remain perfectly 100% bit-identical.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
oharboe added a commit to oharboe/OpenROAD that referenced this pull request Aug 6, 2026
Replace old OpenMP raw iterators and unchunked bounds in `updateBinsGCellDensityArea` and `updateDensityFieldBin` with modern indexed arrays and explicitly enforce `schedule(static)` across all phases of the loop.

This removes sequential clear bottlenecks per Nesterov iteration and eliminates heavy `kmp_flag_64::wait` OpenMP spin-wait barriers during density field evaluation because static scheduling forces threads to take the identical `bin` distribution across all loops without dynamic work-stealing overhead.

**Dogfooding Results**:
Combined with the The-OpenROAD-Project#11084 vectorized forces patch, running the end-to-end global placement flow on a 2.4M-net commercial testcase drops runtime by an additional **100 seconds (1.8%)**, from 5,511s down to **5,411s**. Output placement convergence and metrics remain perfectly 100% bit-identical.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
@maliberty

Copy link
Copy Markdown
Member

Be aware that not everyone has avx512 capable CPUs (see The-OpenROAD-Project/OpenROAD-flow-scripts#4379). Which subset of avx are you using?

@oharboe

oharboe commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator Author

Be aware that not everyone has avx512 capable CPUs (see The-OpenROAD-Project/OpenROAD-flow-scripts#4379). Which subset of avx are you using?

This just makes it possible/worthwhile to compile for whatever vector instructions you have. We will use the smallest common denominator for our servers, but I didnt check what that was, I just started with some conservative settings in my .bazelrc file.

@maliberty

Copy link
Copy Markdown
Member

@codex review

@maliberty

Copy link
Copy Markdown
Member

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request optimizes the updateWireLengthForceWA_native function in nesterovBase.cpp by refactoring the OpenMP parallel loop to utilize SIMD vectorization and thread-local buffers, and by declaring fastExp as a SIMD function. The feedback suggests two key improvements: declaring the thread-private vectors as static thread_local to avoid repeated heap allocations across function calls, and simplifying the SIMD loop by removing redundant ternary operators.

Comment on lines +1426 to +1428
std::vector<float> cx, cy;
std::vector<float> expMinX, expMaxX, expMinY, expMaxY;
std::vector<float> expMinX_fast, expMaxX_fast, expMinY_fast, expMaxY_fast;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Declaring these thread-private vectors as static thread_local avoids allocating and deallocating them on the heap for every thread on every single call to updateWireLengthForceWA_native. Since this function is called in the inner loop of the Nesterov placement across hundreds of iterations, eliminating these repeated heap allocations significantly reduces thread contention and overhead, further boosting performance.

    static thread_local std::vector<float> cx, cy;
    static thread_local std::vector<float> expMinX, expMaxX, expMinY, expMaxY;
    static thread_local std::vector<float> expMinX_fast, expMaxX_fast, expMinY_fast, expMaxY_fast;

Comment on lines +1475 to +1486
expMinX_fast[i] = (expMinX[i] > minBar)
? fastExp(std::max(expMinX[i], minBar))
: 0.0f;
expMaxX_fast[i] = (expMaxX[i] > minBar)
? fastExp(std::max(expMaxX[i], minBar))
: 0.0f;
expMinY_fast[i] = (expMinY[i] > minBar)
? fastExp(std::max(expMinY[i], minBar))
: 0.0f;
expMaxY_fast[i] = (expMaxY[i] > minBar)
? fastExp(std::max(expMaxY[i], minBar))
: 0.0f;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Since expMinX_fast[i] (and the other fast exponential values) are only ever read in the subsequent loop when expMinX[i] > minBar, the ternary operator is redundant. Removing it and directly assigning the result of fastExp(std::max(..., minBar)) simplifies the code and avoids redundant mask generation and blend-with-zero instructions in the vectorized SIMD loop. The std::max clamp is still sufficient to prevent any overflow/infinity issues in the masked lanes.

        expMinX_fast[i] = fastExp(std::max(expMinX[i], minBar));
        expMaxX_fast[i] = fastExp(std::max(expMaxX[i], minBar));
        expMinY_fast[i] = fastExp(std::max(expMinY[i], minBar));
        expMaxY_fast[i] = fastExp(std::max(expMaxY[i], minBar));

@oharboe

oharboe commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator Author

@maliberty Can you have your superior AI complete this PR?

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1309e7293a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +1454 to +1457
float nlx = gNet->lx();
float nux = gNet->ux();
float nly = gNet->ly();
float nuy = gNet->uy();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep coordinate subtraction in the integer domain

For designs whose DBU coordinates exceed float's exact-integer range, converting each bound and pin coordinate to float before subtraction loses relative-distance information. For example, 16,777,216 and 16,777,217 round to the same value, so the new code computes a zero exponent where the previous integer subtraction produced a distance of one; this alters the WA gradient and makes placement results depend on the design's absolute origin. Preserve the coordinates as integers through the subtraction (or use a sufficiently precise type) before converting the difference to float.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants