[LV] Add a statistic for early exit vectorization

[annamthomas]

We currently do not vectorize the epilog loops with early-exits, but the
stats are updated there as well for completeness.

[llvmbot]

@llvm/pr-subscribers-vectorizers

@llvm/pr-subscribers-llvm-transforms

Author: None (annamthomas)

Changes

We currently do not vectorize the epilog loops with early-exits, but the
stats are updated there as well for completeness.

---

Full diff: https://github.com/llvm/llvm-project/pull/145730.diff

2 Files Affected:

• (modified) llvm/lib/Transforms/Vectorize/LoopVectorize.cpp (+11)
• (modified) llvm/test/Transforms/LoopVectorize/single_early_exit.ll (+1-1)

diff --git a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
index 5eda2003920e6..4f365b9f990bf 100644
--- a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
+++ b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
@@ -175,6 +175,8 @@ const char LLVMLoopVectorizeFollowupEpilogue[] =
 STATISTIC(LoopsVectorized, "Number of loops vectorized");
 STATISTIC(LoopsAnalyzed, "Number of loops analyzed for vectorization");
 STATISTIC(LoopsEpilogueVectorized, "Number of epilogues vectorized");
+STATISTIC(LoopsEarlyExitVectorized, "Number of early exit loops vectorized"); 
+
 
 static cl::opt<bool> EnableEpilogueVectorization(
     "enable-epilogue-vectorization", cl::init(true), cl::Hidden,
@@ -10291,6 +10293,9 @@ bool LoopVectorizePass::processLoop(Loop *L) {
         auto ExpandedSCEVs = LVP.executePlan(EPI.MainLoopVF, EPI.MainLoopUF,
                                              *BestMainPlan, MainILV, DT, false);
         ++LoopsVectorized;
+        // TODO: Currently unsupported for early exits.
+        if (BestPlan.hasEarlyExit())
+          ++LoopsEarlyExitVectorized;
 
         // Second pass vectorizes the epilogue and adjusts the control flow
         // edges from the first pass.
@@ -10319,6 +10324,10 @@ bool LoopVectorizePass::processLoop(Loop *L) {
           Inc->setIncomingValueForBlock(BypassBlock, V);
         }
         ++LoopsEpilogueVectorized;
+        // TODO: Currently unsupported for early exits.
+        if (BestEpiPlan.hasEarlyExit())
+          ++LoopsEarlyExitVectorized;
+
 
         if (!Checks.hasChecks())
           DisableRuntimeUnroll = true;
@@ -10328,6 +10337,8 @@ bool LoopVectorizePass::processLoop(Loop *L) {
                                Checks, BestPlan);
         LVP.executePlan(VF.Width, IC, BestPlan, LB, DT, false);
         ++LoopsVectorized;
+        if (BestPlan.hasEarlyExit())
+          ++LoopsEarlyExitVectorized;
 
         // Add metadata to disable runtime unrolling a scalar loop when there
         // are no runtime checks about strides and memory. A scalar loop that is
diff --git a/llvm/test/Transforms/LoopVectorize/single_early_exit.ll b/llvm/test/Transforms/LoopVectorize/single_early_exit.ll
index 00e21f102038a..c7b4a8f915405 100644
--- a/llvm/test/Transforms/LoopVectorize/single_early_exit.ll
+++ b/llvm/test/Transforms/LoopVectorize/single_early_exit.ll
@@ -1,7 +1,7 @@
 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --version 4
 ; RUN: opt -S < %s -p loop-vectorize -enable-early-exit-vectorization -force-vector-width=4 | FileCheck %s
 
-declare void @init_mem(ptr, i64);
+eclare void @init_mem(ptr, i64);
 
 
 define i64 @same_exit_block_phi_of_consts() {

[github-actions]

✅ With the latest revision this PR passed the C/C++ code formatter.

[fhahn]

Might also be worth adding a test for the new statistic

[fhahn]

Would be good to leave the other increments as-is for now.

Doesn't this change now increments LoopsVectorized twice per loop if the epilogue gets vectorized?