Skip to content

Conversation

@mpark
Copy link
Member

@mpark mpark commented Dec 11, 2025

Problem

Given code such as N::foo();, we perform name look-up on N. In the case where N is a namespace declared in imported modules, one namespace decl (the "key declaration") for each module that declares a namespace foo is loaded and stored. In large scales where there are many such modules, (e.g., 1,500) and many uses (e.g., 500,000), this becomes extremely inefficient because every look-up (500,000 of them) return 1,500 results.

The following synthetic script demonstrates the problem:

#/usr/bin/env bash

CLANG=${CLANG:-clang++}
NUM_MODULES=${NUM_MODULES:-1500}
NUM_USES=${NUM_USES:-500000}
USE_MODULES=${USE_MODULES:-true}

TMPDIR=$(mktemp -d)
echo "Working in temp directory: $TMPDIR"
cd $TMPDIR

trap "rm -rf \"$TMPDIR\"" EXIT

echo "namespace N { inline void foo() {} }" > m1.h
for i in $(seq 2 $NUM_MODULES); do echo "namespace N {}" > m${i}.h; done

if $USE_MODULES; then
  seq 1 $NUM_MODULES | xargs -I {} -P $(nproc) bash -c "$CLANG -std=c++20 -fmodule-header m{}.h"
fi

> a.cpp
if $USE_MODULES; then
  for i in $(seq 1 $NUM_MODULES); do echo "import \"m${i}.h\";" >> a.cpp; done
else
  for i in $(seq 1 $NUM_MODULES); do echo "#include \"m${i}.h\"" >> a.cpp; done
fi

echo "int main() {" >> a.cpp
for i in $(seq 1 $NUM_USES); do echo "  N::foo();" >> a.cpp; done
echo "}" >> a.cpp

if $USE_MODULES; then
  time $CLANG -std=c++20 -Wno-experimental-header-units -c a.cpp -o /dev/null \
      $(for i in $(seq 1 $NUM_MODULES); do echo -n "-fmodule-file=m${i}.pcm "; done)
else
  time $CLANG -std=c++20 -Wno-experimental-header-units -c a.cpp -o /dev/null
fi

As of 575d689, without modules (USE_MODULES=false) this takes about 4.5s, whereas with modules (USE_MODULES=true), this takes about 37s.

With this PR, without modules there's no change (as expected) at 4.5s, but with modules it improves to about 5.2s.

Approach

The approach taken here aims to maintain status-quo with respect to the input and output of modules. That is, the ASTReader and ASTWriter both read and write the same declarations as it did before. The difference is in the middle part: the StoredDeclsMap in DeclContext. The StoredDeclsMap is roughly a map<DeclarationName, StoredDeclsList>. Currently, we read all of the external namespace decls from ASTReader, they all get stored into the StoredDeclsList, and the ASTWriter iterates through that list and writes out the results.

This PR continues to read all of the external namespace decls from ASTReader, but only stores one namespace decl in the StoredDeclsList. This is okay since the reading of the decls handles all of the merging and chaining of the namespace decls, and as long as they're loaded and chained, returning one for look-up purposes is sufficient.

The other half of the problem is to write out all of the external namespaces that we used to store in StoredDeclsList but no longer. For this, we take advantage of the KeyDecls data structure in ASTReader. KeyDecls is roughly a map<Decl *, vector<GlobalDeclID>>, and it stores a mapping from the canonical decl of a redeclarable decl to a list of GlobalDeclIDs where each ID represents a "key declaration" from each imported module. More to the point, if we read external namespaces N1, N2, N3 in ASTReader, we'll either have N1 mapped to [N2, N3], or some newly local canonical decl mapped to [N1, N2, N3]. Either way, we can visit N1, N2, and N3 by doing ASTReader::forEachImportedKeyDecls(N1, Visitor), and we leverage this to maintain the current behavior of writing out all of the imported namespace decls in ASTWriter.

Alternatives Attempted

  • Tried reading fewer declarations on the ASTReader side, and writing out fewer declarations on the ASTWriter side, and neither options worked at all.
  • Tried trying to split StoredDeclsList into two pieces, one with non-namespace decls and one with only namespace decls, but that didn't work well... I think because the order of the declarations matter sometimes, and maybe also because the declaration replacement logic gets more complicated.
  • Tried to deduplicate at the SemaLookup level. Basically, retrieve all the stored decls but deduplicate populating the LookupResult here. This did improve things slightly, but not quite enough, and this solution seemed cleaner in the end anyway.

@llvmbot llvmbot added clang Clang issues not falling into any other category clang:modules C++20 modules and Clang Header Modules labels Dec 11, 2025
@llvmbot
Copy link
Member

llvmbot commented Dec 11, 2025

@llvm/pr-subscribers-clang-modules

Author: Michael Park (mpark)

Changes

Problem

Given code such as N::foo();, we perform name look-up on N. In the case where N is a namespace declared in imported modules, one namespace decl (the "key declaration") for each module that declares a namespace foo is loaded and stored. In large scales where there are many such modules, (e.g., 1,500) and many uses (e.g., 500,000), this becomes extremely inefficient because every look-up (500,000 of them) return 1,500 results.

The following synthetic script demonstrates the problem:

#/usr/bin/env bash

CLANG=${CLANG:-clang++}
NUM_MODULES=${NUM_MODULES:-1500}
NUM_USES=${NUM_USES:-500000}
USE_MODULES=${USE_MODULES:-true}

TMPDIR=$(mktemp -d)
echo "Working in temp directory: $TMPDIR"
cd $TMPDIR

trap "rm -rf \"$TMPDIR\"" EXIT

echo "namespace N { inline void foo() {} }" &gt; m1.h
for i in $(seq 2 $NUM_MODULES); do echo "namespace N {}" &gt; m${i}.h; done

if $USE_MODULES; then
  seq 1 $NUM_MODULES | xargs -I {} -P $(nproc) bash -c "$CLANG -std=c++20 -fmodule-header m{}.h"
fi

&gt; a.cpp
if $USE_MODULES; then
  for i in $(seq 1 $NUM_MODULES); do echo "import \"m${i}.h\";" &gt;&gt; a.cpp; done
else
  for i in $(seq 1 $NUM_MODULES); do echo "#include \"m${i}.h\"" &gt;&gt; a.cpp; done
fi

echo "int main() {" &gt;&gt; a.cpp
for i in $(seq 1 $NUM_USES); do echo "  N::foo();" &gt;&gt; a.cpp; done
echo "}" &gt;&gt; a.cpp

if $USE_MODULES; then
  time $CLANG -std=c++20 -Wno-experimental-header-units -c a.cpp -o /dev/null \
      $(for i in $(seq 1 $NUM_MODULES); do echo -n "-fmodule-file=m${i}.pcm "; done)
else
  time $CLANG -std=c++20 -Wno-experimental-header-units -c a.cpp -o /dev/null
fi

As of 575d689, without modules (USE_MODULES=false) this takes about 4.5s, whereas with modules (USE_MODULES=true), this takes about 37s.

With this PR, without modules there's no change (as expected) at 4.5s, but with modules it improves to about 5.2s.

Approach

The approach taken here aims to maintain status-quo with respect to the input and output of modules. That is, the ASTReader and ASTWriter both read and write the same declarations as it did before. The difference is in the middle part: the StoredDeclsMap in DeclContext. The StoredDeclsMap is roughly a map&lt;DeclarationName, StoredDeclsList&gt;. Currently, we read all of the external namespace decls from ASTReader, they all get stored into the StoredDeclsList, and the ASTWriter iterates through that list and writes out the results.

This PR continues to read all of the external namespace decls from ASTReader, but only stores one namespace decl in the StoredDeclsList. This is okay since the reading of the decls handles all of the merging and chaining of the namespace decls, and as long as they're loaded and chained, returning one for look-up purposes is sufficient.

The other half of the problem is to write out all of the external namespaces that we used to have available in StoredDeclsList. For this, we take advantage of the KeyDecls data structure in ASTReader. KeyDecls is roughly a map&lt;Decl *, vector&lt;GlobalDeclID&gt;, and it stores a mapping from the canonical decl of a redeclarable decl to a list of GlobalDeclIDs where each ID represents a "key declaration" from each imported module. More to the point, if we read external namespaces N1, N2, N3 in ASTReader, we'll either have N1 mapped to [N2, N3], or some newly local canonical decl mapped to [N1, N2, N3]. Either way, we can visit N1, N2, and N3 by doing ASTReader::forEachImportedKeyDecls(N1, Visitor), and we leverage this to maintain the current behavior of writing out all of the imported namespace decls in ASTWriter.

Alternatives Attempted

I tried reading fewer declarations on the ASTReader side, and writing out fewer declarations on the ASTWriter side, and neither options worked at all. Also tried trying to split StoredDeclsList into two pieces, one with non-namespace decls and one with only namespace decls, but that didn't work well... I think because the order of the declarations matter sometimes, and maybe also because the declaration replacement logic gets more complicated.


Full diff: https://github.com/llvm/llvm-project/pull/171769.diff

2 Files Affected:

  • (modified) clang/lib/Serialization/ASTReader.cpp (+46-11)
  • (modified) clang/lib/Serialization/ASTWriter.cpp (+26-5)
diff --git a/clang/lib/Serialization/ASTReader.cpp b/clang/lib/Serialization/ASTReader.cpp
index aec61322fb8be..5f66beefd6388 100644
--- a/clang/lib/Serialization/ASTReader.cpp
+++ b/clang/lib/Serialization/ASTReader.cpp
@@ -555,7 +555,26 @@ namespace {
 
 using MacroDefinitionsMap =
     llvm::StringMap<std::pair<StringRef, bool /*IsUndef*/>>;
-using DeclsMap = llvm::DenseMap<DeclarationName, SmallVector<NamedDecl *, 8>>;
+
+class DeclsSet {
+  SmallVector<NamedDecl *, 64> Decls;
+  llvm::SmallPtrSet<NamedDecl *, 8> Found;
+
+public:
+  NamedDecl *const *data() const { return Decls.data(); }
+
+  bool empty() const { return Decls.empty(); }
+  size_t size() const { return Decls.size(); }
+
+  bool insert(NamedDecl *ND) {
+    auto [_, Inserted] = Found.insert(ND);
+    if (Inserted)
+      Decls.push_back(ND);
+    return Inserted;
+  }
+};
+
+using DeclsMap = llvm::DenseMap<DeclarationName, DeclsSet>;
 
 } // namespace
 
@@ -8702,14 +8721,22 @@ bool ASTReader::FindExternalVisibleDeclsByName(const DeclContext *DC,
     return false;
 
   // Load the list of declarations.
-  SmallVector<NamedDecl *, 64> Decls;
-  llvm::SmallPtrSet<NamedDecl *, 8> Found;
+  DeclsSet DS;
 
   auto Find = [&, this](auto &&Table, auto &&Key) {
     for (GlobalDeclID ID : Table.find(Key)) {
       NamedDecl *ND = cast<NamedDecl>(GetDecl(ID));
-      if (ND->getDeclName() == Name && Found.insert(ND).second)
-        Decls.push_back(ND);
+      if (ND->getDeclName() != Name)
+        continue;
+      // Special case for namespaces: There can be a lot of redeclarations of
+      // some namespaces, and we import a "key declaration" per imported module.
+      // Since all declarations of a namespace are essentially interchangeable,
+      // we can optimize namespace look-up by only storing the key declaration
+      // of the current TU, rather than storing N key declarations where N is
+      // the # of imported modules that declare that namespace.
+      if (isa<NamespaceDecl>(ND))
+        ND = cast<NamedDecl>(getKeyDeclaration(ND));
+      DS.insert(ND);
     }
   };
 
@@ -8744,8 +8771,8 @@ bool ASTReader::FindExternalVisibleDeclsByName(const DeclContext *DC,
     Find(It->second.Table, Name);
   }
 
-  SetExternalVisibleDeclsForName(DC, Name, Decls);
-  return !Decls.empty();
+  SetExternalVisibleDeclsForName(DC, Name, DS);
+  return !DS.empty();
 }
 
 void ASTReader::completeVisibleDeclsMap(const DeclContext *DC) {
@@ -8763,7 +8790,15 @@ void ASTReader::completeVisibleDeclsMap(const DeclContext *DC) {
 
     for (GlobalDeclID ID : It->second.Table.findAll()) {
       NamedDecl *ND = cast<NamedDecl>(GetDecl(ID));
-      Decls[ND->getDeclName()].push_back(ND);
+      // Special case for namespaces: There can be a lot of redeclarations of
+      // some namespaces, and we import a "key declaration" per imported module.
+      // Since all declarations of a namespace are essentially interchangeable,
+      // we can optimize namespace look-up by only storing the key declaration
+      // of the current TU, rather than storing N key declarations where N is
+      // the # of imported modules that declare that namespace.
+      if (isa<NamespaceDecl>(ND))
+        ND = cast<NamedDecl>(getKeyDeclaration(ND));
+      Decls[ND->getDeclName()].insert(ND);
     }
 
     // FIXME: Why a PCH test is failing if we remove the iterator after findAll?
@@ -8773,9 +8808,9 @@ void ASTReader::completeVisibleDeclsMap(const DeclContext *DC) {
   findAll(ModuleLocalLookups, NumModuleLocalVisibleDeclContexts);
   findAll(TULocalLookups, NumTULocalVisibleDeclContexts);
 
-  for (DeclsMap::iterator I = Decls.begin(), E = Decls.end(); I != E; ++I) {
-    SetExternalVisibleDeclsForName(DC, I->first, I->second);
-  }
+  for (auto &[Name, DS] : Decls)
+    SetExternalVisibleDeclsForName(DC, Name, DS);
+
   const_cast<DeclContext *>(DC)->setHasExternalVisibleStorage(false);
 }
 
diff --git a/clang/lib/Serialization/ASTWriter.cpp b/clang/lib/Serialization/ASTWriter.cpp
index 667e04049dac8..231e0a206ab11 100644
--- a/clang/lib/Serialization/ASTWriter.cpp
+++ b/clang/lib/Serialization/ASTWriter.cpp
@@ -4396,20 +4396,20 @@ class ASTDeclContextNameLookupTrait
 
   template <typename Coll> data_type getData(const Coll &Decls) {
     unsigned Start = DeclIDs.size();
-    for (NamedDecl *D : Decls) {
+    auto AddDecl = [this](NamedDecl *D) {
       NamedDecl *DeclForLocalLookup =
           getDeclForLocalLookup(Writer.getLangOpts(), D);
 
       if (Writer.getDoneWritingDeclsAndTypes() &&
           !Writer.wasDeclEmitted(DeclForLocalLookup))
-        continue;
+        return;
 
       // Try to avoid writing internal decls to reduced BMI.
       // See comments in ASTWriter::WriteDeclContextLexicalBlock for details.
       if (Writer.isGeneratingReducedBMI() &&
           !DeclForLocalLookup->isFromExplicitGlobalModule() &&
           IsInternalDeclFromFileContext(DeclForLocalLookup))
-        continue;
+        return;
 
       auto ID = Writer.GetDeclRef(DeclForLocalLookup);
 
@@ -4423,7 +4423,7 @@ class ASTDeclContextNameLookupTrait
             ModuleLocalDeclsMap.insert({Key, DeclIDsTy{ID}});
           else
             Iter->second.push_back(ID);
-          continue;
+          return;
         }
         break;
       case LookupVisibility::TULocal: {
@@ -4432,7 +4432,7 @@ class ASTDeclContextNameLookupTrait
           TULocalDeclsMap.insert({D->getDeclName(), DeclIDsTy{ID}});
         else
           Iter->second.push_back(ID);
-        continue;
+        return;
       }
       case LookupVisibility::GenerallyVisibile:
         // Generally visible decls go into the general lookup table.
@@ -4440,6 +4440,27 @@ class ASTDeclContextNameLookupTrait
       }
 
       DeclIDs.push_back(ID);
+    };
+    for (NamedDecl *D : Decls) {
+      if (isa<NamespaceDecl>(D) && D->isFromASTFile()) {
+        // In ASTReader, we stored only the key declaration of a namespace decl
+        // for this TU rather than storing all of the key declarations from each
+        // imported module. If we have an external namespace decl, this is that
+        // key declaration and we need to re-expand it to write out all of the
+        // key declarations from each imported module again.
+        //
+        // See comment 'ASTReader::FindExternalVisibleDeclsByName' for details.
+        ASTReader *Chain = Writer.getChain();
+        assert(Chain && "An external namespace decl without an ASTReader");
+        assert(D == Chain->getKeyDeclaration(D) &&
+               "An external namespace decl that is not "
+               "key declaration of this TU");
+        Chain->forEachImportedKeyDecl(D, [&AddDecl](const Decl *D) {
+          AddDecl(cast<NamedDecl>(const_cast<Decl*>(D)));
+        });
+      } else {
+        AddDecl(D);
+      }
     }
     return std::make_pair(Start, DeclIDs.size());
   }

@llvmbot
Copy link
Member

llvmbot commented Dec 11, 2025

@llvm/pr-subscribers-clang

Author: Michael Park (mpark)

Changes

Problem

Given code such as N::foo();, we perform name look-up on N. In the case where N is a namespace declared in imported modules, one namespace decl (the "key declaration") for each module that declares a namespace foo is loaded and stored. In large scales where there are many such modules, (e.g., 1,500) and many uses (e.g., 500,000), this becomes extremely inefficient because every look-up (500,000 of them) return 1,500 results.

The following synthetic script demonstrates the problem:

#/usr/bin/env bash

CLANG=${CLANG:-clang++}
NUM_MODULES=${NUM_MODULES:-1500}
NUM_USES=${NUM_USES:-500000}
USE_MODULES=${USE_MODULES:-true}

TMPDIR=$(mktemp -d)
echo "Working in temp directory: $TMPDIR"
cd $TMPDIR

trap "rm -rf \"$TMPDIR\"" EXIT

echo "namespace N { inline void foo() {} }" &gt; m1.h
for i in $(seq 2 $NUM_MODULES); do echo "namespace N {}" &gt; m${i}.h; done

if $USE_MODULES; then
  seq 1 $NUM_MODULES | xargs -I {} -P $(nproc) bash -c "$CLANG -std=c++20 -fmodule-header m{}.h"
fi

&gt; a.cpp
if $USE_MODULES; then
  for i in $(seq 1 $NUM_MODULES); do echo "import \"m${i}.h\";" &gt;&gt; a.cpp; done
else
  for i in $(seq 1 $NUM_MODULES); do echo "#include \"m${i}.h\"" &gt;&gt; a.cpp; done
fi

echo "int main() {" &gt;&gt; a.cpp
for i in $(seq 1 $NUM_USES); do echo "  N::foo();" &gt;&gt; a.cpp; done
echo "}" &gt;&gt; a.cpp

if $USE_MODULES; then
  time $CLANG -std=c++20 -Wno-experimental-header-units -c a.cpp -o /dev/null \
      $(for i in $(seq 1 $NUM_MODULES); do echo -n "-fmodule-file=m${i}.pcm "; done)
else
  time $CLANG -std=c++20 -Wno-experimental-header-units -c a.cpp -o /dev/null
fi

As of 575d689, without modules (USE_MODULES=false) this takes about 4.5s, whereas with modules (USE_MODULES=true), this takes about 37s.

With this PR, without modules there's no change (as expected) at 4.5s, but with modules it improves to about 5.2s.

Approach

The approach taken here aims to maintain status-quo with respect to the input and output of modules. That is, the ASTReader and ASTWriter both read and write the same declarations as it did before. The difference is in the middle part: the StoredDeclsMap in DeclContext. The StoredDeclsMap is roughly a map&lt;DeclarationName, StoredDeclsList&gt;. Currently, we read all of the external namespace decls from ASTReader, they all get stored into the StoredDeclsList, and the ASTWriter iterates through that list and writes out the results.

This PR continues to read all of the external namespace decls from ASTReader, but only stores one namespace decl in the StoredDeclsList. This is okay since the reading of the decls handles all of the merging and chaining of the namespace decls, and as long as they're loaded and chained, returning one for look-up purposes is sufficient.

The other half of the problem is to write out all of the external namespaces that we used to have available in StoredDeclsList. For this, we take advantage of the KeyDecls data structure in ASTReader. KeyDecls is roughly a map&lt;Decl *, vector&lt;GlobalDeclID&gt;, and it stores a mapping from the canonical decl of a redeclarable decl to a list of GlobalDeclIDs where each ID represents a "key declaration" from each imported module. More to the point, if we read external namespaces N1, N2, N3 in ASTReader, we'll either have N1 mapped to [N2, N3], or some newly local canonical decl mapped to [N1, N2, N3]. Either way, we can visit N1, N2, and N3 by doing ASTReader::forEachImportedKeyDecls(N1, Visitor), and we leverage this to maintain the current behavior of writing out all of the imported namespace decls in ASTWriter.

Alternatives Attempted

I tried reading fewer declarations on the ASTReader side, and writing out fewer declarations on the ASTWriter side, and neither options worked at all. Also tried trying to split StoredDeclsList into two pieces, one with non-namespace decls and one with only namespace decls, but that didn't work well... I think because the order of the declarations matter sometimes, and maybe also because the declaration replacement logic gets more complicated.


Full diff: https://github.com/llvm/llvm-project/pull/171769.diff

2 Files Affected:

  • (modified) clang/lib/Serialization/ASTReader.cpp (+46-11)
  • (modified) clang/lib/Serialization/ASTWriter.cpp (+26-5)
diff --git a/clang/lib/Serialization/ASTReader.cpp b/clang/lib/Serialization/ASTReader.cpp
index aec61322fb8be..5f66beefd6388 100644
--- a/clang/lib/Serialization/ASTReader.cpp
+++ b/clang/lib/Serialization/ASTReader.cpp
@@ -555,7 +555,26 @@ namespace {
 
 using MacroDefinitionsMap =
     llvm::StringMap<std::pair<StringRef, bool /*IsUndef*/>>;
-using DeclsMap = llvm::DenseMap<DeclarationName, SmallVector<NamedDecl *, 8>>;
+
+class DeclsSet {
+  SmallVector<NamedDecl *, 64> Decls;
+  llvm::SmallPtrSet<NamedDecl *, 8> Found;
+
+public:
+  NamedDecl *const *data() const { return Decls.data(); }
+
+  bool empty() const { return Decls.empty(); }
+  size_t size() const { return Decls.size(); }
+
+  bool insert(NamedDecl *ND) {
+    auto [_, Inserted] = Found.insert(ND);
+    if (Inserted)
+      Decls.push_back(ND);
+    return Inserted;
+  }
+};
+
+using DeclsMap = llvm::DenseMap<DeclarationName, DeclsSet>;
 
 } // namespace
 
@@ -8702,14 +8721,22 @@ bool ASTReader::FindExternalVisibleDeclsByName(const DeclContext *DC,
     return false;
 
   // Load the list of declarations.
-  SmallVector<NamedDecl *, 64> Decls;
-  llvm::SmallPtrSet<NamedDecl *, 8> Found;
+  DeclsSet DS;
 
   auto Find = [&, this](auto &&Table, auto &&Key) {
     for (GlobalDeclID ID : Table.find(Key)) {
       NamedDecl *ND = cast<NamedDecl>(GetDecl(ID));
-      if (ND->getDeclName() == Name && Found.insert(ND).second)
-        Decls.push_back(ND);
+      if (ND->getDeclName() != Name)
+        continue;
+      // Special case for namespaces: There can be a lot of redeclarations of
+      // some namespaces, and we import a "key declaration" per imported module.
+      // Since all declarations of a namespace are essentially interchangeable,
+      // we can optimize namespace look-up by only storing the key declaration
+      // of the current TU, rather than storing N key declarations where N is
+      // the # of imported modules that declare that namespace.
+      if (isa<NamespaceDecl>(ND))
+        ND = cast<NamedDecl>(getKeyDeclaration(ND));
+      DS.insert(ND);
     }
   };
 
@@ -8744,8 +8771,8 @@ bool ASTReader::FindExternalVisibleDeclsByName(const DeclContext *DC,
     Find(It->second.Table, Name);
   }
 
-  SetExternalVisibleDeclsForName(DC, Name, Decls);
-  return !Decls.empty();
+  SetExternalVisibleDeclsForName(DC, Name, DS);
+  return !DS.empty();
 }
 
 void ASTReader::completeVisibleDeclsMap(const DeclContext *DC) {
@@ -8763,7 +8790,15 @@ void ASTReader::completeVisibleDeclsMap(const DeclContext *DC) {
 
     for (GlobalDeclID ID : It->second.Table.findAll()) {
       NamedDecl *ND = cast<NamedDecl>(GetDecl(ID));
-      Decls[ND->getDeclName()].push_back(ND);
+      // Special case for namespaces: There can be a lot of redeclarations of
+      // some namespaces, and we import a "key declaration" per imported module.
+      // Since all declarations of a namespace are essentially interchangeable,
+      // we can optimize namespace look-up by only storing the key declaration
+      // of the current TU, rather than storing N key declarations where N is
+      // the # of imported modules that declare that namespace.
+      if (isa<NamespaceDecl>(ND))
+        ND = cast<NamedDecl>(getKeyDeclaration(ND));
+      Decls[ND->getDeclName()].insert(ND);
     }
 
     // FIXME: Why a PCH test is failing if we remove the iterator after findAll?
@@ -8773,9 +8808,9 @@ void ASTReader::completeVisibleDeclsMap(const DeclContext *DC) {
   findAll(ModuleLocalLookups, NumModuleLocalVisibleDeclContexts);
   findAll(TULocalLookups, NumTULocalVisibleDeclContexts);
 
-  for (DeclsMap::iterator I = Decls.begin(), E = Decls.end(); I != E; ++I) {
-    SetExternalVisibleDeclsForName(DC, I->first, I->second);
-  }
+  for (auto &[Name, DS] : Decls)
+    SetExternalVisibleDeclsForName(DC, Name, DS);
+
   const_cast<DeclContext *>(DC)->setHasExternalVisibleStorage(false);
 }
 
diff --git a/clang/lib/Serialization/ASTWriter.cpp b/clang/lib/Serialization/ASTWriter.cpp
index 667e04049dac8..231e0a206ab11 100644
--- a/clang/lib/Serialization/ASTWriter.cpp
+++ b/clang/lib/Serialization/ASTWriter.cpp
@@ -4396,20 +4396,20 @@ class ASTDeclContextNameLookupTrait
 
   template <typename Coll> data_type getData(const Coll &Decls) {
     unsigned Start = DeclIDs.size();
-    for (NamedDecl *D : Decls) {
+    auto AddDecl = [this](NamedDecl *D) {
       NamedDecl *DeclForLocalLookup =
           getDeclForLocalLookup(Writer.getLangOpts(), D);
 
       if (Writer.getDoneWritingDeclsAndTypes() &&
           !Writer.wasDeclEmitted(DeclForLocalLookup))
-        continue;
+        return;
 
       // Try to avoid writing internal decls to reduced BMI.
       // See comments in ASTWriter::WriteDeclContextLexicalBlock for details.
       if (Writer.isGeneratingReducedBMI() &&
           !DeclForLocalLookup->isFromExplicitGlobalModule() &&
           IsInternalDeclFromFileContext(DeclForLocalLookup))
-        continue;
+        return;
 
       auto ID = Writer.GetDeclRef(DeclForLocalLookup);
 
@@ -4423,7 +4423,7 @@ class ASTDeclContextNameLookupTrait
             ModuleLocalDeclsMap.insert({Key, DeclIDsTy{ID}});
           else
             Iter->second.push_back(ID);
-          continue;
+          return;
         }
         break;
       case LookupVisibility::TULocal: {
@@ -4432,7 +4432,7 @@ class ASTDeclContextNameLookupTrait
           TULocalDeclsMap.insert({D->getDeclName(), DeclIDsTy{ID}});
         else
           Iter->second.push_back(ID);
-        continue;
+        return;
       }
       case LookupVisibility::GenerallyVisibile:
         // Generally visible decls go into the general lookup table.
@@ -4440,6 +4440,27 @@ class ASTDeclContextNameLookupTrait
       }
 
       DeclIDs.push_back(ID);
+    };
+    for (NamedDecl *D : Decls) {
+      if (isa<NamespaceDecl>(D) && D->isFromASTFile()) {
+        // In ASTReader, we stored only the key declaration of a namespace decl
+        // for this TU rather than storing all of the key declarations from each
+        // imported module. If we have an external namespace decl, this is that
+        // key declaration and we need to re-expand it to write out all of the
+        // key declarations from each imported module again.
+        //
+        // See comment 'ASTReader::FindExternalVisibleDeclsByName' for details.
+        ASTReader *Chain = Writer.getChain();
+        assert(Chain && "An external namespace decl without an ASTReader");
+        assert(D == Chain->getKeyDeclaration(D) &&
+               "An external namespace decl that is not "
+               "key declaration of this TU");
+        Chain->forEachImportedKeyDecl(D, [&AddDecl](const Decl *D) {
+          AddDecl(cast<NamedDecl>(const_cast<Decl*>(D)));
+        });
+      } else {
+        AddDecl(D);
+      }
     }
     return std::make_pair(Start, DeclIDs.size());
   }

@mpark mpark force-pushed the modules-perf-namespaces branch from 4c14d68 to bcff2a7 Compare December 11, 2025 06:51
@llvm llvm deleted a comment from github-actions bot Dec 11, 2025
@mpark mpark requested a review from ChuanqiXu9 December 11, 2025 07:44
Copy link
Member

@ChuanqiXu9 ChuanqiXu9 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is the namespace decl special? Can we make this optimization more general?

@mpark
Copy link
Member Author

mpark commented Dec 11, 2025

Why is the namespace decl special? Can we make this optimization more general?

I did try to generalize it for all redeclarable decls, but it failed a bunch of unit tests. My understanding is that namespaces are indeed special in this aspect, according to this comment in getAcceptableDeclSlow:

  if (auto *ND = dyn_cast<NamespaceDecl>(D)) {
    // Namespaces are a bit of a special case: we expect there to be a lot of
    // redeclarations of some namespaces, all declarations of a namespace are
    // essentially interchangeable, all declarations are found by name lookup
    // if any is, and namespaces are never looked up during template
    // instantiation. So we benefit from caching the check in this case, and
    // it is correct to do so.
    ...
  }
  ...

@mpark mpark force-pushed the modules-perf-namespaces branch 3 times, most recently from fe39409 to dcad405 Compare December 12, 2025 00:58
@llvm llvm deleted a comment from github-actions bot Dec 12, 2025
@ChuanqiXu9
Copy link
Member

ChuanqiXu9 commented Dec 12, 2025

Why is the namespace decl special? Can we make this optimization more general?

I did try to generalize it for all redeclarable decls, but it failed a bunch of unit tests. My understanding is that namespaces are indeed special in this aspect, according to this comment in getAcceptableDeclSlow:

  if (auto *ND = dyn_cast<NamespaceDecl>(D)) {
    // Namespaces are a bit of a special case: we expect there to be a lot of
    // redeclarations of some namespaces, all declarations of a namespace are
    // essentially interchangeable, all declarations are found by name lookup
    // if any is, and namespaces are never looked up during template
    // instantiation. So we benefit from caching the check in this case, and
    // it is correct to do so.
    ...
  }
  ...

I think the only point in the comment are all declarations of a namespace are interchangable. While for class we may have:

class Foo;

and

class Foo {
    ...
};

they are not interchangable.

But the idea of the patch still seems valuable and doable for at least classes. Maybe it is not a super stragihtfoward extension. But I believe the problems are solvable and it is worthy to look into the problems.

I am not asking to address this in the PR. But let's add a FIXME to TODO for that.

@ChuanqiXu9
Copy link
Member

The patch itself looks good. But I like to test it internally before merging. Hope I can make it in next week.

And also I think it is worthy to add a test for it. We are able to test it with unittests. e.g., we can test the number of the return decls of DeclContext::lookup, or other similar words. We can add some API for testing such status too.

@mpark
Copy link
Member Author

mpark commented Dec 12, 2025

Why is the namespace decl special? Can we make this optimization more general?

I did try to generalize it for all redeclarable decls, but it failed a bunch of unit tests. My understanding is that namespaces are indeed special in this aspect, according to this comment in getAcceptableDeclSlow:

if (auto *ND = dyn_cast(D)) {

// Namespaces are a bit of a special case: we expect there to be a lot of
// redeclarations of some namespaces, all declarations of a namespace are
// essentially interchangeable, all declarations are found by name lookup
// if any is, and namespaces are never looked up during template
// instantiation. So we benefit from caching the check in this case, and
// it is correct to do so.
...

}

...

I think the only point in the comment are all declarations of a namespace are interchangable. While for class we may have:


class Foo;

and


class Foo {

    ...

};

they are not interchangable.

But the idea of the patch still seems valuable and doable for at least classes. Maybe it is not a super stragihtfoward extension. But I believe the problems are solvable and it is worthy to look into the problems.

I am not asking to address this in the PR. But let's add a FIXME to TODO for that.

Right, I was giving this more thought earlier today and I do think there's likely to be an improvement to be had there.

Probably along the lines of: keeping one class decl, and replacing it with the definition if one exists, etc rather than unconditionally taking one decl like we can with namespaces.

For now though, I'll leave a note for it as you suggest.

@mpark
Copy link
Member Author

mpark commented Dec 13, 2025

I've added a TODO to try to generalize this optimization to other redeclarable decls.

@mpark mpark force-pushed the modules-perf-namespaces branch from dcad405 to 2be2052 Compare December 13, 2025 02:47
@ChuanqiXu9
Copy link
Member

ChuanqiXu9 commented Dec 15, 2025

The patch itself looks good. But I like to test it internally before merging. Hope I can make it in next week.

In our internal workloads, I didn't see noticeable changes in End-to-End build process. Maybe the reason is we've already done a so called "bottom up" build. But the good news is that I didn't find regression too. So LGTM.

And also I think it is worthy to add a test for it. We are able to test it with unittests. e.g., we can test the number of the return decls of DeclContext::lookup, or other similar words. We can add some API for testing such status too.

Would you like to add a test for this? If you don't know how to do, you can look at clang/unittests/Serialization/LoadSpecLazilyTest.cpp for example. This is useful as lit test can't show all things.

@mpark
Copy link
Member Author

mpark commented Dec 15, 2025

The patch itself looks good. But I like to test it internally before merging. Hope I can make it in next week.

In our internal workloads, I didn't see noticeable changes in End-to-End build process. Maybe the reason is we've already done a so called "bottom up" build. But the good news is that I didn't find regression too. So LGTM.

Thanks for checking! Yeah, unless you have fine-grained modules with 1K+ modules declaring the same namespaces, I wouldn't expect to see any improvements with this.

For us, it's because at a certain scale with Thrift generated files, we end up importing 1K+ modules that all declare the apache::thrift namespace.

And also I think it is worthy to add a test for it. We are able to test it with unittests. e.g., we can test the number of the return decls of DeclContext::lookup, or other similar words. We can add some API for testing such status too.

Would you like to add a test for this? If you don't know how to do, you can look at clang/unittests/Serialization/LoadSpecLazilyTest.cpp for example. This is useful as lit test can't show all things.

I'll be working on the test tomorrow! Thanks for the pointer.

@ChuanqiXu9
Copy link
Member

For us, it's because at a certain scale with Thrift generated files, we end up importing 1K+ modules that all declare the apache::thrift namespace.

Maybe this is a nice point of named modules. We use thrift too. But we just wrap it into a single named module. Then we avoid the problem.

@mpark mpark force-pushed the modules-perf-namespaces branch 2 times, most recently from 3b2ecdc to d1cd20d Compare December 16, 2025 06:25
Copy link
Member

@ChuanqiXu9 ChuanqiXu9 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks.

@llvm llvm deleted a comment from github-actions bot Dec 16, 2025
@mpark mpark force-pushed the modules-perf-namespaces branch from d1cd20d to 8b117ab Compare December 16, 2025 06:34
@llvm llvm deleted a comment from github-actions bot Dec 16, 2025
@mpark mpark force-pushed the modules-perf-namespaces branch from 8b117ab to 1af1a44 Compare December 16, 2025 06:38
@mpark mpark merged commit 1928c1e into llvm:main Dec 16, 2025
10 checks passed
@mpark mpark deleted the modules-perf-namespaces branch December 16, 2025 07:33
@llvm-ci
Copy link
Collaborator

llvm-ci commented Dec 16, 2025

LLVM Buildbot has detected a new failure on builder clang-m68k-linux-cross running on suse-gary-m68k-cross while building clang at step 5 "ninja check 1".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/27/builds/20378

Here is the relevant piece of the build log for the reference
Step 5 (ninja check 1) failure: stage 1 checked (failure)
...
[134/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/SymbolCollectorTests.cpp.o
/var/lib/buildbot/workers/suse-gary-m68k-cross/clang-m68k-linux-cross/llvm/clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp: In member function ‘virtual void clang::clangd::{anonymous}::SymbolCollectorTest_SpelledReferences_Test::TestBody()’:
/var/lib/buildbot/workers/suse-gary-m68k-cross/clang-m68k-linux-cross/llvm/clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp:1148:8: warning: suggest explicit braces to avoid ambiguous ‘else’ [-Wdangling-else]
 1148 |     if (!SpelledRanges.empty())
      |        ^
/var/lib/buildbot/workers/suse-gary-m68k-cross/clang-m68k-linux-cross/llvm/clang-tools-extra/clangd/unittests/SymbolCollectorTests.cpp:1151:8: warning: suggest explicit braces to avoid ambiguous ‘else’ [-Wdangling-else]
 1151 |     if (!ImplicitRanges.empty())
      |        ^
[135/1248] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Lex/HeaderMapTest.cpp.o
[136/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/CodeCompleteTests.cpp.o
FAILED: tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/CodeCompleteTests.cpp.o 
/usr/bin/c++ -DGTEST_HAS_RTTI=0 -DLLVM_BUILD_STATIC -D_DEBUG -D_GLIBCXX_ASSERTIONS -D_GLIBCXX_USE_CXX11_ABI=1 -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/var/lib/buildbot/workers/suse-gary-m68k-cross/clang-m68k-linux-cross/stage1/tools/clang/tools/extra/clangd/unittests -I/var/lib/buildbot/workers/suse-gary-m68k-cross/clang-m68k-linux-cross/llvm/clang-tools-extra/clangd/unittests -I/var/lib/buildbot/workers/suse-gary-m68k-cross/clang-m68k-linux-cross/llvm/clang-tools-extra/clangd/../include-cleaner/include -I/var/lib/buildbot/workers/suse-gary-m68k-cross/clang-m68k-linux-cross/stage1/tools/clang/tools/extra/clangd/../clang-tidy -I/var/lib/buildbot/workers/suse-gary-m68k-cross/clang-m68k-linux-cross/llvm/clang/include -I/var/lib/buildbot/workers/suse-gary-m68k-cross/clang-m68k-linux-cross/stage1/tools/clang/include -I/var/lib/buildbot/workers/suse-gary-m68k-cross/clang-m68k-linux-cross/stage1/include -I/var/lib/buildbot/workers/suse-gary-m68k-cross/clang-m68k-linux-cross/llvm/llvm/include -I/var/lib/buildbot/workers/suse-gary-m68k-cross/clang-m68k-linux-cross/llvm/clang-tools-extra/clangd -I/var/lib/buildbot/workers/suse-gary-m68k-cross/clang-m68k-linux-cross/stage1/tools/clang/tools/extra/clangd -I/var/lib/buildbot/workers/suse-gary-m68k-cross/clang-m68k-linux-cross/llvm/third-party/unittest/googletest/include -I/var/lib/buildbot/workers/suse-gary-m68k-cross/clang-m68k-linux-cross/llvm/third-party/unittest/googlemock/include -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-uninitialized -Wno-nonnull -Wno-class-memaccess -Wno-dangling-reference -Wno-redundant-move -Wno-pessimizing-move -Wno-array-bounds -Wno-stringop-overread -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -Wno-misleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -fno-common -Woverloaded-virtual -O3 -DNDEBUG -std=c++17  -Wno-variadic-macros -fno-exceptions -funwind-tables -fno-rtti -UNDEBUG -Wno-suggest-override -MD -MT tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/CodeCompleteTests.cpp.o -MF tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/CodeCompleteTests.cpp.o.d -o tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/CodeCompleteTests.cpp.o -c /var/lib/buildbot/workers/suse-gary-m68k-cross/clang-m68k-linux-cross/llvm/clang-tools-extra/clangd/unittests/CodeCompleteTests.cpp
c++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
[137/1248] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Lex/DependencyDirectivesScannerTest.cpp.o
[138/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/TestWorkspace.cpp.o
[139/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/tweaks/DumpSymbolTests.cpp.o
[140/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/tweaks/MemberwiseConstructorTests.cpp.o
[141/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/tweaks/ExtractFunctionTests.cpp.o
[142/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/tweaks/ObjCLocalizeStringLiteralTests.cpp.o
[143/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/tweaks/PopulateSwitchTests.cpp.o
[144/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/tweaks/ObjCMemberwiseInitializerTests.cpp.o
[145/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/tweaks/OverridePureVirtualsTests.cpp.o
[146/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/tweaks/DumpRecordLayoutTests.cpp.o
[147/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/tweaks/DumpASTTests.cpp.o
[148/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/tweaks/ExpandMacroTests.cpp.o
[149/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/tweaks/ExtractVariableTests.cpp.o
[150/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/tweaks/ExpandDeducedTypeTests.cpp.o
[151/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/tweaks/RemoveUsingNamespaceTests.cpp.o
[152/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/tweaks/AnnotateHighlightingsTests.cpp.o
[153/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/tweaks/SpecialMembersTests.cpp.o
[154/1248] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Lex/HeaderSearchTest.cpp.o
[155/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/tweaks/RawStringLiteralTests.cpp.o
[156/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/tweaks/AddUsingTests.cpp.o
[157/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/tweaks/DefineInlineTests.cpp.o
[158/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/tweaks/ScopifyEnumTests.cpp.o
[159/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/tweaks/ShowSelectionTreeTests.cpp.o
[160/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/tweaks/TweakTesting.cpp.o
[161/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/tweaks/SwapBinaryOperandsTests.cpp.o
[162/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/tweaks/TweakTests.cpp.o
[163/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/tweaks/SwapIfBranchesTests.cpp.o
[164/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/tweaks/DefineOutlineTests.cpp.o
[165/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/TypeHierarchyTests.cpp.o
[166/1248] Building CXX object tools/clang/tools/extra/unittests/clang-tidy/CMakeFiles/ClangTidyTests.dir/ReadabilityModuleTest.cpp.o
[167/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/XRefsTests.cpp.o
[168/1248] Building CXX object tools/clang/tools/extra/clangd/unittests/CMakeFiles/ClangdTests.dir/TestTU.cpp.o
[169/1248] Building CXX object tools/clang/tools/extra/unittests/clang-tidy/CMakeFiles/ClangTidyTests.dir/TransformerClangTidyCheckTest.cpp.o
ninja: build stopped: subcommand failed.

@llvm-ci
Copy link
Collaborator

llvm-ci commented Dec 16, 2025

LLVM Buildbot has detected a new failure on builder reverse-iteration running on hexagon-build-03 while building clang at step 6 "check_all".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/110/builds/6824

Here is the relevant piece of the build log for the reference
Step 6 (check_all) failure: test (failure)
******************** TEST 'Clang :: Interpreter/dynamic-library.cpp' FAILED ********************
Exit Code: 2

Command Output (stdout):
--
# RUN: at line 17
cat /local/mnt/workspace/bots/hexagon-build-03/reverse-iteration/llvm.src/clang/test/Interpreter/dynamic-library.cpp | env LD_LIBRARY_PATH=/local/mnt/workspace/bots/hexagon-build-03/reverse-iteration/llvm.src/clang/test/Interpreter/Inputs:$LD_LIBRARY_PATH /local/mnt/workspace/bots/hexagon-build-03/reverse-iteration/llvm.obj/bin/clang-repl | /local/mnt/workspace/bots/hexagon-build-03/reverse-iteration/llvm.obj/bin/FileCheck /local/mnt/workspace/bots/hexagon-build-03/reverse-iteration/llvm.src/clang/test/Interpreter/dynamic-library.cpp
# executed command: cat /local/mnt/workspace/bots/hexagon-build-03/reverse-iteration/llvm.src/clang/test/Interpreter/dynamic-library.cpp
# .---command stdout------------
# | // REQUIRES: host-supports-jit, x86_64-linux
# | 
# | // To generate libdynamic-library-test.so :
# | // clang -xc++ -o libdynamic-library-test.so -fPIC -shared
# | //
# | // extern "C" {
# | //
# | // int ultimate_answer = 0;
# | // 
# | // int calculate_answer() {
# | //   ultimate_answer = 42;
# | //   return 5;
# | // }
# | //
# | // }
# | 
# | // RUN: cat %s | env LD_LIBRARY_PATH=%S/Inputs:$LD_LIBRARY_PATH clang-repl | FileCheck %s
# | 
# | extern "C" int printf(const char* format, ...);
# | 
# | extern "C" int ultimate_answer;
# | extern "C" int calculate_answer();
# | 
# | %lib libdynamic-library-test.so
# | 
# | printf("Return value: %d\n", calculate_answer());
# | // CHECK: Return value: 5
# | 
# | printf("Variable: %d\n", ultimate_answer);
# | // CHECK-NEXT: Variable: 42
# | 
# | %quit
# `-----------------------------
# executed command: env 'LD_LIBRARY_PATH=/local/mnt/workspace/bots/hexagon-build-03/reverse-iteration/llvm.src/clang/test/Interpreter/Inputs:$LD_LIBRARY_PATH' /local/mnt/workspace/bots/hexagon-build-03/reverse-iteration/llvm.obj/bin/clang-repl
# .---command stderr------------
# | /local/mnt/workspace/bots/hexagon-build-03/reverse-iteration/llvm.obj/bin/clang-repl: error while loading shared libraries: libc++.so.1: cannot open shared object file: No such file or directory
# `-----------------------------
# error: command failed with exit status: 127
# executed command: /local/mnt/workspace/bots/hexagon-build-03/reverse-iteration/llvm.obj/bin/FileCheck /local/mnt/workspace/bots/hexagon-build-03/reverse-iteration/llvm.src/clang/test/Interpreter/dynamic-library.cpp
# .---command stderr------------
# | FileCheck error: '<stdin>' is empty.
...

@llvm-ci
Copy link
Collaborator

llvm-ci commented Dec 16, 2025

LLVM Buildbot has detected a new failure on builder clang-ppc64le-linux-test-suite running on ppc64le-clang-test-suite while building clang at step 6 "test-build-unified-tree-check-all".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/95/builds/19450

Here is the relevant piece of the build log for the reference
Step 6 (test-build-unified-tree-check-all) failure: test (failure)
******************** TEST 'LeakSanitizer-AddressSanitizer-powerpc64le :: TestCases/create_thread_leak.cpp' FAILED ********************
Exit Code: 2

Command Output (stdout):
--
# RUN: at line 3
/home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/build/./bin/clang  --driver-mode=g++ -O0  -m64 -fno-function-sections  -gline-tables-only -fsanitize=address -I/home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/llvm-project/compiler-rt/test/lsan/../ -pthread /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/llvm-project/compiler-rt/test/lsan/TestCases/create_thread_leak.cpp -o /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/build/runtimes/runtimes-bins/compiler-rt/test/lsan/POWERPC64LEAsanConfig/TestCases/Output/create_thread_leak.cpp.tmp
# executed command: /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/build/./bin/clang --driver-mode=g++ -O0 -m64 -fno-function-sections -gline-tables-only -fsanitize=address -I/home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/llvm-project/compiler-rt/test/lsan/../ -pthread /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/llvm-project/compiler-rt/test/lsan/TestCases/create_thread_leak.cpp -o /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/build/runtimes/runtimes-bins/compiler-rt/test/lsan/POWERPC64LEAsanConfig/TestCases/Output/create_thread_leak.cpp.tmp
# RUN: at line 4
not  /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/build/runtimes/runtimes-bins/compiler-rt/test/lsan/POWERPC64LEAsanConfig/TestCases/Output/create_thread_leak.cpp.tmp 10 1 0 0 2>&1 | FileCheck /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/llvm-project/compiler-rt/test/lsan/TestCases/create_thread_leak.cpp --check-prefixes=LEAK,LEAK123
# executed command: not /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/build/runtimes/runtimes-bins/compiler-rt/test/lsan/POWERPC64LEAsanConfig/TestCases/Output/create_thread_leak.cpp.tmp 10 1 0 0
# executed command: FileCheck /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/llvm-project/compiler-rt/test/lsan/TestCases/create_thread_leak.cpp --check-prefixes=LEAK,LEAK123
# RUN: at line 5
not  /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/build/runtimes/runtimes-bins/compiler-rt/test/lsan/POWERPC64LEAsanConfig/TestCases/Output/create_thread_leak.cpp.tmp 10 0 1 0 2>&1 | FileCheck /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/llvm-project/compiler-rt/test/lsan/TestCases/create_thread_leak.cpp --check-prefixes=LEAK,LEAK234
# executed command: not /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/build/runtimes/runtimes-bins/compiler-rt/test/lsan/POWERPC64LEAsanConfig/TestCases/Output/create_thread_leak.cpp.tmp 10 0 1 0
# executed command: FileCheck /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/llvm-project/compiler-rt/test/lsan/TestCases/create_thread_leak.cpp --check-prefixes=LEAK,LEAK234
# RUN: at line 6
not  /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/build/runtimes/runtimes-bins/compiler-rt/test/lsan/POWERPC64LEAsanConfig/TestCases/Output/create_thread_leak.cpp.tmp 10 0 0 1 2>&1 | FileCheck /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/llvm-project/compiler-rt/test/lsan/TestCases/create_thread_leak.cpp --check-prefixes=LEAK,LEAK234
# executed command: not /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/build/runtimes/runtimes-bins/compiler-rt/test/lsan/POWERPC64LEAsanConfig/TestCases/Output/create_thread_leak.cpp.tmp 10 0 0 1
# note: command had no output on stdout or stderr
# error: command failed with exit status: 1
# executed command: FileCheck /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/llvm-project/compiler-rt/test/lsan/TestCases/create_thread_leak.cpp --check-prefixes=LEAK,LEAK234
# .---command stderr------------
# | FileCheck error: '<stdin>' is empty.
# | FileCheck command line:  FileCheck /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/llvm-project/compiler-rt/test/lsan/TestCases/create_thread_leak.cpp --check-prefixes=LEAK,LEAK234
# `-----------------------------
# error: command failed with exit status: 2

--

********************


Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

clang:modules C++20 modules and Clang Header Modules clang Clang issues not falling into any other category

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants