Skip to content

[llvm-debuginfo-analyzer] Remove LVScope::Children container #144750

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

jalopezg-git
Copy link
Contributor

@jalopezg-git jalopezg-git commented Jun 18, 2025

Remove the LVScope::Children container and use llvm::concat() instead to return a view over the types, symbols, and sub-scopes contained in a given LVScope.

Fixes #69160.

@llvmbot
Copy link
Member

llvmbot commented Jun 18, 2025

@llvm/pr-subscribers-llvm-binary-utilities
@llvm/pr-subscribers-llvm-adt

@llvm/pr-subscribers-debuginfo

Author: Javier Lopez-Gomez (jalopezg-git)

Changes

Remove the LVScope::Children container and use llvm::concat()
instead to return a view over the types, symbols, and sub-scopes
contained in a given LVScope.

Fixes #69160.


Full diff: https://github.com/llvm/llvm-project/pull/144750.diff

5 Files Affected:

  • (modified) llvm/include/llvm/ADT/STLExtras.h (+6-5)
  • (modified) llvm/include/llvm/DebugInfo/LogicalView/Core/LVScope.h (+11-10)
  • (modified) llvm/lib/DebugInfo/LogicalView/Core/LVScope.cpp (+21-28)
  • (modified) llvm/unittests/ADT/STLExtrasTest.cpp (+17)
  • (modified) llvm/unittests/DebugInfo/LogicalView/DWARFReaderTest.cpp (+5-7)
diff --git a/llvm/include/llvm/ADT/STLExtras.h b/llvm/include/llvm/ADT/STLExtras.h
index eea06cfb99ba2..6dd9b5a2430d2 100644
--- a/llvm/include/llvm/ADT/STLExtras.h
+++ b/llvm/include/llvm/ADT/STLExtras.h
@@ -1030,14 +1030,15 @@ class concat_iterator
                                   std::forward_iterator_tag, ValueT> {
   using BaseT = typename concat_iterator::iterator_facade_base;
 
-  static constexpr bool ReturnsByValue =
-      !(std::is_reference_v<decltype(*std::declval<IterTs>())> && ...);
+  static constexpr bool ReturnsValueOrPointer =
+      !(std::is_reference_v<decltype(*std::declval<IterTs>())> && ...) ||
+      (std::is_pointer_v<IterTs> && ...);
 
   using reference_type =
-      typename std::conditional_t<ReturnsByValue, ValueT, ValueT &>;
+      typename std::conditional_t<ReturnsValueOrPointer, ValueT, ValueT &>;
 
   using handle_type =
-      typename std::conditional_t<ReturnsByValue, std::optional<ValueT>,
+      typename std::conditional_t<ReturnsValueOrPointer, std::optional<ValueT>,
                                   ValueT *>;
 
   /// We store both the current and end iterators for each concatenated
@@ -1088,7 +1089,7 @@ class concat_iterator
     if (Begin == End)
       return {};
 
-    if constexpr (ReturnsByValue)
+    if constexpr (ReturnsValueOrPointer)
       return *Begin;
     else
       return &*Begin;
diff --git a/llvm/include/llvm/DebugInfo/LogicalView/Core/LVScope.h b/llvm/include/llvm/DebugInfo/LogicalView/Core/LVScope.h
index 5715a37185b2b..afd30a24d0f8d 100644
--- a/llvm/include/llvm/DebugInfo/LogicalView/Core/LVScope.h
+++ b/llvm/include/llvm/DebugInfo/LogicalView/Core/LVScope.h
@@ -14,6 +14,7 @@
 #ifndef LLVM_DEBUGINFO_LOGICALVIEW_CORE_LVSCOPE_H
 #define LLVM_DEBUGINFO_LOGICALVIEW_CORE_LVSCOPE_H
 
+#include "llvm/ADT/STLExtras.h"
 #include "llvm/DebugInfo/LogicalView/Core/LVElement.h"
 #include "llvm/DebugInfo/LogicalView/Core/LVLocation.h"
 #include "llvm/DebugInfo/LogicalView/Core/LVSort.h"
@@ -94,6 +95,11 @@ class LLVM_ABI LVScope : public LVElement {
   LVProperties<LVScopeKind> Kinds;
   LVProperties<Property> Properties;
   static LVScopeDispatch Dispatch;
+  // Empty containers used in `getChildren()` in case there is no Types,
+  // Symbols, or Scopes.
+  static const LVTypes EmptyTypes;
+  static const LVSymbols EmptySymbols;
+  static const LVScopes EmptyScopes;
 
   // Size in bits if this scope represents also a compound type.
   uint32_t BitSize = 0;
@@ -128,14 +134,6 @@ class LLVM_ABI LVScope : public LVElement {
   std::unique_ptr<LVLines> Lines;
   std::unique_ptr<LVLocations> Ranges;
 
-  // Vector of elements (types, scopes and symbols).
-  // It is the union of (*Types, *Symbols and *Scopes) to be used for
-  // the following reasons:
-  // - Preserve the order the logical elements are read in.
-  // - To have a single container with all the logical elements, when
-  //   the traversal does not require any specific element kind.
-  std::unique_ptr<LVElements> Children;
-
   // Resolve the template parameters/arguments relationship.
   void resolveTemplate();
   void printEncodedArgs(raw_ostream &OS, bool Full) const;
@@ -213,7 +211,11 @@ class LLVM_ABI LVScope : public LVElement {
   const LVScopes *getScopes() const { return Scopes.get(); }
   const LVSymbols *getSymbols() const { return Symbols.get(); }
   const LVTypes *getTypes() const { return Types.get(); }
-  const LVElements *getChildren() const { return Children.get(); }
+  auto getChildren() const {
+    return llvm::concat<LVElement *const>(Types ? *Types : EmptyTypes,
+                                          Symbols ? *Symbols : EmptySymbols,
+                                          Scopes ? *Scopes : EmptyScopes);
+  }
 
   void addElement(LVElement *Element);
   void addElement(LVLine *Line);
@@ -222,7 +224,6 @@ class LLVM_ABI LVScope : public LVElement {
   void addElement(LVType *Type);
   void addObject(LVLocation *Location);
   void addObject(LVAddress LowerAddress, LVAddress UpperAddress);
-  void addToChildren(LVElement *Element);
 
   // Add the missing elements from the given 'Reference', which is the
   // scope associated with any DW_AT_specification, DW_AT_abstract_origin.
diff --git a/llvm/lib/DebugInfo/LogicalView/Core/LVScope.cpp b/llvm/lib/DebugInfo/LogicalView/Core/LVScope.cpp
index 55880fab5e88e..70422555ea27e 100644
--- a/llvm/lib/DebugInfo/LogicalView/Core/LVScope.cpp
+++ b/llvm/lib/DebugInfo/LogicalView/Core/LVScope.cpp
@@ -107,11 +107,9 @@ LVScopeDispatch LVScope::Dispatch = {
     {LVScopeKind::IsTryBlock, &LVScope::getIsTryBlock},
     {LVScopeKind::IsUnion, &LVScope::getIsUnion}};
 
-void LVScope::addToChildren(LVElement *Element) {
-  if (!Children)
-    Children = std::make_unique<LVElements>();
-  Children->push_back(Element);
-}
+const LVTypes LVScope::EmptyTypes{};
+const LVSymbols LVScope::EmptySymbols{};
+const LVScopes LVScope::EmptyScopes{};
 
 void LVScope::addElement(LVElement *Element) {
   assert(Element && "Invalid element.");
@@ -175,7 +173,6 @@ void LVScope::addElement(LVScope *Scope) {
 
   // Add it to parent.
   Scopes->push_back(Scope);
-  addToChildren(Scope);
   Scope->setParent(this);
 
   // Notify the reader about the new element being added.
@@ -202,7 +199,6 @@ void LVScope::addElement(LVSymbol *Symbol) {
 
   // Add it to parent.
   Symbols->push_back(Symbol);
-  addToChildren(Symbol);
   Symbol->setParent(this);
 
   // Notify the reader about the new element being added.
@@ -229,7 +225,6 @@ void LVScope::addElement(LVType *Type) {
 
   // Add it to parent.
   Types->push_back(Type);
-  addToChildren(Type);
   Type->setParent(this);
 
   // Notify the reader about the new element being added.
@@ -277,15 +272,12 @@ bool LVScope::removeElement(LVElement *Element) {
   if (Element->getIsLine())
     return RemoveElement(Lines);
 
-  if (RemoveElement(Children)) {
-    if (Element->getIsSymbol())
-      return RemoveElement(Symbols);
-    if (Element->getIsType())
-      return RemoveElement(Types);
-    if (Element->getIsScope())
-      return RemoveElement(Scopes);
-    llvm_unreachable("Invalid element.");
-  }
+  if (Element->getIsSymbol())
+    return RemoveElement(Symbols);
+  if (Element->getIsType())
+    return RemoveElement(Types);
+  if (Element->getIsScope())
+    return RemoveElement(Scopes);
 
   return false;
 }
@@ -356,8 +348,8 @@ void LVScope::updateLevel(LVScope *Parent, bool Moved) {
   setLevel(Parent->getLevel() + 1);
 
   // Update the children.
-  if (Children)
-    for (LVElement *Element : *Children)
+  if (auto Elements = getChildren(); Elements.begin() != Elements.end())
+    for (LVElement *Element : Elements)
       Element->updateLevel(this, Moved);
 
   // Update any lines.
@@ -374,8 +366,8 @@ void LVScope::resolve() {
   LVElement::resolve();
 
   // Resolve the children.
-  if (Children)
-    for (LVElement *Element : *Children) {
+  if (auto Elements = getChildren(); Elements.begin() != Elements.end())
+    for (LVElement *Element : Elements) {
       if (getIsGlobalReference())
         // If the scope is a global reference, mark all its children as well.
         Element->setIsGlobalReference();
@@ -633,8 +625,9 @@ Error LVScope::doPrint(bool Split, bool Match, bool Print, raw_ostream &OS,
         options().getPrintFormatting() &&
         getLevel() < options().getOutputLevel()) {
       // Print the children.
-      if (Children)
-        for (const LVElement *Element : *Children) {
+      if (const auto Elements = getChildren();
+          Elements.begin() != Elements.end())
+        for (const LVElement *Element : Elements) {
           if (Match && !Element->getHasPattern())
             continue;
           if (Error Err =
@@ -692,7 +685,6 @@ void LVScope::sort() {
           Traverse(Parent->Symbols, SortFunction);
           Traverse(Parent->Scopes, SortFunction);
           Traverse(Parent->Ranges, compareRange);
-          Traverse(Parent->Children, SortFunction);
 
           if (Parent->Scopes)
             for (LVScope *Scope : *Parent->Scopes)
@@ -978,8 +970,8 @@ bool LVScope::equals(const LVScopes *References, const LVScopes *Targets) {
 void LVScope::report(LVComparePass Pass) {
   getComparator().printItem(this, Pass);
   getComparator().push(this);
-  if (Children)
-    for (LVElement *Element : *Children)
+  if (auto Elements = getChildren(); Elements.begin() != Elements.end())
+    for (LVElement *Element : Elements)
       Element->report(Pass);
 
   if (Lines)
@@ -1656,8 +1648,9 @@ void LVScopeCompileUnit::printMatchedElements(raw_ostream &OS,
       // Print the view for the matched scopes.
       for (const LVScope *Scope : MatchedScopes) {
         Scope->print(OS);
-        if (const LVElements *Elements = Scope->getChildren())
-          for (LVElement *Element : *Elements)
+        if (const auto Elements = Scope->getChildren();
+            Elements.begin() != Elements.end())
+          for (LVElement *Element : Elements)
             Element->print(OS);
       }
     }
diff --git a/llvm/unittests/ADT/STLExtrasTest.cpp b/llvm/unittests/ADT/STLExtrasTest.cpp
index 286cfa745fd14..98c13a33d35eb 100644
--- a/llvm/unittests/ADT/STLExtrasTest.cpp
+++ b/llvm/unittests/ADT/STLExtrasTest.cpp
@@ -398,6 +398,8 @@ struct some_struct {
   std::string swap_val;
 };
 
+struct derives_from_some_struct : some_struct {};
+
 std::vector<int>::const_iterator begin(const some_struct &s) {
   return s.data.begin();
 }
@@ -532,6 +534,21 @@ TEST(STLExtrasTest, ConcatRangeADL) {
   EXPECT_THAT(concat<const int>(S0, S1), ElementsAre(1, 2, 3, 4));
 }
 
+TEST(STLExtrasTest, ConcatRangePtrToDerivedClass) {
+  auto S0 = std::make_unique<some_namespace::some_struct>();
+  auto S1 = std::make_unique<some_namespace::derives_from_some_struct>();
+  SmallVector<some_namespace::some_struct *> V0{S0.get()};
+  SmallVector<some_namespace::derives_from_some_struct *> V1{S1.get(),
+                                                             S1.get()};
+
+  // Use concat over ranges of pointers to different (but related) types.
+  EXPECT_THAT(
+      concat<some_namespace::some_struct *>(V0, V1),
+      ElementsAre(S0.get(),
+                  static_cast<some_namespace::some_struct *>(S1.get()),
+                  static_cast<some_namespace::some_struct *>(S1.get())));
+}
+
 TEST(STLExtrasTest, MakeFirstSecondRangeADL) {
   // Make sure that we use the `begin`/`end` functions from `some_namespace`,
   // using ADL.
diff --git a/llvm/unittests/DebugInfo/LogicalView/DWARFReaderTest.cpp b/llvm/unittests/DebugInfo/LogicalView/DWARFReaderTest.cpp
index 544c39a3c7b2e..ba6df7489b750 100644
--- a/llvm/unittests/DebugInfo/LogicalView/DWARFReaderTest.cpp
+++ b/llvm/unittests/DebugInfo/LogicalView/DWARFReaderTest.cpp
@@ -163,13 +163,12 @@ void checkUnspecifiedParameters(LVReader *Reader) {
   LVPublicNames::const_iterator IterNames = PublicNames.cbegin();
   LVScope *Function = (*IterNames).first;
   EXPECT_EQ(Function->getName(), "foo_printf");
-  const LVElements *Elements = Function->getChildren();
-  ASSERT_NE(Elements, nullptr);
+  const auto Elements = Function->getChildren();
   // foo_printf is a variadic function whose prototype is
   // `int foo_printf(const char *, ...)`, where the '...' is represented by a
   // DW_TAG_unspecified_parameters, i.e. we expect to find at least one child
   // for which getIsUnspecified() returns true.
-  EXPECT_TRUE(llvm::any_of(*Elements, [](const LVElement *elt) {
+  EXPECT_TRUE(llvm::any_of(Elements, [](const LVElement *elt) {
     return elt->getIsSymbol() &&
            static_cast<const LVSymbol *>(elt)->getIsUnspecified();
   }));
@@ -183,10 +182,9 @@ void checkScopeModule(LVReader *Reader) {
   EXPECT_EQ(Root->getFileFormatName(), "Mach-O 64-bit x86-64");
   EXPECT_EQ(Root->getName(), DwarfClangModule);
 
-  ASSERT_NE(CompileUnit->getChildren(), nullptr);
-  LVElement *FirstChild = *(CompileUnit->getChildren()->begin());
-  EXPECT_EQ(FirstChild->getIsScope(), 1);
-  LVScopeModule *Module = static_cast<LVScopeModule *>(FirstChild);
+  ASSERT_NE(CompileUnit->getScopes(), nullptr);
+  LVElement *FirstScope = *(CompileUnit->getScopes()->begin());
+  LVScopeModule *Module = static_cast<LVScopeModule *>(FirstScope);
   EXPECT_EQ(Module->getIsModule(), 1);
   EXPECT_EQ(Module->getName(), "DebugModule");
 }

Copy link
Member

@kuhar kuhar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please submit ADT changes in a separate PR to make it easy to revert. Would be also nice to add a unit test.

@jalopezg-git
Copy link
Contributor Author

This PR should yield further memory footprint improvements after #144399 is merged.

As suggested in #69160, removing LVScope::Children. Note, however, that children iteration order is now (1) Types, (2) Symbols, and finally (3) Scopes. This may be (or not) considered important - I think it's not.
llvm-debuginfo-analyzer tests should be fixed though due to iteration order and CHECK-NEXT: FileCheck directives used in those tests.

@jalopezg-git
Copy link
Contributor Author

This required a minor change to llvm::concat_iterator; it appears in diff here, but change is tracked separately in PR #144744.

@CarlosAlbertoEnciso
Copy link
Member

This PR should yield further memory footprint improvements after #144399 is merged.

As suggested in #69160, removing LVScope::Children. Note, however, that children iteration order is now (1) Types, (2) Symbols, and finally (3) Scopes. This may be (or not) considered important - I think it's not. llvm-debuginfo-analyzer tests should be fixed though due to iteration order and CHECK-NEXT: FileCheck directives used in those tests.

From #69160 (comment)

The idea behind the Children container is to generate a logical view where its logical elements order is very close to their programming elements from the input file. That is the default printing mode.
By using the chaining iterator, we hope to have a considerable memory reduction, which is very important.
May be we can use the LVObject::ID to recreate the original printing order via the LVSortFunction llvm::logicalview::getSortFunction().

@jalopezg-git
Copy link
Contributor Author

This PR should yield further memory footprint improvements after #144399 is merged.
As suggested in #69160, removing LVScope::Children. Note, however, that children iteration order is now (1) Types, (2) Symbols, and finally (3) Scopes. This may be (or not) considered important - I think it's not. llvm-debuginfo-analyzer tests should be fixed though due to iteration order and CHECK-NEXT: FileCheck directives used in those tests.

From #69160 (comment)

The idea behind the Children container is to generate a logical view where its logical elements order is very close to their programming elements from the input file. That is the default printing mode. By using the chaining iterator, we hope to have a considerable memory reduction, which is very important. May be we can use the LVObject::ID to recreate the original printing order via the LVSortFunction llvm::logicalview::getSortFunction().

@CarlosAlbertoEnciso If order is important, then...

  • (1) getChildren() may return llvm::SmallVector instead, which is composed on demand by adding element from the other vectors and sorting as appropriate (e.g. by LVObject::ID).
    The problem w/ this one is that getChildren() has an associated overhead, but we could likely offer getUnorderedChildren() which iterates as-is (current state of this PR).
  • OR... (2) the Children member may be kept, but its encoding may change s.t. it uses less memory, e.g. instead of using pointers, we may use int32_t where the first N most-significant bits are used to store which vector does the element belong to, and the remaining bits are an index.
    This should cut memory consumption at least by half, depending on the type used.

I'm thinking about this, but likely we could go for approach (1), and if overhead is a concern, one should use getUnorderedChildren(). What do you think?

@jalopezg-git jalopezg-git force-pushed the jalopezg-69160 branch 2 times, most recently from 5f913f3 to 5a31ab8 Compare June 20, 2025 15:09
@CarlosAlbertoEnciso
Copy link
Member

@jalopezg-git I think the base branch for this patch should be #144744.
This patch should have only the changes to the Children container.

@jalopezg-git
Copy link
Contributor Author

jalopezg-git commented Jun 20, 2025

@jalopezg-git I think the base branch for this patch should be #144744. This patch should have only the changes to the Children container.

@CarlosAlbertoEnciso That is right -- I can probably just rebase it atop main once #144744 is landed. On this side, we are missing the element ordering (that's why some tests fail).
But, the current state should be enough to see what are the memory savings and if it pays off for the additional work 🙂. Could you run the tool w/ that big Chrome debug info file again?

@jalopezg-git jalopezg-git force-pushed the jalopezg-69160 branch 3 times, most recently from ce0fb39 to 63fb246 Compare June 23, 2025 15:45
@jalopezg-git jalopezg-git marked this pull request as ready for review June 23, 2025 16:14
@@ -35,8 +35,8 @@
; ONE-NEXT: [002] 1 {TypeAlias} 'INTPTR' -> '* const int'
; ONE-NEXT: [002] 2 {Function} extern not_inlined 'foo' -> 'int'
; ONE-NEXT: [003] {Block}
; ONE-NEXT: [004] 5 {Variable} 'CONSTANT' -> 'const INTEGER'
; ONE-NEXT: +[004] 4 {TypeAlias} 'INTEGER' -> 'int'
Copy link
Contributor Author

@jalopezg-git jalopezg-git Jun 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm... that is weird, given that for comparison, sort mode is line (see LVOptions.cpp:219), unless I am mistaken, INTEGER should have come before CONSTANT, but it was not the case. Was this (partially) broken before?

FYI, @CarlosAlbertoEnciso.

; ONE-NEXT:  [004]     5             {Variable} 'CONSTANT' -> 'const INTEGER'
; ONE-NEXT: +[004]     4             {TypeAlias} 'INTEGER' -> 'int'

@@ -55,8 +55,8 @@
; ONE-NEXT: [002] 1 {TypeAlias} 'INTPTR' -> '* const int'
; ONE-NEXT: [002] 2 {Function} extern not_inlined 'foo' -> 'int'
; ONE-NEXT: [003] {Block}
; ONE-NEXT: [004] 5 {Variable} 'CONSTANT' -> 'const INTEGER'
Copy link
Contributor Author

@jalopezg-git jalopezg-git Jun 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as before (see my other comment) -- also seen in llvm/test/tools/llvm-debuginfo-analyzer/WebAssembly/01-wasm-compare-logical-elements.test.

Fix llvm::concat_iterator for the case of `ValueT` being a pointer
to a common base class to which the result of dereferencing any
iterator in `ItersT` can be casted to.
Remove the `LVScope::Children` container and use `llvm::concat()`
instead to return a view over the types, symbols, and sub-scopes
contained in a given `LVScope`.
@CarlosAlbertoEnciso
Copy link
Member

The changes that introduce the none : unsorted output should be in a separated patch, as it is a general feature, independent of the children removal and associated with LVObject::ID.

@jalopezg-git
Copy link
Contributor Author

jalopezg-git commented Jun 25, 2025

The changes that introduce the none : unsorted output should be in a separated patch, as it is a general feature, independent of the children removal and associated with LVObject::ID.

I removed the change from this PR. Actually, I am not totally convinced that we should have it, given that we also have --output-sort=offset, which should yield the same result (?)

--output-sort=none should perform a bit better though, as it avoids calling LVScope::sort() on the root scope. Thus, I have created #145761.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[llvm-debuginfo-analyzer] Remove 'LVScope::Children' container.
4 participants