Performance: /api/content_node/checklist_nodes #7114

pmattmann · 2025-03-26T21:19:28Z

Add Index ContentNode->strategy
improves joins with strategy-filter
ContentNode->root not nullable
left join becomes inner join

Auswirkung
auf /api/content_node/checklist_nodes?camp=/camps/camp_id

Old: (cost=348.72..348.73 rows=1 width=1453)
New: (cost=186.07..186.07 rows=1 width=1453)

Old:

Explain query

Sort  (cost=348.72..348.73 rows=1 width=1453)
  Sort Key: c0_.createtime DESC, c5_."position"
  ->  Nested Loop Semi Join  (cost=221.45..348.71 rows=1 width=1453)
        Join Filter: ((c0_.rootid)::text = (a.rootcontentnodeid)::text)
        ->  Nested Loop Semi Join  (cost=220.48..317.83 rows=1 width=1466)
              Join Filter: ((c0_.rootid)::text = (a7_.rootcontentnodeid)::text)
              ->  Nested Loop Left Join  (cost=220.17..300.60 rows=17 width=1453)
                    ->  Hash Join  (cost=220.03..297.19 rows=17 width=1375)
                          Hash Cond: ((c0_.contenttypeid)::text = (c4_.id)::text)
                          ->  Hash Right Join  (cost=202.60..279.72 rows=17 width=1162)
                                Hash Cond: ((c2_.id)::text = (c0_.parentid)::text)
                                ->  Seq Scan on content_node c2_  (cost=0.00..65.12 rows=793 width=278)
                                      Filter: ((strategy)::text = ANY ('{contentnode,checklistnode,columnlayout,storyboard,responsivelayout,singletext,materialnode,multiselect}'::text[]))
                                ->  Hash  (cost=202.39..202.39 rows=17 width=884)
                                      ->  Hash Left Join  (cost=134.15..202.39 rows=17 width=884)
                                            Hash Cond: ((c0_.rootid)::text = (c1_.id)::text)
                                            ->  Hash Right Join  (cost=74.84..143.03 rows=17 width=606)
                                                  Hash Cond: ((c3_.parentid)::text = (c0_.id)::text)
                                                  ->  Seq Scan on content_node c3_  (cost=0.00..65.12 rows=793 width=278)
                                                        Filter: ((strategy)::text = ANY ('{contentnode,checklistnode,columnlayout,storyboard,responsivelayout,singletext,materialnode,multiselect}'::text[]))
                                                  ->  Hash  (cost=74.63..74.63 rows=17 width=328)
                                                        ->  Hash Right Join  (cost=56.66..74.63 rows=17 width=328)
                                                              Hash Cond: ((c6_.checklistnode_id)::text = (c0_.id)::text)
                                                              ->  Seq Scan on checklistnode_checklistitem c6_  (cost=0.00..16.30 rows=630 width=100)
                                                              ->  Hash  (cost=56.45..56.45 rows=17 width=278)
                                                                    ->  Seq Scan on content_node c0_  (cost=0.00..56.45 rows=17 width=278)
                                                                          Filter: ((strategy)::text = 'checklistnode'::text)
                                            ->  Hash  (cost=56.45..56.45 rows=229 width=278)
                                                  ->  Seq Scan on content_node c1_  (cost=0.00..56.45 rows=229 width=278)
                                                        Filter: ((strategy)::text = 'columnlayout'::text)
                          ->  Hash  (cost=13.30..13.30 rows=330 width=213)
                                ->  Seq Scan on content_type c4_  (cost=0.00..13.30 rows=330 width=213)
                    ->  Index Scan using checklist_item_pkey on checklist_item c5_  (cost=0.14..0.20 rows=1 width=128)
                          Index Cond: ((id)::text = (c6_.checklistitem_id)::text)
              ->  Materialize  (cost=0.30..16.98 rows=1 width=13)
                    ->  Nested Loop  (cost=0.30..16.98 rows=1 width=13)
                          ->  Nested Loop  (cost=0.16..16.74 rows=1 width=13)
                                ->  Seq Scan on schedule_entry s8_  (cost=0.00..6.77 rows=277 width=26)
                                ->  Memoize  (cost=0.16..0.35 rows=1 width=50)
                                      Cache Key: s8_.periodid
                                      Cache Mode: logical
                                      ->  Index Scan using period_pkey on period p9_  (cost=0.15..0.34 rows=1 width=50)
                                            Index Cond: ((id)::text = (s8_.periodid)::text)
                                            Filter: ((campid)::text = '6973c230d6b1'::text)
                          ->  Index Scan using activity_pkey on activity a7_  (cost=0.14..0.23 rows=1 width=26)
                                Index Cond: ((id)::text = (s8_.activityid)::text)
        ->  Hash Join  (cost=0.97..30.87 rows=1 width=36)
              Hash Cond: ((c.id)::text = (a.campid)::text)
              ->  Append  (cost=0.14..29.19 rows=61 width=132)
                    ->  Nested Loop  (cost=0.14..20.11 rows=60 width=132)
                          ->  Index Only Scan using user_pkey on "user" u  (cost=0.14..8.16 rows=1 width=50)
                                Index Cond: (id = '9145944210a7'::text)
                          ->  Seq Scan on camp c  (cost=0.00..11.20 rows=60 width=50)
                                Filter: isprototype
                    ->  Subquery Scan on "*SELECT* 2"  (cost=0.14..8.17 rows=1 width=132)
                          ->  Index Scan using idx_c93898a7b00651c on camp_collaboration cc  (cost=0.14..8.16 rows=1 width=150)
                                Index Cond: ((status)::text = 'established'::text)
                                Filter: ((userid)::text = '9145944210a7'::text)
              ->  Hash  (cost=0.81..0.81 rows=2 width=72)
                    ->  Append  (cost=0.14..0.81 rows=2 width=72)
                          ->  Index Scan using uniq_ac74095af886581c on activity a  (cost=0.14..0.30 rows=1 width=26)
                                Index Cond: ((rootcontentnodeid)::text = (a7_.rootcontentnodeid)::text)
                          ->  Index Scan using uniq_64c19c1f886581c on category ca  (cost=0.14..0.50 rows=1 width=100)
                                Index Cond: ((rootcontentnodeid)::text = (a7_.rootcontentnodeid)::text)

New:

Explain query

Sort  (cost=186.07..186.07 rows=1 width=1453)
  Sort Key: c0_.createtime DESC, c5_."position"
  ->  Nested Loop Semi Join  (cost=61.38..186.06 rows=1 width=1453)
        Join Filter: ((c0_.rootid)::text = (a.rootcontentnodeid)::text)
        ->  Nested Loop Semi Join  (cost=60.58..155.35 rows=1 width=1466)
              Join Filter: ((c0_.rootid)::text = (a7_.rootcontentnodeid)::text)
              ->  Nested Loop Left Join  (cost=60.13..151.28 rows=3 width=1453)
                    ->  Nested Loop Left Join  (cost=59.99..150.67 rows=3 width=1375)
                          ->  Nested Loop  (cost=59.84..141.51 rows=3 width=1325)
                                ->  Nested Loop Left Join  (cost=59.69..132.52 rows=3 width=1112)
                                      ->  Nested Loop Left Join  (cost=58.21..107.76 rows=3 width=834)
                                            ->  Hash Join  (cost=57.94..91.30 rows=3 width=556)
                                                  Hash Cond: ((c0_.rootid)::text = (c1_.id)::text)
                                                  ->  Bitmap Heap Scan on content_node c0_  (cost=4.28..37.60 rows=17 width=278)
                                                        Recheck Cond: ((strategy)::text = 'checklistnode'::text)
                                                        ->  Bitmap Index Scan on idx_481d0580144645ed  (cost=0.00..4.28 rows=17 width=0)
                                                              Index Cond: ((strategy)::text = 'checklistnode'::text)
                                                  ->  Hash  (cost=50.79..50.79 rows=229 width=278)
                                                        ->  Bitmap Heap Scan on content_node c1_  (cost=5.93..50.79 rows=229 width=278)
                                                              Recheck Cond: ((strategy)::text = 'columnlayout'::text)
                                                              ->  Bitmap Index Scan on idx_481d0580144645ed  (cost=0.00..5.87 rows=229 width=0)
                                                                    Index Cond: ((strategy)::text = 'columnlayout'::text)
                                            ->  Index Scan using content_node_pkey on content_node c2_  (cost=0.28..5.48 rows=1 width=278)
                                                  Index Cond: ((id)::text = (c0_.parentid)::text)
                                                  Filter: ((strategy)::text = ANY ('{contentnode,checklistnode,columnlayout,storyboard,responsivelayout,singletext,materialnode,multiselect}'::text[]))
                                      ->  Bitmap Heap Scan on content_node c3_  (cost=1.48..8.24 rows=2 width=278)
                                            Recheck Cond: ((c0_.id)::text = (parentid)::text)
                                            Filter: ((strategy)::text = ANY ('{contentnode,checklistnode,columnlayout,storyboard,responsivelayout,singletext,materialnode,multiselect}'::text[]))
                                            ->  Bitmap Index Scan on idx_481d058010ee4cee  (cost=0.00..1.48 rows=3 width=0)
                                                  Index Cond: ((parentid)::text = (c0_.id)::text)
                                ->  Index Scan using content_type_pkey on content_type c4_  (cost=0.15..2.99 rows=1 width=213)
                                      Index Cond: ((id)::text = (c0_.contenttypeid)::text)
                          ->  Index Scan using idx_5a2b5b31de6b6f00 on checklistnode_checklistitem c6_  (cost=0.15..3.03 rows=3 width=100)
                                Index Cond: ((checklistnode_id)::text = (c0_.id)::text)
                    ->  Index Scan using checklist_item_pkey on checklist_item c5_  (cost=0.14..0.20 rows=1 width=128)
                          Index Cond: ((id)::text = (c6_.checklistitem_id)::text)
              ->  Nested Loop  (cost=0.45..1.35 rows=1 width=13)
                    ->  Nested Loop  (cost=0.29..0.62 rows=2 width=26)
                          ->  Index Scan using uniq_ac74095af886581c on activity a7_  (cost=0.14..0.25 rows=1 width=26)
                                Index Cond: ((rootcontentnodeid)::text = (c1_.id)::text)
                          ->  Index Scan using idx_d7785d2c1335e2fc on schedule_entry s8_  (cost=0.15..0.35 rows=2 width=26)
                                Index Cond: ((activityid)::text = (a7_.id)::text)
                    ->  Memoize  (cost=0.16..0.35 rows=1 width=50)
                          Cache Key: s8_.periodid
                          Cache Mode: logical
                          ->  Index Scan using period_pkey on period p9_  (cost=0.15..0.34 rows=1 width=50)
                                Index Cond: ((id)::text = (s8_.periodid)::text)
                                Filter: ((campid)::text = '6973c230d6b1'::text)
        ->  Hash Join  (cost=0.80..30.69 rows=1 width=36)
              Hash Cond: ((c.id)::text = (a.campid)::text)
              ->  Append  (cost=0.14..29.19 rows=61 width=132)
                    ->  Nested Loop  (cost=0.14..20.11 rows=60 width=132)
                          ->  Index Only Scan using user_pkey on "user" u  (cost=0.14..8.16 rows=1 width=50)
                                Index Cond: (id = '9145944210a7'::text)
                          ->  Seq Scan on camp c  (cost=0.00..11.20 rows=60 width=50)
                                Filter: isprototype
                    ->  Subquery Scan on "*SELECT* 2"  (cost=0.14..8.17 rows=1 width=132)
                          ->  Index Scan using idx_c93898a7b00651c on camp_collaboration cc  (cost=0.14..8.16 rows=1 width=150)
                                Index Cond: ((status)::text = 'established'::text)
                                Filter: ((userid)::text = '9145944210a7'::text)
              ->  Hash  (cost=0.63..0.63 rows=2 width=72)
                    ->  Append  (cost=0.14..0.63 rows=2 width=72)
                          ->  Index Scan using uniq_ac74095af886581c on activity a  (cost=0.14..0.25 rows=1 width=26)
                                Index Cond: ((rootcontentnodeid)::text = (c1_.id)::text)
                          ->  Index Scan using uniq_64c19c1f886581c on category ca  (cost=0.14..0.37 rows=1 width=100)
                                Index Cond: ((rootcontentnodeid)::text = (c1_.id)::text)

carlobeltrame · 2025-03-26T23:14:13Z

api/src/Entity/ContentNode.php

@@ -51,6 +51,9 @@
 #[ORM\InheritanceType('SINGLE_TABLE')]
 #[ORM\DiscriminatorColumn(name: 'strategy', type: 'string')]
 #[ORM\UniqueConstraint(name: 'contentnode_parentid_slot_position_unique', columns: ['parentid', 'slot', 'position'])]
+#[ORM\Index(columns: ['createTime'])]


Are the createTime and updateTime inices also new? Why are they not in the migration?

Actually, these indexes are already on BaseEntity. If they are not repeated here, they are omitted.
I suspect it is a bug in Doctrine.

what do you mean by omitted?
they are in the schema and in the migrations. why do you need to add them again on the subclass entity?

If I add only the Index(category) to ContentNode, then the two indexes of BaseEntity are lost.
It looks as if the index attributes are not aggregated - but the mostly specific class with index attributes determines the list of indexes.

carlobeltrame · 2025-03-26T23:14:40Z

api/src/Entity/ContentNode.php

@@ -62,7 +65,7 @@ abstract class ContentNode extends BaseEntity implements BelongsToContentNodeTre
    #[Gedmo\SortableGroup] // this is needed to avoid that all root nodes are in the same sort group (parent:null, slot: '')
    #[Groups(['read'])]
    #[ORM\ManyToOne(targetEntity: ColumnLayout::class, inversedBy: 'rootDescendants')]
-    #[ORM\JoinColumn(nullable: true, onDelete: 'CASCADE')] // TODO make not null in the DB using a migration, and get fixtures to run
+    #[ORM\JoinColumn(nullable: false, onDelete: 'CASCADE')]


Nice, so this finally works now

yes - and it improves performance :)

carlobeltrame · 2025-03-26T23:15:58Z

api/tests/Api/ContentNodes/RootColumnLayout/CreateRootColumnLayoutTest.php

@@ -35,7 +35,7 @@ public function testCreateColumnLayoutValidatesMissingParent() {
        $this->assertJsonContains([
            'violations' => [
                [
-                    'propertyPath' => 'parent',
+                    'propertyPath' => 'root',


The test name still says the parent may not be missing, not the root. Should this be a new test instead? Or do we just need to adjust the test's name?

Unfortunately no.
parent is the mandatory field. But without parent root is not determined.
Unfortunately root is checked first.
It is correct that a ValidationError occurs. However, the message is unfortunate.
Unfortunately, I have not found a better solution.

BacLuc · 2025-03-29T14:17:11Z

api/fixtures/activity1.yml

    parent: null
+    root: '@self'


I think you don't need to change the order of the properties.
(i reverted this could still run the CreateColumnLayout tests).

I would have preferred it if we did not have this unnecessary change in the history.

pmattmann requested a review from a team March 26, 2025 21:19

pmattmann changed the title ~~Performance /api/content_node/checklist_nodes~~ Performance: /api/content_node/checklist_nodes Mar 26, 2025

pmattmann added the Performance label Mar 26, 2025

carlobeltrame reviewed Mar 26, 2025

View reviewed changes

BacLuc approved these changes Mar 29, 2025

View reviewed changes

BacLuc requested a review from a team March 29, 2025 14:18

Index ContentNode.Strategy; ContentNode.RootId not nullable

be75ad0

pmattmann force-pushed the feature/perf-content-nodes branch from 177a351 to be75ad0 Compare March 29, 2025 15:19

pmattmann mentioned this pull request Mar 30, 2025

Improve Performance #6461

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Performance: /api/content_node/checklist_nodes #7114

Performance: /api/content_node/checklist_nodes #7114

pmattmann commented Mar 26, 2025 •

edited

Loading

carlobeltrame Mar 26, 2025

pmattmann Mar 26, 2025

BacLuc Mar 29, 2025

pmattmann Mar 29, 2025

carlobeltrame Mar 26, 2025

pmattmann Mar 26, 2025

carlobeltrame Mar 26, 2025

pmattmann Mar 26, 2025

BacLuc Mar 29, 2025

pmattmann Mar 29, 2025

Performance: /api/content_node/checklist_nodes #7114

Are you sure you want to change the base?

Performance: /api/content_node/checklist_nodes #7114

Conversation

pmattmann commented Mar 26, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pmattmann commented Mar 26, 2025 •

edited

Loading