Revise section about millions of Entities (#1927)

Author: lognaturel

docs/entities-intro.rst

@@ -124,11 +124,21 @@ Can I have millions of Entities?
 
 There are two current limitations that make millions of Entities impractical: data transfer and form performance.
 
-Currently, all Entities that have not been deleted are sent to every device on every update. Depending on your data connection, this may be a limiting factor for your project. We will eventually add support for archiving Entities to address this limitation.
+Currently, all Entities that have not been deleted are sent to every device on every update. Depending on your data connection, this may be a limiting factor for your project. To address this, we will eventually add functionality such as archiving Entities, synchronizing only updated Entities, or assigning a subset of Entities to a user.
 
-Entities are currently represented in memory for access by forms. Modern devices can easily process multiple tens of thousands of entities in this way, but your form may become slow or crash if you have more than 50,000 Entities.
+Starting in Collect v2024.3 and Central v2024.3, Entities are optimized so that larger numbers can be used in forms. If your choice filters and :ref:`lookup expressions <referencing-values-in-datasets>` use simple expressions with ``=``, ``and`` and ``or``, they will be performed quickly. We verify that a form with 100,000 Entities with 6 properties that are each 36 random characters long is usable on a device from 2021 with 3 GB of RAM.
 
-We are actively working on addressing these performance limitations and expect significant improvements by late 2024. In the mean time, one possible workaround is to use `pulldata <https://xlsform.org/en/#how-to-pull-data-from-csv>`_ and `search() <https://xlsform.org/en/#dynamic-selects-from-pre-loaded-data>`_ instead of `instance` and `select_one_from_file`. These methods are less flexible but they will perform better.
+If you use function calls in choice filters or lookup expressions, the entire Entity List may need to be stored in memory. Modern devices can easily process multiple tens of thousands of entities in this way, but your form may become slow or crash if you have more than 50,000 Entities.
+
+The number of properties that each Entity has and the size and uniqueness of values saved will also affect both data transfer and form performance.
+
+.. seealso::
+    :ref:`What form fields should I save to my Entities as properties? <entities-intro-form-fields>`
+
+If you have a form with may Entities that feels slow, we encourage you to post about it on `the forum <https://forum.getodk.org/c/support/6>`_ so that we can recommend approaches that will make it work faster or design improvements to the system.
+
+.. note::
+    We generally do not recommend using `search() <https://xlsform.org/en/#dynamic-selects-from-pre-loaded-data>`_ instead of ``select_one_from_file`` because it does not work with offline Entities and is less flexible. Starting in Collect v2024.3 and Central v2024.3, there is limited performance benefit to ``search()``.
 
 My form captures data on multiple different things, can I create multiple Entities with a single submission?
 ------------------------------------------------------------------------------------------------------------
@@ -232,6 +242,8 @@ You can access a specific Entity's properties using a :ref:`lookup expression <r
     * :doc:`Community reporting tutorial <tutorial-community-reporting>`
     * :ref:`Build a form that uses Entities <central-entities-follow-up-forms>`
 
+.. _entities-intro-form-fields:
+
 What form fields should I save to my Entities as properties?
 ------------------------------------------------------------