Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix refill logic in nextDoc(). #14185

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

jpountz
Copy link
Contributor

@jpountz jpountz commented Jan 30, 2025

The recent optimization from #14164 interfered in a bad way with a prior optimization.

The recent optimization from apache#14164 interfered in a bad way with a prior
optimization.
@jpountz jpountz added this to the 10.2.0 milestone Jan 30, 2025
@jpountz
Copy link
Contributor Author

jpountz commented Jan 30, 2025

Benchmarks suggest that the additional overhead in nextDoc() is fine:

                            TaskQPS baseline      StdDevQPS my_modified_version      StdDev                Pct diff p-value
                      OrHighRare      517.98      (4.5%)      506.55      (6.5%)   -2.2% ( -12% -    9%) 0.211
                DismaxOrHighHigh      163.10      (2.2%)      160.59      (3.4%)   -1.5% (  -6% -    4%) 0.090
             FilteredOrStopWords       71.58      (6.1%)       70.50      (6.1%)   -1.5% ( -12% -   11%) 0.433
                       OrHighMed      292.53      (2.3%)      288.61      (2.4%)   -1.3% (  -5% -    3%) 0.066
              FilteredOrHighHigh       97.76      (6.8%)       96.47      (6.0%)   -1.3% ( -13% -   12%) 0.514
                     OrStopWords       44.42      (3.4%)       43.84      (3.7%)   -1.3% (  -8% -    5%) 0.240
                      OrHighHigh       67.46      (2.2%)       66.59      (1.9%)   -1.3% (  -5% -    2%) 0.050
                        Wildcard      131.06      (3.0%)      129.44      (4.2%)   -1.2% (  -8% -    6%) 0.280
             And2Terms2StopWords      267.40      (3.7%)      264.15      (3.2%)   -1.2% (  -7% -    5%) 0.264
                    AndStopWords       40.26      (3.9%)       39.78      (3.8%)   -1.2% (  -8% -    6%) 0.322
                        PKLookup      374.88      (3.7%)      370.62      (3.9%)   -1.1% (  -8% -    6%) 0.342
                       And3Terms      236.13      (3.1%)      233.61      (3.3%)   -1.1% (  -7% -    5%) 0.287
                 DismaxOrHighMed      249.56      (2.0%)      247.08      (2.6%)   -1.0% (  -5% -    3%) 0.178
                          OrMany       25.96      (5.5%)       25.71      (4.0%)   -1.0% (  -9% -    9%) 0.529
                         Prefix3      235.88      (3.1%)      233.63      (4.0%)   -1.0% (  -7% -    6%) 0.395
              Or2Terms2StopWords      253.41      (3.5%)      251.17      (3.7%)   -0.9% (  -7% -    6%) 0.439
                  FilteredOrMany       20.15      (9.8%)       19.98      (9.3%)   -0.8% ( -18% -   20%) 0.779
                      AndHighMed      176.43      (2.6%)      174.96      (2.8%)   -0.8% (  -6% -    4%) 0.328
                     AndHighHigh       57.64      (3.0%)       57.16      (3.5%)   -0.8% (  -7% -    5%) 0.423
                FilteredOr3Terms      234.70      (6.7%)      233.02      (5.9%)   -0.7% ( -12% -   12%) 0.719
      FilteredOr2Terms2StopWords      237.64      (5.6%)      235.96      (5.3%)   -0.7% ( -11% -   10%) 0.683
                      TermDTSort      606.25      (4.7%)      602.01      (3.8%)   -0.7% (  -8% -    8%) 0.605
              CombinedOrHighHigh       29.13      (2.7%)       28.93      (2.7%)   -0.7% (  -5% -    4%) 0.432
               FilteredOrHighMed      236.89      (6.5%)      235.59      (5.9%)   -0.5% ( -12% -   12%) 0.781
               CombinedOrHighMed      116.23      (2.4%)      115.60      (2.7%)   -0.5% (  -5% -    4%) 0.503
                      DismaxTerm      823.43      (4.1%)      819.10      (5.2%)   -0.5% (  -9% -    9%) 0.721
                        Or3Terms      220.05      (3.9%)      219.03      (3.9%)   -0.5% (  -7% -    7%) 0.706
                 CountAndHighMed      425.17      (3.3%)      423.34      (2.8%)   -0.4% (  -6% -    5%) 0.656
                  TermBGroup1M1P       53.55      (2.7%)       53.35      (2.8%)   -0.4% (  -5% -    5%) 0.673
     FilteredAnd2Terms2StopWords      308.51      (2.3%)      307.49      (2.6%)   -0.3% (  -5% -    4%) 0.669
              FilteredAndHighMed      184.14      (2.4%)      183.64      (2.7%)   -0.3% (  -5% -    4%) 0.735
                AndMedOrHighHigh       91.28      (3.5%)       91.04      (3.1%)   -0.3% (  -6% -    6%) 0.803
                    CombinedTerm       50.34      (2.4%)       50.27      (3.1%)   -0.2% (  -5% -    5%) 0.863
             FilteredAndHighHigh       98.97      (4.6%)       98.83      (4.4%)   -0.1% (  -8% -    9%) 0.918
                     CountPhrase        7.07      (2.9%)        7.06      (2.7%)   -0.1% (  -5% -    5%) 0.878
                    TermGroup10K       30.48      (2.4%)       30.44      (1.8%)   -0.1% (  -4% -    4%) 0.851
               TermDayOfYearSort     3408.11      (1.9%)     3403.87      (1.5%)   -0.1% (  -3% -    3%) 0.821
                         Respell       76.33      (2.3%)       76.24      (2.4%)   -0.1% (  -4% -    4%) 0.875
                  CountOrHighMed      522.17      (3.3%)      521.60      (2.9%)   -0.1% (  -6% -    6%) 0.912
                     TermGroup1M       32.16      (2.1%)       32.14      (1.7%)   -0.1% (  -3% -    3%) 0.917
         CountFilteredOrHighHigh      221.09      (3.1%)      220.96      (2.7%)   -0.1% (  -5% -    5%) 0.949
          CountFilteredOrHighMed      241.72      (2.3%)      241.63      (1.8%)   -0.0% (  -4% -    4%) 0.953
                    TermBGroup1M       43.00      (2.6%)       42.98      (2.5%)   -0.0% (  -5% -    5%) 0.968
             CountFilteredOrMany       40.87      (4.9%)       40.86      (4.2%)   -0.0% (  -8% -    9%) 0.988
             CombinedAndHighHigh       24.90      (3.4%)       24.90      (3.1%)   -0.0% (  -6% -    6%) 0.988
                       CountTerm    15466.46      (3.6%)    15465.67      (4.6%)   -0.0% (  -7% -    8%) 0.997
                    TermGroup100       33.90      (2.4%)       33.91      (2.9%)    0.0% (  -5% -    5%) 0.993
              CombinedAndHighMed       96.43      (3.3%)       96.44      (2.9%)    0.0% (  -6% -    6%) 0.992
                 CountOrHighHigh      465.82      (6.8%)      466.21      (5.3%)    0.1% ( -11% -   13%) 0.965
                CountAndHighHigh      509.45      (6.6%)      509.90      (5.7%)    0.1% ( -11% -   13%) 0.964
                          Fuzzy1      111.93      (2.9%)      112.05      (2.6%)    0.1% (  -5% -    5%) 0.907
                          Fuzzy2      105.56      (2.6%)      105.69      (2.5%)    0.1% (  -4% -    5%) 0.887
               FilteredAnd3Terms      270.45      (2.4%)      270.82      (3.6%)    0.1% (  -5% -    6%) 0.888
                   TermTitleSort      236.14      (2.2%)      236.48      (2.2%)    0.1% (  -4% -    4%) 0.836
                   TermMonthSort     3512.78      (2.0%)     3518.37      (2.5%)    0.2% (  -4% -    4%) 0.820
            FilteredAndStopWords       81.77      (4.4%)       81.90      (4.4%)    0.2% (  -8% -    9%) 0.908
                          IntNRQ      222.55     (12.7%)      223.07      (9.2%)    0.2% ( -19% -   25%) 0.946
                    FilteredTerm      269.47      (2.4%)      270.24      (2.2%)    0.3% (  -4% -    4%) 0.693
                  FilteredPhrase       55.62      (2.7%)       55.89      (2.8%)    0.5% (  -4% -    6%) 0.575
                 AndHighOrMedMed       71.48      (2.5%)       71.83      (2.5%)    0.5% (  -4% -    5%) 0.533
             CountFilteredPhrase       50.57      (3.3%)       50.82      (2.9%)    0.5% (  -5% -    6%) 0.618
                          Phrase       23.97      (3.5%)       24.13      (3.4%)    0.6% (  -6% -    7%) 0.561
                            Term      786.42      (4.5%)      799.69      (3.5%)    1.7% (  -6% -   10%) 0.187

@jpountz jpountz requested a review from gf2121 January 30, 2025 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant