Reduce quantization optimization steps at ivf query time

[benwtrent]

Since we are quantizing for posting list centroid, I think we can get away with fewer optimization iterations.

Dropping from 5 to 2 reduces latency when hitting many centroids, with no recall impact (at least on my data sets).

baseline:

index_name                      index_type  n_probe  latency(ms)  net_cpu_time(ms)  avg_cpu_count     QPS  recall    visited
------------------------------  ----------  -------  -----------  ----------------  -------------  ------  ------  ---------
cohere-wikipedia-docs-768d.vec         ivf      100         2.43              0.00           0.00   411.52    0.91  23766.65

candidate:

index_name                      index_type  n_probe  latency(ms)  net_cpu_time(ms)  avg_cpu_count     QPS  recall    visited
------------------------------  ----------  -------  -----------  ----------------  -------------  ------  ------  ---------
cohere-wikipedia-docs-768d.vec         ivf      100         1.84              0.00           0.00   543.48    0.91  23766.65

Here is a more extreme case (many segments):

baseline:

index_name                      index_type  n_probe  latency(ms)  net_cpu_time(ms)  avg_cpu_count     QPS  recall    visited
------------------------------  ----------  -------  -----------  ----------------  -------------  ------  ------  ---------
cohere-wikipedia-docs-768d.vec         ivf      100        36.10              0.00           0.00   27.70    0.87  364480.37

candidate:

index_name                      index_type  n_probe  latency(ms)  net_cpu_time(ms)  avg_cpu_count     QPS  recall    visited
------------------------------  ----------  -------  -----------  ----------------  -------------  ------  ------  ---------
cohere-wikipedia-docs-768d.vec         ivf      100        24.94              0.00           0.00   40.10    0.87  364480.37

Need to test against more data sets, but this is a nice improvement.

[elasticsearchmachine]

Pinging @elastic/es-search-relevance (Team:Search Relevance)

[john-wagster]

agreed need to test this more but good results are good and I buy that it makes sense; lgtm

[benwtrent]

@john-wagster

I am still running more benchmarks, but I ran over

glove-normalized-200.train
corpus-dbpedia-entity-arctic-0.fvec
cohere-wikipedia-docs-768d.vec
corpus-quora-E5-small.fvec.flat

And observed no recall difference.

I am running a larger 8M run of cohere-wikipedia-docs-768d.vec. But even with iter=1 at query time, there has been zero impact on recall, with nice query time latency improvements when hitting many centroids.

[benwtrent]

@john-wagster I ran over all 8M of the cohere 768, observed no recall difference between 5 iterations and 1 iteration for multiple segments and force merged to one.

I am thinking we should call it. If we noticed funky recall in other datasets, we can revisit.