[JENKINS-75586] add maximum indexing capacity configuration capability

[jenkins-infra-bot]

looking at this line https://github.com/mattmoor/branch-api-plugin/blob/f4890e44a3686e5ae19d0a09ee408148a8f1fb16/src/main/java/jenkins/branch/MultiBranchProject.java#L2293

this is the entire code snippet from the git repo - notice the hard coded number in the second if statement....

public CauseOfBlockage canRun(Queue.Item item) {
            if (item.task instanceof MultiBranchProject) {
                if (indexingCount() > 5) {
                    // TODO make the limit configurable
                    return CauseOfBlockage.fromMessage(Messages._MultiBranchProject_MaxConcurrentIndexing());
                }
            }
            return null;
        } 

I see the maximum indexing capacity is hard coded.

looking at the next line shows a TODO line with "make the limit configurable".

in our environment, we have hundreds of multibranch pipelines and the configuration of this parameter is critical to increase the amount of scanned branches/tag in parallel.

---

Originally reported by amidar, imported from: add maximum indexing capacity configuration capability

• status: Open
• priority: Minor
• component(s): branch-api-plugin
• resolution: Unresolved
• votes: 0
• watchers: 3
• imported: 20251211-141027

Raw content of original issue

looking at this line https://github.com/mattmoor/branch-api-plugin/blob/f4890e44a3686e5ae19d0a09ee408148a8f1fb16/src/main/java/jenkins/branch/MultiBranchProject.java#L2293

this is the entire code snippet from the git repo - notice the hard coded number in the second if statement....

public CauseOfBlockage canRun(Queue.Item item) {
            if (item.task instanceof MultiBranchProject) {
                if (indexingCount() > 5) {
                    // TODO make the limit configurable
                    return CauseOfBlockage.fromMessage(Messages._MultiBranchProject_MaxConcurrentIndexing());
                }
            }
            return null;
        } 

I see the maximum indexing capacity is hard coded.
looking at the next line shows a TODO line with "make the limit configurable".
in our environment, we have hundreds of multibranch pipelines and the configuration of this parameter is critical to increase the amount of scanned branches/tag in parallel.

• environment: All - the feature has not been developed yet.

[jenkins-infra-bot]

amidar:

• Original comment link
• Raw content of original comment:

  hi alice45richards ,
  
  i'll split my answer into two parts:
  
  	
  we are using gitlab. we work in an on prem, air gapped, offline environment, and we control all aspects of the services (hardware, software, configration). we just want to be able to set the number I mentioned in the description, even if it will not take into account any hardware limitations (this will be the "quick and dirty" solution, and will be apapted manually, over time, to our specific needs).
  	
  a better approach for any type of environment/hardware or software might be to simply let the users define all the parameters you mentioned in your reply, and let the system "calculate" the best indexing capacity. I guess it will be wise to set a minimum/maximum limit for safer results, and take into account the hardware settings of the jenkins controller.
  
  
  
  I assume the solution for my first approach shouldn't be that difficult to implement, but will impose all the liability on the user (fine by me, since it is a fully manual solution). the second approach divides the liability between the user, the code and the hardware, but also requires a more complicated logic to implement - hence it is not the quickest way to approach the solution.
  
   
  
  both solutions should also have default values (lets say - the current value) which can be used as a fallback in case some error occurs during operation.
  
   
  
  what do you think?
  

hi alice45richards ,

i'll split my answer into two parts:

1. we are using gitlab. we work in an on prem, air gapped, offline environment, and we control all aspects of the services (hardware, software, configration). we just want to be able to set the number I mentioned in the description, even if it will not take into account any hardware limitations (this will be the "quick and dirty" solution, and will be apapted manually, over time, to our specific needs).
2. a better approach for any type of environment/hardware or software might be to simply let the users define all the parameters you mentioned in your reply, and let the system "calculate" the best indexing capacity. I guess it will be wise to set a minimum/maximum limit for safer results, and take into account the hardware settings of the jenkins controller.

I assume the solution for my first approach shouldn't be that difficult to implement, but will impose all the liability on the user (fine by me, since it is a fully manual solution). the second approach divides the liability between the user, the code and the hardware, but also requires a more complicated logic to implement - hence it is not the quickest way to approach the solution.

 

both solutions should also have default values (lets say - the current value) which can be used as a fallback in case some error occurs during operation.

 

what do you think?

[DidierLoiseau]

I think this issue is no longer relevant since 836c63c. The link in the description points to a repository that hasn’t been updated for 12 years.

I think the relevant code is now https://github.com/jenkinsci/cloudbees-folder-plugin/blob/master/src/main/java/com/cloudbees/hudson/plugins/folder/computed/ThrottleComputationQueueTaskDispatcher.java#L57-L61, and it seems you can control the control this limit with com.cloudbees.hudson.plugins.folder.computed.ThrottleComputationQueueTaskDispatcher.LIMIT. It seems undocumented though, as per jenkinsci/cloudbees-folder-plugin#680.

Cc. @amitdar (in the off chance that you are OP)