Common shared WorkerPool for both Potree1 and Potree2 pointclouds #182
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,74 +1,54 @@ | ||
| import { AutoTerminatingWorker, WorkerQueue } from './worker-queue'; | ||
|
|
||
| export enum WorkerType { | ||
| // Potree 1 workers | ||
| BINARY_DECODER_WORKER = 'BINARY_DECODER_WORKER', | ||
|
|
||
| // Potree 2 workers | ||
| DECODER_WORKER = 'DECODER_WORKER', | ||
| DECODER_WORKER_GLTF = 'DECODER_WORKER_GLTF', | ||
| } | ||
|
|
||
| export const DEFAULT_MAX_WORKERS_PER_POOL = 32; | ||
|
|
||
| export class WorkerPool { | ||
|
There was a problem hiding this comment. The new WorkerPool is a Singleton and has a pool that consists of multiple WorkerQueues for each worker type. We set a maximum number of workers per queue, and this allows us to manage all the different Queues in a single place |
||
| public _maxWorkersPerPool = DEFAULT_MAX_WORKERS_PER_POOL; | ||
|
|
||
| private static instance: WorkerPool | undefined; | ||
| private constructor() {} | ||
|
|
||
| private pool: { [key in WorkerType]: WorkerQueue } = { | ||
| BINARY_DECODER_WORKER: new WorkerQueue( | ||
| this._maxWorkersPerPool, | ||
| require('../workers/binary-decoder.worker.js').default, | ||
| ), | ||
| DECODER_WORKER: new WorkerQueue( | ||
| this._maxWorkersPerPool, | ||
| require('../loading2/decoder.worker.js').default, | ||
| ), | ||
| DECODER_WORKER_GLTF: new WorkerQueue( | ||
| this._maxWorkersPerPool, | ||
| require('../loading2/gltf-decoder.worker.js').default, | ||
| ), | ||
| }; | ||
|
|
||
| static getInstance(): WorkerPool { | ||
| if (!this.instance) { | ||
| this.instance = new WorkerPool(); | ||
| } | ||
|
|
||
| return this.instance; | ||
| } | ||
|
|
||
| set maxWorkersPerPool(count: number) { | ||
| Object.entries(this.pool).forEach(([_, pool]) => (pool.maxWorkers = count)); | ||
| } | ||
|
|
||
| public getWorker(workerType: WorkerType): Promise<AutoTerminatingWorker> { | ||
| return this.pool[workerType].getWorker(); | ||
| } | ||
|
|
||
| public releaseWorker(workerType: WorkerType, worker: AutoTerminatingWorker): void { | ||
| return this.pool[workerType].releaseWorker(worker); | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| import { AsyncBlockingQueue } from './async-blocking-queue'; | ||
|
|
||
| export class AutoTerminatingWorker { | ||
| private timeoutId: number | undefined = undefined; | ||
| private terminated: boolean = false; | ||
|
|
||
| constructor(private wrappedWorker: Worker, private maxIdle: number) {} | ||
|
|
||
| public get worker(): Worker { | ||
| return this.wrappedWorker; | ||
| } | ||
|
|
||
| get isTerminated(): boolean { | ||
| return this.terminated; | ||
| } | ||
|
|
||
| markIdle(): void { | ||
| this.timeoutId = window.setTimeout(() => { | ||
| this.terminated = true; | ||
| this.wrappedWorker.terminate(); | ||
| }, this.maxIdle); | ||
| } | ||
|
|
||
| markInUse(): void { | ||
| if (this.timeoutId) { | ||
| window.clearTimeout(this.timeoutId); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| export class WorkerQueue { | ||
|
There was a problem hiding this comment. Renamed and moved the previous Potree1 WorkerPool -> WorkerQueue. The WorkerQueue is responsible for returning a single worker type The AsyncBlockingQueue with AutoTerminatingWorkers are awesome. By reusing this functionality as a Queue, we know that workers will be cleaned up when they are not being used |
||
| /** | ||
| * The maximum amount of idle time that can elapse before a worker from this pool is automatically terminated | ||
| */ | ||
| private static readonly QUEUE_MAX_IDLE = 7000; | ||
|
|
||
| private queue = new AsyncBlockingQueue<AutoTerminatingWorker>(); | ||
| private queueSize = 0; | ||
|
|
||
| constructor(public maxWorkers: number, private workerType: any) {} | ||
|
|
||
| /** | ||
| * Returns a worker promise which is resolved when one is available. | ||
| */ | ||
| public getWorker(): Promise<AutoTerminatingWorker> { | ||
| // If the number of active workers is smaller than the maximum, return a new one. | ||
| // Otherwise, return a promise for worker from the pool. | ||
| if (this.queueSize < this.maxWorkers) { | ||
| this.queueSize++; | ||
| return Promise.resolve( | ||
| new AutoTerminatingWorker(new this.workerType(), WorkerQueue.QUEUE_MAX_IDLE), | ||
| ); | ||
| } else { | ||
| return this.queue.dequeue().then(worker => { | ||
| worker.markInUse(); | ||
| // If the dequeued worker has been terminated, decrease the pool size and make a recursive call to get a new worker | ||
| if (worker.isTerminated) { | ||
| this.queueSize--; | ||
| return this.getWorker(); | ||
| } | ||
| return worker; | ||
| }); | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Releases a Worker back into the pool | ||
| * @param worker | ||
| */ | ||
| public releaseWorker(worker: AutoTerminatingWorker): void { | ||
| worker.markIdle(); | ||
| this.queue.enqueue(worker); | ||
| } | ||
| } | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Previously setting
maxLoaderWorkershad no impact on Potree2 pointclouds, which could spawn an unlimited number of workers. Now, we can set the maximum number of workers that each pool is allowed to spawn