-
Notifications
You must be signed in to change notification settings - Fork 5
Description
Problem: Currently the chunksize found in the infoList of EntityInfo.java objects results in the FilePartitioner splitting the file into symmetric chunks. The issue being that we sometimes want to resize the chunksize to reduce or increase strain on the network/service in general.
Solution: Make a class that manages the chunksize found in the FilePartitioner that will enable re-sizing of all the chunks in the ConcurrentQueue found in the FIlePartitioner. Create a controller that will enable a user to re-size the chunks on the fly of an ongoing transfer. An interesting idea is that the filePartioiner will produce a chunk of a size that is the current chunkSize in the class. This way we dont need to pre-compute all the chunks, we can simply query the next chunk based on the chunkSize found in this class.
What we want to do is create a single service that is responsible for sizing the next chunk that the reader will produce.