Skip to content

Commit c16cf49

Browse files
committed
Fix chunking logic when bsz > 1
1 parent 91b9b12 commit c16cf49

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

model.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -955,7 +955,7 @@ def forward(self,
955955
max_a = self.config.max_attention_size
956956
if attn_size > max_a:
957957
cs = (math.sqrt(past_len ** 2 + 4 * max_a) - past_len) / 2
958-
chunk_size = math.floor(cs)
958+
chunk_size = min(chunk_size, math.floor(cs))
959959

960960
# Process chunk
961961

0 commit comments

Comments
 (0)