Skip to content

Conversation

@LeslieXMOS
Copy link
Contributor

The function mul_boggle will find the closest output scale and bias, and vlashr_shr parameters such that it will not overflow in vpu calculation.

The current code will have an infinite loop when the required vlashr_shr is smaller than vdepth8_shr or (vlsat_shr + vlmul_shr + vdepth8_shr).
As vdepth8_shr and vlsat_shr are unsigned variable, it will underflow when vlashr_shr is too small.
This cause pow(2, y) in L51 and L63 can never output value smaller than 1, thus the loop never ends.

This pull request updated vdepth8_shr and vlsat_shr to signed variable, prevent underflow in the operation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant