Skip to content

fix(apiquery): format float32 query values at float32 precision - #87

Open
latent-9 wants to merge 1 commit into
anthropics:mainfrom
latent-9:fix-query-float32-precision
Open

fix(apiquery): format float32 query values at float32 precision#87
latent-9 wants to merge 1 commit into
anthropics:mainfrom
latent-9:fix-query-float32-precision

Conversation

@latent-9

Copy link
Copy Markdown

What

The query encoder shared a single case between reflect.Float32 and reflect.Float64 and formatted both with bitSize 64.

Why

strconv.FormatFloat(value.Float(), 'f', -1, 64) produces the shortest representation that round-trips as a float64. For a float32 like 0.1 that yields 0.10000000149011612 instead of 0.1, so float32 query parameters went out with float64 noise digits. The form encoder in this repo already splits the two cases (internal/apiform/encoder.go) and formats Float32 at bitSize 32; the query encoder now mirrors that.

Change

Split the case and format Float32 at bitSize 32, plus a float32(0.1) row in the encoder table test.

The query encoder shared one case between Float32 and Float64 and
formatted both with bitSize 64, so a float32 like 0.1 was serialized as
0.10000000149011612 instead of 0.1. The form encoder already splits the
two cases; mirror that here, and cover float32 in the encoder table.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant