Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Numeric Data Type Loading in Power BI #478

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

slabko
Copy link
Contributor

@slabko slabko commented Mar 7, 2025

MS Power BI and other applications were not able to recognize the types of uncommon SQL-world data types, such as UInt8 or Int64. It was unable to parse the data types from SQLColumns because the dataset returned by this function did not comply with the requirements. This is fixed by changing the SQLColumns dataset.

Additionally, SQLGetTypeInfo returned SQL types such as INT, SMALLINT, BIGINT, etc. Each of these types had to have two entries in the SQLGetTypeInfo result: one for signed and one for unsigned types. This duplication confused MS Power BI. The fix for this issue involved following the documentation and using database-native types such as Int32, UInt64, etc.

@slabko slabko force-pushed the unique-type-names branch 6 times, most recently from 56713a8 to 55bf285 Compare March 12, 2025 09:58
slabko added 2 commits March 12, 2025 12:15
Update SQLGetTypeInfo results to comply with
[SQLGetTypeInfo](https://learn.microsoft.com/en-us/sql/odbc/reference/syntax/sqlgettypeinfo-function).
Applications must use the type names returned in the TYPE_NAME column of
the SQLGetTypeInfo result set in ALTER TABLE and CREATE TABLE statements
SQLColumns should return a dataset with columns of specific types. If
incorrect types are returned, some applications fail to deduce column
types and either ignore them or treat them as binary data.

This change ensures each column in the SQLColumns result is converted to
the expected type.
@slabko slabko force-pushed the unique-type-names branch from 1774ca4 to 50c168b Compare March 12, 2025 11:15
@slabko slabko changed the title [WIP]: Fix SQLGetTypeInfo output to match documentation Fix Numeric Data Type Loading in Power BI Mar 12, 2025
@slabko slabko marked this pull request as ready for review March 12, 2025 11:23
@mshustov mshustov requested a review from slvrtrn March 12, 2025 15:03
@mshustov
Copy link
Member

mshustov commented Mar 12, 2025

discussed offline: we need to test how the existing dashboards react to the data type mapping changes

@slabko slabko marked this pull request as draft March 12, 2025 21:32
@slabko
Copy link
Contributor Author

slabko commented Mar 12, 2025

Unfortunately, after testing, we noticed that this change breaks the MS Power BI Direct Query aggregation functions, such as sum. Power BI relies on conversion to Decimal to avoid integer overflow. However, with the new implementation of the SQLGetTypeInfo function, Power BI selects Float32 instead, which prevents overflow but results in very imprecise calculations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Int64 columns are loaded as binary columns in PowerBI
2 participants