Skip to content

Conversation

@VivekY1098
Copy link
Owner

@VivekY1098 VivekY1098 commented Apr 9, 2025

User description

Fixes #<issue_number_goes_here>

It's a good idea to open an issue first for discussion.

  • Tests pass
  • Appropriate changes to README are included in PR

CodeAnt-AI Description

  • Enhanced integration tests to support NUMERIC type and expanded test coverage for core data types.
  • Added conversion functions for NUMERIC type in both MySQL and PostgreSQL data handling.
  • Updated type mapping logic to map NUMERIC to Spanner's NUMERIC, with comments on precision differences.
  • Adjusted test cases and data to reflect new NUMERIC type handling across MySQL and PostgreSQL.
  • Updated GitHub workflow to use emulator version 1.1.1 and enabled NUMERIC type support.
  • Revised documentation to reflect changes in NUMERIC type handling and clarified precision details.

This PR enhances the handling of the NUMERIC data type by mapping it directly to Spanner's NUMERIC type, improving precision and compatibility. It updates tests, documentation, and configuration to support these changes, ensuring comprehensive coverage and clarity.


Changes walkthrough

Relevant files
Tests
8 files
integration_test.go
Enhance integration tests to support NUMERIC type                           

integration_test.go

  • Added support for NUMERIC type in integration tests.
  • Expanded checkCoreTypes function to include more data types.
  • Updated test assertions for new data types.
  • +27/-10 
    infoschema_test.go
    Update MySQL info schema tests for NUMERIC type                               

    mysql/infoschema_test.go

  • Updated test cases to reflect NUMERIC type mapping.
  • Removed issues related to decimal mapping.
  • +7/-8     
    mysqldump_test.go
    Adjust MySQL dump tests for NUMERIC type support                             

    mysql/mysqldump_test.go

  • Changed expected type for decimal to NUMERIC in tests.
  • Adjusted test data to match new NUMERIC type handling.
  • +2/-2     
    toddl_test.go
    Verify NUMERIC type mapping in MySQL to Spanner conversion         

    mysql/toddl_test.go

  • Updated tests to verify NUMERIC type mapping.
  • Removed previous issues related to float64 mapping.
  • +2/-2     
    infoschema_test.go
    Update PostgreSQL info schema tests for NUMERIC type                     

    postgres/infoschema_test.go

  • Updated NUMERIC type mapping in test cases.
  • Removed unnecessary schema issues.
  • +1/-2     
    pgdump_test.go
    Adjust PostgreSQL dump tests for NUMERIC type support                   

    postgres/pgdump_test.go

  • Changed expected type for numeric to NUMERIC in tests.
  • Adjusted test data to reflect NUMERIC type handling.
  • +5/-5     
    report_test.go
    Update report tests for NUMERIC type support                                     

    postgres/report_test.go

  • Removed outdated warning about NUMERIC type mapping.
  • Updated report test to reflect new NUMERIC support.
  • +2/-5     
    toddl_test.go
    Verify NUMERIC type mapping in PostgreSQL to Spanner conversion

    postgres/toddl_test.go

  • Updated tests to verify NUMERIC type mapping.
  • Removed previous issues related to float64 mapping.
  • +1/-2     
    Enhancement
    4 files
    data.go
    Add conversion support for NUMERIC type in MySQL                             

    mysql/data.go

  • Added convNumeric function to handle NUMERIC type conversion.
  • Updated convScalar to include NUMERIC type.
  • +16/-0   
    toddl.go
    Map MySQL NUMERIC type to Spanner NUMERIC                                           

    mysql/toddl.go

  • Modified type mapping logic to map NUMERIC to Spanner's NUMERIC.
  • Added comments explaining NUMERIC type precision differences.
  • +9/-8     
    data.go
    Add NUMERIC type conversion support for PostgreSQL                         

    postgres/data.go

  • Added convNumeric function for NUMERIC type conversion.
  • Updated convScalar to handle NUMERIC type.
  • +16/-0   
    toddl.go
    Map PostgreSQL NUMERIC type to Spanner NUMERIC                                 

    postgres/toddl.go

  • Modified type mapping logic to map NUMERIC to Spanner's NUMERIC.
  • Added comments on NUMERIC type precision differences.
  • +17/-8   
    Configuration changes
    1 files
    integration-tests-against-emulator.yaml
    Update GitHub workflow for NUMERIC type support in emulator       

    .github/workflows/integration-tests-against-emulator.yaml

  • Updated emulator image version to 1.1.1 for NUMERIC support.
  • Enabled NUMERIC type in emulator configuration.
  • +5/-1     
    Documentation
    2 files
    README.md
    Update MySQL README for NUMERIC type support                                     

    mysql/README.md

  • Updated documentation to reflect NUMERIC type mapping.
  • Clarified precision details for NUMERIC type.
  • +15/-13 
    README.md
    Update PostgreSQL README for NUMERIC type support                           

    postgres/README.md

  • Updated documentation to reflect NUMERIC type mapping.
  • Clarified precision details for NUMERIC type.
  • +11/-12 
    💡 Usage Guide

    Checking Your Pull Request

    Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

    Talking to CodeAnt AI

    Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

    @codeant-ai ask: Your question here
    

    This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

    Retrigger review

    Ask CodeAnt AI to review the PR again, by typing:

    @codeant-ai: review
    

    Check Your Repository Health

    To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.


    EntelligenceAI PR Summary

    Purpose

    Enhance integration tests and type mappings for improved compatibility with Spanner's NUMERIC type.

    Changes

    Enhancement

    • Updated the emulator to version 1.1.1 and added support for NUMERIC type in integration tests, ensuring accurate data type conversions across MySQL and PostgreSQL.

    Documentation

    • Revised README files for MySQL and PostgreSQL to clarify the mapping of DECIMAL and NUMERIC types to Spanner's NUMERIC type.

    Test

    • Renamed and updated test functions to validate NUMERIC and other data types, ensuring robustness in type handling.

    Impact

    These changes enhance the reliability and accuracy of data type handling in applications using Spanner, leading to a better user experience and reduced potential for data-related issues.

    @codeant-ai codeant-ai bot added the size:L This PR changes 100-499 lines, ignoring generated files label Apr 9, 2025
    @entelligence-ai-pr-reviews
    Copy link

    Review Summary

    Skipped posting 1 drafted comments based on your review threshold. Feel free to update them here.

    Draft Comments
    .github/workflows/integration-tests-against-emulator.yaml:26-26
    Typo in the TODO comment: `emuator` should be `emulator`. This doesn't affect functionality but should be fixed for clarity.
    

    Scores:

    • Production Impact: 1
    • Fix Specificity: 5
    • Urgency Impact: 1
    • Total Score: 7

    Reason for filtering: The comment meets the minimum threshold for inclusion

    Analysis: This is a simple typo fix in a comment that has no impact on production functionality. The fix is extremely clear and specific, providing the exact correction needed. However, since it's just a typo in a comment, it has very low urgency. With a total score of 7, this comment falls below the threshold of 10 required for inclusion under the current filtering rules.

    @entelligence-ai-pr-reviews
    Copy link

    Walkthrough

    This PR enhances integration tests and type mappings for MySQL and PostgreSQL to support the NUMERIC type in Spanner. It upgrades the emulator to version 1.1.1, adds numeric support flags, and modifies tests to ensure accurate data type conversions. README files are updated to clarify type mapping differences, improving compatibility and accuracy.

    Changes

    File(s) Summary
    .github/workflows/integration-tests-against-emulator.yaml Updated emulator image to version 1.1.1 and added argument for numeric type support.
    integration_test.go Renamed function to checkCoreTypes and updated to check for NUMERIC and other data types.
    mysql/README.md Updated DECIMAL and NUMERIC type mapping to Spanner's NUMERIC type.
    postgres/README.md Updated NUMERIC type mapping to reflect Spanner's NUMERIC type instead of FLOAT64.
    mysql/data.go / postgres/data.go Added convNumeric function to convert numeric values to Spanner's numeric format.
    Entelligence.ai can learn from your feedback. Simply add 👍 / 👎 emojis to teach it your preferences. More shortcuts below

    Emoji Descriptions:

    • ⚠️ Potential Issue - May require further investigation.
    • 🔒 Security Vulnerability - Fix to ensure system safety.
    • 💻 Code Improvement - Suggestions to enhance code quality.
    • 🔨 Refactor Suggestion - Recommendations for restructuring code.
    • ℹ️ Others - General comments and information.

    Interact with the Bot:

    • Send a message or request using the format:
      @bot + *your message*
    Example: @bot Can you suggest improvements for this code?
    
    • Help the Bot learn by providing feedback on its responses.
      @bot + *feedback*
    Example: @bot Do not comment on `save_auth` function !
    

    @entelligence-ai-pr-reviews
    Copy link

    LGTM 👍

    var stringVal string
    iter := client.Single().Read(ctx, "test2", spanner.Key{1}, []string{"a", "b", "c", "d", "e", "f", "g"})
    defer iter.Stop()
    for {
    Copy link

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    Suggestion: Add a check for the iterator.Done condition to gracefully exit the loop when no more rows are available. [possible bug]

    @codeant-ai
    Copy link

    codeant-ai bot commented Apr 9, 2025

    Pull Request Feedback 🔍

    🔒 No security issues identified
    ⚡ Recommended areas for review

    Iterator Termination
    The newly added function checkCoreTypes does not appear to handle the end-of-iteration scenario (e.g. checking for iterator.Done). This could lead to an infinite loop if no more rows are returned. Please consider adding proper termination logic.

    Numeric Conversion
    The convNumeric function converts numeric strings using a big.Rat and then formats them via spanner.NumericString. It would be beneficial to double-check for edge cases where precision or rounding might be an issue, ensuring that the output conforms exactly to Spanner's NUMERIC format.

    Type Mapping Change
    The mapping for MySQL's "numeric" and "decimal" types has been updated to ddl.Numeric. Please ensure that this change is thoroughly tested for backward compatibility and that any precision differences are properly communicated through schema issues.

    Type Mapping Update
    PostgreSQL's NUMERIC type is now mapped to ddl.Numeric with detailed commentary about precision differences. It would be helpful to verify that this update consistently reflects the capabilities of Spanner and that any warnings about precision mismatches are correctly generated.

    Emulator Update
    The Spanner emulator image has been updated to version 1.1.1 and now includes an extra argument (-enable_numeric_type) for enabling numeric support. Please confirm that this flag is still required and that future emulator upgrades will not break integration tests.

    Test Data Update
    Test output has been updated to reflect schema changes, including updated data types and numeric formatting. Verify that these changes are intentional and consistent with the new conversion logic, and that they do not inadvertently affect other tests.

    Copy link

    @greptile-apps greptile-apps bot left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    PR Summary

    This pull request reverts recent changes and restores the original numeric conversion behavior so that both MySQL and PostgreSQL DECIMAL/NUMERIC types map to Spanner's NUMERIC type, ensuring precision is preserved.

    • mysql/README.md: Updated documentation now accurately reflects that DECIMAL/NUMERIC convert to NUMERIC (29/9) instead of FLOAT64.
    • .github/workflows/integration-tests-against-emulator.yaml: Emulator updated to 1.1.1 with the "-enable_numeric_type" flag to support numeric types.
    • mysql/data.go: Reintroduced convNumeric handling via big.Rat with spanner.NumericString for proper conversion.
    • MySQL and PostgreSQL tests (e.g. infoschema_test.go, mysqldump_test.go, toddl_test.go, pgdump_test.go): Updated expectations to validate NUMERIC mapping and precision preservation.
    • Documentation updates for PostgreSQL (postgres/README.md): Revised numeric type mapping explanations to match the restored conversion logic.

    16 file(s) reviewed, 1 comment(s)
    Edit PR Review Bot Settings | Greptile

    Comment on lines +24 to +27
    # needed because numeric support isn't on by default in emulator:1.1.1
    # note: future versions will default enable this and deprecate the flag
    # TODO: upgrade emuator version, confirm enable_numeric_type is no longer needed and delete it
    args: ["-enable_numeric_type"]
    Copy link

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    syntax: Fix typo in the TODO comment on line 26: change 'emuator' to 'emulator'.

    Suggested change
    # needed because numeric support isn't on by default in emulator:1.1.1
    # note: future versions will default enable this and deprecate the flag
    # TODO: upgrade emuator version, confirm enable_numeric_type is no longer needed and delete it
    args: ["-enable_numeric_type"]
    # needed because numeric support isn't on by default in emulator:1.1.1
    # note: future versions will default enable this and deprecate the flag
    # TODO: upgrade emulator version, confirm enable_numeric_type is no longer needed and delete it
    args: ["-enable_numeric_type"]

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Labels

    size:L This PR changes 100-499 lines, ignoring generated files

    Projects

    None yet

    Development

    Successfully merging this pull request may close these issues.

    3 participants