Skip to content

Add a flag to check the new metadata api is called for DTS users only #100

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 7 commits into
base: develop
Choose a base branch
from

Conversation

prince-cs
Copy link
Contributor

@prince-cs prince-cs commented Jun 26, 2025

As the schema changed logic is changed based on the value type being used. Older pipelines are not compatible with this change.

Added a flag, which ensures that for DTS users, schema will be fetched via column metadata api and for CDF users it will be fetched via previous schema api.

this.setUrl(String.format(METADATA_API_URL_TEMPLATE, instanceBaseUrl, tableName));
} else if (isSchemaRequired) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the difference between metadata api vs schema api?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

metadata api uses column info which in turn is used to create a schema based on value type and schema api provides with the schema based on actual type only.

Schema api was being used earlier

@vikasrathee-cs
Copy link
Contributor

Reverse the naming convention, let the old schema response and field name class be same and new class can be MetaDataAPISchema and field.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants