Skip to content

data validation for serviceNow #92

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from

Conversation

AnkitCLI
Copy link
Contributor

@AnkitCLI AnkitCLI commented Jun 1, 2025

This PR contains changes related to fix the data validation logic for service now and bigquery.

@AnkitCLI AnkitCLI force-pushed the dataValidationServiceNow branch 2 times, most recently from 35e2570 to ed5f4a3 Compare June 1, 2025 08:05
@AnkitCLI AnkitCLI added the build label Jun 1, 2025
@AnkitCLI AnkitCLI force-pushed the dataValidationServiceNow branch from ed5f4a3 to e37c6e7 Compare June 1, 2025 08:48
String columnType = field.getType().toString();

switch (columnType) {

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: remove extra line

boolean targetAsBoolean = bigQueryRow.get(columnName).getAsBoolean();
Assert.assertEquals("Different values found for column : %s", sourceAsBoolean, targetAsBoolean);
break;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: remove extra line

int targetAsInteger = bigQueryRow.get(columnName).getAsInt();
Assert.assertEquals("Different values found for column : %s", sourceAsInteger, targetAsInteger);
break;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: remove extra line

LocalDateTime targetDateTime = LocalDateTime.parse(bigQueryRow.get(columnName).getAsString());
Assert.assertEquals("Different values found for column : %s", sourceDateTime, targetDateTime);
break;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: remove extra line

Assert.assertEquals(String.format("Different values found for column: %s", columnName), 0,
Double.compare(sourceVal, targetVal));
break;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: remove extra line

@AnkitCLI AnkitCLI force-pushed the dataValidationServiceNow branch from e37c6e7 to b3e4645 Compare June 2, 2025 07:13
@AnkitCLI AnkitCLI marked this pull request as ready for review June 2, 2025 09:13
Copy link
Member

@itsankit-google itsankit-google left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not see any AfterHooks configured to delete the BQ tables?

Please add them in all the tests.

@AnkitCLI
Copy link
Contributor Author

AnkitCLI commented Jun 2, 2025

I do not see any AfterHooks configured to delete the BQ tables?

Please add them in all the tests.

added for both sink and source

@itsankit-google
Copy link
Member

e2e-tests build is failing & branch is out of date with base branch :
image

@AnkitCLI AnkitCLI force-pushed the dataValidationServiceNow branch from f723415 to 64cd3b1 Compare June 3, 2025 04:58
@AnkitCLI AnkitCLI force-pushed the dataValidationServiceNow branch from 408705a to 4b5336e Compare June 3, 2025 05:54
@AnkitCLI AnkitCLI requested a review from itsankit-google June 3, 2025 08:35
throws IOException, ServiceNowAPIException {
BeforeActions.scenario.write("Create new record in Date time table");
ServiceNowTableAPIClientImpl tableAPIClient = new ServiceNowTableAPIClientImpl(config.getConnection());
String recordDetails = "{'u_date':'2025-05-28 00:00:00','u_datetime':'2025-05-28 15:07:56'}";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is just testing one format, plan was to test all formats added in https://github.com/data-integrations/servicenow-plugins/pull/88/files

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants