feat(ingestion/superset): add timeout values to config to prevent hanging queries from blocking ingestion

[PeteMango]

Currently, if a request to superset hangs indefinitely, this will stall the entire ingestion pipeline and there are no errors that is currently being surfaced (not even with --debug flag). This aims to prevent that by adding a timeout value to the config file.

Checklist

• The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
• Links to related issues (if applicable)
• Tests for the changes have been added/updated (if applicable)
• Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
• For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

[codecov]

Codecov Report

Attention: Patch coverage is 66.66667% with 1 line in your changes missing coverage. Please review.

✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
...ingestion/src/datahub/ingestion/source/superset.py	66.66%	1 Missing ⚠️

❌ Your patch check has failed because the patch coverage (66.66%) is below the target coverage (75.00%). You can increase the patch coverage or adjust the target coverage.

📢 Thoughts on this report? Let us know!

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[gabe-lyons]

Are these really all the times we call the superset API? That seems like a surprisingly low amount.

[PeteMango]

Are these really all the times we call the superset API? That seems like a surprisingly low amount.

Yes, all of the constructions for charts, dashboards and datasets call the paginate_entity_api_results function, which means we only had to change it in one place. Only datasets require specific dataset info which is why we also updated get_dataset_info