Helm deployment fixes #735
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ion, in order to attempt to get production traefik deployment to function as it did in another branch. Updated docker compose files to separate keycloak into a profile and remove docs service from production. Updated manage script to utilize new compose files for deployment and testing.
…update app css and include other minor fixes.
…oid using the conditional statements that were being used which seemed to not be working for traefik deployment. Updated the deployment script to make sure local and production custm overlays are applied last so their variables aren't overriden.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## v1.3.1-release #735 +/- ##
==================================================
Coverage ? 100.00%
==================================================
Files ? 34
Lines ? 1086
Branches ? 0
==================================================
Hits ? 1086
Misses ? 0
Partials ? 0
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
…deployment container types. Updated the config helper script to generate production overlay
bstrdsmkr
approved these changes
Feb 14, 2025
…l background in dark theme.
… updateProjects.sh script and updated the README's regarding updating project data. Added the update functions to the manage_metagrid.sh scripts instead. Still need to test in production, but should work.
downiec
added a commit
that referenced
this pull request
Apr 9, 2025
* Updated version number to 1.3.0 and added changelog file for 1.3.0, to be updated as needed. * Staticbuild v1.3.0 update (#690) * Harden images and reorg * fix timezone with package (#680) * Joyride tutorials update (#679) * Merged in latest v1.2.1 * Updates to the joyride tutorials and some minor bugfixes. * Revert Markdown update to fix current jest testing error * Updated React Markdown library and fixed the jest test errors. * Some modifications to remove the build and linting errors that came up after the merge with latest branch. * Load frontend config before rendering App Stop tests from leaking calls to globus.org * Some fixes to get the react front-end to build and get the postgres to start up. There are still errors with django session store. * Fixed some linting errors so frontend should build correctly. * Updated the local config file used for local testing. The new file .local_config, contains the configuration used when running Metagrid locally. Updated the manage_metagrid script to run Metagrid from the single docker-compose file. Updated some settings to use llnl instead of ornl for local development and testing. Removed some unused environment variable files after after testing that they aren't used. Left the default variables in place because Metagrid fails to build when several of the defaults aren't set, still working to troubeshoot why. Globus transfer steps in the front-end are working, however transfers seem to fail and need to be resolved. * Switch from CRA to Vite and clean up tests (#692) * Switch from CRA to Vite and clean up tests * Fixed issue with loading screen continuing even after cancel button click. Updated local test config. Globus transfers still not working. --------- Co-authored-by: Zach Price <[email protected]> Co-authored-by: downiec <[email protected]> * Staticbuild v1.3 fixes (#697) * Fix proxy and redirects for globus auth on docker compose * Various lint and test fixes * Update yarn.lock * Some changes made to try and get globus transfers working. * Update docs to match new config (#707) * Updated the local config file used for local testing. The new file .local_config, contains the configuration used when running Metagrid locally. Updated the manage_metagrid script to run Metagrid from the single docker-compose file. Updated some settings to use llnl instead of ornl for local development and testing. Removed some unused environment variable files after after testing that they aren't used. Left the default variables in place because Metagrid fails to build when several of the defaults aren't set, still working to troubeshoot why. Globus transfer steps in the front-end are working, however transfers seem to fail and need to be resolved. * Switch from CRA to Vite and clean up tests (#692) * Switch from CRA to Vite and clean up tests * Fixed issue with loading screen continuing even after cancel button click. Updated local test config. Globus transfers still not working. --------- Co-authored-by: Zach Price <[email protected]> Co-authored-by: downiec <[email protected]> * Fix proxy and redirects for globus auth on docker compose * Various lint and test fixes * Update yarn.lock * Some changes made to try and get globus transfers working. * Update docs to match new config Auto generate configuration documentation with pre-commit hooks * Revert yarn.lock, fix more tests * Includes some updates and minor fixes so that everything builds and runs correctly. Removed the GLOBUS_REDIRECT setting and did some cleanup of the manage_metagrid.sh functions. * Updated the test files to no longer use the act() functions, as they are deprecated. Cleaned up and fixed some failing tests. * Fix esgsearch json tests * Minor test fixes to see if tests will pass on github (they pass locally). --------- Co-authored-by: downiec <[email protected]> Co-authored-by: Zach Price <[email protected]> --------- Co-authored-by: Zach Price <[email protected]> Co-authored-by: bstrdsmkr <[email protected]> --------- Co-authored-by: Zach Price <[email protected]> Co-authored-by: Sasha Ames <[email protected]> Co-authored-by: bstrdsmkr <[email protected]> * Fixed some linting errors related to unused imports * Some minor modifications to the pre-commit files, including removing repeated settings-doc-markdown config, to see if that resolves the pre-commit failure. * Fix broken link in Globus transfer panel (#719) * Fix broken link in Globus transfer panel * Globus link display update (#3) * Added a warning case for when a transfer request returns no failures or successes in the backend response. No successes indicates no transfer occurred although there wasn't any specified errors provided. Added new test for the added warning case. * Fix pre-commit prerequisites --------- Co-authored-by: Zach Price <[email protected]> Co-authored-by: Carlos Downie <[email protected]> * V1.3.0 additional fixes (#721) * Update version number, add version changelog (to be updated with the changes that will be included in this update) * Updated documentation references to config variables (#706) * Updated documentation, specifically references to the configuration parameters which have changed over time, to use the latest variable names currently used in configuration. Updated requirements to use the latest readthedocs them of Sphinx * Update version number, add version changelog (to be updated with the changes that will be included in this update) * fixed the pagination so that it will reset to page one when doing a new search (#705) * update for obs4MIPs (#711) Co-authored-by: Sasha Ames <[email protected]> * Updated the settings file to autoformat on save to hopefully reduce linting errors. Fixed linting errors for the initial_projects_data update. Fixed the status tooltip to correctly display the node's status. Updated test fixtures to accurately reflect the node status response (the node names included the https:// in front and thats why status tooltip wasn't finding them. (#714) * Added alert for the case that a collection search fails in the manage collections popup (#710) * Added alert for the case that a collection search fails in the collection management popup. Added a test to bring the coverage up to 100% for the search table component. Removed some unused code, the get endpoint function is not used. * Minor linting error fixes. * Fixed some backend tests based on the removed/unused functions * Small fix to test. * Minor update to update summary. * Made it so the globus ready column does not appear if no globus ready nodes are configured (as it was before). Changed ActiveFacets type to be more strict in order to avoid the issue of calling join on a string which was causing errors. Fixed and updated some tests related to the url to string and string to url functions in the utils file. * Fixed failing test. Updated the manage_metagrid.sh helper scripts to include commands for pre-commits, backend and frontend tests. Updated setup.cfg to allow backend tests to run correctly. Removed unneeded commands in backend README.md. Removed the autoformat setting on save. --------- Co-authored-by: Sasha Ames <[email protected]> * Dark mode (#718) * Add dark mode * Persist theme choice * Add tests for Dark mode feature * Merged in the latest changes from v1.3.0-release. Updated the styling to fix issues with the dark-mode rendering of navbar and certain colors. Added custom colors for dark and light version of status icons. Modified how colors are applied to tooltips and node status pages. I'd like to update/improve the way custom colors and styles are applied to make future updates smoother. (#4) --------- Co-authored-by: Zach Price <[email protected]> Co-authored-by: downiec <[email protected]> * Custom footer (#727) * Add dark mode * Persist theme choice * Add tests for Dark mode feature * Merged in the latest changes from v1.3.0-release. Updated the styling to fix issues with the dark-mode rendering of navbar and certain colors. Added custom colors for dark and light version of status icons. Modified how colors are applied to tooltips and node status pages. I'd like to update/improve the way custom colors and styles are applied to make future updates smoother. * Added a new configuration option for setting the footer of the metagrid frontend so that links and text can be displayed as desired. Updated the manage_metagrid script to include more developer actions, including run migrations, and installing packages for local development. Modified styling to better fit the current footer. Will need to add option for adjusting the footer height based on the height of the footer text. Updated tests and documentation for the custom footer options. * Final updates for the v1.3.0 release. Includes updates to the joyride tutorial to add the theme switch. Updated the manage_metagrid script and created a new configHelper script that sets the proper initial configuration settings for first time deployment or developers. Added tests for the theme buttons in the app. --------- Co-authored-by: Zach Price <[email protected]> * V1.3.1 release * Traefik for prod (#734) Co-authored-by: Zach Price <[email protected]> * Helm deployment fixes (#735) * Include helm chart and healthchecks * Modified the traefik.yml config to separate between local and production, in order to attempt to get production traefik deployment to function as it did in another branch. Updated docker compose files to separate keycloak into a profile and remove docs service from production. Updated manage script to utilize new compose files for deployment and testing. * Updated some docs and the manage scripts * Merged in some minor changes from traefik-deployment-fixes branch to update app css and include other minor fixes. * Delete docker-compose.prod-overlay.yml * Pre-comfig fixes, update to scripts. * Some updates to docker deployment to test if they will work in production * Separated trafik.yml into one for production and one for local, to avoid using the conditional statements that were being used which seemed to not be working for traefik deployment. Updated the deployment script to make sure local and production custm overlays are applied last so their variables aren't overriden. * Removed unnecessary prod_traefik file. * Updated prod overlay to name the containers with prod to distinguish deployment container types. Updated the config helper script to generate production overlay * Updated changelog. Added dark theme styling to the React Joyride tutorials. * Implemented some suggestions from PR comments. Brightened the tutorial background in dark theme. * Minor fix to compose file. * Reverted container names to leave them be. * Updated obs4mips url and modified the display button. Remove obsolete updateProjects.sh script and updated the README's regarding updating project data. Added the update functions to the manage_metagrid.sh scripts instead. Still need to test in production, but should work. * Fix minor lint error. Added build step to project migration function. --------- Co-authored-by: Zach Price <[email protected]> * Minor fix that was missed for changing duplicate Originals and Replicas label, to Replicas Only. * Initial commit for version 1.3.2. Includes a refactoring of how messages are updated and how the changelog is updated. The data for messages and changes is now located in a messageData.json file, removing the need to modify the messageDisplayData.ts file in multiple locations every time a new version and changelog entry is created. * Fixed minor issues with helm and wget downloads. * Updated management script to properly quit if tests fail. Updated do_request to add option for obtaining object using request.body, this fixes the wget download which expects data in a different format than other requests. Updated the changelog. * Initial v1.3.3 release version update, includes changes to News display, to make version history collapsed as it was getting long. * Added a dev action that will automatically generate necessary changes to increment/create a new version. It will create a new changelog file, update the package.json and update the messageData.json file to add the new version. * Initial v1.3.4 commit includes fix for path updating to proper value. * Fixed linting error, tests regard changes to path and added some extra save steps to see if that will resolve globus transfer token auth loop issue in Firefox. * Fixed linting error, tests regard changes to path and added some extra save steps to see if that will resolve globus transfer token auth loop issue in Firefox. * Updated the way paths are set so that an auth step is included if no globus permissions were requested. This seems to allow the path set redirect to request the appropriate permissions for allowing transfers to occur. * Updated change log to include mention of how path set functionality will behave. * Add hash annotation to Deployments to ensure restart when config changes, fix typos in ingress template (#743) Co-authored-by: Zach Price <[email protected]> * Removed unnecessary settings from the config helper. * Updated app version number for the Helm chart, added step in updateVersion script so helm chart version is automatically updated when the version number is updated. * Fixes helm & containers (#744) * Fixes hash for automatic deployment rolls * Fixes container trigger and pushing images * Fixes workflow tags * Fixes default filename for configHelper.sh * Fixes frontend makefile * Adds minimal frontend-config.js * Revert "Adds minimal frontend-config.js" This reverts commit 8856c9d. * Adds helm chart testing job * Fixes working directory * Updates workflow and adds test values * Fixes building dependency * Adds readme and helmfile to test locally * Fixes NOTES rendering * Adds defaults to support no-config install * Removes test.yaml since defaults were fixed * Fixes spelling error * Fixes workflow * Updates readme * Feature/v1.3.4 minor updates (#747) * Some minor updates to config helper and manage_metagrid scripts. * Added option to cancel endpoint path selection from globus side and it will return user to metagrid properly. * Updated to the manage script for updating the version values in helm files. Fixes some linting errors. * Added a tooltip to the download button to hint to user why it's disabled. Updated download button behavior ro make sure user can't start a Globus download without selecting a collection or start a wget download without selecting a cart item. * V1.3.4 patch (#749) * Created a DataBundlePersister class which will bundle key value pairs into a single object, stringify and then save to session storage as a single bundle. This will reduce requests to session storage by consolidating into a single request when values need to be persisted or loaded. * Minor updates to changelogs for consistent punctuation. * Some minor updates to persistent class * Updated the way session storage is used by the datasetdownload component to improve globus transfer reliability. Includes extensive refactoring of tests and test components to pass using the new DataBundlePersister class. Updated the cart summary to display info on selected cart items not just on the entire cart totals. * Some minor changes to the documentation to update the frontend project structure. Moved the reset tokens button to a menu in the globus transfer button. Globus transfer and wget download are separated into their own buttons using download type to select which is displayed. Updated tests as needed and added new ones for the reset token feature. Updated react joyride tour to mention changes on the summary table. * Made some minor updates, refactors and added new tests to increase test coverage * Update version used in README.md for helm * Initial update to version numbers. * Improve cart items and selections synchronization (#752) * Updated the App.tsx to make sure the selected items list is properly updated when new items are added to the cart or when items are removed from the cart. Fixes the issue where items were removed from the cart but the selection summary had the old selections there, which caused an issue with cart selections. Made it so that items added to cart are automatically selected for download, to make downloads more intuitive. * Fixed some broken tests, updated the changelog. * Fix the issue with duplicate saved searches and deleting saved searches. * Update v1.3.5.md * Fixes the JSON buttons by updating the clickableRoute function (#757) * Fixes the JSON buttons by removing th clickableRoute function which is no longer needed and was causing the button to fail (do nothing). Removed some commented out code that has been there a while. Update version in helm README. * Updated the JSON output links so that the url updates the location origin to match the current origin. Otherwise some older saved search cards will point to an older origin and the JSON feature doesn't work. * Added some tests for the createSearchRouteURL function and fixed some fixtures so other tests will pass. * Updated changelog * Minor change to the docker-compose to hide internal ports to only work with localhost. * Fix Helm Chart (#759) * Fixes version format in Chart.yaml * Fixes updating version in chart readme * Adds new job to link helm chart * Updates the chart version and script * Fixes to trigger on all pull requests * Removes updating the helm chart version from update script * App recoil refactoring (#760) * Significant refactoring to reduce the complexity and dependency among various components upon the main App component. Updated tests and added new recoil wrapper helper class that can set recoil state for tests. * Refactored some components to utilize recoil. Fixed tests to use the new recoil method. Updated customRender so that it can apply initial recoil state. * Another round or refactoring to simplify more components by using recoil state. Updated and modified the recoilWrapper to make it a singleton and then have it set defaults at the start of all the tests. Hopefully updating the test suite after refactoring will be more straight forward however still need to fix some broken tests. * Some updates to the test suite to fix broken tests and fix how the recoil state is initialized for each test. * Updated docker compose to close off internal ports to use only localhost * Performed another round of refactoring and updated utils, test functions and components to improve readability and testability. Fixed some minor issue with custom-render function not properly setting the auth state. Added coverage to bring test coverage up over 94%. * Updated the FacetsForm to correctly handle globus ready status * Fix minor linting error for recoil tests. * Fixed issue with joyride tutorials, where the add/minus buttons weren't being selected. Updated tutorial to work for the case where user is logged in. * Increase the testing timeout. Tests pass locally but seemed to fail/timeout on github. * Updated the timeout value for globus tests only, reverting the overall timeout for other tests. * More changes to see why test fails in github but not locally * Test with additional troubleshooting info. * Increase timeout time again * Increase timeout time again for specific failing test. * Skipping test for now, it passes locally but consistently fails in github, will need to troubleshoot further. * Patch the backend docker file to make sure pip and setup tools are upgraded to latest in order to avoid current installation issues when run without. --------- Co-authored-by: Zach Price <[email protected]> Co-authored-by: Sasha Ames <[email protected]> Co-authored-by: bstrdsmkr <[email protected]> Co-authored-by: Sasha Ames <[email protected]> Co-authored-by: Jason Boutte <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Fixed issues with traefik production deployment. Includes some updates to the management script and updates to the tutorial styling for dark mode.
Fixes: #635
Type of change
How Has This Been Tested?
Checklist