Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/transmission #35

Merged
merged 7 commits into from
Mar 7, 2025
Merged

Feat/transmission #35

merged 7 commits into from
Mar 7, 2025

Conversation

MaryKilewe
Copy link
Contributor

@MaryKilewe MaryKilewe commented Mar 7, 2025

Summary by Sourcery

Implements data transmission functionality, including WebSocket communication for progress updates and data loading counts. It also enhances the site configuration management by adding the ability to edit site configurations.

New Features:

  • Adds real-time progress updates for data loading via WebSockets.
  • Introduces the ability to edit existing site configurations.
  • Displays the count of loaded data for a given repository.
  • Validates required mappings before testing, preventing errors due to missing data.
  • Adds a toggle to set a site configuration as active or inactive.
  • Displays mandatory base variables with an asterisk in the selector form.
  • Adds a tooltip to display the full name of the base variable and indicate if it is mandatory or not

Tests:

  • Adds validation to ensure all required mappings are present before testing mappings.

Copy link
Contributor

sourcery-ai bot commented Mar 7, 2025

Reviewer's Guide by Sourcery

This pull request introduces several new features and improvements, including real-time progress updates for data loading and transmission, enhanced error handling, improved user experience in the SelectorForm component, and the ability to edit site configurations. The changes include modifications to the DataExtraction, TestMappings, SelectorForm, SiteConfigsList, MainRoutes, and EditSiteConfigDetails components, as well as updates to the site configurations queries.

No diagrams generated as the changes look simple and do not need a visual representation.

File-Level Changes

Change Details Files
Implemented WebSocket communication for real-time progress updates during data loading and transmission.
  • Established WebSocket connections to track data loading progress.
  • Implemented WebSocket connections to track data transmission progress.
  • Updated UI to display real-time progress updates.
  • Added handling for WebSocket errors and connection closures.
src/pages/data-extraction/DataExtraction.js
Improved error handling and user feedback during data loading.
  • Enhanced error messages to provide more context.
  • Added checks for missing mappings before testing.
  • Improved alert messages to be more informative.
  • Fixed bug where alert messages were not being dismissed.
src/pages/data-extraction/DataExtraction.js
src/pages/mapper/test-mappings/TestMappings.js
Enhanced the SelectorForm component to handle required base variables and improve the user experience.
  • Added visual indicators for required base variables.
  • Implemented logic to prevent testing mappings with missing required variables.
  • Ensured base variables are initialized only once.
  • Added tooltips to display the full base variable name and indicate if it is mandatory.
src/pages/mapper/selector-form/SelectorForm.js
Added functionality to edit site configurations.
  • Created a new component, EditSiteConfigDetails, to handle editing site configurations.
  • Implemented API calls to fetch and update site configuration details.
  • Added a route to navigate to the edit site configuration page.
  • Implemented form validation and error handling for the edit form.
src/store/site_configurations/queries.js
src/pages/configs/Site/SiteConfigsList.js
src/routes/MainRoutes.js
src/pages/configs/Site/EditSiteConfigDetails.js

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!
  • Generate a plan of action for an issue: Comment @sourcery-ai plan on
    an issue to generate a plan of action for it.

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@MaryKilewe MaryKilewe merged commit 93e55e4 into master Mar 7, 2025
Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @MaryKilewe - I've reviewed your changes - here's some feedback:

Overall Comments:

  • Consider using environment variables for WebSocket URLs instead of hardcoding them.
  • The websocket connection logic is duplicated, consider refactoring into a shared function.
Here's what I looked at during the review
  • 🟡 General issues: 2 issues found
  • 🟢 Security: all looks good
  • 🟢 Testing: all looks good
  • 🟡 Complexity: 1 issue found
  • 🟢 Documentation: all looks good

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.


newProgressSocket.onclose = function () {
console.log("ws connection closed");
setSendingSpinner(false);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: Possible incorrect spinner state update in checkLoadedCount.

The function calls setSendingSpinner but the rest of the component uses setSpinner. Verify that setSendingSpinner exists or consider using setSpinner consistently.

@@ -96,6 +109,7 @@ const SelectorForm = () => {
formData.push({
"base_repository": baselookup,
"base_variable_mapped_to": baseVariable,
"is_required": baseVariable,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (bug_risk): Potential type issue for the 'is_required' property.

In other parts of the code, 'is_required' is set to a boolean (e.g., o.is_required). Passing baseVariable directly might be unintended; confirm that the correct boolean value is assigned.

@@ -25,10 +25,16 @@ const DataExtraction = ({baseRepo}) =>{
const [progress, setProgress] = useState(null);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (complexity): Consider creating a custom hook to encapsulate the WebSocket logic and reduce code duplication across the component.

Consider extracting your duplicated WebSocket handling into a custom hook. This reduces duplication and isolates event handling logic. For example, you could implement a hook like:

// useWebSocket.js
import { useEffect, useRef } from 'react';

export function useWebSocket(url, { onOpen, onMessage, onError, onClose, sendMsg } = {}) {
  const socketRef = useRef(null);

  useEffect(() => {
    const ws = new WebSocket(url);
    socketRef.current = ws;

    ws.onopen = (event) => {
      if (onOpen) onOpen(event);
      if (sendMsg) ws.send(sendMsg);
    };
    ws.onmessage = (event) => onMessage && onMessage(event);
    ws.onerror = (error) => onError && onError(error);
    ws.onclose = (event) => onClose && onClose(event);

    return () => {
      ws.close();
    };
  }, [url, onOpen, onMessage, onError, onClose, sendMsg]);

  return socketRef.current;
}

Then refactor your WebSocket-based functions (both for sending and counting data) to use this hook:

// Example for checkLoadedCount
function checkLoadedCount(baseRepo) {
  console.log("count loaded data...");
  setDataLoadedCount(0);

  useWebSocket(`ws://${WS_API}/api/dictionary_mapper/ws/load/progress/${baseRepo}`, {
    onOpen: (event) => {
      event.target.send(baseRepo);
    },
    onMessage: (event) => {
      const data = event.data;
      if (data.includes("Error")) {
        console.error(data);
        event.target.close();
      } else {
        setDataLoadedCount(Number(data.replace("%", "")));
      }
    },
    onError: (error) => console.error("connection failed:", error),
    onClose: () => {
      console.log("ws connection closed");
      setSendingSpinner(false);
      setAlertType("success");
      setLoadMessage("Loading data completed");
    }
  });
}

Repeat a similar refactoring for the WebSocket in sendData or any other component needing WebSocket functionality. This approach centralizes the connection logic and makes maintenance easier.

Comment on lines +174 to +211
function checkLoadedCount(baseRepo) {

console.log("count loaded data... ")

setDataLoadedCount(0); // Reset progress to 0
const newProgressSocket = new WebSocket(`ws://${WS_API}/api/dictionary_mapper/ws/load/progress/${baseRepo}`);
console.log("count newProgressSocket... ",newProgressSocket)

newProgressSocket.onopen = () => {
newProgressSocket.send(baseRepo);
};
// Set up the WebSocket connection
newProgressSocket.onmessage = function (event) {
console.log("ws connection established ")

const data = event.data;
if (data.includes("Error")) {
console.error(data);
newProgressSocket.close();
} else {
console.log("data count--->",data)
setDataLoadedCount(Number(data.replace("%", ""))); // Update progress
}
};

newProgressSocket.onerror = function (error) {
console.log("connection failed with error: ", error)
};

newProgressSocket.onclose = function () {
console.log("ws connection closed");
setSendingSpinner(false);
setAlertType("success");
setLoadMessage("Loading data completed");
};

setSocket(newProgressSocket);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (code-quality): Avoid function declarations, favouring function assignment expressions, inside blocks. (avoid-function-declarations-in-blocks)

ExplanationFunction declarations may be hoisted in Javascript, but the behaviour is inconsistent between browsers. Hoisting is generally confusing and should be avoided. Rather than using function declarations inside blocks, you should use function expressions, which create functions in-scope.

newProgressSocket.onmessage = function (event) {
console.log("ws connection established ")

const data = event.data;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (code-quality): Prefer object destructuring when accessing and using properties. (use-object-destructuring)

Suggested change
const data = event.data;
const {data} = event;


ExplanationObject destructuring can often remove an unnecessary temporary reference, as well as making your code more succinct.

From the Airbnb Javascript Style Guide

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant