Migrate to Dividend Chain of Events and Improve Regex Parsing#147
Open
letssayx wants to merge 9 commits into
Open
Conversation
…arsing * Improved regex in `nse_lib.py` to aggressively strip "face value", handle `\u20b9`, `Re`, and fractional amounts to extract exact dividend values. * Removed problematic logic from `nse_importer.py` that injected fake/synthesized duplicate `CorporateAction` rows based on Board Meetings. * Refactored `special_sit_routes.py` to dynamically construct a chronological "Chain of Events" from pristine `BoardMeeting` and `CorporateAction` models. * Fixed loop bugs where symbols with only a Board Meeting (but no historical CA) were skipped. * Maintained accurate exact-time timestamping for extraordinary yield calculations. Co-authored-by: letssayx <[email protected]>
Contributor
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
…regex parsing * Improved regex in `nse_lib.py` to aggressively strip "face value", handle `\u20b9`, `Re`, and fractional amounts to correctly extract exact dividend values without false positives. * Removed problematic logic from `nse_importer.py` that injected fake/synthesized duplicate `CorporateAction` rows based on Board Meetings. * Refactored `special_sit_routes.py` to dynamically construct a chronological "Chain of Events" from pristine `BoardMeeting` and `CorporateAction` models. * Fixed the `workbench.html` Dividends Data Bank frontend to properly merge distinct distinct lifecycle events based on strict `parsed_dividend_amount` matching, preventing distinct interim dividends (like BHEL) from being accidentally destroyed. * Removed legacy flawed frontend Javascript regex amount parser. Co-authored-by: letssayx <[email protected]>
- Update nse_lib.py regex to strip "face value", "fv", and other false positives, and correctly parse "Re", "Rs", and fractional dividend values. - Remove synthetic CorporateAction row injection from nse_importer.py. - Implement a true "Chain of Events" compiler in special_sit_routes.py using distinct BoardMeeting and CorporateAction lookups. - Fix UI deduplication logic in workbench.html to merge only a synthetic/intimation event with an official CorporateAction, avoiding the destruction of sequential quarterly dividends. Co-authored-by: letssayx <[email protected]>
- Improved regex in `nse_lib.py` to aggressively strip Face Value (e.g. "face value of Rs 10/- each") before extracting dividend amounts. This fixes issues like BHEL missing 'Re' amounts and Reliance capturing 16 instead of 6. - Fixed the chronological 'Chain of Events' in `workbench.html` by ensuring that identically-priced dividends are not incorrectly merged into one another. Specifically, new upcoming Board Meetings will no longer be destructively merged into historical Corporate Actions that already have an ex-date in the past. Co-authored-by: letssayx <[email protected]>
…idends - Updated `nse_lib.py` regex to robustly handle fractional amounts (Re), Rupee symbols (₹), and aggressively strip face value noise (each, /-) to fix BHEL and Reliance extraction. - Rewrote `special_sit_routes.py` to query both `CorporateAction` and `BoardMeeting`, linking intimations to outcomes chronologically without destructive deduplication based on matching amounts, fixing missing upcoming dividends like HDFCAMC. - Refactored `workbench.html` to eliminate destructive sliding-window deduplication, matching synthetic intimations to official actions 1:1, rendering a single-row timeline per cycle. Co-authored-by: letssayx <[email protected]>
* Update `nse_lib.py` regex to correctly strip 'face value' and 'paid-up capital' * Update `nse_lib.py` regex to correctly handle and sum fractional 'Re' amounts * Refactor `workbench.html` to chronologically group 'chain of events' (intimations and corporate actions) using a flat list * Ensure `workbench.html` correctly sorts synthesized records using safe date parsing Co-authored-by: letssayx <[email protected]>
* Update `nse_lib.py` regex to correctly strip 'face value' and 'paid-up capital' * Update `nse_lib.py` regex to correctly handle and sum fractional 'Re' amounts * Refactor `workbench.html` to chronologically group 'chain of events' (intimations and corporate actions) using a flat list * Ensure `workbench.html` correctly sorts synthesized records using safe date parsing, including falling back to `_matchedMeeting.meeting_date` for upcoming board meetings. Co-authored-by: letssayx <[email protected]>
- Update regex in `nse_lib.py` and `field_mapper.py` to correctly parse edge-case fractions (e.g. `Re0.25`, `\u20b9`).
- Prevent historical synthetic corporate action destruction by removing the overly aggressive `ca_model.purpose.like('Dividend (%')` deletion filter from `nse_importer.py`.
- Ensure expected dividends properly overwrite forecasts with official 'Announced' amounts (without HTML tag bleed) in `special_sit_routes.py` and `specialSitTool.js`.
Co-authored-by: letssayx <[email protected]>
- `nse_importer.py`: Fixed a critical deduplication flaw where all official `CorporateAction` historical rows with `purpose == 'Dividend'` were being deleted inadvertently when replacing synthetic placeholders. - `workbench.html`: Rewrote the frontend timeline logic to link intimation Board Meetings with official Corporate Actions matching precisely by amount and falling within a bi-directional 180-day window (`[-90, 180]` days), preventing erroneous destructive merges of distinct events (e.g., POWERGRID). - `field_mapper.py` & `nse_lib.py`: Vastly improved extraction regex by dynamically stripping terms like "face value", "fv", and variations *before* searching for dividend numbers. Added strict parsing for fractional `Re` / `Re.` identifiers (e.g., BHEL fractional cases). - `special_sit_routes.py`: Re-verified exact market hour timing logic (`>= 15:30` vs `< 15:30`) to precisely map pre/post market `broadcast_date` events to previous day vs same day Eq. close prices for >2% Extra-ordinary calculations. Co-authored-by: letssayx <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes critical dividend tracking bugs where the old deduplication logic hid rows and the regex parsing falsely captured face values instead of exact dividend amounts. Transitions the architecture to a "Chain of Events" timeline structure.
PR created automatically by Jules for task 13805983119605612887 started by @letssayx