Improve date extraction#432
Closed
rajeee wants to merge 334 commits into
Closed
Conversation
* Add new kwargs for prompt * Update name and jurisdiction prompts * Don't lean so heavily on zoning, since we look for other kinds of ordinances now * Add emphasis to prompt * Minor prompt clarification * Add clarifications to prompts * Fix density description * Add clarifier about density units * Increase location thresh * Fix docstring * Add total cost to meta * Add cost via tracker * Optional arg * Pass tracker to updater * Allow Utility and commercial "tech" to be treated similarly * Add geopandas as a dev req * `prop_line` -> `property line` * `struct` -> `structures` * Format scores * `rail` -> `railroads` * `trans` -> `transmission` * `density` -> `tower density` * `max size` -> `max project size` * `max` -> `maximum` * `min` -> `minimum` * Define `LLMConfig` interface and implement `OpenAIConfig` * namespace imports * Rename arguments * Deprecate old class * Update tutorial * Rename var * Rename function * Emphasize numerical * Clarification for numeric ords * Add log call for cache * Add ability to specify DL prog bar * Short pause to acknowledge PB completion * Add `TempFileCachePB` class * Use `TempFileCachePB` class * Update log message * New compass exception * Better control flow for tree evaluation * Catch new error type * Add `_setup_logging_levels` function * Use new `_setup_logging_levels` function * Refactor `_setup_cli_logging` to accept verbosity level and dynamically set libraries based on verbosity * Update verbose option to count and adjust logging setup in CLI * Clarify logging output message in async decision tree * Update logging levels and messages for clarity across parsing and downloading modules * Refactor logging setup in `_setup_logging` to only attach logging for COMPASS * Remove unnecessary clarification about error-level messages in README for structured extraction * Replace `logger.debug` with `logger.debug_to_file` for ordinance and text processing logs * `DEBUG` -> `DEBUG_TO_FILE` when running script * Logging call before progress bar starts * `_MofNCompleteColumn` now allows customizable text style for completed/total display * Remove SpinnerColumn from progress bar configurations and replace with _MofNCompleteColumn for improved display of completed tasks. * Quiet OCR errors * Update logging messages for jurisdiction count to use pluralization * Replace `UserWarning` with `COMPASSWarning` in warning messages for ordinance extraction and county utilities * Add warnings filter to suppress warnings during processing in CLI
* feat: Prototype of Quantitative Ordinance * feat: Ordinance::init_db * feat: Including Ordinance init_db into Scrapper's init_db * clean: Removing prototype DB code from core library * feat: Prototype of Ordinance::open() * refact: Ordinance and OrdinanceRecord * test: Create samples and use to test * style: * feat: Using Ordinance with sample at module level * style: * cfg: Allowing dead_code while developing * fix: offset is reserved * refact: value is required * feat: ordinance::write * feat: Loading ordinance into DB * cfg: Updating cargo lock with latest CLI * style: * Downgrading warn used during development to trace * test: Test on hold until finished quantitative ordinance
* Update docs * Add `ordinances_bool_index` function * Expose in namespace * Update docstring * Filter outputs
* Update docstring * Add tag parameter * Track model names and update tag accordingly * Use tag to set up service * Give tag a separator
Add `effective_year`, `effective_month`, and `effective_day` to output
* feat: Prototype of jurisdictions * Missing field * feat: Collection::init_db * feat: Collection::write() * wip: Moving new Jurisdiction into Source There are too much overlap in the concept of the new Jurisdiction and the original Source, so I'm merging both into Source. It is still missing some functionalities to validate the files, such as validating the checksum. * Updating mod ScrappedOrdinance to use new Source * refact: FIPS -> fips to conform with expected lower case name * clean: Removing old Source * style: * fix: Have to define alias for FIPS * clean: * Downgrading all logs to trace * style: * clean: Unused import * wip: Not showing number of documents for now * feat: Validating calculated checksum with parsed checksum While jurisdictions already provides a checksum, we can re-calculate on the given files and compare if the file we have is what we expected. * style: * clean: Removing unused fields Some fields used during development are not required anymore. * log: Adjusting log levels * feat, test: Adding samples to sources
* Add citation file * Fix typo
* wip: Qualitative ordinance * refact: Re-organizing code ordinance module structure * feat: Implementing qualitative ordinance module * cfg: Importing qualitative ordinance * wip: Extracting quantitative from mod ordinance * test: Re-routing ordinance test to sub-modules * refact: Ordinance is now composed by [quali + quanti]tative * refact: Renaming former Ordinance to Quantitative * refact: Ordinance::sample to use sub-modules samples * style: * style: * fix: New table name for quantitative ordinance * doc, fix: Updating module title for quantitative ordinance * style:
* cfg: Updating Rust * cfg: Updating some dependencies * cfg: pixi update * cfg: Adding dependabot updates Only security updates on the Rust side, and everything on GA side. I'll leave for @ppinchuk to decide if want to add for Python as well. * cfg: Minimum Rust 1.87 * cfg: Updating Cargo.lock
* Bump prefix-dev/setup-pixi from 0.8.1 to 0.8.8 Bumps [prefix-dev/setup-pixi](https://github.com/prefix-dev/setup-pixi) from 0.8.1 to 0.8.8. - [Release notes](https://github.com/prefix-dev/setup-pixi/releases) - [Commits](prefix-dev/setup-pixi@v0.8.1...v0.8.8) --- updated-dependencies: - dependency-name: prefix-dev/setup-pixi dependency-version: 0.8.8 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Updating pixi used in CI to 0.47 * Updating pixi used in CI * Updating pixi --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Guilherme Castelão <guilherme@castelao.net>
Bumps [release-drafter/release-drafter](https://github.com/release-drafter/release-drafter) from 5.15.0 to 6.1.0. - [Release notes](https://github.com/release-drafter/release-drafter/releases) - [Commits](release-drafter/release-drafter@v5.15.0...v6.1.0) --- updated-dependencies: - dependency-name: release-drafter/release-drafter dependency-version: 6.1.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Guilherme Castelão <guilherme@castelao.net>
Bumps [peaceiris/actions-gh-pages](https://github.com/peaceiris/actions-gh-pages) from 3.6.1 to 4.0.0. - [Release notes](https://github.com/peaceiris/actions-gh-pages/releases) - [Changelog](https://github.com/peaceiris/actions-gh-pages/blob/main/CHANGELOG.md) - [Commits](peaceiris/actions-gh-pages@v3.6.1...v4.0.0) --- updated-dependencies: - dependency-name: peaceiris/actions-gh-pages dependency-version: 4.0.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Top-level title now uses title markdown * New category * Add dependabot tag
* Add tip * Wind regs now include project size and max lot size * Update legal text prompt * Avoid definitions only * Only check value and summary columns * Prompt update * Fix function logic * Add support for `exclude_features` input * Use `exclude_features` input * Minor clarification update * Better sanitize * More detailed prompts * prompt updates * prompt clarification * Add extra check for bans * Extra ban text
* Update function name * Rename class * Update parameter * More renaming * More renames * Rename module * Rename function * More renaming * More re-naming * Update jurisdiction to be more generic * Add another test * Rework jurisdiction parsing * Add support script and update data * Almost fix all jurisdiction loading tests * Update jurisdictions * Fix missing attribute * Add subdivision name and jurisdiction type to outputs * Update tests * More test fixes * JSON prompt clarification * Consistent prompt formatting * Add dictionary to prompt * Add logging statement * Minor fix to init * Rename old class * Rename old class * Rename class * Add jurisdiction phrases + tests * Add support + test for parish * Add `setup_graph_correct_jurisdiction_type` + tests * Generalize `_weighted_vote` and add test * Rename class * Fix loading counties only + test * Allow `usage_sub_label` input * Updaet query templates * prefer "jurisdiction" to "location" * Add `setup_graph_correct_jurisdiction_from_url` function + tests * Docstring * Rename class * Add `Jurisdiction Type` to error output * No `np.nan` values + test * OCR loads in a separate process * Jurisdiction can now be town * Add emphasis * Use jurisdiction * Formatting * Allow `setup_async_decision_tree` to take a `usage_sub_label` as input * Update enums * Add decision tree validators * Minor doc update * Update docstring
* Only include `OCRPDFLoader` if pytesseract is requested * Tests don't hand anymore * Add flaky markers * Tests now use `DTreeURLCountyValidator` * Test `DTreeJurisdictionValidator` * Test `JurisdictionValidator` * Patch up tests * Minor docstring update * Use `DTreeJurisdictionValidator.META_SCORE_KEY` * Soften tone in prompts to avoid jailbreak flag * Correctly set one worker * Log messages on bad request * Deprecate unused classes * Fix up tests
* Update pixi caching * Use a shared key for rust cache to avoid one cache for each job * Update text * Update docs * Typo fixes
* Remove confusing assignment * Soften prompts to help avoid jailbreak flag * Add `briefly` clarification * Minor re-structure * Add missing import * Update deps * Add `JURISDICTION_TYPES_AS_PREFIXES` global constant * Add OCR test * Update prompts to better capture some jurisdictions * Short circuit URL validation * Do URL check first * `ParseChunksWithMemory` now takes an llm validation callback * `LegalTextValidator` now derives from `StructuredLLMCaller` * Text collectors now derive from `StructuredLLMCaller` * Correctly initialize text collectors * Add `setup_graph_correct_document_type` * Fix test * Fix tests * Set up conftest * Add legal text validation tests * Fix tests running without env keys * Add docs for test * Add extra moratorium check
* feat: CLI returns anyhow::Result * fix: Unecessary use of reference * cfg: Updating Cargo.lock with anyhow * refact: Passing lib's error to CLI * feat: Adding context to CLI * style:
* More details for `setup_participating_owner` * Minor typo fix * Fix bug in `_remove_key_for_empty_value` * Add extra property line type * Add `ignore_features` to `EXTRACT_ORIGINAL_TEXT_PROMPT` * Add utility scale clarification to `EXTRACT_ORIGINAL_TEXT_PROMPT` * Add `ignore_features` to base setback graph * `SETBACKS_SYSTEM_MESSAGE` now formatted with all feature kwargs * Clarify density prompt * Add shadow flicker clarification * Break min and max distance conditionals into separate graphs * Ask for numerical qualifiers in system size prompt * Add utility scale verification to base setback prompts * Adder prompt clarification * Put equation in backticks * Add an `out_no_adder` node * Clarify prompt * Formatting * Add ignore types to system message * Add utility-scale focus to system message * Add utility-scale clarification to `RESTRICTIONS_SYSTEM_MESSAGE` * Add utility-scale check to base setback graph * Simplify `setup_participating_owner` by adding `ignore_features` check * Clarify instructions * Formatting * Add utility scale clarifications * Formatting * Fix prompts * Add missing spaces * Soften tone to avoid jailbreak flag
* Initializing DB schema * Script to build diagram * Extending for other tables * Adding linkage from usage_per_item -> usage * Adding foreign keys connection * Figure with current model * Removing connection labels for now * cfg: Adding graphviz in rust-dev The app to generate the DB model.
* feat: Extending arguments for subcommand `export` * style: * cfg: Defining repository at workspace level * Proof of concept on exporting to CSV * refact: export() now receives a Writer * refact: For now, output is required * feat: CLI export gets arguments and log those * clean: * Inheriting CSV errors * refact: export() now returns a Result With availability of Result, propagate errors. * feat: bail out if output file already exists * refact: Propagate export's error * refact: Reinforce create new without overwriting * style: Pre load tracing::info * style: * refact: Unecessary use of reference * style: * style: * clean: * refact: Adding missing linkage to usage * log: Mark when bookkeeping table is created * style: * style: Allowing to use 'CSV' (upper case) * Bump version * Bump library version
* clean: Old Jurisdiction model * refact: Using a common limit to json size * clean: * refact: Renaming write() to record() * doc: Documenting module source * clean: * log: Ugrading initializing log to debug * doc: Documenting ScrappedOrdinance::init_db
…drop correct + month/day
… by duck-typed signature
…via autouse teardown
… manifest+Result+CSV
…ounty'/'Parish' suffix)
…sdiction_from helper
Collaborator
Author
|
Moved here: #450 (to use git-lfs for the eval data management) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Starting by adding a accuracy test for
extract_datethat feeds production-shaped documents (real source URL + parsed pages, loaded from local disk) and asserts the extracted year against a ground-truth set built from the solar-validation data. Includes about 65 documents with known enactment years and 5 "no date" cases (asserting no year is invented).The test makes live, billable LLM calls, so it is gated behind an
expensivepytest marker (deselected by default everywhere). A manually-triggeredexpensive-testsworkflow runs it on demand. A terminal-summary hook prints a per-case cost + accuracy breakdown.