Skip to content

Release: v0.1.0#4

Merged
RemingtonRohel merged 81 commits into
mainfrom
develop
Apr 9, 2026
Merged

Release: v0.1.0#4
RemingtonRohel merged 81 commits into
mainfrom
develop

Conversation

@RemingtonRohel
Copy link
Copy Markdown
Collaborator

@RemingtonRohel RemingtonRohel commented Sep 27, 2024

Implements the RAWACF -> FITACF -> GRID pipeline, with both Rust and Python APIs.

Installation

Use maturin to build from source with Python API. See README for details.

APIs

See the README for the relevant Python and Rust functions.

Binary

raw2fit: Command-line tool for fitting a rawacf file with the FITACF3 algorithm.

Usage: raw2fit <INFILE> <OUTFILE>

Arguments:
  <INFILE>   Rawacf file to fit
  <OUTFILE>  Output fitacf file path

Options:
  -h, --help     Print help
  -V, --version  Print version

fit2grid: Command-line tool for gridding a fitacf file.

Usage: fit2grid [OPTIONS] <OUTFILE> [-- <INFILES>...]

Arguments:
  <OUTFILE>     Output grid file path
  [INFILES]...  Fitacf file(s) to grid

Options:
      --start-time <START_TIME>
          Start time in HH:MM format [aliases: st]
      --end-time <END_TIME>
          End time in HH:MM format [aliases: et]
      --start-date <START_DATE>
          Start date in YYYYMMDD format [aliases: sd]
      --end-date <END_DATE>
          End date in YYYYMMDD format [aliases: ed]
      --interval <INTERVAL>
          Use interval of length HH:MM [aliases: ex]
      --scan-length <SCAN_LENGTH>
          Scan length specification in whole seconds, overriding the scan flag [aliases: tl]
  -i, --record-interval <RECORD_INTERVAL>
          Time interval to store in each grid record, in whole seconds [default: 120]
      --channel <CHANNEL>
          Stereo channel identifier, either 'a' or 'b' [aliases: cn]
      --channel-fix <CHANNEL_FIX>
          User-defined channel identifier for the output file only [aliases: cn_fix]
      --exclude-beams <EXCLUDE_BEAMS>
          Beams to exclude, as a comma-separated list [aliases: ebm]
      --min-range-gate <MIN_RANGE_GATE>
          Minimum range gate [aliases: minrng]
      --max-range-gate <MAX_RANGE_GATE>
          Maximum range gate [aliases: maxrng]
      --min-slant-range <MIN_SLANT_RANGE>
          Minimum slant range in km [aliases: minsrng]
      --max-slant-range <MAX_SLANT_RANGE>
          Maximum slant range in km [aliases: maxsrng]
      --filter-weighting <FILTER_WEIGHTING>
          Filter weighting mode [default: 0] [aliases: fwgt]
      --max-power <MAX_POWER>
          Maximum power (linear scale) [default: 60] [aliases: pmax]
      --max-velocity <MAX_VELOCITY>
          Maximum velocity in m/s [default: 2500] [aliases: vmax]
      --max-spectral-width <MAX_SPECTRAL_WIDTH>
          Maximum spectral width in m/s [default: 1000] [aliases: wmax]
      --max-velocity-error <MAX_VELOCITY_ERROR>
          Maximum velocity error in m/s [default: 200] [aliases: vemax]
      --min-power <MIN_POWER>
          Minimum power (linear scale) [default: 3] [aliases: pmin]
      --min-velocity <MIN_VELOCITY>
          Minimum velocity in m/s [default: 35] [aliases: vmin]
      --min-spectral-width <MIN_SPECTRAL_WIDTH>
          Minimum spectral width in m/s [default: 10] [aliases: wmin]
      --min-velocity-error <MIN_VELOCITY_ERROR>
          Minimum velocity error in m/s [default: 0] [aliases: vemin]
      --altitude <ALTITUDE>
          Altitude at which mapping is done in km [default: 300] [aliases: alt]
      --max-frequency-var <MAX_FREQUENCY_VAR>
          Maximum allowed frequency variation in Hz [default: 500000] [aliases: fmax]
      --boxcar-filter-flag
          Flag to disable boxcar median filtering [aliases: nav]
      --no-limits-flag
          Flag to include data that exceeds limits [aliases: nlm]
      --op-param-flag
          Flag to exclude data that doesn't match operating parameter requirements [aliases: nb]
      --exclude-neg-scan-flag
          Flag to exclude data with scan flag of -1 [aliases: ns]
      --extended-mode-flag
          Extended output, include power and width in output file [aliases: xtd]
      --sort-params-flag
          If using a median filter, sort parameters independent of the velocity [aliases: isort]
      --ionosphere-only-flag
          Exclude data marked as ground scatter [aliases: ion]
      --groundscatter-only-flag
          Exclude data not marked as ground scatter [aliases: gs]
      --all-data-flag
          Do not exclude data based on scatter flag [aliases: both]
      --inertial-frame-flag
          Use inertial reference frame [aliases: inertial]
      --chisham-flag
          Map data using Chisham virtual height model [aliases: chisham]
  -v, --verbose
          Verbose mode
  -h, --help
          Print help
  -V, --version
          Print version

RemingtonRohel and others added 30 commits November 9, 2023 22:41
* So far have copied all of the command line arguments from RST that are relevant
* Have yet to implement any functionality
* Added default arguments and types for some cli arguments
* Adding structs for gridding
* Adding RadarScan struct
* Adding in rpos utilities
* Finished GridTable::map() method, working through GridTable::add_beam()
* Using geodesy crate for coordinate transformations.
* Including IGRF calls is going to be extremely tedious.
…Records.

* Method used in the grid.rs binary.
* Added function to find the first record after a timestamp in an array of FitacfRecords
* Added more functionality to bin_main() of grid.rs binary, up to calculating end_time for the grid.
* Need to iterate through GridPoints of Table and convert them to vectors suitable for GridRecord
* Rounded out the rest of bin_main in bin/grid.rs, still lots of TODOs
* Corrected RadarScan::reset_beams() after reviewing equivalent RST function
* Updated dmap dependency
* Created new file utils/dmap.rs and put function there, as it will have shared usage by gridding.
* Updated dmap dependency to fix some types in GridRecord
* Removed unused use statements
* Fixed moved value errors in bin/grid.rs
* Completed TODOs around filtering data from scans when gridding in bin/grid.rs
* Still have to apply boxcar median filter, and load AACGM coefficients
* Write the GridRecords to file once done looping through files
Started working on median filter kernel for gridding.

* Still very much a work in progress, only about 1/8 implemented.
* Updated dmap crate dependency to use new error_refactor branch
* Still untested, but IDE not throwing any warnings in the file.
* fieldpoint calculations now mostly added. Still have some functions to add but pretty close to complete.
* Still have some differences in numerical results vs. RST, working through those.
…` command.

* Uses new DMAP API
* Benchmarked faster than RST, 8x on my machine with 8 cores
* Fixed clippy warnings
* Made fit_fitacf3 arguments positional
* Created a convenience function to get the hdw info associated with a rawacf record, reducing copied code in bench and tests.
Update following `dmap` API change
* Removed Rust binary `fit_fitacf3`, exposed `fit_fitacf3` as a Python CLI tool using maturin.
* Created another Python-available function `fitacf3` which takes a list of rawacf records, like what is read in by `dmap.read_rawacf()`
* Created two rust functions, `fitacf3` and `par_fitacf3` which are single- and multi-threaded fitacf3 callers on `Vec<RawacfRecord>`, used in bench and test files and Python API functions.
* Avoids passing data through the Python-Rust interface
RemingtonRohel and others added 29 commits March 3, 2025 16:28
* Now compiles, currently working through more bugs that cause errors at runtime.
* AACGM_v2_DAT_PREFIX not set - need to figure out how to get this from dependency, if possible.
* Currently, stumped at GridTable::map() panicking with an index error.
* median_filter calculated max number of beams correctly; skips averaging beams with no cells
* Made all GridArgs members public; removed `old_aacgm_flag` member
* Fixed some mixups between `no_limits_flag` and `op_param_flag`
* let `end_time` be an Option; doesn't error out now if not specified
* Gated some printing with `args.verbose`
* Used `extended_mode_flag` to determine whether pwr + spectral width written to file
* Fixed `RadarScan::add_beam()` so it initializes vectors of correct size
* Handle partial records (no slist + other vectors in fitacf record)
* Ran cargo fmt
* Testing shows differences vs. RST, but is able to produce files. Need
  to investigate where the differences arise.
* Added a suite of gridding tests, all passing so far.
* Added structs for different coordinate systems.
* Now present two fit2grid functions - one for providing file paths, and another for providing FITACF records.
* GridArgs no longer requires `infiles` or `outfile` - these are provided with `GridArgsCLI` struct that wraps `GridArgs` for exposed `fit2grid` binary.
Added RST's `make_grid`, calling it `fit2grid`
* raw2fit and fit2grid are the new binaries
@RemingtonRohel RemingtonRohel merged commit 55ca3c2 into main Apr 9, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant