Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

speed up large polygon intersections using STRtrees. #259

Open
wants to merge 47 commits into
base: as/stabletasks
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
375ea0a
speed up large polygon intersections using an STRtree on poly b
asinghvi17 Feb 14, 2025
32e77db
add staticarrays + strtree
asinghvi17 Feb 14, 2025
9995e7b
works but bad perf "do-everything-function"
asinghvi17 Feb 14, 2025
d98949c
Fix type instability
asinghvi17 Feb 14, 2025
cc5545d
WIP dual query
asinghvi17 Feb 14, 2025
05ee402
make the benchmark script work again
asinghvi17 Feb 14, 2025
c19730f
fixes
asinghvi17 Feb 14, 2025
7411f3b
add comments + simplify edgelist generation
asinghvi17 Feb 14, 2025
c7c1fbe
Final clean up
asinghvi17 Feb 14, 2025
25c414d
Move things around + loop state machine
asinghvi17 Feb 14, 2025
edc6123
some things got left behind
asinghvi17 Feb 14, 2025
b5c09f2
add a Return action
asinghvi17 Feb 14, 2025
0b98d8b
plotting quickfix
asinghvi17 Feb 14, 2025
6aa1187
Better tests and testsets (#260)
asinghvi17 Feb 14, 2025
dace097
Fix package.json that stopped doc build (#261)
asinghvi17 Feb 14, 2025
ae01d9b
Move doctests to a separate CI job
asinghvi17 Feb 14, 2025
1416bd5
Fix typo
asinghvi17 Feb 14, 2025
e25ef73
make watersheds reproducible, move it to the bottom
asinghvi17 Feb 14, 2025
ae1189b
fix++
asinghvi17 Feb 14, 2025
cd87f7b
edge thinning
rafaqz Feb 15, 2025
6385fd9
Merge branch 'as/non_working_do_everything' into edge_thinning
rafaqz Feb 15, 2025
1ba7bb8
abstracted indices
rafaqz Feb 15, 2025
5cbf798
working filtered single tree
rafaqz Feb 15, 2025
6d86135
all working
rafaqz Feb 15, 2025
726106b
Fix vancouver polygons benchmark
asinghvi17 Feb 15, 2025
1e00098
some minor edits to man-alg-op system
asinghvi17 Feb 15, 2025
0bba556
Merge remote-tracking branch 'origin/edge_thinning' into as/non_worki…
asinghvi17 Feb 15, 2025
9d01309
Add an algorithm (plus manifold and accelerator) to clipping
asinghvi17 Feb 15, 2025
047403d
add STRtree branch in CI
asinghvi17 Feb 15, 2025
95bd6f2
use package extents
rafaqz Feb 15, 2025
a136e2d
better circle benchmarks
asinghvi17 Feb 15, 2025
db39f96
fix branch name in CI
asinghvi17 Feb 15, 2025
a0e2772
better benchmark plot
asinghvi17 Feb 15, 2025
f7485c8
don't unnecessarily use densified geometries at this point
asinghvi17 Feb 15, 2025
09a549b
Fix cut
asinghvi17 Feb 15, 2025
800c131
fix difference and union
asinghvi17 Feb 15, 2025
49790bf
Remove DoubleSTR from optimization and tests
asinghvi17 Feb 15, 2025
f93665f
fix cut
asinghvi17 Feb 15, 2025
ad48339
Short circuit SingleSTRTree if there is no possible interaction
asinghvi17 Feb 15, 2025
c9878fe
force the loop to be at maximum n^2 iterations in vancouver bench
asinghvi17 Feb 16, 2025
eb1f921
`@processloopaction` -> `@controlflow`
asinghvi17 Feb 16, 2025
ae908b1
remove dual query code
asinghvi17 Feb 16, 2025
e904f65
fix
asinghvi17 Feb 16, 2025
38c731b
fix tests
asinghvi17 Feb 16, 2025
925eb1f
Implement difference targeting multipolygontrait
asinghvi17 Feb 16, 2025
53e2c61
add multipolygon-target intersection too
asinghvi17 Feb 16, 2025
a7d4743
add STRtree main
asinghvi17 Feb 18, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 18 additions & 3 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,8 @@ jobs:
version: ${{ matrix.version }}
arch: ${{ matrix.arch }}
- uses: julia-actions/cache@v1
- name: Dev GeometryOpsCore`
run: julia --project=. -e 'using Pkg; Pkg.develop(; path = joinpath(".", "GeometryOpsCore"))'
- name: Dev GeometryOpsCore and add other packages
run: julia --project=. -e 'using Pkg; Pkg.develop(; path = joinpath(".", "GeometryOpsCore")); Pkg.add(; name = "SortTileRecursiveTree", rev = "main")'
- uses: julia-actions/julia-buildpkg@v1
- uses: julia-actions/julia-runtest@v1
- uses: julia-actions/julia-processcoverage@v1
Expand All @@ -60,13 +60,28 @@ jobs:
with:
version: '1'
- name: Build and add versions
run: julia --project=docs -e 'using Pkg; Pkg.develop([PackageSpec(path = "."), PackageSpec(path = joinpath(".", "GeometryOpsCore"))]); Pkg.add([PackageSpec(name = "GeoMakie", rev = "master"), PackageSpec(name="GeoInterface", rev="bugfix_vars")])'
run: julia --project=docs -e 'using Pkg; Pkg.develop([PackageSpec(path = "."), PackageSpec(path = joinpath(".", "GeometryOpsCore"))]); Pkg.add([PackageSpec(name = "GeoMakie", rev = "master")])'
- uses: julia-actions/julia-docdeploy@v1
with:
install-package: false
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
DOCUMENTER_KEY: ${{ secrets.DOCUMENTER_KEY }}
doctests:
name: Doctests
runs-on: ubuntu-latest
permissions:
contents: write
statuses: write
actions: write
steps:
- uses: actions/checkout@v2
- uses: julia-actions/cache@v1
- uses: julia-actions/setup-julia@v1
with:
version: '1'
- name: Build and add versions
run: julia --project=docs -e 'using Pkg; Pkg.develop([PackageSpec(path = "."), PackageSpec(path = joinpath(".", "GeometryOpsCore"))]); Pkg.add([PackageSpec(name = "GeoMakie", rev = "master")])'
- run: |
julia --project=docs -e '
using Documenter: DocMeta, doctest
Expand Down
9 changes: 9 additions & 0 deletions GeometryOpsCore/src/types/algorithm.jl
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,15 @@ Algorithms are:

abstract type Algorithm{M <: Manifold} end

struct AutoAlgorithm{T, M <: Manifold} <: Algorithm{M}
manifold::M
x::T
end

AutoAlgorithm(m::Manifold; kwargs...) = AutoAlgorithm(m, kwargs)
AutoAlgorithm(; kwargs...) = AutoAlgorithm(AutoManifold(), kwargs)


abstract type ManifoldIndependentAlgorithm{M <: Manifold} <: Algorithm{M} end

abstract type SingleManifoldAlgorithm{M <: Manifold} <: Algorithm{M} end
Expand Down
23 changes: 18 additions & 5 deletions GeometryOpsCore/src/types/manifold.jl
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,14 @@ In GeometryOps (and geodesy more generally), there are three manifolds we care a
- [`Planar`](@ref): the 2d plane, a completely Euclidean manifold
- [`Spherical`](@ref): the unit sphere, but one where areas are multiplied by the radius of the Earth. This is not Euclidean globally, but all map projections attempt to represent the sphere on the Euclidean 2D plane to varying degrees of success.
- [`Geodesic`](@ref): the ellipsoid, the closest we can come to representing the Earth by a simple geometric shape. Parametrized by `semimajor_axis` and `inv_flattening`.
- [`AutoManifold`](@ref): a special manifold that automatically selects the best manifold for the operation when it's executed. Resolves to [`Planar`](@ref), [`Spherical`](@ref), or [`Geodesic`](@ref) depending on the input geometry.

Generally, we aim to have `Linear` and `Spherical` be operable everywhere, whereas `Geodesic` will only apply in specific circumstances.
Currently, those circumstances are `area` and `segmentize`, but this could be extended with time and https://github.com/JuliaGeo/SphericalGeodesics.jl.
Currently, those circumstances are [`area`](@ref), [`arclength`](@ref), and [`segmentize`](@ref), but this could be extended with time and https://github.com/JuliaGeo/SphericalGeodesics.jl.
=#

export Manifold, Planar, Spherical, Geodesic
export Manifold, AutoManifold, Planar, Spherical, Geodesic

"""
abstract type Manifold

Expand All @@ -30,6 +32,16 @@ Currently we have [`Planar`](@ref), [`Spherical`](@ref), and [`Geodesic`](@ref)
"""
abstract type Manifold end

"""
AutoManifold()

The `AutoManifold` is a special manifold that automatically selects the best manifold for the operation.
It does not carry any parameters, nor does it indicate anything about the nature of the space.

This gets resolved to a specific manifold when an operation is applied.
"""
struct AutoManifold <: Manifold end

"""
Planar()

Expand Down Expand Up @@ -57,7 +69,7 @@ A spherical manifold means that the geometry is on the 3-sphere (but is represen
`-90` (south pole) and `90` (north pole).
"""
Base.@kwdef struct Spherical{T} <: Manifold
radius::T = 6371008.8
radius::T = 6371008.8 # this should be theWGS84 defined mean radius
end

"""
Expand All @@ -69,12 +81,13 @@ and `inv_flattening` (``1/f``).
Usually, this is only relevant for area and segmentization calculations. It becomes more relevant as one grows closer to the poles (or equator).
"""
Base.@kwdef struct Geodesic{T} <: Manifold
semimajor_axis::T = 6378137.0
inv_flattening::T = 298.257223563
semimajor_axis::T = 6378137.0 # WGS84 by default
inv_flattening::T = 298.257223563 # WGS84 by default
end


# specifically for manifolds
# not used now but will be used later
abstract type EllipsoidParametrization end

struct SemimajorAxisInvFlattening{T} <: EllipsoidParametrization
Expand Down
16 changes: 14 additions & 2 deletions GeometryOpsCore/src/types/operation.jl
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,18 @@ but if we ever implement e.g. RelateNG in GeometryOps, we can add that in.
abstract type Operation{Alg <: Algorithm} end

# example
struct XPlusOneOperation <: Operation{NoAlgorithm}
x::Int
# struct XPlusOneOperation{M <: Manifold} <: Operation{NoAlgorithm{M}}
# m::M # the manifold always needs to be stored, since it's not a singleton
# x::Int
# end


struct Area{Alg<: Algorithm} <: Operation{Alg}
alg::Alg
end

Area() = Area(NoAlgorithm(Planar()))

function (op::Area{Alg})(data; threaded = _False(), init = 0.0) where {Alg <: Algorithm}
return GO.area(op.alg, data; threaded, init)
end
2 changes: 2 additions & 0 deletions Project.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ authors = ["Anshul Singhvi <[email protected]>", "Rafael Schouten <rafaels
version = "0.1.12"

[deps]
AbstractTrees = "1520ce14-60c1-5f80-bbc7-55ef81b5835c"
CoordinateTransformations = "150eb455-5306-5404-9cee-2592286d6298"
DataAPI = "9a962f9c-6df0-11e9-0e5d-c546b8b5ee8a"
DelaunayTriangulation = "927a84f5-c5f4-47a5-9785-b46e178433df"
Expand All @@ -27,6 +28,7 @@ GeometryOpsLibGEOSExt = "LibGEOS"
GeometryOpsProjExt = "Proj"

[compat]
AbstractTrees = "0.4.5"
CoordinateTransformations = "0.5, 0.6"
DataAPI = "1"
DelaunayTriangulation = "1.0.4"
Expand Down
4 changes: 3 additions & 1 deletion benchmarks/benchmark_plots.jl
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ function plot_trials(
legend_orientation = :horizontal,
legend_halign = 1.0,
legend_valign = -0.25,
legend_kws = (;),
)

xs, ys, labels = [], [], []
Expand Down Expand Up @@ -118,7 +119,8 @@ function plot_trials(
tellheight = legend_position isa Union{Tuple, Makie.Automatic} && (legend_position isa Makie.Automatic || length(legend_position) != 3) && legend_orientation == :horizontal,
halign = legend_halign,
valign = legend_valign,
orientation = legend_orientation
orientation = legend_orientation,
legend_kws...,
)
ax.xtickcolor[] = ax.xgridcolor[]
ax
Expand Down
Loading
Loading