Methodology for deriving a utility threshold for Dijkstra's algorithm (scipy) in bike model

[bhargavasana]

#329 Follow-up item.
@dhensle @americalexander @aber-sandag please propose a methodology.

[dhensle]

I propose plotting the 99th percentile of distance from the created paths as a function of utility threshold. This will allow us to interpolate the utility threshold cutoff that would produce our desired distance cutoff. We would run this for both TAZs and MAZs.

(The 99th percentile distance point may be too high or too low -- we would have to see where that mark is on the histogram of path distances to see if we are happy with that value.)

This methodology would allow us to see both the sensitivity of distances for the utility threshold as well as allow us to pick an appropriate value.

[aber-sandag]

I have some concerns about this approach after testing @americalexander's threshold recommendations. I tested the 0.75 utile threshold for MGRA and found that the total number of OD pairs found dropped from 19,150,646 to 391,282. A significant drop was expected, but this felt too extreme, so I did some additional checking. Filtering the previous MGRA output to only OD pairs with an average path distance less than 2 miles still returned 3,483,836 paths, meaning that our new output that is intended more-or-less as an equivalent to a 2 mile threshold is giving us 1/10 as many paths as an actual 2 mile threshold would.

I believe that we need to rethink this methodology. By looking at the 99th percentile of distance as a function of utility threshold, we are completely ignoring the number of OD pairs that exceed the given utility threshold. 99th percentile of distance only means that most included paths are within 2 miles, not that most paths within 2 miles are included.

I am unsure what a better approach would be. 99th percentile of utility as a function of distance threshold may be more informative, we cannot set distance threshold but we could run with a threshold that we know is too high and then filter by results within the desired distance threshold. I am not certain if this is a reasonable approach either though, and we would likely need a lower percentile.

Let's discuss this a little further without making it too big an issue. Ultimately this is only important for finding a balance between results and performance, I would rather have a threshold that is too high with slightly worse performance than one that significantly affects the outputs. I will be out until 9/22, but @JiaXu1024 and @bhargavasana are available.