-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Reduce libcudf binary size by trimming instantiations #23706
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
af1932c
d3cfcc7
784fdfd
3fae4f7
1fcd78d
5fc3e75
defbc05
090ee57
aae612c
85b0ce3
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -194,8 +194,7 @@ dispatch_tuple_t dispatch_from_arrow_device::operator()<cudf::string_view>( | |
| d_indices.begin(), | ||
| binary_view_to_string_index_pair{d_items, d_ptrs, d_mask, skip_mask}); | ||
| // gather strings into output column | ||
| auto out_col = | ||
| cudf::strings::detail::make_strings_column(d_indices.begin(), d_indices.end(), stream, mr); | ||
| auto out_col = cudf::make_strings_column(d_indices, stream, mr); | ||
|
vyasr marked this conversation as resolved.
|
||
| owned.emplace_back(std::move(out_col)); | ||
| stream.synchronize(); | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🚀 Performance & Scalability | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- candidate file map ---'
ast-grep outline cpp/src/interop/from_arrow_device.cu
ast-grep outline cpp/src/interop/from_arrow_host_strings.cu
printf '%s\n' '--- relevant source sections ---'
sed -n '150,220p' cpp/src/interop/from_arrow_device.cu
sed -n '70,135p' cpp/src/interop/from_arrow_host_strings.cu
printf '%s\n' '--- factory declarations and definitions ---'
rg -n -C 4 'make_strings_column|from_arrow_device|from_arrow_host_strings' cpp include
printf '%s\n' '--- synchronization and stream usage in these paths ---'
rg -n -C 3 'stream\.synchronize|make_strings_column|exec_policy_nosync|cudaStream' cpp/src/interop/from_arrow_device.cu cpp/src/interop/from_arrow_host_strings.cuRepository: NVIDIA/cudf Length of output: 50368 🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- from_arrow_device includes and relevant implementation ---'
sed -n '1,75p' cpp/src/interop/from_arrow_device.cu
sed -n '160,215p' cpp/src/interop/from_arrow_device.cu
printf '%s\n' '--- from_arrow_host_strings full implementation ---'
sed -n '1,130p' cpp/src/interop/from_arrow_host_strings.cu
printf '%s\n' '--- string factory implementation ---'
sed -n '35,90p' cpp/include/cudf/strings/detail/strings_column_factories.cuh
sed -n '875,925p' cpp/src/strings/strings_column_factories.cu cpp/src/transform/transform.cu 2>/dev/null || true
printf '%s\n' '--- exact factory source locations ---'
rg -n -C 5 'make_strings_column\(IndexPairIterator|make_chars_buffer\(' cpp/include/cudf/strings/detail/strings_column_factories.cuh cpp/srcRepository: NVIDIA/cudf Length of output: 20335 🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- make_chars_buffer implementation ---'
sed -n '120,205p' cpp/src/strings/strings_column_factories.cu
sed -n '130,190p' cpp/src/transform/transform.cu
printf '%s\n' '--- host interop callers and return usage ---'
rg -n -C 8 'string_column_from_arrow_host|from_arrow_stringview|from_arrow_host' cpp/src/interop cpp/tests/interop
printf '%s\n' '--- synchronization in Arrow interop sources and tests ---'
rg -n -C 4 'synchronize\(\)|stream\.sync\(\)|make_strings_column' cpp/src/interop cpp/tests/interop
printf '%s\n' '--- stream-ordered destruction and async factory patterns ---'
rg -n -C 4 'device_uvector<.*>.*stream|device_buffer.*stream|make_strings_column\(.*stream' cpp/src cpp/include | head -n 240Repository: NVIDIA/cudf Length of output: 50367 🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- single-column string factory source ---'
sed -n '1,135p' cpp/src/strings/strings_column_factories.cu
printf '%s\n' '--- device conversion return path ---'
sed -n '340,525p' cpp/src/interop/from_arrow_device.cu
printf '%s\n' '--- host conversion return path ---'
sed -n '430,530p' cpp/src/interop/from_arrow_host.cu
printf '%s\n' '--- focused synchronization inventory ---'
python3 - <<'PY'
from pathlib import Path
files = [
Path("cpp/src/interop/from_arrow_device.cu"),
Path("cpp/src/interop/from_arrow_host_strings.cu"),
Path("cpp/src/interop/from_arrow_host.cu"),
Path("cpp/src/strings/strings_column_factories.cu"),
]
for path in files:
print(f"--- {path} ---")
for n, line in enumerate(path.read_text().splitlines(), 1):
if "synchronize()" in line or ".sync()" in line or "make_strings_column(" in line:
print(f"{n}: {line.strip()}")
PYRepository: NVIDIA/cudf Length of output: 18606 🌐 Web query:
💡 Result: In RAPIDS Memory Manager (RMM), both Citations:
🌐 Web query:
💡 Result: In cuDF (libcudf), the Citations:
Remove both redundant stream synchronizations.
📍 Affects 2 files
🤖 Prompt for AI AgentsSource: MCP tools |
||
| return std::make_tuple<column_view, owned_columns_t>(owned.front()->view(), std::move(owned)); | ||
|
|
||
|
vyasr marked this conversation as resolved.
|
This file was deleted.
|
vyasr marked this conversation as resolved.
|
This file was deleted.
Uh oh!
There was an error while loading. Please reload this page.