feat(data_drip): add self-documenting backfills + catalog - #21
Open
jmjurado23 wants to merge 6 commits into
Open
feat(data_drip): add self-documenting backfills + catalog#21jmjurado23 wants to merge 6 commits into
jmjurado23 wants to merge 6 commits into
Conversation
jmjurado23
changed the title
3 tasks
jmjurado23
changed the title
jmjurado23
marked this pull request as ready for review
June 11, 2026 15:10
miguellarranaga-factorial
approved these changes
Jun 26, 2026
… UI) Re-implements the feature from PR #21 on top of the restyled main, using the current design system and conventions instead of the pre-restyle UI. - Add `description` and `instructions` DSL to DataDrip::Backfill, mirroring DataDrip::Script.description. `description` is a one-line catalog summary; `instructions` is Markdown guidance shown in the New Backfill Run form. - Add a searchable, paginated Backfills Catalog at /data_drip/backfills, reachable from a new "Catalog" pill in the shared header. Server-side search (autosubmit) and the shared pagination partial replace the PR's client-side JS. - Render `instructions` server-side via a tiny, dependency-free Markdown renderer (headings, bold, inline code, bullet lists, fenced code blocks) with Tailwind classes, reusing the existing backfill-options fetch — dropping the PR's markdown.js and backfill_form_controller.js. - Generalize Paginatable to page an in-memory array (the catalog list). - Document the DSLs + catalog in the README, seed the example backfills, and add the description/instructions hints to the generator template. - Recompile the shipped tailwind.css for the new classes. Specs: full suite green (312 examples). Restyle of #21. Co-Authored-By: Miguel Larrañaga <miguel.larranaga@factorial.co> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
jmjurado23
force-pushed
the
feat/add_description
branch
from
August 3, 2026 08:47
4f6c1f9 to
fd6ce23
Compare
javascript_importmap_tags resolves every pinned module through asset_path when it builds the import map, so a host on a strict Sprockets pipeline (check_precompiled_asset) raised "asset ... was not declared to be precompiled" for data_drip/controllers/*.js. The engine put its JS on config.assets.paths but never declared it for precompilation. Register data_drip/**/*.js (JS only — the compiled CSS is served outside the pipeline by DataDrip::AssetsController) on config.assets.precompile, guarded to a real Array so Propshaft hosts are unaffected. Add regression coverage. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
sprockets-rails resolves config.assets.precompile through
Sprockets::Manifest#find, which calls start_with? on each entry — a Regexp
entry raises NoMethodError ("undefined method `start_with?' for an instance of
Regexp") on a strict Sprockets host. Enumerate the engine's JS files and
register their explicit logical-path strings instead.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The class-name column dominated under the default auto table layout because the fully-qualified backfill class names are long. Switch to table-fixed with explicit widths (class 1/4, description 1/2, fields 1/4) and wrap long class names, so the description column is the widest. Recompile tailwind.css. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In development, Zeitwerk reloading leaves stale class copies in DataDrip::Backfill.descendants (DataDrip.all), so the catalog listed the same backfill several times — and stale copies show "No description". Resolve each name to its current constant and dedupe. The class picker already dedupes by name, so only the catalog was affected. Production (eager load) is unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ription The previous dedup resolved each name to its current constant, but under Zeitwerk reloading the live copy can lose its `description` while a stale copy keeps it — so the catalog showed "No description" for backfills that have one. Group the (possibly duplicated) copies by name and pick one that still carries a description. Production (eager load, no reload) is unaffected. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🚪 Why?
Problem
DataDrip's web UI only listed past backfill runs — there was no way to see which backfills (datadrips) exist, no place to document what each one does, and no guidance for operators on how to fill in a backfill's options before running it. On apps with many backfills, understanding any of them meant reading the source.
🔑 What?
Changes
description, shown in the catalog so anyone can tell what a datadrip does at a glance.instructionswritten in Markdown, declared with the same DSL idiom asdescription(instructions <<~MARKDOWN ... MARKDOWN; a plaindef self.instructionsoverride also works): rendered as formatted rich text in the New Backfill Run form the moment a backfill is selected, so operators see what it does and how to fill in the options before running. Rendered by a tiny dependency-free Markdown renderer (headings, bold, inline code, bullet lists, fenced code blocks)./data_drip/backfillslisting every datadrip with its description and the configurable fields (options) it accepts.company_ids).description/instructionsDSLs, the:enumattribute type, and the catalog page in the README.✅ Verification
Tests
description/custom_fields/instructionsDSLs (spec/lib/data_drip/backfill_spec.rb), instructions returned by the options endpoint (spec/controllers/data_drip/backfill_runs_controller_spec.rb), the catalog controller (spec/controllers/data_drip/backfills_controller_spec.rb), and the helper's optional-description / legacy-class fallbacks (spec/helpers/data_drip/backfills_helper_spec.rb).Manual Verification
🏷️ Skill tags
skill:backend🤖 Generated with Claude Code