Skip to content

Split gem to avoid FFI dependency for core - #127

Closed
DavidS wants to merge 7 commits into
enkessler:devfrom
DavidS:split-ffi-core
Closed

DavidS wants to merge 7 commits into
enkessler:devfrom
DavidS:split-ffi-core

Conversation

@DavidS

@DavidS DavidS commented Oct 11, 2017

Copy link
Copy Markdown
Contributor

This splits off all of the pure ruby code into a childprocesscore gem that doesn't depend on FFI, thereby allowing users to trade off between full-featured, and always-deployable by depending on the right gem. It is even possible for grandchildren to add the childprocess and FFI gems on top of something using only childprocesscore to enable posix_spawn, or other platform support.

This still needs updates to the README, but I wanted to get early feedback on the details of the approach.

Fixes #124

@DavidS

DavidS commented Oct 11, 2017

Copy link
Copy Markdown
Contributor Author

Another thing that needs fixed is merging coverage reports from the two parts. After some googling, the best I found was switching to codecov.io instead, which does automerging of ALL reports sent in, so it could also ingest the reports from non-default rubies, and windows into a single result.

I've played around with it a little bit at another project this morning: https://codecov.io/gh/DavidS/puppet-resource_api/commit/53af7f92b209d07bb1cac838bd9d33e06906d103/build

@coveralls

coveralls commented Oct 11, 2017 •

Copy link
Copy Markdown

Coverage Status

Coverage decreased (-4.4%) to 90.485% when pulling dc75dc9 on DavidS:split-ffi-core into 0bce27d on enkessler:master.

1 similar comment
@coveralls

coveralls commented Oct 11, 2017 •

Copy link
Copy Markdown

Coverage Status

Coverage decreased (-4.4%) to 90.485% when pulling dc75dc9 on DavidS:split-ffi-core into 0bce27d on enkessler:master.

@DavidS
DavidS force-pushed the split-ffi-core branch 2 times, most recently from bc882e6 to 480028b Compare October 13, 2017 12:52
@coveralls

coveralls commented Oct 13, 2017 •

Copy link
Copy Markdown

Coverage Status

Coverage decreased (-4.5%) to 90.388% when pulling bc882e6 on DavidS:split-ffi-core into 0bce27d on enkessler:master.

4 similar comments
@coveralls

Copy link
Copy Markdown

Coverage Status

Coverage decreased (-4.5%) to 90.388% when pulling bc882e6 on DavidS:split-ffi-core into 0bce27d on enkessler:master.

@coveralls

Copy link
Copy Markdown

Coverage Status

Coverage decreased (-4.5%) to 90.388% when pulling bc882e6 on DavidS:split-ffi-core into 0bce27d on enkessler:master.

@coveralls

Copy link
Copy Markdown

Coverage Status

Coverage decreased (-4.5%) to 90.388% when pulling bc882e6 on DavidS:split-ffi-core into 0bce27d on enkessler:master.

@coveralls

Copy link
Copy Markdown

Coverage Status

Coverage decreased (-4.5%) to 90.388% when pulling bc882e6 on DavidS:split-ffi-core into 0bce27d on enkessler:master.

@coveralls

coveralls commented Oct 13, 2017 •

Copy link
Copy Markdown

Coverage Status

Coverage decreased (-4.5%) to 90.379% when pulling 480028b on DavidS:split-ffi-core into 0bce27d on enkessler:master.

3 similar comments
@coveralls

coveralls commented Oct 13, 2017 •

Copy link
Copy Markdown

Coverage Status

Coverage decreased (-4.5%) to 90.379% when pulling 480028b on DavidS:split-ffi-core into 0bce27d on enkessler:master.

@coveralls

coveralls commented Oct 13, 2017 •

Copy link
Copy Markdown

Coverage Status

Coverage decreased (-4.5%) to 90.379% when pulling 480028b on DavidS:split-ffi-core into 0bce27d on enkessler:master.

@coveralls

coveralls commented Oct 13, 2017 •

Copy link
Copy Markdown

Coverage Status

Coverage decreased (-4.5%) to 90.379% when pulling 480028b on DavidS:split-ffi-core into 0bce27d on enkessler:master.

@codecov-io

codecov-io commented Oct 24, 2017 •

Copy link
Copy Markdown

Codecov Report

❗ No coverage uploaded for pull request base (dev@5fb308d). Click here to learn what that means.
The diff coverage is n/a.

Impacted file tree graph

@@          Coverage Diff           @@
##             dev     #127   +/-   ##
======================================
  Coverage       ?   83.13%           
======================================
  Files          ?       29           
  Lines          ?     1755           
  Branches       ?        0           
======================================
  Hits           ?     1459           
  Misses         ?      296           
  Partials       ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5fb308d...59fe72a. Read the comment docs.

@coveralls

coveralls commented Oct 25, 2017 •

Copy link
Copy Markdown

Coverage Status

Coverage decreased (-4.3%) to 90.541% when pulling aebb493 on DavidS:split-ffi-core into 0bce27d on enkessler:master.

@enkessler

Copy link
Copy Markdown
Owner

Ah. Now it makes more sense what you wanted to do. Admittedly, I only mostly followed what your plan was when you opened the issue.

I'll take a better look later this week but, at a glance, it'd be nice if the two top level project folders were named the same as their gem (childprocess and childprocess-core) instead of main and core, respectively. Then have their libs shaped in the conventional fashion.

Also, please base the PR off of the development branch (dev) instead of the master branch.

I appreciate all of the effort!

@DavidS

DavidS commented Oct 25, 2017

Copy link
Copy Markdown
Contributor Author

I've renamed the top-level directories, and rebased everything. That was the easy part.

I also tried to rename the ChildProcess module in core. This would split the exiting public API on the ChildProcess module across namespaces, which will have a big impact on downstream users. While I tried it out, I also found pieces where it's not really clear that the way it's setup currently makes sense in a multi-gem world. E.g. https://github.com/enkessler/childprocess/blob/master/lib/childprocess.rb#L203-L205 might need more work.

Looking at that, I wonder if splitting it even more into childprocess-core, childprocess-unix, childprocess-posix_spawn, childprocess-windows, childprocess-jruby, and childprocess. Where all the library/glue code goes into -core, each of the "drives" resides in its own "plugin", and the main gem only ties the dependencies together. For the simple case, it would still remain a dependency on childprocess, and a single require, that pulls in everything. Maybe I'm overthinking this :-D

@DavidS
DavidS changed the base branch from master to dev October 25, 2017 13:24
@coveralls

coveralls commented Oct 25, 2017 •

Copy link
Copy Markdown

Coverage Status

Coverage decreased (-15.0%) to 79.899% when pulling e7ea7a7 on DavidS:split-ffi-core into 0bce27d on enkessler:master.

2 similar comments
@coveralls

coveralls commented Oct 25, 2017 •

Copy link
Copy Markdown

Coverage Status

Coverage decreased (-15.0%) to 79.899% when pulling e7ea7a7 on DavidS:split-ffi-core into 0bce27d on enkessler:master.

@coveralls

coveralls commented Oct 25, 2017 •

Copy link
Copy Markdown

Coverage Status

Coverage decreased (-15.0%) to 79.899% when pulling e7ea7a7 on DavidS:split-ffi-core into 0bce27d on enkessler:master.

The final goal here is a separate childprocesscore gem that contains
all ruby-only code, specifically not FFI). posix_spawn, windows support,
the related tooling, and the ffi dependency stays in the main gem. This
way people used to the existing ways can keep trucking along without
disturbance, while people who want to avoid FFI can depend on just
childprocesscore.

This is only the first patch in the series, to keep the code patches
cleaner.
This could be done better by moving more of the machinery to main, but
it'll serve for now.
@coveralls

coveralls commented Oct 27, 2017 •

Copy link
Copy Markdown

Coverage Status

Coverage decreased (-15.04%) to 79.842% when pulling 59fe72a on DavidS:split-ffi-core into 5fb308d on enkessler:dev.

9 similar comments
@coveralls

Copy link
Copy Markdown

Coverage Status

Coverage decreased (-15.04%) to 79.842% when pulling 59fe72a on DavidS:split-ffi-core into 5fb308d on enkessler:dev.

@coveralls

Copy link
Copy Markdown

Coverage Status

Coverage decreased (-15.04%) to 79.842% when pulling 59fe72a on DavidS:split-ffi-core into 5fb308d on enkessler:dev.

@coveralls

Copy link
Copy Markdown

Coverage Status

Coverage decreased (-15.04%) to 79.842% when pulling 59fe72a on DavidS:split-ffi-core into 5fb308d on enkessler:dev.

@coveralls

Copy link
Copy Markdown

Coverage Status

Coverage decreased (-15.04%) to 79.842% when pulling 59fe72a on DavidS:split-ffi-core into 5fb308d on enkessler:dev.

@coveralls

Copy link
Copy Markdown

Coverage Status

Coverage decreased (-15.04%) to 79.842% when pulling 59fe72a on DavidS:split-ffi-core into 5fb308d on enkessler:dev.

@coveralls

Copy link
Copy Markdown

Coverage Status

Coverage decreased (-15.04%) to 79.842% when pulling 59fe72a on DavidS:split-ffi-core into 5fb308d on enkessler:dev.

@coveralls

Copy link
Copy Markdown

Coverage Status

Coverage decreased (-15.04%) to 79.842% when pulling 59fe72a on DavidS:split-ffi-core into 5fb308d on enkessler:dev.

@coveralls

Copy link
Copy Markdown

Coverage Status

Coverage decreased (-15.04%) to 79.842% when pulling 59fe72a on DavidS:split-ffi-core into 5fb308d on enkessler:dev.

@coveralls

Copy link
Copy Markdown

Coverage Status

Coverage decreased (-15.04%) to 79.842% when pulling 59fe72a on DavidS:split-ffi-core into 5fb308d on enkessler:dev.

@DavidS

DavidS commented Oct 27, 2017

Copy link
Copy Markdown
Contributor Author

Something weird is still ongoing with the codcov reports (see https://codecov.io/gh/DavidS/childprocess/compare/aebb49369bcebc401f0c59af8e336d47a18944c1...59fe72a87c0264ab724f21c1f27d56971fa25434/changes), but I'll chalk that up to weirdness around the changes in the repos.

And appveyor failed on SSL when trying to upload reports. Everything else shows up on https://codecov.io/gh/DavidS/childprocess/commit/59fe72a87c0264ab724f21c1f27d56971fa25434/build

@enkessler

enkessler commented Nov 5, 2017 •

Copy link
Copy Markdown
Owner

@DavidS The more I think about it, the more that I just want the correct version to be downloaded automatically. If they are on Windows, then they get the windows version that will have FFI as a dependency and if they are not then they get the other version.

This looks relevant: http://guides.rubygems.org/specification-reference/#platform=

It would require us to build two versions of the gem but that can be a Rake task and the only difference would be the dependencies in the gemspec and a conditional around the FFI parts in the library code.

@DavidS

DavidS commented Nov 6, 2017

Copy link
Copy Markdown
Contributor Author

@enkessler

Copy link
Copy Markdown
Owner

Possibly, yes. I take it that that is a script that is kicked off during the gem installation process that conditionally triggers additional gem installations? And it looks like you are already using it, so it presumably works.

Out of curiosity, what does the console output look like for the two different installation paths?

Also, I see that this was one of your first suggestions before but I glossed over it and focused on the alternative because you said

In this case downstream users of childprocess would have to make sure that FFI is available, if they want to use posix_spawn. It is a relatively simple change on childprocess itself, but might not be easily discoverable by developers, and adds more hassle both when upgrading, and in future maintenance.

Can you elaborate on that? What do you mean by 'available'? Won't it be downloaded the same as any other gem just, you know, conditionally?

@DavidS

DavidS commented Nov 7, 2017

Copy link
Copy Markdown
Contributor Author

The code I linked installs a package conditionally on a platform.

If we choose to "install FFI only on windows", FFI would not be available by default for posix_spawn.

@DavidS

DavidS commented Feb 21, 2018

Copy link
Copy Markdown
Contributor Author

Closing this in favour of #132

@DavidS DavidS closed this Feb 21, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants