mbed-cli can clone mbed-os.lib more than once

[wdwalker]

As a creator of an mbed library that people can add to their projects, I want to be able to build and test my library against mbed-os. To do so, I put an mbed-os.lib file in my top level directory so that mbed-cli will clone mbed-os when I do an "mbed import" of my library.

However, the presence of the mbed-os.lib file in my library wreaks havoc on projects that import my library. The reason is that mbed-cli seems to recurse through directories looking for *.lib files. So, the presence of mbed-os.lib in my library will end up with mbed-os being cloned twice by an app that uses my library: once for the app's own mbed-os.lib and again for my libraries mbed-os.lib.

I'm not sure how to fix this. I'm looking at using the solution outlined in http://stackoverflow.com/questions/229186/os-walk-without-digging-into-directories-below. I've coded this up but haven't tested it yet. If I can get this to work, and the mbed-cli maintainers agree this is a valid request, I will do a pull request.

[wdwalker]

BTW - the workaround is to not put an mbed-os.lib in my library's repo. Instead, I do a git clone of the repo and then do an "mbed add mbed-os" inside it whenever I want to work on it. Or, I do an mbed import of an app that uses my library and then cd into the clone mbed-cli made of library and do a "git checkout develop" and work on the directory from there. Both are kind of clunky solutions.
[Mirrored to Jira]

[bridadan]

I do a git clone of the repo and then do an "mbed add mbed-os" inside it whenever I want to work on it. Or, I do an mbed import of an app that uses my library

This is pretty much what we do (usually the latter) and it seems to generally work, if being a bit clunky.

If you have thoughts on alternative ways of doing this, I think we'd definitely be interested in hearing them!
[Mirrored to Jira]

[wdwalker]

@bridadan - I have an approach in the pull request associated with this issue, and I also suggest a more generic approach that may be possible. Please let me know what you think.
[Mirrored to Jira]

[betzw]

Apart mbed-os, what is happening when several libraries depend on other libraries?
[Mirrored to Jira]

[wdwalker]

IMO, that's a very good question and is something I allude to in #474 (comment). This is a problem that will surface in time, and is common across many platforms (golang's 'vendoring' quagmire, autotools, etc.).
[Mirrored to Jira]

[wdwalker]

I'm still struggling with this. The problem I face is that the current implementation seems to make it very cumbersome to develop and test a library. My directory structure is similar to this:

example_program
|- my-library (separate git repo added via 'mbed add')
`- TESTS
|- mbed-os
|- main.cpp
|- other-stuff.h

What I want to do is be able to simultaneously develop example_program and my-library. I can get pretty far if I use example_program as my primary development bed, but a problem arises when I want to run the TESTS in my-library via mbed test. The only way I've been able to run those TESTS is by having mbed-os added in my-library. But, if I do that in the structure above, compiling example_program results in multiple references to mbed-os (one for the mbed-os in example_program and another for the mbed-os in my-library).

So, what I do now is very very cumbersome. I check out my-library in two different directories: one under example_program and one in its own directory. I do all my-library development in the directory under example_program. Once I'm satisfied, I then create a patch and apply it to the my-library that is in its own directory so I can run mbed add mbed-os and mbed test there. Clunky.

Perhaps I'm missing some easier way to run the TESTS of my-library when my-library is checked out under example_program. If you know the magic incantations to do that, you'd solve the core problem that caused me to file this issue.

Any thoughts?
[Mirrored to Jira]

[betzw]

@screamerbg, @nikapov-ST,

we are actually facing this issue with a library (lets call it libA) (different from mbed-os) on which two other libraries (libB1 & libB2) depend on. As expected the compilation, or rather the build process is failing due to the double inclusion of library libA. This is happening also when using the mbed online compiler!!!
Pls. give some advise on how to proceed.

Note: we cannot remove the dependencies in libB1 and libB2, as the user when writing an application just knows about these libraries and as he might even write an application which requires only one of them!

[Mirrored to Jira]

[wdwalker]

@betzw - I proposed a possible approach in my comment at #474 (comment). The problem here is very similar to the 'vendoring' problem the golang folks have yet to solve gracefully: you're going to run into this problem in any system that automatically fetches dependencies, especially when it fetches them recursively.
[Mirrored to Jira]

[seppestas]

I'm facing a similar issue. My approach is to add an EXAMPLE folder in the root of the library and add EXAMPLE/* to the .mbedignore file.

Inside of the EXAMPLE folder is my mbed-os.lib file and a symlink to the library (e.g ln -s .. my-library). This way I can build and test the example code that relies on the library (yes, I realize that this creates a symbolic link loop, but this is what mbed forced me to do). It has the added benefit of providing up-to-date documentation to consumers of the library.

However, when using mbed add my-libary, the mbed cli automatically clones the mbed-os libary in the example, causing the add command to take way longer than necessary if no cache was set up. I think a nice solution would be to have mbed add respect the .mbedignore file.
[Mirrored to Jira]

[bmcdonnell-ionx]

@wdwalker,

BTW - the workaround is to not put an mbed-os.lib in my library's repo. Instead, ... I do an mbed import of an app that uses my library and then cd into the clone mbed-cli made of library and do a "git checkout develop" and work on the directory from there.

How about you make a "canonical" test app (as a peer repo to your lib), which includes both your lib and mbed-os as libraries?
[Mirrored to Jira]

[devangel77b]

It doesn't look like there's a good solution to this yet? I have been doing the workaround where I have a different repository for testing the library I am working on but it would be much better if
mbed add https://github.com/me/mylibrary
could "add it as a library" and ignore mbed-os and main.cpp
versus
mbed import https://github.com/me/mylibary
could "import it as a program" (like the --help seems to imply)

I have had the same odd behavior where it puts a copy of mbed-os inside the library directory when added. I also have the odd behavior if bot library has dependencies spektrum and nmea, and if they are used elsewhere it is not smart enough to realize that and clones it again.

[ciarmcom]

Thank you for raising this issue. Please note we have updated our policies and
now only defects should be raised directly in GitHub. Going forward questions and
enhancements will be considered in our forums, https://forums.mbed.com/ . If this
issue is still relevant please re-raise it there.
This GitHub issue will now be closed.