Advanced CmakeLists Parser/Lexer, Yaml config

[roscopecoltran]

Hi,

Hope you are all well !

I tried to install and use cmakepp with hunter in order to add hunter_add_package on the fly when a find_package command is referenced in a project.

Goal/Background:

My goal is to create a small utility, that I will write in golang for ease and cross-platform use, allowing to parse CMakeLists.txt and export,import package dependencies with yaml/json input file. It would help me to bundle lots of projects, cmake based, to a Docker container, alpine based (for those not familiar Docker, it will just reduce the size of containers, making it more easy to deploy in production). Often, I would to extract all dependencies, and pick a specific version and build them from sources.

Summary: Manage cmakelists package dependencies with commit-ids and vcs, and be able to manage cmake args through a yaml file.

My little script can either write/generate *.cmake file from a yaml config, then parse the content from cmake with json-cmake (link) and loop over the dependencies to find or add with the ExternalProject to build a project. As a preliminary step, I would need to extract some informations, or the whole ast from cmake based project in order to build this configuration file.

My inspiration comes from these golang utilities:

• Glide: Manage package dependencies with versioning, and multi-vcs. Official Repository (link), Yaml config file example (link)
• nut: Manage macros and bundle a project into a docker container. Official Repository (link), Yaml config example (link)

Real case:

For example, this dockerfile as lot of packages provided by apt package manager. So I would like to build them from source but be able to checkout a specifc version from a specifc vcs and iterate through a macro.

In a nutshell, I wanted to pre-generate, with a kind of wizard (like with glide in golang) the extraction of the cmake ast, then to extract some config nodes related to commands like: find_package, find_library, add_target, add_library...

Where is my bottleneck ? :-)
So it requires to get the precise mapping of arguments names, values and properties. But CmakeLists does not have uniform ways to provide command flags or args ....the order of some commands can be different, present or missing...

examples:

• Custom functions/macros:
  • https://github.com/headupinclouds/gatherer/blob/develop/CMakeLists.txt#L66
• Packages,Components,Version
  • https://github.com/halismai/bpvo/blob/master/CMakeLists.txt#L124
  • https://github.com/halismai/bpvo/blob/master/CMakeLists.txt#L136

I was thinking about a CMake lexer (reference example), and to write an ast tree, but I would like to do more with the CMakeLists.txt so that's how I found your interesting project.

Question:

Is it possible to manipulate, parse CMakeLists.txt with cmakepp ? Is there any example that would help me to start some tests or development quickly ? ps. I could not make work all examples 3 last examples.

Hope I was clear enough with this lengthy explanation ^^

Have all a great day ! And thanks for any inputs or any recommendations in order to achive this smart cmake lexer.

Cheers,
Richard

[toeb]

Hi Richard,

it is possible to manipulate cmake lists with cmakepp... however cmakepp is a pure cmake library. ie it could be hard to incorporate it and it is slow compared to doing this natively in c++ for example.

creating a lexer and parser for cmake yourself is however very simple because cmake script is a very simple language - this is also what i would suggest.

Here is the complete language definition: https://cmake.org/cmake/help/v3.0/manual/cmake-language.7.html#syntax

possibly you could even use a parser like antlr or boost::spirit to create the asts simply.

you can look at how i did it in cmakepp eventhough its very specific.
https://github.com/toeb/cmakepp/blob/master/cmake/cmake/reflection/parsing/cmake_tokens_parse.cmake

here are the regular expressions used: https://github.com/toeb/cmakepp/blob/c597ef81cfa3c09cecc4129600b7ed082900cc7d/cmake/regex/regex_cmake.cmake