massive compile time using g++

[andi2011]

Hi,

I'm compiling a program that uses rxcpp with g++.
Its taking a really long time to compile!
Is this common?
Can I do anything to speed up compilation?

Greets,
Andreas

[kirkshoop]

Improvements in compile-time would require a series of changes to rxcpp that would convert existing meta programming patterns into faster ones.

@odinthenerd gave a presentation at cppnow PDF, YouTube that contains a slide that ranks the compile-time perf of various meta programming features.

Moving rxcpp to use cheaper meta features would take time and might require breaking compatibility with VS2013 and perhaps require changing from c++11 compatibility to c++14 compatibility.

Moving to C++14 would allow so many improvements to the implementation in general that I would expect that to be a rewrite. I wrote a prototype already and there is an active issue #347 to track development and eventual migration to this new implementation.

[odinthenerd]

you summoned ;) VS2013 is super picky about complex aliases so working with it is a total nightmare. I don't believe C++14 is really necessary, C++11 provides enough.

I skimmed a bit but not long enough to really understand what the common case is, if you could give a tip which part of the code is going to be compiled the most I can give more useful tips.

https://github.com/Reactive-Extensions/RxCpp/blob/53eac3736ae9587a91e33746ace1eb3641068e8a/Rx/v2/src/rxcpp/rx-coordination.hpp#L77 is pretty low hanging fruit. Here all three predicates are evaluated even if the first is true, one could rewrite it so that its short circuiting.

These could also be written as pure aliases https://github.com/Reactive-Extensions/RxCpp/blob/53eac3736ae9587a91e33746ace1eb3641068e8a/Rx/v2/src/rxcpp/rx-coordination.hpp#L53 this saves a type instantiation which is a big constant savings but cost a little more each call with the same type because aliases are not memoized so its a question of use pattern.

Hope this was helpfull.

[kirkshoop]

Hi, @odinthenerd!

Thanks for the pointers, that would be a great place to start, since it is fairly isolated.

This library is similar to range v3 in that it is a collection of algorithms that are composable using operator|. These algorithms often take lambdas and avoid type-forgetting. This results in very long types with many ephemeral types built to choose an implementation or to constrain or to derive one type from another. I have assumed that the long types will require compiler improvements. I expect that there is a lot of room for improvement in how the types are constructed.

The code was made more complex for having been started some years ago and targeting multiple compilers that had more limitations than those available today. Also, C++ 14 auto lambdas would avoid so many manually built structs with operator()().

I suspect that patterns like this tag check and this signature check that are repeated a lot and then applied often contribute to the compile-time.

I suspect that the make_subscriber and make_observer overloads are expensive to process.

I suspect that the contract enforcement of these observer methods by each subscriber is costly.

However, when I have tried to experiment with void_t and other ways of speeding up compile-time, I have been thwarted by VS2013 and have never seen any change make a dent in the compile-time.

[odinthenerd]

yes the make_subscriber and make_observer overloads are probably super expensive, the disadvantage to managing overload resolution like that is that the compiler has to instantiate most of the overloads for any particular argument type set because only with a complete set of overload candidates can it check for ambiguities etc. I would expect changing that to tag dispatch rather than SFINAE should speed things up considerably.

[gsauthof]

Just an example how long is a long time: I just executed the doc target with ninja - which is a very efficient build tool and does parallel execution, by default. For generating the docs, some stuff is compiled, i.e. 73 translation units.

On this on 4 core Skylake CPU machine the compiling took 11 minutes at 354 % CPU usage.

[kirkshoop]

Yes. I expect that if Rxcpp drops support for msvc 2013 and moves to c++14, replacing the existing traits and sfinae overloading with the concepts compatibility macros @eniebler built for pushmi and range-v3 would result in big improvements in compilation speed.

[serg06]

Faster compilation times would be amazing.

Personal anecdote, we want to use RxCpp more in our project (VS 2019 / C++17) but the compilation time is holding us back.