#1 2014-10-07 18:06:02

WayneBailey
Member
Registered: 2013-06-12
Posts: 15

Building service/client modules with different, incompatible compilers

Please correct me if I’m wrong. It appears Cpp Micro Services requires all modules to be built with the same compiler or at least compatible ones. If that’s so, have you considered using something like CppComponents ( https://github.com/jbandela/cppcomponents ) to get around that limitation?

Offline

#2 2014-10-07 18:36:21

sascha
Administrator
Registered: 2012-05-04
Posts: 46

Re: Building service/client modules with different, incompatible compilers

I indeed looked through the slides of the C++Now 2014 presentation about CppComponents recently.

You are right that there is no special mechanism in CppMicroServices to achieve some level of improved cross-compiler ABI support. To overcome that, we thought about several different approaches:

  • Just compile everything with the same compiler and flags

  • Use remote services (using IPC or sockets)

  • Decouple clients from the CppMicroServices API using declarative services and use

    • header only service interfaces

    • or C-style service interfaces (structs)

    • or CppComponents based service interfaces

  • Use a hour-glass API design for the CppMicroServices API and interfaces (my personal favourite currently)

Regarding CppComponents, here are my personal thoughts:

  • It is amazing what you can do with plain C++(11) ;-)

  • It depends on C++11 while CppMicroServices is quite conservative and still supports VS 2008 SP1

  • It seems to be quite intrusive on the service interface declaration (haven't looked into that much)

  • There is a proposal about a standardized C++ ABI (subset) by Herb Sutter - this would help a lot but will come with C++17 at the earliest if at all

Btw, the new resources system will allow to embed a compiler specific version string and I am thinking about a light-weight verification mechanism when doing a ModuleContext::Install(...) call in the future. Apart from that, we have always required people to use the same compilers for distributed/partial builds or they are just building everything themselves any way. While obviously not elegant, it is "the way it is" in C++ land.

If you are experimenting with CppComponents, please keep me up-to-date. Any thoughts about advantages/disadvantages of one approach over an other is very welcome as well.

- Sascha

Offline

#3 2014-10-08 13:31:39

WayneBailey
Member
Registered: 2013-06-12
Posts: 15

Re: Building service/client modules with different, incompatible compilers

The status quo is adequate for our current needs; however, having plugin compatibility across compilers offers unique opportunities.

Herb’s proposal looks quite promising, but as you point out, it’ll be many years before we see it in practice. In the meantime, John’s CppComponents demonstrates there are intermediate alternatives. My major concern about CppComponents is the overhead of runtime mediation across compiler interfaces. This is eliminated by Herb’s proposal. But as Herb points out his solution also has performance and flexibility costs. Thus he proposes C++ ABIs be specified rather than being the default. 

If CppMicroServices is to support cross-compiler plugins, it makes sense (as you point out) to have a mechanism to opt into this feature for performance and backward compatibility reasons. Although remote services could be used, it seems to be overkill for this purpose. Can you elaborate on how declarative services would be used to achieve this?

As you point out, CppComponents intrudes on the CppMicroServices service interface declaration. I believe this is because you’re both addressing the same problem, and it appears you’re both using the same design pattern. Which means there’s an opportunity to combine the two. However, such a blending would not be at a superficial level. You may wish to consider collaborating with John in this regard. He indicated a desire to implement a “package manager” (which you have) and you’d like cross-compiler plugins (which he has). Just a thought.

If CppMicroServices had cross-compiler plugin support that could be selectively “switched” on/off, then a light-weight verification mechanism to detect compiler version could be used to determine when to use this feature. Thus making this support transparent.

-Wayne

Offline

#4 2014-10-09 13:34:00

WayneBailey
Member
Registered: 2013-06-12
Posts: 15

Re: Building service/client modules with different, incompatible compilers

Oh, by the way, there are alternatives to CppComponents such as the Hourglass Pattern recently presented at CppCon. See slides at http://www.slideshare.net/StefanusDuToi … for-c-apis and interesting comments at https://news.ycombinator.com/item?id=8308389 (note John's plug for his CppComponents).

This approach is more lightweight than CppComponents, but also more restrictive.

-Wayne

Offline

#5 2014-10-24 00:03:29

sascha
Administrator
Registered: 2012-05-04
Posts: 46

Re: Building service/client modules with different, incompatible compilers

Thanks a lot for your input. While it is not on my short-term todo list, I will keep monitoring the CppComponents project and eventually contact its developer. The hourglass pattern you mentioned also looks promising to me (it actually was in my original list of possible approaches).

The approach with declarative services + C-style service interfaces (plus maybe a convenience header-only C++ interface wrapping the C-style interface) could work in a restricted scenario where modules communicate exclusively via service interfaces and do not make any CppMicroServices API calls at all. The declarative services module itself would take care of the service wiring by reading the xml files from the modules and taking care of the CppMicroServices API calls on behalf of the modules. Hence such modules would not have any linker dependencies to the CppMicroServices library (at least not to its C++ API). Dependencies to other modules would only involve the C API for the C-style service interface. Therefore no ABI incompatibilities would exist. Currently, the module initialization code would still force a linker dependency on the CppMicroServices library, but this is going to change soon. So far the theory...

In practice I would still be concerned about different runtimes needed by modules compiled with different compilers. E.g. on Windows, one would get dependencies to different versions of msvcrt.dll, msvcr*.dll, etc. libraries which use different heaps. Correct memory management across module boundaries would have to be enforced (at least in the service interfaces).

Offline

#6 2014-10-24 13:49:19

WayneBailey
Member
Registered: 2013-06-12
Posts: 15

Re: Building service/client modules with different, incompatible compilers

Sorry, I missed your reference to the hour-glass pattern in your first post.

Plugins using different CRTs isn’t a problem per say. Vcredist can be used to distribute the run-time needed by a plugin and an manifest directive embedded into the plugin can be used to ensure the correct run-time is used. The problem is that global resources are not managed consistently across run-times which heap management is one. This problem is particular to Windows and Microsoft refuses to address it (see http://connect.microsoft.com/VisualStud … -libraries ). Even if Herb’s proposal is implemented, this will continue to be a problem. Minimally plugin developers need to be made aware of the specifics regarding this issue. However, mitigating this problem within CppMicroService seems a daunting task.

Using Declarative services (as you describe) appears a good choice to handle the ABI issue. This would hide the underlying implementation from the declared modules, and allow transparent improvements in the ABI solution as technologies evolve (e.g. Herb’s proposal). Obviously there would need to be a way for a module to declare that it’s implementing a C ABI (e.g. a C-ABI property).  I’m a bit concerned that Declarative services may become a performance bottleneck.

-Wayne

Offline

Board footer

Powered by FluxBB