C++20 Modules: Practical Insights, Status and TODOs

36 pointsposted 3 days ago
by ashvardanian

12 Comments

ho_schi

41 minutes ago

Looking at the examples from using Clang (I use GCC btw.):

   clang++ -std=c++20 Hello.cppm --precompile -o Hello.pcm
   clang++ -std=c++20 use.cpp -fmodule-file=Hello=Hello.pcm Hello.pcm -o Hello.out
   ./Hello.out
Why is something which shall makes things easy and secure so complicated?

I'm used to:

    g++ -o hello hello.cpp

It can use headers. Or doesn't use headers. I doesn't matter. That's the decision of the source file. To be fair, the option -std=c++20 probably isn't necessary in future.

I recommend skimming over this issue from Meson:

https://github.com/mesonbuild/meson/issues/5024

Reading the last few blog posts from a developer of Meson, providing some insights why Meson doesn't support modules until now:

https://nibblestew.blogspot.com/

delta_p_delta_x

28 minutes ago

> Why is something which shall makes things easy and secure so complicated? > I'm used to: > > g++ -o hello hello.cpp

That is simple, because C++ inherited C's simplistic, primitive, and unsafe compilation and abstraction model of brute-force textual inclusion. When you scale this to a large project with hundreds of thousands of translation units, every command-line invocation becomes a huge list of flag soup that plain Makefiles become intractable.

Almost all other reasonably-recent programming languages have all of the following:

- a strong coupling of dependency management, building, installation, and publishing tools

- some description of a directed acyclic graph of dependencies, whether it be requirements.txt, cargo.toml, Maven, dotnet and Nuget .csproj files, Go modules, OPAM, PowerShell gallery, and more

- some way to describe the dependencies within the source code itself

C++20 modules are a very good thing, and enforce a stronger coupling between compiler and build tool; it's no longer just some weekend project chucking flags at g++/clang++/cl.exe but analysing source code, realising it needs a, b, c, x, y, z modules, ensuring those modules are built and export the necessary symbols, and then compiling the source at hand correctly. That is what `clang-scan-deps` does: https://clang.llvm.org/docs/StandardCPlusPlusModules.html#di...

I concede there are two problems with C++20 modules: we didn't have a working and correct compiler implementation before the paper was accepted into C++20, and secondly, the built/binary module interface specification is not fixed, so BMIs aren't (yet) portable across compilers.

The Meson developer is notorious for stirring the pot with respect to both the build system competition, and C++20 modules. The Reddit thread on his latest blog post provides a searing criticism for why he is badly mistaken: https://www.reddit.com/r/cpp/comments/1n53mpl/we_need_to_ser...

bluGill

34 minutes ago

Because hello is so simple you don't need complicated. When you are doing something complicated though you have to accept that it is complicated. I could concatenate all 20 million lines of C++ (round number) I work with into one file and building would be as simple as your hello example - but that simple building comes at great cost (you try working with a 20 million line file, and then merging it with changes someone else made) and so I'm willing to accept more complex builds.

triknomeister

3 hours ago

> Yes, we can. C++20 Modules are usable in a Linux + Clang environment. There are also examples showing that C++20 Modules are usable in a Windows environment with MSVC. I have not yet heard of GCC’s C++20 Modules being used in non-trivial projects.

People keep saying this and yet I do not know of a good example from a real life project which did this which I can test. This seems very much still an experimental thing.

juliangmp

39 minutes ago

I think it still is in a "well technically its possible" state. And I fear it'll remain that way for a bit longer.

A while ago I made a small example to test how it would work in an actual project and that uses cmake (https://codeberg.org/JulianGmp/cpp-modules-cmake-example). And while it works™, you can't use any compiler provided modules or header modules. Which means that 1) so you'll need includes for anything from the standard library, no import std 2) you'll also need includes for any third party library you want to use

When I started a new project recently I was considering going with modules, but in the end I chose against it because I dont want to mix modules and includes in one project.

bluGill

an hour ago

It is just beyond experimental now, and finally in the early adopter phase. Those early adopters are trying things are trying to develop best practices - which is to say as always: they will be trying things that future us will laugh at how stupid it was to do.

There are still some features that are missing from compilers, but enough is there that you can target all 3 major compilers and still get most of modules and benefit from them. However if you do this remember you are an early adopter and you need to be prepared to figure out the right way to do things - including fixing things that you get wrong once you figure out what is right.

Also, if you are writing a library you cannot benefit from modules unless you are willing to force all your consumers to adopt modules. This is not reasonable for major libraries used by many so they will be waiting until more projects adopt modules.

Still modules need early adopters and they show great promise. If you write C++ you should spend a little time playing with them in your current project even if you can't commit anything.

Kelteseth

2 hours ago

For me it is simply because Qt moc does not support modules yet. This is my last comment from the tracking issue https://bugreports.qt.io/browse/QTBUG-86697

> C++ 26 reflections have now been voted in. This would get rid of moc entirely, but I really do not see how this will become widely available in the next 5-10 Years+. This would require Qt to move to C++ 26, but only if compiler support is complete for all 3 compilers AND older Linux distros that ship these compilers. For example, MSVC still has no native C++ 23 flag (In CMake does get internally altered to C++ latest aka. C++ 26) , because they told me that they will only enable it is considered 100% stable. So I guess we need to add modules support into moc now, waiting another 10 years is not an option for me .

menaerus

3 hours ago

> The data I have obtained from practice ranges from 25% to 45%, excluding the build time of third-party libraries, including the standard library.

> Online, this number varies widely. The most exaggerated figure I recall is a 26x improvement in project compilation speed after a module-based refactoring.

> Furthermore, if a project uses extensive template metaprogramming and stores constexpr variable values in Modules, the compilation speed can easily increase by thousands of times, though we generally do not discuss such cases.

> Apart from these more extreme claims, most reports on C++20 Modules compilation speed improvements are between 10% and 50%.

I'd like to see references to those claims and experiments, size of the codebase etc. I find it hard to believe the figures since the bottleneck in large codebases is not a compute, e.g. headers preprocessing, but it's a memory bandwidth.

unddoch

2 hours ago

> I'd like to see references to those claims and experiments, size of the codebase etc. I find it hard to believe the figures since the bottleneck in large codebases is not a compute, e.g. headers preprocessing, but it's a memory bandwidth.

Edit: I think I misunderstood what you meant by memory bandwidth at first? Modules reduce the amount of work being done by the compiler in parsing and interpreting C++ code (think constexpr). Even if your compilation infrastructure is constrained by RAM access, modules replace a compute+RAM heavy part with a trivial amount of loading a module into compiler memory so it's a win.

lingolango

2 hours ago

>since the bottleneck in large codebases is not a compute, e.g. headers preprocessing, but it's a memory bandwidth.

SSD bandwidth: 4-10GB/s RAM bandwidth: 5-10x that, say 40GB/s.

If compute was not a bottleneck, the entire linux kernel should compile in less than 1 second.

skeezyboy

2 hours ago

> I find it hard to believe the figures since the bottleneck in large codebases is not a compute, e.g. headers preprocessing, but it's a memory bandwidth.

source? language? what exactly does memory bandwidth have to do with compilation times in your example?