Using ccache with clang 5 causes different results from just clang5

So with this contrived bit of code - compiling with clang works just fine, but when ccache is used extra warnings/errors appear - I thought ccache is supposed to transparently pass these through. This is ccache 3.1.6 on CentOS 6 from epel repo - upgrading is not an option as this is production environment.

#include <byteswap.h>

int main()
{
    int i = 42;
    auto j = bswap_32(i);
    (void)j;

    return 0;
}

So example 1 with unused include path gives no errors:

clang++ -Wno-c++98-compat -Wall -Werror -std=c++17 -I/usr/local/include -c ccache.cpp

But with ccache I get:

ccache clang++ -Wno-c++98-compat -Wall -Werror -std=c++17 -I/usr/include/xmlib -c ccache.cpp 
clang-5.0: error: argument unused during compilation: '-I /usr/include/xmlib' [-Werror,-Wunused-command-line-argument]

Example 2 without the extra include works just fine:

clang++ -Wno-c++98-compat -Wall -Werror -std=c++17  -c ccache.cpp

And with ccache

cache clang++ -Wno-c++98-compat -Wall -Werror -std=c++17  -c ccache.cpp 
ccache.cpp:6:32: error: ISO C++17 does not allow 'register' storage class specifier [-Wregister]
    auto j = (__extension__ ({ register unsigned int __v, __x = (i); if (__builtin_constant_p (__x)) __v = ((((__x) & 0xff000000) >> 24) | (((__x) & 0x00ff0000) >> 8) | (((__x) & 0x0000ff00) << 8) | (((__x) & 0x000000ff) << 24)); else __asm__ ("bswap %0" : "=r" (__v) : "0" (__x)); __v; }));
                               ^~~~~~~~~
ccache.cpp:6:32: error: ISO C++17 does not allow 'register' storage class specifier [-Wregister]
    auto j = (__extension__ ({ register unsigned int __v, __x = (i); if (__builtin_constant_p (__x)) __v = ((((__x) & 0xff000000) >> 24) | (((__x) & 0x00ff0000) >> 8) | (((__x) & 0x0000ff00) << 8) | (((__x) & 0x000000ff) << 24)); else __asm__ ("bswap %0" : "=r" (__v) : "0" (__x)); __v; }));
                                   ^~~~~~~~~
    2 errors generated.

Why does using ccache change the results?

The basic problem is that by default, ccache will pre-process the file (run the C-preprocessor on it) to get a single pre-processed translation unit which it uses to check its cache. Then, for efficiency's sake, if it "misses" in its cache and has to compile the file, it passes the pre-processed file directly to clang, telling clang not to run the pre-processor, only the compiler (e.g., by giving it the extension .i).

Since it is not running the pre-processor, -I arguments are inherently ignored, and this is what clang is complaining about. gcc is not as picky about this so it doesn't pose a problem there.

In ccache 3.2, fixes have been made and this problem goes away based on my local testing.

If you're stuck on 3.1, you can export CCACHE_CPP2=yes in your environment, which tells ccache to not bother with the above optimization and just pass the original C or C++ file into clang. How much performance this costs depends on the cost of pre-processing versus compilation.

I experienced this issue on TravisCI (which has an old 3.1 version of ccache) and this fixed it for me.

Your compilation error is probably rooted in the same reason: clang probably ignores errors in some or all system headers like <byteswap.h> but not in "user code". Since ccache passes it a big pre-processd blob, it all looks like user-code to clang.

This three part blog series explains this issue in more detail and contains links to relevant ccache bugs and discussion.