Google Test discovery is adding and running all of my tests twice on mac OS X

I'll try to keep this short, but concise.

I have been porting the google test framework (and a suite of tests) my company uses FROM Windows, TO mac os. And I have almost everything working except for some reason when I build/run my test on MAC... Google Test is including each of my defined tests twice.

On MAC, my Makefile outputs "main_test", and to quickly show the issue if I use the --gtest_list_tests option you can see my dumbed down list of test are all duplicated a second time.

$ ./main_test --gtest_list_tests
BEGINNING Google Test!
sman_drift_test.
  slow_forward
  medium_back
  slow_forward
  medium_back
motor_self_test/motor_test.
  self_test/0  # GetParam() = "rev"
  self_test/1  # GetParam() = "fwd"
  self_test/0  # GetParam() = "rev"
  self_test/1  # GetParam() = "fwd"

The exact same source files, compiled in VS on windows result in my "unit_test_d.exe" file and if I run the same --gtest_list_tests option from the "cmd" shell I get this:

>unit_test_d.exe --gtest_list_tests
BEGINNING Google Test!
sman_drift_test.
  slow_forward
  medium_back
motor_self_test/motor_test.
  self_test/0  # GetParam() = "rev"
  self_test/1  # GetParam() = "fwd"

There is only ONE copy of these tests in my servo_drift_test.cpp file, likewise in motor_self_test.cpp file.

TEST_F(sman_drift_test, slow_forward) 
{
     // Do my test stuff...
}

TEST_F(sman_drift_test, medium_back) 
{
     // Do my test stuff...
}

I have been messing with this for a few days now... and I have eliminated a lot of possible options. Including basically ALL of my companies code.

I originally followed the setup instructions for MAC on this web link: http://hack.limbicmedia.ca/installing-google-test/

NOTE: I should note... while following those instructions, I did change the directories and use the "gtest" files already in our tree (version listed at the bottom), so it IS possible I could have messed up a file placement, or possibly a build option when recompiling the source we already had downloaded, and placing those directories in the correct locations. Hopefully not. Our original libraries were compliled with VS 2015 for Windows... I built the same source files with the cmake and install steps listed in the link above. I do have gtest.h and libgtest.a in /usr/local/inc and lib, so the include path stuff should be good(?).

And run just the sample code as it instructs. And likewise... I see the same issue... NOTE: I did comment out the "TEST(FactorialTest, Zero)" case as well, just to verify commenting out 1 test... really does eliminate BOTH instances. And it did. But you can see... the "Negative" and "Positive" both get added twice, as well as all 3 of the "IsPrime()" tests.

$ ./sample1 --gtest_list_tests
Running main() from gtest_main.cc
FactorialTest.
  Negative
  Positive
  Negative
  Positive
IsPrimeTest.
  Negative
  Trivial
  Positive
  Negative
  Trivial
  Positive

And likewise, actually running the tests results in duplicate tests being run. There are 6 test cases in the sample, 3 in FactorialTest and 3 in IsPrimeTest. You can see it's building/running each set twice... 6 in each set, 12 total:

$ ./sample1
Running main() from gtest_main.cc
[==========] Running 12 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 6 tests from FactorialTest
[ RUN      ] FactorialTest.Negative
[       OK ] FactorialTest.Negative (0 ms)
[ RUN      ] FactorialTest.Zero
[       OK ] FactorialTest.Zero (0 ms)
[ RUN      ] FactorialTest.Positive
[       OK ] FactorialTest.Positive (0 ms)
[ RUN      ] FactorialTest.Negative
[       OK ] FactorialTest.Negative (0 ms)
[ RUN      ] FactorialTest.Zero
[       OK ] FactorialTest.Zero (0 ms)
[ RUN      ] FactorialTest.Positive
[       OK ] FactorialTest.Positive (0 ms)
[----------] 6 tests from FactorialTest (0 ms total)

[----------] 6 tests from IsPrimeTest
[ RUN      ] IsPrimeTest.Negative
[       OK ] IsPrimeTest.Negative (0 ms)
[ RUN      ] IsPrimeTest.Trivial
[       OK ] IsPrimeTest.Trivial (0 ms)
[ RUN      ] IsPrimeTest.Positive
[       OK ] IsPrimeTest.Positive (0 ms)
[ RUN      ] IsPrimeTest.Negative
[       OK ] IsPrimeTest.Negative (0 ms)
[ RUN      ] IsPrimeTest.Trivial
[       OK ] IsPrimeTest.Trivial (0 ms)
[ RUN      ] IsPrimeTest.Positive
[       OK ] IsPrimeTest.Positive (0 ms)
[----------] 6 tests from IsPrimeTest (0 ms total)

[----------] Global test environment tear-down
[==========] 12 tests from 2 test cases ran. (0 ms total)
[  PASSED  ] 12 tests.

I have done a lot of research on this issue, and don't think it's a linking issue to my own libraries or object files.

It is certainly possible the issue could be related to some build options with Google Test, and the libraries created/installed by it (-lgtest). It's possible it's getting linked twice(?), or is there some sort of define or parameter setup somewhere in google test that says "run all my tests twice"? NOTE: This issue is NOT the "./sample_test --gtest_repeat=2" option. This repeats my repeated tests twice, so they run 4 times each.

Lastly, and this could be a potential hint... and how I got down this rabbit hole in the first place... is that when I use my full code stack, and NOT the "sample"... where I connect to a serial port and maintain my serial handle structure... as you can see... once I setup my pointer with Address (0x6d009f0) and File Descriptor FD[3]... I LOSE it... or more likely have an additional one defined (as NULL)... with the "duplicate" instance of the test.

Also, what is strange is where it goes NULL... and where it comes back. The flow looks like this:

Start Google Test
 - Do Init.  IE: Establish and init Serial port (init, open, config port)
    - Pointer GOOD Addr(0x6d009f0) FD[3]
    - Get Device Serial Number (Works)
 - Begin servo_drift_test
    - Do "SetUpTestCase()" Init Code
       - Set Device Mode to 2, required for this "set" of tests.
       - Pointer GOOD  Addr(0x6d009f0) FD[3]
    - Run First Test Case *** Now I presume this is the "duplicate" case
       - Pointer LOST Addr(0)... NULL (Not initialized)
       - TEST instance ONE FAILS
    - Run Second Test Case *** This appears to have the matching memory space as the original init and setup code!!!
       - Runs a SUCCESSFUL "get_params" command.
       - Pointer GOOD  Addr(0x6d009f0) FD[3]
       - TEST instance TWO PASSED
    - Servo Drift Test "TearDownTestCase()"
       - Nothing To Do Here
 - Complete Test Summary

As you can see, it ran 2 instances of the test case... although there is only 1 "test" in this very simple stripped down example. I tried to clean it up as best I can, but still show the debug prints. :)

I find it peculiar that the "SetUp" for the test case only runs once... and runs just fine. But the TWO instances of the same test, clearly one is not in the same memory space. This is probably a symptom of how Google Test adds "tests" versus "test sets". But I'm guessing this is a hint to the overall problem.

  $ ./main_test port=/dev/cu.usbserial-FTFMEXDK
unit_test version: 6.00.00
  Set Port String[64]: /dev/cu.usbserial-FTFMEXDK <---
  Global Port String: /dev/cu.usbserial-FTFMEXDK <---
using serial port /dev/cu.usbserial-FTFMEXDK
SUCCESS - GET_PORT_BY_NAME
             PORT - Struct Addr port2print[6d009f0]
                  - Name: /dev/cu.usbserial-FTFMEXDK
                  - Baudrate: -1
                  - Bits: -1
                  - Parity: -1
                  - Stopbits: -1
                  - FD[-1]

SUCCESS - SP_OPEN in Mode[3]
             PORT - Struct Addr port2print[6d009f0]
                  - Name: /dev/cu.usbserial-FTFMEXDK
                  - Baudrate: 9600
                  - Bits: 8
                  - Parity: -1
                  - Stopbits: 1
                  - FD[3]

SUCCESS - SET_BAUDRATE 460800
             PORT - Struct Addr port2print[6d009f0]
                  - Name: /dev/cu.usbserial-FTFMEXDK
                  - Baudrate: 460800
                  - Bits: 8
                  - Parity: 0
                  - Stopbits: 1
                  - FD[3]



  WRITE
    - [29]: sys_config get_smart_sn
  PROXY RECEIVE START
 TOK[19] EXP[19] ==>  0x00000000  1 0 E 2 0 1 1 7 0 6 2 7 0 0 1 4 8 R  
Target ALIVE, unit serial no: '10E20117062700148R'


BEGINNING Google Test!
Note: Google Test filter = *.slow_forward
[==========] Running 2 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 2 tests from sman_drift_test
             PORT - Struct Addr port2print[6d009f0]
                  - FD[3]
  WRITE
    - [24]: sys_config set_mode  2 
  PROXY RECEIVE START
 TOK[1] EXP[1] ==>  0x00000000   

[ RUN      ] sman_drift_test.slow_forward
             PORT - Struct Addr port2print[0]
                  - NULL PORT Structure!!! 

          ERROR - comm port initialization failed
servo_drift_test.cpp:485: Failure
Failed
UNABLE TO COMMUNICATE WITH DEVICE


[  FAILED  ] sman_drift_test.slow_forward (15013 ms)
[ RUN      ] sman_drift_test.slow_forward
              - PreTestFlags[0x7]
             PORT - Struct Addr port2print[6d009f0]
                  - FD[3]

  WRITE
    - [17]: sman get_params 
  PROXY RECEIVE START
 TOK[13] EXP[13] ==>  0x00000000  1 0 0 -1471 1 0 0 0 0 0 1 75000  

[       OK ] sman_drift_test.slow_forward (29114 ms)
[----------] 2 tests from sman_drift_test (44127 ms total)

[----------] Global test environment tear-down
[==========] 2 tests from 1 test case ran. (45607 ms total)
[  PASSED  ] 1 test.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] sman_drift_test.slow_forward

However, here is my Makefile if anyone cares to see it (I am using "libserialport" for my non-windows serial connection code... Ideally this will build on many platforms). To my knowledge the Makefile runs and builds just fine. I'm using it to build, everything works. Note: I did change the name of "rpclibstuff" for this example, because the actual name was part of our product... so I hid it for security reasons. :) So if there is a syntax mistake related to "rpclibstuff" it's probably just in this example. :)a

$ cat Makefile 
# Include directories for rpc .h files
RPCINC=../../../Common/rpc/inc/
RPCSRC=../../../Common/rpc/src/

# Include directory for the "system_mode.h" include
TAINC=../../../Tensioner/application/inc/

# Direcroty path for libserialport
LSDIR=../libserialport/

all: main_test

main_test: rpclibstuff.a 
    g++ -std=c++11 -stdlib=libc++ main.cpp servo_drift_test.cpp motor_test.cpp -lgtest -lpthread -o main_test -I $(RPCINC) -I $(LSDIR) -L. -lrpclibstuff -lserialport

rpclibstuff.a: rpc_util.o rpc_proxy.o rpc_proxy_util.o rpc_dispatch_util.o
    ar rcs librpclibstuff.a rpc_dispatch_util.o rpc_proxy.o rpc_proxy_util.o rpc_util.o


# gcc "-c" option compiles source files without linking.
rpc_util.o: $(RPCSRC)rpc_util.c
    gcc -c $(RPCSRC)rpc_util.c -o rpc_util.o -I $(RPCINC) -I $(TAINC)

rpc_proxy.o: $(RPCSRC)rpc_proxy.c
    gcc -c $(RPCSRC)rpc_proxy.c -o rpc_proxy.o -I $(RPCINC)

rpc_proxy_util.o: $(RPCSRC)rpc_proxy_util.c
    gcc -c $(RPCSRC)rpc_proxy_util.c -o rpc_proxy_util.o -I $(RPCINC)

rpc_dispatch_util.o: $(RPCSRC)rpc_dispatch_util.c
    gcc -c $(RPCSRC)rpc_dispatch_util.c -o rpc_dispatch_util.o -I $(RPCINC)

clean:
    rm *.o *.a main_test

Additional Background: My company gave me a blob of code, which was a google test framework (v1.7.0) and an entire suite of tests that interface through a serial connection to our product. No one has looked at this for a couple years, and it's all windows based. Builds a windows based "unit_test_d.exe" file output. I was able to build and expand it significantly and the project is built in VS 2015... I was able to port it forward and open/build the project in VS 2017 v15.6.7

I was then given the task to port the entire project over to mac, so we can interface with a BLE (Bluetooth) engine we already have existing for our product. Likewise to continue testing going forward from a mac terminal rather than needing a windows VM or windows box. I have successfully ported the code so it still runs on BOTH platforms (a LOT of #ifdef _WIN32 or #ifdef APPLE), and I can now build the project in VS on Windows, as well as on mac os x with a terminal shell.

Appologies if any of that was not the right wording. :)

Any assistance with this issue would be greatly appreciated. Sorry for the length of this post, but I wanted to give as much specific info as I could the first time! :) Thanks!!!

I had originally thought this might be an issue with my compiler (gcc/g++), because when I build in XCODE the issue was gone.

However, what XCODE did was simply put the output file "sample1" in a different location. It turns out the actual LOCATION I was running the file from mattered. I could run it from my test/src/ directory and all the tests were duplicated, when I ran it from /use/local/bin/ (where XCODE put it), it ran fine. Additionally... moving the (exact same) file from one spot to the other, and executing it would result in either working, or not working... depending on the location I ran the file from.

I was able to use Activity Monitor (mac) to see the "Open Files/Ports" when running my test file... and narrowed it down to a mysterious library being included when running from one location, but not the other.

That Library was /usr/lib/libprelaod.dylib

I tracked down this library which I also found in a "dgagent" folder (and copied to /usr/lib... both have the same time/date stamp), to what I believe is part of Digital Guardian product... probably that my company has installed on my machine.

I don't know what is in this library, or why it makes Google Test executables run all their tests twice. I also don't know why moving an executable to a different location would affect how many times a test is run, and why that's not done at compile/build time.

But as it turns out, I don't care. I'm now putting my outputed test file in /usr/local/bin/ and running it from there, and it works just fine.

Anyway, I wanted to post an update to this in case anyone else sees a similar problem... or just feels like answering a little about why this is happening. :)

Thanks, J