How to overlay text on image in opencv without modifying the underlying matrix

I need a way to overlay text on opencv image without actually modifying the underlying image. For example inside a while loop i create a random point for each iteration and I want to display it. If I use puttext the matrix gets overwritten with the added text in each cycle.

My questions is how to overlay text on opencv image without modifying the underlying matrix. I know I could use a temporary copy of the original image and upload it every time. However I want to avoid it.

My attempt(which fails) is as below:

#include <opencv2/opencv.hpp>
#include <iostream>

using namespace cv;

int main(int argc, char * argv[])
{

    RNG rng( 0xFFFFFFFF );
    cv::Mat image(480, 640, CV_8UC3, cv::Scalar(0,255,0));

    int fontFace = FONT_HERSHEY_COMPLEX_SMALL;
    double fontScale_small=1.5;
    double fontScale_large=10.;
    std::string text="X";

    Point p;

    while(1)
    {

        p.x = rng.uniform( 0, 639 );
        p.y = rng.uniform( 0, 479 );

        putText(image, "X", p, fontFace, 1, Scalar(0,0,255), 2);

        imshow("IMAGE", image);
    waitKey(1);
    }

    return 0;
}

If you are on OpenCV 3+ and have built OpenCV with Qt, then you can use the highgui functions on the Qt GUI window to overlay text, instead of actively modifying your image. See displayOverlay(). You can also simply change the status bar text, which is sometimes more useful (so that you don't cover the image or have to deal with color clashes) with displayStatusBar(). However, note that displayOverlay() does not allow you to give specific positions for your text.

Without QT support, I don't believe this is possible in OpenCV. You'll need to either modify your image via putText() or addText(), or roll your own GUI window that you can overlay text on.