Memory leak (OpenCV + QML)

3

I’m trying to display OpenCV Mat in QML image.

I grab frames from camera with OpenCV, the frames are displayed successfully in QML, but memory usage increases with time. How can I fix it? Here is the my code:

main.cpp

#include <QGuiApplication>
#include "videoprovider.h"

int main(int argc, char *argv[])
{
    QGuiApplication app(argc, argv);
    VideoProvider videoProvider;
    return app.exec();
}

VideoProvider.h

#ifndef VIDEOPROVIDER_H
#define VIDEOPROVIDER_H

#include <QObject>
#include <QFuture>
#include <QImage>
#include <QQmlApplicationEngine>
#include <QQuickImageProvider>
#include <opencv2/opencv.hpp>

class VideoProvider : public QObject, public QQuickImageProvider
{
    Q_OBJECT
public:
    explicit VideoProvider();
    QPixmap requestPixmap(const QString &id, QSize *size, const QSize& requestedSize);

signals:
    void frameChanged();

public slots:
    void framePainted();

private:
    QQmlApplicationEngine engine;
    bool readyfor;
    cv::Mat mat;
    QImage outputImage;
    void process();
};

#endif // VIDEOPROVIDER_H

VideoProvider.cpp

#include <QQmlContext>
#include <QtConcurrent/QtConcurrent>
#include <QDebug>
#include <QThread>
#include "videoprovider.h"
#include <QQuickImageProvider>

VideoProvider::VideoProvider() : QQuickImageProvider (QQuickImageProvider :: Pixmap)
{
    engine.rootContext()->setContextProperty("videoProvider", this);
    engine.addImageProvider(QLatin1String ("videoCapture"), this);
    engine.load(QUrl(QStringLiteral("qrc:/main.qml")));
    readyfor = true;
    QtConcurrent::run(this, VideoProvider::process);
}

void VideoProvider::framePainted()
{
    readyfor = true;
}

void VideoProvider::process()
{
    cv::VideoCapture capture(0);

    while(true){

        QThread::currentThread()->msleep(80);

        if(!readyfor) continue;

        mat.release();
        capture >> mat;

        if(mat.empty())
        {
            qDebug()<<"disconnect";
        }
        else
        {
            readyfor = false;
            cv::cvtColor(mat, mat, cv::COLOR_BGR2RGB);
            outputImage = QImage((uchar*)mat.data, mat.cols, mat.rows, mat.step, QImage::Format_RGB888);
            emit frameChanged();
        }
    }

    capture.release();
}

QPixmap VideoProvider::requestPixmap(const QString &id, QSize *size, const QSize& requestedSize)
{
    return QPixmap::fromImage(outputImage);
}

main.qml

import QtQuick 2.6
import QtQuick.Window 2.2

Window
{
    visible: true
    width: 640
    height: 480
    title: qsTr("Hello World")
    id: root

    Image{
        id: videoLayer
        anchors.fill: parent
        cache: false

        onSourceChanged:{
            videoProvider.framePainted();
        }
    }

    Connections
    {
        target: videoProvider
        property int frameCounter: 0

        onFrameChanged:
        {
            videoLayer.source = "image://videoCapture/hoge" + frameCounter;
            frameCounter ^= 1;
        }
    }
}

I found that it happened when I send signal (emit frameChanged();) to QML.

UPD:

Valgrind log:

==18038== Memcheck, a memory error detector
==18038== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==18038== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==18038== Command: ./prog
==18038== 
QML debugging is enabled. Only use this in a safe environment.
==18038== Warning: noted but unhandled ioctl 0x30000001 with no size/direction hints.
==18038==    This could cause spurious value errors to appear.
==18038==    See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18038== Warning: noted but unhandled ioctl 0x27 with no size/direction hints.
==18038==    This could cause spurious value errors to appear.
==18038==    See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18038== Warning: noted but unhandled ioctl 0x7ff with no size/direction hints.
==18038==    This could cause spurious value errors to appear.
==18038==    See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18038== Warning: noted but unhandled ioctl 0x25 with no size/direction hints.
==18038==    This could cause spurious value errors to appear.
==18038==    See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18038== Warning: noted but unhandled ioctl 0x17 with no size/direction hints.
==18038==    This could cause spurious value errors to appear.
==18038==    See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18038== Warning: set address range perms: large range [0x200000000, 0x500000000) (noaccess)
==18038== Warning: set address range perms: large range [0x500000000, 0x700000000) (noaccess)
==18038== Warning: noted but unhandled ioctl 0x19 with no size/direction hints.
==18038==    This could cause spurious value errors to appear.
==18038==    See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18038== Warning: noted but unhandled ioctl 0x21 with no size/direction hints.
==18038==    This could cause spurious value errors to appear.
==18038==    See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18038== Warning: noted but unhandled ioctl 0x1b with no size/direction hints.
==18038==    This could cause spurious value errors to appear.
==18038==    See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18038== Thread 8 Thread (pooled):
==18038== Invalid read of size 4
==18038==    at 0xAF682D0: QImage::~QImage() (in /home/dmytro/Qt/5.9.1/gcc_64/lib/libQt5Gui.so.5.9.1)
==18038==    by 0x40570F: VideoProvider::process() (videoprovider.cpp:50)
==18038==    by 0x406AEB: QtConcurrent::VoidStoredMemberFunctionPointerCall0<void, VideoProvider>::runFunctor() (qtconcurrentstoredfunctioncall.h:205)
==18038==    by 0x405FC6: QtConcurrent::RunFunctionTask<void>::run() (qtconcurrentrunbase.h:136)
==18038==    by 0xBC44BA2: ??? (in /home/dmytro/Qt/5.9.1/gcc_64/lib/libQt5Core.so.5.9.1)
==18038==    by 0xBC48849: ??? (in /home/dmytro/Qt/5.9.1/gcc_64/lib/libQt5Core.so.5.9.1)
==18038==    by 0xCE456B9: start_thread (pthread_create.c:333)
==18038==    by 0xC9773DC: clone (clone.S:109)
==18038==  Address 0x23d1b260 is 0 bytes inside a block of size 128 free'd
==18038==    at 0x4C2F24B: operator delete(void*) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==18038==    by 0xAF682F3: QImage::~QImage() (in /home/dmytro/Qt/5.9.1/gcc_64/lib/libQt5Gui.so.5.9.1)
==18038==    by 0x4047CB: VideoProvider::~VideoProvider() (videoprovider.h:11)
==18038==    by 0x403A34: main (main.cpp:8)
==18038==  Block was alloc'd at
==18038==    at 0x4C2E0EF: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==18038==    by 0xAF656BE: QImageData::create(unsigned char*, int, int, int, QImage::Format, bool, void (*)(void*), void*) (in /home/dmytro/Qt/5.9.1/gcc_64/lib/libQt5Gui.so.5.9.1)
==18038==    by 0xAF65971: QImage::QImage(unsigned char*, int, int, int, QImage::Format, void (*)(void*), void*) (in /home/dmytro/Qt/5.9.1/gcc_64/lib/libQt5Gui.so.5.9.1)
==18038==    by 0x4056E2: VideoProvider::process() (videoprovider.cpp:50)
==18038==    by 0x406AEB: QtConcurrent::VoidStoredMemberFunctionPointerCall0<void, VideoProvider>::runFunctor() (qtconcurrentstoredfunctioncall.h:205)
==18038==    by 0x405FC6: QtConcurrent::RunFunctionTask<void>::run() (qtconcurrentrunbase.h:136)
==18038==    by 0xBC44BA2: ??? (in /home/dmytro/Qt/5.9.1/gcc_64/lib/libQt5Core.so.5.9.1)
==18038==    by 0xBC48849: ??? (in /home/dmytro/Qt/5.9.1/gcc_64/lib/libQt5Core.so.5.9.1)
==18038==    by 0xCE456B9: start_thread (pthread_create.c:333)
==18038==    by 0xC9773DC: clone (clone.S:109)
c++
qt
opencv
qml
asked on Stack Overflow Nov 2, 2017 by DYY • edited Nov 2, 2017 by DYY

1 Answer

1

I grab frames from camera with OpenCV, the frames are displayed successfully in QML, but memory usage increases with time. How can I fix it?

The below preconditions the video capture to process images no faster than 1000 ms / 80 ms per frame = 12.5 frames per second (actually less due to processing time) while the standard camera rates are in the range of 25 to 80. That is why image frames are stuck in internal buffer of OpenCV etc. It won't hurt to wait just for 1 to 5 milliseconds (must be sufficient) here as long as the OS will still be able to relinquish the time slice for other threads. I personally use condition variable from either C++ 11 or from Qt and not "sleep". With condition variable we can interrupt the wait more gracefully etc.

while(true){

    QThread::currentThread()->msleep(80); // too much wait
    // also just do QThread::msleep instead

    if(!readyfor) continue;

    mat.release();
    capture >> mat
answered on Stack Overflow Nov 2, 2017 by Alexander V • edited Nov 3, 2017 by Alexander V

User contributions licensed under CC BY-SA 3.0