I’m trying to display OpenCV Mat in QML image.
I grab frames from camera with OpenCV, the frames are displayed successfully in QML, but memory usage increases with time. How can I fix it? Here is the my code:
main.cpp
#include <QGuiApplication>
#include "videoprovider.h"
int main(int argc, char *argv[])
{
QGuiApplication app(argc, argv);
VideoProvider videoProvider;
return app.exec();
}
VideoProvider.h
#ifndef VIDEOPROVIDER_H
#define VIDEOPROVIDER_H
#include <QObject>
#include <QFuture>
#include <QImage>
#include <QQmlApplicationEngine>
#include <QQuickImageProvider>
#include <opencv2/opencv.hpp>
class VideoProvider : public QObject, public QQuickImageProvider
{
Q_OBJECT
public:
explicit VideoProvider();
QPixmap requestPixmap(const QString &id, QSize *size, const QSize& requestedSize);
signals:
void frameChanged();
public slots:
void framePainted();
private:
QQmlApplicationEngine engine;
bool readyfor;
cv::Mat mat;
QImage outputImage;
void process();
};
#endif // VIDEOPROVIDER_H
VideoProvider.cpp
#include <QQmlContext>
#include <QtConcurrent/QtConcurrent>
#include <QDebug>
#include <QThread>
#include "videoprovider.h"
#include <QQuickImageProvider>
VideoProvider::VideoProvider() : QQuickImageProvider (QQuickImageProvider :: Pixmap)
{
engine.rootContext()->setContextProperty("videoProvider", this);
engine.addImageProvider(QLatin1String ("videoCapture"), this);
engine.load(QUrl(QStringLiteral("qrc:/main.qml")));
readyfor = true;
QtConcurrent::run(this, VideoProvider::process);
}
void VideoProvider::framePainted()
{
readyfor = true;
}
void VideoProvider::process()
{
cv::VideoCapture capture(0);
while(true){
QThread::currentThread()->msleep(80);
if(!readyfor) continue;
mat.release();
capture >> mat;
if(mat.empty())
{
qDebug()<<"disconnect";
}
else
{
readyfor = false;
cv::cvtColor(mat, mat, cv::COLOR_BGR2RGB);
outputImage = QImage((uchar*)mat.data, mat.cols, mat.rows, mat.step, QImage::Format_RGB888);
emit frameChanged();
}
}
capture.release();
}
QPixmap VideoProvider::requestPixmap(const QString &id, QSize *size, const QSize& requestedSize)
{
return QPixmap::fromImage(outputImage);
}
main.qml
import QtQuick 2.6
import QtQuick.Window 2.2
Window
{
visible: true
width: 640
height: 480
title: qsTr("Hello World")
id: root
Image{
id: videoLayer
anchors.fill: parent
cache: false
onSourceChanged:{
videoProvider.framePainted();
}
}
Connections
{
target: videoProvider
property int frameCounter: 0
onFrameChanged:
{
videoLayer.source = "image://videoCapture/hoge" + frameCounter;
frameCounter ^= 1;
}
}
}
I found that it happened when I send signal (emit frameChanged();) to QML.
UPD:
Valgrind log:
==18038== Memcheck, a memory error detector
==18038== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==18038== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==18038== Command: ./prog
==18038==
QML debugging is enabled. Only use this in a safe environment.
==18038== Warning: noted but unhandled ioctl 0x30000001 with no size/direction hints.
==18038== This could cause spurious value errors to appear.
==18038== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18038== Warning: noted but unhandled ioctl 0x27 with no size/direction hints.
==18038== This could cause spurious value errors to appear.
==18038== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18038== Warning: noted but unhandled ioctl 0x7ff with no size/direction hints.
==18038== This could cause spurious value errors to appear.
==18038== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18038== Warning: noted but unhandled ioctl 0x25 with no size/direction hints.
==18038== This could cause spurious value errors to appear.
==18038== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18038== Warning: noted but unhandled ioctl 0x17 with no size/direction hints.
==18038== This could cause spurious value errors to appear.
==18038== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18038== Warning: set address range perms: large range [0x200000000, 0x500000000) (noaccess)
==18038== Warning: set address range perms: large range [0x500000000, 0x700000000) (noaccess)
==18038== Warning: noted but unhandled ioctl 0x19 with no size/direction hints.
==18038== This could cause spurious value errors to appear.
==18038== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18038== Warning: noted but unhandled ioctl 0x21 with no size/direction hints.
==18038== This could cause spurious value errors to appear.
==18038== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18038== Warning: noted but unhandled ioctl 0x1b with no size/direction hints.
==18038== This could cause spurious value errors to appear.
==18038== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18038== Thread 8 Thread (pooled):
==18038== Invalid read of size 4
==18038== at 0xAF682D0: QImage::~QImage() (in /home/dmytro/Qt/5.9.1/gcc_64/lib/libQt5Gui.so.5.9.1)
==18038== by 0x40570F: VideoProvider::process() (videoprovider.cpp:50)
==18038== by 0x406AEB: QtConcurrent::VoidStoredMemberFunctionPointerCall0<void, VideoProvider>::runFunctor() (qtconcurrentstoredfunctioncall.h:205)
==18038== by 0x405FC6: QtConcurrent::RunFunctionTask<void>::run() (qtconcurrentrunbase.h:136)
==18038== by 0xBC44BA2: ??? (in /home/dmytro/Qt/5.9.1/gcc_64/lib/libQt5Core.so.5.9.1)
==18038== by 0xBC48849: ??? (in /home/dmytro/Qt/5.9.1/gcc_64/lib/libQt5Core.so.5.9.1)
==18038== by 0xCE456B9: start_thread (pthread_create.c:333)
==18038== by 0xC9773DC: clone (clone.S:109)
==18038== Address 0x23d1b260 is 0 bytes inside a block of size 128 free'd
==18038== at 0x4C2F24B: operator delete(void*) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==18038== by 0xAF682F3: QImage::~QImage() (in /home/dmytro/Qt/5.9.1/gcc_64/lib/libQt5Gui.so.5.9.1)
==18038== by 0x4047CB: VideoProvider::~VideoProvider() (videoprovider.h:11)
==18038== by 0x403A34: main (main.cpp:8)
==18038== Block was alloc'd at
==18038== at 0x4C2E0EF: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==18038== by 0xAF656BE: QImageData::create(unsigned char*, int, int, int, QImage::Format, bool, void (*)(void*), void*) (in /home/dmytro/Qt/5.9.1/gcc_64/lib/libQt5Gui.so.5.9.1)
==18038== by 0xAF65971: QImage::QImage(unsigned char*, int, int, int, QImage::Format, void (*)(void*), void*) (in /home/dmytro/Qt/5.9.1/gcc_64/lib/libQt5Gui.so.5.9.1)
==18038== by 0x4056E2: VideoProvider::process() (videoprovider.cpp:50)
==18038== by 0x406AEB: QtConcurrent::VoidStoredMemberFunctionPointerCall0<void, VideoProvider>::runFunctor() (qtconcurrentstoredfunctioncall.h:205)
==18038== by 0x405FC6: QtConcurrent::RunFunctionTask<void>::run() (qtconcurrentrunbase.h:136)
==18038== by 0xBC44BA2: ??? (in /home/dmytro/Qt/5.9.1/gcc_64/lib/libQt5Core.so.5.9.1)
==18038== by 0xBC48849: ??? (in /home/dmytro/Qt/5.9.1/gcc_64/lib/libQt5Core.so.5.9.1)
==18038== by 0xCE456B9: start_thread (pthread_create.c:333)
==18038== by 0xC9773DC: clone (clone.S:109)
I grab frames from camera with OpenCV, the frames are displayed successfully in QML, but memory usage increases with time. How can I fix it?
The below preconditions the video capture to process images no faster than 1000 ms / 80 ms per frame = 12.5 frames per second (actually less due to processing time) while the standard camera rates are in the range of 25 to 80. That is why image frames are stuck in internal buffer of OpenCV etc. It won't hurt to wait just for 1 to 5 milliseconds (must be sufficient) here as long as the OS will still be able to relinquish the time slice for other threads. I personally use condition variable from either C++ 11 or from Qt and not "sleep". With condition variable we can interrupt the wait more gracefully etc.
while(true){
QThread::currentThread()->msleep(80); // too much wait
// also just do QThread::msleep instead
if(!readyfor) continue;
mat.release();
capture >> mat
User contributions licensed under CC BY-SA 3.0