Saving an image of a webpage with a WebEngineView works fine, but when I want to scroll and save another image, the resulting image does not show the website has been scrolled (it shows the top of the webpage).
My question is: how do I scroll down in the QWebEngineView then save a screen shot that shows the correctly scrolled webpage?
I take a screenshot at the top of the webpage, scroll down ~700 pixels, wait for a javascript callback to trigger which then takes another screenshot. The javascript and callback works fine (I observe the QWebEngineView scrolling).
this->setScrollPageHandlerFunc([&] (const QVariant &result) {
saveSnapshotScroll();
});
saveSnapshotScroll();
view->page()->runJavaScript("scrollPage();",this->scrollPageHandlerFunc);
Screenshot code:
void MainWindow::saveSnapshotScroll()
{
QPixmap pixmap(this->size());
view->page()->view()->render(&pixmap);
pixmap.save(QString::number(QDateTime::currentMSecsSinceEpoch()) + ".png");
}
Javascript:
function scrollPage()
{
var y = qt_jq.jQuery(window).scrollTop();
qt_jq.jQuery(window).scrollTop(y+708);
}
UPDATE: I've found that if I put the saveSnapshotScroll() on a timer of ~100ms or more (i.e. wait 100ms to save the snapshot after scrolling), instead of taking the screenshot as soon as the page is scrolled, it works. So there is some latency between the javascript callback when a scroll has been performed, and the rendering of the scrolled page. I wouldn't call this a complete solution and thus why I'm only updating the post. What I would really like is a callback from QT that says the rendered webpage has been updated in the screen buffer. Does something like this exist?
When the callback of runJavaScript is triggered the script is finished. However, the window should be repainted (or at least prepared for repainting) to use QWidget::render(&pixmap).
It seems that some paint event can be useful to detect repainting of the widget. Unfortunately the QWebEngineView does not catch almost any event (except mouse enter and exit, recently added unhandled keyboard events), for example see "[QTBUG-43602] WebEngineView does not handle mouse events".
Almost all events (like mouse move or paint) are handled by QWebEngineView child delegate of private type RenderWidgetHostViewQtDelegateWidget that is derived from QOpenGLWidget.
It is possible to catch new child of QWebEngineView of type QOpenGLWidget and to install on this child the event filter hook for all needed events.
That solution relies on undocumented structure of QWebEngineView. Thus it may be not supported by future Qt releases. However, it is usable for projects with current Qt versions. Maybe in the future some more convenient interface to catch QWebEngineView events will be implemented.
The following example implements that magic:
#ifndef WEBENGINEVIEW_H
#define WEBENGINEVIEW_H
#include <QEvent>
#include <QChildEvent>
#include <QPointer>
#include <QOpenGLWidget>
#include <QWebEngineView>
class WebEngineView : public QWebEngineView
{
Q_OBJECT
private:
QPointer<QOpenGLWidget> child_;
protected:
bool eventFilter(QObject *obj, QEvent *ev)
{
// emit delegatePaint on paint event of the last added QOpenGLWidget child
if (obj == child_ && ev->type() == QEvent::Paint)
emit delegatePaint();
return QWebEngineView::eventFilter(obj, ev);
}
public:
WebEngineView(QWidget *parent = nullptr) :
QWebEngineView(parent), child_(nullptr)
{
}
bool event(QEvent * ev)
{
if (ev->type() == QEvent::ChildAdded) {
QChildEvent *child_ev = static_cast<QChildEvent*>(ev);
// there is also QObject child that should be ignored here;
// use only QOpenGLWidget child
QOpenGLWidget *w = qobject_cast<QOpenGLWidget*>(child_ev->child());
if (w) {
child_ = w;
w->installEventFilter(this);
}
}
return QWebEngineView::event(ev);
}
signals:
void delegatePaint();
};
#endif // WEBENGINEVIEW_H
Child adding is caught by WebEngineView::event. The child pointer is saved and the event filter is installed on this child. On the child paint event the signal WebEngineView::delegatePaint is emitted in WebEngineView::eventFilter.
The signal delegatePaint is always emitted when the web view is changed by some script or by highlighting of some web controls due to mouse hover or for any other reason.
The signal is emitted from the event filter before actual execution of QOpenGLWidget::paintEvent(). So, it looks that it is needed to take page snapshot only after full paint is finished (maybe using asynchronous Qt::QueuedConnection connection). It appears that at this point in the event filter when delegatePaint is triggered because of the JavaScript the widget is ready for render(). However, it possible to receive the paint event for some other reason (for example due to window activation) and that may lead to the warning message:
QWidget::repaint: Recursive repaint detected
So, it is still better to use Qt::QueuedConnection to avoid such issues.
Now the trick is to use the event delegatePaint only once when the JavaScipt is finished. That part can be adjusted to actual requirements.
The page view can be repainted at any moment due to some scripts or loading new images. Let's assume that we need to capture the page how it looks after the script execution. So, it possible to connect delegatePaint signal to saveSnapshotScroll slot only in the script callback and disconnect that connection in saveSnapshotScroll. The following test generates snapshots in a loop for three different scroll positions. The similar snapshots are orginized by folders 0, 1 and 2:
void MainWindow::runJavaScript()
{
// count initialized by 0
if (++count > 1000)
return;
QString script = QString::asprintf("window.scrollTo(0, %d);", 708 * (count % 3));
view->page()->runJavaScript(script,
[&] (const QVariant&) {
connect(view, &WebEngineView::delegatePaint,
this, &MainWindow::saveSnapshotScroll,
Qt::QueuedConnection);
}
);
}
void MainWindow::saveSnapshotScroll()
{
disconnect(view, &WebEngineView::delegatePaint,
this, &MainWindow::saveSnapshotScroll);
QPixmap pixmap(view->size());
view->render(&pixmap);
pixmap.save(QString::number(count % 3) + "/" +
QString::number(QDateTime::currentMSecsSinceEpoch()) + ".png");
runJavaScript();
}
In those cases when the event is trigged by some other window interaction it is possible to get wrong snapshot. If the window is not touched during script execution the result is correct.
To avoid handling of wrong paint events it is possible to compare Web view pixmap with previously saved image. If difference between those images is small it means that the current paint event should be skipped and it is needed to wait for the next paint event:
void MainWindow::saveSnapshotScroll()
{
QSharedPointer<QPixmap> pixmap(new QPixmap(view->size()));
view->render(pixmap.data());
// wait for another paint event if difference with saved pixmap is small
if (!isNewPicture(pixmap))
return;
pixmap->save(QString::number(count % 3) + "/" +
QString::number(QDateTime::currentMSecsSinceEpoch()) + ".png");
disconnect(view, &WebEngineView::delegatePaint,
this, &MainWindow::saveSnapshotScroll);
runJavaScript();
}
bool MainWindow::isNewPicture(QSharedPointer<QPixmap> pixmap)
{
// initialized by nullptr
if (!prevPixmap) {
prevPixmap = pixmap;
return true;
}
// <pixmap> XOR <previously saved pixmap>
QPixmap prev(*prevPixmap);
QPainter painter;
painter.begin(&prev);
painter.setCompositionMode(QPainter::RasterOp_SourceXorDestination);
painter.drawPixmap(0, 0, *pixmap);
painter.end();
// check difference
QByteArray buf;
QBuffer buffer(&buf);
buffer.open(QIODevice::WriteOnly);
prev.save(&buffer, "PNG");
// almost empty images (small difference) have large compression ratio
const int compression_threshold = 50;
bool isNew = prev.width() * prev.height() / buf.size() < compression_threshold;
if (isNew)
prevPixmap = pixmap;
return isNew;
}
The above solution is just an example and it is based on tools provided by Qt. It is possible to think about other comparison algorithms. Also similarity threshold may be adjusted to specific case. There is a limitation of such comparison if scrolled view is very similar to the previous image (for example in case of long empty space).
