I am trying to create a minimal POC of evaluating javascript using using QtWebEngine and no GUI involved.
One attempt
int main(void)
{
int argc = 1;
char *argv[] = {"Hello"};
QApplication app(argc, argv);;
QtWebEngine *e = QtWebEngine::initialize()
QWebEnginePage *page = new QWebEnginePage;
page->runJavaScript("'Java''' 'Script'",
[](const QVariant &result){ qDebug() << result; });
std::cout << "Compiled and Linked" << std::endl;
return 0;
}
but this errored with a display not attached error, something to do with guis I take it.
I also found something about the distinction of console app and gui app, but it was not clear to me and I tried using variantions of
QCoreApplication *app = QCoreApplication::instance();
but this also erred.
I also ran into the issue of
:[0712/072409:ERROR:browser_main_loop.cc(217)] Running without the
SUID sandbox! See
https://chromium.googlesource.com/chromium/src/+/master/docs/linux_suid_sandbox_development.md
for more information on developing with the sandbox on.
and apparently --disable-setuid-sandbox to chromium is an escape hatch but I'm not quite clear where I could set that programmatically.
Related
.pro
LIBS += -LC:\Qt\Tools\OpenSSL\Win_x86\lib -llibssl
LIBS += -LC:\Qt\Tools\OpenSSL\Win_x86\lib -llibcrypto
INCLUDEPATH += C:\Qt\Tools\OpenSSL\Win_x86\include
main.qml
import QtQuick 2.12
import QtQuick.Window 2.12
import QtQuick.Controls 2.0
Window {
visible: true
width: 640
height: 480
Component.onCompleted: getPage(logResults)
function logResults(results) {
console.log("RESULTS: " + results)
}
function getPage(callback) {
var xhttp = new XMLHttpRequest();
var url = "https://www.google.com/"
xhttp.onreadystatechange = function() {
if (xhttp.readyState === 4 && xhttp.status === 200) {
console.log("calling callback")
callback(xhttp.responseText)
}
};
xhttp.open("GET", url);
xhttp.send();
}
}
expected output
qml: calling callback
qml: RESULTS: <html>
actual output
qt.network.ssl: QSslSocket: cannot resolve SSL_CTX_set_ciphersuites
qt.network.ssl: QSslSocket: cannot resolve SSL_set_psk_use_session_callback
qt.network.ssl: QSslSocket: cannot call unresolved function SSL_set_psk_use_session_callback
qml: calling callback
Windows 10 64-bit OS, running MSVC2017 QML project
I ran C:\Qt\MaintenanceTool.exe to install Developer and Designer Tools > OpenSSL 1.1.1d Toolkit
I've tried following a previous tutorial and another one for MSVC2017 but no luck in resolving the errors or getting xhttp.responseText. Found out the code works in ubuntu 19.4 so it just has to be that I'm running it on my windows machine that something funky is happening with the OpenSSL. I couldn't find any resolution by googling the outputted error messages. I've read that accidentally installing openSSL to "the windows directory" can cause errors, but I've not been able to actually locate "the windows directory" in question to check if I did.
edit
From C:\Qt\Tools\OpenSSL\Win_x64\bin I copied libcrypto-1_1-x64.dll and libssl-1_1-x64.dll to my project's \debug and \release folders. This removed the qt.network.ssl errors, however I am still not getting the expected output of qml: RESULTS: <html>
You ran C:\Qt\MaintenanceTool.exe to install Developer and Designer
Tools > OpenSSL 1.1.1d Toolkit
That is also my recommendation. The alternative is to compile OpenSSL yourself, or download a binary package from a third party provider. I have one of those packages installed at "C:\Program Files\OpenSSL-Win64\bin", and programs using Qt+=network are able to locate and load the libraries when their path is included in the PATH environment variable. The problem is that you will need to take care of the updates yourself, but the Qt packages are automatically updated with the MaintenanceTool along with Qt and Qt Creator. So pick your choice.
Anyway, even if you have another set of OpenSSL DLLs in your path, if you copy the libraries to the output directory of your executable, these libraries will be loaded instead. Two questions need to be answered here: 1) how do you copy the DLLs automatically each time they are needed?, and 2) how do you verify which DLLs are loaded when you run your program?
1) You may add the following to your project .pro:
win32 {
CONFIG += file_copies
CONFIG(debug, debug|release) {
openssllibs.path = $$OUT_PWD/debug
} else {
openssllibs.path = $$OUT_PWD/release
}
contains(QMAKE_TARGET.arch, x86_64) {
openssllibs.files = C:/Qt/Tools/OpenSSL/Win_x64/bin/libcrypto-1_1-x64.dll \
C:/Qt/Tools/OpenSSL/Win_x64/bin/libssl-1_1-x64.dll
} else {
openssllibs.files = C:/Qt/Tools/OpenSSL/Win_x86/bin/libcrypto-1_1.dll \
C:/Qt/Tools/OpenSSL/Win_x86/bin/libssl-1_1.dll
}
COPIES += openssllibs
}
That is it. Now your program will always have the latest libraries from Qt/Tools copied to the output directory of your project, without worrying if you compile in debug or release mode, or 32/64 bits, or another Qt Kit.
2) Run your program while inspecting the loaded DLLs with Process Explorer, by Mark Russinovich. To do so, in Process Explorer->View->Show lower pane, and select your running program in the upper pane. The lower pane lists all your loaded DLLs and origins. There are other similar utilities out there, like the open source Process Hacker.
Even understanding all of the above, and following exactly the recipe, your program still does not print the desired output. Please change the function logResults() in your qml like this:
function logResults(results) {
console.log("RESULTS Length=", results.length);
console.log("results.substr=", results.substr(0, 20));
}
You will get the following output:
qml: calling callback
qml: RESULTS Length= 47932
qml: results.substr= <!doctype html><html
Explanation: looks like console.log() has a limitation of about 32K in Windows (it doesn't on Linux). The document retrieved from the remote host is much larger, and this breaks the logging function. This is probably a bug in Qt (it should not fail silently like that).
Another advice for anybody coming here in the future: It is not strictly needed, but you may want to verify in your main() function that SSL is available, with something like this code:
#include <QDebug>
#include <QSslSocket>
int main(int argc, char *argv[])
{
QGuiApplication app(argc, argv);
if (!QSslSocket::supportsSsl()) {
qDebug() << "Sorry, OpenSSL is not supported";
return -1;
}
qDebug() << "build-time OpenSSL version:" << QSslSocket::sslLibraryBuildVersionString();
qDebug() << "run-time OpenSSL version:" << QSslSocket::sslLibraryVersionString();
[...]
}
I'm building an app that contains a WebView that runs some JavaScript code. That JavaScript code is quite allocation heavy and can require a lot of memory.
Sometimes, the amount of required memory exceeds the amount JavaScript can require and crashes the Chromium process of the WebView which crashes my app.
I listen to onMemoryTrim in my application - but it is never called in this scenario on devices with more than 1GB of memory. (Not even with TRIM_MEMORY_RUNNING_LOW).
Is there any way I could detect my WebView is running low on memory and either kill it or let it know (so it can free memory)?
I've tried polling performance.memory but it did not work. The following script crashes the WebView if executed in it:
var a = [];
var kek = () => {
var b = [];
for(var i = 0; i < 1024 * 1024 * 2; i++) b.push(Math.random());
return b;
}
var ival = setInterval(() => {
let m = performance.memory;
if(m.jsHeapSizeLimit - m.usedJSHeapSize < 1e5) {
console.log("Memory limited")
} else {
a.push(kek());
}
});
Is there any way to detect memory is about to run out so I can handle it gracefully without the app crashing?
I have discussed this with the Chromium team and the Android team and at the moment (they think and I believe them) that this is impossible.
Sometimes, the amount of required memory exceeds the amount JavaScript can require and crashes the Chromium process of the WebView which crashes my app.
You can however catch out of memory crashes in Android 8.0+ using the new termination handle API. So this works around my problem by not having to check the available memory required in the first place.
By overriding onRenderProcessGone - we get to catch the bug and recreate the WebView.
Is it possible to load a web page in c++ and get the rendered DOM? Not just the HTTP response, but the rendered DOM that occurs after java-script runs (maybe after letting it run for some amount of time). Specifically the dynamic HTML that may have changed over time? Is there a library for this?
Or if not c++, do you know of any other language which this can be done in?
Edit here's an example to illustrate better why one might want to do this:
Imagine you want to crawl a website written in angular. You can't just make an http request and use the HTTP response, because most of the DOM is rendered after javascript/dynamic html manipulates the DOM. The initial http response for an angular site probably doesn't have all the contents, its requested and rendered later through javascript/AJAX/dyanmic html.
Since DOM is something implemented differently by each browser, how you use that from C++ will be different with each browser.
I'll give an example for IE. You can use the WebBrowser ActiveX control which exposes the IWebBrowser2 interface. From there you can call IWebBrowser2::get_Document to get an IHTMLDocument2 object, which is the root of the DOM.
#include "StdAfx.h"
using namespace ATL;
using namespace std;
void ThrowIfFailed(HRESULT hr)
{
if (FAILED(hr))
throw CAtlException(hr);
}
int main()
{
::CoInitialize(nullptr);
try
{
CComPtr<IWebBrowser2> pWebBrowser;
HRESULT hr = ::CoCreateInstance(CLSID_InternetExplorer, nullptr, CLSCTX_LOCAL_SERVER, IID_PPV_ARGS(&pWebBrowser));
ThrowIfFailed(hr);
hr = pWebBrowser->put_Visible(VARIANT_TRUE);
ThrowIfFailed(hr);
hr = pWebBrowser->GoHome();
ThrowIfFailed(hr);
CComPtr<IDispatch> pDispatch;
hr = pWebBrowser->get_Document(&pDispatch);
ThrowIfFailed(hr);
CComPtr<IHTMLDocument2> pDocument;
hr = pDispatch->QueryInterface(&pDocument);
ThrowIfFailed(hr);
CComBSTR bstrTitle;
hr = pDocument->get_title(&bstrTitle);
ThrowIfFailed(hr);
wcout << bstrTitle.m_str << endl;
}
catch (const CAtlException& e)
{
wcout << L"Error (" << hex << e.m_hr << L")" << endl;
}
::CoUninitialize();
return 0;
}
This code just opens an IE window, navigates to the home page, and writes the title of the page to the console. You can also control whether the IE window becomes visible by removing the call to IWebBrowser2::put_Visible.
As I am understanding, you are asking:
"How to manipulate DOM of already rendered HTML Page through C++?"
If that's what you wanted to ask, here is my answer:
Technically, you can do it through C++. However, you need a right tool/lib/framework/ ... for doing this.
Normally, we manipulate DOM by Javascript.
In my experience, mobile developer have built-in control for load the page, usually called "webview". Android (Java) and iOS (Objective-C) have it. Then they manipulate DOM like this manner: "webview.evaluteScript("your javascript").
If you want to do it with C++. I think you can read these link:
How to embed WebKit into my C/C++/Win32 application?
How do I embed WebKit in a window?
I am a beginner in developing Chrome extensions. I am trying to achieve a native messaging between my extension and a C++ code. Here is the C++ code
int main(int argc, char* argv[]) {
// Define our message
char message[] = "{\"text\": \"This is a response message\"}";
// Collect the length of the message
unsigned int len = strlen(message);
// We need to send the 4 bytes of length information
printf("%c%c%c%c", (char) (len & 0xff),
(char) ((len>>8) & 0xFF),
(char) ((len>>16) & 0xFF),
(char) ((len>>24) & 0xFF));
// Now we can output our message
printf("%s", message);
return 0;
}
The problem is that the extension does receive any thing and I don't know how to debug the program. I've tried opening Chrome from the terminal so that errors are displayed, but nothing is displayed there. here is the code from the background.js
var port = chrome.runtime.connectNative('com.my_company.my_application');
port.onMessage.addListener(function(msg) {
console.log("Received" + msg);
});
port.onDisconnect.addListener(function() {
console.log("Disconnected");
});
Any way I could debug the program?
You can run Chrome with logging enabled and then review the errors using Sawbuck - a handy GUI designed just for that. If the problem is with Chrome, it might shed some light.
see here - http://www.chromium.org/for-testers/enable-logging
There are multiple ways to debug but no end-end IDE like debug exists.
to debug your background and content scripts--- background and page content section in chrome tools
to debug the host --- open chrome from terminal with logging enabled
Also native messaging API offers you some methods which will return appropriate error messages like, runtime.lastError
can refer this
Is it possible to query the enable/disable status of a window with Microsoft JScript?
I have this working code
#include <windows.h>
#include <iostream>
#include <string>
BOOL CALLBACK EnumWindowsProc(HWND hwnd,LPARAM lParam)
{
wchar_t text[80];
if ( ::GetWindowText( hwnd, text, 80 ) ) {
if ( std::wstring(text).find(L"MyApp") != std::wstring::npos ) {
std::wcout << ::IsWindowEnabled( hwnd ) << L"\t" << text << L"\n";
}
}
return TRUE;
}
int main(int, char**)
{
while(true){
::EnumWindows( EnumWindowsProc, 0 );
::Sleep(1000);
}
return 0;
}
and I would like to convert it into a JScript script (and running it through cscript.exe).
Maybe is there a way of doing it with WMI?
Thank you.
Neither JScript/WSH nor WMI have this functionality (WSH's GUI interaction capabilities are limited to SendKeys and AppActivate, and WMI doesn't work with GUI at all).
However, if you definitely prefer scripts to compiled code and don't mind using third-party components or scripting environments other than WSH, you can do this.
For example, in an AutoIt script, you can check the enabled/disabled state of a window using the WinGetState function. You can also call this function from JScript through the AutoIt ActiveX/COM control:
// Note: this code is untested
var oAutoIt = new ActiveXObject("AutoItX3.Control");
var state = oAutoIt.WinGetState("MyApp");
var isEnabled = state & 4;