Develop Reference

How to return memory of terminated web worker safely?

I made to use web workers to upload files.
It handle with small size files.
But with large size file, the speed is getting very slow and my script causes web page collapse.
It does not return memory of web workers.
See the attachment.
The Dedicated Workers keep being accumulated and consume GB memory, when large file is being uploaded.
And I see this warning accumulating whenever web worker call close()
Scripts may close only the windows that were opened by them.
I throttled threadsQuantity as 5.
I think the number of web workers should not be exceeded more than 5.
class Queue {
constructor() {
this.timestamp = new Date().getTime();
this.activeConnections = {};
this.threadsQuantity = 5;
}
async sendNext() {
const activeConnections = Object.keys(this.activeConnections).length;
if (activeConnections >= this.threadsQuantity) {
return;
}
if (!this.chunksQueue.length) {
if (!activeConnections) {
this.complete();
}
return;
}
let chunkId = this.chunksQueue.pop();
this.activeConnections[chunkId] = true;
this.sendChunk( chunkId) ;
}
sendChunk( chunkId) {
if (window.Worker) {
let chunk = this.getChunk( chunkId)
const myWorker = new Worker("/assets/js/worker.js?v=" + this.timestamp);
myWorker.postMessage([this.timestamp, chunkId, chunk]);
myWorker.onmessage = (e) => {
var obj = JSON.parse(e.data)
if( obj.success) {
delete this.activeConnections[chunkId];
this.sendNext()
close();
} else {
sendChunk( chunkId);
}
}
}
}
}
I tried with close() , self.close() but all got same warning and failed.
I tried with this.close(), but it cause this error.
app.0a4dcc55.js:32 Uncaught TypeError: this.close is not a function
at _.onmessage
How can I kill terminated web workers safely during process ?

Nightwatch.js function not 'closing'

I'm trying to perform a function at the beginning of my test, then the rest of the test should be executed.
This is my custom-command (named internalAdviceLinksHtml):
var solr = require('solr-client')
exports.command = function() {
this
var client = solr.createClient('solr.dev.bauerhosting.com', 8080, 'cms', '/www.parkers.co.uk');
var globalSettingsQuery = client.createQuery()
.q({TypeName:'Bauer.Parkers.GlobalSettings'})
.start(0)
.rows(10);
client.search(globalSettingsQuery,function(err,obj) {
if (err) {
console.log(err);
} else {
var myresult = (obj.response.docs[0].s_InternalAdviceLinksHtml);
console.log(myresult.length);
if (myresult.length === 0) {
console.log('content block not configured');
} else {
console.log('content block configured');
}
}
});
return this;
};
Test-file (script):
module.exports = {
'set up the solr query': function (browser) {
browser
.solr_query.global_settings.internalAdviceLinksHtml();
},
'links above footer on advice landing page displayed': function (browser) {
browser
.url(browser.launch_url + browser.globals.carAdvice)
.assert.elementPresent('section.seo-internal-links')
},
'closing the browser': function (browser) {
browser
.browserEnd();
},
};
The function works correctly (i.e. if myresult length is 0 then "content block is not configured" is displayed, etc), but the following test ("links above footer on advice landing page displayed") is never invoked.
It seems like the execution stops after the custom-command. I'm sure this will be something quite obvious to someone, but I just can't seem to see what it is.
Any help would be greatly appreciated.

Regarding your internalAdviceLinksHtml custom-command, everything looks good from my point of view (I presume that lonely this was a typo).
Your hunch is correct, it seems that the Nightwatch test-runner fails to go to the next test, which is probably due to some promise not being resolved upstream (client.search function from internalAdviceLinksHtml).
I would recommend doing a return this immediately after outputting to console (content block not configured, or content block configured) and see if that fixes the problem:
client.search(globalSettingsQuery,function(err,obj) {
if (err) {
console.log(err);
} else {
var myresult = (obj.response.docs[0].s_InternalAdviceLinksHtml);
console.log(myresult.length);
if (myresult.length === 0) {
console.log('content block not configured');
} else {
console.log('content block configured');
}
}
return this
});
Also, a few extra pointers:
make use of the Nightwatch test-hooks to make your tests easier to read/maintain & create a separation of concern (setup => before/beforeEach hooks | teardown (e.g: browser.end()) => after/afterEach hooks);
you need not do an explicit browser.end() at the end of your test case. Check this answer for more information on the matter.
Your test-file would become:
module.exports = {
// > do your setup here <
before(browser) {
browser
.solr_query.global_settings.internalAdviceLinksHtml();
},
'links above footer on advice landing page displayed': function (browser) {
browser
.url(browser.launch_url + browser.globals.carAdvice)
.assert.elementPresent('section.seo-internal-links');
},
// > do your cleanup here <
after(browser) {
browser
.browserEnd();
},
};

Failed to execute 'start' on 'SpeechRecognition': recognition has already started

I am using a wrapper of Web Speech API for Angular6. I am trying to implement a system of starting-stopping after each 3.5s in order to be able to manipulate the results for these small parts.
Even though I stop the recognition, before starting it again, I keep getting this error Failed to execute 'start' on 'SpeechRecognition': recognition has already started.
As suggested in this post, I first verify whether the speech recognition is active or not and only if not active, I try to start it. https://stackoverflow.com/a/44226843/6904971
Here is the code:
constructor( private http: Http, private service: SpeechRecognitionService, private links: LinksService) {
var recognizing; // will get bool values to verify if recognition is active
this.service.onresult = (e) => {
this.message = e.results[0].item(0).transcript;
};
this.service.onstart = function () {
recognizing = true;
};
this.service.onaudiostart = function () {
recognizing = true;
};
this.service.onerror = function (event) {
recognizing = false;
};
this.service.onsoundstart = function () {
recognizing = true;
};
this.service.onsoundstart = function () {
recognizing = true;
};
this.record = () => {
this.service.start();
setInterval(root.ongoing_recording(), 3500);
};
var root = this;
var speech = '';
this.stop_recording = () => {
this.service.stop();
};
this.ongoing_recording = ()=> {
setTimeout(function(){
if( recognizing === true){
root.service.stop();
root.service.onend = (e) => {
recognizing = false;
speech = root.message;
var sentence = document.createElement('span');
sentence.innerHTML = speech + " ";
document.body.appendChild(sentence);
}
}
}, 3500);
setTimeout(function(){
if(recognizing === false){
root.service.start();
}
}, 3510);
};
}
start() {
this.service.start();
}
stop() {
this.service.stop();
}
record(){
this.record();
}
stop_recording(){
this.stop_recording();
}
ongoing_recording(){
this.ongoing_recording();
}
I think that the timing might not be good (with the setTimeout and interval). Any help would be much appreciated. Thank you! :)

I used Web Speech API for voice search functionality in my site and I was facing a similar sort of situation. It has one microphone icon which toggles the speech recognition on and off. It was working fine in the normal on and off of the button that started speech recognition but was breaking only if you test it rigorously with a continuous button toggle.
Solution:
The thing that worked for me is:
try{
//line of code to start the speech recognition
}
catch{
//line of code to stop the speech recognition
}
So I wrapped the .start() method which was breaking the application in a try block and then added the catch block to stop it. And even if it comes across this problem, on the next button click to turn on the speech recognition, it works. I hope you would be able to extract something from it.

one observation:
you run setInterval() every 3500 ms to invoke ongoing_recording(), but then use setTimeout() with 3500 ms again within ongoing_recording().
Besides that, maybe logging the error handler --where recognizing is also set to false-- could help finding a solution:
in past versions of the SpeechRecognition implementation, not every error did actually stop the recognition (I don't know if that is still the case).
So it might be the case, that recognizing is reset due to an error that did not actually stop the recognition; if this is really the cause of the error when restarting recognition, it could be just catched & ignored.
Also it might be worth trying to re-start the recognition in the onend handler (and onerror).

I am not sure what is the reason that is causing it in your code, but i had the same error and what caused it in my case was that I was calling start() twice in a row, so what fixed it was adding a variable to check if the recognition has started or stopped, so if it has started and I clicked it again it would return speach.stop() to avoid using start() again.
let recognition = new SpeechRecognition();
let status = 0;
document.querySelector(".mic").addEventListener("click",() => {
if (status == 1) {
status = 0;
return recognition.stop();
}
recognition.start();
status = 1;
recognition.onresult = function (event) {
status=0;
var text = event.results[0][0].transcript;
recognition.stop();
};
recognition.onspeechend = function () {
status = 0;
recognition.stop();
};
});

Update HTML object with node.js and javascript

I'm new to nodejs and jquery, and I'm trying to update one single html object using a script.
I am using a Raspberry pi 2 and a ultrasonic sensor, to measure distance. I want to measure continuous, and update the html document at the same time with the real time values.
When I try to run my code it behaves like a server and not a client. Everything that i console.log() prints in the cmd and not in the browesers' console. When I run my code now i do it with "sudo node surveyor.js", but nothing happens in the html-document. I have linked it properly to the script. I have also tried document.getElementsByTagName("h6").innerHTML = distance.toFixed(2), but the error is "document is not defiend".
Is there any easy way to fix this?
My code this far is:
var statistics = require('math-statistics');
var usonic = require('r-pi-usonic');
var fs = require("fs");
var path = require("path");
var jsdom = require("jsdom");
var htmlSource = fs.readFileSync("../index.html", "utf8");
var init = function(config) {
usonic.init(function (error) {
if (error) {
console.log('error');
} else {
var sensor = usonic.createSensor(config.echoPin, config.triggerPin, config.timeout);
//console.log(config);
var distances;
(function measure() {
if (!distances || distances.length === config.rate) {
if (distances) {
print(distances);
}
distances = [];
}
setTimeout(function() {
distances.push(sensor());
measure();
}, config.delay);
}());
}
});
};
var print = function(distances) {
var distance = statistics.median(distances);
process.stdout.clearLine();
process.stdout.cursorTo(0);
if (distance < 0) {
process.stdout.write('Error: Measurement timeout.\n');
} else {
process.stdout.write('Distance: ' + distance.toFixed(2) + ' cm');
call_jsdom(htmlSource, function (window) {
var $ = window.$;
$("h6").replaceWith(distance.toFixed(2));
console.log(documentToSource(window.document));
});
}
};
function documentToSource(doc) {
// The non-standard window.document.outerHTML also exists,
// but currently does not preserve source code structure as well
// The following two operations are non-standard
return doc.doctype.toString()+doc.innerHTML;
}
function call_jsdom(source, callback) {
jsdom.env(
source,
[ 'jquery-1.7.1.min.js' ],
function(errors, window) {
process.nextTick(
function () {
if (errors) {
throw new Error("There were errors: "+errors);
}
callback(window);
}
);
}
);
}
init({
echoPin: 15, //Echo pin
triggerPin: 14, //Trigger pin
timeout: 1000, //Measurement timeout in µs
delay: 60, //Measurement delay in ms
rate: 5 //Measurements per sample
});

Node.js is a server-side implementation of JavaScript. It's ok to do all the sensors operations and calculations on server-side, but you need some mechanism to provide the results to your clients. If they are going to use your application by using a web browser, you must run a HTTP server, like Express.js, and create a route (something like http://localhost/surveyor or just http://localhost/) that calls a method you have implemented on server-side and do something with the result. One possible way to return this resulting data to the clients is by rendering an HTML page that shows them. For that you should use a Template Engine.
Any DOM manipulation should be done on client-side (you could, for example, include a <script> tag inside your template HTML just to try and understand how it works, but it is not recommended to do this in production environments).
Try searching google for Node.js examples and tutorials and you will get it :)

Can't run Selenium PhantomJS instances in parallell

I'm using Selenium's node.js API to run PhantomJS instances against a series of web pages. The code I use to execute the actions on the pages work fine, but it seems only one instance of Selenium/PhantomJS can run at a time. This function is called multiple times from the same module and steps through pages in a webshop where the pagination is handled client side (which is why I need the Selenium/PhantomJS environment - to extract data from each page).
Once again, the code in and of itself works fine, but it can't execute in parallell. What could be causing this?
module.exports = function (crawler, page, parsePage, done) {
"use strict";
var _ = require("lodash"),
format = require("util").format,
path = require("path"),
webdriver = require("selenium-webdriver"),
By = webdriver.By,
until = webdriver.until;
var phantomPath = path.resolve(__dirname, "../node_modules/.bin/phantomjs"),
isWin = process.platform === "win32";
var driver = new webdriver.Builder()
.withCapabilities({
"phantomjs.binary.path": isWin ? phantomPath + ".cmd" : phantomPath
})
.forBrowser("phantomjs")
.build();
var windowHandle = new webdriver.WebDriver.Window(driver);
windowHandle.setSize(1100, 1000);
var getAllPagesContent = function (driver) {
var pagesContent = [],
pageNo = 1;
var getNextPage = function () {
var nextPageLink;
return driver.findElements(By.css(".pagination li")).then(function (elements) {
return elements[elements.length - 1];
}).then(function (element) {
nextPageLink = element;
return element.getAttribute("class");
}).then(function (className) {
return _.includes(className, "active");
}).then(function (isLastPage) {
return (!isLastPage) ? driver.getPageSource() : false;
}).then(function (content) {
if (content)
pagesContent.push(content);
content && console.log("Got page %d", pageNo++);
return nextPageLink.findElement(By.css("a")).then(function (element) {
return element.click();
}).then(function () {
return driver.wait(until.stalenessOf(nextPageLink), 10 * 1000);
}).then(function () {
return content ? getNextPage() : pagesContent;
});
});
};
return getNextPage();
};
var processTimeout = setTimeout(function () {
console.log("PhantomJS for page %s took too long to execute", page.url);
driver.quit().then(done);
}, 60 * 1000);
driver.get(page.url).then(function () {
var pageOverlay = driver.findElement(By.css("#overlay-the-new"));
return pageOverlay.isDisplayed().then(function (visible) {
if (visible) {
pageOverlay.click();
return driver.wait(until.elementIsNotVisible(pageOverlay), 10000);
}
}).then(function () {
return getAllPagesContent(driver);
});
}).then(function (contents) {
clearTimeout(processTimeout);
console.log("Got %d pages for %s", contents.length, page.url);
_.forEach(contents, function (pageContent) {
parsePage(page.url, pageContent);
});
return driver.quit();
}).then(function () {
done();
});
}

Although PhantomJS is now deprecated you can still run it in parallel isolated Docker containers by using Selenoid. There is a ready to use image with latest release here: https://hub.docker.com/r/selenoid/phantomjs/tags/

Parallel execution with Selenium tends to be done using Remote WebDrivers and the Selenium Grid2 Framework.
This tutorial at WeDoQA seems to be the sort of thing you want. At a brief glance it has each test in a separate class, while a central test base class points towards Grid2's hub, which then (in the tutorial) executes the tests in parallel using a Firefox driver. You could easily retool this to use phantomjs, but you might have to rework your test structure.

It seems you're only using one driver. I'd initialize a second driver, then use threading to run in parallel. I think this could get the job done.

Use Thread for running in parallel or you can use any test framework which can take care of running the tests in parallel.