I want to do some stress test against my websocket server runnig this javascript example in my browser (chrome) below:
function create_ws() {
var ws=new WebSocket("ws://127.0.0.1/");
ws.onopen=function(evt){
var binary = new Uint8Array(2);
binary[0]=1;
binary[1]=2;
ws.send(binary.buffer);
};
ws.onclose=function(evt){};
ws.onmessage=function(evt){};
ws.onerror=function(evt){};
}
var connections = [];
var numberOfconnections=100;
for(var i = 0; i < numberOfconnections; i++) {
connections.push(create_ws());
}
The problem is the script above let me run only about 100 connections at the same time. If i increase numberOfconnections to 300 it throws following error:
WebSocket connection to 'ws://127.0.0.1/' failed: Error in connection establishment: net::ERR_INSUFFICIENT_RESOURCES
Is there a way to increase the number of websocket connections in browser?
Try to open new tabs with your stress test manually or with window.open in code.
Looks like the limit for chromium is set to 256 per the discussion below:
Chromium Discussion
It's possible other browsers have different limits. As I only use so many connections for stress testing, I plan to open a few windows to go outside the limit.
Related
I'm working on adding WebAuthn support to a newly-minted web site and am running into a problem during the navigator.credentials.get() call. The client is Firefox 85.0 on Fedora 33. In case it matters, the server is Apache httpd on Fedora 33. The token is either a Yubikey 4 or a Yubikey 5NFC (the results are the same). This is the function making the API call. Obviously the credential IDs hard-coded here are for testing, not part of the final product:
function handleUserAuthenticationResponse(r) {
var cid1 = {type: "public-key", id: base64ToArrayBuffer("gL0Ig10uA2tn8L0kn2L9FoGqIPSrqvc1lLBwgQhcVDa200b1P94kPv94T6O1bDZyYRrfLbTrLRsubDxuYUxHCg==")};
var cid2 = {type: "public-key", id: base64ToArrayBuffer("tjW1RPqtAJm69I/qeV7eRFJx6h87J3NPeJ/hhbkjttcCc2BWHQ2v2nueoKBSGabw1eYsT8S+lhJv1l1mYWX+Uw==")};
var options = {
rpID: "http://localhost",
challenge: base64ToArrayBuffer(r.challenge),
allowCredentials: [cid1,cid2],
timeout: 60000
};
if (!window.PublicKeyCredential) {
throw new Error("Unable to access credentials interface");
}
navigator.credentials.get({"publicKey":options})
.then(assertion => handleTokenAssertion(assertion))
.catch(e => {console.log("Error fetching token assertion:",e);});
}
function base64ToArrayBuffer(base64) {
var binary_string = window.atob(base64);
var len = binary_string.length;
var bytes = new Uint8Array(len);
for (var i = 0; i < len; i++) {
bytes[i] = binary_string.charCodeAt(i);
}
return bytes.buffer;
}
function handleTokenAssertion(a) {
alert("Got an assertion!");
}
Everything seems to work, the Yubikey LED blinks, I press the touchpad, but then I get back an exception:
Error fetching token assertion: DOMException: An attempt was made to use an object that is not, or is no longer, usable
This seems to be a bit of a Firefox catch-all. It could indicate that the token doesn't match one of the allowedCredentials[], or perhaps other things. It's hard to tell. The FIDO2 credential on the Yubikey was created with fido2-cred(1) tool packaged with the libfido2 source. In this case the credentialId is from the fido2-cred -M output:
CuCEGL10uPhBmNCY4NsGaAz0gir/68UMGFQn0pfb6tc=
http://localhost
fido-u2f
WMSGBbi6CMINQvnkVRUYcYltDg3pgFlihvtzbRHuwBPipEEAAAAAAAAAAAAAAAAAAAAAAAAAAABAgL0Ig10uA2tn8L0kn2L9FoGqIPSrqvc1lLBwgQhcVDa200b1P94kPv94T6O1bDZyYRrfLbTrLRsubDxuYUxHCqUBAgMmIAEhWCA5itRRCBO0lnsztvPvI1waVZLBCZ1XMJjOvlN2oZmBCyJYILFaRjThs5Paj1sOp81iID1LpUBYHJhp4dizC0eI/RrE
gL0Ig10uA2tn8L0kn2L9FoGqIPSrqvc1lLBwgQhcVDa200b1P94kPv94T6O1bDZyYRrfLbTrLRsubDxuYUxHCg==
MEQCIFfs8PagKhNnDgzxfurVzdkTDVTT6ixKk0ak/2qrbSPUAiAf64w390rX1cyY58JgSC/Ac97w6TLcYKuqxOSn5lxV0g==
<long assertion certificate>
You can see the credentialId on line 5, and that it matches cid1 in the Javascript function. Furthermore, if I request an assertion from the token using this credentialId and all else identical (except the challenge) with fido2-assert -G, everything works fine: I get the assertion and it verifies correctly using fido2-assert -V.
Without a more meaningful diagnostic it's hard to know what to try, so I thought I would ask here and see if anyone has any hints. Perhaps I've made some basic error with either Javascript or the credentials API?
Thanks!
UPDATE: One possibility I thought might be worth trying was removing the scheme from the RP ID but that made no difference.
UPDATE: Looking at the firefox source code, the error is apparently NS_ERROR_DOM_INVALID_STATE_ERR, which covers several different situations but in this case is most likely a translation of U2F_ERROR_INVALID_STATE (in dom/webauthn/U2FHIDTokenManager.h). U2F_ERROR_INVALID_STATE, in turn, is defined in third_party/rust/authenticator/src/u2fhid-capi.h as a simple numerical value (3), with no indication of where the value came from. Perhaps it's defined by the underlying HID driver for the Yubikey, but it's not clear what driver that corresponds to. The hunt continues...
It turns out that the problem was indeed the format of the relying party ID. Based on example code from the net (which may have worked with other browsers or versions of the code?), I initially used the full scheme://domain format for the rpID (so in my code above, http://localhost), but it turns out that what's needed is just the domain (localhost). Modifying the rpID in this way allows the assertion process to succeed.
Initially I thought this did not work, but it turned out that I'd simply forgotten to commit the change. Having belatedly done that, it works.
We were using Redis for a plenty of time until we have come to the conclusion that moving to KeyDB may be a good choice for its features.
Environment
OS: Centos 7
NodeJs: v12.18.0
Redis: v6.0.5
Targeted KeyDB: v0.0.0 (git:1069d0b4) // keydb-cli -v showed this. Installed Using Docker.
ioredis: v4.17.3
pm2: v4.2.1 // used for clustering my application.
Background
Referring to the KeyDB documentation, KeyDB is compatible with the latest version of Redis.
KeyDB remains fully compatible with Redis modules API and protocol. As such a migration from Redis to KeyDB is very simple and would be similar to what you would expect migrating in a Redis to Redis scenario. https://docs.keydb.dev/docs/migration/
In the same page they provide a list of redis clients which are compatible with KeyDB. The list contains ioredis which I'm using.
KeyDB is compatible with all Redis clients as listed here so this should not be a concern. Simply use your client as you would with Redis.
https://docs.keydb.dev/docs/migration/
Problem
As said in the documentation. I should be able to migrate easily to KeyDB in a few hours. Well that is not the case! At least not for me! I spent my last 3 days searching on the internet for the solution. I came to the conclusion that I should write to stackoverflow :)
The issue is somehow interesting. The Client is actually working with KeyDb and the process is actually setting and retrieving keys (Not sure but may lose some data during the error.). But On 10% of time it gives me the following error, And continues to work again after a while. As I'm using Redis for storing sessions and other stuff on my production environment; I can not risk ignoring such insisting error.
error: message=write EPIPE, stack=Error: write EPIPE
./app-error-1.log:37: at WriteWrap.onWriteComplete [as oncomplete] (internal/stream_base_commons.js:92:16), errno=EPIPE, code=EPIPE, syscall=write
I searched nearly all the internet for this error but no one provides a solution nor an explanation for what is going wrong.
Luckily the process "sometimes" shows a stack for the error. It points to lib/redis/index.ts:711 inside the ioredis codes. Which I have no idea what it does.
(stream || this.stream).write(command.toWritable());
https://github.com/luin/ioredis/blob/master/lib/redis/index.ts#L711
I found some issues on ioredis github repository mentioning some EPIPE error. But most of them were about error handling stuff and all marked as resolved.
I also found some general EPIPE errors on google(Most of them about socket.io which is not something I use.)
Wrap Up
What is wrong with this thing?
As no one wrote an answer at the end of bounty. I am writing my experience on solving the issue for the people who will get this error later.
Please note that this is not a canonical answer. But It is rather a workaround
I am starting with sharing what was happening.
We were trying to migrate from a Redis server hosting nearly 600 000 keys. Standard migration process was taking a lot of time for transfer that amount of keys from Redis to keyDB. So I came across a different solution.
Our KeyDB works on 2 Active-Active replica servers. I will provide the link to those are wondering how this system works.
https://medium.com/faun/failover-redis-like-cluster-from-two-masters-with-keydb-9ab8e806b66c
The solution was re-building up our Redis data using some MongoDB database aggregation and doing some batch operations on KeyDB.
Here is a simulation(Not exact as source. Also I did not tested for syntax errors)
const startPoint =
(Number.parseInt(process.env.NODE_APP_INSTANCE) || 0) * 40000;
let skip = 0 + startPoint;
let limit = 1000;
let results = await SomeMongooseSchema.find({someQueries}).limit(1000).skip(skip);
let counter = 0;
while (results.length){
if(counter > 39) break;
for(const res of results){
const item = {
key: '',
value: ''
};
// do some build ups on item
...
// end n
app.ioRedisClient.set(item.key, item.value);
}
counter++;
skip = i * limit + startPoint;
results = await SomeMongooseSchema.find({someQueries}).limit(limit).skip(skip);
}
Running this code on 16 processes using pm2 sets all the keys to keyDB in about 45 minutes. (compared to 4-5 hours)
pm2 start app.js -i 16
When we run the code on a Redis server. It works but giving the following error on KeyDB.
error: message=write EPIPE, stack=Error: write EPIPE
./app-error-1.log:37: at WriteWrap.onWriteComplete [as oncomplete] (internal/stream_base_commons.js:92:16), errno=EPIPE, code=EPIPE, syscall=write
Firstly I started by tuning the code by creating a transaction instead of setting every key separately. and setted a 1-second gap between each 1000 operation. the code changed as following.
const startPoint =
(Number.parseInt(process.env.NODE_APP_INSTANCE) || 0) * 40000;
let skip = 0 + startPoint;
let limit = 1000;
let results = await SomeMongooseSchema.find({someQueries}).limit(1000).skip(skip);
const batch = app.ioredisClient.multi();
let counter = 0;
while (results.length){
if(counter > 39) break;
for(const res of results){
const item = {
key: '',
value: ''
};
// do some build ups on item
...
// end n
batch.set(item.key, item.value);
}
counter++;
await batch.exec();
await sleep();
skip = i * limit + startPoint;
results = await SomeMongooseSchema.find({someQueries}).limit(limit).skip(skip);
}
This reduced the error rate as long as the operation time to 20 minutes. But the error was still persisting.
I suspected that this error may be due to some permission errors on the docker version. So I asked our server administrator to check and if possible remove the docker version and install from rpm repository.
https://download.keydb.dev/packages/rpm/centos7/x86_64/
Did that and it worked. All the errors vanished and successfully managed to migrate in 20 minutes.
I don't consider this as a real answer. But this should be useful for some experts to find out what was wrong.
I am trying to allocate memory in JavaScript to study memory leak/consumption using the code snippet below. However
performance.memory.usedJSHeapSize
always shows the same number, 10000000 in my case. How come that number never changes despite dynamically creation of elements and attaching to DOM ?
I need a JavaScript snippet to create memory leak and monitor the usage using performance.memory.usedJSHeapSize dynamically( or any other functions if exists).
I tried this code but performance.memory.usedJSHeapSize remains at 10000000:
<body>
<p id="memory" style="position: fixed; top:10px; left:10px"></p>
<script>
setInterval(() => {
document.getElementById("memory").innerHTML = performance.memory.usedJSHeapSize
}, 300);
btn = [];
let i = 0;
setInterval(() => {
for (let j = 0; j < 1000; j++) {
++i;
let k=i;
btn[k] = document.createElement("BUTTON");
document.body.appendChild(btn[k]);
btn[k].innerHTML = k;
btn[k].addEventListener("click", function () {
alert(k);
});
}
}, 5000);
</script>
</body>
I already tired the example given in 2013 in this post, but this one no longer create memory leak either.
How do I create a memory leak in JavaScript?
performance.memory.usedJSHeapSize does not update when the page is opened directly from the local file system.
The image below shows that the exact same code copy-pasted from the question shows increasing memory usage when accessed at localhost:
Or, you can check for yourself: https://memory-leak.surge.sh/simple/ (You can also check the original code: https://memory-leak.surge.sh/ but your browser might freeze up if left open for more than a few seconds.)
How to host the HTML like I did above:
The simplest option is to use dev tools like Browsersync or Parcel. These tools will let you open files from your local file system as if they were hosted from a server with a URL like http://localhost:1234/ . (Because a temporary web server is started on your computer.)
Another option is to actually host the files on a server. There are many options to do this:
surge The tool I used for the examples above
Glitch (This one is cool because you can edit the files online and see changes right away)
Github pages
Note: results may vary based on browser/hardware. My environment:
Chrome Version 74.0.3729.131 (Official Build) (64-bit)
Windows 10
I have a legacy application which is making use of ActiveX control for serial communication. This is rightly communicating in IE.
Since ActiveX is IE specific technology obviously it will not run in Chrome or other browser
So I don't want to disturb the IE functionality and thought of introducing browser specific function as below.
$(document).ready(function(e){
var isIEBrowserFlag = true;
if(isIEBrowserFlag)
{
var obj = new ActiveXObject("MSCommLib.MSComm");
obj.CommPort = commPort;
obj.RThreshold = thresHold;
obj.Settings = settings;
obj.PortOpen = true;
obj.DTREnable = true;
obj.Output = "test";
obj.PortOpen = false;
//other stuff
}
else
{
//chrome
}
//sendBagToPrinter(obj);
});
On googling I came to know about jQuery.parseXML() but how do I implement the same functionality as of IE in Chrome using $.parseXML()
Similar other plugins such as juart were tried, but not fitting my requirement.
chrome.serial API provides access to client serial ports, but from a Chrome App.
My suggestion, for future support, use/write some kind of server for serial device and connect from browser to server via a json type communication. See https://github.com/johnlauer/serial-port-json-server
I'm building an app that contains a WebView that runs some JavaScript code. That JavaScript code is quite allocation heavy and can require a lot of memory.
Sometimes, the amount of required memory exceeds the amount JavaScript can require and crashes the Chromium process of the WebView which crashes my app.
I listen to onMemoryTrim in my application - but it is never called in this scenario on devices with more than 1GB of memory. (Not even with TRIM_MEMORY_RUNNING_LOW).
Is there any way I could detect my WebView is running low on memory and either kill it or let it know (so it can free memory)?
I've tried polling performance.memory but it did not work. The following script crashes the WebView if executed in it:
var a = [];
var kek = () => {
var b = [];
for(var i = 0; i < 1024 * 1024 * 2; i++) b.push(Math.random());
return b;
}
var ival = setInterval(() => {
let m = performance.memory;
if(m.jsHeapSizeLimit - m.usedJSHeapSize < 1e5) {
console.log("Memory limited")
} else {
a.push(kek());
}
});
Is there any way to detect memory is about to run out so I can handle it gracefully without the app crashing?
I have discussed this with the Chromium team and the Android team and at the moment (they think and I believe them) that this is impossible.
Sometimes, the amount of required memory exceeds the amount JavaScript can require and crashes the Chromium process of the WebView which crashes my app.
You can however catch out of memory crashes in Android 8.0+ using the new termination handle API. So this works around my problem by not having to check the available memory required in the first place.
By overriding onRenderProcessGone - we get to catch the bug and recreate the WebView.