Develop Reference

How to replace ajax with webrtc data channel

** JAVASCRIPT question **
I'm using regularly ajax via XMLHttpRequest. But in 1 case, I need 1 ajax call per seconds....
but long term wise and with growing number of simultaneous users, it could bloat easily...
I'm reading stuff about webRTC data channel and it seems interesting and promissing.
Here my working AJAX function as an example of how easy and there is a few lines of codes to communicate from the browser to the server and vice-versa
function xhrAJAX ( divID , param2 ) {
// random value for each call to avoid cache
var pcache = (Math.floor(Math.random() * 100000000) + 1);
// parameters
var params = "divID="+encodeURIComponent(divID)+"&param2="+encodeURIComponent(param2);
// setup XMLHttpRequest with pcache
var xhr = new XMLHttpRequest();
xhr.open("POST", "/file.php?pcache="+pcache, true);
// setup headers
xhr.setRequestHeader("Content-Type", "application/x-www-form-urlencoded");
// prepare onready scripts
xhr.onreadystatechange = function(e) { if (xhr.readyState == 4) { $("#"+divID).html(e.currentTarget.responseText) ; } }
// send the ajax call
xhr.send(params);
}
How can I "transpose" or "convert" this ajax workflow into a webRTC data channel ? in order to avoid to setup a setInterval 1000...
Note: I mean how to replace the javascript portion of the code. PHP here is only to illustrate, I don't want to do a webRTC via PHP...
Is there a simple few lines of code way to push/receive data like this ajax function ?
the answer I'm looking for is more like a simple function to push and receive
(once the connection with STUN, ICE, TURN is established and working...)
If I need to include a javascript library like jquery or the equivalent for webRTC, I'm welcoming good and simple solution.
*** The main goal is this kind of scenario :
I have a webapp : users in desktop and users within webview in Android and IOS
right now I have this workflow => ajax every 3 seconds to "tell" the main database that the user is still active and using the browser (or the app)
But I'd like to replace with this kind : when the user uses the browser => do a webrtc data chata in background between the browser and the server
While reading on the web I think that webRTC is a better solution than websocket.
** I did a bit of search and found peerjs....
https://github.com/jmcker/Peer-to-Peer-Cue-System/blob/main/send.html
I'll do some testing, but in the meantime, if someone can trow ideas, it could be fun.
Cheers

ERR_INSUFFICIENT_RESOURCES ajax code for REAL-TIME CLOCK

I created a real-time clock that updates every second. When I run it locally I see no errors. However when I uploaded it to my web host I saw a lot of error messages in the console every time the AJAX code runs.
I think that's the reason why web host suspended my site, telling that my site has performed too many requests.
Here is my code:
$(document).ready(function() {
setInterval(function() {
$('#time').load('timewithdate.php')
}, 1000);
setInterval(function(){
$('#time2').load('time.php')
}, 1000);
})

I'm not sure what kind of answer you are looking for. Your code is a certain way to kill a server: it's making 2 calls to the server every second for each client (read more about DDOS). There is no need to make a server call, just use javascript to get the current time and format it the way you want. You can have something like this:
$(document).ready(function() {
setInterval(function() {
let curTime = new Date();
let date = curTime.toLocaleDateString();
let time = curTime.toLocaleTimeString();
$('#time').text(date + " " + time);
$('#time2').text(time);
}, 1000);
})

The error is because you have effectively DDOS'd your own server with 2 requests every second * number of concurrent clients. I would strongly suggest you remove these AJAX requests and perform the countdown on the client side.
If you're trying to keep the clocks in sync with server time, get the time from the server when the page loads, then add seconds to it on the client side. Do not use AJAX for this, and do not use AJAX polling in future. It's an anti-pattern.

Weird (caching) issue with Express/Node

I've built an angular/express/node app that runs in google cloud which currently uses a JSON file that serves as a data source for my application. For some reason, (and this only happens in the cloud) when saving data through an ajax call and writing it to the json file, everything seems to work fine. However, when refreshing the page, the server (sometimes!) sends me the version before the edit. I can't tell whether this is an Express-related, Node-related or even Angular-related problem, but what I know for sure is that I'm checking the JSON that comes in the response from the server, and it really is sometimes the modified version, sometimes not, so it most probably isn't angular cache-related.
The GET:
router.get('/concerts', function (request, response) {
delete require.cache[require.resolve('../database/data.json')];
var db = require('../database/data.json');
response.send(db.concerts);
});
The POST:
router.post('/concerts/save', function (request, response) {
delete require.cache[require.resolve('../database/data.json')];
var db = require('../database/data.json');
var concert = request.body;
console.log('Received concert id ' + concert.id + ' for saving.');
if (concert.id != 0) {
var indexOfItemToSave = db.concerts.map(function (e) {
return e.id;
}).indexOf(concert.id);
if (indexOfItemToSave == -1) {
console.log('Couldn\'t find concert with id ' + concert.id + 'in database!');
response.sendStatus(404);
return;
}
db.concerts[indexOfItemToSave] = concert;
}
else if (concert.id == 0) {
concert.id = db.concerts[db.concerts.length - 1].id + 1;
console.log('Concert id was 0, adding it with id ' + concert.id + '.');
db.concerts.push(concert);
}
console.log("Added stuff to temporary db");
var error = commit(db);
if (error)
response.send(error);
else
response.status(200).send(concert.id + '');
});
This probably doesn't say much, so if someone is interested in helping, you can see the issue live here. If you click on modify for the first concert and change the programme to something like asd and then save, everything looks fine. But if you try to refresh the page a few times (usually even up to 6-7 tries are needed) the old, unchanged programme is shown. Any clue or advice greatly appreciated, thanks.

To solve: Do not use local files to store data in cloud! This is what databases are for!
What was actually the problem?
The problem was caused by the fact that the App Engine had 2 VM instances running for my application. This caused the POST request to be sent to one instance, it did its job, saved the data by modifying its local JSON file, and returned a 200. However, after a few refreshes, the load balancing causes the GET to arrive at the other machine, which has its individual source code, including the initial, unmodified JSON. I am now using a MongoDB instance, and everything seems to be solved. Hopefully this discourages people who attempt to do the same thing I did.

Is it possible to control Firefox's DNS requests in an addon?

I was wondering if it was possible to intercept and control/redirect DNS requests made by Firefox?
The intention is to set an independent DNS server in Firefox (not the system's DNS server)

No, not really. The DNS resolver is made available via the nsIDNSService interface. That interface is not fully scriptable, so you cannot just replace the built-in implementation with your own Javascript implementation.
But could you perhaps just override the DNS server?
The built-in implementation goes from nsDNSService to nsHostResolver to PR_GetAddrByName (nspr) and ends up in getaddrinfo/gethostbyname. And that uses whatever the the system (or the library implementing it) has configured.
Any other alternatives?
Not really. You could install a proxy and let it resolve domain names (requires some kind of proxy server of course). But that is a very much a hack and nothing I'd recommend (and what if the user already has a real, non-resolving proxy configured; would need to handle that as well).

You can detect the "problem loading page" and then probably use redirectTo method on it.
Basically they all load about:neterror url with a bunch of info after it. IE:
about:neterror?e=dnsNotFound&u=http%3A//www.cu.reporterror%28%27afew/&c=UTF-8&d=Firefox%20can%27t%20find%20the%20server%20at%20www.cu.reporterror%28%27afew.
about:neterror?e=malformedURI&u=about%3Abalk&c=&d=The%20URL%20is%20not%20valid%20and%20cannot%
But this info is held in the docuri. So you have to do that. Here's example code that will detect problem loading pages:
var listenToPageLoad_IfProblemLoadingPage = function(event) {
var win = event.originalTarget.defaultView;
var docuri = window.gBrowser.webNavigation.document.documentURI; //this is bad practice, it returns the documentUri of the currently focused tab, need to make it get the linkedBrowser for the tab by going through the event. so use like event.originalTarget.linkedBrowser.webNavigation.document.documentURI <<i didnt test this linkedBrowser theory but its gotta be something like that
var location = win.location + ''; //I add a " + ''" at the end so it makes it a string so we can use string functions like location.indexOf etc
if (win.frameElement) {
// Frame within a tab was loaded. win should be the top window of
// the frameset. If you don't want do anything when frames/iframes
// are loaded in this web page, uncomment the following line:
// return;
// Find the root document:
//win = win.top;
if (docuri.indexOf('about:neterror') == 0) {
Components.utils.reportError('IN FRAME - PROBLEM LOADING PAGE LOADED docuri = "' + docuri + '"');
}
} else {
if (docuri.indexOf('about:neterror') == 0) {
Components.utils.reportError('IN TAB - PROBLEM LOADING PAGE LOADED docuri = "' + docuri + '"');
}
}
}
window.gBrowser.addEventListener('DOMContentLoaded', listenToPageLoad_IfProblemLoadingPage, true);

How to process streaming HTTP GET data?

Right now, I have a node.js server that's able to stream data on GET request, using the stream API. The GET request is Transfer-encoded set to 'chunked'. The data can be on the order of 10 to 30 MBs. (They are sometimes 3D models)
On the browser side, I wish to be able to process the data as I'm downloading it--I wish to be able to display the data on Canvas as I'm downloading it. So you can see the 3D model appear, face by face, as the data is coming in. I don't need duplex communication, and I don't need a persistent connection. But I do need to process the data as soon as it's downloaded, rather than waiting for the entire file to finish downloading. Then after the browser downloads the data, I can close the connection.
How do I do this?
JQuery ajax only calls back when all the data has been received.
I also looked at portal.js (which was jquery-streaming) and socket.io, but they seem to assume persistent reconnection.
So far, I was able to hack a solution using raw XMLHttpRequest, and making a callback when readyStead >= 2 && status == 200, and keeping track of place last read. However, that keeps all the data downloaded in the raw XMLHttpRequest, which I don't want.
There seems to be a better way to do this, but I'm not sure what it is. Any one have suggestions?

oboe.js is a library for streaming responses in the browser.
However, that keeps all the data downloaded in the raw XMLHttpRequest, which I don't want.
I suspect this may be the case with oboe.js as well and potentially a limitation of XMLHttpRequest itself. Not sure as I haven't directly worked on this type of use case. Curious to see what you find out with your efforts and other answers to this question.

So I found the answer, and it's Server-sent events. It basically enables one-way http-streams that the browser can handle a chunk at a time. It can be a little tricky because some existing stream libs are broken (they don't assume you have \n in your stream, and hence you get partial data), or have little documentation. But it's not hard to roll your own (once you figure it out).
You can define your sse_transform like this:
// file sse_stream.coffee
var Transform = require('stream').Transform;
var util = require('util');
util.inherits(SSEStream, Transform);
function SSEStream(option) {
Transform.call(this, option);
this.id = 0;
this.retry = (option && option.retry) || 0;
}
SSEStream.prototype._transform = function(chunk, encoding, cb) {
var data = chunk.toString();
if (data) {
this.push("id:" + this.id + "\n" +
data.split("\n").map(function (e) {
return "data:" + e
}).join("\n") + "\n\n");
//"retry: " + this.retry);
}
this.id++;
cb();
};
SSEStream.prototype._flush = function(next) {
this.push("event: end\n" + "data: end" + "\n\n");
next();
}
module.exports = SSEStream;
Then on the server side (I was using express), you can do something like this:
sse_stream = require('sse_stream')
app.get '/blob', (req, res, next) ->
sse = new sse_stream()
# It may differ here for you, but this is just a stream source.
blobStream = repo.git.streamcmd("cat-file", { p: true }, [blob.id])
if (req.headers["accept"] is "text/event-stream")
res.type('text/event-stream')
blobStream.on("end", () -> res.removeAllListeners()).stdout
.pipe(
sse.on("end", () -> res.end())
).pipe(res)
else
blobStream.stdout.pipe(res)
Then on the browser side, you can do:
source = new EventSource("/blob")
source.addEventListener('open', (event) ->
console.log "On open..."
, false)
source.addEventListener('message', (event) ->
processData(event.data)
, false)
source.addEventListener('end', (event) ->
console.log "On end"
source.close()
, false)
source.addEventListener('error', (event) ->
console.log "On Error"
if event.currentTarget.readyState == EventSource.CLOSED
console.log "Connection was closed"
source.close()
, false)
Notice that you need to listen for the event 'end', that is sent from the server in the transform stream's _flush() method. Otherwise, EventSource in the browser is just going to request the same file over and over again.
Note that you can use libraries on the server side to generate SSE. On the browser side, you can use portal.js to handle SSE. I just spelt things out, so you can see how things would work.