Develop Reference

Weird (caching) issue with Express/Node

I've built an angular/express/node app that runs in google cloud which currently uses a JSON file that serves as a data source for my application. For some reason, (and this only happens in the cloud) when saving data through an ajax call and writing it to the json file, everything seems to work fine. However, when refreshing the page, the server (sometimes!) sends me the version before the edit. I can't tell whether this is an Express-related, Node-related or even Angular-related problem, but what I know for sure is that I'm checking the JSON that comes in the response from the server, and it really is sometimes the modified version, sometimes not, so it most probably isn't angular cache-related.
The GET:
router.get('/concerts', function (request, response) {
delete require.cache[require.resolve('../database/data.json')];
var db = require('../database/data.json');
response.send(db.concerts);
});
The POST:
router.post('/concerts/save', function (request, response) {
delete require.cache[require.resolve('../database/data.json')];
var db = require('../database/data.json');
var concert = request.body;
console.log('Received concert id ' + concert.id + ' for saving.');
if (concert.id != 0) {
var indexOfItemToSave = db.concerts.map(function (e) {
return e.id;
}).indexOf(concert.id);
if (indexOfItemToSave == -1) {
console.log('Couldn\'t find concert with id ' + concert.id + 'in database!');
response.sendStatus(404);
return;
}
db.concerts[indexOfItemToSave] = concert;
}
else if (concert.id == 0) {
concert.id = db.concerts[db.concerts.length - 1].id + 1;
console.log('Concert id was 0, adding it with id ' + concert.id + '.');
db.concerts.push(concert);
}
console.log("Added stuff to temporary db");
var error = commit(db);
if (error)
response.send(error);
else
response.status(200).send(concert.id + '');
});
This probably doesn't say much, so if someone is interested in helping, you can see the issue live here. If you click on modify for the first concert and change the programme to something like asd and then save, everything looks fine. But if you try to refresh the page a few times (usually even up to 6-7 tries are needed) the old, unchanged programme is shown. Any clue or advice greatly appreciated, thanks.

To solve: Do not use local files to store data in cloud! This is what databases are for!
What was actually the problem?
The problem was caused by the fact that the App Engine had 2 VM instances running for my application. This caused the POST request to be sent to one instance, it did its job, saved the data by modifying its local JSON file, and returned a 200. However, after a few refreshes, the load balancing causes the GET to arrive at the other machine, which has its individual source code, including the initial, unmodified JSON. I am now using a MongoDB instance, and everything seems to be solved. Hopefully this discourages people who attempt to do the same thing I did.

socket.io stop re-emitting event after x seconds/first failed attempt to get a response

I noticed that whenever my server is offline, and i switch it back online, it receives a ton of socket events, that have been fired while server was down. ( events that are ... by now outdated ).
Is there a way to stop socket.io from re-emitting the events after they have not received a response for x seconds ?.

When all else fails with open source libraries, you go study the code and see what you can figure out. After spending some time doing that with the socket.io source code...
The crux of the issue seems to be this code that is here in socket.emit():
if (this.connected) {
this.packet(packet);
} else {
this.sendBuffer.push(packet);
}
If the socket is not connected, all data sent via .emit() is buffered in the sendBuffer. Then, when the socket connects again, we see this:
Socket.prototype.onconnect = function(){
this.connected = true;
this.disconnected = false;
this.emit('connect');
this.emitBuffered();
};
Socket.prototype.emitBuffered = function(){
var i;
for (i = 0; i < this.receiveBuffer.length; i++) {
emit.apply(this, this.receiveBuffer[i]);
}
this.receiveBuffer = [];
for (i = 0; i < this.sendBuffer.length; i++) {
this.packet(this.sendBuffer[i]);
}
this.sendBuffer = [];
};
So, this fully explains why it buffers all data sent while the connection is down and then sends it all upon reconnect.
Now, as to how to prevent it from sending this buffered data, here's a theory that I will try to test later tonight when I have more time.
Two things look like they present an opportunity. The socket notifies of the connect event before it sends the buffered data and the sendBuffer is a public property of the socket. So, it looks like you can just do this in the client code (clear the buffer upon connect):
// clear previously buffered data when reconnecting
socket.on('connect', function() {
socket.sendBuffer = [];
});
I just tested it, and it works just fine. I have a client socket that sends an increasing counter message to the server every second. I take the server down for 5 seconds, then when I bring the server back up before adding this code, all the queued up messages arrive on the server. No counts are missed.
When, I then add the three lines of code above, any messages sent while the server is down are not sent to the server (technically, they are cleared from the send buffer before being sent). It works.
FYI, another possibility would be to just not call .emit() when the socket is not connected. So, you could just create your own function or method that would only try to .emit() when the socket is actually connected, thus nothing would ever get into the sendBuffer.
Socket.prototype.emitWhenConnected = function(msg, data) {
if (this.connected) {
return this.emit(msg, data);
} else {
// do nothing?
return this;
}
}
Or, more dangerously, you could override .emit() to make it work this way (not my recommendation).

Volatile events are events that will not be sent if the underlying connection is not ready (a bit like UDP, in terms of reliability).
https://socket.io/docs/v4/emitting-events/#volatile-events
socket.volatile.emit("hello", "might or might not be received");

How to check if the user is online using javascript or any library?

I need some help on how I could check the internet connection using Javascript or jQuery or any library if available. cause i'm developing an offline application and I want to show a version if the user is offline and another version if the user is online.
For the moment i'm using this code :
if (navigator.onLine) {
alert('online');
} else {
alert('offline');
}
But this is working very slow to detect. sometimes it's just connected to a network without internet, it takes 5 to 10 seconds to alert false (No internet).
I took a look at Offline.js library, but I'm not sure if this library is useful in my case. and I don't know how to use it

I just got this bit of code functionality from a Mozilla Site:
window.addEventListener('load', function(e) {
if (navigator.onLine) {
console.log('We\'re online!');
} else {
console.log('We\'re offline...');
}
}, false);
window.addEventListener('online', function(e) {
console.log('And we\'re back :).');
}, false);
window.addEventListener('offline', function(e) {
console.log('Connection is down.');
}, false);
They even have a link to see it working. I tried it in IE, Firefox and Chrome. Chrome appeared the slowest but it was only about half a second.

i think you should try OFFLINE.js.. it looks pretty easy to use, just give it a try.
it even provides the option checkOnLoad which checks the connection immediately on page load.
Offline.check(): Check the current status of the connection.
Offline.state: The current state of the connection 'up' or 'down'
haven't tried it, would be nice to know if it works as intended.
EDIT took a little peak into the code, it uses the method with FAILED XHR REQUEST suggested in THIS SO Question

Take a look at Detect that the Internet connection is offline? Basically, make an ajax request to something you know is likely to be up (say google.com) and if it fails, there is no internet connection.

navigator.onLine is a property that maintains a true/false value (true for online, false for offline). This property is updated whenever the user switches into "Offline Mode".
window.addEventListener('load', function() {
function updateOnlineStatus(event) {
document.body.setAttribute("data-online", navigator.onLine);
}
updateOnlineStatus();
window.addEventListener('online', updateOnlineStatus);
window.addEventListener('offline', updateOnlineStatus);
});

// check if online/offline
// http://www.kirupa.com/html5/check_if_internet_connection_exists_in_javascript.htm
function doesConnectionExist() {
var xhr = new XMLHttpRequest();
var file = "http://www.yoursite.com/somefile.png";
var randomNum = Math.round(Math.random() * 10000);
xhr.open('HEAD', file + "?rand=" + randomNum, false);
try {
xhr.send();
if (xhr.status >= 200 && xhr.status < 304) {
return true;
} else {
return false;
}
} catch (e) {
return false;
}
}

My solution is to grab a very small image (1x1), not cached and always onLine.
<head>
<script src="jquery.min.js"></script>
</head>
<body>
<script>
$( document ).ready(function() {
function onLine() {
alert("onLine")
}
function offLine() {
alert("offLine")
}
var i = new Image();
i.onload = onLine;
i.onerror = offLine;
i.src = 'http://www.google-analytics.com/__utm.gif';
});
</script>
<body>
Notes:
Use a local copy of jQuery otherwise it won't work offLine.
I've tested the code onLine/offLine and it works without delay.
Works with all browsers, Desktop or Mobile.
In case you wonder, there's no tracking made from Google Analytics as we don't use any arguments.
Feel free to change the image, just make sure it doesn't get cached and it's small in size.

Try utilizing WebRTC , see diafygi/webrtc-ips; in part
Additionally, these STUN requests are made outside of the normal
XMLHttpRequest procedure, so they are not visible in the developer
console or able to be blocked by plugins such as AdBlockPlus or
Ghostery. This makes these types of requests available for online
tracking if an advertiser sets up a STUN server with a wildcard
domain.
modified minimally to log "online" or "offline" at console
// https://github.com/diafygi/webrtc-ips
function online(callback){
//compatibility for firefox and chrome
var RTCPeerConnection = window.RTCPeerConnection
|| window.mozRTCPeerConnection
|| window.webkitRTCPeerConnection;
var useWebKit = !!window.webkitRTCPeerConnection;
//bypass naive webrtc blocking using an iframe
if(!RTCPeerConnection) {
//NOTE: you need to have an iframe in the page
// right above the script tag
//
//<iframe id="iframe" sandbox="allow-same-origin" style="display: none"></iframe>
//<script>...getIPs called in here...
//
var win = iframe.contentWindow;
RTCPeerConnection = win.RTCPeerConnection
|| win.mozRTCPeerConnection
|| win.webkitRTCPeerConnection;
useWebKit = !!win.webkitRTCPeerConnection;
}
//minimal requirements for data connection
var mediaConstraints = {
optional: [{RtpDataChannels: true}]
};
//firefox already has a default stun server in about:config
// media.peerconnection.default_iceservers =
// [{"url": "stun:stun.services.mozilla.com"}]
var servers = undefined;
//add same stun server for chrome
if(useWebKit)
servers = {iceServers: [{urls: "stun:stun.services.mozilla.com"}]};
//construct a new RTCPeerConnection
var pc = new RTCPeerConnection(servers, mediaConstraints);
//create a bogus data channel
pc.createDataChannel("");
var fn = function() {};
//create an offer sdp
pc.createOffer(function(result){
//trigger the stun server request
pc.setLocalDescription(result, fn, fn);
}, fn);
//wait for a while to let everything done
setTimeout(function(){
//read candidate info from local description
var lines = pc.localDescription.sdp.split("\n");
// return `true`:"online" , or `false`:"offline"
var res = lines.some(function(line) {
return line.indexOf("a=candidate") === 0
});
callback(res);
}, 500);
}
//Test: Print "online" or "offline" into the console
online(function(connection) {
if (connection) {
console.log("online")
} else {
console.log("offline")
}
});

You can use SignalR, if you're developing using MS web technologies. SignalR will establish either long polling or web sockets depending on your server/client browser technology, transparent to you the developer. You don't need to use it for anything else than determining if you have an active connection to the site or not.
If SignalR disconnects for any reason, then you have lost connection to the site, as long as your SignalR server instance is actually installed on the site. Thus, you can use $.connection.hub.disconnected() event/method on the client to set a global var which holds your connection status.
Read up about SignalR and how to use it for determining connection states here...
http://www.asp.net/signalr/overview/guide-to-the-api/handling-connection-lifetime-events#clientdisconnect

See How do I check connection type (WiFi/LAN/WWAN) using HTML5/JavaScript? answers:
Rob W suggests navigator.connection;
Bergi suggests Windows.Networking.Connectivity API through this tutorial;
Gerard Sexton suggests Gmail approach.

You can use the new Fetch API which will trigger an error almost immediately if no network is present.
The problem with this is that the Fetch API has infant support at the moment (currently Chrome has the most stable implementation, Firefox and Opera is getting there, IE does not support it). There exists a polyfill to support the fetch principle but not necessarily the rapid return as with a pure implementation. On the other hand, an offline app would require a modern browser...
An example which will try to load a plain text file over HTTPS to avoid CORS requirements (link is picked at random, you should set up a server with a tiny text file to test against - test in Chrome, for now):
fetch("https://link.to/some/testfile")
.then(function(response) {
if (response.status !== 200) { // add more checks here, ie. 30x etc.
alert("Not available"); // could be server errors
}
else
alert("OK");
})
.catch(function(err) {
alert("No network"); // likely network errors (incl. no connection)
});
Another option is to set up a Service worker and use fetch from there. This way you could serve an optional/custom offline page or a cached page when the requested page is not available. Also this is a very fresh API.

best one liner
console.log(navigator.onLine ? 'online' : 'offline');

Sending mouse clicks to a differnet computer using webrtc

I am trying to connect computer "a" to computer "b" using webrtc and print out the "Click" on computer "b" when the mouse is clicked on computer "a"'s canvas. I already created a working webrtc example where I make a connection between computer "a" and "b" and send messages between them using textboxes(chat).
I know to Attach a click event to the document. When the user clicks anywhere in the document, output "Click" will be displayed.
document.addEventListener("click", function(){
message.value= "Click!";
});
And these are some of the webrtc functions I have, I didnt post all my webRTC functions because I dont wanna make the question longer, it already is.
// a nice wrapper to send data
function send (room, key, data) {
roomRef.child(room).child(key).set(data);
}
// wrapper function to receive data
function recv (room, type, cb) {
roomRef.child(room).child(type).on("value", function (snapshot, key) {
var data = snapshot.val();
if (data) { cb(data); }
});
}
// get references to the document tags
var chatlog = document.getElementById("chatlog");
var message = document.getElementById("message");
function bindEvents () {
channel.onopen = function () { console.log("Channel Open"); }
channel.onmessage = function (e) {
// add the message to the chat log
chatlog.innerHTML += "<div>Peer says: " + e.data + "</div>";
};
}
// send a message the textbox throught
// the data channel for a chat program
function sendMessage () {
var msg = message.value;
channel.send(msg);
message.value = "";
}
My question is I dont know how to connect these two codes together or even if i did, I am not sure if it would work. So my question is how can I click on the canvas on computer "a" and get the textbox to print out "Click" on computer "b".
Thanks for reading

You could look into node.js and socket.io.
With these two you could connect multiple clients together and have a real-time communication between them. Other alternative is to use ajax with php, and make one browser to poll for new commands from server, and other browser to send them to server.

You're mostly there. What you can do is, after setting the message.value property, is call the sendMessage() function. This should trigger the application to send the correct value through the WebRTC Connection.

strophe.js and openfire who terminates connection on page reload?

I am using symfony2 to build some app. In that app I have chat app. I am using attached session in chat.
1) On login I fire up event listener to catch user/pass from login, connect to openfire server and get sid and rid.
2) After that i am storing that data in session so I can use them later on every page where I have chat.
Problem occurs when page is reloaded/refreshed.
My guess this is because ajax request to url:7070/httpd-bind is canceled strophe sends terminate to openfire server. Bu I can not find anywhere terminate stanza.
I am have patched strophe.js to use sync on page unload but again that is not working.
Chat.connection.flush();
Chat.connection.sync = true; // Switch to using synchronous requests since this is typically called onUnload.
Chat.connection.disconnect();
Please suggest solution for this, I am on 10 hour coding and I have no idea how to solve this.
I can sotre user/pass in session but that is just stupid. Why attached session exists if I have to do that.

UPDATE
After trying to figure about this rid plus+1 etc I noticed that rid is changing on presence, on message on message sent on roster on roster change so I made a XMLHttpRequest on each to remember new rid in session. For some reason localstorage is sometimes working sometimes not.
Now i have rid up to date all the time.
I think I got this. Problem was in rid and presence.
1) First you have to figure out from your logs if your rid is increasing or decreasing.
My was decreasing by one. So I substract -1 from my Chat.connection.rid
2) In my openfire logs I figured out that I was sending unavailable status on page refresh
so I changed my window.unload function to send presence to online. N
Now I am refreshing page million times and i never got disconnected.
Now I just have to figure out how to remember connection.rid to localStorage for non HTML browsers.
To start openfire in debug mode you just add ./openfire.sh -debug. Then you will be able to se everything in debug.log
This did trick for me. If this is doing trick for you please +1 and accept answer.
Do not forget to terminate session on logout :)
UPDATE
This is my on window.onunload function
window.onunload = function(ev){
var initialPresence = $pres().c('show').t("cao").up().c('status').t("sad");
Chat.connection.send(initialPresence);
store.set('session_rid', parseInt(Chat.connection.rid)-1);
//save rooster contacts state
var contacts = document.getElementById('records').getElementsByTagName('li');
var id_value;
var class_value;
var status;
var el;
for(i= 0; i < contacts.length; i++){
el = contacts[i].getElementsByClassName("mood")[0];
status = el.textContent || el.innerText;
Array.prototype.slice.call(contacts[i].attributes).forEach(function(item) {
if(item.name == "id"){
id_value = item.value;
}
if(item.name == "class"){
class_value = item.value;
}
store.set('user'+i, { id: id_value, class_name: class_value, status : status });
});
}
Chat.disconnect();
}
This is my on window.onload function
window.onload = function(){
if(store.get("session_rid")){
var obj;
var id;
var class_name;
var status;
store.forEach(function(val, key) {
if(val !== "session_rid"){
setTimeout(function(){
obj = eval(key);
id = obj.id;
class_name = obj.class_name;
status = obj.status;
if(document.getElementById(id)){
document.getElementById(id).className = class_name;
document.getElementById(id).getElementsByClassName("mood")[0].innerHTML = "<span>"+status+"</span>";
}
}, 1000);
}
})
}
}
This is working for me. I used store.js to store data so it can work on IE.
I used attached sessions.
//json is from ajax call on some php script that has started attached session on user login
var obj = JSON.parse(json);
connection = new Strophe.Connection(BOSH_SERVICE);
connection.attach(obj.fulljid,
obj.sid,
(store.get("session_rid") ? store.get("session_rid"):obj.rid),
justDoIt);
full_jid = obj.fulljid;