I started to learn websockets today, since I want to have an architecture, with which I can get real-time updates.
I created my first websockets node.js and socket.io app, so I can communicate via javascript between a client and server. That works fine.
But I need something, that can communicate with a MySQL, so that for every change on a certain table, it has to tell the clients, that there is a change.
So I was thinking that the node.js server communicates with a PHP script that observes the database. But then I would also need long-pulling to request changes, so I could do it with ajax anyway, so it would be useless.
So my question: How can I achieve to get real-time data changes from a table of the database or from a certain query, which will send the update to all clients, that are connected in real-time, without long polling?
Thanks !
So my question: How can I achieve to get real-time data changes from a
table of the database or from a certain query...
There is a module to watch for MySQL events: mysql-events
Official Example :
var MySQLEvents = require('mysql-events');
var dsn = {
host: _dbhostname_,
user: _dbusername_,
password: _dbpassword_,
};
var mysqlEventWatcher = MySQLEvents(dsn);
var watcher =mysqlEventWatcher.add(
'myDB.table.field.value',
function (oldRow, newRow, event) {
//row inserted
if (oldRow === null) {
//insert code goes here
}
//row deleted
if (newRow === null) {
//delete code goes here
}
//row updated
if (oldRow !== null && newRow !== null) {
//update code goes here
}
//detailed event information
//console.log(event)
},
'match this string or regex'
);
... which will send the update to all clients, that are connected in real-time,
without long polling?
You can use socket.io and prevent the initial http polling entirely by doing this on the client:
var socket = io({transports: ['websocket'], upgrade: false});
To avoid clients from using polling, add this line to the server:
io.set('transports', ['websocket']);
and to send the events to all connected (socket.io) clients from mysql-event use the following :
io.sockets.emit('db-event',eventDataObj)
Related
I'm making an application where when user submits a form, i want DOM to be updated for everyone on that page, realtime, without refreshing the page.
I have tried doing that with Socket.IO, and it kinda works, but the problem is that it only works if someone is already on that page and i don't need that functionality, i need that when users SUBMITS a form, view is not updated only for existing connection but also when someone loads the page first time and requests were already done.
So, i decided to create a database and check for changes and it works as expected
basically the work flow of the app is this
user submits form => fetch function that checks database for changes is fired => it finds new database entry => updates React state => change is sent to the view
But the problem is that if i do the updating of dom this way, i'm afraid i may be overloading the server unnecessary. I Checked, and every new open instance of
"http://localhost:3000/seek" checks to see if database is changed and so, if i had 1000 users on my web app that would be 1000 requests every second :o
Maybe i should combine both socket.io and database and use that approach for updating dom realtime?
Seek.js (Server Side)
router.post('/', (req,res) => {
// Processes form
// Saving to database
// Sending response
});
router.get('/:fetch', (req,res,next) => {
if(req.params.fetch === 'fetch'){
Seek.find(function(err,games){
if(err) console.log(err);
console.log('FETCHED')
res.status(200).send({games});
})
}else{
next();
}
});
seekDiv.jsx
class MyComponent extends React.Component {
constructor(props) {
super(props);
this.state = { games:[] };
}
componentWillMount(){
this.fetchGames()
}
fetchGames(){
fetch('http://www.localhost:3000/seek/fetch')
.then(res => res.json())
.then(data => {this.setState({games: data.games})})
.catch(err => console.log(err))
}
componentDidMount(){
setInterval( () => this.fetchGames(), 1000)
}
render() {
var games = this.state.games;
let zero;
if(games.length === 0){
zero = ''
}else{
zero = games.map(x => <div>{x.userAlias}</div>)
}
return(
<div>
{zero}
</div>
);
}
}
I'm hoping that i presented my problem clear enough but in case i didn't this is the functionality i want
users submits form => the DOM is updated for EVERY user without refresh containing that form data and is also there until it's manually removed.
Any help on how to proceed is greatly appreciated.
But the problem is that if i do the updating of dom this way, i'm afraid i may be overloading the server unnecessary. I Checked, and every new open instance of "http://localhost:3000/seek" checks to see if database is changed and so, if i had 1000 users on my web app that would be 1000 requests every second :o
Yeah - that is a problem. Conceptually, you need to make this a "push" not "pull" system.
Rather than having every client constantly ask the server if there are updates, you simply need to leave a socket connection open to every page (very low resource use) and on your server, after receiving a new form/post - you then push to every connected client the update.
The socket.io docs have a good example of how to do this in the "broadcast" section. It's for chat messages, but it works the same way for your forms.
You'll want to minimize the data you send to every client to the bare minimum needed. So if you record any additional data (say, a timestamp of when the new post was added) unless you are displaying or using that data on the front end, you wouldn't want to send it to all of the listening client.
You'll want your front end to be monitoring for incoming updates, and when it does, use react to update the DOM accordingly.
Background
I am trying to design an interactive classroom type of environment. The room has different slides, a chat box and some other basic features.
My understanding
A sure way to update a page in real time for all users is for one person to persist a change via ajax to a database, then all the other users poll the server at a timed interval (one second) to check for changes... if there are their view gets updated.
My code
Each room has a unique URL... http://www.example.com/room/ajc73
Users slide through the clides using this code:
showCard();
function showCard() {
$('#card-' + (cardId)).show();
}
$('#nextCard').click(function() {
nextCard();
});
$('#previousCard').click(function() {
previousCard();
});
function nextCard() {
if ($('#card-' + (cardId + 1)).length != 0) { // if there is a next card
$('#card-' + (cardId)).hide(); // hide current card
cardId++; // increment the card id
$('#card-' + (cardId)).show(); // and show the next card
location.hash = cardId;
}
}
function previousCard() {
if (cardId != 1) { // if we are not at the first card
$('#card-' + (cardId)).hide(); // hide the current card
cardId--; // decrement the card id
$('#card-' + (cardId)).show(); // and show the previous card
location.hash = cardId;
}
}
My question
Am I required to persist data from user1 to the database in order for it to be called and displayed to user2 or is there a way to cut out the database part and push changes directly to the browser?
Go for websockets. that will be a better option. since its real-time and just a simpler logic will help you achieve the result.
If you are not sure that whether you will be able to use websockets(like if you are using shared hosting and your provider doesn't allow this or any other reason) you can go for various services like pusher(easier to understand) that will help to do your job but with some cost.
You could use sockets and just broadcast any input to every client.
Of course, the same can be done with ajax and a rest api but it'll be harder, i'll use pseudocode:
clients = {};
fn newclient() {
clients[client_id] = {
pool: [];
....
}
}
fn onNewMessage(newmessage) {
forEach(client, fn(c) {
c.pool.push(newmessage);
})
}
fn clientRequestNews() {
response = clients[client].pool;
clients[client].pool.length = 0;
return response;
}
the point here is, in server memory there would be a entry for each client, each of them has a pool, when a new message is sent to the server, it's pushed to every client's pool.
When a client ask's for news, the server will return the clients pool, after that, it'll clean the pool of that client.
With this you dont need persistence.
Use web sockets. Please see here
You need websockets, a datastructure server and a pub/serve model with events:
A hint
I want to implement Server side events with grails. What i want is that only when there is a change in the DataBase my DataTable should refresh. I want to user HTML5 Server Side events for this. My first question is that while using SSE i observed that the client keeps making a request to the server and if data is available it pulls it. Its similar to an Ajax call being sent every 3-4 seconds which can be changed but what i really want is that only and only when the Data in the DataBase changes should there be a refresh in the DataTable. Also i want to send JSON data to the client but am unable to send it in the right format. Below is my controller code.
def test(){
def action
something.each(){
action = "<a href=\"javascript:fetchDetails('"+it.id+"','comments'"+")\" class='btn btn-primary'><span class='glyphicon glyphicon-upload'></span></a>"
dataArr << [
it.Number,
it.product,
it.description,
OrgChart.findByid(it.Owner)?.displayName,
OrgChart.findByid(it.Coordinator)?.displayName,
startDate,
endDate,
it.status.status,
action
]
}
println dataArr
response.setContentType("text/event-stream;charset=UTF-8")
response << "data: ${dataArr}\n\n"
render "Hi"
}
}
Below is the gsp or the client side code
console.log("Starting eventSource");
var eventSource = new EventSource("/ops/test");
console.log("Started eventSource");
eventSource.onmessage = function(event) {
var data = JSON.stringify(event.data)
console.log("Message received: " + JSON.parse(data));
changeRequestsTable.clear().rows.add(JSON.parse(event.data)).draw()
};
eventSource.onopen = function(event) { console.log("Open " + event); };
eventSource.onerror = function(event) { console.log("Error " + event); };
console.log("eventState: " + eventSource.readyState);
// Stop trying after 10 seconds of errors
setTimeout(function() {eventSource.close();}, 10000);
I know i am a long way from implementing what i intend to but any help would be really appreciated
Going to answer this since it is getting to be a long conversation. As it stands the solution is too broad to give a proper answer since the angle as to how you do things could vary in such a dramatic range from using events that get triggered upon record save that then go off to either make client socket connections through application through to direct socket client connection at point of save that triggers something to be sent to clients. These methods are probably all more complex and more entangled and in short can be done all in a much easier way.
As the users go their interface get the users to make a ws connection to a backend listener. It can be the same location as in no room/separation (additional complexity needed).
As they join room declare a static concurrent map in the top of your websocket listener that collections each session and userId. I wouldn't do it the chat way since this is injected through a service that keeps it as a collection instead like this and changing RunnableFuture to be Websocket sessions like seen in the service link example.
Once you have this you can simply call a broadcast something like this that gets hold of your static concurrent map and for each session either broadcasts the entire new list in json and user processes html update with it or sends a trigger to say update page and they go off doing ajax call to update list.
See the following guide for a tutorial on how to implement Server Sent Events with Grails 3:
http://guides.grails.org/server-sent-events/guide/index.html
My code is working perfectly , But i need to refresh my page to get
new message . I want the real time message.
Angular controller's function -
$scope.getAllMessages = function() {
$scope.messages=[];
restservice.get('', "api/v1/message/getMessageList").then(function(response) {
if (response != null) {
for (var i = 0; i < response.length; i++) {
$scope.messages.push(response[i]);
}
}
});
}
My UI looks like -
<li ng-repeat="message in messages" class="{{(message.me == '1')? 'me' : 'other'}}">
<div>
{{message.userMessage}}
</div>
</li>
Please make it simple for me and give me a proper guideline/tutorial to do it. I'm bad at front end.
Another way is to use WebSockets.
Javascript has built-it WebSocket object wich you can use. And you also need to handle websockets on your server side. But for messaging websockets or long polling requests looks as more useful technologies, as for me.
WebSockets are very simple and if you want real-time, they could be useful. But you would have to implement websockets backend on your server. Here is a sample of WS usage from the JS side:
var exampleSocket = new WebSocket("ws://www.example.com/socketserver", "protocolOne");
exampleSocket.send("Here's some text that the server is urgently awaiting!");
exampleSocket.onmessage = function (event) {
console.log(event.data);
}
This is example from here: https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API/Writing_WebSocket_client_applications
You need to call getAllMessages method on interval basis so that you get live data or real time data
Link here
https://docs.angularjs.org/api/ng/service/$interval
check with the above link for Example how to implement
To avoid getting ALL messages again i will considere do another .get maybe like this:
$interval(getLastMessagesFrom, 1000);
function getLastMessagesFrom () {
// Extract the last message_ID from $scope.messages, to do this maybe
// you will need to change your backend code to add the message ID in
// the message object. This deppends on your backend code & backend
// message response.
var last_message_id = $scope.messages[$scope.messages.length-1].id;
restservice.get('', "api/v1/message/getMessageList/" + last_message_id).then(function(response) {
if (response != null) {
// Now make sure that your server only returns messages with
// ID higher than last_message_id
for (var i = 0; i < response.length; i++) {
$scope.messages.push(response[i]);
}
}
});
}
Anyways this is still a bad practice if you have a lower time interval (1000 = 1 second) and a lot of concurrent users.
Considere use sockets to avoid repeated and unecessary calls.
use nodeJS's sockets
where you can achieve real-time experience.
e.g
once you inserted to database,you can fire JS command to emit msg realtime
io.sockets.connected[setsocketid].emit("clssifiedMessage", {msg:"Hi!!!how are yyou?" });
I want to make a homepage where several pieces of data are published, but only when the user first visits the page : one would get the latest 10 articles published but that's it - it won't keep changing.
Is there a way to make the inbuilt pub/sub mechanism turn itself off after a set amount of time or number of records, or another mechanism?
Right now I'm using a very simple setup that doesn't "turn off":
latestNews = new Mongo.Collection('latestNews');
if (Meteor.isClient) {
Meteor.subscribe("latestNews");
}
if (Meteor.isServer) {
Meteor.publish('latestNews', function() {
return latestNews.find({}, {sort: { createdAt: -1 }, limit : 10});
});
}
The pub/sub pattern as it is implemented in Meteor is all about reactive data updates. In your case that would mean if the author or last update date of an article changes then users would see this change immediately reflected on their home page.
However you want to send data once and not update it ever again.
Meteor has a built-in functionality to handle this scenario : Methods. A method is a way for the client to tell the server to execute computations and/or send pure non-reactive data.
//Server code
var lastTenArticlesOptions = {
sort : {
createdAt : -1
},
limit : 10
}
Meteor.methods({
'retrieve last ten articles' : function() {
return latestNews.find({}, lastTenArticlesOptions).fetch()
}
})
Note that contrary to publications we do not send a Mongo.Cursor! Cursors are used in publications as a handy (aka magic) way to tell the server which data to send.
Here, we are sending the data the data directly by fetching the cursor to get an array of articles which will then be EJSON.stringifyied automatically and sent to the client.
If you need to send reactive data to the client and at a later point in time to stop pushing updates, then your best bet is relying on a pub/sub temporarily, and then to manually stop the publication (server-side) or the subscription (client-side) :
Meteor.publish('last ten articles', function() {
return latestNews.find({}, lastTenArticlesOptions)
})
var subscription = Meteor.subscribe('last ten articles')
//Later...
subscription.stop()
On the server-side you would store the publication handle (this) and then manipulate it.
Stopping a subscription or publication does not destroy the documents already sent (the user won't see the last ten articles suddenly disappear).