I want to implement Server side events with grails. What i want is that only when there is a change in the DataBase my DataTable should refresh. I want to user HTML5 Server Side events for this. My first question is that while using SSE i observed that the client keeps making a request to the server and if data is available it pulls it. Its similar to an Ajax call being sent every 3-4 seconds which can be changed but what i really want is that only and only when the Data in the DataBase changes should there be a refresh in the DataTable. Also i want to send JSON data to the client but am unable to send it in the right format. Below is my controller code.
def test(){
def action
something.each(){
action = "<a href=\"javascript:fetchDetails('"+it.id+"','comments'"+")\" class='btn btn-primary'><span class='glyphicon glyphicon-upload'></span></a>"
dataArr << [
it.Number,
it.product,
it.description,
OrgChart.findByid(it.Owner)?.displayName,
OrgChart.findByid(it.Coordinator)?.displayName,
startDate,
endDate,
it.status.status,
action
]
}
println dataArr
response.setContentType("text/event-stream;charset=UTF-8")
response << "data: ${dataArr}\n\n"
render "Hi"
}
}
Below is the gsp or the client side code
console.log("Starting eventSource");
var eventSource = new EventSource("/ops/test");
console.log("Started eventSource");
eventSource.onmessage = function(event) {
var data = JSON.stringify(event.data)
console.log("Message received: " + JSON.parse(data));
changeRequestsTable.clear().rows.add(JSON.parse(event.data)).draw()
};
eventSource.onopen = function(event) { console.log("Open " + event); };
eventSource.onerror = function(event) { console.log("Error " + event); };
console.log("eventState: " + eventSource.readyState);
// Stop trying after 10 seconds of errors
setTimeout(function() {eventSource.close();}, 10000);
I know i am a long way from implementing what i intend to but any help would be really appreciated
Going to answer this since it is getting to be a long conversation. As it stands the solution is too broad to give a proper answer since the angle as to how you do things could vary in such a dramatic range from using events that get triggered upon record save that then go off to either make client socket connections through application through to direct socket client connection at point of save that triggers something to be sent to clients. These methods are probably all more complex and more entangled and in short can be done all in a much easier way.
As the users go their interface get the users to make a ws connection to a backend listener. It can be the same location as in no room/separation (additional complexity needed).
As they join room declare a static concurrent map in the top of your websocket listener that collections each session and userId. I wouldn't do it the chat way since this is injected through a service that keeps it as a collection instead like this and changing RunnableFuture to be Websocket sessions like seen in the service link example.
Once you have this you can simply call a broadcast something like this that gets hold of your static concurrent map and for each session either broadcasts the entire new list in json and user processes html update with it or sends a trigger to say update page and they go off doing ajax call to update list.
See the following guide for a tutorial on how to implement Server Sent Events with Grails 3:
http://guides.grails.org/server-sent-events/guide/index.html
Related
I'm trying to build real time notification system for users using
Server sent events in python.
The issue I'm facing is when I refresh the browser, which means the
browser then will try to hit the EventSource url again, and as per
docs it should send the event.lastEventId as part of headers in next
request. I'm getting None every time I refresh the page.
<!DOCTYPE html>
<html>
<header><title>SSE test</title></header>
<body>
<ul id="list"></ul>
<script>
const evtSource = new EventSource("/streams/events?token=abc");
evtSource.onmessage = function(event) {
console.log('event', event)
console.log('event.lastEventId', event.lastEventId)
const newElement = document.createElement("li");
const eventList = document.getElementById("list");
newElement.innerHTML = "message: " + event.data;
eventList.appendChild(newElement);
}
</script>
</body>
</html>
On the server side
from sse_starlette.sse import EventSourceResponse
from asyncio.queues import Queue
from starlette.requests import Request
#event_router.get(
"/streams/events",
status_code=HTTP_200_OK,
summary='',
description='',
response_description='')
async def event_stream(request: Request):
return EventSourceResponse(send_events(request))
async def send_events(request: Request):
try:
key = request.query_params.get('token')
last_id = request.headers.get('last-event-id')
print('last_id ', last_id) # this value is always None
connection = Queue()
connections[key] = connection
while RUNNING:
next_event = await connection.get()
print('event', next_event)
yield dict(data=next_event, id=next_event['id'])
connection.task_done()
except asyncio.CancelledError as error:
pass
Now, as per every doc on SSE, when client reconnects or refreshes the
page it will send the last-event-id in headers. I'm trying to read it
using request.headers.get('last-event-id'), but this is always null.
Any pointers on how to get the last event id would be helpful. Also,
how would I make sure that I do't send the same events even later once
the user has seen the events, as my entire logic would be based on
last-event-id received by the server, so if its None after reading
events with Id 1 to 4, how would I make sure in server that I should
not send these back even if last-event-id is null for the user
.
Adding browser snaps
1st pic shows that the events are getting received by the browser.
e.g. {alpha: abc, id:4}
2nd pic shows that the event received is
setting the lastEventId correctly.
I think this is a wrong understanding. Where did you get the part that says "when client reconnects or refreshes the page it will send the last-event-id in headers".
My understanding is that the last ID is sent on a reconnect. When you refresh the page directly, that is not seen as a broken connection and a re-connect. You are completely re-starting the whole page and forming a brand new SSE connection to the server from the browser. Keep in mind if your SSE had a certain request parameter for example and this could be driven by something on the page. You could have two tabs open from the same "page" but put different items into the page which causes this request parameter to be different in your SSE. They are two different SSE connections altogether. One does not affect the other except that you can reach the maximum connection limit of a browser if you have too many. In the same way, if you click refresh, this is like a new tab being created. It is a totally new connection.
I've got a web service under development that uses Django and Django Channels to send data across websockets to a remote application. The arrangement is asynchronous and I pass information between the 2 by sending JSON formatted commands across websockets and then receive replies back on the same websocket.
The problem I'm having is figuring out how to get the replies back to a Javascript call from a Django template that invokes a Python function to initiate the JSON websocket question. Since the command question & data reply happen in different Django areas and the originating Javascript/Python functions call does not have a blocking statement, the Q&A are basically disconnected and I can't figure out how to get the results back to the browser.
Right now, my idea is to use Django global variables or store the results in the Django models. I can get either to work, but I beleive the Django global variables would not scale beyond multiple workers from runserver or if the system was eventually spread across multiple servers.
But since the reply data is for different purposes (for example, list of users waiting in a remote lobby, current debugging levels in remote system, etc), the database option seems unworkable because the reply data is varying structure. That, plus the replies are temporal and don't need to be permanently stored in the database.
Here's some code showing the flow. I'm open to different implementation recommendations or a direct answer to the question of how to share information between the 2 Django functions.
In the template, for testing, I just have a button defined like this:
<button id="request_lobby">Request Lobby</button>
With a Javascript function. This function is incomplete as I've yet to do anything about the response (because I can't figure out how to connect it):
$("#request_lobby").click(function(){
$.ajax({
type: "POST",
url: "{% url 'test_panel_function' %}",
data: { csrfmiddlewaretoken: '{{ csrf_token }}', button:"request_lobby" },
success: function(response){
}
});
});
This is the Django/Python function in views.py . The return channel for the remote application is pre-stored in the database as srv.server_channel when the websocket is initially connected (not shown):
#login_required
def test_panel_function(request):
button = request.POST.get('button', '')
if button == "request_lobby" :
srv = Server.objects.get(server_key="1234567890")
json_res = []
json_res.append({"COMMAND": "REQUESTLOBBY"})
message = ({
"text": json.dumps(json_res)
})
Channel(srv.server_channel).send(message)
return HttpResponse(button)
Later, the remote application sends the reply back on the websocket and it's received by a Django Channels demultiplexer in routing.py :
class RemoteDemultiplexer(WebsocketDemultiplexer):
mapping = {
"gLOBBY" : "gLOBBY.receive",
}
http_user = True
slight_ordering = True
channel_routing = [
route_class(RemoteDemultiplexer, path=r"^/server/(?P<server_key>[a-zA-Z0-9]+)$"),
route("gLOBBY.receive" , command_LOBBY),
]
And the consumer.py :
#channel_session
def command_LOBBY(message):
skey = message.channel_session["server_key"]
for x in range(int(message.content['LOBBY'])):
logger.info("USERNAME: " + message.content[str(x)]["USERNAME"])
logger.info("LOBBY_ID: " + message.content[str(x)]["LOBBY_ID"])
logger.info("OWNER_ID: " + message.content[str(x)]["IPADDRESS"])
logger.info("DATETIME: " + message.content[str(x)]["DATETIME"])
So I need to figure out how to get the reply data in command_LOBBY to the Javascript/Python function call in test_panel_function
Current ideas, both of which seem bad and why I think I need to ask this question for SO:
1) Use Django global variables:
Define in globals.py:
global_async_result = {}
And include in all relevant Django modules:
from test.globals import global_async_result
In order to make this work, when I originate the initial command in test_panel_function to send to the remote application (the REQUESTLOBBY), I'll include a randomized key in the JSON message which would be round-tripped back to command_LOBBY and then global_async_result dictionary would be indexed with the randomized key.
In test_panel_function , I would wait in a loop checking a flag for the results to be ready in global_async_result and then retrieve them from the randomized key and delete the entry in global_async_result.
Then the reply can be given back to the Javascript in the Django template.
That all makes sense to me, but uses global variables (bad), and seems that it wouldn't scale as the web service is spread across servers.
2) Store replies in Django mySQL model.py table
I could create a table in models.py to hold the replies temporarily. Since Django doesn't allow for dynamic or temporary table creations on the fly, this would have to be a pre-defined table.
Also, because the websocket replies would be different formats for different questions, I could not know in advance all the fields ever needed and even if so, most fields would not be used for differing replies.
My workable idea here is to create the reply tables using a field for the randomized key (which is still routed back round-trip through the websocket) and another large field to just store the JSON reply entirely.
Then in test_panel_function which is blocking in a loop waiting for the results, pull the JSON from the table, delete the row, and decode. Then the reply can be given back to the Javascript in the Django template.
3) Use Django signals
Django has a signals capability, but the response function doesn't seem to be able to be embedded (like inside test_panel_function) and there seems to be no wait() function available for an arbitrary function to just wait for the signal. If this were available, it would be very helpful
I want to make a homepage where several pieces of data are published, but only when the user first visits the page : one would get the latest 10 articles published but that's it - it won't keep changing.
Is there a way to make the inbuilt pub/sub mechanism turn itself off after a set amount of time or number of records, or another mechanism?
Right now I'm using a very simple setup that doesn't "turn off":
latestNews = new Mongo.Collection('latestNews');
if (Meteor.isClient) {
Meteor.subscribe("latestNews");
}
if (Meteor.isServer) {
Meteor.publish('latestNews', function() {
return latestNews.find({}, {sort: { createdAt: -1 }, limit : 10});
});
}
The pub/sub pattern as it is implemented in Meteor is all about reactive data updates. In your case that would mean if the author or last update date of an article changes then users would see this change immediately reflected on their home page.
However you want to send data once and not update it ever again.
Meteor has a built-in functionality to handle this scenario : Methods. A method is a way for the client to tell the server to execute computations and/or send pure non-reactive data.
//Server code
var lastTenArticlesOptions = {
sort : {
createdAt : -1
},
limit : 10
}
Meteor.methods({
'retrieve last ten articles' : function() {
return latestNews.find({}, lastTenArticlesOptions).fetch()
}
})
Note that contrary to publications we do not send a Mongo.Cursor! Cursors are used in publications as a handy (aka magic) way to tell the server which data to send.
Here, we are sending the data the data directly by fetching the cursor to get an array of articles which will then be EJSON.stringifyied automatically and sent to the client.
If you need to send reactive data to the client and at a later point in time to stop pushing updates, then your best bet is relying on a pub/sub temporarily, and then to manually stop the publication (server-side) or the subscription (client-side) :
Meteor.publish('last ten articles', function() {
return latestNews.find({}, lastTenArticlesOptions)
})
var subscription = Meteor.subscribe('last ten articles')
//Later...
subscription.stop()
On the server-side you would store the publication handle (this) and then manipulate it.
Stopping a subscription or publication does not destroy the documents already sent (the user won't see the last ten articles suddenly disappear).
I'm developing a text based adventure game with Meteor and I'm running into an issue with how to handle certain elements. Namely, how to emit data from the Server to the Client without any input from the Client.
The idea is that when a player is engaged in combat with a monster, the combat damage and updating the Player and Monster objects will be occurring in a loop on the server. When the player takes damage it should accordingly update the client UI. Is something like this possible with Publish / Subscribe?
I basically want something that sits and listens for events from the server to update the combat log accordingly.
In pseudo-code, this is something along the lines of what I'm looking for:
// Client Side:
Meteor.call("runCommand", "attack monster");
// Server Side
Meteor.methods({
runCommand: function(input) {
// Take input, run the loop to begin combat,
// whenever the user takes damage update the
// client UI and output a line saying how much
// damage the player just received and by who
}
});
I understand that you can publish a collection to the client, but that's not really as specific of a function I'm looking for, I don't want to publish the entire Player object to the client, I just want to tell the client to write a line to a textbox saying something like "You were hit for 12 damage by a monster!".
I was hoping there was a function similar to SocketIO where I could, if I wanted to, just emit an event to the client telling it to update the UI. I think I can use SocketIO for this if I need to, but people seemed to be adamant that something like this was doable with Meteor entirely without SocketIO, I just don't really understand how.
The only outs I see to this scenario are: writing all of the game logic client-side which feels like a bad idea, writing all of the combat logs to a collection which seems extremely excessive (but maybe it's not?), or using some sort of SocketIO type-tool to just emit messages to the client to tell it to write a new line to the text box.
Using Meteor, create a combat log collection seem to be the simplest option you have.
You can only listen on added event and then clear the collection when the combat is over.
It should be something like this :
var cursor = Combat_Log.find();
var handleCombatLog = cursor.observe({
added: function (tmp)
{
// do your stuff
}
});
I ask a similar question here, hope this will help ^^
Here's how I did it without a collection. I think you are right to be concerned about creating one. That would not be a good idea. First install Streamy.
https://atmospherejs.com/yuukan/streamy
Then on the server
//find active sockets for the user by id
var sockets = Streamy.socketsForUsers(USER_ID_HERE)._sockets
if (!Array.isArray(sockets) || !sockets.length) {
//user is not logged in
} else {
//send event to all open browser windows for the user
sockets.forEach((socket) => {
Streamy.emit('ddpEvent', { yourKey:yourValue }, socket);
})
}
Then in the client, respond to it like this:
Streamy.on('ddpEvent', function(data) {
console.log("data is ", data); //prints out {yourKey:yourValue}
});
I'm working on an online, turned based game in order to teach myself Node.js and Socket.IO. Some aspects of the game are resolved serverside. At one point during one of these functions, the server may require input from the clients. Is there a way I can "pause" the resolution of the server's function in order to wait for the clients to respond (via a var x = window.prompt)?
Here's an idea of the code I'm working with:
Server:
for (some loop){
if (some condition){
request input via io.sockets.socket(userSocket[i]).emit('requestInput', data)
}
}
Client:
socket.on('requestInput', function (data) {
var input = window.prompt('What is your input regarding ' + data + '?');
//send input back to the server
socket.emit('refresh', input)
});
Any thoughts?
I don't think that is possible.
for (some loop){
if (some condition){
request input via io.sockets.socket(userSocket[i]).emit('requestInput', data)
/* Even if you were able to pause the execution here, there is no way to resume it when client emits the 'refresh' event with user input */
}
}
What you can do instead is emit all 'requestInput' events without pausing and save all responses you will get in socket.on('refresh',function(){}) event in an array, then you can process this array later. I don't know what your exact requirement is but let me know if that works.
Since you are emitting socket.emit('refresh', input) on the client side, you just need to set up a socket event listener on the server side as well. For example:
io.sockets.on('connection', function (socket) {
socket.on('refresh', function (data) {
console.log(data) //input
});
})
I will also point out, so that you don't run into trouble down the line, that indefinite loops are a big nono in node. Nodejs runs on a single thread so you are actually blocking ALL clients as long as your loop is running.