Develop Reference

SignalR with long process: send intermediate messages

I am using SignalR to start long computations on server side and post a message to the client when the result is available.
The input bindings is an HTTP request.
I would like to be able to send multiple messages back in order to notify the client of the differents steps of the process (eg, computation starts, computation ends, etc..).
I tried pushing different messages to context.bindings.signalRMessages but I see that everything is sent together at the end of the whole process. Is there a way to send several messages at different times?
Another related issue is that my HTTP request on client side remains stuck until the end of the process. I would like to be able to post a quick response early, since I get the response via a signalR message.
Here is my server code :
module.exports = async function(context, req) {
let ID = context.bindingData.invocationId;
context.bindings.signalRMessages = [];
const messageQueue = context.bindings.signalRMessages;
var postMessage = (message) => {
message.userId = req.query.userId;
message.isPrivate = true;
messageQueue.push(message);
};
let preProcessData = preProcess(req.body.input);
let startMessage = {
"target": "optimStart",
"arguments": [{ preProcessData: preProcessData }]
};
postMessage(startMessage); // <<<< I want this one to be sent immediately
try {
let optimOutput = await computeOptim(req.body.input, ID); // that's the long process
let response = {
optimId: ID,
optimOutput: optimOutput
};
let optimCompleteMessage = {
"target": "optimComplete",
"arguments": [response]
};
postMessage(optimCompleteMessage);
} catch (err) {
// ....
}
};
Am I doing anything wrong or is it just not possible ?
Thanks!

This is not possible with a simple HTTP triggered function since bindings resolve only once the execution of the function completes.
For your scenario, durable functions would be the perfect choice.
You would still have a HTTP Triggered function (client function) to start on orchestration and return immediately. In the orchestration function, you would have separate activity functions for the processing and for sending updates to the client using the SignalR binding.

How can I tell when a loop is done.using JavaScript and Vue and AXIOS

I have a dataGrid from Syncfusion and I have a column of checkboxes. When I press a button the code reads all of the selected rows creates an array and loops until the process is ended.
this.selectedRecords = this.$refs.grid.ej2Instances.getSelectedRecords();
this.selectedRecords.forEach(function(arg, index) {
call HTTP API request. with AXIOS
get the return values and store it to the database
}
I could have 100+ rows selected and I need to be able to tell when all of the API calls are finished.
I have slowed down my calls so I only have a maximum of 10 calls per second using
axios.interceptors.request.use(
function(config) {
setTimeout(console.log("here request interceptor"), 100);
return config;
},
function(error) {
return Promise.reject(error);
}
);
And I have tried
if (self.selectedRecords.length - 1 === index) {
alert("Done");
}
but since there is not a guarantee that the rows are processed in order it can call "Done" too early.
I hope I've given you enough code to understand my problem without giving you too much to make it sloppy.

If I've understood correctly then you should just need to gather up the promises in an array and then use Promise.all to wait for them to all complete:
var requests = this.selectedRecords.map(function(arg, index) {
return axios.get(/* request details here */);
});
Promise.all(requests).then(function() {
console.log('Done')
});
If you need to process the individual requests using a then that's fine, just chain it on the end of the axios.get call:
return axios.get(/* request details here */)
.then(function(response) {
// Handle response
})
Update:
Request interceptors can return promises, which will be necessary if you want to hold up the execution of the request:
http.interceptors.request.use(function (config) {
return new Promise(function (resolve) {
setTimeout(function () {
resolve(config)
}, 5000)
})
})
Note that the example above is not performing proper throttling, it's merely delaying the request. It is purely to illustrate how promises can be used with interceptors. You haven't included the real interceptor in the question so I can't be more specific.

Error handling over websockets a design dessision

Im currently building a webapp that has two clear use cases.
Traditional client request data from server.
Client request a stream from the server after wich the server starts pushing data to the client.
Currently im implementing both 1 and 2 using json message passing over a websocket. However this has proven hard since I need to handcode lots of error handling since the client is not waiting for the response. It just sends the message hoping it will get a reply sometime.
Im using Js and react on the frontend and Clojure on the backend.
I have two questions regarding this.
Given the current design, what alternatives are there for error handling over a websocket?
Would it be smarter to split the two UC using rest for UC1 and websockets for UC2 then i could use something like fetch on the frontend for rest calls.
Update.
The current problem is not knowing how to build an async send function over websockets can match send messages and response messages.

Here's a scheme for doing request/response over socket.io. You could do this over plain webSocket, but you'd have to build a little more of the infrastructure yourself. This same library can be used in client and server:
function initRequestResponseSocket(socket, requestHandler) {
var cntr = 0;
var openResponses = {};
// send a request
socket.sendRequestResponse = function(data, fn) {
// put this data in a wrapper object that contains the request id
// save the callback function for this id
var id = cntr++;
openResponses[id] = fn;
socket.emit('requestMsg', {id: id, data: data});
}
// process a response message that comes back from a request
socket.on('responseMsg', function(wrapper) {
var id = wrapper.id, fn;
if (typeof id === "number" && typeof openResponses[id] === "function") {
fn = openResponses[id];
delete openResponses[id];
fn(wrapper.data);
}
});
// process a requestMsg
socket.on('requestMsg', function(wrapper) {
if (requestHandler && wrapper.id) {
requestHandler(wrapper.data, function(responseToSend) {
socket.emit('responseMsg', {id: wrapper.id, data; responseToSend});
});
}
});
}
This works by wrapping every message sent in a wrapper object that contains a unique id value. Then, when the other end sends it's response, it includes that same id value. That id value can then be matched up with a particular callback response handler for that specific message. It works both ways from client to server or server to client.
You use this by calling initRequestResponseSocket(socket, requestHandler) once on a socket.io socket connection on each end. If you wish to receive requests, then you pass a requestHandler function which gets called each time there is a request. If you are only sending requests and receiving responses, then you don't have to pass in a requestHandler on that end of the connection.
To send a message and wait for a response, you do this:
socket.sendRequestResponse(data, function(err, response) {
if (!err) {
// response is here
}
});
If you're receiving requests and sending back responses, then you do this:
initRequestResponseSocket(socket, function(data, respondCallback) {
// process the data here
// send response
respondCallback(null, yourResponseData);
});
As for error handling, you can monitor for a loss of connection and you could build a timeout into this code so that if a response doesn't arrive in a certain amount of time, then you'd get an error back.
Here's an expanded version of the above code that implements a timeout for a response that does not come within some time period:
function initRequestResponseSocket(socket, requestHandler, timeout) {
var cntr = 0;
var openResponses = {};
// send a request
socket.sendRequestResponse = function(data, fn) {
// put this data in a wrapper object that contains the request id
// save the callback function for this id
var id = cntr++;
openResponses[id] = {fn: fn};
socket.emit('requestMsg', {id: id, data: data});
if (timeout) {
openResponses[id].timer = setTimeout(function() {
delete openResponses[id];
if (fn) {
fn("timeout");
}
}, timeout);
}
}
// process a response message that comes back from a request
socket.on('responseMsg', function(wrapper) {
var id = wrapper.id, requestInfo;
if (typeof id === "number" && typeof openResponse[id] === "object") {
requestInfo = openResponses[id];
delete openResponses[id];
if (requestInfo) {
if (requestInfo.timer) {
clearTimeout(requestInfo.timer);
}
if (requestInfo.fn) {
requestInfo.fn(null, wrapper.data);
}
}
}
});
// process a requestMsg
socket.on('requestMsg', function(wrapper) {
if (requestHandler && wrapper.id) {
requestHandler(wrapper.data, function(responseToSend) {
socket.emit('responseMsg', {id: wrapper.id, data; responseToSend});
});
}
});
}

There are a couple of interesting things in your question and your design, I prefer to ignore the implementation details and look at the high level architecture.
You state that you are looking to a client that requests data and a server that responds with some stream of data. Two things to note here:
HTTP 1.1 has options to send streaming responses (Chunked transfer encoding). If your use-case is only the sending of streaming responses, this might be a better fit for you. This does not hold when you e.g. want to push messages to the client that are not responding to some sort of request (sometimes referred to as Server side events).
Websockets, contrary to HTTP, do not natively implement some sort of request-response cycle. You can use the protocol as such by implementing your own mechanism, something that e.g. the subprotocol WAMP is doing.
As you have found out, implementing your own mechanism comes with it's pitfalls, that is where HTTP has the clear advantage. Given the requirements stated in your question I would opt for the HTTP streaming method instead of implementing your own request/response mechanism.

Abort elasticsearch request using elastic.js

Is there a way to cancel requests/queries to Elasticsearch using elasticjs? The web app I am working on performs a request/query every 5 seconds, but I would like to cancel the request if for some reason the response doesn't show up in 5 seconds (so the browser doesn't pile up a bunch of requests that are unnecessary since the queries are happening repeatedly). I understand this would not prevent Elasticsearch from completing the query, but I would like to at least cancel the request in the browser.
Example:
var request = ejs.Request().doSearch();
var dataFromElasticsearch;
request.then(function (data) {
dataFromElasticsearch = data;
});
setTimeout(function () {
if (!dataFromElasticsearch) {
//do something here to cancel request
}
}, 5000)

Per documentation for elasticsearch.js (3.1, at the time of writing):
...calling the API will return an object (either a promise or just a plain object) which has an abort() method. Calling that abort method ends the HTTP request, but it will not end the work Elasticsearch is doing.
Specifically for your example:
setTimeout(function () {
if (!dataFromElasticsearch) {
request.abort();
}
}, 5000)

A design pattern for async requests to handle success, failure, retry ? (javascript)

I'm writing a mobile app with Appcelerator Titanium that makes a lot of different xhr requests. This is not really an Appcelerator Titanium specific question. But if you do write some code, I hope it's javascript.
The app needs to authenticate itself, the user must be logged for some interactions, etc.
I've come to a point where any request might get any kind of response such as:
not authenticated
not logged
bad params
successful
...
The requests are wrapped in different model methods or helpers.
The thing is, I'm not familiar with this kind of app. I was wondering what are the best practices.
Some real questions for example would be:
If the app is not authenticated (token expired, first launch), should the app try to authenticate itself and then send again the request that was denied ? (transparent to user)
Should I send an authentication request each time the app launches and then "forget" about it?
The problem I'm facing is that the code becomes quickly big if I try to handle this for each request. Full of nested callbacks, retry conditions, various events listeners to manage, etc. It just does not feel very "nice". And it's not DRY at all, when what I really need is for any request, check what was wrong, try to fix it (authenticate if not, automatic login if possible or show the login UI, etc..) then if that works retry the original request a couple of times, abort if needed.
I've been looking at the promise pattern but only know theory and don't know if it could be what I need.
So I welcome any advice regarding this particular problem. I wonder how apps like "Facebook" handle this.
Thank you for your help

This question is not easily answered, but let me try to give you some Ideas:
The most important thing, before coding anything in your app, is the API itself. It has to be reliable and adhere to standards. I will not go into too much detail here, but a well written RESTful API can reduce the complexity of your httpClient significantly. It has to respond with standard http status codes and to methods like POST, GET, PUT, DELETE...
A pretty good read is The REST API Design Handbook by George Reese.
My approach to httpClients with Titanium is a single module, which is loaded via require() wherever needed. I stick to one single client at a time, as I had massive problems with multiple parallel calls. Whenever a call is made, the client checks if there is already a call in progress and sends it to a queue if necessary.
Let me show you an example. I have left out lots of stuff for sake of brevity:
// lib/customClient.js
var xhrRequest; // This will be our HTTPClient
var callQueue = []; // This will be our queue
// Register the request
// params are:
// method (e.g. 'GET')
// url (e.g. 'http://test.com/api/v1/user/1')
// done (callback function)
function registerRequest(params) {
if(!xhrRequest) {
sendRequest(params);
} else {
queueRequest(params);
}
}
// This simply sends the request
// to the callQueue
function queueRequest(params) {
callQueue.push(params);
}
// Send the request with the params from register
// Please note that I do not hardcode error messages,
// I just do it here so it is easier to read
function sendRequest(params) {
// Set callback if available and valid
var callback = params.done && typeof(params.done) === "function" ? params.callback : null;
// Set method
var method = params.method || 'GET';
// Create the HTTP Client
xhrRequest = Ti.Network.createHTTPClient({
// Success
onload: function() {
// You can check for status codes in detail here
// For brevity, I will just check if it is valid
if (this.status >= 200 && this.status < 300) {
if(this.responseText) {
// You might want to check if it can be parsed as JSON here
try {
var jsonData = JSON.parse(this.responseText);
if(callback) callback({ success: true, response: jsonData });
} catch(e) {
if(callback) callback({ success: false, errormessage: 'Could not parse JSON data' });
}
processQueue();
} else {
if(callback) callback({ success: false, errormessage: 'No valid response received' });
processQueue();
}
} else {
if(callback) callback({ success: false, errormessage: 'Call response is success but status is ' + this.status });
processQueue();
}
},
// Error
onerror: function(e) {
if(this.responseText) {
try {
var jsonData = JSON.parse(this.responseText);
if(callback) callback({ success: false, reponse: jsonData });
} catch(e) {};
}
processQueue();
},
});
// Prepare and send request
// A lot more can (and should) be configured here, check documentation!
xhrRequest.setTimeout(10000);
xhrRequest.open(method, params.url);
xhrRequest.send();
}
// Checks if there is anything else in the queue
// and sends it
function processQueue() {
xhrRequest = null;
var nextInQueue = callQueue.shift();
if(nextInQueue) sendRequest(nextInQueue);
}
// Our public API
var publicAPI = {
sendRequest: function(params) {
registerRequest(params);
}
};
module.exports = publicAPI;
I can then send a call from any other controller/view
var customClient = require('lib/customClient'); // omit 'lib' if you use alloy
// Send the request
customClient.sendRequest({
method : 'GET',
url : 'http://test.com/api/v1/user/1',
done : function(response) {
Ti.API.debug(JSON.stringify(response));
}
});
Note that this is not complete and does not check for connectivity, has no real error handling etc., but it might help you to get an idea.
I think there is loads of stuff to talk about here, but I will stop here for now...