I'm using RxJs to listen to a amqp queu (not really relevant).
I have a function createConnection that returns an Observable that emits the new connection object. Once I have a connection, I want to send messages through it every 1000ms and after 10 messages I want to close the connection.
I'm trying to avoid external state, but if I don't store the connection in an external variable, how can I close it? See I begin with the connection, then flatMap and push messages, so after a few chains I no longer have the connection object.
This is no my flow but imagine something like this:
createConnection()
.flatMap(connection => connection.createChannel())
.flatMap(channel => channel.send(message))
.do(console.log)
.subscribe(connection => connection.close()) <--- obviously connection isn't here
Now I understand that it's stupid to do that, but now how do I access the connection? I could of course begin with var connection = createConnection()
and later on somehow join that. But how do I do this? I don't even know how to ask this question properly. Bottomline, what I have is an observable, that emits a connection, after the connection is opened I want an observable that emits messages every 1000ms (with a take(10)), then close the connection
The direct answer to your question is "you can carry it through each step". For example, you can replace this line
.flatMap(connection => connection.createChannel())
with this one:
.flatMap(connection => ({ connection: connection, channel: connection.createChannel() }))
and retain access to the connection all the way down.
But there's another way to do what you want to do. Let's assume your createConnection and createChannel functions look something like this:
function createConnection() {
return Rx.Observable.create(observer => {
console.log('creating connection');
const connection = {
createChannel: () => createChannel(),
close: () => console.log('disposing connection')
};
observer.onNext(connection);
return Rx.Disposable.create(() => connection.close());
});
}
function createChannel() {
return Rx.Observable.create(observer => {
const channel = {
send: x => console.log('sending message: ' + x)
};
observer.onNext(channel);
// assuming no cleanup here, don't need to return disposable
});
}
createConnection (and createChannel, but we'll focus on the former) returns a cold observable; each subscriber will get their own connection stream containing a single connection, and when that subscription expires, the dispose logic will be called automatically.
This allows you to do something like this:
const subscription = createConnection()
.flatMap(connection => connection.createChannel())
.flatMap(channel => Rx.Observable.interval(1000).map(i => ({ channel: channel, data: i })))
.take(10)
.subscribe(x => x.channel.send(x.data))
;
You don't actually have to dispose the subscription for cleanup to occur; after take(10) is satisfied, the whole chain will finish and cleanup will be triggered. The only reason you'd need to call dispose on the subscription explicitly is if you wanted to tear things down before the 10 1000ms intervals were up.
Note that this solution also contains an instance of the direct answer to your question: we cart the channel down the line so we can use it in the onNext lambda passed to the subscribe call (which is customarily where such code would appear).
Here's the whole thing working: https://jsbin.com/korihe/3/edit?js,console,output
This code gave me a error because flatmap wait for a observable<(T)> and ({ connection: connection, channel: connection.createChannel() }) it's a Object.
.flatMap(connection => ({ connection: connection, channel: connection.createChannel() }))
instead you can use the combineLatest operator
.flatMap(connection => Observable.combineLatest( Observable.of(connection), connection.createChannel(), (connection, channel) => {
... code ....
});
Related
I'm using a Python based socketio server and JS socketio client. The workflow looks roughly like this:
client emits <<action1>> -> server receives <<action1>> -> server performs some calculations -> server emits <<action2>> -> client receives <<action2>>
It should work in a loop.
The problem is that for some reason the last step isn't performed, client doesn't seem to catch action2 event. I can see that it has been fired from the server logs, they state that: emitting event "action2" to all [/]
Server:
#sio.on('action1')
def handle_analysis():
... some calculations ...
sio.emit('action2')
Client:
import { io } from "socket.io-client";
export const socket = io("http://localhost:5550",
{
timeout: 200000
});
async runCalculations() {
console.log('progress1');
let stdoutChunks: string = '';
const getCalculations = () => {
socket.emit('action1');
return new Promise((resolve, reject) => {
socket.on('action2', (msg) => {
stdoutChunks += msg;
resolve(stdoutChunks);
})
});
};
await getCalculations()
console.log('progress2');
return {};
}
progress2 is never logged, because js keeps on awaiting the getCalculations function.
Can anyone point where the issue might be?
Each calculation takes roughly 50 seconds, I wonder if that might be the cause.
I used same approach for different events that take much less time and it works perfectly, server communicates with the client without problems.
I'm not that good/versed with python-socket.io but isn't getCalculations() method defined inside runCalculations() in Client. So, await getCalculations() won't directly work and also in Server shouldn't sio.emit('action2') also have a second argument like this sio.emit('action2', 'hello'). Hope, this may be useful
I have a reactjs client that connects to a websocket server, however when I try to initialize it outside of an effect hook, the websocket object is undefined, but it initializes when its in an effect hook
like so:
effect(()=>{
connection = new WebSocket(url)
//do some stuff
})
the issue is that I want to send things based on an event that happens in the reactjs app
effect(()=>{
connection.send(messages)
},[messages])
at that point, connection is undefined, despite the connection being made then, I can only assume that since I have to use the Native WS for browsers, I have to initialize the websocket elsewhere, but I don't know where
Maintain state variable for connection. Create only one time and have close connection in effect. (Empty dependency array to run once)
const [connection, setConnection] = useState(null);
useEffect(() => {
connection = new WebSocket(url);
setConnection(connection);
return () => connection.close()
}, [])
useEffect(() => {
connection.send(messages)
}, [messages])
I know that Background Sync API is not supported in the apple ecosystem, so how would you get around it and make a solution that would work in the apple ecosystem and other platforms as well, now i have a solution that uses Background Sync API and for some reason it literally does not do anything on IOS, it just saves the failed requests, and then never sync-s, could i just access the sync queue somehow, with a indexedDB wrapper and then sync at an arbitrary time?
I tried it once and it broke everything, do you guys have an idea how?
const bgSyncPlugin = new workbox.backgroundSync.Plugin('uploadQueue', {
maxRetentionTime: 60 * 24 * 60,
onSync: async ({ queue }) => {
return getAccessToken().then((token) => {
replayQueue(queue, token).then(() => {
return showNotification();
});
});
},
});
This is the code i have, they all. have a purpose, since my token has a timeout i have to check if the token is expired or not and proceed after that and replace the token in the headers if it is expired, and i have to change data as well when i sync in the request bodies, but it all works good on anything other than apple devices. Apple devices never trigger the onsync, i tried to do listen to fetch events and trigger onsync with:
self.registration.sync.register('uploadQueue');
But to no awail, i tried to register sync on servvice worker registration, nothing seems to help.
If the sync registration is not viable on ios, then can i access the upload queue table somehow?
P.S.: I`m using dexie.js as a indexedDB wrapper, it is a vue.js app, with laravel api, and the sync process is quite complex, but it is working, just have to figure out how to do it on IOS!
I have found an answer to this after like 2 weeks of it being on my mind and on my to do list.
Now get some popcorn and strap yourself the heck in, because this is quite a chonker.
In my case the sync process was pretty complex as my users could be away from any connection for such a long time that my accessTokens would expire so i had to do a check for the access token expiration as well and reFetch it.
Furthermore my users could add new people to the database of people, which all had their on unique server side id-s, so i had to order my requests in a way that the person registrations are sent first then the tasks and campaigns that were completed for them, so i can receive the respective ids from the API.
Now for the fun part:
Firstly you cant use a bgSyncPlugin, because you cant access the replayQueue, you have to use a normal queue, like this:
var bgSyncQueue = new workbox.backgroundSync.Queue('uploadQueue', {
maxRetentionTime: 60 * 24 * 60,
onSync: () => syncData(),
});
And push the failed requests to the queue inside the fetch listener:
this.onfetch = (event) => {
let requestClone = event.request.clone();
if (requestClone.method === 'POST' && 'condition to match the requests you need to replay') {
event.respondWith(
(() => {
const promiseChain = fetch(requestClone).catch(() => {
return bgSyncQueue.pushRequest(event);
});
event.waitUntil(promiseChain);
return promiseChain;
})()
);
} else {
event.respondWith(fetch(event.request));
}
};
When user has connection we trigger the "syncData()" function, on ios this is a bit complicated(more on this later), on android it happens automatically, as the service worker sees it has connection, now lets just check out what syncData does:
async function syncData() {
if (bgSyncQueue) //is there data to sync?
return getAccessToken() //then get the access token, if expired refresh it
.then((token) => replayQueue(bgSyncQueue, token).then(() => showNotification({ body: 'Succsesful sync', title: 'Data synced to server' })))
.catch(() => showNotification({ title: 'Sync unsuccessful', body: 'Please find and area with better coverage' })); //replay the requests and show a notification
return Promise.resolve('empty');//if no requests to replay return with empty
}
For the android/desktop side of thing we are finished you can be happy with your modified data being synced, now on iOS we cant just have the users data be uploaded only when they restart the PWA, thats bad user experience, but we are playing with javascript everything is possible in a way or another.
There is a message event that can be fired every time that the client code sees that it has internet, which looks like this:
if (this.$online && this.isIOSDevice) {
if (window.MessageChannel) {
var messageChannel = new MessageChannel();
messageChannel.port1.onmessage = (event) => {
this.onMessageSuccess(event);
};
} else {
navigator.serviceWorker.onmessage = (event) => {
this.onMessageSuccess(event);
};
}
navigator.serviceWorker.ready.then((reg) => {
try {
reg.active.postMessage(
{
text: 'sync',
port: messageChannel && messageChannel.port2,
},
[messageChannel && messageChannel.port2]
);
} catch (e) {
//firefox support
reg.active.postMessage({
text: 'sync',
});
}
});
}
this is inside a Vue.js watch function, which watches whether we have connection or not, if we have connection it also checks if this is a device from the apple ecosystem, like so:
isIosDevice() {
return !!navigator.platform && /iPad|iPhone|MacIntel|iPod/.test(navigator.platform) && /^((?!chrome|android).)*safari/i.test(navigator.userAgent);
}
And so it tells the service worker that it has internet and it has to sync, in that case this bit of code gets activated:
this.onmessage = (event) => {
if (event.data.text === 'sync') {
event.waitUntil(
syncData().then((res) => {
if (res !== 'empty') {
if (event.source) {
event.source.postMessage('doNotification');//this is telling the client code to show a notification (i have a built in notification system into the app, that does not use push notification, just shows a little pill on the bottom of the app with the message)
} else if (event.data.port) {
event.data.port.postMessage('doNotification'); //same thing
}
return res;
}
})
);
}
};
Now the most useful part in my opinion, the replay queue function, this guy gets the queue and the token from getAccessToken, and then it does its thing like clockwork:
const replayQueue = async (queue, token) => {
let entry;
while ((entry = await queue.shiftRequest())) {//while we have requests to replay
let data = await entry.request.clone().json();
try {
//replay the person registrations first and store them into indexed db
if (isPersonRequest) {
//if new person
await fetchPerson(entry, data, token);
//then replay the campaign and task submissions
} else if (isTaskOrCampaignRequest) {
//if task
await fetchCampaigns(entry, data, token);
}
} catch (error) {
showNotification({ title: 'no success', body: 'go for better internet plox' });
await queue.unshiftRequest(entry); //put failed request back into queue, and try again later
}
}
return Promise.resolve();
};
Now this is the big picture as how to use this guy on iOS devices and make Apple mad as heck :) I am open to any questions that are related, in this time i think i have become pretty good with service worker related stuff as this was not the only difficult part of this project but i digress, thats a story for another day.
(you may see that error handling is not perfect and maybe this thing is not he most secure of them all, but this project has a prettty small amount of users, with a fixed number which know how to use it and what it does, so im not really afraid of security in this case, but you may want to improve on things if you use in in a more serious project)
Hope i could help and all of you have a grea day.
I am writing a web software with node js. I am also using Mssql database. I have a table that has a datetime value and a bit value. The bit value is always 0 until the real time is equals the value that datetime has. If they equals, the bit value will be 1.
Okey the question is "How can I check the realtime and the datetime value in the table, everytime?"
I used setInterval func and the server closed itself after 3 or 4 time. Weird thing is I can't see any error code or anything like that after the server closed.
I need help. If you have any idea about how to solve, please help me. I heard something about socket.io but I don't now how to adapt it.
Querying a database every second is probably too frequent. It's possible one query won't finish before the next one starts.
To work around this you should use a new database connection each time your setInterval handler runs. That way your new query still has a chance of working if the previous query is not done. Getting a new database connection each time is reasonable if you use a connection pool in your program, and request your new connections from that pool.
Something like this: not debugged.
const sql = require('mssql')
const pool = new sql.ConnectionPool(config)
const poolConnect = pool.connect()
pool.on('error', err => {
// ... error handler
})
function intervalTimer() {
return poolConnect.then((pool) => {
pool
.request()
.execute('SELECT whatever', (err, result) => {
// ... error checks
console.dir(result)
})
}).catch(err => {
// ... error handler
})
}
Or, you can even skip the query if the previous one isn't done. This code does that with an intervalTimerInProgress boolean value. It sets it right before starting the query, and clears it when the query finishes.
const sql = require('mssql')
const pool = new sql.ConnectionPool(config)
const poolConnect = pool.connect()
pool.on('error', err => {
// ... error handler
})
let intervalTimerInProgress = false
function intervalTimer() {
if (!intervalTimerInProgress) {
intervalTimerInProgress = true
return poolConnect.then((pool) => {
pool
.request()
.execute('SELECT whatever', (err, result) => {
// ... error checks
console.dir(result)
intervalTimerInProgress = false
})
}).catch(err => {
// ... error handler
})
}
}
Again, polling a DBMS once a second is a design choice you should reconsider. DBMSs are the scarcest resource and the bottleneck in many web apps, and polling them can put bigger loads on them than you want.
I'm creating a server in my Node.js application with net.createServer.
I add an event listener to the "connection"-event to be able to determine when a connection has been established. All well so far.
The problem is that when I disconnect it seems to trigger the "connection"-event again! And there is no "disconnect" event, so I don't know how to know if it is connected or disconnected!
const server = createServer({ allowHalfOpen: true }, (socket) => {
this._socket.on("data", (buffer) => {
// ...
});
this._socket.on("end", () => {
// ...
});
server.on("connection", () => {
console.log("Connected or Disconnected!")
});
How can I determine if the connection is established or ended?
I would like to avoid having a flag keeping track of the state if possible.
Thanks!
server.on("close", () => {
console.log("Disconnected!")
});
https://nodejs.org/api/http.html#http_event_close
Within the connection listening event, there is a listening event for disconnections:
var clients = [];
server.on('connection', (socket) => {
// add client
clients.push({id:15, name:'client15'});
socket.on('disconnect', () => {
// first search client and remove
});
});
Now the next question would be: how do I know which person disconnected?
The answer to this question is more than putting logic to the process and you can create a global variable for save client, when the person starts session or connects to the socket, it is registered in that variable and when it is disconnected it must be taken out of said variables, this is not ma than an example but there may be better ways.
So my naive approach to this issue that seems to work is to use getConnections, which will increase on every connection event, and see if it is even or uneven to be able to determine if it is connected or disconnected:
server.on("connection", () => {
server.getConnections((e, c) => {
if (c % 2 === 0) {
// Disconnected
} else {
// Connected
}
});
It does not feel like a good solution, but maybe good enough until someone can show me a better way.