I am trying to implement a auto-reconnect client for a server which receives a command and then replies with a single byte. However, the problem is that one cannot send any additional commands to the server while it is processing commands. So I need to somehow serialise the commands, is that possible to achieve in a pragmatic way in RxJS?
const onClient = new BehaviourSubject(...) // Auto-reconnecting client
function sendCommand(cmd) {
return onClient
.concatMap(client => {
client.write(cmd + '\r\n')
return Rx.Observable.fromEvent(client, 'data').take(1)
})
}
sendCommand('CMD1').subscribe(x => console.log(x))
sendCommand('CMD2').subscribe(x => console.log(x)) // Oops, sent a command while another one is active...
Here is one possible solution that lacks error handling and looks quite inefficient.
const input = new Rx.Subject()
const output = new Rx.Subject()
input.concatMap(({cmd, id})) => onClient
.filter(client => client != null)
.concatMap(client => {
client.write(cmd + '\r\n')
return Rx.Observable.fromEvent(client, 'data').take(1)
})
.map(value => ({value, id}))
.subscribe(output)
function sendCommand(cmd) {
const id = cuid()
input.onNext(id)
return output
.filter(res => res.id === id)
.map(res => res.value)
}
Any better ideas or suggestions on improvement?
Here is my gut instinct. I've only ever used JavaRX, and that just barely. Note that this assumes you want 1 invocation of CMD2 for every return of CMD1.
const onClient = new BehaviourSubject(...) // Auto-reconnecting client
function sendCommand(cmd) {
return onClient
.concatMap(client => {
client.write(cmd + '\r\n')
return Rx.Observable.fromEvent(client, 'data').take(1)
})
}
sendCommand('CMD1').subscribe(function(x) {
console.log(x);
sendCommand('CMD2').subscribe(y => console.log(y))
});
For what it's worth, you may want to consider using Promises for this stuff. My understanding of Rx is that it is useful for complex streams of async data, such as event streams. But if all you want is the async part, I believe Promises might be easier. We were considering using it on a Java project and decided it wasn't what we needed. See: When to Use Rx
I don't know what you are working on, but a command-response pattern seems to me like it might be better served by Promises, especially if you expect the lambda you're passing into subscribe to only be invoked once.
Here is the rather complicated try I ended up with:
import stampit from 'stampit'
import Rx from 'rx'
import cuid from 'cuid'
let input = new Rx.Subject()
let output = new Rx.Subject()
input
.concatMap(({fn, id}) => Rx.Observable
.defer(() => fn())
.map(value => ({value, id}))
.catch(error => Rx.Observable.return({error, id}))
.concat(Rx.Observable.return({id})))
.subscribe(output)
async function enqueue(fn) {
const id = cuid()
input.onNext({fn, id})
output
.filter(res => res.id === id)
.takeWhile(res => res.error || res.value)
.concatMap(res => res.error
? Rx.Observable.throw(res.error)
: Rx.Observable.return(res.value))
}
})
const onClient = new BehaviourSubject(...) // Auto-reconnecting client
function sendCommand(cmd) {
return enqueue(() => onClient
.concatMap(client => {
client.write(cmd + '\r\n')
return Rx.Observable.fromEvent(client, 'data').take(1)
}))
}
sendCommand('CMD1').subscribe(x => console.log(x))
sendCommand('CMD2').subscribe(x => console.log(x))
Related
I have some difficulties with some rxjs calls and I could use some help. My problems is to parcours a list of ids, make an async call for each id and then pass on the next id once this async call is finish. Unfortunaly I did not manage to find a proper solution yet.
Here my code so far:
import { from, interval, of, Subject, timer } from 'rxjs';
import { concatMap, filter, flatMap, map, take } from 'rxjs/operators';
const callFinised$ = new Subject();
const makeAsyncCall = val => {
console.log('start call for ', val);
interval(1000)
.pipe(take(1))
.subscribe(() => {
callFinised$.next(true);
console.log('call finished for ', val);
});
};
from(['id1', 'id2'])
.pipe(
concatMap(val => {
return of(val).pipe(
flatMap(() => {
makeAsyncCall(val);
return callFinised$;
}),
filter(callFinised => callFinised === true),
take(1),
map(() => {
return val + '_updated';
})
);
})
)
.subscribe(finalVal => console.log('finalVal', finalVal));
Here the stackblitz associated: stackblitz
The second id is emited before the first async is finished, so I'm a bit lost.
Best regards
Here's a version of your code that (I think) does what you're after.
const makeAsyncCall = val => defer(() => {
console.log('start call for ', val);
return timer(1000).pipe(
map(_ => true),
tap(_ => console.log('call finished for ', val))
);
});
from(['id1', 'id2']).pipe(
concatMap(id => makeAsyncCall(id).pipe(
filter(callFinished => callFinished === true),
map(_ => id + '_updated')
))
).subscribe(finalVal => console.log('finalVal', finalVal));
Update:
Defer doesn't accomplish anything for you in the code above since you call makeAsyncCall at basically the same time you subscribe to the returned observable.
However, creation and subscription to Observables are different things.
In the following code, I create all the Observables first and then afterward I concatenate them and call them one after another. Without defer, this code wouldn't output what you're expecting.
const idCalls = ["id1", "id2"].map(id =>
makeAsyncCall(id).pipe(
filter(callFinished => callFinished === true),
map(_ => id + '_updated')
)
);
concat(...idCalls).subscribe(
finalVal => console.log('finalVal', finalVal)
);
I'm writing a server route that makes api calls.
I need to make two different fetch requests cause I need more info that's coming in the first fetch.
The problem is that I'm declaring a variable out of the promise scope and for some reason, my res.send is not awaiting until the array gets full.
I need to iterate until result 9 (I can't use theDogApi's predefined filters to show nine results!)
if (req.query.name) {
var myRes = [];
fetch(`https://api.thedogapi.com/v1/breeds/search?name=${req.query.name}&apikey=${key}`)
.then(r => r.json())
.then( data => {
for (let i = 0; i < 8 && i < data.length; i++) {
fetch(`https://api.thedogapi.com/v1/images/${data[i].reference_image_id
}`)
.then(r => r.json())
.then(datos => {
myRes.push({ ...data[i], ...datos });
})
}
})
.then(res.send(myRes))
}
I'll appreciate the help!
You can try using Promise.all to turn your array of fetch calls into an aggregate promise that resolves to an array of responses when all have arrived. If any fail, the whole thing fails (use Promise.allSettled if you don't want all-or-nothing semantics). Don't forget to catch the error.
Although the code doesn't show it, be sure to check response.ok to make sure the request actually succeeded before calling .json(). Throwing an error if !repsonse.ok and handling it in the .catch block is a typical strategy. Writing a wrapper on fetch is not a bad idea to avoid verbosity.
Lastly, note that Array#slice replaces the for loop. For arrays with fewer than 8 elements, it'll slice as many as are available without issue.
// mock everything
const fetch = (() => {
const responses = [
{
json: async () =>
[...Array(10)].map((e, i) => ({reference_image_id: i}))
},
...Array(10)
.fill()
.map((_, i) => ({json: async () => i})),
];
return async () => responses.shift();
})();
const req = {query: {name: "doberman"}};
const key = "foobar";
const res = {send: response => console.log(`sent ${response}`)};
// end mocks
fetch(`https://api.thedogapi.com/v1/breeds/search?name=${req.query.name}&apikey=${key}`)
.then(response => response.json())
.then(data =>
Promise.all(data.slice(0, 8).map(e =>
fetch(`https://api.thedogapi.com/v1/images/${e.reference_image_id}`)
.then(response => response.json())
))
)
.then(results => res.send(results))
.catch(err => console.error(err))
;
Here is an example of an async function unsing await:
async function fun(queryName, key){
const a = [], p, j = [];
let firstWait = await fetch(`https://api.thedogapi.com/v1/breeds/search?name=${req.query.name}&apikey=${key}`);
let firstJson = await firstWait.json(); // must be an Array
for(let i=0,n=8,j,l=firstJson.length; i<n && i<l; i++){
a.push(fetch('https://api.thedogapi.com/v1/images/'+firstJson[i].reference_image_id));
}
p = await Promise.all(a);
for(let v of p){
j.push(v.json());
}
return Promise.all(j);
}
// assumes req, req.query, req.query.name, and key are already defined
fun(req.query.name, key).then(a=>{
// a is your JSON Array
});
JSON
Here's my hot take: Stop using low-level functions like fetch every time you want to get JSON. This tangles up fetching logic every time we want to get a bit of JSON. Write getJSON once and use it wherever you need JSON -
const getJSON = s =>
fetch(s).then(r => r.json())
const data =
await getJSON("https://path/to/some/data.json")
// ...
URL and URLSearchParams
Another hot take: Stop writing all of your URLs by hand. This tangles URL writing/rewriting with all of your api access logic. We can setup a DogApi endpoint once, with a base url and an apikey -
const DogApi =
withApi("https://api.thedogapi.com/v1", {apikey: "0xdeadbeef"})
And now whenever we need to touch that endpoint, the base url and default params can be inserted for us -
const breed =
// https://api.thedogapi.com/v1/breeds/search?apikey=0xdeadbeef&name=chihuahua
await getJSON(DogApi("/breeds/search", {name}))
// ...
withApi has a simple implementation -
const withApi = (base, defaults) => (pathname, params) =>
{ const u = new URL(url) // <- if you don't know it, learn URL
u.pathname = pathname
setParams(u, defaults)
setParams(u, params)
return u.toString()
}
function setParams (url, params = {})
{ for (const [k,v] of Object.entries(params))
url.searchParams.set(k, v) // <- if you don't know it, learn URLSearchParams
return url
}
fruits of your labor
Now it's dead simple to write functions like imagesForBreed, and any other functions that touch JSON or your DogApi -
async function imagesForBreed (name = "")
{ if (name == "")
return []
const breed =
await getJSON(DogApi("/breeds/search", {name}))
const images =
data.map(v => getJSON(DogAPI(`/images/${v.reference_image_id}`))
return Promise.all(images)
}
And your entire Express handler is reduced to a single line, with no need to touch .then or other laborious API configuration -
async function fooHandler (req, res)
{
res.send(imagesForBreed(req.query.name))
}
I am trying to set up an observable that currently receives an array of location IDs and then makes a get request for all of these at once and waits for the response for them all. Here is a sample:
const fetchPhotosEpic = action$ =>
action$.ofType(LOCATIONS_RECEIVED)
.map(action => action.payload)
.mergeMap((data) => {
let promiseArray = data.map(location => Observable.fromPromise(axios.get(photosUrl(location.id))))
return Observable.forkJoin(
promiseArray
)
})
.map(responses => responses.map((response) => response.data.location))
Where data looks like:
[
{
id: "aoeuaeu",
name: "Test"
},
...
]
The issue I have right now is I get a 404 on one of the requests and it's messing everything up. I am probably doing something wrong as I am just learning RX. Any help would be great!
You can try adding a catch to each call and returning a new observable with the error message, which should stop the forkJoin failing if one request fails. You can then either filter out the failures, or add logic to handle them in your final .map. eg.
const fetchPhotosEpic = action$ =>
action$.ofType(LOCATIONS_RECEIVED)
.map(action => action.payload)
.mergeMap((data) => {
let promiseArray = data.map(location => {
return Observable.fromPromise(axios.get(photosUrl(location.id)))
.catch(error => Observable.of({error}))
})
return Observable.forkJoin(
promiseArray
)
})
.filter(response => !Boolean(response.error))
.map(responses => responses.map((response) => response.data.location))
The following code
const endpoint = 'https://raw.githubusercontent.com/Hipo/university-domains-list/master/world_universities_and_domains.json';
const universities = [];
fetch(endpoint)
.then(results => results.json())
.then(data => universities.push(...data));
console.log(universities);
function findMatches(wordToMatch, universities) {
return universities.filter(uni => {
const regex = new RegExp(wordToMatch, 'gi');
return uni.name.match(regex)
})
}
export default findMatches;
returns the error below
'Uncaught TypeError: Cannot read property 'filter' of undefined'
I can log the data using console.log(universities). So why can't I filter through it? FYI the data is an array of objects. Any help is massivaly appreciated. Thank you.
You need to remove universities as a parameter in your findMatches function because it is overriding the value of the local universities variable:
function findMatches(wordToMatch) {
return universities.filter(uni => {
const regex = new RegExp(wordToMatch, 'gi');
return uni.name.match(regex)
})
}
You can then proceed to use the findMatches function as follows:
findMatches("hi") // returns a filtered array
Edit:
You have a race condition in your code, wherein findMatches might be called
before your fetch is complete. To fix the issue, findMatches should return a promise like so:
const endpoint = 'https://raw.githubusercontent.com/Hipo/university-domains-list/master/world_universities_and_domains.json';
const universities = [];
const promise = fetch(endpoint)
.then(results => results.json())
.then(data => universities.push(...data));
console.log(universities);
function findMatches(wordToMatch) {
return promise.then(() => universities.filter(uni => {
const regex = new RegExp(wordToMatch, 'gi');
return uni.name.match(regex)
}));
}
findMatches("hi").then(arr => console.log(arr));
If you're absolutely sure that findMatches will always be called after fetch is complete, you can go with the first solution. Otherwise, it is strongly recommended you go with the second solution that uses promises.
I just want to let everybody know I finally got things working. I had to install babel-polyfill and babel-preset-env and add to webpack to get UglifyJS to work with async await and to optimise bundle size.
I had to use async await instead of regular promises to get the HTML to render into the DOM for some reason, not sure why. Anyway here is the code that finally worked as expected:
UniFetch.js
const endpoint = 'https://raw.githubusercontent.com/Hipo/university-domains-list/master/world_universities_and_domains.json';
const universities = [];
const promise = fetch(endpoint)
.then(blob => blob.json())
.then(data => universities.push(...data.slice(8286, 8456)));
function findMatches(wordToMatch) {
return promise.then(() => universities.filter(uni => {
const regex = new RegExp(wordToMatch, 'gi');
return uni.name.match(regex)
}));
}
async function displayMatches() {
searchResults.innerHTML = await findMatches(this.value)
.then(arr => arr.map(uni => {
return `
<li>${uni.name}</li>
`
}));
}
const searchInput = document.querySelector("input[name='unisearch']");
const searchResults = document.querySelector('.unisearch__results');
searchInput.addEventListener('change', displayMatches);
searchInput.addEventListener('keyup', displayMatches);
export default findMatches
App.js
import FindMatches from '../UniFetch.js'
FindMatches()
Hope this helps some people to implement a typeahead, autocomplete API fetch.
The code below works just fine. Console outputs an array [1, 2].
const getAsyncValue = () => {
return new Promise(resolve => {
setTimeout(() => {
resolve(1);
}, 1000)
})
}
const getSyncValue = () => {
return Rx.Observable.of(2);
}
const observer = (arrayOfValues) => {
console.log(arrayOfValues);
}
Rx.Observable.of(getPromise(), getSyncValue())
.concatAll()
.toArray()
.subscribe(observer)
I'd like to change function getSyncFunction to the following (because in a real world application this function might not have a reference to RxJs library):
const getSyncValue = () => {
return 2;
}
If I just do that without anything else I get an error:
You provided '2' where a stream was expected
What other changes to the code do I need? Maybe hint me the operator to use.
The problem is not in getSyncValue() but in concatAll() that works with higher-order Observables. When you pass it just 2 it throws the error. Using Rx.Observable.of(2) is correct because it's an Observable that emits a single value 2 which is remitted by concatAll().
I don't know what your code should do but you could do for example:
Rx.Observable.of(getPromise(), getSyncValue())
.map(v => typeof(v) == 'object' ? v : Rx.Observable.of(v))
.concatAll()
...
However, I do recommend to rethink this because I guess you could use some easier approach than this.