Develop Reference

How to merge and observe two collections in Firestore based on reference ID in documents?

I'm creating a StencilJS app (no framework) with a Google Firestore backend, and I want to use the RxFire and RxJS libraries as much as possible to simplify data access code. How can I combine into a single observable stream data coming from two different collections that use a reference ID?
There are several examples online that I've read through and tried, each one using a different combination of operators with a different level of nested complexity. https://www.learnrxjs.io/ seems like a good resource, but it does not provide line-of-business examples that make sense to me. This question is very similar, and maybe the only difference is some translation into using RxFire? Still looking at that. Just for comparison, in SQL this would be a SELECT statement with an INNER JOIN on the reference ID.
Specifically, I have a collection for Games:
{ id: "abc000001", name: "Billiards" },
{ id: "abc000002", name: "Croquet" },
...
and a collection for Game Sessions:
{ id: "xyz000001", userId: "usr000001", gameId: "abc000001", duration: 30 },
{ id: "xyz000002", userId: "usr000001", gameId: "abc000001", duration: 45 },
{ id: "xyz000003", userId: "usr000001", gameId: "abc000002", duration: 55 },
...
And I want to observe a merged collection of Game Sessions where gameId is essentially replace with Game.name.
I current have a game-sessions-service.ts with a function to get sessions for a particular user:
import { collectionData } from 'rxfire/firestore';
import { Observable } from 'rxjs';
import { GameSession } from '../interfaces';
observeUserGameSesssions(userId: string): Observable<GameSession[]> {
let collectionRef = this.db.collection('game-sessions');
let query = collectionRef.where('userId', '==', userId);
return collectionData(query, 'id);
}
And I've tried variations of things with pipe and mergeMap, but I don't understand how to make them all fit together properly. I would like to establish an interface GameSessionView to represent the merged data:
export interface GameSessionView {
id: string,
userId: string,
gameName: string,
duration: number
}
observeUserGameSessionViews(userId: string): Observable<GameSessionView> {
this.observeUserGameSessions(userId)
.pipe(
mergeMap(sessions => {
// What do I do here? Iterate over sessions
// and embed other observables for each document?
}
)
}
Possibly, I'm just stuck in a normalized way of thinking, so I'm open to suggestions on better ways to manage the data. I just don't want too much duplication to keep synchronized.

You can use the following code (also available as Stackblitz):
const games: Game[] = [...];
const gameSessions: GameSession[] = [...];
combineLatest(
of(games),
of(gameSessions)
).pipe(
switchMap(results => {
const [gamesRes, gameSessionsRes] = results;
const gameSessionViews: GameSessionView[] = gameSessionsRes.map(gameSession => ({
id: gameSession.id,
userId: gameSession.userId,
gameName: gamesRes.find(game => game.id === gameSession.gameId).name,
duration: gameSession.duration
}));
return of(gameSessionViews);
})
).subscribe(mergedData => console.log(mergedData));
Explanation:
With combineLatest you can combine the latest values from a number of Obervables. It can be used if you have "multiple (..) observables that rely on eachother for some calculation or determination".
So assuming you lists of Games and GameSessions are Observables, you can combine the values of each list.
Within the switchMap you create new objects of type GameSessionView by iterating over your GameSessions, use the attributes id, userId and duration and find the value for gameName within the second list of Games by gameId. Mind that there is no error handling in this example.
As switchMap expects that you return another Observable, the merged list will be returned with of(gameSessionViews).
Finally, you can subscribe to this process and see the expected result.
For sure this is not the only way you can do it, but I find it the simplest one.

In rxjs, how do I chain mapping through arrays of data received from different API's?

I'm calling an API and receiving an array of results, I'm checking for pagination and if more pages exist I call the next page, repeat until no more pages.
For each array of results, I call another endpoint and do the exact same thing: I receive an array of results, check for another page and call endpoint again. Wash, rinse repeat.
For instance:
I want to grab a list of countries that might be a paginated response, then for each country I want to grab a list of cities, which might also be paginated. And for each city I execute a set of transformations and then store in a database.
I already tried this, but got stuck:
const grabCountries = Observable.create(async (observer) => {
const url = 'http://api.com/countries'
let cursor = url
do {
const results = fetch(cursor)
// results = {
// data: [ 'Canada', 'France', 'Spain' ],
// next: '47asd8f76358df8f4058898fd8fab'
// }
results.data.forEach(country => { observer.next(country) })
cursor = results.next ? `${url}/${results.next}` : undefined
} while(cursor)
})
const getCities = {
next: (country) => {
const url = 'http://api.com/cities'
let cursor = url
do {
const results = fetch(cursor)
// results = {
// data: [
// 'Montreal', 'Toronto',
// 'Paris', 'Marseilles',
// 'Barcelona', 'Madrid'
// ],
// next: '89ghjg98nd8g8sdfg98gs9h868hfoig'
// }
results.data.forEach(city => {
`**** What do I do here?? ****`
})
cursor = results.next ? `${url}/${results.next}` : undefined
} while(cursor)
}
}
I tried a few approaches:
Making a subject (sometimes I'll need to do parallel processed base on the results of 'grabCountries'. For example I may want to store the countries in a DB in parallel with grabbing the Cities.)
const intermediateSubject = new Subject()
intermediateSubject.subscribe(storeCountriesInDatabase)
intermediateSubject.subscribe(getCities)
I also tried piping and mapping, but it seems like it's basically the same thing.
As I was writing this I thought of this solution and it seems to be working fine, I would just like to know if I'm making this too complicated. There might be cases where I need to make more that just a few API calls in a row. (Imagine, Countries => States => Cities => Bakeries => Reviews => Comments => Replies) So this weird mapping over another observer callback pattern might get nasty.
So this is what I have now basically:
// grabCountries stays the same as above, but the rest is as follows:
const grabCities = (country) =>
Observable.create(async (observer) => {
const url = `http://api.com/${country}/cities`
let cursor = url
do {
const results = fetch(cursor)
// results = {
// data: [
// 'Montreal', 'Toronto',
// 'Paris', 'Marseilles',
// 'Barcelona', 'Madrid'
// ],
// next: '89ghjg98nd8g8sdfg98gs9h868hfoig'
// }
results.data.forEach(city => {
observer.next(city)
})
cursor = results.next ? `${url}/${results.next}` : undefined
} while (cursor)
})
const multiCaster = new Subject()
grabCountries.subscribe(multiCaster)
multiCaster.pipe(map((country) => {
grabCities(country).pipe(map(saveCityToDB)).subscribe()
})).subscribe()
multiCaster.pipe(map(saveCountryToDB)).subscribe()
tl;dr - I call an API that receives a paginated set of results in an array and I need to map through each item and call another api that receives another paginated set of results, each set also in an array.
Is nesting one observable inside another and mapping through the results via 'callApiForCountries.pipe(map(forEachCountryCallApiForCities))' the best method or do you have any other recommendations?

Here's the code that should work with sequential crawling of next url.
You start with a {next:url} until res.next is not available.
of({next:http://api.com/cities}).pipe(
expand(res=>results.next ? `${url}/${results.next}` : undefined
takeWhile(res=>res.next!==undefined)
).subscribe()

OK, so I have spent a lot of brain power on this and have come up with two solutions that seem to be working.
const nestedFlow = () => {
fetchAccountIDs.pipe(map(accountIds => {
getAccountPostIDs(accountIds) // Has the do loop for paging inside
.pipe(
map(fetchPostDetails),
map(mapToDBFormat),
map(storeInDB)
).subscribe()
})).subscribe()
}
const expandedflow = () => {
fetchAccountIDs.subscribe((accountId) => {
// accountId { accountId: '345367geg55sy'}
getAccountPostIDs(accountId).pipe(
expand((results) => {
/*
results : {
postIDs: [
131424234,
247345345,
],
cursor: '374fg8v0ggfgt94',
}
*/
const { postIDs, cursor } = results
if (cursor) return getAccountPostIDs({...accountId, cursor})
return { postIDs, cursor }
}),
takeWhile(hasCursor, true), // recurs until cursor is undefined
concatMap(data => data.postIDs),
map(data => ({ post_id: data })),
map(fetchPostDetails),
map(mapToDBFormat),
map(storeInDB)
).subscribe()
})
}
Both seem to be working with similar performance. I read some where that leaving the data flow is a bad practice and you should pipe everything, but I don't know how to eliminate the first exit in the 'expandedFlow' because the 'expand' needs to call back an observable, but maybe it can be done.
Now I just have to solve the race condition issues from the time the 'complete' is called in getAccountPostIDs the the last record is stored in the DB. Currently in my test, the observer.complete is finishing before 3 of the upsert actions.
Any comments are appreciated and I hope this helps someone out in the future.

What you need is the expand operator. It behaves recursively so it fits the idea of having paginated results.

#ngrx return one object from array with .single

I want to return an Observable of one object, selected from an array of objects from my store. I would like to use the .single operator, because this would throw an exception if there are less or more than 1 objects present. I can't get it to work though, because my store is returning an array instead of an observable. I know I could use .filter on the array, but I would like to understand why I can't get it to work. I even tried using a selector.
My App reducer has:
export interface AppState {
batStores: fromBatStores.State;
}
export const reducers: ActionReducerMap<AppState> = {
batStores: fromBatStores.batStoresReducer,
};
The reducer of my module:
export interface State {
batStores: BatStore[];
}
const initialState = {
batStores: [
new BatStore(0, 0, true, 'Cell 11', 11, 11),
new BatStore(1, 0, false, 'Cel 12', 12, 11),
new BatStore(2, 0, true, 'Cel 13', 13, 11),
new BatStore(2, 0, true, 'Cel 14', 14, 11),
]
};
export const selectBatStores = (state: fromApp.AppState) => state.batStores;
export const selectBatStoresBatStores = createSelector(selectBatStores, (state: State) => state.batStores);
Now I try something like this in my ngOnInit:
ngOnInit() {
const editId = 11;
const batStore$: Observable<BatStore> = this.store.select(fromBatStores.selectBatStoresBatStores)
.single(bs => bs.number === editId);
this.batStore$ = batStore$;
}
PhpStorm says the number property doesn't exist on BatStore[]. Which I don't understand, because at the example documentation single is also used as an operator on observable array.
Update
After reading the answers, still very confused. To test, I wrote my function like this:
getBatStore(id: number) {
const versionA$ = this.store.select(fromBatStores.selectBatStoresBatStores)
.switchMap((batStores: BatStore[]) => Observable.from(batStores))
.filter((bs: BatStore) => {
console.log('Analyzing ', bs);
console.log('Result ', bs.number === id);
return bs.number === id;
});
const versionB$ = this.store.select(fromBatStores.selectBatStoresBatStores)
.switchMap((batStores: BatStore[]) => Observable.from(batStores))
.single((bs: BatStore) => {
console.log('Analyzing ', bs);
console.log('Result ', bs.number === id);
return bs.number === id;
});
console.log('VersionA: ', versionA$);
console.log('VersionB: ', versionB$);
console.log('Same? ', versionA$ === versionB$);
return versionB$;
}
Both versions output the same thing to the console. VersionA works when using with async pipe in my view, VersionB doesn't. The code compiles and PhpStorm doesn't complain.

The select is getting a reference to the whole array and so the number property does not exist on it. Try a switchMap to switch to a new observable that expands the list so they can be selected one at a time like so:
const batStore$: Observable<BatStore> = this.store.select(fromBatStores.selectBatStoresBatStores)
.switchMap((batStores : BatStore[]) => Observable.from(batStores))
.single((bs: BatStore) => bs.number === editId);

I understand your confusion, PhpStorm is right to say "the number property doesn't exist on BatStore[]". The example documentation is also correct. The key here, in this example from RxJs, is you need to know how the Observable.from([array]) works. An Observable.from([array]) creates a "cold" observable that does one emission for every item in the array. So, the resulting Observable is of type number, not of type array, as you can see here:
const source: Observable<number> = Rx.Observable.from([1,2,3,4,5]);
The Observable that are you facing in your store, is an Observable of type Array. For example if you use the operator Observable.of, this will emit the array, not every item of the array:
const source: Observable<number[]> = Rx.Observable.of([1,2,3,4,5]);
Said that, I would do an Array.filter, for me it's better than any other transformation on the Observable stream.
Let me know if you have any doubt,
Hope this helps.

The select method will retorn an array object, for that reason you cant access the number property. You could try the following:
ngOnInit() {
const editId = 11;
this.batStore$ = this.store.select(fromBatStores.selectBatStoresBatStores)
.flatMap(list => list)// flat list to single elements, emit 1 value per element
.single(bs => bs.number === editId);
}
Another approach would be to store the elements in your state as key,value pairs and then either create a selector that takes the Id argument, or store that Id in the state and then compose 2 selectors (1 for collection of elements, 1 for Id) into one.

Updating json with lodash

Actually I need to handle mysite frontend fully with json objects(React and lodash).
I am getting the initial data via an ajax call we say,
starred[] //returns empty array from server
and am adding new json when user clicks on star buton it,
starred.push({'id':10,'starred':1});
if the user clicks again the starred should be 0
current_star=_findWhere(starred,{'id':10});
_.set(curren_star,'starred',0);
but when doing console.log
console.log(starred); //returns
[object{'id':10,'starred':0}]
but actually when it is repeated the global json is not updating,while am performing some other operations the json is like,
console.log(starred); //returns
[object{'id':10,'starred':1}]
How to update the global , i want once i changed the json, it should be changed ever.Should I get any idea of suggesting some better frameworks to handle json much easier.
Thanks before!

Working with arrays is complicated and usually messy. Creating an index with an object is usually much easier. You could try a basic state manager like the following:
// This is your "global" store. Could be in a file called store.js
// lodash/fp not necessary but it's what I always use.
// https://github.com/lodash/lodash/wiki/FP-Guide
import { flow, get, set } from 'lodash/fp'
// Most basic store creator.
function createStore() {
let state = {}
return {
get: path => get(path, state),
set: (path, value) => { state = set(path, value, state) },
}
}
// Create a new store instance. Only once per "app".
export const store = createStore()
// STARRED STATE HANDLERS
// Send it an id and get back the path where starred objects will be placed.
// Objects keyed with numbers can get confusing. Creating a string key.
const starPath = id => ['starred', `s_${id}`]
// Send it an id and fieldId and return back path where object will be placed.
const starField = (id, field) => starPath(id).concat(field)
// import to other files as needed
// Add or replace a star entry.
export const addStar = item => store.set(starPath(item.id), item)
// Get a star entry by id.
export const getStar = flow(starPath, store.get)
// Get all stars. Could wrap in _.values() if you want an array returned.
export const getStars = () => store.get('starred')
// Unstar by id. Sets 'starred' field to 0.
export const unStar = id => store.set(starField(id, 'starred'), 0)
// This could be in a different file.
// import { addStar, getStar, getStars } from './store'
console.log('all stars before any entries added:', getStars()) // => undefined
const newItem = { id: 10, starred: 1 }
addStar(newItem)
const star10a = getStar(10)
console.log('return newItem:', newItem === star10a) // => exact match true
console.log('star 10 after unstar:', star10a) // => { id: 10, starred: 1 }
console.log('all stars after new:', getStars())
// Each request of getStar(10) will return same object until it is edited.
const star10b = getStar(10)
console.log('return same object:', star10a === star10b) // => exact match true
console.log('return same object:', newItem === star10b) // => exact match true
unStar(10)
const star10c = getStar(10)
console.log('new object after mutate:', newItem !== star10c) // => no match true
console.log('star 10 after unstar:', getStar(10)) // => { id: 10, starred: 0 }
console.log('all stars after unstar:', getStars())

I think the problem is in mutating original state.
Instead of making push, you need to do the following f.e.:
var state = {
starred: []
};
//perform push
var newItem = {id:10, starred:1};
state.starred = state.starred.concat(newItem);
console.log(state.starred);
//{ id: 10, starred: 1 }]
var newStarred = _.extend({}, state.starred);
var curr = _.findWhere(newStarred, {id: 10});
curr.starred = 0;
state = _.extend({}, state, {starred: newStarred});
console.log(state.starred)
//{ id: 10, starred: 0 }]
To solve this in a more nice looking fashion, you need to use either React's immutability helper, or ES6 stuff, like: {...state, {starred: []}} instead of extending new object every time. Or just use react-redux =)

How to debounce by ID in RxJS

My problem is the next: I want to debounce the liking functionality of my app. I using actions to make changes in my app, for example:
dispatch(likePost(1)) // => dispatch({ type: 'LIKE_POST', id: 1 })
Given the next example:
I dispatch an action at time: 0
dispatch(likePost(1))
This gonna trigger my actionSource:
actionSource$.
pipe(
filter(isActionOf(likePost)),
debounceTime(500)
mapTo('make-api-request-to-like-the-post')
)
So it's gonna looks like:
--DISPATCH(LIKE(1))---DEBOUNCE(500)---MAKE_API_CALL(1)---->
But there is a problem with this, as what happens if I make the next actions:
dispatch(likePost(1)) // at time 0ms
dispatch(likePost(2)) // at time 200ms
here we got a problem as the likePost(2) debounced the likePost(1), resulting only making a request with the likePost(2)
--DISPATCH(LIKE(1))--200ms--DISPATCH(LIKE(2))--DEBOUNCE(500)--MAKE_API_CALL(2)--->
So how could I debounce by id, or make it unique branches by id.

You can groupBy events by id and then apply debounceTime on each group individually.
For example:
const { of } = rxjs; // = require("rxjs")
const { map, mergeMap, groupBy, debounceTime } = rxjs.operators; // = require("rxjs/operators")
of(1, 1, 2, 3, 1, 1).pipe(
groupBy(id => id),
mergeMap(likeById$ =>
likeById$.pipe(
debounceTime(500)
)
)
).subscribe(e => console.log(e));
<script src="https://unpkg.com/rxjs#6.5.3/bundles/rxjs.umd.min.js"></script>