Develop Reference

How to call a function after subscriber complete in angular 8+

Hello everyone I am learning angular and firebase. So the idea is I fetch some ticket from my ticket collection and after fetching add some new property in it but the problem is when I receive half data my ordering function call (the function which is responsible for adding a property). or in simple word we can say I receive data in stream form.
get_ticket() {
console.log('have permission');
this.get_ticket_service
.get_ticket_company(this.user.company)
.subscribe((res) => {
console.log('get response');
this.unStructure_ticket = res.map((e) => {
return {
id: e.payload.doc.id,
item: e.payload.doc.data(),
};
});
this.odering_ticket(this.unStructure_ticket)
});
ordering function
odering_ticket(data) {
const info = [];
console.log('hi');
data.map((ticket) => {
if (ticket.item.seen == false) {
ticket.item.ticketID = ticket.id;
ticket.item.ticketType = 'new';
info.push(ticket.item);
} else if (
ticket.item.new == true &&
ticket.item.currently_handle_sme_ID == undefined
) {
ticket.item.ticketID = ticket.id;
ticket.item.ticketType = 'not assigned';
info.push(ticket.item);
} else if (
ticket.item.currently_handle_sme_ID == localStorage.getItem('uid') &&
!this.has_view_all_ticket_permission
) {
ticket.item.ticketID = ticket.id;
ticket.item.ticketType = 'assigned';
info.push(ticket.item);
} else if (
ticket.item.currently_handle_sme_ID != undefined &&
this.has_view_all_ticket_permission
) {
ticket.item.ticketID = ticket.id;
ticket.item.ticketType = 'assigned';
info.push(ticket.item);
}
});
console.log('end map');
return info;
}
service.ts
get_ticket_company(company) {
return this.firebase_store
.collection('Ticket', (ref) => ref.where('company', '==', company))
.snapshotChanges();
}
output
have permission ,
get response ,
hi ,
end map ,
get response ,
hi ,
end map,
I want to call my ordering function one's after I receive all data

You must try to use the last RxJs operator which will make your pipeline wait until the last emitted value.
this.get_ticket_service
.get_ticket_company(this.user.company).pipe(
tap((poRes) => {
console.log('get response');
if (this.unStructure_ticket === undefined) {
// Initialize your data structure.
this.unStructure_ticket = res.map((e) => {
return {
id: e.payload.doc.id,
item: e.payload.doc.data(),
};
});
} else {
// Add or concate the data received
// this.unStructure_ticket.push()//;
}
}),
last(), // Wait for the last emitted value.
tap((poRes) => {
this.odering_ticket(this.unStructure_ticket);
}),
).subscribe();
Example of how this should work, but not tested code.

multiple http.post in Angular

I am trying to send an http.post request for each element of an array, my method works well, but when I subscribe, it does it for each of the requests, if someone could help me optimize this, I will I would really appreciate it, here I leave the snippets of my code.
component.ts
saveExclusion() {
this.indForm.value.Centers.forEach(element => {
for (const days of this.exclusionDays) {
delete days.horadesde;
delete days.horahasta;
delete days.id;
for (const key in days) {
if (days[key] === true) {
days[key] = true;
}else if (days[key] === false) {
delete days[key];
}
}
}
const valueForm = this.indForm.value;
valueForm.ResourceId = this.idResource;
valueForm.TimeZoneId = 'America/Santiago';
valueForm.CenterId = element;
this.exclusionFunc = false;
this.apiFca.saveNew(valueForm, this.exclusionDays)
.pipe(last()).subscribe((res: any) => {
console.log(res)
if (res === '200') {
this.successMessage = true;
this.exclusionDays = [];
this.indForm.reset();
this.ngOnInit();
setTimeout(() => {
this.successMessage = false;
}, 3000);
}
}, err => {
console.log('error', err);
});
});
}
service.ts
saveNew(exclusionData, daysBlock) {
let reason = '';
const dt = new Date();
const n = dt.getTimezoneOffset();
const tz = new Date(n * 1000).toISOString().substr(14, 5);
if (exclusionData.OtherReason) {
reason = exclusionData.ExclusionReason + ' ' + exclusionData.OtherReason;
} else {
reason = exclusionData.ExclusionReason;
}
if (exclusionData.ExclusionType !== 'Partial' ) {
daysBlock = [];
}
const data = {Exclusion: new ExclusionClass(
[],
reason,
exclusionData.ExclusionType,
exclusionData.Repetition,
exclusionData.CenterId,
exclusionData.ProfessionalName,
exclusionData.ResourceId,
daysBlock,
exclusionData.TimeZoneId,
'Exclude',
exclusionData.Unit,
exclusionData.ValidFrom + 'T' + exclusionData.ValidTimeFrom + ':00-' + tz,
exclusionData.ValidTo + 'T' + exclusionData.ValidTimeUntil + ':59.999-' + tz
)};
if (exclusionData.CenterId === '') {
delete data.Exclusion.CenterId;
}
return this.http
.post("url", data)
.pipe(
map((res: any) => {
return res.code;
})
);
}
greetings, and I look forward to your comments, thanks.

I'm not fully confident in my rxjs knowledge but it looks like, because of .pipe(last()), you are only watching the last request? I'd recommend you only set success if all completed without error, like
this.apiFca.saveNew(valueForm, this.exclusionDelays)
.subscribe(
res => {
console.log(res);
},
err => {
console.log(err);
},
() => {
this.successMessage = true;
// etc. etc. etc.
});
or maybe instead of using this.successMessage use something like this.saveState$ that would be the a BehaviorSubject object initialized with 'idle' (or some enum thereof) that your saveExclusion() function manages. That way, the beginning of your saveExclusion() function could
set const saveState$ = this.saveState$
assert that saveState$.getValue() === 'in process' or if not, do something about it,
saveState$.next('in process');
and you could change your subscribe line to
this.apiFca.saveNew(valueForm, this.exclusionDelays)
.subscribe(
res => {
if (res !== '200') {
saveState$.next('unexpected result')
} },
err => {
console.log(err);
saveState$.next('error');
},
() => {
if (saveState$.getValue() === 'in process') {
saveState$.next('success');
} }
);
And then you can subscribe to your component's saveState$ as well (though outside of the component you'd want to provide saveState$.asObservable() so values can't be injected by outside code). This affords some elegant event-driven code in your component initialization:
saveState$.pipe(filter(val => val === 'error'))
.subscribe(functionToTellYourUserThereWasAnError);
// if successful, we want other code to know, but immediately change it back to 'idle' even if other code errors
saveState$.pipe(filter(val => val === 'success')
.subscribe(val => saveState$.next('idle'));
// upon success, reset me
saveState$.pipe(filter(val => val === 'success'))
.subscribe(
val => {
this.exclusionDays = [];
// etc. etc.
// setTimeout not needed because set to 'idle' in a different thread.
}
)
Plus, I think your template could reflect and change the UI in response to changes in saveState$ as well, so your save button can be enabled/disabled based on whether or not saveState is 'idle', etc.

Counter not increasing in async map function

I am working with mongodb and nodejs. I have an array of customers I have to create each inside database.
const promises2 = customers.map(async customer => {
if (!customer.customerId) {
const counter = await Counter.findOne({ type: "Customer" });
console.log({counter});
const payload = {
customerId: counter.sequence_value,
};
await Customer.create(payload);
await Counter.findOneAndUpdate({ type: "Customer" }, { $inc: { sequence_value: 1 } });
}
});
await Promise.all([...promises2]);
The issue is counter is not increasing every time. I am getting same counter in all the created customers. What is the issue here?
Issue is something like this but don't have an answer.

The problem is that all the calls overlap. Since the first thing they each do is get the current counter, they all get the same counter, then try to use it. Fundamentally, you don't want to do this:
const counter = await Counter.findOne({ type: "Customer" });
// ...
await Counter.findOneAndUpdate({ type: "Customer" }, { $inc: { sequence_value: 1 } });
...because it creates a race condition: overlapping asynchronous operations can both get the same sequence value and then both issue an update to it.
You want an atomic operation for incrementing and retrieving a new ID. I don't use MongoDB, but I think the findOneAndUpdate operation can do that for you if you add the returnNewDocument option. If so, the minimal change would be to swap over to using that:
const promises2 = customers.map(async customer => {
if (!customer.customerId) {
const counter = await Counter.findOneAndUpdate(
{ type: "Customer" },
{ $inc: { sequence_value: 1 } },
{ returnNewDocument: true }
);
console.log({counter});
const payload = {
customerId: counter.sequence_value,
};
await Customer.create(payload);
}
});
await Promise.all([...promises2]);
...but there's no reason to create an array and then immediately copy it, just use it directly:
await Promise.all(customers.map(async customer => {
if (!customer.customerId) {
const counter = await Counter.findOneAndUpdate(
{ type: "Customer" },
{ $inc: { sequence_value: 1 } },
{ returnNewDocument: true }
);
console.log({counter});
const payload = {
customerId: counter.sequence_value,
};
await Customer.create(payload);
}
}));
The overall operation will fail if anything fails, and only the first failure is reported back to your code (the other operations then continue and succeed or fail as the case may be). If you want to know everything that happened (which is probably useful in this case), you can use allSettled instead of all:
// Gets an array of {status, value/reason} objects
const results = await Promise.allSettled(customers.map(async customer => {
if (!customer.customerId) {
const counter = await Counter.findOneAndUpdate(
{ type: "Customer" },
{ $inc: { sequence_value: 1 } },
{ returnNewDocument: true }
);
console.log({counter});
const payload = {
customerId: counter.sequence_value,
};
await Customer.create(payload);
}
}));
const errors = results.filter(({status}) => status === "rejected").map(({reason}) => reason);
if (errors.length) {
// Handle/report errors here
}
Promise.allSettled is new in ES2021, but easily polyfilled if needed.
If I'm mistaken about the above use of findOneAndUpdate in some way, I'm sure MongoDB gives you a way to get those IDs without a race condition. But in the worst case, you can pre-allocate the IDs instead, something like this:
// Allocate IDs (in series)
const ids = [];
for (const customer of customers) {
if (!customer.customerId) {
const counter = await Counter.findOne({ type: "Customer" });
await Counter.findOneAndUpdate({ type: "Customer" }, { $inc: { sequence_value: 1 } });
ids.push(counter.sequence_value);
}
}
// Create customers (in parallel)
const results = await Promise.allSettled(customers.map(async(customer, index) => {
const customerId = ids[index];
try {
await Customer.create({
customerId
});
} catch (e) {
// Failed, remove the counter, but without allowing any error doing so to
// shadow the error we're already handling
try {
await Counter.someDeleteMethodHere(/*...customerId...*/);
} catch (e2) {
// ...perhaps report `e2` here, but don't shadow `e`
}
throw e;
}
});
// Get just the errors
const errors = results.filter(({status}) => status === "rejected").map(({reason}) => reason);
if (errors.length) {
// Handle/report errors here
}

Your map function is not returning a promise.
Try this :
const promises2 = [];
customers.map((customer) => {
return new Promise(async (resolve) => {
if (!customer.customerId) {
const counter = await Counter.findOne({ type: 'Customer' });
console.log({ counter });
const payload = {
customerId: counter.sequence_value,
};
await Customer.create(payload);
await Counter.findOneAndUpdate({ type: 'Customer' }, { $inc: { sequence_value: 1 } });
}
resolve();
});
});
await Promise.all(promises2);

Issues when testing Epic with TestScheduler

I'm using an rxjs epic as a middleware for an async action in a react-redux app.
I'm trying to simulate an ajax request (through a dependency injection) and test the behavior of this epic based on the response.
This is my epic :
export const loginEpic = (action$, store$, { ajax }) => { // Ajax method is injected
return action$.ofType(LoginActions.LOGIN_PENDING).pipe(
mergeMap(action => {
if (action.mail.length === 0) {
return [ loginFailure(-1) ]; // This action is properly returned while testing
} else {
return ajax({ ... }).pipe(
mergeMap(response => {
if (response.code !== 0) {
console.log(response.code); // This is logged
return [ loginFailure(response.code) ]; // This action is expected
} else {
return [ loginSuccess() ];
}
}),
catchError(() => {
return [ loginFailure(-2) ];
})
);
}
})
);
};
This part test if the mail adress is empty and works just fine (Or at least just as expected):
it("empty mail address", () => {
testScheduler.run(({ hot, expectObservable }) => {
let action$ = new ActionsObservable(
hot("a", {
a: {
type: LoginActions.LOGIN_PENDING,
mail: ""
}
})
);
let output$ = loginEpic(action$, undefined, { ajax: () => ({}) });
expectObservable(output$).toBe("a", {
a: {
type: LoginActions.LOGIN_FAILURE,
code: -1
}
});
});
});
However, I have this second test that fails because the actual value is an empty array (There is no login failed returned):
it("wrong credentials", () => {
testScheduler.run(({ hot, cold, expectObservable }) => {
let action$ = new ActionsObservable(
hot("a", {
a: {
type: LoginActions.LOGIN_PENDING,
mail: "foo#bar.com"
}
})
);
let dependencies = {
ajax: () =>
from(
new Promise(resolve => {
let response = {
code: -3
};
resolve(response);
})
)
};
let output$ = loginEpic(action$, undefined, dependencies);
expectObservable(output$).toBe("a", {
a: {
type: LoginActions.LOGIN_FAILURE,
code: -3
}
});
});
});
Any idea on what I'm doing wrong / why this part returns an empty array (The console.log does actually log the code):
if (response.code !== 0) {
console.log(response.code);
return [ loginFailure(response.code) ];
}
While this part returns a populated array:
if (action.mail.length === 0) {
return [ loginFailure(-1) ];
}

I'm guessing the use of Promise is causing the test to actually be asynchronous. Try changing the stub of ajax to use of(response) instead of from

mongodb node cursor not found with timeout false

I have a nodejs/express server and I'm trying to merge and sort sorted results from multiple mongodb collections in order to create a sorted CSV file. The way I achieve this requires that I keep the mongodb cursors alive (no timeout) until I read/exhaust all data, or until an error occurs, in which case I have to close them manually. It seems to work when there aren't many data points. However, when the mongo queries request data for one year for example, at some point after almost half an hour, I get the following mongo error: Cursor not found: cursor id: 59427962835.
Promises are bluebird promises. Written in Typescript.
import * as _ from 'lodash';
import * as moment from 'moment-timezone';
function findNative(db, collection, spec={}) {
const {query, fields, sort, limit, skip, hint, timeout=true} = spec;
// internal function that gets a connection from the connection pool
// returns promise with connection
return ensureConnection(db)
.then(connection => {
const cursor = connection.collection(collection).find(
query || {},
{fields, sort, limit, skip, hint, timeout});
// For sorted queries we have to limit batchSize
// see https://jira.mongodb.org/browse/SERVER-14228
if (connection.serverConfig.capabilities().maxWireVersion == 0 && sort && !limit) {
cursor.batchSize(0);
}
return cursor;
});
}
function getMongoStream(col, startdate, enddate) {
return findNative('testDb', col, {
query: { t: { $gte: startdate, $lte: enddate }},
sort: { t: 1 },
fields: { i: 0, _id: 0 },
timeout: false
});
}
async function fetchNextCursorData(cursor) {
const hasMore = await cursor.hasNext();
console.log(hasMore, cursor.cursorState.cursorId.toString());
return hasMore ? cursor.next() : Promise.resolve(null);
}
function findEarliestDate(buffer: any[]): [string, number[]] {
let earliestDateMS;
const indices = _(buffer)
.map(x => x && x.t.getTime())
.forEach(t => {
// make sure timestamp is defined
// buffer also contains null values
if(t && (!earliestDateMS || (earliestDateMS && t < earliestDateMS))) {
earliestDateMS = t;
}
})
.reduce((acc, t, i) => {
if(t === earliestDateMS) {
acc.push(i);
}
return acc;
}, []);
return [moment(earliestDateMS).utc().format('YYYY-MM-DD HH:mm:ss.SSS'), indices];
}
function closeAllCursors(cursors: any[]) {
const openCursors = cursors
.filter(c => !c.isClosed());
openCursors.forEach(c => c.close());
}
async function csvData(req, res) {
const collections: string[] = req.swagger.params.collections.value.split(',').sort(),
sources: string[] = req.swagger.params.sources.value.split(',').sort(),
startdate = new Date(Number(req.swagger.params.startdate.value)),
enddate = new Date(Number(req.swagger.params.enddate.value));
const filename = `${moment.utc().format('YYYY-MM-DD_HH:mm')}.csv`;
res.set({
'Content-Type': 'text/csv',
'Content-Disposition': `attachment; filename="${filename}"`
});
res.write('Date UTC,' + sources.join(',') + '\n');
const colPromises = collections.map(col => getMongoStream(col, startdate, enddate));
let cursorsMap: { [rec: string]: any; };
try {
let buffer = [], dateCSVBuffer: any[] = _.fill(Array(sources.length), '');
// fetch first doc from all cursors
const cursors = await Promise.all(colPromises);
cursorsMap = _.zipObject<any>(collections, cursors);
let docs = await Promise.all(cursors.map(fetchNextCursorData));
// initial request made for all collections
let requestedIdx = _.range(0, collections.length);
while(true) {
docs.forEach((doc, i) => {
buffer[requestedIdx[i]] = doc;
});
// null indicates that cursor won't return more data =>
// all cursors are exhausted
if(buffer.every(d => d === null)) {
break;
}
const [date, indices] = findEarliestDate(buffer);
requestedIdx = indices;
indices.forEach(idx => {
// update csv buffer
const {data} = buffer[idx];
Object.keys(data)
.forEach(ch => {
const sourceIndex = sources.indexOf(ch);
if(sourceIndex > -1) {
dateCSVBuffer[sourceIndex] = data[ch];
}
});
// remove doc from buffer
buffer[idx] = null;
});
// send csv string
dateCSVBuffer.unshift(date);
res.write(dateCSVBuffer.join(',') + '\n');
// empty buffer
dateCSVBuffer = dateCSVBuffer.map(() => '');
// request new entry from cursors
const nextDocPromises = indices
.map(idx => cursorsMap[collections[idx]])
.map(fetchNextCursorData);
docs = await Promise.all(nextDocPromises);
}
// end data stream
res.end();
} catch(err) {
// make sure to close all cursors
// will catch all nested promise errors
closeAllCursors(_.values(cursorsMap));
console.error(err);
res.status(500).json(err);
}
}
Mongodb connection created with following options:
{
auto_reconnect: true,
poolSize: 30,
connectTimeoutMS: 90000
}
Could the problem be that I keep the cursor references in the map and thus they are not updated? And when I do a cursor.hasNext() cursor is already dead? I also tried checking whether cursor.isClosed() but it always returns false.
Mongodb driver is "mongodb": "2.2.15" and the queries are tested against a v3.0 database.
EDIT: I did a small count test to see how many docs have been processed at the time when the program crashes.
The 3 cursors (test case requested only data from 3 collections) have the following counts and ids:
3097531 '59427962835'
31190333 '53750510295'
32007475 '101213786015'
and the last document cursor with id '59427962835' processed was number 4101. So not even close to finishing

Turns out that adding the timeout to the find query doesn't work. I had to use the noCursorTimeout flag like so:
const cursor = connection.collection(collection)
.find(query || {}, {fields, sort, limit, skip, hint})
.addCursorFlag('noCursorTimeout', !timeout);