In the following exported function which is from a Nextjs app as an API page, the domainnames array is returning nothing in the 200 response.
However, if I do not use the GetDomainStatus() function and just push items from response.data.results into domainnames, then the JSON response is filled.
export default function GetSuggestions(req, res){
const keyword = req.query.q;
const tlds = '.com,.net,.io,.org,.co,.xyz,.app,.us,.blog,.shop,.land,.video,.review,.host,.dev';
let queryPath = `${suggestionsURL}?include-registered=false&tlds=${tlds}&include-suggestion-type=true&sensitive-content-filter=true&use-numbers=true&max-length=20&lang=eng&max-results=100&name=${keyword}&use-idns=false`
let domainnames = [];
axios.get(queryPath).then(response => {
response.data.results.forEach(item => {
GetDomainStatus(item.name).then(a => {
domainnames.push({
name: item.name,
avail: a
})
})
})
res.status(200).json(domainnames);
});
}
is this a scope issue where I actually cannot access domainnames array from within the promise?
This solution worked. Not sure if its the best, but uses the promise.all solution.
const domainnames = [];
const promises = [];
axios.get(queryPath).then(response => {
response.data.results.forEach(item => {
let newPromise = GetDomainStatus(item.name).then(a => {
domainnames.push({
name: item.name,
avail: a
})
});
promises.push(newPromise);
})
Promise.all(promises).then(r => res.status(200).json(domainnames));
});
In my Vuex store, I have an action that takes a list of uids of followed users from the current user's Firestore UserDataCollection document, iterates through them, and produces the data for each document to be shown on the UI. It works fine using .get(), but I'm trying to convert it to .onSnapshot() so that I can get real-time updates.
I have been totally unsuccessful trying to use .onSnapshot(), as I can't find any references online or in the Firebase docs on how to implement this after mapping through the array of uids.
I tried removing the promises, since onSnapshot doesn't seem to work with promises, and replaced the .get() with .onSnapshot(), but that didn't work.
Does anyone know the correct way to implement the Firestore .onSnapshot() listener given the code below?
getCircle({state, commit}) {
const circle = state.userProfile.circle
let promises = circle.map(u => userDataCollection.doc(u).get())
return Promise.all(promises)
.then(querySnapShot => {
let circleData = []
if (querySnapShot.empty) {
console.log("empty")
} else {
querySnapShot.forEach(doc => {
let item = doc.data()
circleData.push(item)
}
)
}
commit('setUserCircle', circleData)
})
},
Edit based on response
I added .onSnapshot within the forEach as shown in the code below. In vue devtools it's showing the correct number of data entries in my Vuex store, however they are all undefined.
getCircle({state, commit}) {
const circle = state.userProfile.circle
let promises = circle.map(u => userDataCollection.doc(u).get())
return Promise.all(promises)
.then(querySnapShot => {
let circleData = []
if (querySnapShot.empty) {
console.log("empty")
} else {
querySnapShot.forEach(x => {
let itemId = x.data().uid
userDataCollection.doc(itemId)
.onSnapshot((doc) => {
let item = doc.data()
console.log(doc.data())
circleData.push(item)
})
}
)
}
commit('setUserCircle', circleData)
})
},
In presented code commit is run only once before onSnapshot callback. I tried to create example showing the mechanism (I am assuming that circle array is assigned properly and I have 3 items hardcode'ed in mine):
let promises = circle.map(u => userDataCollection.doc(u).get())
Promise.all(promises)
.then( querySnapShot => {
var circleData = []
querySnapShot.forEach( x => {
let itemId = x.data().uid
userDataCollection.doc(itemId).onSnapshot(doc => {
let item = doc.data()
circleData.push(item)
console.log("inside snapshot: ",circleData.length)
})
})
console.log("outside foreach: ",circleData.length)
})
If you run this code you should see in console something like this (I have run this in node):
outside foreach: 0
inside snapshot: 1
inside snapshot: 2
inside snapshot: 3
And if you do any change in Firestore you will get another console log inside snapshot: 4.
I do not have full understanding of your application logic, however I think that commit statement should be inside onSnapshot listener. But of course, with information I have, this is only guess.
Thank you everyone for the help. The solution that worked was to remove the .map and Promise and used an 'in' query. Code shown below:
getCircle({state, commit}) {
const circle = state.userProfile.circle
userDataCollection.where('__name__', 'in', circle).onSnapshot(querySnapShot => {
var circleData = []
querySnapShot.forEach(doc => {
let item = doc.data()
circleData.push(item)
commit('setUserCircle', circleData)
})
})
},
I'm trying to find the best way to go about this service call where I retain all the data in a single object. The call returns an object that has a property of next_page_url. If there is a next_page_url the function should keep chaining. Since I don't know what the url is until the next call resolves I need to call these in order and resolve them in order. I'm also collecting data from each call. I haven't been able to figure out what the structure should be
what I have so far
getDataFromAllPages = (url) => {
waniKaniAxios.get(url).then(object => {
if(object.data.pages.next_url){
return this.getDataFromAllPages(object.data.pages.next_url.replace(waniKaniAxios.defaults.baseURL, ''));
}
});
}
getWanikaniData = () => {
this.getDataFromAllPages('/subjects?types=vocabulary').then(result => {
console.log(result);
});
}
Abstract away the wanikaniaxios.get in another function to make recursion clearer.
Here's my badly formatted code (don't know how SF editor works) , feel to ask any questions if you have any. Happy coding.
getWanikaniData = () => {
this.getDataFromAllPages("/subjects?types=vocabulary")
.then((result) => {
console.log(result);
})
.catch((err) => {
console.log(err); // always put a .catch when you're using prmomises.
});
};
getDataFromAllPages = async (url) => {
// using async await;
try {
let arr = []; // i am assuming you'll improve upon what data structure you might want to return. Linked list seems best to me.
const object = await waniKaniAxios.get(url);
if (object.data.pages.next_url) {
const laterData = await this.getDataFromAllPages(
object.data.pages.next_url.replace(waniKaniAxios.defaults.baseURL, "")
);
arr = [...arr, ...laterData];
} else {
arr = [...arr, object];
}
Promise.resolve(arr);
} catch (err) {
Promise.reject(new Error(`Oops new wanikani error, ${err}`));
}
};
FINAL UPDATE
Using part of the answer below I managed to get it working. Had to partially give up on the recursion aspect because I didn't how to make the promise resolve into data
Here's the final solution that I came up with
getDataFromAllPages = async (url) => {
let results = {};
try {
//getting intial data
const initialData = await waniKaniAxios.get(url);
//using the intial data and mapping out the levels then saving it into results object
results = this.mapOutLevels(initialData.data, results);
//get the next page url
let nextPageUrl = initialData.data.pages.next_url;
//while there is a next page url keep calling the service and adding it to the results object
while (nextPageUrl) {
const laterData = await waniKaniAxios.get(nextPageUrl);
nextPageUrl = laterData.data.pages.next_url;
results = this.mapOutLevels(laterData.data, results);
}
} catch (err) {
Promise.reject(new Error(`Opps new wanikani error, ${err}`));
}
return Promise.resolve(results);
};
getWanikaniData = () => {
this.getDataFromAllPages("/subjects?types=vocabulary")
.then((result) => {
console.log(result);
})
.catch((err) => {
console.log(err);
});
};
I know there are similar questions, but I can't find the answer.
First, please tell me if I'm doing something really wrong
I need to populate my state with data from an API call. This was working fine with code above:
export const GetPlanets = async () => {
const planets = await axios.get(`${BASE_URL}`).catch((e) => {
console.error(e);
})
return planets.data.results
}
But then I needed to make a second call to several links from one json response filed, and I managed to make it work (don't know if it is the correct approach, though)
const GetPlanets = async () => {
let planetas = {}
await axios.get(`${PLANETS_URL}`)
.then((p) => {
planetas = p.data.results
return axios.get(`${FILMS_URL}`)
}).then((f) => {
planetas.films.forEach((v, i) => {
planetas[i].film = f
})
})
})
.catch((e) => {
console.error(e);
})
return planetas
}
This is my component file, where I try to get the object, like I was doing before
useEffect(() => {
const fetchPlanetas = async () => { // ME TRYING...
const planetas = await GetPlanets()
setPlanetas(planetas)
setToShow(planetas[0])
};
fetchPlanetas()
}, [])
But all I get is undefined
You're getting an array of undefined because .map() needs a return value. In both your .map() callbacks, you are not returning anything.
const results = [1, 2, 3, 4]
const results2 = results.map(elem => {
elem = elem + 1
})
console.log(results2)
But, even if you did return something in your .map() callback, GetFilms(f) is asynchronous, so you would not get the results of GetFilms() mapped into the array as you would expect.
You have a couple of options:
If you have access to the API, send the films data along with the rest of the data when you do your first request.
Use async/await and Promise.all() to get responses.
Hi I'm starting with javascript and react-native and I'm trying to figure out this problem for hours now. Can someone explain me how to get all the documents from firestore collection ?
I have been trying this:
async getMarkers() {
const events = await firebase.firestore().collection('events').get()
.then(querySnapshot => {
querySnapshot.docs.map(doc => {
console.log('LOG 1', doc.data());
return doc.data();
});
});
console.log('LOG 2', events);
return events;
}
Log 1 prints all the objects(one by one) but log 2 is undefined, why ?
The example in the other answer is unnecessarily complex. This would be more straightforward, if all you want to do is return the raw data objects for each document in a query or collection:
async getMarker() {
const snapshot = await firebase.firestore().collection('events').get()
return snapshot.docs.map(doc => doc.data());
}
if you want include Id
async getMarkers() {
const events = await firebase.firestore().collection('events')
events.get().then((querySnapshot) => {
const tempDoc = querySnapshot.docs.map((doc) => {
return { id: doc.id, ...doc.data() }
})
console.log(tempDoc)
})
}
Same way with array
async getMarkers() {
const events = await firebase.firestore().collection('events')
events.get().then((querySnapshot) => {
const tempDoc = []
querySnapshot.forEach((doc) => {
tempDoc.push({ id: doc.id, ...doc.data() })
})
console.log(tempDoc)
})
}
I made it work this way:
async getMarkers() {
const markers = [];
await firebase.firestore().collection('events').get()
.then(querySnapshot => {
querySnapshot.docs.forEach(doc => {
markers.push(doc.data());
});
});
return markers;
}
if you need to include the key of the document in the response, another alternative is:
async getMarker() {
const snapshot = await firebase.firestore().collection('events').get()
const documents = [];
snapshot.forEach(doc => {
const document = { [doc.id]: doc.data() };
documents.push(document);
}
return documents;
}
The docs state:
import { collection, getDocs } from "firebase/firestore";
const querySnapshot = await getDocs(collection(db, "cities"));
querySnapshot.forEach((doc) => {
// doc.data() is never undefined for query doc snapshots
console.log(doc.id, " => ", doc.data());
});
However I am using the following (excuse the TypeScript):
import { collection, Firestore, getDocs, Query, QueryDocumentSnapshot, QuerySnapshot } from 'firebase/firestore'
const q: Query<any> = collection(db, 'videos')
const querySnapshot: QuerySnapshot<IVideoProcessed> = await getDocs(q)
const docs: QueryDocumentSnapshot<IVideoProcessed>[] = querySnapshot.docs
const videos: IVideoProcessed[] = docs.map((doc: QueryDocumentSnapshot<IVideoProcessed>) => doc.data())
where db has the type Firestore
You could get the whole collection as an object, rather than array like this:
async function getMarker() {
const snapshot = await firebase.firestore().collection('events').get()
const collection = {};
snapshot.forEach(doc => {
collection[doc.id] = doc.data();
});
return collection;
}
That would give you a better representation of what's in firestore. Nothing wrong with an array, just another option.
Two years late but I just began reading the Firestore documentation recently cover to cover for fun and found withConverter which I saw wasn't posted in any of the above answers. Thus:
If you want to include ids and also use withConverter (Firestore's version of ORMs, like ActiveRecord for Ruby on Rails, Entity Framework for .NET, etc), then this might be useful for you:
Somewhere in your project, you probably have your Event model properly defined. For example, something like:
Your model (in TypeScript):
./models/Event.js
export class Event {
constructor (
public id: string,
public title: string,
public datetime: Date
)
}
export const eventConverter = {
toFirestore: function (event: Event) {
return {
// id: event.id, // Note! Not in ".data()" of the model!
title: event.title,
datetime: event.datetime
}
},
fromFirestore: function (snapshot: any, options: any) {
const data = snapshot.data(options)
const id = snapshot.id
return new Event(id, data.title, data.datetime)
}
}
And then your client-side TypeScript code:
import { eventConverter } from './models/Event.js'
...
async function loadEvents () {
const qs = await firebase.firestore().collection('events')
.orderBy('datetime').limit(3) // Remember to limit return sizes!
.withConverter(eventConverter).get()
const events = qs.docs.map((doc: any) => doc.data())
...
}
Two interesting quirks of Firestore to notice (or at least, I thought were interesting):
Your event.id is actually stored "one-level-up" in snapshot.id and not snapshot.data().
If you're using TypeScript, the TS linter (or whatever it's called) sadly isn't smart enough to understand:
const events = qs.docs.map((doc: Event) => doc.data())
even though right above it you explicitly stated:
.withConverter(eventConverter)
Which is why it needs to be doc: any.
(But! You will actually get Array<Event> back! (Not Array<Map> back.) That's the entire point of withConverter... That way if you have any object methods (not shown here in this example), you can immediately use them.)
It makes sense to me but I guess I've gotten so greedy/spoiled that I just kinda expect my VS Code, ESLint, and the TS Watcher to literally do everything for me. 😇 Oh well.
Formal docs (about withConverter and more) here: https://firebase.google.com/docs/firestore/query-data/get-data#custom_objects
I prefer to hide all code complexity in my services... so, I generally use something like this:
In my events.service.ts
async getEvents() {
const snapchot = await this.db.collection('events').ref.get();
return new Promise <Event[]> (resolve => {
const v = snapchot.docs.map(x => {
const obj = x.data();
obj.id = x.id;
return obj as Event;
});
resolve(v);
});
}
In my sth.page.ts
myList: Event[];
construct(private service: EventsService){}
async ngOnInit() {
this.myList = await this.service.getEvents();
}
Enjoy :)
Here's a simple version of the top answer, but going into an object with the document ids:
async getMarker() {
const snapshot = await firebase.firestore().collection('events').get()
return snapshot.docs.reduce(function (acc, doc, i) {
acc[doc.id] = doc.data();
return acc;
}, {});
}
General example to get products from Cloud Firestore:
Future<void> getAllProducts() async {
CollectionReference productsRef =
FirebaseFirestore.instance.collection('products');
final snapshot = await productsRef.get();
List<Map<String, dynamic>> map =
snapshot.docs.map((doc) => doc.data() as Map<String, dynamic>).toList();
}
In version 9 sdk of firebase you can get all the documents from a collection using following query:
const querySnapshot = await getDocs(collection(db, "cities"));
querySnapshot.docs.forEach((doc) => {
console.log(doc.id, " => ", doc.data());
});
See Get multiple documents from a collection
I understand your query, This is because how Javascript handles promises and variables. So basically events variable is hoisted with the value undefined and printed on the LOG 2 console log, while the Event Loop responsible for the promise call resulted in an array of objects as the value of the events variable and then the console log (LOG 1) was printed with the resolved promise response
All answers are true, but when you have heavy data you will face memory and bandwidth problems, so you have to write a [cursor] function to read data part by part.
also, you may face to Bandwidth Exhausted error, please have a look at this solution I have implemented on a gist
https://gist.github.com/navidshad/973e9c594a63838d1ebb8f2c2495cf87
Otherwise, you can use this cursor I written to read a collection doc by doc:
async function runCursor({
collection,
orderBy,
limit = 1000,
onDoc,
onDone,
}) {
let lastDoc;
let allowGoAhead = true;
const getDocs = () => {
let query = admin.firestore().collection(collection).orderBy(orderBy).limit(limit)
// Start from last part
if (lastDoc) query = query.startAfter(lastDoc)
return query.get().then(sp => {
if (sp.docs.length > 0) {
for (let i = 0; i < sp.docs.length; i++) {
const doc = sp.docs[i];
if (onDoc) onDoc(doc);
}
// define end of this part
lastDoc = sp.docs[sp.docs.length - 1]
// continue the cursor
allowGoAhead = true
} else {
// stop cursor if there is not more docs
allowGoAhead = false;
}
}).catch(error => {
console.log(error);
})
}
// Read part by part
while (allowGoAhead) {
await getDocs();
}
onDone();
}