I'm currently making my first network call in a paginated series of calls on the server side. Prior to doing this I was making all of the calls client side and stored the last document in the collection call as an offset.
The offset was then sent as a .startAfter call for the same collection. The offset document looked like this:
exists: (...)
id: (...)
metadata: (...)
ref: (...)
_document: Document {key: DocumentKey, version: SnapshotVersion, data: ObjectValue, proto: {…}, hasLocalMutations: false, …}
_firestore: Firestore {_queue: AsyncQueue, INTERNAL: {…}, _config: FirestoreConfig, _databaseId: DatabaseId, _dataConverter: UserDataConverter, …}
_fromCache: false
_hasPendingWrites: false
_key: DocumentKey {path: ResourcePath}
__proto__: DocumentSnapshot
When I make the call on the server I'm currently able to see the document looks the same, but when I send it over the network it seems to be stripped or at least looks very different by the time it's sent over JSON and parsed back.
Sent like this:
res.json(offset)
then parsed like this:
feedData = await dataWithOffset.json();
After the parsing it looks like this:
{ _ref:
{ _firestore:
{ _settings: [Object],
_settingsFrozen: true,
_serializer: [Object],
_projectId: '***-prod',
_lastSuccessfulRequest: 1566308918946,
_preferTransactions: false,
_clientPool: [Object] },
_path:
{ segments: [Array],
projectId: '***-prod',
databaseId: '(default)' } },
_fieldsProto:
{ lastModified: { timestampValue: [Object], valueType: 'timestampValue' },
...,
_serializer: { timestampsInSnapshots: true },
_readTime: { _seconds: 1566308918, _nanoseconds: 909566000 },
_createTime: { _seconds: 1565994031, _nanoseconds: 304997000 },
_updateTime: { _seconds: 1565994031, _nanoseconds: 304997000 } }
Any idea why it is losing its shape and what I can do to fix it so it returns to working as a proper offset? Should I not be converting to JSON and back, as that may strip some important things?
Solution (tested)
So I went ahead and made a sandbox to test the implementation, and like I mentioned in the comments .startAfter expects a DocumentSnapshot not a DocumentReference
Given that I created a collection named cars with 4 records like this:
[
{
"name": "car 1",
"hp": 5
},
{
"name": "car 2",
"hp": 10
},
{
"name": "car 2.5",
"hp": 10
},
{
"name": "car 3",
"hp": 15
}
]
Then I configured an express endpoint:
router.get('/cars/:lastDocId', (req, res) => {
const query = db.collection('cars')
.orderBy('hp')
.limit(2);
const handleQueryRes = (snap) => {
return res.send({
docs: snap.docs.map(doc => doc.data()),
last: snap.docs.length > 0 ? snap.docs[snap.docs.length - 1].id : null
});
}
if(req.params.lastDocId != -1) {
// this makes an "auxiliar" read from the DB to transform the given ID in a DocumentSnapshot needed for startAfter
return db.collection('cars').doc(req.params.lastDocId).get().then(snap => {
return query.startAfter(snap).get();
}).then(handleQueryRes).catch(console.log);
} else {
return query.get().then(handleQueryRes).catch(console.log);
}
});
The path parameter lastDocId is either -1 for first page or the last document fetched in the previous page
When I make a GET request to /cars/-1 it returns this:
{
"docs": [
{
"hp": 5,
"name": "car 1"
},
{
"name": "car 2",
"hp": 10
}
],
"last": "9sRdLOvV8REwEpHDjEw7"
}
Now if I grab the last prop in the response and use it in my next GET like so /cars/9sRdLOvV8REwEpHDjEw7 I get my next 2 cars:
{
"docs": [
{
"name": "car 2.5",
"hp": 10
},
{
"name": "car 3",
"hp": 15
}
],
"last": "tZPF7Wav7jZuchoCp6zM"
}
Even though I only have 4 records and the second request should be the last the function does not know the length of the collection so it returns another last ID
If you make the last request /cars/tZPF7Wav7jZuchoCp6zM it returns:
{
"docs": [],
"last": null
}
Thus giving indication that is the last page.
I'm not too fond of having to read the document again to transform it into a DocumentSnapshot but I guess that's a firebase limitation
Hope it helped
Answer with steps
When converting to JSON and back you lose the DocumentReference prototype
The solution to pass that type of data through network requests is:
Client sends a plain-text ref to your endpoint ( or doesn't for first page)
Have the server side code re-create the DocumentReference with the received ref
Use the re-created DocumentReference to paginate query
Return from server the requested page's documents alongside with a plain-text refto get the next page
Initial answer (obsolete) but for future reference
You need to send offset.data() instead of sending offset alltogether
One is the document's data (probably what you're after) the other is a DocumentReference which has methods that cannot be represented in JSON.
If you need anything else from the document (like it's id or reference) you're better off building a new object:
res.json({
data: offset.data(),
ref: offset.ref,
id: offset.id
});
Related
This question already has answers here:
Safely turning a JSON string into an object
(28 answers)
Closed 2 years ago.
Good day Developers in the house. I need help. I have a json string that i have gotten from API in Angular and console log to see it. I want to get a particular data from the json file and store in a variable because i need to query the data again. Below is how my Json data look like.
{
"done": true,
"record_id": "5fc0a7ac88d2f8534f8e59d8",
"customer_id": "5fa1541f6bd09b290f736608",
"balance": {
"clientId": "qh8RKp9BGhmKCn8ZAAED",
"status": true,
"balance_callback": "https://webhook.site/92b09e29-f08e-4472-b7e8-5875155360d67",
"data": {
"formatted": [
{
"available_balance": 31500,
"ledger_balance": 32000,
"ref": "saving-account",
"status": "active",
"account": "5fa9e536f6b7bb837cb22byu",
"connected": true
},
{
"available_balance": 11200,
"ledger_balance": 11200,
"ref": "current-account",
"status": "active",
"account": "5fa9e535f6b7bb837cb22buy",
"connected": false
},
{
"available_balance": 2000,
"ledger_balance": 2000,
"ref": "current-account",
"status": "active",
"account": "5fa9e536f6b7bb837cb22bty",
"connected": false
}
]
}
},
"Post": {
"callback_url": "https://webhook.site/92b09e29-f08e-4472-b7e8-5875155360d67"
},
"guarantors": [],
"redirect_url": "",
"launchAgain": true,
"hideExit": "",
"options": {},
"directors": null,
"auth": {
"clientId": "qh8RKp9BGhmKCn8ZAythj",
"status": true,
}
I want to get the callback-url under Post in the json file above.. Please any idea on how to do that with any javascript Method.
Let's say you call the date like that
fetch('urlOfTheApi')
.then( response => resonse.json()
.then( json => {
console.log(json);
});
So the solution is
fetch('urlOfTheApi')
.then( response => resonse.json()
.then( json => {
let callback_url = json.POST.callback;
return fetch(callback_url, {method:'POST'}).
})
.then( () => console.log('ok')
.catch( console.error );
Based on your JSON you should create an interface to define the object of the JSON and the particular part of the JSON that you need if you just need that property you could create the following interface
interface Response {
Post: {
callback_url: string;
}
}
In the service where you are requesting the endpoint from your API, you set that this method returns a Response type of data.
And inside of the subscription on your method you could access the property of the object of the JSON.
I would like to get a modified response object. For example I dont know how to get the user object without the roles.
The default response is:
{
"id": 6,
"username": "username",
"email": "user#email.com",
"provider": "local",
"confirmed": true,
"blocked": false,
"role": {
"id": 2,
"name": "Authenticated",
"description": "Default role given to authenticated user.",
"type": "authenticated"
}
}
Now I want to get the same response without the role attribute.
{
"id": 6,
"username": "username",
"email": "user#email.com",
"provider": "local",
"confirmed": true,
"blocked": false
}
Currently you cannot do this in the Rest API unless you change the UserController provided by permissions plugin, which is not recommended.
What you can do then is to use the GraphQL plugin provided by Strapi, so you can query only the fields you need on client side.
The docs about how to use GraphQL plugin are here.
For anyone still struggling with this problem:
The latest versions of strapi do support custom queries, you can pass an array containing all the names of relations you wish to populate (only relations!).
If you don't want to populate any relationships, you can keep it empty, your controller would then look something like this:
module.exports = {
UserWithoutRoles: ctx => {
return strapi.query('user').findOne({ id: ctx.params.id }, ['']);
}
}
If you do wish to populate it, it would be like this:
module.exports = {
UserWithoutRoles: ctx => {
return strapi.query('user').findOne({ id: ctx.params.id }, ['role']);
}
}
Also see:
[https://strapi.io/documentation/3.0.0-beta.x/concepts/queries.html#api-reference][1]
Does anyone know if it's possible to populate a list of IDs for another model using waterline associations? I was trying to get the many-to-many association working but I don't think it applies here since one side of the relationship doesn't know about the other. Meaning, a user can be a part of many groups but groups don't know which users belong to them. For example, I'm currently working with a model with data in mongodb that looks like:
// Group
{
_id: group01,
var: 'somedata',
},
{
_id: group02,
var: 'somedata',
},
{
_id: group03,
var: 'somedata',
}
// User
{
_id: 1234,
name: 'Jim',
groups: ['group01', 'group03']
}
And I'm trying to figure out if it's possible to setup the models with an association in such a way that the following is returned when querying the user:
// Req: /api/users/1234
// Desired result
{
id: 1234,
name: 'Jim',
groups: [
{
_id: group01,
var: 'somedata',
},
{
_id: group03,
var: 'somedata',
}
]
}
Yes, associations are supported in sails 0.10.x onwards. Here is how you can setup the models
Here is how your user model will look like:
// User.js
module.exports = {
tableName: "users",
attributes: {
name: {
type: "string",
required: true
},
groups: {
collection: "group",
via: "id"
}
}
};
Here is how your group model will look like:
// Group.js
module.exports = {
tableName: "groups",
attributes: {
name: {
type: "string",
required: "true"
}
}
};
Setting up models like this will create three tables in your DB:
users,
groups and
group_id__user_group
The last table is created by waterline to save the associations. Now go on and create groups. Once groups are created, go ahead and create user.
Here is a sample POST request for creation a new user
{
"name": "user1",
"groups": ["547d84f691bff6663ad08147", "547d850c91bff6663ad08148"]
}
This will insert data into the group_id__user_group in the following manner
{
"_id" : ObjectId("547d854591bff6663ad0814a"),
"group_id" : ObjectId("547d84f691bff6663ad08147"),
"user_groups" : ObjectId("547d854591bff6663ad08149")
}
/* 1 */
{
"_id" : ObjectId("547d854591bff6663ad0814b"),
"group_id" : ObjectId("547d850c91bff6663ad08148"),
"user_groups" : ObjectId("547d854591bff6663ad08149")
}
The column user_groups is the user id. And group_id is the group id. Now if you fetch the user using GET request, your response will look like this:
{
"groups": [
{
"name": "group1",
"createdAt": "2014-12-02T09:23:02.510Z",
"updatedAt": "2014-12-02T09:23:02.510Z",
"id": "547d84f691bff6663ad08147"
},
{
"name": "group2",
"createdAt": "2014-12-02T09:23:24.851Z",
"updatedAt": "2014-12-02T09:23:24.851Z",
"id": "547d850c91bff6663ad08148"
}
],
"name": "user1",
"createdAt": "2014-12-02T09:24:21.182Z",
"updatedAt": "2014-12-02T09:24:21.188Z",
"id": "547d854591bff6663ad08149"
}
Please note that groups are not embedded in the user collection. Waterline does the fetch from groups, users and group_id__user_group to show this result to you.
Also, if you want to do this in your controller, you will need to execute like this
User.findOne({'id': "547d854591bff6663ad08149"})
.populate('groups')
.exec(function (err, user){
// handle error and results in this callback
});
Without populate('groups'), you won't get the groups array. Hope this serves your purpose
I have this schema defined:
var bookCollection = new mongoose.Schema({
book:[{
bookTitle: String,
bookOrder: Number,
bookChapters: [{
chapterTitle: String,
chapterIntro: String,
chapterOrder: Number,
chapterArticles: [{
articleTitle: String,
articleIntro: String,
articleOrder: Number,
articleHeadings: [{
headingTitle: String,
headingOrder: Number
}]
}]
}]
}]
});
var bookModel = mongoose.model('bookModel', bookCollection);
I then saved 1 document to mongoDB, this is the JSON object when checking using db.bookmodels.find()
{
"_id": ObjectId("530cc92710f774355434b394"),
"book": [
{
"bookTitle": "Javascript",
"bookOrder": 300,
"_id": ObjectId("530cc92710f774355434b395"),
"bookChapters": [
{
"chapterTitle": "Functions",
"chapterIntro": "All about javascript functions",
"chapterOrder": 500,
"_id": ObjectId("530cc92710f774355434b396"),
"chapterArticles": [
{
"articleTitle": "A visual illustration of the JS function",
"articleIntro": "Something to see here, check it out",
"articleOrder": 500,
"_id": ObjectId("530cc92710f774355434b397"),
"articleHeadings": [
{
"headingTitle": "Parts of a function",
"headingOrder": 500,
"_id": ObjectId("530cc92710f774355434b398")
}
]
}
]
}
]
}
],
"__v": 0
}
If i want to change headingOrder to 100 instead of 500, how would i update the database using mongoose.js? I've been trying several things and i can't seem to get my head around it.
Everywhere you see examples with simple schema's but never with complex schema's like this one.
thx.
You can always load the document in memory, made modifications, i.e. book[0].bookChapters[0].chapterArticles[0].articleHeadings[0].headingOrder = 100 then save() it.
The example at the link above does exactly that, document.find() followed by save().
I'm using REST adapter, when I call App.Message.find() Ember.js makes call to the /messages to retrieve all messages and expect to see JSON structure like this:
{
"messages": [] // array contains objects
}
However API I have to work with response always with:
{
"data": [] // array contains objects
}
I only found the way1 to change namespace or URL for the API. How to tell REST adapter to look for data instead of messages property?
If this is not possible how to solve this problem? CTO said we can adapt API to use with REST adapter as we want, but from some reason we can't change this data property which will be on each response.
Assuming you are ok with writing your own adapter to deal with the difference, in the success callback you can simply modify the incoming name from "data" to your specific entity -in the case above "messages"
I do something like this to give you and idea of what if possible in a custom adapter
In the link below I highlighted the return line from my findMany
The json coming back from my REST api looks like
[
{
"id": 1,
"score": 2,
"feedback": "abc",
"session": 1
},
{
"id": 2,
"score": 4,
"feedback": "def",
"session": 1
}
]
I need to transform this before ember-data gets it to look like this
{
"sessions": [
{
"id": 1,
"score": 2,
"feedback": "abc",
"session": 1
},
{
"id": 2,
"score": 4,
"feedback": "def",
"session": 1
}
]
}
https://github.com/toranb/ember-data-django-rest-adapter/blob/master/packages/ember-data-django-rest-adapter/lib/adapter.js#L56-57
findMany: function(store, type, ids, parent) {
var json = {}
, adapter = this
, root = this.rootForType(type)
, plural = this.pluralize(root)
, ids = this.serializeIds(ids)
, url = this.buildFindManyUrlWithParent(store, type, ids, parent);
return this.ajax(url, "GET", {
data: {ids: ids}
}).then(function(pre_json) {
json[plural] = pre_json; //change the JSON before ember-data gets it
adapter.didFindMany(store, type, json);
}).then(null, rejectionHandler);
},