I have a versioning schema that versions object entries in a object_version table
const Object = sequelize.define('object', {})
const ObjectVersion = sequelize.define('object_version', {
title: {
allowNull: false,
type: DataTypes.STRING
}
})
Object.hasMany(ObjectVersion, { as: 'versions' })
Using express, i have a put route where an existing object entry can be updated, in this simple example by setting a new title, e.g. by sending a PUT request with a body of {"title":"new title"}
app.put('object/:id', (req, res) => {
const id = parseInt(req.params.id, 10)
Object.findOne({
where: { id },
include: [ Object.associations.versions ]
}).then(object => {
const newVersion = ObjectVersion.build(req.body, { objectId: id })
Object.addVersion(newVersion).then(object => {
// shouldn't this instance have all versions, including the new one?
// do i need to `findOne` again to get them?
console.log(object.toJSON().versions)
res.send(object)
})
}
})
})
When using the addVersion method that is created on my Object model by associating it to the ObjectVersion model via hasMany, the returned promise is resolved with an object instance.
The problem is that i want to return the objects JSON in the response, but the object instance the promise is resolved with does not contain the version that was just added.
To me, this looks like an oversight. It should not be necessary to perform another query. When the promise for the addXyz operation is resolved, the new association is already saved in the database so it should be possible to resolve the promise with an updated version of the object.
Maybe i'm going at this the wrong way or missed an easier way to perform an update operation that adds new associations. Any help is welcome!
After an interesting talk on Slack i realised that the addAssociation, addAssociations and setAssociations methods on model instances connect existing rows in the db with the target.
In my example, the addVersion method connects an existing object_version row with the current version instance and can not be used to insert a new row into the database.
I've opened a PR to sequelize trying to clarify this in the docs as i skipped right over this. An example that works might look like this:
app.put('object/:id', (req, res) => {
const id = parseInt(req.params.id, 10)
Object.findOne({
where: { id },
include: [ Object.associations.versions ]
}).then(object => {
object.createVersion(req.body).then(version => {
object.versions.push(version)
res.send(object)
})
})
})
Related
question is possibly a duplicate but I haven't found anything that provides an appropriate answer to my issue.
I have an ExpressJS server which is used to provide API requests to retrieve data from a MongoDB database. I am using mongoosejs for the MongoDB connection to query/save data.
I am building a route that will allow me to find all data that matches some user input but I am having trouble when doing the query. I have spent a long while looking online for someone with a similar issue but coming up blank.
I will leave example of the code I have at the minute below.
code for route
// -- return matched data (GET)
router.get('/match', async (req, res) => {
const style_data = req.query.style; // grab url param for style scores ** this comes in as a string **
const character_data = req.query.character; // grab url param for character scores ** this comes in as a string **
// run matcher systems
const style_matches = style_match(style_data);
res.send({
response: 200,
data: style_matches
}); // return data
});
code for the query
// ---(Build the finder)
const fetch_matches_using = async function(body, richness, smoke, sweetness) {
return await WhiskyModel.find({
'attributes.body': body,
'attributes.richness': richness,
'attributes.smoke': smoke,
'attributes.sweetness': sweetness
});
}
// ---(Start match function)---
const style_match = async function (scores_as_string) {
// ---(extract data)---
const body = scores_as_string[0];
const richness = scores_as_string[1];
const smoke = scores_as_string[2];
const sweetness = scores_as_string[3];
const matched = [];
// ---(initialise variables)---
let match_count = matched.length;
let first_run; // -> exact matches
let second_run; // -> +- 1
let third_run; // -> +- 2
let fourth_run; // -> +- 3
// ---(begin db find loop)---
first_run = fetch_matches_using(body, richness, smoke, sweetness).then((result) => {return result});
matched.push(first_run);
// ---(return final data)---
return matched
}
example of db object
{
_id: mongoid,
meta-data: {
pagemd:{some data},
name: whiskyname
age: whiskyage,
price: price
},
attributes: {
body: "3",
richness: "3",
smoke: "0",
sweetness: "3",
some other data ...
}
}
When I hit the route in postman the JSON data looks like:
{
response: 200,
data: {}
}
and when I console.log() out matched from within the style match function after I have pushed the it prints [ Promise(pending) ] which I don't understand.
if I console.log() the result from within the .then() I get an empty array.
I have tried using the populate() method after running the find which does technically work, but instead of only returning data that matches it returns every entry in the collection so I think I am doing something wrong there, but I also don't see why I would need to use the .populate() function to access the nested object.
Am I doing something totally wrong here?
I should also mention that the route and the matching functions are in different files just to try and keep things simple.
Thanks for any answers.
just posting an answer as I seem to have fixed this.
Issue was with my .find() function, needed to pass in the items to search by and then also a call back within the function to return error/data. I'll leave the changed code below.
new function
const fetch_matches_using = async function(body, richness, smoke, sweetness) {
const data = await WhiskyModel.find({
'attributes.body': body,
'attributes.richness': richness,
'attributes.smoke': smoke,
'attributes.sweetness': sweetness
}, (error, data) => { // new ¬
if (error) {
return error;
}
if (data) {
console.log(data)
return data
}
});
return data; //new
}
There is still an issue with sending the found results back to the route but this is a different issue I believe. If its connected I'll edit this answer with the fix for that.
I'm currently trying Firestore, and I'm stuck at something very simple: "updating an array (aka a subdocument)".
My DB structure is super simple. For example:
proprietary: "John Doe",
sharedWith:
[
{who: "first#test.com", when:timestamp},
{who: "another#test.com", when:timestamp},
],
I'm trying (without success) to push new records into shareWith array of objects.
I've tried:
// With SET
firebase.firestore()
.collection('proprietary')
.doc(docID)
.set(
{ sharedWith: [{ who: "third#test.com", when: new Date() }] },
{ merge: true }
)
// With UPDATE
firebase.firestore()
.collection('proprietary')
.doc(docID)
.update({ sharedWith: [{ who: "third#test.com", when: new Date() }] })
None works. These queries overwrite my array.
The answer might be simple, but I could'nt find it...
Firestore now has two functions that allow you to update an array without re-writing the entire thing.
Link: https://firebase.google.com/docs/firestore/manage-data/add-data, specifically https://firebase.google.com/docs/firestore/manage-data/add-data#update_elements_in_an_array
Update elements in an array
If your document contains an array field, you can use arrayUnion() and
arrayRemove() to add and remove elements. arrayUnion() adds elements
to an array but only elements not already present. arrayRemove()
removes all instances of each given element.
Edit 08/13/2018: There is now support for native array operations in Cloud Firestore. See Doug's answer below.
There is currently no way to update a single array element (or add/remove a single element) in Cloud Firestore.
This code here:
firebase.firestore()
.collection('proprietary')
.doc(docID)
.set(
{ sharedWith: [{ who: "third#test.com", when: new Date() }] },
{ merge: true }
)
This says to set the document at proprietary/docID such that sharedWith = [{ who: "third#test.com", when: new Date() } but to not affect any existing document properties. It's very similar to the update() call you provided however the set() call with create the document if it does not exist while the update() call will fail.
So you have two options to achieve what you want.
Option 1 - Set the whole array
Call set() with the entire contents of the array, which will require reading the current data from the DB first. If you're concerned about concurrent updates you can do all of this in a transaction.
Option 2 - Use a subcollection
You could make sharedWith a subcollection of the main document. Then
adding a single item would look like this:
firebase.firestore()
.collection('proprietary')
.doc(docID)
.collection('sharedWith')
.add({ who: "third#test.com", when: new Date() })
Of course this comes with new limitations. You would not be able to query
documents based on who they are shared with, nor would you be able to
get the doc and all of the sharedWith data in a single operation.
Here is the latest example from the Firestore documentation:
firebase.firestore.FieldValue.ArrayUnion
var washingtonRef = db.collection("cities").doc("DC");
// Atomically add a new region to the "regions" array field.
washingtonRef.update({
regions: firebase.firestore.FieldValue.arrayUnion("greater_virginia")
});
// Atomically remove a region from the "regions" array field.
washingtonRef.update({
regions: firebase.firestore.FieldValue.arrayRemove("east_coast")
});
You can use a transaction (https://firebase.google.com/docs/firestore/manage-data/transactions) to get the array, push onto it and then update the document:
const booking = { some: "data" };
const userRef = this.db.collection("users").doc(userId);
this.db.runTransaction(transaction => {
// This code may get re-run multiple times if there are conflicts.
return transaction.get(userRef).then(doc => {
if (!doc.data().bookings) {
transaction.set({
bookings: [booking]
});
} else {
const bookings = doc.data().bookings;
bookings.push(booking);
transaction.update(userRef, { bookings: bookings });
}
});
}).then(function () {
console.log("Transaction successfully committed!");
}).catch(function (error) {
console.log("Transaction failed: ", error);
});
Sorry Late to party but Firestore solved it way back in aug 2018 so If you still looking for that here it is all issues solved with regards to arrays.
https://firebase.googleblog.com/2018/08/better-arrays-in-cloud-firestore.htmlOfficial blog post
array-contains, arrayRemove, arrayUnion for checking, removing and updating arrays. Hope it helps.
To build on Sam Stern's answer, there is also a 3rd option which made things easier for me and that is using what Google call a Map, which is essentially a dictionary.
I think a dictionary is far better for the use case you're describing. I usually use arrays for stuff that isn't really updated too much, so they are more or less static. But for stuff that gets written a lot, specifically values that need to be updated for fields that are linked to something else in the database, dictionaries prove to be much easier to maintain and work with.
So for your specific case, the DB structure would look like this:
proprietary: "John Doe"
sharedWith:{
whoEmail1: {when: timestamp},
whoEmail2: {when: timestamp}
}
This will allow you to do the following:
var whoEmail = 'first#test.com';
var sharedObject = {};
sharedObject['sharedWith.' + whoEmail + '.when'] = new Date();
sharedObject['merge'] = true;
firebase.firestore()
.collection('proprietary')
.doc(docID)
.update(sharedObject);
The reason for defining the object as a variable is that using 'sharedWith.' + whoEmail + '.when' directly in the set method will result in an error, at least when using it in a Node.js cloud function.
#Edit (add explanation :) )
say you have an array you want to update your existing firestore document field with. You can use set(yourData, {merge: true} ) passing setOptions(second param in set function) with {merge: true} is must in order to merge the changes instead of overwriting. here is what the official documentation says about it
An options object that configures the behavior of set() calls in DocumentReference, WriteBatch, and Transaction. These calls can be configured to perform granular merges instead of overwriting the target documents in their entirety by providing a SetOptions with merge: true.
you can use this
const yourNewArray = [{who: "first#test.com", when:timestamp}
{who: "another#test.com", when:timestamp}]
collectionRef.doc(docId).set(
{
proprietary: "jhon",
sharedWith: firebase.firestore.FieldValue.arrayUnion(...yourNewArray),
},
{ merge: true },
);
hope this helps :)
addToCart(docId: string, prodId: string): Promise<void> {
return this.baseAngularFirestore.collection('carts').doc(docId).update({
products:
firestore.FieldValue.arrayUnion({
productId: prodId,
qty: 1
}),
});
}
i know this is really old, but to help people newbies with the issue
firebase V9 provides a solution using the arrayUnion and arrayRemove
await updateDoc(documentRef, {
proprietary: arrayUnion( { sharedWith: [{ who: "third#test.com", when: new Date() }] }
});
check this out for more explanation
Other than the answers mentioned above. This will do it.
Using Angular 5 and AngularFire2. or use firebase.firestore() instead of this.afs
// say you have have the following object and
// database structure as you mentioned in your post
data = { who: "third#test.com", when: new Date() };
...othercode
addSharedWith(data) {
const postDocRef = this.afs.collection('posts').doc('docID');
postDocRef.subscribe( post => {
// Grab the existing sharedWith Array
// If post.sharedWith doesn`t exsit initiated with empty array
const foo = { 'sharedWith' : post.sharedWith || []};
// Grab the existing sharedWith Array
foo['sharedWith'].push(data);
// pass updated to fireStore
postsDocRef.update(foo);
// using .set() will overwrite everything
// .update will only update existing values,
// so we initiated sharedWith with empty array
});
}
We can use arrayUnion({}) method to achive this.
Try this:
collectionRef.doc(ID).update({
sharedWith: admin.firestore.FieldValue.arrayUnion({
who: "first#test.com",
when: new Date()
})
});
Documentation can find here: https://firebase.google.com/docs/firestore/manage-data/add-data#update_elements_in_an_array
Consider John Doe a document rather than a collection
Give it a collection of things and thingsSharedWithOthers
Then you can map and query John Doe's shared things in that parallel thingsSharedWithOthers collection.
proprietary: "John Doe"(a document)
things(collection of John's things documents)
thingsSharedWithOthers(collection of John's things being shared with others):
[thingId]:
{who: "first#test.com", when:timestamp}
{who: "another#test.com", when:timestamp}
then set thingsSharedWithOthers
firebase.firestore()
.collection('thingsSharedWithOthers')
.set(
{ [thingId]:{ who: "third#test.com", when: new Date() } },
{ merge: true }
)
If You want to Update an array in a firebase document.
You can do this.
var documentRef = db.collection("Your collection name").doc("Your doc name")
documentRef.update({
yourArrayName: firebase.firestore.FieldValue.arrayUnion("The Value you want to enter")});
Although firebase.firestore.FieldValue.arrayUnion() provides the solution for array update in firestore, at the same time it is required to use {merge:true}. If you do not use {merge:true} it will delete all other fields in the document while updating with the new value. Here is the working code for updating array without loosing data in the reference document with .set() method:
const docRef = firebase.firestore().collection("your_collection_name").doc("your_doc_id");
docRef.set({yourArrayField: firebase.firestore.FieldValue.arrayUnion("value_to_add")}, {merge:true});
If anybody is looking for Java firestore sdk solution to add items in array field:
List<String> list = java.util.Arrays.asList("A", "B");
Object[] fieldsToUpdate = list.toArray();
DocumentReference docRef = getCollection().document("docId");
docRef.update(fieldName, FieldValue.arrayUnion(fieldsToUpdate));
To delete items from array user: FieldValue.arrayRemove()
If the document contains a nested object in the form of an array, .dot notation can be used to reference and update nested fields.
Node.js example:
const users = {
name: 'Tom',
surname: 'Smith',
favorites: {
sport: 'tennis',
color: 'red',
subject: 'math'
}
};
const update = await db.collection('users').doc('Tom').update({
'favorites.sport': 'snowboard'
});
or Android sdk example:
db.collection("users").document("Tom")
.update(
'favorites.sport': 'snowboard'
);
There is a simple hack in firestore:
use path with "." as property name:
propertyname.arraysubname.${id}:
db.collection("collection")
.doc("docId")
.update({arrayOfObj: fieldValue.arrayUnion({...item})})
I have the following GraphQL schema, which defines 3 types: a CondaPackage which hasmany CondaVersion, which hasmany CondaExecutable. I want to be able to query a CondaVersion and ask "how many CondaExecutables do you own which succeeded my analysis". Currently I've written a succeededExeCount and allExeCount which resolve this field by loading all children and manually counting the number of children that succeeded.
exports.createSchemaCustomization = ({ actions: { createTypes }, schema }) => {
createTypes([
schema.buildObjectType({
name: "CondaPackage",
fields: {
succeededExeCount: {
type: "Int!",
resolve(source, args, context){
// TODO
}
},
allExeCount: {
type: "Int!",
resolve(source, args, context){
// TODO
}
}
},
interfaces: ["Node"]
}),
schema.buildObjectType({
name: "CondaVersion",
fields: {
succeededExeCount: {
type: "Float!",
resolve(source, args, context){
const children = context.nodeModel.getNodesByIds({
ids: source.children,
type: "CondaExecutable"
})
return children.reduce((acc, curr) => acc + curr.fields.succeeded, 0)
}
},
allExeCount: {
type: "Int!",
resolve(source, args, context){
return source.children.length;
}
}
},
interfaces: ["Node"]
}),
schema.buildObjectType({
name: "CondaExecutable",
fields: {
succeeded: {
type: "Boolean!",
resolve(source, args, context, info) {
return source.fields.succeeded || false;
}
},
},
interfaces: ["Node"]
})
])
}
My first problem is that this seems incredibly inefficient. For each CondaVersion I'm running a separate query for its children, which is a classic N+1 query problem. Is there a way to tell Gatsby/GraphQL to simply "join" the two tables like I would using SQL to avoid this?
My second problem is that I now need to count the number of succeeding children from the top level type: CondaPackage. I want to ask "how many CondaExecutables do your child CondaVersions own which succeeded my analysis". Again, in SQL this would be easy because I would just JOIN the 3 types. However, the only way I can currently do this is by using getNodesByIds for each child, and then for each child's child, which is n*m*o runtime, which is terrifying. I would like to run a GraphQL query as part of the field resolution which lets me grab the succeededExeCount from each child. However, Gatsby's runQuery seems to return nodes without including derived fields, and it won't let me select additional fields to return. How can I access fields on a node's child's child in Gatsby?
Edit
Here's the response from a Gatsby maintainer regarding the workaround:
Gatsby has an internal mechanism to filter/sort by fields with custom resolvers. We call it materialization. [...] The problem is that this is not a public API. This is a sort of implementation detail that may change someday and that's why it is not documented.
See the full thread here.
Original Answer
Here's a little 'secret' (not mentioned anywhere in the docs at the time of writing):
When you use runQuery, Gatsby will try to resolve derived fields... but only if that field is passed to the query's options (filter, sort, group, distinct).
For example, in CondaVersion, instead of accessing children nodes and look up fields.succeeded, you can do this:
const succeededNodes = await context.nodeModel.runQuery({
type: "CondaExecutable",
query: { filter: { succeeded: { eq: true } } }
})
Same thing for CondaPackage. You might try to do this
const versionNodes = await context.nodeModel.runQuery({
type: "CondaVersion",
query: {}
})
return versionNodes.reduce((acc, nodes) => acc + node.succeededExeCount, 0) // Error
You'll probably find that succeededExeCount is undefined.
The trick is to do this:
const versionNodes = await context.nodeModel.runQuery({
type: "CondaVersion",
- query: {}
+ query: { filter: { succeededExeCount: { gte: 0 } } }
})
It's counter intuitive, because you'd think Gatsby would just resolve all resolvable fields on a type. Instead it only resolves fields that is 'used'. So to get around this, we add a filter that supposedly does nothing.
But that's not all yet, node.succeededExeCount is still undefined.
The resolved data (succeededExeCount) is not directly stored on the node itself, but in node.__gatsby_resolved source. We'll have to access it there instead.
const versionNodes = await context.nodeModel.runQuery({
type: "CondaVersion",
query: { filter: { succeededExeCount: { gte: 0 } } }
})
return versionNodes.reduce((acc, node) => acc + node.__gatsby_resolved.succeededExeCount, 0)
Give it a try & let me know if that works.
PS: I notice that you probably use createNodeField (in CondaExec's node.fields.succeeded?) createTypes is also accessible in exports.sourceNodes, so you might be able to add this succeeded field directly.
I'm work with NodeJS and Sequelize. I have the following issue:
Read the Settings Table:
Settings.findOne({where: {
user_id: data
}})
.then(settings => {
// Next request
});
I need to save the settings.device (Example) outside of the .then block.
But if I do that like
var device;
Settings.findOne({where: {
user_id: data
}})
.then(settings => {
device = settings.device;
});
It doesn't work.
Already the error output undefined
The output in .then result block with console.log(settings.device); works perfect.
Update
I need it like:
var array = [];
// Get Settings from table
Settings.findOne({where: {
user_id: data
}})
.then(settings => {
// Get token from other Table
Example.findOne({where: {
user_id: data
}})
.then(example => {
// push to array
array.push({
"user_id" : data,
"device":settings.device, // output: settings.device is undefined
"token": example.token
});
});
});
// Send array to client
This is really a question of how to handle multiple resolved values in a Promise chain. You can search for that and see lots of great examples on how to handle it. For example, you could return an array or object in each then handler, or re-assign values to higher-scoped variables (as you're doing with settings). I've used both methods heavily in the past, and the resulting code is obviously inelegant and not fun to write.
However, async/await is readily available in Node and simplifies your code quite a bit:
const array = [];
// Get settings.
const settings = await Settings.findOne({where: {
user_id: data
}});
// Get token.
const example = await Example.findOne({where: {
user_id: data
}});
array.push({
user_id : data,
device: settings.device,
token: example.token
});
Sequelize return the model object you can get value by dataValue
console.log(settings.dataValues.device);
or if you want lean data
Settings.findOne({where: {
user_id: data,
raw:true,
}})
.then(settings => {
device = settings.device;
console.log(device);
});
I am having difficulty figuring out what is happening (and not happening) in my action creator.
I need to make a call to one API endpoint, get the ids and names of all the items returned, then for each of those ids, make another call. I want to store the return of the last call and the ids/names from the first call in an object and dispatch it.
{
category: name //name of category
subcategory: [] //array of categories in the category above.
}
Right now, my reducer does end up having what I want, but when I attempt to log that particular prop in the component it is empty. (below I am using OpenSocialAPI or osapi. This is just a basic wrapper for an ajax request. Allows for me to not have to authenticate as it sees I am already authenticated.)
export function fetchCategories(id){
let categories = []
return function(dispatch){
dispatch(requestCategories(id))
return osapi.core.get({
v: "v3",
href: "/places/" + id + "/places"
}).execute(function(response) {
response.content.list.forEach(function(category) {
osapi.core.get({
v: "v3",
href: "/places/" + category.id+ "/categories"
}).execute(function(response) {
categories.push({
category: category.name,
subcategories: response.content.list.map(category => category.name)
})
})
})
console.log("Category response: ", categories)
dispatch(receiveCategories(id, categories))
})
}
}
export function receiveCategories(id,array){
return {
type: RECEIVE_CATEGORIES,
id,
categories: array,
recievedAt: new Date(Date.now()),
isFetching: false
}
}
And in my app I am dispatching the action creator in componentDidMount
componentDidMount() {
const { dispatch } = this.props
dispatch(fetchCategoriesIfNeeded(id))
}
Right now when I console log in my Category component and in the execute above, it is empty. But looking at my state in my logger, when recieveCategories is completed, I have the array of objects I want
[{category:...,
subcategories:[...]
},
{category:...,
subcategories:[...]
}]
I suspect this is because of something asynchronous but I'm unsure how to proceed.
I attempted to create my own wrapper for the call that was promise based, but I had similar issues, probably more so because I'm not sure if resolve(response) is what I want.
function httpService(path){
return new Promise(function(resolve, reject){
osapi.core.get({
v: 'v3',
href: path
}).execute(function(response, error){
if(error){
return reject(new Error("Error: ", error))
}
resolve(response)
})
})
}
export function fetchCategories(spaceId) {
let categories = []
return function(dispatch) {
dispatch(requestCategories(id))
return httpService("/places/" + id + "/places")
.then(function(response) {
response.content.list.forEach(function(category) {
fetchSubCategories("/places/" + category.id + "/categories")
.then(function(response) {
categories.push({
category: category.name,
subcategories: response
})
})
})
console.log("CATEGORIES: ", categories)
dispatch(receiveCategories(id, categories))
})
}
}
function fetchSubCategories(url){
return httpService(url)
.then(function(response){
return response.content.list.map(category => category.name)
})
}
Can you look at this and give guidance? Also, is me dispatching an array that I built based on the API responses a valid way of doing things or is there someway better? Thank you
I was only able to find 1 other question with similar use case but they are using bluebird or something similar. I'd really like to keep this without anything extra besides Redux.
It looks like you just need to dispatch your categories inside your .execute() callback, not outside of it. You're doing osapi.core.get().execute((response) => but then outside of that execute callback, you dispatch receiveCategories, which will execute long before your Promise resolves, and dispatch the empty array you initialize.
You also need to use Promise.all to get the response of all of your nested GET requests.
There's also no reason to keep a mutating array around.
I guess osapi.core.get is some kind of promise based fetch library? And .execute is called when the get succeeds?
If so, then what you're missing is that you're not waiting for all asynchronous calls to finish.
I'm going to show a solution based on generic fetch and native Promises so you can understand the solution and adopt it based on your specific libraries.
const promises = [];
response.content.list.forEach(function(category) {
const promise = fetch("/places/" + category.id+ "/categories");
promises.push(promise);
})
Promise.all(promises).then(responses => {
categories = responses.map(response => ({/* do your object mapping here */}))
// now you can dispatch with all received categories
dispatch(receiveCategories(id, categories))
});
Also, you're using the same variable in your nested functions - while this may work and the computers may understand it, it makes it super hard for any human to figure out which response belongs to which scope. So you may want to take a second look at your variable names as well.