I have been looking at the docs indexes for FQL and Fauna DB. Is there a way to reverse the order of the data returned from the query?
Here is the code I have:
q.Map(
q.Paginate(q.Match(q.Index("all_pages")), {
//! Find way to change order
size: 3
}),
ref => q.Get(ref)
)
The docs mention the reverse flag.
Does anyone know how to use it?
Imagine you have a collection that contains documents with a price field, let's call it ... ermm ... priceables!
Let's add two documents to the collection with price 1 and price 2.
[{
"price": 1
},
{
"price": 2
}]
Image in UI console of the documents
Create a regular index
Create a regular range index on that price value with the following snippet:
CreateIndex({
name: "regular-value-index",
unique: false,
serialized: true,
source: Collection("priceables"),
terms: [],
values: [
{
field: ["data", "price"]
}
]
})
Create a reverse index
There can be multiple values (e.g. a composite index) and the reverse field can be set per value.
To create a reverse index, set reverse to true for that specific value.
CreateIndex({
name: "reverse-value-index",
unique: false,
serialized: true,
source: Collection("priceables"),
terms: [],
values: [
{
field: ["data", "price"],
reverse: true
}
]
})
If you go to the UI console and open the index you will notice the values are sorted from highest to low.
UI console, reverse index
I assume that what confused you is that you can't set the reverse boolean in the UI yet. But you can just go to Shell and paste in the FQL code to create the index: Image of the shell
I just reversed an Index successfully. The docs are helpful, but they don't give an example where the Map function is implemented. The secret is to wrap the Map function as well, with q.Reverse:
q.Reverse(
q.Map(
q.Paginate(q.Match(q.Index("all_pages")), {
//! Find way to change order
size: 3
}),
ref => q.Get(ref)
)
)
This should work for you!
The best way to reverse the order of data is to use an index, per Brecht's answer. Fauna stores index entries in sorted order, so when your index uses reverse: true matching entries from the index are already sorted in reverse order.
WΔ_'s answer is technically correct, but probably not what you want. Using Reverse around a Paginate reverses the items in the page, but not the overall set. Suppose your set was the letters of the alphabet, and your page size is 3. Without Reverse, you'd expect to see pages with ABC, DEF, etc. If you're trying to change the order of the set, you'd expect to see ZYX, WVU, etc. WΔ_'s answer results in CBA, FED, etc.
Reverse can operate on a set. So you can Reverse(Match(Index("all_pages"))), and Paginate/Map that result. Note that for large sets, your query might fail since Reverse has to operate on the entire set.
Related
So I have a collection looking like this:
[
{"url":"website.com/test", "links":[ {"url": "www.something.fr/page.html","scoreDiff": 0.44} ], "justUpdated": true, "score": 0.91},
{"url":"domain.com/", "links":[], "justUpdated": true, "score": 0.81},
{"url":"www.something.fr/page.html", "links":[], "justUpdated": false, "score": 0.42},
]
The goal here is to get the third document, because in one of the documents where "justUpdated" equals true (the first one here), there is its url as a value in one of the "links" array elements.
To achieve that, I tried:
To find all the documents with "justUpdated" equals to true, then in NodeJS concatenate all the urls in their "links" arrays (let's call this array urlsOfInterest). And finally do another query to find all the documents where the url is in urlsOfInterest.
The problem is that it takes some time to do the first query then process the result and do the second query.
So I thought maybe I could do it all at once in an aggregate query. I use $group (with $cond to check if justUpdated equals true) to get all the arrays of "links" in one new variable named urlsOfInterest. For now this is an array of arrays of object so I then use $project with $reduce to have all these {url: "...", score: X} objects as one big array. Finally I use $project and $map to only have the url as the score value doesn't interest me here.
So I get an output looking like this:
_id:"urlsOfInterest",
urlsOfInterest: ["www.something.fr/page.html"]
Which is pretty great but I am stuck because now I just need to get the documents where url is in this variable named urlsOfInterest except I can't because all my documents have "disappeared" after the $group stage.
Please help me to find a way to perform this final query :) Or if this isn't the right way to do this, please point me in the right direction !
PS: the real goal here would be to update for all the documents where justUpdated equals true, every scoreDiff values in their links array. For our exemple, we do abs(0.91 - 0.42) = 0.49 so we replace our scoreDiff value of 0.44 by 0.49 (0.91 being the score of the document where justUpdated equals true and 0.42 the score of the document where url equals www.something.fr/page.html, explaining why I need to fetch this last document.) I don't believe there could be a way of doing all of this at once but if there is, please tell me !
You can use $lookup to get all matching documents in an array:
db.collection.aggregate([
{
"$match": {
"justUpdated": true
}
},
{
"$lookup": {
"from": "collection",
"localField": "links.url",
"foreignField": "url",
"as": "result"
}
},
{
"$match": {
"result": {
$gt: []
}
}
}
])
Then either $unwind and $replaceRoot the results array to get the documents as a cursor and do the math on the application level or do the calculations in the same pipeline, e.g. with $reduce
The "PS: the real goal" is not quite clear as it is based on a particular example but if you play a little bit with it in the playground I am sure you can calculate the numbers as per your requirements.
Here is simplified version of my schema:
var MySchema = new Schema({
createdDate: {
type: Date,
required: true,
default: Date.now,
index: true
},
vote: {
type: Number,
index: true,
default: 0
}
});
I have large amount of data, so for paging with good performance I use range query like: .find({_id: {$gt: lastId}}).limit(20). Now I also want to sort my data by vote field. How should I do this?
Fairly much the same thing as the looking for a greater value concept, but this time on the "vote", but with another twist:
var query = Model.find({
"vote": { "$lte": lastVoteValue },
"_id": { "$nin": seenIds }
}).sort({ "vote": -1 }).limit(20);
So if you think about what is going on here, since you are doing a "descending" sort you want values of vote that are either "less than or equal to" the last value seen from your previous page.
The second part would be the "list" of previously seen _id values from either just the last page or possibly more. That part depends on how "granular" your "vote" values are in order to maintain that none of the items already paged are seen in the next page.
So the $nin operator does the exclusion here. You might want to track how that "vote" value varies to help you decide when to reduce the list of "seenIds".
That's the way to do range queries for paging, but if you need to jump to "specific" pages by number don't forget that you would still need to .limit() and .skip(). But this will work with single page jumps, or otherwise just incremental paging.
I am trying to update a value in the nested array but can't get it to work.
My object is like this
{
"_id": {
"$oid": "1"
},
"array1": [
{
"_id": "12",
"array2": [
{
"_id": "123",
"answeredBy": [], // need to push "success"
},
{
"_id": "124",
"answeredBy": [],
}
],
}
]
}
I need to push a value to "answeredBy" array.
In the below example, I tried pushing "success" string to the "answeredBy" array of the "123 _id" object but it does not work.
callback = function(err,value){
if(err){
res.send(err);
}else{
res.send(value);
}
};
conditions = {
"_id": 1,
"array1._id": 12,
"array2._id": 123
};
updates = {
$push: {
"array2.$.answeredBy": "success"
}
};
options = {
upsert: true
};
Model.update(conditions, updates, options, callback);
I found this link, but its answer only says I should use object like structure instead of array's. This cannot be applied in my situation. I really need my object to be nested in arrays
It would be great if you can help me out here. I've been spending hours to figure this out.
Thank you in advance!
General Scope and Explanation
There are a few things wrong with what you are doing here. Firstly your query conditions. You are referring to several _id values where you should not need to, and at least one of which is not on the top level.
In order to get into a "nested" value and also presuming that _id value is unique and would not appear in any other document, you query form should be like this:
Model.update(
{ "array1.array2._id": "123" },
{ "$push": { "array1.0.array2.$.answeredBy": "success" } },
function(err,numAffected) {
// something with the result in here
}
);
Now that would actually work, but really it is only a fluke that it does as there are very good reasons why it should not work for you.
The important reading is in the official documentation for the positional $ operator under the subject of "Nested Arrays". What this says is:
The positional $ operator cannot be used for queries which traverse more than one array, such as queries that traverse arrays nested within other arrays, because the replacement for the $ placeholder is a single value
Specifically what that means is the element that will be matched and returned in the positional placeholder is the value of the index from the first matching array. This means in your case the matching index on the "top" level array.
So if you look at the query notation as shown, we have "hardcoded" the first ( or 0 index ) position in the top level array, and it just so happens that the matching element within "array2" is also the zero index entry.
To demonstrate this you can change the matching _id value to "124" and the result will $push an new entry onto the element with _id "123" as they are both in the zero index entry of "array1" and that is the value returned to the placeholder.
So that is the general problem with nesting arrays. You could remove one of the levels and you would still be able to $push to the correct element in your "top" array, but there would still be multiple levels.
Try to avoid nesting arrays as you will run into update problems as is shown.
The general case is to "flatten" the things you "think" are "levels" and actually make theses "attributes" on the final detail items. For example, the "flattened" form of the structure in the question should be something like:
{
"answers": [
{ "by": "success", "type2": "123", "type1": "12" }
]
}
Or even when accepting the inner array is $push only, and never updated:
{
"array": [
{ "type1": "12", "type2": "123", "answeredBy": ["success"] },
{ "type1": "12", "type2": "124", "answeredBy": [] }
]
}
Which both lend themselves to atomic updates within the scope of the positional $ operator
MongoDB 3.6 and Above
From MongoDB 3.6 there are new features available to work with nested arrays. This uses the positional filtered $[<identifier>] syntax in order to match the specific elements and apply different conditions through arrayFilters in the update statement:
Model.update(
{
"_id": 1,
"array1": {
"$elemMatch": {
"_id": "12","array2._id": "123"
}
}
},
{
"$push": { "array1.$[outer].array2.$[inner].answeredBy": "success" }
},
{
"arrayFilters": [{ "outer._id": "12" },{ "inner._id": "123" }]
}
)
The "arrayFilters" as passed to the options for .update() or even
.updateOne(), .updateMany(), .findOneAndUpdate() or .bulkWrite() method specifies the conditions to match on the identifier given in the update statement. Any elements that match the condition given will be updated.
Because the structure is "nested", we actually use "multiple filters" as is specified with an "array" of filter definitions as shown. The marked "identifier" is used in matching against the positional filtered $[<identifier>] syntax actually used in the update block of the statement. In this case inner and outer are the identifiers used for each condition as specified with the nested chain.
This new expansion makes the update of nested array content possible, but it does not really help with the practicality of "querying" such data, so the same caveats apply as explained earlier.
You typically really "mean" to express as "attributes", even if your brain initially thinks "nesting", it's just usually a reaction to how you believe the "previous relational parts" come together. In reality you really need more denormalization.
Also see How to Update Multiple Array Elements in mongodb, since these new update operators actually match and update "multiple array elements" rather than just the first, which has been the previous action of positional updates.
NOTE Somewhat ironically, since this is specified in the "options" argument for .update() and like methods, the syntax is generally compatible with all recent release driver versions.
However this is not true of the mongo shell, since the way the method is implemented there ( "ironically for backward compatibility" ) the arrayFilters argument is not recognized and removed by an internal method that parses the options in order to deliver "backward compatibility" with prior MongoDB server versions and a "legacy" .update() API call syntax.
So if you want to use the command in the mongo shell or other "shell based" products ( notably Robo 3T ) you need a latest version from either the development branch or production release as of 3.6 or greater.
See also positional all $[] which also updates "multiple array elements" but without applying to specified conditions and applies to all elements in the array where that is the desired action.
I know this is a very old question, but I just struggled with this problem myself, and found, what I believe to be, a better answer.
A way to solve this problem is to use Sub-Documents. This is done by nesting schemas within your schemas
MainSchema = new mongoose.Schema({
array1: [Array1Schema]
})
Array1Schema = new mongoose.Schema({
array2: [Array2Schema]
})
Array2Schema = new mongoose.Schema({
answeredBy": [...]
})
This way the object will look like the one you show, but now each array are filled with sub-documents. This makes it possible to dot your way into the sub-document you want. Instead of using a .update you then use a .find or .findOne to get the document you want to update.
Main.findOne((
{
_id: 1
}
)
.exec(
function(err, result){
result.array1.id(12).array2.id(123).answeredBy.push('success')
result.save(function(err){
console.log(result)
});
}
)
Haven't used the .push() function this way myself, so the syntax might not be right, but I have used both .set() and .remove(), and both works perfectly fine.
Given an example in http://dojotoolkit.org/reference-guide/1.9/dojo/store/util/SimpleQueryEngine
SimpleQueryEngine(function(object){
return object.id > 1;
})(someData) // Returns an array with matching objects
Im using DataStore http://livedocs.dojotoolkit.org/dojo/store/DataStore
DataStore contains a store of type ItemFileWriteStore http://livedocs.dojotoolkit.org/dojo/data/ItemFileWriteStore
I'm trying the following:
var myNumber = 12; // but could be any number like -12, 0.12, 12345 or -12345.1
dataStore.store.query(function(storeItem){
return storeItem.number < myNumber;
})
and that doesn't really work.
As far as I can see SimpleQueryEngine uses dojo/_base/array which uses a filter method that takes in parameters such as an array and callback function to filter by.
As you can see here: https://github.com/dojo/dojo/blob/master/store/DataStore.js, DataStore uses SimpleQueryEngine so it should work...
My store contains objects like this:
[{id: 1, number: 2345},{id: 2, number: 23.45},{id: 3, number: -2345},{id: 4, number: 2345},{id: 5, number: 0.2345}]
I would like to query given store to find a number that is less than a given number.
And to clarify what I'm really trying to understand is why passing in a function as a parameter to query() method doesn't work and how to make it work.
Thanks,
Since you have an ItemFileWriteStore, you could use it's function called fetch. It might be able to do what you need. Let's say store is your ItemFileWriteStore, you would do:
store.fetch({
sort: [{attribute: "number", descending: true}]
onComplete: function(items, request) {
// items will be an array of the items in your store,
// sorted by the "number" attributed in descending order
}
});
In your onComplete function you would have the array of sorted store items in descending order based on the number parameter. At this point it would be trivial to find all numbers that are less than your specified number (rather than just 1 as you stated).
So I am using mongoose and node.js to access a mongodb database. I want to bump up each result based on a number (they are ordered by date created if none are bumped up). For example:
{ name: 'A',
bump: 0 },
{ name: 'B',
bump: 0 },
{ name: 'C',
bump: 2 },
{ name: 'D',
bump: 1 }
would be retreived in the order: C, A, D, B. How can this be accomplished (without iterating through every entry in the database)?
Try something like this. Store a counter tracking the total # of threads, let's call it thread_count, initially set to 0, so have a document somewhere that looks like {thread_count:0}.
Every time a new thread is created, first call findAndModify() using {$inc : {thread_count:1}} as the modifier - i.e., increment the counter by 1 and return its new value.
Then when you insert the new thread, use the new value for the counter as the value for a field in its document, let's call it post_order.
So each document you insert has a value 1 greater each time. For example, the first 3 documents you insert would look like this:
{name:'foo', post_order:1, created_at:... } // value of thread_count is at 1
{name:'bar', post_order:2, created_at:... } // value of thread_count is at 2
{name:'baz', post_order:3, created_at:... } // value of thread_count is at 3
etc.
So effectively, you can query and order by post_order as ASCENDING, and it will return them in the order of oldest to newest (or DESCENDING for newest to oldest).
Then to "bump" a thread in its sorting order when it gets upvoted, you can call update() on the document with {$inc:{post_order:1}}. This will advance it by 1 in the order of result sorting. If two threads have the same value for post_order, created_at will differentiate which one comes first. So you will sort by post_order, created_at.
You will want to have an index on post_order and created_at.
Let's guess your code is the variable response (which is an array), then I would do:
response.sort(function(obj1, obj2){
return obj2.bump - obj1.bump;
});
or if you want to also take in mind name order:
response.sort(function(obj1, obj2){
var diff = obj2.bump - obj1.bump;
var nameDiff = (obj2.name > obj1.name)?-1:((obj2.name < obj1.name)?1:0);
return (diff == 0) ? nameDiff : diff;
});
Not a pleasant answer, but the solution you request is unrealistic. Here's my suggestion:
Add an OrderPosition property to your object instead of Bump.
Think of "bumping" as an event. It is best represented as an event-handler function. When an item gets "bumped" by whatever trigger in your business logic, the collection of items needs to be adjusted.
var currentOrder = this.OrderPosition
this.OrderPosition = currentOrder - bump; // moves your object up the list
// write a foreach loop here, iterating every item AFTER the items unadjusted
// order, +1 to move them all down the list one notch.
This does require iterating through many items, and I know you are trying to prevent that, but I do not think there is any other way to safely ensure the integrity of your item ordering - especially when relative to other pulled collections that occur later down the road.
I don't think a purely query-based solution is possible with your document schema (I assume you have createdDate and bump fields). Instead, I suggest a single field called sortorder to keep track of your desired retrieval order:
sortorder is initially the creation timestamp. If there are no "bumps", sorting by this field gives the correct order.
If there is a "bump," the sortorder is invalidated. So simply correct the sortorder values: each time a "bump" occurs swap the sortorder fields of the bumped document and the document directly ahead of it. This literally "bumps" the document up in the sort order.
When querying, sort by sortorder.
You can remove fields bump and createdDate if they are not used elsewhere.
As an aside, most social sites don't directly manipulate a post's display position based on its number of votes (or "bumps"). Instead, the number of votes is used to calculate a score. Then the posts are sorted and displayed by this score. In your case, you should combine createdDate and bumps into a single score that can be sorted in a query.
This site (StackOverflow.com) had a related meta discussion about how to determine "hot" questions. I think there was even a competition to come up with a new formula. The meta question also shared the formulas used by two other popular social news sites: Y Combinator Hacker News and Reddit.