I have a schema like this.
peopleSchema = Schema({
followerRefs: {
type: [
{ type: Schema.Types.ObjectId, ref: 'peoples'}
],
select: false
},
followRefs: {
type: [
{ type: Schema.Types.ObjectId, ref: 'peoples'}
],
select: false
}
});
Every time, I just want to select part of the followerRefs or followRefs. For example, I want to implement a paging, so I just want to select first 20 in the followRefs, or first 21 ~ 40 in the followRefs.
So, are there any way to get part of the followerRefs with select all of the list?
It seems that I didn't explain my question clearly. I assume that there are over one million entity in the followerRefs in database, and I just want to get the first 20 of them, which mean I just want to get the index of 0~19 of them. So I don't want to load all of the one million entity into the memory.
So I'm wondering whether there are any way to get the first 20 entity without load all of them?
If it's just a list of names, I don't think you should handle the sorting via mongoose nor the backend.
But it depends on the size of data still.
Via javascript(frontend) I would sort the list first depending on what category, then I'd sort it.
Sorting with plain javascript could be complicated and I'm not an expert with it, since I use plugins. XD
So here's a link with a link with an answer about javascript sorting.
Sorting an array of JavaScript objects
Since you are asking about parts of the array, just use javascript slice() function.
var myList = ["Follower1", "Follower2", "Follower3", ...]; //pretend it goes til 20
var page1 = myList.slice(0, 19);
Related
I'm trying to generate a series of actions, where the actions that should be generated depend on what the previous actions were.
Let's say my state is a set of numbers stored as an array:
[1, 2]
And there are the following actions:
{ type: "add", value: number }
{ type: "remove", value: number }
I want to generate a series of actions to dispatch before checking properties of the state. If a remove action is generated, I want to ensure that it's value is in the state.
Valid Examples:
initial state: [1, 2]
[{ type: "remove", value: 1 }, { type: "remove", value: 2 }]
[{ type: "add", value: 3 }, { type: "remove", value: 3 }]
Invalid Examples:
initial state: [1, 2]
[{ type: "remove", value: 3 }]
[{ type: "remove", value: 2 }, { type: "remove", value: 2 }]
Is this something that is possible using a property based testing library, and if so how would I accomplish it?
I am using https://github.com/dubzzz/fast-check, but if this is easier using another library I'm open to examples from others.
Yes, this is perfectly suited for property-based testing. From a quick skimming of the docs of fast-check, I can see three approaches to this:
Make a precondition. This will ignore tests where an invalid sequence of actions was generated.
Prior to running a remove action, you'd count how often the number was contained in the initial state, how often in add actions before now, and how often in remove actions before now. Then you'd know whether you can remove it another time.
Use model-based testing (also here). This seems to perfectly fit your use case. Each action would be represented by one Command, all commands would simply apply their respective action, and in their check method you would validate whether the action is eligible.
This requires building a model, where you need to make sure that the model is a simplification of the actual state and that it uses a different implementation approach (so that you won't re-implement your bugs here). In your example, that could mean keeping a Set of occurring numbers or a Map of their counts, not an ordered array.
Generate only valid sequences in the first place.
This is much more efficient than the first two approaches, but usually also much more complicated. It might be necessary though if the generated numbers to be removed are too unbiased and rarely ever match one that is in the list. I got two ideas here:
generate your list of actions recursively, and keep a model similar to the one from model-based testing. You have to update it yourself however. With this, you can generate remove actions only exactly for those numbers that are currently in your model.
I'm not sure whether letrec or memo help here, whether you might need to use chain, or ask the library author to provide an extra function for this use case. (Maybe even as part of model-based testing, where Command instances could be dynamically derived from the current model?)
generate a remove action always together with a preceding add action with the same number. After having generated a list of [add(x)] and [add(y), remove(y)] lists, merge these in arbitrary order but keeping the respective order between the elements of each sublist.
This is probably the most elegant method to do this, as it looks nothing like the model of your state. However, I'm pretty certain that you will need to build your own Arbitrary for the randomMerge function - maybe ask the library author for help with this or request a new feature.
I'm wanting to make a system that gets a users inventory then displays it as the image and name. I only know how to do the JSON part and I'm unsure as what to do next.
All I have at the moment is:
http://steamcommunity.com/profiles/<PROFILEID>/inventory/json/753/1
Is anyone able to help me turn that data into what I am looking for?
First off - for CS:GO, at least - the URL you are looking for is:
http://steamcommunity.com/profiles/<PROFILEID>/inventory/json/730/2
The two numbers at the end of the URL refer to the app ID and context ID, respectively. CS:GO's app ID is 730 and most games use a context ID of 2 for user inventories.
The JSON returned from this request is an object in the following format:
{
"success": true,
"rgInventory": { ... },
"rgCurrency": { ... },
"rgDescriptions": { ... },
"more": false,
"more_start": false
}
For the use-case you described (getting the item names and icons), you can ignore everything except the rgDescriptions object. This object contains an object for each item in the user's inventory. The object keys are the result of concatenating the item's classid and instanceid, but that doesn't really matter for you - you can just iterate over it like you would for any other object.
The two data points that you're interested in are market_hash_name, which is the name of the item, and icon_url, which is part of what you need to display the actual image. The full path to the image is https://steamcommunity-a.akamaihd.net/economy/image/{icon_url}. For example, this link loads the icon for a G3SG1 | Polar Camo in my inventory.
One thing to note is that the market_hash_name includes the wear pattern (e.g., Minimal Wear, Factory New, etc.). If you don't need those, you can just use the name from the object.
Should I store objects in an Array or inside an Object with top importance given Write Speed?
I'm trying to decide whether data should be stored as an array of objects, or using nested objects inside a mongodb document.
In this particular case, I'm keeping track of a set of continually updating files that I add and update and the file name acts as a key and the number of lines processed within the file.
the document looks something like this
{
t_id:1220,
some-other-info: {}, // there's other info here not updated frequently
files: {
log1-txt: {filename:"log1.txt",numlines:233,filesize:19928},
log2-txt: {filename:"log2.txt",numlines:2,filesize:843}
}
}
or this
{
t_id:1220,
some-other-info: {},
files:[
{filename:"log1.txt",numlines:233,filesize:19928},
{filename:"log2.txt",numlines:2,filesize:843}
]
}
I am making an assumption that handling a document, especially when it comes to updates, it is easier to deal with objects, because the location of the object can be determined by the name; unlike an array, where I have to look through each object's value until I find the match.
Because the object key will have periods, I will need to convert (or drop) the periods to create a valid key (fi.le.log to filelog or fi-le-log).
I'm not worried about the files' possible duplicate names emerging (such as fi.le.log and fi-le.log) so I would prefer to use Objects, because the number of files is relatively small, but the updates are frequent.
Or would it be better to handle this data in a separate collection for best write performance...
{
"_id": ObjectId('56d9f1202d777d9806000003'),"t_id": "1220","filename": "log1.txt","filesize": 1843,"numlines": 554
},
{
"_id": ObjectId('56d9f1392d777d9806000004'),"t_id": "1220","filename": "log2.txt","filesize": 5231,"numlines": 3027
}
From what I understand you are talking about write speed, without any read consideration. So we have to think about how you will insert/update your document.
We have to compare (assuming you know the _id you are replacing, replace {key} by the key name, in your example log1-txt or log2-txt):
db.Col.update({ _id: '' }, { $set: { 'files.{key}': object }})
vs
db.Col.update({ _id: '', 'files.filename': '{key}'}, { $set: { 'files.$': object }})
The second one means that MongoDB have to browse the array, find the matching index and update it. The first one means MongoDB just update the specified field.
The worst:
The second command will not work if the matching filename is not present in the array! So you have to execute it, check if nMatched is 0, and create it if it is so. That's really bad write speed (see here MongoDB: upsert sub-document).
If you will never/almost never use read queries / aggregation framework on this collection: go for the first one, that will be faster. If you want to aggregate, unwind, do some analytics on the files you parsed to have statistics about file size and line numbers, you may consider using the second one, you will avoid some headache.
Pure write speed will be better with the first solution.
I am starting to build an app and have a collection that is used to create a pet post.
Note: I'm using the following packages to build my collection schemas, Simple Schema https://atmospherejs.com/aldeed/simple-schema
The page that I'm using to insert the data into this particular collection has a section that based on the choice the user makes they are presented with more options.
For example focusing in on one question there are some checkboxes, by default none are selected but if the user selects any checkbox they are given more options to choose from either via checkboxes or text areas. The point is that they will have additional options to choose from based on their selection.
I think the best way to handle how the data is inserted is via the template helpers for that specific section so that's not the question.
Instead where i'm confused is at the collection level and how to Schema that information or if I even need to.
For example let's say you have a question that involves pets.
The question could be: Do you have pets?
The checkbox options could be:
Dogs
Cats
Other
By default all of these checkboxes are unchecked but if the user checks any of the boxes then a text area is revealed (again i realize all of this happens at the Template.events and Template.helpers level).
What my real question is, now that I've set the scenario, is how do you store information like that or schema it to allow these kinds of values.
Assuming that all of the functions to grab this data are all on the Template.helpers level so this is simple a question about how to store kind of data in this scenario.
My notion was to do something like this (using the simpleSchema package):
Pet = new SimpleSchema({
petInstructions: {
type: String,
optional: true,
allowedValues: ['Dogs', 'Cats', 'Other']
}
});
However I'm not certain this is right again let me re-iterate that I do want the checkboxes to have the values 'Dogs', 'Cats', 'Other' and the text area when any of these are checked so that the end-user can enter more information about their pets.
What i'm unclear about is if I'm thinking about how this data's Schema could be or how this kind of information should be stored in the MongoDB.
I feel like there should also be a property for the TextArea input, would that be a good way to think about it?
For example (adding on to the example above):
Pet = new SimpleSchema({
petInstructions: {
type: String,
optional: true,
allowedValues: ['Dogs', 'Cats', 'Other'],
textArea: ""
}
});
Should I just leave another property/value for that particular content?
This is somewhat of a granular level question that understanding something simple like this would help clarify other questions I had.
Thank you.
This is how I would structure it. This is from the autoform instructions for how to validate a field against another field and how to conditionally require a value - https://github.com/aldeed/meteor-simple-schema#best-practice-code-examples
Pet = new SimpleSchema({
pet: {
type: Object,
optional: true
},
"pet.type": {
type: String,
optional: true,
allowedValues: ['Dogs', 'Cats', 'Other']
},
"pet.instructions": {
type: String,
optional: true,
custom: function(){
var petType = this.field('pet.type').value;
// if the user selects a pet type then instructions should be required
if (petType){
return "required";
};
}
}
});
In your old code:
Pet = new SimpleSchema({
petInstructions: {
type: String,
optional: true,
allowedValues: ['Dogs', 'Cats', 'Other'],
textArea: ""
}
});
You are literally stating that for the petInstructions key, you only allow the values to be Dogs, Cats, or Other, but you also want one of the keys to be textArea with some other data, which doesn't make too much sense. SimpleSchema wants you to define a specific data point for each key.
I have the following generic schema to represent different types of information.
var Record = new Schema (
{
type: {type: String}, // any string (foo, bar, foobar)
value: {type: String}, // any string value
o_id: {type:String}
}
);
Some of the records based on this schema have:
type="car"
value="ferrari" or
value="ford"
Some records have type "topspeed" with value "210" but they always share o_id (e.g. related "ferrari has this topspeed"). So if "ferrari has top speed 300", then both records have same o_id.
How can I make query to find "ferrari with topspeed 300" when I don't know o_id?
The only solution I found out is to select cars "ferrari" first and then with knowledge of all o_id for all "ferrari" use it to find topspeed.
In pseudocode:
Record.find({type:"car", value:"ferrari"}, function(err, docs)
{
var condition = [];// create array of all found o_id;
Record.find({type:"topspeed", value:"300"}...
}
I know that some merging or joining might not be possible, but what about some chaining these conditions to avoid recursion?
EDIT:
Better example:
Lets imagine I have a HTML document that contains DIV elements with certain id (o_id).
Now each div element can contain different type of microdata items (Car, Animal...).
Each microdata item has different properties ("topspeed", "numberOfLegs"...) based on the type (Car has a topspeed, animal numberOfLegs)
Each property has some value (310 kph, 4 legs)
Now I'm saving these microdata items to the database but in a general way, agnostic of the type and values they contain since the user can define custom schemas from Car, to Animal, to pretty much anything). For that I defined the Record schema: type consists of "itemtype_propertyname" and value is value of the property.
I would eventually like to query "Give me o_id(s) of all DIV elements that contain item Ferrari and item Dog" at the same time.
The reason for this general approach is to allow anyone the ability to define custom schema and corresponding parser that stores the values.
But I will have only one search engine to find all different schemas and value combinations that will treat all possible schemas as a single definition.
I think it'd be far better to combine all records that share an o_id into a single record. E.g.:
{
_id: ObjectId(...),
car: "ferarri",
topspeed: 300
}
Then you won't have this problem, and your schema will be more efficient both in speed and storage size. This is how MongoDB is intended to be used -- heterogenous data can be stored in a single collection, because MongoDB is schemaless. If you continue with your current design, then no, there's no way to avoid multiple round-trips to the database.