I need to do some additional filtering of the data I'm getting, so that I'm only getting the data that match certain criteria. I'm trying to find a combination of operators I can use to accomplish this via a functional programming approach. I'm thinking perhaps a combination of filter and reduce might work? Could I simply chain a combination of them together to get the result I'm looking for?
My data structure looks like this:
"data": [
{
"_id": "53545",
"services": [
{
"service": "service1",
"history": [
{
"_id": "6546",
"status": "stage1",
"completed": true
},
{
**"_id": "6547",
"status": "stage2",
"completed": false**
}
],
{
"service": "service2",
"history": [
{
"_id": "6743",
"status": "stage3",
"completed": false
},
{
"_id": "3742",
"status": "stage2",
"completed": true
}
]
},
{
"service": "service3",
"history": [
{
"_id": "6448",
"status": "stage4",
"completed": false
},
{
"_id": "4443",
"status": "stage1",
"completed": true
}
]
}
]
To be clear, here data, services and history are ALL arrays. And in the case of the above data, only one object within "history" should pass the tests, as there is only one object where "status:stage2" and "completed:false". BUT that means that, within the "data" array, this particular object, the one with the "_id": "53545" SHOULD pass, as the tests returned true for it. And ultimately, I'm wanting to return an array, from within the objects within the "data" array, that match like this one does.
Very specifically, in my case, there are always 3 services. Within each of those 3, there might be 1 or as many as 30 or more history objects. I need to look through all 3 "services", and find the history objects where the status is "stage2" and "completed:false". So this should return true if ANY of the three services have a history object with "status:stage2" and "completed:false".
I'm getting tripped up iterating over arrays within arrays.
I could get it directly in this manner:
let stage2Data = data.filter(data => data.services[0].history[1].status === 'stage2' && data.services[0].history[1].completed === false);
However, I obviously don't know what item number in these various arrays will contain the value I'm looking for. So what's the best way to tackle this? I suppose a series of for-of loops could work as well, but I'd like to use a more functional approach, and was thinking a combination of "filter" and "reduce", or perhaps "map" could work here. How would I only return the objects within the "data" array where ""status": "stage2" and "completed": false is true for at least one of the objects within the "history" array?
We've been going back and forth in the comments to try to get a valid data structure in your question. Let me give you a simple example of how you can make this much easier for everyone.
First, use the snippet button at the top of the editor box to create a snippet. It's the icon that looks like a typical "folded corner" document icon with <> inside it.
This will open up a popup box with places for HTML/CSS/JavaScript code. Ignore the HTML and CSS boxes and paste your data object into the JavaScript box.
Because it's not JavaScript "as is", wrap it inside an assignment statement to get a JavaScript variable that you can work with.
Then add a console.log statement at the end to display the data. Use the Run button to make sure the code actually parses and runs and displays the data correctly. Once you do that, it will be much easier for people to help you.
Here is a simplified example:
const input = {
"data": [
{
"one": "two"
},
{
"three": "four"
}
]
};
console.log( JSON.stringify( input, null, 4 ) );
Click the Run code snippet button above to see this example run. Notice how it prints out the same data structure that I used above.
If you do the same in your question, this will force you to clean up your data so that it is actually valid. Then you are giving people something they can work with.
One other note: don't be hung up on trying to do this in a "functional" manner. It's very likely that you will get more understandable code by creating an empty array with let output = [];, then using .forEach() on each of your nested arrays (or for...of loops if you are targeting the latest browsers), and using output.push() on each matching element.
To illustrate, assuming I understand your data structure correctly (and assuming you wrap it inside an input variable as in my simplified example) it might look something like this:
let output = [];
input.data.forEach( function( dataItem ) {
dataItem.services.forEach( function( service ) {
service.history.forEach( function( historyItem ) {
if( ! historyItem.completed && historyItem.status == "stage2" ) {
output.push( historyItem );
}
});
});
});
Or using for...of loops:
let output = [];
for( let dataItem of input.data ) {
for( let service of dataItem.services ) {
for( let historyItem of service.history ) {
if( ! historyItem.completed && historyItem.status == "stage2" ) {
output.push( historyItem );
}
}
}
}
Both of these are untested, of course. And either way it's not fancy, but it seems pretty easy to understand.
Depending on how you want the result data, here's a start for you. You can brute-force this with a few nested filters:
result = data.filter((datum) => {
return datum.services.filter((service) => {
return service.history.filter((h) => {
return h.completed && h.status === "stage2"
}).length > 0;
}).length > 0;
});
tl;dr - Filter the main list by any datum that has services that have at least one history record that was completed in stage 2.
Working fiddle: http://jsbin.com/jozujezidu/edit?js,console,output
If you want your resultant array to only have the history records that pass the filter, you'll have to modify the code. Plus, this is an On^3, so if this data can be enormous, you should probably modify the approach. But this should get you going.
Don't worry about being militant functional. If you can come to a working solution that people can understand and maintain, then that's a good start.
A filter operation will be based on a predicate matching your values.
If you are wanting a list of the service objects try this approach:
let result = data[0].services.filter((item) => item.history.some((h) => h.status=='stage2' && h.completed==false));
Related
I've already tried to find a solution on stack, but I didn't found a possible reply, so I decided to open a topic to ask:
Let's say we have 2 arrays: one containing "keys" and another one containing "values"
Example:
keys = [CO2, Blood, General, AnotherKey, ... ]
values = [[2,5,4,6],[4,5,6],[1,3,34.5,43.4],[... [
I have to create a Json with a specific structure like:
[{
name: 'CO2',
data: [2,5,4,6]
}, {
name: 'Blood',
data: [4,5,6]
}, {
name: 'General',
data: [1,3,34.5,43.4]
}, {
...
},
}]
I've tried to make some test bymyself, like concatenate strings and then encode it as json, but I don't think is the correct path to follow and a good implementation of it ... I've also take a look on JSON.PARSE, JSON.stringify, but I never arrived at good solution so... I am asking if someone know the correct way to implements it!
EDIT:
In reality, i didn't find a solution since "name" and "data" are no strings but object
Here's one way to get your desired output:
keys = ["CO2", "Blood", "General", "AnotherKey"]
values = [[2,5,4,6],[4,5,6],[1,3,34.5,43.4],[0] ]
const output = keys.map((x, i) => {
return {"name": x, "data": values[i]}
})
console.log(output)
However, since you're literally constructing key/value pairs, you should consider whether an object might be a better data format to output:
keys = ["CO2", "Blood", "General", "AnotherKey"]
values = [[2,5,4,6],[4,5,6],[1,3,34.5,43.4],[0] ]
const output = {}
for (let i=0; i<keys.length; i++) {
output[keys[i]] = values[i]
}
console.log(output)
With this data structure you can easily get the data for any keyword (e.g. output.CO2). With your array structure you would need to iterate over the array every time you wanted to find something in it.
(Note: The reason you weren't getting anywhere useful by searching for JSON methods is that nothing in your question has anything to do with JSON; you're just trying to transform some data from one format to another. JSON is a string representation of a data object.)
I need suggestions on improving a UI component that receives an object that contains all data types i.e. string, int, boolean, arrays and object and displays all the data types.
example of the object. PS: this object is also provide in the codesandbox link.
const data = {
foo1: {},
foo9: true,
foo3: "some random string",
foo4: "some random string",
created_at: {
$date: 1637368143.618
},
sources: {
foo1: {
first_date: {
$date: 1637368143.618
}
}
},
download: {
status: "pending"
},
foo8: "some random string",
foo5: "some random string",
foo7: ["sms"],
foo10: "some random string",
foo11: {
bar5: 0,
bar3: null,
bar1: null,
bar2: 0
},
foo28: ["some random string", "some random string2"],
foo19: "some random string"
};
currently i loop through the object and display the data in a card.
Object.entries(objSorted).map(([key, value])
but first the object is sorted to the info in the following order of string, int or boolean, array and object.
this is the link to the codesandbox. here
PS: This is more of a suggestion giving than a question, i don't have any problem with the code. i only want to know if the way am currently displaying this data is good and user friendly(i don't think so that's why i am asking). and if you think there is a better way you can comment or edit the codesandbox or create a new codesandbox.
Your intuition is correct! We can make this a little easier to understand by decomposing everything in a way similar to what you mentioned. What we really want is better separation of concerns. All of this hand waves details away, but the concepts will be what you take with you.
You have data.
Sort the data.
Loop through the outputs from step 2 and render the cards.
profit?
Step 1/2. You have data. Why not digest this before rendering? There are tons of ways of doing this, but for the sake of readability lets create a hook and be conscience about our downstream consumers. We can make our ObjectExtract component way simpler if it has well formatted data... And we can memo the results so that our digested data is cached and therefor doesn't recompute ever render.
const useCardData = (data) => {
return useMemo(() => {
const entries = Object.entries(data)
.map(([key, value]) => {
let type = "unknown";
if (Array.isArray(key)) {
type = "array";
} else if (typeof value === "boolean") {
type = "boolean";
} else if { ... };
return { type, value, key, priority: getTypeSortId(type) };
})
.sort((x, y) => x.priority - y.priority)
return entries;
}, [data]);
};
We digest our data as much as we can. The key is needed later because using index as a key is almost never what you want since when the order changes, react uses keys to track how rows should be shifted. Numeric indexes from arrays really don't describe the uniqueness of the record. Imagine sorting the records, index 1 could mean completely different things before and after sort, and your data may or may not show up correctly. We use the object property name as our unique key, since objects do a good job of not allowing duplicate keys.
Step 3. Let's use the hook and render the data.
const CardRenderer = ({ data }) => {
const cards = useCardData(data);
return (
<>
{cards.map((thing) => {
switch (thing.type) {
case "boolean":
return <BooleanCard key={thing.key} data={thing} />;
case "string":
return <StringCard key={thing.key} data={thing} />;
case "...":
return ...;
}
}}
</>
)
};
Since our data is well formed we can simply loop over it with a switch. Instead of defining all cases and their results in one component, notice how we just create smaller card components to handle each specific case. Separation of concern. The boolean card cares about what booleans look like ect. This makes it SUPER easy to test since you can test each card individually and then test the output of our CardRenderer to make sure its output is reasonable.
Step 4. profit. We broke everything down to components. Each piece only cares about a specific responsibility and that makes it easy to glue things together. It makes it composable. We can test each piece by itself and make sure its doing the right thing. We can make some really complex things like this while keeping the complexity of each individual hidden away.
What I described is SOLID. The examples in the link are in PHP but I think you'll get the gist. We can and should use the same patterns in react to build really cool things while managing the every growing complexity of cooler things.
I have not been able to find a solution for this on StackoverlFlow and i have been working on this issue for some time now. Its a bit difficult so please let me know if i should provide more information.
My problem:
I have a JSON object of Lands, each land has an ID and a Blocks array associated with it and that blocks array has block with ID's too.
Preview:
var Lands = [{
'LandId':'123',
'something1':'qwerty',
'something2':'qwerty',
'Blocks': [
{
'id':'456',
'LandId':'123'
},
{
'BlockId':'789',
'LandId':'123'
}
]
},
...More Land Objects];
Note: The data is not setup the way i would have done it but this was done along time ago and i have to work with what i have for right now.
I am trying to write a lodash function that will take the blockId's that i have and match them and return the landId from the Blocks.
so the end result would be a list of LandId's that where returned from the Blocks.
I was using something like this and it was returning no results:
selectedLand = function(Lands, landIDs){
return _.filter(Lands, function(land){
return land === landIDs[index];
});
};
I know i am going about this the wrong way and would like to know the appropriate way to approach this and solve it. any help is much appreciated.
selectedLand = function(Lands, landIDs){
return _.filter(Lands, function(land){
return land === landIDs[index];
});
};
Note that index lacks any definition in this scope, so this function will essentially never return true, barring a lucky(?) accident with an externally defined index. And anytime you call _.filter(someArr,function(){return false;}) you'll get []. Undefined index aside, this does strict comparison of a land object against (maybe) a string in landIDs
I'm a bit unclear on the exact requirements of your selection, so you can tailor this filter to suit your needs. The function filters the lands array by checking if the .blocks array property has some values where the .landID property is included in the landsID array.
In closing, if you want to make the most out of lodash (or my favorite, ramda.js) I suggest you sit down and read the docs. Sounds deathly boring, but 75% of the battle with data transforms is knowing what's in your toolbox. Note how the English description of the process matches almost exactly with the example code (filter, some, includes).
var Lands = [{
'LandId': '123',
'something1': 'qwerty',
'something2': 'qwerty',
'Blocks': [{
'id': '456',
'LandId': '123'
}, {
'BlockId': '789',
'LandId': '123'
}]
}
];
// assuming lands is like the above example
// and landIDs is an array of strings
var selectLandsWithBlocks = function(lands, landIDs) {
return _.filter(lands, function(land) {
var blocks = land.Blocks;
var blockHasALandId = function(block) {
return _.includes(landIDs,block.LandId);
};
return _.some(blocks, blockHasALandId);
});
};
console.log(selectLandsWithBlocks(Lands,[]));
console.log(selectLandsWithBlocks(Lands,['mittens']));
console.log(selectLandsWithBlocks(Lands,['123']));
<script src="//cdnjs.cloudflare.com/ajax/libs/lodash.js/4.13.1/lodash.min.js"></script>
I am trying to update a value in the nested array but can't get it to work.
My object is like this
{
"_id": {
"$oid": "1"
},
"array1": [
{
"_id": "12",
"array2": [
{
"_id": "123",
"answeredBy": [], // need to push "success"
},
{
"_id": "124",
"answeredBy": [],
}
],
}
]
}
I need to push a value to "answeredBy" array.
In the below example, I tried pushing "success" string to the "answeredBy" array of the "123 _id" object but it does not work.
callback = function(err,value){
if(err){
res.send(err);
}else{
res.send(value);
}
};
conditions = {
"_id": 1,
"array1._id": 12,
"array2._id": 123
};
updates = {
$push: {
"array2.$.answeredBy": "success"
}
};
options = {
upsert: true
};
Model.update(conditions, updates, options, callback);
I found this link, but its answer only says I should use object like structure instead of array's. This cannot be applied in my situation. I really need my object to be nested in arrays
It would be great if you can help me out here. I've been spending hours to figure this out.
Thank you in advance!
General Scope and Explanation
There are a few things wrong with what you are doing here. Firstly your query conditions. You are referring to several _id values where you should not need to, and at least one of which is not on the top level.
In order to get into a "nested" value and also presuming that _id value is unique and would not appear in any other document, you query form should be like this:
Model.update(
{ "array1.array2._id": "123" },
{ "$push": { "array1.0.array2.$.answeredBy": "success" } },
function(err,numAffected) {
// something with the result in here
}
);
Now that would actually work, but really it is only a fluke that it does as there are very good reasons why it should not work for you.
The important reading is in the official documentation for the positional $ operator under the subject of "Nested Arrays". What this says is:
The positional $ operator cannot be used for queries which traverse more than one array, such as queries that traverse arrays nested within other arrays, because the replacement for the $ placeholder is a single value
Specifically what that means is the element that will be matched and returned in the positional placeholder is the value of the index from the first matching array. This means in your case the matching index on the "top" level array.
So if you look at the query notation as shown, we have "hardcoded" the first ( or 0 index ) position in the top level array, and it just so happens that the matching element within "array2" is also the zero index entry.
To demonstrate this you can change the matching _id value to "124" and the result will $push an new entry onto the element with _id "123" as they are both in the zero index entry of "array1" and that is the value returned to the placeholder.
So that is the general problem with nesting arrays. You could remove one of the levels and you would still be able to $push to the correct element in your "top" array, but there would still be multiple levels.
Try to avoid nesting arrays as you will run into update problems as is shown.
The general case is to "flatten" the things you "think" are "levels" and actually make theses "attributes" on the final detail items. For example, the "flattened" form of the structure in the question should be something like:
{
"answers": [
{ "by": "success", "type2": "123", "type1": "12" }
]
}
Or even when accepting the inner array is $push only, and never updated:
{
"array": [
{ "type1": "12", "type2": "123", "answeredBy": ["success"] },
{ "type1": "12", "type2": "124", "answeredBy": [] }
]
}
Which both lend themselves to atomic updates within the scope of the positional $ operator
MongoDB 3.6 and Above
From MongoDB 3.6 there are new features available to work with nested arrays. This uses the positional filtered $[<identifier>] syntax in order to match the specific elements and apply different conditions through arrayFilters in the update statement:
Model.update(
{
"_id": 1,
"array1": {
"$elemMatch": {
"_id": "12","array2._id": "123"
}
}
},
{
"$push": { "array1.$[outer].array2.$[inner].answeredBy": "success" }
},
{
"arrayFilters": [{ "outer._id": "12" },{ "inner._id": "123" }]
}
)
The "arrayFilters" as passed to the options for .update() or even
.updateOne(), .updateMany(), .findOneAndUpdate() or .bulkWrite() method specifies the conditions to match on the identifier given in the update statement. Any elements that match the condition given will be updated.
Because the structure is "nested", we actually use "multiple filters" as is specified with an "array" of filter definitions as shown. The marked "identifier" is used in matching against the positional filtered $[<identifier>] syntax actually used in the update block of the statement. In this case inner and outer are the identifiers used for each condition as specified with the nested chain.
This new expansion makes the update of nested array content possible, but it does not really help with the practicality of "querying" such data, so the same caveats apply as explained earlier.
You typically really "mean" to express as "attributes", even if your brain initially thinks "nesting", it's just usually a reaction to how you believe the "previous relational parts" come together. In reality you really need more denormalization.
Also see How to Update Multiple Array Elements in mongodb, since these new update operators actually match and update "multiple array elements" rather than just the first, which has been the previous action of positional updates.
NOTE Somewhat ironically, since this is specified in the "options" argument for .update() and like methods, the syntax is generally compatible with all recent release driver versions.
However this is not true of the mongo shell, since the way the method is implemented there ( "ironically for backward compatibility" ) the arrayFilters argument is not recognized and removed by an internal method that parses the options in order to deliver "backward compatibility" with prior MongoDB server versions and a "legacy" .update() API call syntax.
So if you want to use the command in the mongo shell or other "shell based" products ( notably Robo 3T ) you need a latest version from either the development branch or production release as of 3.6 or greater.
See also positional all $[] which also updates "multiple array elements" but without applying to specified conditions and applies to all elements in the array where that is the desired action.
I know this is a very old question, but I just struggled with this problem myself, and found, what I believe to be, a better answer.
A way to solve this problem is to use Sub-Documents. This is done by nesting schemas within your schemas
MainSchema = new mongoose.Schema({
array1: [Array1Schema]
})
Array1Schema = new mongoose.Schema({
array2: [Array2Schema]
})
Array2Schema = new mongoose.Schema({
answeredBy": [...]
})
This way the object will look like the one you show, but now each array are filled with sub-documents. This makes it possible to dot your way into the sub-document you want. Instead of using a .update you then use a .find or .findOne to get the document you want to update.
Main.findOne((
{
_id: 1
}
)
.exec(
function(err, result){
result.array1.id(12).array2.id(123).answeredBy.push('success')
result.save(function(err){
console.log(result)
});
}
)
Haven't used the .push() function this way myself, so the syntax might not be right, but I have used both .set() and .remove(), and both works perfectly fine.
I'm trying to generate a tree structure in JavaScript from a flat array. This would usually be a fairly straightforward proposition - simply retain a 'stack' array with references to ancestor objects of the current working scope ordered by nesting depth - push a new element onto the stack when entering another nested level, and pop it off when leaving one, replacing the current working element with the object referenced by the (new) last array item.
Unfortunately this requires the capability to pass-by-reference, which JavaScript doesn't have (well, doesn't have in any meaningful way that I know how I could use for this problem.)
To give a bit of background, I'm trying to turn an arbitrarily long/complicated string containing nested XML-style (but not XML, so an XML parser can't be used instead) tokens into a structure similar to the one below:
Expected Input:
[
"<token>",
"<my non compliant token>",
"some text at this level",
"<some other token>",
"some more text",
"<yet another token>",
"more text",
"</yet another token>",
"blah!",
"</some other token>",
"</token>",
"more text"
]
Expected Output
[
{
"token": "<token>",
"children": [
{
"token": "<my non compliant token>",
"children": [
"some text at this level",
{
"token": "<some other token>",
"children": [
"some more text",
{
"token": "<yet another token>",
"children": [ "more text" ]
},
"blah!"
]
}
]
}
]
},
"more text"
]
To clarify - I'm not after an entire algorithm (but I'd be interested if you want to provide your implementation) - just a good method for maintaining current position in the outputted tree (or an entirely different/better way of generating tree objects!) Don't get too caught up on how the tokens work - they're not XML and for the purpose of the exercise could be formatted entirely differently.
Any input would be greatly appreciated!
Your strings look easy to parse. I think I would do something like this:
var stack = [];
var array = [];
for (var i in strings) {
var s = strings[i];
if (s.indexOf("</") == 0) {
array = stack.pop();
} else if (s.indexOf("<") == 0) {
var obj = {token: s, children: []};
array.push(obj);
stack.push(array);
array = obj.children;
} else {
array.push(s);
}
}
Idea #1
Here's an answer you probably weren't anticipating.
Looking at your expect output, I was wondering if it's easiest to just generate JSON and then eval it when you're done. No references at all.
When going through your flat array, you basically have three operations:
You add more data to the current object
You close off the current object
You create a new child object
You can do all three of those fairly easily by just appending the appropriate text onto a JSON string you're building as you iterate through your source array to literally just generate the text you show in your expected output. When done, run that JSON string through eval. You may need a few safety checks to verify that each array and object is closed properly if there are errors in the input, but it should work fine.
Idea #2
You can still use your stack array. I'm not sure exactly why you need to pass by reference, but you can just pass an index into the array around and have everyone modify the master copy of the array that way via index. Using local functions, the master array can be a common data value that is local to your main function, but essentially global to all your sub-functions so they can all shared access to it.
This would look something like this:
function ParseRawData(rawData)
{
var parentScopeArray = []; // main parent scope of objects
function processTag(x)
{
// you can access parentScopeArray directly here and
// and be accessing it by reference
}
// other code or local functions here
}
Idea #3
If you want to pass the array into a function and have the master copy modified (perhaps the reason you're thinking of pass by reference), the javascript design pattern is to pass the array in and return a modified array, replacing the entire original array with the modified one that is returned.