I have big json that I pass to stored procedure in Cosmos DB. I need to split the json document and create multiple documents. How do I loop to the json inside the stored procedure.
Example: Using Json below I need to create a separate document for each children and another one without Children nodes. A total of 3 documents. How can I loop and split it inside the stored procedure?
{
"id": "d93af7d3706e4f28882920366c017cd7",
"FamilyId": 989,
"LastName": "Andersen",
"Parents": [
{
"FamilyName": null,
"FirstName": "Thomas"
},
{
"FamilyName": null,
"FirstName": "Mary Kay"
}
],
"Children": [
{
"ChildId":001,
"FamilyName": null,
"FirstName": "Henriette Thaulow",
"Gender": "female",
"Grade": 5,
"Pets": [
{
"GivenName": "Fluffy"
}
]
},
{
"ChildId":002
"FamilyName": null,
"FirstName": "Maria Thaulow",
"Gender": "female",
"Grade": 12,
"Pets": [
{
"GivenName": "Grizzly"
}
]
}
],
"Address": {
"State": "WA",
"County": "King",
"City": "Seattle"
},
"IsRegistered": false
}
Take a look at this sample: https://github.com/Azure/azure-documentdb-js-server/blob/master/samples/stored-procedures/BulkImport.js. Note that currently stored procedure can run within context of one partition key (which is provided in request options to execute the sproc), i.e. if there are multiple PK values in your docs, the docs need to be grouped by PK value.
Actually , you can implement bulk import data via Stored Procedure.
Please refer to the code from official tutorial.
function bulkImport(docs) {
var collection = getContext().getCollection();
var collectionLink = collection.getSelfLink();
// The count of imported docs, also used as current doc index.
var count = 0;
// Validate input.
if (!docs) throw new Error("The array is undefined or null.");
var docsLength = docs.length;
if (docsLength == 0) {
getContext().getResponse().setBody(0);
}
// Call the create API to create a document.
tryCreate(docs[count], callback);
// Note that there are 2 exit conditions:
// 1) The createDocument request was not accepted.
// In this case the callback will not be called, we just call setBody and we are done.
// 2) The callback was called docs.length times.
// In this case all documents were created and we don’t need to call tryCreate anymore. Just call setBody and we are done.
function tryCreate(doc, callback) {
var isAccepted = collection.createDocument(collectionLink, doc, callback);
// If the request was accepted, callback will be called.
// Otherwise report current count back to the client,
// which will call the script again with remaining set of docs.
if (!isAccepted) getContext().getResponse().setBody(count);
}
// This is called when collection.createDocument is done in order to process the result.
function callback(err, doc, options) {
if (err) throw err;
// One more document has been inserted, increment the count.
count++;
if (count >= docsLength) {
// If we created all documents, we are done. Just set the response.
getContext().getResponse().setBody(count);
} else {
// Create next document.
tryCreate(docs[count], callback);
}
}
}
When stored procedure is executed from client, RequestOptions specify the partition key, stored procedure will run in context of this partition and cannot operate (e.g. create) on docs that have different partition key value.
What you can do is to execute the stored procedure from client for each partition key. For instance, if stored procedure is to bulk-create documents, you can group the docs by partition key and send each group (can be done in parallel) to the stored procedure providing partition key value in RequestOptions.
Hope it helps you.
Related
I am running a JSON server to create a simple login system. The data from the server looks like this:
{
"users": [
{
"name": "user1",
"password": "pass1",
"email": "useer1#user.com",
"id": 2,
"details": {
"first_name": "user1_1",
"last_name": "user1_2",
"gender": "male",
"pic": "",
"about": "fnbewhbdwie"
}
},
{
"name": "user2",
"password": "pass2",
"email": "user2#user.com",
"details": {
"first_name": "user2_1",
"last_name": "user2_2",
"gender": "male",
"pic": "",
"about": "cjkdbvcwvebxcnoewbvcu"
},
"id": 4
}
]
}
I have created a function to check if the name and password match from the user input, and if the condition is true I want to run another function that will return only the details of the object that have the name and password from the user input.
I have tried the find() method but it returns the details for the first object only
async getAcc() {
const {data} = await Axios
.get(`${BASE_URL}users`)
data.forEach(({name, password}) => {
if(this.nameInput.value === name && this.passInput.value === password){
showAcc();
}
else {
return false
}
}
)
function showAcc() {
let result = data.find(a => a.details)
console.log(result)
}
}
Putting aside the obvious problem with the user/pwd on the response, as mentioned on comments, I believe that what is wrong here is this:
First, your data is an array of objects.
So, by definition, 'find' will return the first entry that the lamba function successfully satisfies. In this case, you are passing the array into the find and saying, give me the first details object you find. It is doing the right thing.
I believe that your problem is on the forEach. Since you added name and pwd as parameters , it is only picking those properties for each item on the array. Try and change it to something like
.forEach(item => {,
which will send into the lamba function the complete object. Then, inside the method you will need to modify and use
if(this.nameInput.value === item.name
Finally, inside your if statement, you'll be able to even simplify your showAcc by sending the "details" property straight into the method like:
showAcc(item.details);
and
function showAcc(details) {
console.log(details);
}
Would be better if you had a sample we could edit , but I think this should work.
Good luck
I'd use the lodash filter method instead of find method.
Something like: _.filter(data, { 'name': 'user2', 'password': 'pass2' });
It should get you all the objects that match your criteria passed in the argument object as shown above.
WebDever may have pointed you to a better solution. I'll just point out my observation, which directly explains why you fail to print all the answers.
I don't know anything about any of the technology used, but it seems that the problem has to be the return value (or lack thereof) of the code block inside the forEach. Just showing the result, it appears, should in and of itself have no side effect in terms of stopping the iteration. So the key thing that happens when you first find a match is that you don't return 'false'. Up to that point, you've always been returning 'false'.
So my guess is that when you find a match, you're returning the result of the last expression rather than 'false', and that return value causes the iteration to stop. So change the code within the foreach to:
if(this.nameInput.value === name && this.passInput.value === password){
showAcc();
}
return false
I don't know if there is a word for this, guess there is, but right now I couldn't explain it better than "where array contains item(s) from array".
It might sound weird, but actually it not (I think), and I'm having a hard time figuring out how I can do this in Azure CosmosDB.
Here goes. I have a document like this (simplified):
{
"id": "2a62fcf4-988f-4ebe-aedc-fb0c664b85d8",
"Title": "Seks års fængsel for overgreb",
"ZipCodes": [
{
"Code": "6500",
"Name": "Vojens",
"FoundViaTerm": "Vojens"
},
{
"Code": "6400",
"Name": "Sønderborg",
"FoundViaTerm": "Sønderborg"
},
{
"Code": "6700",
"Name": "Esbjerg",
"FoundViaTerm": "Esbjerg"
}
],
"_rid": "k1sVAPf7SQAMAAAAAAAAAA==",
"_self": "dbs/k1sVAA==/colls/k1sVAPf7SQA=/docs/k1sVAPf7SQAMAAAAAAAAAA==/",
"_etag": "\"00001000-0000-0000-0000-5a14898e0000\"",
"_attachments": "attachments/",
"_ts": 1511295374
}
Ok, now I want to query documents like this and find all, where ZipCodes.Code is in a list of zipcodes, ex. ('6500', '2700').
I'm puzzle here...
I found the ARRAY_CONTAINS method and it works, if I only come in with one zipcode - my problem is I come with a list.
Hope somebody can help, thanks in advance.
Per my experience , expr in ARRAY_CONTAINS (arr_expr, expr [, bool_expr]) method is not supported list arguments.
According to your situation , I suggest you use UDF in Cosmos DB.
I created 3 sample documents as your description.
[
{
"id": "1",
"zip": [
{
"code": "1111"
},
{
"code": "2222"
}
]
},
{
"id": "2",
"zip": [
{
"code": "2222"
},
{
"code": "3333"
}
]
},
{
"id": "3",
"zip": [
{
"code": "4444"
},
{
"code": "1111"
},
{
"code": "2222"
}
]
}
]
Please refer to the snippet of UDF code as below :
function test(zipcode){
var arrayList = ["1111","2222"]
var ret = false ;
for(var i=0 ;i <zipcode.length;i++){
if(arrayList.indexOf(zipcode[i].code)){
ret= true;
}else{
ret = false;
break;
}
}
return ret;
}
You could select zip array (select c.zip from c) ,then loop the results and invoke the UDF above in your code with the zip[i] arguments.
Hope it helps you.
Just for summary:
Use the IN operator from Cosmos DB SQL APIs to query entry which is included in the list condition.
Like
SELECT * FROM c WHERE c.ZipCodes[0].Code IN ("6500", "6700")
Or
SELECT DISTINCT c FROM c JOIN zc IN c.ZipCodes WHERE zc.Code IN ("2720", "2610")
I would like to propose another solution to this problem.
Use EXISTS with ARRAY_CONTAINS in this way:
SELECT * FROM c
WHERE EXISTS
(SELECT VALUE z FROM z in c.ZipCodes WHERE ARRAY_CONTAINS(["6500","6700"], z))
You can do something like this:
For each item in ZipCodes, you get a zip and compare with the array of codes you are checking. This, IMHO, is much better than using UDF.
{
query: '
SELECT DISTINCT value r
FROM root r
JOIN zip IN r.zipCodes
WHERE ARRAY_CONTAINS(#zipIds, zip, true)
',
parameters: [{name: "#zipIds", value: zipIds}]
}
The last param of ARRAY_CONTAINS tells the function to accept partial matches.
Apart from the fact that using UDF looks as the easier option, i would not use UDFs in your query's filter, since it compromises the performance of your query. I faced the same problem in my work environment, where things are designed to use UDFs to help in the queries, but the reality is that most of the times we are doing queries by using single values, and using UDF will actually result on the query not using the index. So in that case if you need to validate multiple values in the array, depending on the volume of values you need to validate, you can always write something like ARRAY_CONTAINS(c, 1) or ARRAY_CONTAINS(c, 2) or ....
It doesn't look so elegant solution, but will ensure that it will use the index and will do the best performance in your query.
Just started with Firebase and loving it.
Was watching this video https://www.youtube.com/watch?v=Idu9EJPSxiY and wanted some help with a particular query. If anyone can help?
In the schema/structure below if i wanted to query all the events that a person named "David" attended, what would it be?
{
"users": {
"key1": {
"name": "David",
"age": 33,
"profileImg": "http/......"
}
"key2": {
"name": "John",
"age": 25,
"profileImg": "http/......"
}
},
"events": {
"fm": {
"name": "event1",
"date": 12332443456
}
},
"eventAttendees": {
"fm": {
"key1": "David",
"key2": "John"
}
}
}
I'm kinda stuck and would like some help. Thanks.
DB structure
You can try something like this (I'm using node.js firebase module to show the example)
firebase.database().ref('eventAttendees').once('value', function (eventAttendeesList) {
var events = []; //array to store the events
eventAttendeesList.forEach(function (eachEvent) { //forEach list of eventAttendees node, Eg: fm
eachEvent.forEach(function (data) { //forEach value of current eventAttendees, Eg: jey1: "David", key2: "John"
if (data.val() == "David") {
firebase.database().ref('events').child(eachEvent.key).once('value', function (snapshot) { // get the event data
events.push(snapshot) // add to array of events David attended
})
} //end of if
})// end of eachEvent.forEach
})// end of eventAttendeesList.forEach
return events;
})
I might have some syntax error somewhere, but you get the idea
First is to query for the list of data under the eventAttendees node ("fm") and loop through each value
In each of your value for the eventAttendees node there are your key1, key2 which we need to do another loop to get the data ("David", "John"). Here we will check if the data matches the person you are looking for
If it matches we will query firebase for the event data under the events node with the key that matches eachEvent.key and add it to our events array
I have an app that makes requests to 3 different API REST servers, gather and manipulate the info received into objects and uses mongoose's model to send the info different Mongo collections (one for each API REST).
My objective is to have a function operate on the different sets of info right before or at the same time (or right after) it is sent to the MongoDB.
Here's an example with 2 concrete objects that are being retrieved and sent to the db:
Object1:
{
"name": "X",
"a": 10,
"b": 9,
"n": 1502642712481,
"iname": "ix",
"_id": "59907f34040eb562d8d11d8c"
}
Object2:
{
"name": "Y",
"n": 1502642712552,
"iname": "ix",
"_id": "59907f34040eb562d8d11d8d",
"b": "10",
"a": "9",
}
These have their own models and are correctly being inserted into the MongoDB with a save and I want to compare the "a" and "b" keys of each objects (FYI: "iname" is a common key for both objects and "n" is a timestamp created at the moment the request is being returned).
The type of operation I want to make on 'a' and 'b' are relatively basic mathematics (at the moment): ax - ay and bx -by
Any idea on how I could do that ?
Right now the code to save each object to the db are nested within an infinite loop and looks like:
async function loopY() {
setTimeout(
async function () {
var list = ['Y'];
var data = await y(list); //<---- This is the call to wait whatever the request of Y will return
data.forEach((object) => {
if (object.name === 'Y') {
var iname = 'ix'
} else {
var iname = 'N/A' };
var tick = new Ymodel({
mk: object.mk,
name: object.name,
a: object.a,
b: object.b,
n: object.n,
iname: iname,
});
// tick.sendToCompare(); //<---- HERE I THOUGHT OF USING A FUNCTION FROM THE MODEL BUT NOT SURE HOW TO HANDLE THE DIVISION BETWEEN THE INFORMATION
tick.save(function(err, tick) {
if (err) return console.log(err);
console.log(`[SUCCESS]: ${tick.name} added to db!`);
});
});
loopY();
}
}, 1000); // <------- THE TIMEOUT HELPS ME MANAGE THE REQUESTS TO THE API SERVERS AND AVOID A "TOO MANY REQUESTS" ERROR.
};
loopY();
I have the exact same function for "X" (loopX()) with the slight difference that it adds an "sn" key to the object.
I hope this was clear enough and respected the rules... Also, thanks a lot in advance for your help!
I am writing an application in Node.js. I need to convert JavaScript objects returned from my database layer to JavaScript objects that I need to return from my REST service. I may need to add or remove certain properties or perhaps flatten out some structures. For example, here's a User object returned from the database layer:
{
"id": 1234,
"username": "jsmith",
"person": {
"first_name": "John",
"last_name": "Smith"
}
}
The object that I want to return from the REST layer is
{
"_links": {
"self": { "href": "/users/jsmith" },
},
"username": "jsmith",
"firstName": "John",
"lastName": "Smith"
}
Here are the changes:
"id" property is removed
"links" section has been added
"person" object has been flattened out to the parent object
"first_name" and "last_name" properties have beed converted from snake_case to camelCase
I could obviously hand-code each one of these transformations, but that would be very inefficient and error prone. Do you know of any framework or approach that would ease this pain?
P.S. I am essentially looking for something similar to the Java framework called dozer.
You will need a custom function to convert the object returned from the DB before pushing them to the RESTAPI.
An example:
var dbObject = db.query(...); //this is what you get from your DB query
//Here is your REST API object that you will send back
var restObject = (function(obj){
var link = {
self: {
href: "/users/" + obj.username
}
};
var username = obj.username;
var firstname = obj.person.first_name;
var lastname = obj.person.last_name;
return JSON.stringify({
_links: link,
username: username,
firstName: firstname,
lastName: lastname
})
})(dbObject);
Obviously, this can be cleaned up/customized to fit your code (e.g: your DB query might be asynchronous, so the restObject will need to reflect that). Also, asssuming your DB sends back a JSON object, you will need to account for that.