I'm using Axios / promises to make AJAX requests, but the structure is a bit confusing. Basically I want this to happen:
Get masterlist.xml.
Parse the master list to get an array of category list URLs.
Get each category.xml.
Parse each category list to get an array of manifest URLs.
Get each manifest.xml and parse data.
Example structure:
masterlist.xml
<master>
<category>action</category>
<category>romance</category>
</master>
Category XMLs (e.g. action.xml and romance.xml)
<manifestUrls>
<manifest>foo/bar/manifest.xml</manifest>
<manifest>alice/bob/manifest.xml</manifest>
<manifest>hello/world/manifest.xml</manifest>
</manifestUrls>
Manifest XMLs
<manifest>
<data>Blah blah blah</data>
<manifest>
I'm a bit stuck on how to structure this as an Axios request with then. I'd like to have three functions, getMaster(), getCategory(url), and getManifest(url) that are preferably independent (e.g. getMaster doesn't have to call getCategory directly), but it seems like it might be necessary.
How would this be structured in Axios?
One of the main benefits of promises is that they allow you to easily avoid interdependence between your methods.
Here is a rough outline of how you could do this.
// put it all together
getMaster()
.then(parseMaster)
.then(function (categories) {
return Promise.all(categories.map(getAndParseCategory));
})
.then(flatten) // the previous then() resolves to an array-of-arrays
.then(function (manifestUrls) {
return Promise.all(manifestUrls.map(getManifest));
})
.then(function (manifests) {
// manifests is an array of all manifests
});
// Examples of what each of the functions used above would do
function getMaster() {
return axios.get('masterUrl')
.then(function (response) { return response.data; });
}
function parseMaster(masterContent) {
// parse and return an array of categories
}
function getCategory(name) {
var url = // ... build the URL based on the name
return axios.get(url)
.then(function (response) { return response.data; });
}
function parseCategory(categoryContent) {
// parse and return an array of URLs synchronously for one category
}
function getAndParseCategory(name) {
return getCategory(name).then(parseCategory);
}
function getManifest(url) {
return axios.get(url)
.then(function (response) { return response.data; });
}
function flatten(arrayOfArrays) {
return [].concat.apply([], arrayOfArrays);
}
If you're using Bluebird or something else that gives promises a .map() method, then you can tidy that pipeline up a bit:
// using Promise.resolve() at the beginning to ensure
// the chain is based of the desired kind of promise
Promise.resolve()
.then(getMaster)
.then(parseMaster)
.map(getCategory)
.map(parseCategory)
.then(flatten) // previous line resolves to an array-of-arrays
.map(getManifest)
.then(function (manifests) {
// manifests is an array of all manifests
});
Of course, you could also define your own .map method if you don't want to import a whole third party promise library:
if (!Promise.prototype.map) {
Promise.prototype.map = function (func) {
return this.then(function (result) {
return Promise.all(result.map(func));
});
};
}
Edit: To respond to your question in the comments below. If you wanted to pass the category text along so that it could be included in the manifest URLs, I think a clean way to do this would be to include that in the data returned from getCategory() so that parseCategory can make use of it. Everything else could stay the same.
Example:
function getCategory(name) {
var url = // ... build the URL based on the name
return axios.get(url)
.then(function (response) {
return {
name: name,
data: response.data
};
});
}
function parseCategory(categoryContent) {
var urls = // parse URLs from categoryContent.data
return urls.map(function (url) {
return categoryContent.name + '/' + url;
});
}
Related
Consider the following exerpt from this example in the InfluxDB documentation:
const fluxObserver = {
next(row, tableMeta) {
const o = tableMeta.toObject(row)
console.log(
`${o._time} ${o._measurement} in ${o.region} (${o.sensor_id}): ${o._field}=${o._value}`
)
},
error(error) {
console.error(error)
console.log('\nFinished ERROR')
},
complete() {
console.log('\nFinished SUCCESS')
}
}
/** Execute a query and receive line table metadata and rows. */
queryApi.queryRows(fluxQuery, fluxObserver)
The fluxObserver defines a sequence of operations at different stages of execution of the query, all of which are just console.log calls. I can imagine that people want to do something similar in a way that is actually useful, like for instance return an array that contains the result of the query, instead of logging something to a console. My problem is that queryApi.queryRows does not actually return anything, so it is not possible to do something like:
...
complete() {
console.log('\nFinished SUCCESS')
return data;
}
}
/** Execute a query and receive line table metadata and rows. */
var result = queryApi.queryRows(fluxQuery, fluxObserver);
return result;
So how can I return the data collected by the fluxObserver?
Via a microservice, I retrieve several packages of JSON data and spit them out onto a Vue.js-driven page. The data looks something like this:
{"data":{"getcompanies":
[
{"id":6,"name":"Arena","address":"12 Baker Street","zip":"15090"},
{"id":7,"name":"McMillan","address":null,"zip":"15090"},
{"id":8,"name":"Ball","address":"342 Farm Road","zip":"15090"}
]
}}
{"data":{"getusers":
[{"id":22,"name":"Fred","address":"Parmesean Street","zip":"15090"},
{"id":24,"name":"George","address":"Loopy Lane","zip":"15090"},
{"id":25,"name":"Lucy","address":"Farm Road","zip":"15090"}]}}
{"data":{"getdevices":
[{"id":2,"name":"device type 1"},
{"id":4,"name":"device type 2"},
{"id":5,"name":"device type 3"}]}}
...and I successfully grab them individually via code like this:
getCompanies() {
this.sendMicroServiceRequest({
method: 'GET',
url: `api/authenticated/function/getcompanies`
})
.then((response) => {
if(response.data) {
this.dataCompanies = response.data.getcompanies
} else {
console.error(response)
}
}).catch(console.error)
}
...with getUsers() and getDevices() looking respectively the same. getCompanies() returns:
[{"id":6,"name":"Arena","address":"12 Baker Street","zip":"15090"},
{"id":7,"name":"McMillan","address":null,"zip":"15090"},
{"id":8,"name":"Ball","address":"342 Farm Road","zip":"15090"}]
...which I relay to the Vue template in a table, and this works just fine and dandy.
But this is obviously going to get unwieldy if I need to add more microservice calls down the road.
What I'm looking for is an elegant way to jump past the response.data.*whatever* and get to those id-records with a re-useable call, but I'm having trouble getting there. response.data[0] doesn't work, and mapping down to the stuff I need either comes back undefined, or in bits of array. And filtering for response.data[0].id to return just the rows with ids keeps coming back undefined.
My last attempt (see below) to access the data does work, but looks like it comes back as individual array elements. I'd rather not - if possible - rebuild an array into a JSON structure. I keep thinking I should be able to just step past the next level regardless of what it's called, and grab whatever is there in one chunk, as if I read response.data.getcompanies directly, but not caring what 'getcompanies' is, or needing to reference it by name:
// the call
this.dataCompanies = this.getFullData('companies')
getFullData(who) {
this.sendMicroServiceRequest({
method: 'GET',
url: 'api/authenticated/function/get' + who,
})
.then((response) => {
if(response) {
// attempt 1 to get chunk below 'getcompanies'
Object.keys(response.data).forEach(function(prop) {
console.log(response.data[prop])
})
// attempt 2
// for (const prop in response.data) {
// console.log(response.data[prop])
// }
let output = response.data[prop] // erroneously thinking this is in one object
return output
} else {
console.error(response)
}
}).catch(console.error)
}
...outputs:
(63) [{…}, {…}, {…}] <-- *there are 63 of these records, I'm just showing the first few...*
0: {"id":6,"name":"Arena","address":"12 Baker Street","zip":"15090"}
1: {"id":7,"name":"McMillan","address":null,"zip":"15090"},
2: {"id":8,"name":"Ball","address":"342 Farm Road","zip":"15090"}...
Oh, and the return above comes back 'undefined' for some reason that eludes me at 3AM. >.<
It's one of those things where I think I am close, but not quite. Any tips, hints, or pokes in the right direction are greatly appreciated.
I feel it's better to be explicit about accessing the object. Seems like the object key is consistent with the name of the microservice function? If so:
getData(functionName) {
return this.sendMicroServiceRequest({
method: 'GET',
url: "api/authenticated/function/" + functionName
})
.then( response => response.data[functionName] )
}
getCompanies(){
this.getData("getcompanies").then(companies => {
this.dataCompanies = companies
})
}
let arrResponse = {data: ['x']};
let objResponse = {data: {getcompanies: 'x'}};
console.log(arrResponse.data[0]);
console.log(Object.values(objResponse.data)[0]);
response.data[0] would work if data was an array. To get the first-and-only element of an object, use Object.values(response.data)[0] instead. Object.values converts an object to an array of its values.
Its counterparts Object.keys and Object.entries likewise return arrays of keys and key-value tuples respectively.
Note, order isn't guaranteed in objects, so this is only predictable in your situation because data has exactly a single key & value. Otherwise, you'd have to iterate the entry tuples and search for the desired entry.
firstValue
Let's begin with a generic function, firstValue. It will get the first value of an object, if present, otherwise it will throw an error -
const x = { something: "foo" }
const y = {}
const firstValue = t =>
{ const v = Object.values(t)
if (v.length)
return v[0]
else
throw Error("empty data")
}
console.log(firstValue(x)) // "foo"
console.log(firstValue(y)) // Error: empty data
getData
Now write a generic getData. We chain our firstValue function on the end, and be careful not to add a console.log or .catch here; that is a choice for the caller to decide -
getData(url) {
return this
.sendMicroServiceRequest({ method: "GET", url })
.then(response => {
if (response.data)
return response.data
else
return Promise.reject(response)
})
.then(firstValue)
}
Now we write getCompanies, getUsers, etc -
getCompanies() {
return getData("api/authenticated/function/getcompanies")
}
getUsers() {
return getData("api/authenticated/function/getusers")
}
//...
async and await
Maybe you could spruce up getData with async and await -
async getData(url) {
const response =
await this.sendMicroServiceRequest({ method: "GET", url })
return response.data
? firstValue(response.data)
: Promise.reject(response)
}
power of generics demonstrated
We might even suggest that these get* functions are no longer needed -
async getAll() {
return {
companies:
await getData("api/authenticated/function/getcompanies"),
users:
await getData("api/authenticated/function/getusers"),
devices:
await getData("api/authenticated/function/getdevices"),
// ...
}
}
Above we used three await getData(...) requests which happen in serial order. Perhaps you want all of these requests to run in parallel. Below we will show how to do that -
async getAll() {
const requests = [
getData("api/authenticated/function/getcompanies"),
getData("api/authenticated/function/getusers"),
getData("api/authenticated/function/getdevices")
]
const [companies, users, devices] = Promise.all(requests)
return { companies, users, devices }
}
error handling
Finally, error handling is reserved for the caller and should not be attempted within our generic functions -
this.getAll()
.then(data => this.render(data)) // some Vue template
.catch(console.error)
I am reading a JSON object and looping through each item. I am first checking to see if the item already exists in the database and if so I want to log a message. If it doesn't already exist I want to add it.
This is working correctly however, I would like to add a callback or finish the process with process.exit();
Because the mongoose calls are asynchronous I can't put it at the end of the for loop because they haven't finished.
Whats the best way I should handle this?
function storeBeer(data) {
data.forEach((beer) => {
let beerModel = new Beer({
beer_id: beer.id,
name: beer.name,
image_url: beer.image_url
});
Beer.findOne({
'name': beer.name
}).then(function (result) {
if (result) {
console.log(`Duplicate ${result.name}`)
} else {
beerModel.save(function (err, result) {
console.log(`Saved: ${result.name}`)
});
}
});
});
}
Is there anything I should read up on to help solve this?
One means of managing asynchronous resources is through Promise objects, which are first-class objects in Javascript. This means that they can be organized in Arrays and operated on like any other object. So, assuming you want to store these beers in parallel like the question implies, you can do something like the following:
function storeBeer(data) {
// This creates an array of Promise objects, which can be
// executed in parallel.
const promises = data.map((beer) => {
let beerModel = new Beer({
beer_id: beer.id,
name: beer.name,
image_url: beer.image_url
});
return Beer.findOne({
'name': beer.name
}).then(function (result) {
if (result) {
console.log(`Duplicate ${result.name}`)
} else {
beerModel.save(function (err, result) {
console.log(`Saved: ${result.name}`)
});
}
});
);
});
return Promise.all(promises);
}
Don't forget that the storeBeer function is now asynchronous, and will need to be utilized either through a Promise chain or through async/await.
For example, to add process exit afterwards:
async function main() {
const beers = [ ... ];
await storeBeer(beer);
process.exit(0);
}
You can also modify the above example to invoke the storeBeer function within a try / catch block to exit with a different error code based on any thrown errors.
So I have a situation where I need to build an Object, then inject some items into that Object, and then make an API call and also inject the API's response into that Object. It's a complex Object so we'll just assume that's alright, but anyway, I think that I need to do these 3 things synchronously and using array.forEach means they run asynchronously.
I hope my example is simple enough to understand, but basically I do 3 things:
Create an empty Array of Classes/Classrooms
Loop through an Array of Class IDs and create an Object for each Class
Loop through an Array of Students and push them into a students Array inside the Class Object
Loop through an Array of Options and push them into an options array inside the Class Object
Finally, I have something that could look like this for each Class:
{
class_id: "abc123",
students: [{}, {}, {}],
options: [{}, {}, {}]
}
And finally, here is my code:
// Create Array of Class Objects
var classes = [];
function processArray(array, func) {
return $q(function(resolve, reject) {
array.forEach(function(item, index) {
func(item);
if (index === (array.length - 1)) resolve();
})
})
}
// Create Courier Objects
processArray(classIds, function(id) {
classes.push({class_id: id, students: [], options: []});
}).then(function(response) {
// Inject Students into each Class
processArray(students, function(students) {
_.find(classes, {'class_id': student.class_id}).students.push(student);
}).then(function(response) {
// Inject classOptions into each Class
processArray(classOptions, function(classOption) {
_.find(classes, {'class_id': classOption.class_id}).classOptions.push(classOption);
}).then(function(response) {
// Print the classes
console.log(classes);
})
})
});
I've create a function which does it synchronously but I'd like to know if anyone can think of a much, much cleaner and more efficient way of doing the above. It seems extremely hacky, and maybe I don't even need to do it synchronously if I arranged my functions correctly.
Working with Promise Based APIs that Return Arrays
The processArray function returns a promise that resolves to null.
//WRONG
//Returns a promise that resolves to null
//
function processArray(array, func) {
return $q(function(resolve, reject) {
array.forEach(function(item, index) {
func(item);
if (index === (array.length - 1)) resolve();
})
})
}
To return a promise that resolves to an array, use $q.all.
//RIGHT
//Returns a promise that resolves to an array
//
function processArray(array, func) {
var promiseArray = [];
array.forEach(function(item, index) {
promiseArray.push($q.when(func(item));
});
return $q.all(promiseArray);
}
In either case whether func(item) returns either a value or a promise, $q.when will return a promise.
Be aware that $q.all is not resilient. It will resolve fulfilled with an array of values or it will resolve rejected with the first error.
processArray(array, func)
.then( function onFulfilled(dataList) {
//resolves with an array
$scope.dataList = dataList;
}).catch( function onRejected(firstError) {
console.log(firstError);
});
For more information, see AngularJS $q Service API Reference.
you are using nested promises. These suck. One major advantage of promises is to avoid the 'callback nightmare' of deep nesting.
Use this syntax ...
promise1.then(function(response1) {
// do something with response 1 (and possibly create promise2 here based on response1 if required)
return promise2
}).then(function(response2) {
// do something with response 2
return promise3
}).then(function(response3) {
// do something with response 3
// do something with promise 3???
}).catch(function(errorResponse) {
// will trigger this on a failure in any of the above blocks
// do something with error Response
});
In my Angular project i have a Promise that returns an array of Products:
{"Products":[{"Code":123},{"Code":456}]}
Then for each product code, i must call two other promises that return Price and Quantity respectively.
I use ui-router and my current implementation code in my resolve is below :
$stateProvider.state('root.products', {
abstract: true,
url: '/products',
template: '<div data-ui-view=""></div>',
resolve: {
products: ['ProductsService', function (ProductsService) {
return ProductsService.getProducts()
.then(function (response){
var data = response.data;
return data.Products.map(function (product) {
var viewModelProduct = {};
angular.copy(product, viewModelProduct);
//get Price for Each Product
ProductsService.getPrice(product.Code)
.then(function (response) {
viewModelProduct.Price = response.data.Info.Price;
})
//get Quantity for Each Product
ProductsService.getQuantity(product.Code)
.then(function (response) {
viewModelProduct.Quantity = response.data.Info.Quantity;
})
return viewModelProduct;
});
});
}]
}
})
It works but my question is if i can write it better. I read about the use of $q and $q.all but I don't really know how to use them.
Is there any way i can write the above code, better and safer??
Thanks in advance for your help!
It works but my question is if i can write it better.
Yes. You can write promise chain. The advantage is: Error propagates, you can catch it on the end of the chain and its not piramide like async call in Javascript
See flow example bellow:
[EDIT]
if the order is not critical you can use $q.all that gets list of promises and notify you when all of them resolved successfully.
It should be something like:
$q.all([
ProductsService.getPrice(product.Code),
ProductsService.getQuantity(product.Code)
])
.then(function(result){
viewModelProduct.Price = result[0].data.Info.Price;
iewModelProduct.Quantity = result[1].data.Info.Quantity;
}, function(error) {
alert(error.message);
});
or even:
var promises = [];
promises.push(ProductsService.getPrice(product.Code));
promises.push(ProductsService.getQuantity(product.Code));
$q.all(promises).then(function(result){
viewModelProduct.Price = result[0].data.Info.Price;
iewModelProduct.Quantity = result[1].data.Info.Quantity;
}, function(error) {
alert(error.message);
});
If you need to execute promises one after other you will use chains like Maxim mentioned.
More one topic:
https://docs.angularjs.org/api/ng/service/$q
http://solutionoptimist.com/2013/12/27/javascript-promise-chains-2/
If you need to execute more promises and then resolve it you will use "$q.all".
See documenttion:
https://docs.angularjs.org/api/ng/service/$q
UPDATE
Not sure if I understand well but you should use combination of q.all and chaining.
Again, sorry if I didn't understand you well.
var p1 = ProductsService.getPrice(product.Code)
.then(function (response) {
return response.data.Info.Price;
});
var p2 = ProductsService.getQuantity(product.Code)
.then(function (response) {
return response.data.Info.Quantity;
});
return $q.all([p1,p2]); // next "then" will get array with price and quantity respectively.