I am very new to NodeJS. I was trying to get a function written that can simply return a configuration value from DB. I might need to call it multiple times.
In PHP or other synchronous languages, I would use the following code for it
function getConfigValue($configKeyName)
{
// DB READ OPERATIONS
return $confguration_value_fetched_from_db
}
getConfigValue("key1");
getConfigValue("key2");
etc
But in NodeJS, I found it too difficult to do this operation because of the Asynchronous nature of the code. After asking some questions here, and after spending hours to learn Callbacks, Promises, Async/await keywords, being a beginner the below is best code I could reach.
// Below function defines the 'get' function
var get = async (key) => {
var result = await COLLECTIONNAME.findOne({key}); //MongoDB in backend
return result.value;
}
// Here I am forced to define another async function so that I can await for the get function.
function async anotherfunction()
{
var value_I_am_lookingfor1 = await get("key1");
var value_I_am_lookingfor2 = await get("key2");
}
anotherfunction();
While it might work, I am not fully happy with the result, mainly because I really don't want to do all my further coding based on the fetched value within this function anotherfunction(). All I want is to fetch a single value? Also I might need to easily call it from many places within the application, not just from here (I was planning to place it in a module)
Any better or easier methods? Or should I always get the value I want, and then nest it with a 'then.' to do the subsequent operation? I even doubt the fundamental approach I take on NodeJS coding itself may be wrong.
Can anyone guide me?
mainly because I really don't want to do all my further coding based on the fetched value within this function anotherfunction(). All I want is to fetch a single value?
Because the request is asynchronous, and your code depends on having the fetched values first, there's no option other than to wait for the values to be retrieved before continuing. Somewhere in the script, control flow needs to halt until the values are retrieved before other parts of the script continue.
Also I might need to easily call it from many places within the application, not just from here (I was planning to place it in a module)
You should have that module make the requests and export a Promise that resolves to the values needed. Rather than using await (which forces requests to be processed in serial), you should probably use Promise.all, which will allow multiple requests to be sent out at once. For example:
valuegetter.js
const get = key => COLLECTIONNAME.findOne({ key }).then(res => res.value);
export default Promise.all([
get('key1'),
get('key2')
]);
main.js
import prom from './valuegetter.js';
prom.then(([val1, val2]) => {
// do stuff with val1 and val2
})
.catch((err) => {
// handle errors
});
If other modules need val1 and val2, call them from main.js with the values they need.
Related
I've created a code snippet here that fetches data about a certain country using REST Countries API. The function works fine, and returns a pending promise. That promise's fulfilled value will equal an object containing key value pairs for the country's capital, name, and code.
But if I wanted to use them for something, why wouldn't I just set that very same object equal to a variable and continue to program new actions inside my async function? Why bother trying to use values gained asynchronously on the global scope?
function getJSON(url, errorMSG = 'Something went wrong') {
// feed it the fetch URL, then your customized error message
return fetch(url).then(response => {
if (!response.ok) throw new Error(errorMSG);
return response.json();
});
}
async function countryData(nation) {
try {
const info = await getJSON(
`https://restcountries.eu/rest/v2/name/${nation}?fullText=true`,
'Invalid country selected'
);
return {
// Fullfilled value of promise
capital: info[0].capital,
name: info[0].name,
code: info[0].cioc,
};
} catch (err) {
console.error(err); // display your custom error message
}
}
console.log(countryData('Canada'));
fetch is an async function. Why do they resolve the promise to a response object, instead of continuing to run extra actions once they have it? Because they don't know what you want to do with it. It would be impossible for fetch to handle every possible thing that should happen next, so it just has a single job: get the data, then return it (in a promise). You can then combine this with whatever other code you like.
On a smaller scale, this may happen with countryData too. You might have 10 different parts of your app that want to do things with the result from countryData. It may not be "impossible" for countryData to implement all 10 things, but it's definitely impractical and not a good idea. Instead, countryData can be written to have one job: get the country data and return it. Then each of the 10 pieces of code can do their own things with the result.
This isn't about it being async, the same principles apply to synchronous code to. If you can keep code focused on a single task, without entangling it with the needs of other pieces of code, then your code becomes easier to maintain.
I'm having trouble understanding control flow with asynchronous programming in JS. I come from classic OOP background. eg. C++. Your program starts in the "main" -- top level -- function and it calls other functions to do stuff, but everything always comes back to that main function and it retains overall control. And each sub-function retains control of what they're doing even when they call sub functions. Ultimately the program ends when that main function ends. (That said, that's about as much as I remember of my C++ days so answers with C++ analogies might not be helpful lol).
This makes control flow relatively easy. But I get how that's not designed to handle event driven programming as needed on something like a web server. While Javascript (let's talk node for now, not browser) handles event-driven web servers with callbacks and promises, with relative ease... apparently.
I think I've finally got my head around the idea that with event-driven programming the entry point of the app might do little more than set up a bunch of listeners and then get out of the way (effectively end itself). The listeners pick up all the action and respond.
But sometimes stuff still has to be synchronous, and this is where I keep getting unstuck.
With callbacks, promises, or async/await, we can effectively build synchronous chains of events. eg with Promises:
doSomething()
.then(result => doSomethingElse(result))
.then(newResult => doThirdThing(newResult))
.then(finalResult => {
console.log(`Got the final result: ${finalResult}`);
})
.catch(failureCallback);
});
Great. I've got a series of tasks I can do in order -- kinda like more traditional synchronous programming.
My question is: sometimes you need to deviate from the chain. Ask some questions and act differently depending on the answers. Perhaps conditionally there's some other function you need to call to get something else you need along the way. You can't continue without it. But what if it's an async function and all it's going to give me back is a promise? How do I get the actual result without the control flow running off and eloping with that function and never coming back?
Example:
I want to call an API in a database, get a record, do something with the data in that record, then write something back to the database. I can't do any of those steps without completing the previous step first. Let's assume there aren't any sync functions that can handle this API. No problem. A Promise chain (like the above) seems like a good solution.
But... Let's say when I call the database the first time, the authorization token I picked up earlier for it has expired and I have to get a new one. I don't know that until I make that first call. I don't want to get (or even test for the need for) a new auth token every time. I just want to be able to respond when a call fails because I need one.
Ok... In synchronous pseudo-code that might look something like this:
let Token = X
Step 1: Call the database(Token). Wait for the response.
Step 2: If response says need new token, then:
Token = syncFunctionThatGetsAndReturnsNewToken().
// here the program waits till that function is done and I've got my token.
Repeat Step 1
End if
Step 3: Do the rest of what I need to do.
But now we need to do it in Javascript/node with only async functions, so we can use a promise (or callback) chain?
let Token = X
CallDatabase(Token)
.then(check if response says we need new token, and if so, get one)
.then(...
Wait a sec. That "if so, get one" is the part that's screwing me. All this asynchronicity in JS/node isn't going to wait around for that. That function is just going to "promise" me a new token sometime in the future. It's an IOU. Great. Can't call the database with an IOU. Well ok, I'd be happy to wait, but node and JS won't let me, because that's blocking.
That's it in a (well, ok, rather large) nutshell. What am I missing? How do I do something like the above with callbacks or Promises?
I'm sure there's a stupid "duh" moment in my near future here, thanks to one or more of you wonderful people. I look forward to it. š Thanks in advance!
What you do with the .then call is to attach a function which will run when the Promise resolves in a future task. The processing of that function is itself synchronous, and can use all the control flows you'd want:
getResponse()
.then(response => {
if(response.needsToken)
return getNewToken().then(getResponse);
})
.then(() => /* either runs if token is not expired or token was renewed */)
If the token is expired, instead of directly scheduling the Promise returned by .then, a new asynchronous action gets started to retrieve a new token. If that asynchronous action is done, in a new task it'll resolve the Promise it returns, and as that Promise was returned from the .then callback, this will also then resolve the outer Promise and the Promise chain continues.
Note that these Promise chains can get complicated very quick, and with async functions this can be written more elegantly (though under the hood it is about the same):
do {
response = await getResponse();
if(response.needsToken)
await renewToken();
} while(response.needsToken)
Fist of all, I would recommend against using then and catch method to listen to Promise result. They tend to create a too nested code which is hard to read and maintain.
I worked a prototype for your case which makes use of async/await. It also features a mechanism to keep track of attempts we are making to authenticate to database. If we reach max attempts, it would be viable to send an emergency alert to administrator etc for notification purposes. This avoid the endless loop of trying to authenticate and instead helps you to take proper actions.
'use strict'
var token;
async function getBooks() {
// In case you are not using an ORM(Sequelize, TypeORM), I would suggest to use
// at least a query builder like Knex
const query = generateQuery(options);
const books = executeQuery(query)
}
async function executeQuery(query) {
let attempts = 0;
let authError = true;
if (!token) {
await getDbAuthToken();
}
while (attemps < maxAttemps) {
try {
attempts++;
// call database
// return result
}
catch(err) {
// token expired
if (err.code == 401) {
await getDbAuthToken();
}
else {
authError = false;
}
}
}
throw new Error('Crital error! After several attempts, authentication to db failed. Take immediate steps to fix this')
}
// This can be sync or async depending on the flow
// how the auth token is retrieved
async function getDbAuthToken() {
}
Foo is an example of a function that works "now" but in the future it will possibly throw an error or return some different value.
const foo = () => {
if (new Date() < new Date('2019-09-05') {
return 'Hello world"
}
throw new Error('failure')
}
const x = cacheLock(foo())
What I am looking for is a way to wrap the return value of foo() so that If a value is returned successfully on initial execution it will cache a snapshot of the result to a file locally and on subsequent executions of this code the stored value is automatically returned and the contents of foo() is never run.
I'm looking for a way to serialize functions results as well as async / awaited promises locally, then playback the stored results.
An ideal feature would be a way to also re-run the code and update the cache.
Is this possible? I know there are issues when it comes to serializing objects and using .toJSON() .fromJSON() methods is one way to solve
that.
Are there any libraries out there that can do this? Is there a means of achieving this without serialization?
I'd love a generic solution to this problem that's not tied to testing, or databases. But my top use case is easily mocking / stubbing API calls and Database reads. The sources of which can change with time and aren't reliable to run in tests. There are many solutions when it comes to those mocking databases or mocking api calls, but I would find that some way to cache live calls, and store them for later the most intuitive.
This idea is more of a post-runtime memoization, a cache that sustains even when the process is finished, while traditional memoization just lasts within the runtime of a process.
In my classes, I've been learning indexeddb and how to handle asynchronous-ity. In the interest of stretching myself and learning more, I've been trying to use the functional paradigm in my code.
I'd been using cursors before but I've realized that my code isn't completely immutable or stateless, which bothers me. I was wondering if there was a way to use cursors without having to resort to pushing elements to an array.
Currently, I'm using something like:
async function getTable(){
return new Promise(function(resolve, reject){
const db = await connect();
const transaction = await db.transaction(["objectStore"], "readonly");
const store = await transaction.objectStore("objectStore");
var myArray = [];
store.openCursor().onsuccess = function(evt) {
var cursor = evt.target.result;
if (cursor) {
myArray.push(cursor.value);
//I don't want to use push, because it's impure. See link:
cursor.continue();
} else {
resolve(myArray);
}
}
}
//link: https://en.wikipedia.org/wiki/Purely_functional_programming
And it works fine. But it's not pure, and it uses push. I'd like to learn another way to do it, if there is one.
Thank you!
You could do several things in the spirit of functional programming, but it is probably not worth it in JavaScript.
For example, to implement immutability arrays, at least in spirit, you simply create and return a new array everytime you want to add an element to an array. I think if I recall my Scheme correctly the function was called cons.
function push(array, newValue) {
const copy = copyArray(array);
copy.push(newValue);
return copy;
}
function copyArray(array) {
const copy = [];
for(const oldValue of array) {
copy.push(oldValue);
}
return copy;
}
// Fancy spread operator syntax implementation if you are so inclined
function copyArray2(inputArray) {
return [...inputArray];
}
Now instead of mutating the input array, you are creating a modified copy of it. Keep in mind this is absolutely horrific performance, and you probably never ever ever want to do this in a real app.
You could take this further probably and use some stack based approach. Again, this is hilariously bad, but it would be something that basically creates a push function that returns a function. The stack increases in size as you append items, and then when you unwind it, it unwinds into an array of values.
My second point is that you can avoid this array building entirely by using newer, less-well-documented features of indexedDB. Specifically, IDBObjectStore.prototype.getAll. This function would create the array for you, opaquely, and provided that it is opaque, you will never know about any FP anti-patterns hidden within its abstraction, and therefore are not breaking the rules.
function getTable(){
return new Promise(function(resolve, reject){
const db = await connect();
const transaction = await db.transaction(["objectStore"], "readonly");
const store = await transaction.objectStore("objectStore");
const request = store.getAll();
request.onsuccess = () => resolve(request.result);
request.onerror = () => reject(request.error);
}
}
My third point is simple, use db.transaction("objectStore", "readonly"); instead of db.transaction(["objectStore"], "readonly");. The array parameter is optional, and it is preferable to keep it simple.
My fourth point is simple, use db.transaction("objectStore"); instead of db.transaction(["objectStore"], "readonly");. "readonly" is the default mode of a transaction, so there is no need to specify it. The purpose of your code is sufficiently clearly communicated by not specifying the parameter, and omitting the parameter is less verbose.
My fifth point is your use of the async specifier(?) in the function definition. You don't need to use it here. You have a synchronous function returning an instance of a Promise. If anything, specifying async leads to increased confusion about what your code is doing. Instead, you probably want to be using the async qualifier when using the function.
My sixth point is that you are violating some FP principles in your call to connect(). What is connect connecting to? Implied global state. This is quite the violation of the spirit of functional programming. Therefore, your connect parameters must be parameters to the function itself, so that you do not rely on the implied knowledge of which database is used.
My seventh point is that you are using a database. Functional programmers have so many problems with databases, or any I/O or interaction with the outside world, they seem to like to pretend there is no such thing. Therefore, you probably shouldn't be using a database at all if you want to use functional approach.
My eight point is that connecting within the promise (calling and awaiting connect) is definitively an anti-pattern. The goal is to chain promises, so that one starts after the other. Either the caller has to call connect and then call getTable, or getTable has to call connect and then do the rest of the promise work.
My ninth point is that I am not even sure how this executes. The executor function that you pass to the Promise constructor is not qualified as async. So you are using the await modifier in a non-qualified function. This should be throwing an error. Technically, a promise swallowed exception, meaning that this promise should always be rejecting.
My tenth point is your use of async everywhere. I have no idea what is going on, unless your connect function is returning some kind of wrapper library, but calls to IDBDatabase.prototype.transaction and IDBTransaction.prototype.objectStore are synchronous. It makes no sense why you are awaiting them. They do not return promises.'
My eleventh point is that you are not watching for errors. request.onsuccess is not called back when there is an error. This could lead to your promise never settling. You need to also consider the failure case.
My twelfth point is you seem to missing a closing parentheses for your onsuccess handler function. I am not sure how this code ever is interpreted successfully.
Could you please help me to understand javascirpt async hell?
I think I am missing something important ā¹ The thing is that js examples and most of the answers on the internet are related to just one part of code ā a small snippet. But applications are much more complicated.
I am not going write it directly in JS since I am more interested of the design and how to write it PROPERLY.
Imagine these functions in my application:
InsertTestData();
SelectDataFromDB_1(āUSERSā);
SelectDataFromDB_2(āUSER_CARSā,āUSERSā);
FillCollections(āUSERā,āUSER_CARSā);
DoTheWork();
DeleteData();
I did not provide any description for the functions but I think it is obvious based on names. They need to go in THIS SPECIFIC ORDER. Imagine that I need to run a select into the db to get USERS and then I need run a select to get USER_CARS for these USERS. So it must be really in this order (consider the same for other functions). The thing is that need to call 6 times Node/Mysql which is async but I need results in specific order. So how can I PROPERLY make that happen?
This could work:
/* not valid code I want to present the idea and keep it short */
InsertTestData(
Mysql.query(select, data, function(err,success)
{
SelectDataFromDB_1(āUSERSā); -- in that asyn function I will call the next procedure
}
));
SelectDataFromDB_1 (
Mysql.query(select, data, function(err,success)
{
SelectDataFromDB_2(āUSERSā); -- in that asyn function I will call the next procedure
}
));
SelectDataFromDB_2 (
Mysql.query(select, data, function(err,success)
{
FillCollections (āUSERSā); -- in that asyn function I will call the next procedure
}
));
etc..
I can āeasilyā chain it but this looks as a mess. I mean really mess.
I can use some scheduler/timmers to schedule and testing if the previous procedure is done)
Both of them are mess.
So, what is the proper way to do this;
Thank you,
AZOR
If you're using a recent version of Node, you can use ES2017's async/await syntax to have synchronous-looking code that's really asynchronous.
First, you need a version of Mysql.query that returns a promise, which is easily done with util.promisify:
const util = require('util');
// ...
const query = util.promisify(Mysql.query);
Then, wrap your code in an async function and use await:
(async () => {
try {
await InsertTestData();
await SelectDataFromDB_1(āUSERSā);
await SelectDataFromDB_2(āUSER_CARSā,āUSERSā);
await FillCollections(āUSERā,āUSER_CARSā);
await DoTheWork();
await DeleteData();
} catch (e) {
// Handle the fact an error occurred...
}
})();
...where your functions are async functions, e.g.:
async InsertTestData() {
await query("INSERT INTO ...");
}
Note the try/catch in the async wrapper, it's essential to handle errors, because otherwise if an error occurs you'll get an unhandled rejection notice (and future versions of Node may well terminate the process). (Why "unhandled rejections"? Because async functions are syntactic sugar for promises; an async function returns a promise.) You can either do that with the try/catch shown, or alternate by using .catch on the result of calling it:
(async () => {
await InsertTestData();
// ...
})().catch(e => {
// Handle the fact an error occurred
});