I have a function that checks the existence of token in the database. The problem is it takes some amount of time to return that bool value and I kinda need to pause the function so that the function realizes that the token has existed and run the query again.
const registerToken = dispatch => {
var tokenExisted = null
do {
let token = generateRandomToken();
firebase.database().ref(`/Token`).orderByChild("token").equalTo(token).once("value", snapshot => { // check whether token exists
if (!snapshot.val()) { // if token not exist
token = false;
// register token to firebase
} else {
token = true; // continue the loop to generate a new token and query again
}
})
} while (tokenExisted === true);
}
My setup is basically a do-while loop, when the function first gets call
tokenExisted = null, then a random 4 digit token will be generated and a query will be dispatched to firebase and verify it token has existed.
If token has existed, then tokenExisted = true. I expect it the assignment to be executed but the single threaded nature of Javascript will reach the end of the loop before the query return anything.
I figured to use setTimeout and periodically add some small amount of time whenever tokenExisted = null to kinda safe guard so that the function will always catch when query function returns anything.
Has anyone had a better approach to achieve the same thing?
You might want to call the function itself recursively, as such.
const registerToken = dispatch => {
let token = generateRandomToken();
const tokenObjectRef = firebase.database().ref(`/Token`);
tokenObjectRef.orderByChild("token").equalTo(token).once("value")
.then(snapshot => {
if (!snapshot.val()) {
// success!
} else {
registerToken(dispatch) // call itself again
}
})
.catch(error => {} ))
}
The logic is that token will be refreshed during each new iteration, should the process fails and a new query is needed (if this is what you need).
Note: avoid using do-while in async logic. Plan carefully ahead as you might encounter lots of logic error and it is hard to trace.
Call the function recursively.
function get_token_then(callback_when_token_found) {
firebase.database().etc.etc(function (data) {
if (data == what_you_want) {
callback_when_token_found(data);
} else {
// You might want to wrap this in setTimeout in order to throttle your database calls
get_token_then(callback_when_token_found);
}
}
}
Related
I'm creating a YouTube upload notification bot for a Discord Server I am in using the YouTube RSS Feed and am having problems with it. I have issues with the bot sending the same video twice even though I've tried everything to fix it. The bot cycles through different users in a for loop and checks the user's latest video's ID with one stored in a JSON file. If they do not match, it sends a message and updates the JSON. Here is my current code:
function update(videoId, n) {
var u = JSON.parse(fs.readFileSync("./jsons/uploads.json"))
u[n].id = videoId
fs.writeFile("./jsons/uploads.json", JSON.stringify(u, null, 2), (err) => {
if (err) throw err;
// client.channels.cache.get("776895633033396284").send()
console.log('Hey, Listen! ' + n + ' just released a new video! Go watch it: https://youtu.be/' + videoId + "\n\n")
});
}
async function uploadHandler() {
try {
var u = require('./jsons/uploads.json');
var users = require('./jsons/users.json');
for (i = 0; i < Object.keys(users).length; i++) {
// sleep(1000)
setTimeout(function(i) {
var username = Object.keys(users)[i]
let xml = f("https://www.youtube.com/feeds/videos.xml?channel_id=" + users[username]).text()
parseString(xml, function(err, result) {
if (err) {} else {
let videoId = result.feed.entry[0]["yt:videoId"][0]
let isMatch = u[username].id == videoId ? true : false
if (isMatch) {} else {
if (!isMatch) {
u[username] = videoId
update(videoId, username)
}
}
}
});
}, i * 1000, i)
}
} catch (e) {
console.log(e)
}
}
My code is rather simple but I've had the same issue with other codes that use this method; therefore what would be the best way to accomplish this? Any advice is appreciated
There are a few issues with your code that I would call out right off the bat:
Empty blocks. You use this especially with your if statements, e.g. if (condition) {} else { // Do the thing }. Instead, you should negate the condition, e.g. if (!condition) { // Do the thing }.
You declare the function uploadHandler as async, but you never declare that you're doing anything asynchronously. I'm suspecting that f is your asynchronous Promise that you're trying to handle.
You've linked the duration of the timeout to your incrementing variable, so in the first run of your for block, the timeout will wait zero seconds (i is 0, times 1000), then one second, then two seconds, then three...
Here's a swag at a refactor with some notes that I hope are helpful in there:
// Only require these values once
const u = require('./jsons/uploads.json');
const users = require('./jsons/users.json');
// This just makes the code a little more readable, I think
const URL_BASE = 'https://www.youtube.com/feeds/videos.xml?channel_id=';
function uploadHandler() {
Object.keys(users).forEach(username => {
// We will run this code once for each username that we find in users
// I am assuming `f` is a Promise. When it resolves, we'll have xml available to us in the .then method
f(`${URL_BASE}${username}`).then(xml => {
parseString(xml, (err, result) => {
if (!err) {
const [videoId] = result.feed.entry[0]['yt:videoId']; // We can use destructuring to get element 0 from this nested value
if (videoId !== u[username].id) {
// Update the in-memory value for this user's most recent video
u[username].id = videoId;
// Console.log the update
console.log(`Hey listen! ${username} just released a new video! Go watch it: https://youtu.be/${videoId}\n\n`);
// Attempt to update the json file; this won't affect the u object in memory, but will keep your app up to date
// when you restart it in the future.
fs.writeFile('./jsons/uploads.json', JSON.stringify(u, null, 2), err => {
if (err) {
console.err(`There was a problem updating uploads.json with the new videoId ${videoId} for user ${username}`);
}
});
}
}
});
})
// This .catch method will run if the call made by `f` fails for any reason
.catch(err => console.error(err));
});
}
// I am assuming that what you want is to check for updates once every second.
setInterval(uploadHandler, 1000);
Example code:
Hub.listen('auth', event => {
const { event: type, data } = event.payload;
if (type === 'signIn') {
const session = data.signInUserSession;
console.log('SESSION', data.signInUserSession);
setTimeout(() => {
console.log('SESSION', data.signInUserSession);
}, 100);
}
});
When using oath, after the provider redirects to my app, the Hub fires a signIn event. However, the signInUserSession property is null when the event is fired, but gets a value some time later (within 100 ms). This does not seem to occur when using Auth.signIn(email, password) directly; signInUserSession is populated when the event is fired.
What is happening here, and how can I get around it? Currently, I have an explicit delay in the code, which is a terrible hack.
Perhaps the old way of JavaScript for waiting for value to be populated is useful to ensure that code does not fail even if the it takes longer than expected in populating the value.
Here is a sample code that I normally use when no other options are available.
waitForValue(){
if(myVar!= null && typeof myVar !== "undefined"){
//value exists, do what you want
console.log(myVar)
}
else{
setTimeout(() => {this.waitForValue()}, 100);
}
}
You can refactor this sample code as per your need.
Alternatively, AWS Amplify also have other ways to get current logged in user session. e.g. Auth.currentAuthenticatedUser() and Auth.currentSession() return promise. They can be used like this
private async getUser(){
let user = null;
try {
user = await Auth.currentAuthenticatedUser();
//console.log(user);
} catch (err) {
//console.log(err);
}
//return user;
}
i am not used to aws amplify - just read some github and so far i can see we will need info about your userPool implementation - i guess some weird callback issue
But for a workaround you can proxy the reference:
const event = {type: "signIn", data: {signInProperty: "null"}}
setTimeout(()=>event.data.signInProperty = "{Stack: Overflow}", 1000)
// mock events
function emit(type, args){
console.log(type, args)
}
//initialize
let watchedValue = event.data.signInProperty
document.getElementById("app").innerHTML = event.data.signInProperty
// protect reference
Object.defineProperty(event.data, "signInProperty", {
set(newValue){
watchedValue = newValue
document.getElementById("app").innerHTML = newValue
emit("event:signInCompleted", event.data)
},
get(){
return watchedValue
}
})
<div id="app"></div>
I am creating a Node.js module that takes a list of movie titles and fetches their respective metadata from omdbapi.com.
These lists are often very large and sometimes (with my current slow internet connection) the connection stalls due to too many concurrent connections. So I set up a timeout/abort method that restarts the process after 30 seconds.
The problem I'm having is that whenever I lose internet connection or the connection stalls, it just bails out of the process, and doesn't restart the connection.
Example:
async function getMetadata () {
const remainingMovies = await getRemainingMovies();
for (let i = 0; i < remainingMovies.length;i++) {
const { data, didTimeout } = await fetchMetadata(remainingMovies[i]);
// Update "remainingMovies" Array...
if (didTimeout) {
getMetadata();
break;
}
}
if (!didTimeout) {
return data;
}
}
This is obviously a simplified version but essentially:
The getMetadata Function gets the remainingMovies Array from the global scope.
Fetches the metadata from the server with the fetchMetadata Function.
Checks if the connection timed out or not.
If it did it should restart the Function and attempt to connect again.
If it didn't timeout then finish the for loop and continue.
I guess you want something similar to below script. Error handling using try/catch for async/await which probably is what you are looking for as a missing puzzle.
async function getMetadata() {
const remainingMovies = await getRemainingMovies();
remainingMovies.map(movie => {
try {
return await fetchMetadata(movie);
} catch (err) {
return getMetadata();
}
});
}
Actually I'm not sure that Title of my question is 'correct', if you
have any idea with it, you could leave a comment and I'll rename it.
I am trying to rewrite my old function which make http-requests and insert many object at mongoDB via mongoose. I already have a working version of it, but I face a problem while using it. Basically, because when I'm trying to insertMany 20 arrays from 20+ request with ~50'000 elements from one request it cause a huge memory leak. Even with MongoDB optimization.
Logic of my code:
function main() {
server.find({locale: "en_GB"}).exec(function (err, server) {
for (let i = 0; i < server.length; i++) { //for example 20 servers
rp({url: server[i].slug}).then(response => {
auctions.count({
server: server[i].name,
lastModified: {$gte: response.data.files[0].lastModified}
}).then(function (docs) {
if (docs < 0) {
//We don't insert data if they are already up-to-date
}
else {
//I needed response.data.files[0].url and server[i].name from prev. block
//And here is my problem
requests & insertMany and then => loop main()
})
}
})
}).catch(function (error) {
console.log(error);
})
}
})
}
main()
Actually I have already trying many different things to fix it. First-of-all I was trying to add setInterval after else block like this:
setTimeout(function () {
//request every server with interval, instead of all at once
}, 1000 * (i + 1));
but I create another problem for myself because I needed to recursive my main() function right after. So I can't use: if (i === server[i].length-1) to call garbage collector or to restart main() because not all server skip count validation
Or let's see another example of mine:
I change for (let i = 0; i < server.length; i++) from 3-rd line to .map and move it from 3-rd line close to else block but setTimeout doesn't work with .map version, but as you may already understand script lose correct order and I can't make a delay with it.
Actually I already understand how to fix it at once. Just re-create array via let array_new = [], array_new.push = response.data.files[0].url with use of async/await. But I'm not a big expert in it, so I already waste a couple of hours. So the only problem for now, that I don't know how to return values from else block
As for now I'm trying to form array inside else block
function main() {
--added let array_new = [];
[v1]array_new.url += response.data.files[0].url;
[v2]array_new.push(response.data.files[0].url);
return array_new
and then call array_new array via .then , but not one of these works fine for now. So maybe someone will give me a tip or show me already answered question #Stackoverflow that could be useful in my situation.
Since you are essentially dealing with promises, you can refactor your function logic to use async await as follows:
function async main() {
try {
const servers = await server.find({locale: "en_GB"}).exec()
const data = servers.map(async ({ name, slug }) => {
const response = await rp({ url: slug })
const { lastModified, url } = response.data.files[0]
const count = await auctions.count({
server: name,
lastModified: { $gte: lastModified }
})
let result = {}
if (count > 0) result = { name, url }
return result
}).filter(d => Object.keys(d).length > 0)
Model.insertMany(data)
} catch (err) {
console.error(err)
}
}
Your problem is with logic obscured by your promises. Your main function recursively calls itself N times, where N is the number of servers. This builds up exponentially to eat memory both by the node process and MongoDB handling all the requests.
Instead of jumping into async / await, start by using the promises and waiting for the batch of N queries to complete before starting another batch. You can use [Promise.all] for this.
function main() {
server.find({locale: "en_GB"}).exec(function (err, server) {
// need to keep track of each promise for each server
let promises = []
for (let i = 0; i < server.length; i++) {
let promise = rp({
url: server[i].slug
}).then(function(response) {
// instead of nesting promises, return the promise so it is handled by
// the next then in the chain.
return auctions.count({
server: server[i].name,
lastModified: {
$gte: response.data.files[0].lastModified
}
});
}).then(function (docs) {
if (docs > 0) {
// do whatever you need to here regarding making requests and
// inserting into DB, but don't call main() here.
return requestAndInsert();
}
}).catch(function (error) {
console.log(error);
})
// add the above promise to out list.
promises.push(promise)
}
// register a new promise to run once all of the above promises generated
// by the loop have been completed
Promise.all(promises).then(function () {
// now you can call main again, optionally in a setTimeout so it waits a
// few seconds before fetchin more data.
setTimeout(main, 5000);
})
})
}
main()
I'm a beginner when it comes to promises and I'm trying to understand how to work with them.
I have a firebase trigger where I am performing some validation. If the validation fails, I want to "exit" the trigger, meaning I don't want any code after the validation to execute (assuming the validation failed). But it does. Even though I'm sure that the validation fails (the "You have been timed out due to inactivity. Please go back to the bed booking map and start again" is sent to the android app I'm developing), the code after keeps executing. I know this because I've placed console logs inside it.
I've put comments in my code for the validation I'm talking about, and what code I don't want executed.
exports.createCharge = functions.database.ref('/customers/{userId}/charges/{id}')
.onCreate((snap, context) => {
console.log("Entered createharge function");
const val = snap.val();
return admin.database().ref(`/customers/${context.params.userId}/customer_id`)
.once('value').then((snapshot) => {
return snapshot.val();
}).then((customer) => {
// Do stuff
if (val.type === 'beds') {
// Check if user email is the same as bed email
for (var i = 0; i < val.beds.length; i++) {
var bedEmailRef = db.ref(`beds/${val.hid}/${val.beds[i]}/email`);
bedEmailRef.on("value", function(bedEmailSnap) {
var bedEmail = bedEmailSnap.val();
if (val.email !== bedEmail) { // VALIDATION
snap.ref.child('error').set("You have been timed out due to inactivity. Please go back to the bed booking map and start again");
return null; // Here, I want to exit the whole function.
}
});
}
// If val.email !== bedEmail, I NEVER want to reach here!
return admin.database().ref(`/hotels/${val.hid}/bedPrice`)
.once('value').then((tempBedPrice) => {
// do stuff
console.log("got here");
return stripe.charges.create(charge, {idempotency_key: idempotencyKey});
}).then((response) => {
// Do more stuff
return snap.ref.set(response);
}).catch((error) => {
snap.ref.child('error').set(userFacingMessage(error));
return reportError(error, {user: context.params.userId});
})
} else throw Error('No type');
});
});
Why am I getting this behaviour? How can I stop the code after the validation from executing?
The problem is that you are adding a "listener" after checking for "beds" type. A solution would be to fetch data once:
// This is a snippet from Firebase documentation
return firebase.database().ref('/users/' +
userId).once('value').then(function(snapshot) {
var username = (snapshot.val() && snapshot.val().username) ||
'Anonymous';
// ...
});
Or refactor your code so your validation can be set as a callback that can be returned if some criteria is met.
Additionally, here's a link for Cloud functions best practices that can help you writing better cloud functions.
In cases where you want to exit / don't want anymore events on the listeners, refactor your code as below:
bedEmailRef.on("value", function(bedEmailSnap) {
var bedEmail = bedEmailSnap.val();
if (val.email !== bedEmail) { // VALIDATION
snap.ref.child('error').set("You have been timed out due to inactivity. Please go back to the bed booking map and start again");
bedEmailRef.off('value'); // This will stop listening for further events
return null; // This is useless because returning inside a listener has no effect
}
});