Develop Reference

Passing in list elements in javascript to a different function

I'm trying to do a web scraping exercise where some values are being retrieved and are being stored in a list variable. I am then passing in the list variable as a parameter in a different function. The problem with my approach is I am getting an error when calling the different function. I believe I'm getting this error because I am not passing in the list elements into the function appropriately. In the function, I am reading from a Yahoo Stock API used to retrieve stock data. If I were to hardcode a stock symbol into the parameter for the function, it works without any issue. Since I am passing in a parameter, I am getting this error. Below is my code and the error I am getting. Any feedback would be helpful.
Code
const cheerio = require('cheerio');
const axios = require('axios');
const yahooStockPrices = require('yahoo-stock-prices');
var stockSymbol = []
async function read_fortune_500() {
try {
const { data } = await axios({ method: "GET", url: "https://en.wikipedia.org/wiki/List_of_S%26P_500_companies", })
const $ = cheerio.load(data)
const elemSelector = '#constituents > tbody > tr > td:nth-child(1)'
$(elemSelector).each((parentIndex, parentElem) => {
let keyIndex = 0
if (parentIndex <= 9){
$(parentElem).children().each((childIndex, childElem) => {
const tdValue = $(childElem).text()
if (tdValue) {
//stockObject[keys[keyIndex]] = tdValue
stockSymbol = tdValue
}
})
console.log(stockSymbol)
}
})
} catch (err) {
console.error(err)
}
return stockSymbol;
}
async function collect_stocks(stockSymbol) {
stockSymbol = read_fortune_500()
const stockResult = await yahooStockPrices.getCurrentData(stockSymbol);
console.log(stockResult);
}
collect_stocks(stockSymbol)
Error
/node_modules/yahoo-stock-prices/yahoo-stock-prices.js:75
.split('regularMarketPrice')[1]
^
TypeError: Cannot read properties of undefined (reading 'split')
at Request._callback (/node_modules/yahoo-stock-prices/yahoo-stock-prices.js:75:21)
at Request.self.callback (/node_modules/request/request.js:185:22)
at Request.emit (node:events:390:28)
at Request.emit (node:domain:475:12)
at Request.<anonymous> (/node_modules/request/request.js:1154:10)
at Request.emit (node:events:390:28)
at Request.emit (node:domain:475:12)
at IncomingMessage.<anonymous> (/node_modules/request/request.js:1076:12)
at Object.onceWrapper (node:events:509:28)
at IncomingMessage.emit (node:events:402:35)

The parameter stockSymbol seems to be empty when you pass it to your desired function therefore, when yahoo-stock-prices try to apply a split on it, it fails.

Can anyone tell me why it is showing array is empty?

I want to send notification to multiple devices and for that am getting the token via querying the docs and saving the token to array but it shows that array is empty. Most probably error is because am not able to add elements in the array.
My code is:-
var registrationTokens=[];
const indexOfSender=usersList.indexOf(chatItem.senderUsername);
let removedUsername=usersList.splice(indexOfSender,1); //to remove the senders name from list
usersList.forEach(async(element)=>{
const query = admin.firestore().collection('users').where("username","==",element);
const querySnapshot = await query.get();
if (querySnapshot.docs.length > 0) {
const doc = querySnapshot.docs[0];
const data = doc.data();
registrationTokens.push(data.androidNotificationToken); //adding token over here
}
else {
console.log("Unable to get token for the username ", element);
}
});
const message =
{
notification: {
title:'Message',
body: body,
imageUrl: url,
},
tokens: registrationTokens,
data: { recipient: senderUserId },
};
admin.messaging().sendMulticast(message)
.then(response =>
{
if (response.failureCount > 0) {
const failedTokens = [];
response.responses.forEach((resp, idx) => {
if (!resp.success) {
failedTokens.push(registrationTokens[idx]);
}
});
console.log('List of tokens that caused failures: ' + failedTokens);
}
else
{
console.log('Successfully sent messages ', response);
}
});
Error
Error: tokens must be a non-empty array
at FirebaseMessagingError.FirebaseError [as constructor] (/workspace/node_modules/firebase-admin/lib/utils/error.js:42:28)
at FirebaseMessagingError.PrefixedFirebaseError [as constructor] (/workspace/node_modules/firebase-admin/lib/utils/error.js:88:28)
at new FirebaseMessagingError (/workspace/node_modules/firebase-admin/lib/utils/error.js:254:16)
at Messaging.sendMulticast (/workspace/node_modules/firebase-admin/lib/messaging/messaging.js:294:19)
at sendNotificationForGroupChat (/workspace/index.js:238:35)
at exports.onCreateMessage.functions.region.firestore.document.onCreate (/workspace/index.js:116:9)
at process._tickCallback (internal/process/next_tick.js:68:7)

async inside forEach does not work the way you expect. If you add some logging, you will see that the loop ends before any of its async work is complete, leaving your tokens array empty before you pass it to FCM. Each iteration through the loop simply generates a promise that is not resolved. You will need to rewrite the code to actually wait for all those promises before calling FCM.
Read more about that:
Using async/await with a forEach loop

for-of loop will work just fine with asynchronous calls :)
Cheers

GCloud Function with Puppeteer - 'Process exited with code 16' error

Having trouble finding any documentation or cause for this sort of issue. I'm trying to run a headless chrome browser script that pulls the the current song playing from kexp.org and returns it as a JSON object. Testing with the NPM package #Google-clound/functions-framework does return the correct response however when deployed into GCloud, I receive the following error when hitting the API trigger:
Error: could not handle the request
Error: Process exited with code 16
at process.on.code (invoker.js:396)
at process.emit (events.js:198)
at process.EventEmitter.emit (domain.js:448)
at process.exit (per_thread.js:168)
at logAndSendError (/workspace/node_modules/#google-cloud/functions framework/build/src/invoker.js:184)
at process.on.err (invoker.js:393)
at process.emit (events.js:198)
at process.EventEmitter.emit (domain.js:448)
at emitPromiseRejectionWarnings (internal/process/promises.js:140)
at process._tickCallback (next_tick.js:69)
Full Script:
const puppeteer = require('puppeteer');
let browserPromise = puppeteer.launch({
args: [
'--no-sandbox'
]
})
exports.getkexp = async (req, res) => {
const browser = await browserPromise
const context = await browser.createIncognitoBrowserContext()
const page = await context.newPage()
try {
const url = 'https://www.kexp.org/'
await page.goto(url)
await page.waitFor('.Player-meta')
let content = await page.evaluate(() => {
// finds elements by data type and maps to array note: needs map because puppeeter needs a serialized element
let player = [...document.querySelectorAll('[data-player-meta]')].map((player) =>
// cleans up and removes empty strings from array
player.innerHTML.trim());
// creates object and removes empty strings
player = {...player.filter(n => n)}
let songList = {
"show":player[0],
"artist":player[1],
"song":player[2].substring(2),
"album":player[3]
}
return songList
});
context.close()
res.set('Content-Type', 'application/json')
res.status(200).send(content)
} catch (e) {
console.log('error occurred: '+e)
context.close()
res.set('Content-Type', 'application/json')
res.status(200).send({
"error":"occurred"
})
}
}
Is there documentation for this error type? It's been deployed on GCloud via CLI shell with the following parameters:
gcloud functions deploy getkexp --trigger-http --runtime=nodejs10 --memory=1024mb

Firebase database query returning TypeError: obj.hasOwnProperty is not a function

I'm trying to make a request to my Firebase database running the following code:
const lastCrawl = admin
.database()
.ref('/crawls')
.orderByChild("dateAdded")
.limitToLast(1)
.once('value')
.then((snapshot) => {
return snapshot.val();
});
I've written this based on the docs and just added some extra steps to deliver only the last record inside the target directory. However, even hardcoding one record from the directory (e.g.: '/crawls/RECORD-ID') would throw the same error, which is strange as the code is basically the same as in the documentation.
Currently it's returning the following error:
Unhandled rejection
TypeError: obj.hasOwnProperty is not a function
at each (/srv/node_modules/#firebase/database/dist/index.node.cjs.js:553:17)
at validateFirebaseData (/srv/node_modules/#firebase/database/dist/index.node.cjs.js:1511:9)
at /srv/node_modules/#firebase/database/dist/index.node.cjs.js:1528:13
at each (/srv/node_modules/#firebase/database/dist/index.node.cjs.js:554:13)
at validateFirebaseData (/srv/node_modules/#firebase/database/dist/index.node.cjs.js:1511:9)
at /srv/node_modules/#firebase/database/dist/index.node.cjs.js:1528:13
at each (/srv/node_modules/#firebase/database/dist/index.node.cjs.js:554:13)
at validateFirebaseData (/srv/node_modules/#firebase/database/dist/index.node.cjs.js:1511:9)
at /srv/node_modules/#firebase/database/dist/index.node.cjs.js:1528:13
at each (/srv/node_modules/#firebase/database/dist/index.node.cjs.js:554:13)

As Frank commented, there's a fundamental issue, I was trying to return from the callback, which wouldn't work as it's called asynchronously.
The way I decided to solve was to return a promise as done bellow:
const getLastCrawl = async () => {
const ref = admin.database().ref('/crawls/')
return ref.orderByChild("dateAdded")
.limitToLast(1)
.once('value')
.then((snapshot) => {
return snapshot.val();
});
}
const lastCrawl = await getLastCrawl()
Thanks a lot for the help Frank!

Why i get Error 500 fetching data with axios

I am doing little project to learn and put it in my project section
I get problem like this:
GET https://cors-anywhere.herokuapp.com/http://www.recipepuppy.com/api/?q=rice 500 (Internal Server Error)
Uncaught (in promise) Error: Request failed with status code 500
at createError (createError.js:16)
at settle (settle.js:17)
at XMLHttpRequest.handleLoad (xhr.js:61)
I tried with using axios headers (i found this sugestion on another forum but it dosn't work)
My code for fetching data looks like this
export async function fetchData(text) {
const proxy = `https://cors-anywhere.herokuapp.com/`;
const base = `http://www.recipepuppy.com/api/`;
const baseEnd = `q=${text}`;
const data = await axios
.get(`${proxy}${base}?${baseEnd}`)
.then(data =>console.log(data));
const recipes = data.data.results;
return recipes.length > 0 ? recipes : false;
}
function called here:
async function getRecipes(e){
e.preventDefault();
const text = e.target[1].value;
const loader = '<div class="loader"></div>';
container.insertAdjacentHTML('beforebegin', loader);
const recipes = await fetchData(text);
document.querySelector('.loader').remove();
displayRecipes(recipes);
}

You need to catch possible error from Promise (as axios.get() return Promise type object).
axios.get(`${proxy}${base}?${baseEnd}`)
.then(data =>console.log(data))
.catch(error => {
console.log(error)
})