Develop Reference

Javascript - Loop through API 10 times, although, using the ID of each returned API call as a parameter for the NEXT api call

I am trying to make 10 API calls to the reddit API to get a users most recent comments.
Basically, Reddit uses a parameter 'after' in the URI to get the next page. So basically, if there is some dummy example comment data below, I need the LAST ID.
Example.
API Call: https://www.reddit.com/user/USERNAME/comments.json?limit=3
returns:
{
id: 'd4gf125',
comment: 'blah blah blah'
},
{
id: 'dag42ra',
comment: 'blah blah blah'
},
{
id: 'hq6ir3j', // I need this ID right here, the LAST in the list for the next API call
comment: 'blah blah blah'
},
Then, another API call needs to be made directly AFTER the previous, using the ID as a parameter.
API Call:
`https://www.reddit.com/user/USERNAME/comments.json?limit=3&after=tl_hq6ir3j`
This will run 10 times, which will get 10 pages, so in total, 30 results.
I have tried using this for loop but as for loops are not asynchronous, I cannot change the variable 'lastID' in time.
First attempt:
const handleSubmit = async () => {
setLoading(true);
axios.get(`${apiUrl}user/${username}/about.json`).then((res) => {
setUserData({
about: res.data.data,
});
axios
.get(`${apiUrl}user/${username}/comments.json?limit=100`)
.then((res) => {
let data = res.data.data.children;
let lastItem = data[data.length - 1];
for (i = 0; i < 10; i++) {
axios
.get(
`${apiUrl}comments/${lastItem.data.id}/comments.json?limit=100&after=tl_${lastItem}`
)
.then((res) => {
lastItem =
res.data.data.children[res.data.data.children.length - 1];
});
}
});
});
};
If you need some example REAL data, see the link below:
https://www.reddit.com/user/bulkorcut99/comments.json?limit=1000&after=t1_hq6ir3j

To do async things in sequence, chain promises together with then(). The generic form looks like ths...
let promise = /* starting promise */
for (/* loop stuff */) {
promise = promise.then(/*another promise*/)
}
Refactoring your code to make this more clear...
function getAbout(username) {
return axios.get(`${apiUrl}user/${username}/about.json`).then(res => {
return res.data.data
});
}
function getComment(username, afterId) {
const url = `${apiUrl}comments/${lastItem.data.id}/comments.jsonlimit=100`;
if (afterId) url += `&after=tl_${afterId}`;
return axios.get(url).then(res => res.data.data);
}
// create a chain of promises to get comments, pushing results as we go
function getComments(username, results) {
let promise = getComment(username);
for (let i=0; i<10; i++) { // not sure why 10 here... from the OP
promise = promise.then(data => {
results.push(data);
const lastChild = data.children[data.children.length-1];
return getComment(username, lastChild.id);
});
}
return promise;
}
const handleSubmit = async () => {
setLoading(true);
return getAbout(username).then(data => {
setUserData({ about: data });
}).then(() => {
let comments = [];
return getComments(username, comments).then(() => comments)
}).then(comments => {
// comments is the result from all of the get comments calls
})
}
The real API might stop finding comments in fewer than 10 calls. If so, rather than looping to 10, you'd loop until the results were empty or whatever condition indicates there are no more.

You can simply await each of your axios.get calls in your for loop. To do so you can do something like this:
const handleSubmit = async () => {
setLoading(true);
axios.get(`${apiUrl}user/${username}/about.json`).then((res) => {
setUserData({
about: res.data.data,
});
axios
.get(`${apiUrl}user/${username}/comments.json?limit=100`)
.then(async (res) => {
let data = res.data.data.children;
let lastItem = data[data.length - 1];
for (i = 0; i < 10; i++) {
var {data} = await axios
.get(`${apiUrl}comments/${lastItem.data.id}/comments.json?limit=100&after=tl_${lastItem}`)
lastItem = data.data.children[data.data.children.length - 1];
}
});
});
};

How can I cache this call to the Google Sheets API?

I have followed the instructions here to use a Google Sheet as a JSON endpoint. I am then using that data in Eleventy. This code is currently working:
module.exports = async function() {
let url = `https://docs.google.com/spreadsheets/d/${process.env.GOOGLE_SHEETS_ID}/gviz/tq?tqx=out:json`;
console.log("Fetching from Google Sheets...");
return await fetch(url)
.then(res => res.text()) // node-fetch option to transform to json
.then(text => {
let json = JSON.parse(text.substr(47).slice(0, -2));
return {
items: json.table.rows
};
});
}
...however, this is making for slow build times, so I am trying to tie this in with the eleventy-cache-assets plugin as described in the 11ty docs.
Here is what I've tried:
module.exports = async function() {
let url = `https://docs.google.com/spreadsheets/d/${process.env.GOOGLE_SHEETS_ID}/gviz/tq?tqx=out:json`;
var text = await Cache(url, {
duration: "1s",
type: "text"
})
.then(text => {
var json = JSON.parse(text.substr(47).slice(0, -2));
console.log(json);
return {
items: json.table.rows
};
});
};
In the console, it does return the JSON, but then when I try to make a collection out of the data in .eleventy.js like this:
eleventyConfig.addCollection("myGSheets", (collection) => {
return collection.getAll()[0].data.myGSheets.items;
});
I get an error: Cannot read property 'items' of undefined
I'm not sure what is happening in between the JSON data appearing in the console, and then being undefined.
I am guessing maybe I need to do the string manipulation of the response before calling Cache, perhaps? I'm just not sure how to put it all together...

It looks like you're not actually returning anything from your data function, since your return statement is inside the callback function, not the top-level data function.
module.exports = async function() { // <= top level data function
let url = `https://docs.google.com/spreadsheets/d/${process.env.GOOGLE_SHEETS_ID}/gviz/tq?tqx=out:json`;
var text = await Cache(url, {
duration: "1s",
type: "text"
})
.then(text => { // <= inner callback function
var json = JSON.parse(text.substr(47).slice(0, -2));
console.log(json);
return { // <= This return statement returns from the inner callback
items: json.table.rows
};
});
// <= This function doesn't return anything!
};
Since you're using async/await, there isn't a need for using Promise.then. You can just await the promises, which prevents needing all the callbacks.
Try:
module.exports = async function() {
let url = `https://docs.google.com/spreadsheets/d/${process.env.GOOGLE_SHEETS_ID}/gviz/tq?tqx=out:json`;
var text = await Cache(url, {
duration: "1s",
type: "text"
})
var json = JSON.parse(text.substr(47).slice(0, -2));
console.log(json);
return {
items: json.table.rows
};
};

Knex promises inside for loop

I'm new to Node/asynchronous coding and I realize this is a basic question that speaks to some fundamentals I'm missing, but for the life of me I just can't understand how this is supposed to work. I'm seeding a database using knex, by reading in data from a CSV and iterating through the rows in a for loop. Within that loop, I need to query another table in the database and return an associated value as part of the new row to be created.
I understand that knex returns a pending Promise, so it doesn't have access to the returned value yet. But I can't seem to get the structure right with async/await or Promise.all; I've tried it a bunch of ways based on other answers I've seen, but none seem to actually explain what's happening well enough that I can apply it successfully to my case. Any help hugely appreciated.
My seed file currently looks like this:
exports.seed = function(knex) {
const fs = require('fs');
function createImage(row, event_id) {
return {
name: row[1],
event_id: event_id
}
};
function get_event_id(loc) {
knex('events')
.where({location: loc})
.first()
.then(result => {console.log(result)}); // <-- this never executes
};
let images = [];
const file = fs.readFileSync('./data.csv');
const lines = file.toString().replace(/[\r]/g, '').split('\n');
for (i=1; i<lines.length; i++) {
var row = lines[i].split(',');
var event_id = (async function () { return await get_event_id(row[0]) }) ();
images.push(createImage(row, event_id));
};
console.log(images);
// Inserts seed entries
// return knex('table_name').insert(images);
};
Output:
[ { name: '003.jpg', event_id: undefined } ]
My data.csv is structured like:
Location,Name
Amsterdam,003.jpg,
...

You can change your for loop into an Array.map and use Promise.all on returned promises.
Also, seed will return a promise so invoke it properly
exports.seed = async function (knex) {
const fs = require('fs');
function createImage(row, event_id) {
return {
name: row[1],
event_id: event_id
}
};
function get_event_id(loc) {
return knex('events')
.where({ location: loc })
.first()
.then(result => { console.log(result); return result }); // <-- this never executes
};
const file = fs.readFileSync('./data.csv');
const lines = file.toString().replace(/[\r]/g, '').split('\n');
let promises = lines.map(async (line) => {
let [row] = line.split(',');
let event_id = await get_event_id(row)
return createImage(row, event_id)
});
let images = await Promise.all(promises);
console.log(images);
// Inserts seed entries
// return knex('table_name').insert(images);
};

How to get rid of promise when getting json data

I'm trying to simply get data from a weather API. Here's my api link to get http://api.wunderground.com/api/5b81d144ae2d1942/conditions/q/46.838260,-71.293689.json
In my api.js file, i have this basic function :
const baseUrl = `http://api.wunderground.com/api/5b81d144ae2d1942/conditions/q`;
export const getCurrent = (lat,lon) => {
return fetch(`${baseUrl}/${lon},${lat}.json`)
.then((response) => response.json())
.then((json) => {
console.log(json.current_observation.weather)
return json.current_observation
})
.catch(() => {
console.error('unable to fetch tasks')
})
}
Notice the console.log, in this function i'm able to fetch the json data, i got the value i want.
Now, in my Vue, i call this function this way :
export default {
data: () => ({
current: []
}),
created: function () {
if (navigator.geolocation) {
navigator.geolocation.getCurrentPosition(this.showPosition);
}
},
methods: {
showPosition(position) {
const data = api.getCurrent(position.coords.longitude,position.coords.latitude);
this.current = data;
console.log(this.current);
}
}
}
For some reason, the console.log in here gives me this :
Promise {<pending>}__proto__:
Promise[[PromiseStatus]]: "resolved"
[[PromiseValue]]: Object
I don't know what's going on but i can't access the data. I searched on the net, a lot of pages talk about this, but couldn't find the exact solution, only long texts ...
Is there a solution for this (code please)
Thanks a lot.

To "get rid" of the Promise and access it's data, use .then() in .getCurrent()'s result, just like you are when using fetch():
methods: {
showPosition(position) {
api.getCurrent(position.coords.longitude,position.coords.latitude)
.then((data) => {
this.current = data;
console.log(this.current);
}
}
}
Alternatively, you could declare showPosition as async, and use await:
methods: {
showPosition: async function(position) {
const data = await api.getCurrent(position.coords.longitude,position.coords.latitude);
this.current = data;
console.log(this.current);
}
}
Just keep in mind the results of both executions will be processed asynchronously, meaning this.current will not have the value of data right away.

ReactJS - pushing to a list outside of a promise

I want to update a list which is defined outside of a promise (API call).
here is the code:
let data = [];
api.getData(this.state.ID).then((resp) => {
data = [...data, { title : resp.title }];
data = [...data, { name : resp.name }];
});
console.log(data); // --> it returns []
the data is [ ] and no value have pushed to it. how can I change my code to fix this issue? for some reasons I don't want to use setState()
Regards.

Promise is ALWAYS asynchronous, meaning that your data array will still be empty after you define the promise.
To get access to filled data array you have to refer to it inside the promise.
let data = [];
api.getData(this.state.ID).then((resp) => {
data = [...data, { title: resp.title }, { name: resp.name }];
console.log(data);
});

Async/Await will help you:
let data = [];
async function getData() {
await api.getData(this.state.ID).then((resp) => {
data = [...data, { title : resp.title }];
data = [...data, { name : resp.name }];
});
console.log(data); // --> it shouldn't return empty array
}
getData();
Example:
function resolveAfter2Seconds(x) {
return new Promise(resolve => {
setTimeout(() => {
resolve("You see it after promis end");
}, 2000);
});
}
var arr = [];
async function f1() {
await resolveAfter2Seconds(10).then(res => arr.push(res));
alert(arr);
}
f1();

Your console.log function is getting executed before api returns you data. Although data got updated after api call. You could not see it because it is consoled before that.
let data = [];
api.getData(this.state.ID).then((resp) => {
data = [...data, { title : resp.title }];
data = [...data, { name : resp.name }];
Console.log(data)
})
Now you can see data consoles.
But if you want to console the data after then method then you can make the execution of api synchronized by using npm module q.