I am running 3 API requests and they are making my JS script very slow.
The goal is to fetch data from the API and push it my database.
First 2 API is called using date and next_page_token. So until there is a next page token i keep calling these 2 APIs. I call them recursively.
I store the id that i get from above in an array and pass it to the next 2 APIs.
The last API call run in a loop. I loop through the ids and call the API each time.
Code:
export async function getFirstAPI(access_token, start, end, globalObject){
let url = 'baseAPI/meetings/from=start/to=end/next_page_token=globalObject.next_page_token';
var obj = {
method: 'GET',
headers: {
authorization: 'Bearer {yourtokenhere}'
}
}
let response = await fetch(url, obj)
let data = await response.json()
return data
}
export async function getSecondAPI(access_token, start, end, globalObject){
let url = 'baseAPI/chats/from=start/to=end/next_page_token=globalObject.next_page_token';
var obj = {
method: 'GET',
headers: {
authorization: 'Bearer {yourtokenhere}'
}
}
let response = await fetch(url, obj)
let data = await response.json()
return data
}
export async function getThirdAPI(access_token, id_array, globalObject){
for(let i=0; i<id_array.length; i++){
let url = 'baseAPI/participants/{id}';
var obj = {
method: 'GET',
headers: {
authorization: 'Bearer {yourtokenhere}'
}
}
let response = await fetch(url, obj)
let data = await response.json()
globalObject.store_data.push(data)
}
return globalObject
}
When i run the above for a single day. That alone takes 14min 20sec. If i run it for a wider date range i am guessing this will go on for hours!!
Is there a way i can optimize this code? Or is it supposed to take this much time to run?
Any suggestions would be great!!
Switch from await in your loop to Promise.all or Promise.allSettled. It will work much faster. Await is not a silver bullet. It helps but in your case you are waiting for a response on each iteration instead of.. something like "fire all and collect results"
export async function getThirdAPI(access_token, id_array, globalObject) {
const apiPromises = id_array.map(async (id) => {
let url = `${baseAPI}/participants/${id}`;
var obj = {
method: 'GET',
headers: { /* */ },
authorization: `Bearer ${access_token}`
}
const response = await fetch(url, obj);
return await response.json();
});
const results = await Promise.all(apiPromises);
// in case globalObject.store_data is an array
globalObject.store_data.push(...results);
return globalObject;
}
Related
I'm using javascript and calling an API which returns some JSON. For some reason, when I assign the JSON to a variable, it removes some of the data. If I console log the variable (in this case data2) it is not the same output as when I just put a function JSON.parse(data) inside the console log.
let data = await res.text();
console.log("data1: ",JSON.parse(data))
const data2 = JSON.parse(data);
console.log("data2: ",data2);
Both console logs should be the same but they are not.
I've tried everything to figure this out but in the end my understanding is the console should be printing the exact same thing
All of this code is in a function so it shouldn't be reused or mutated (that I'm aware of). The full function is here (cleaned up and using res.josn())
const post = async (url, params) => {
const res = await fetch(BASE_URL + url, {
method: "POST",
headers: {
"Content-Type": "application/json",
},
body: JSON.stringify(params),
});
const data = await res.json();
console.log("data1: ",data)
return data1;
};
In this case, the data variable is incomplete and not showing the full response
I'm relatively new when it comes to Typescript/Javascript, but I'm trying to set all indices of this.articles[i].result within setBias() equal to setBiasedRating(this.articles[i].url);, the function containing the API call. So far I have this:
async getBiasedRating(url) {
const response = await fetch(
'https://news-bias-detection-api.herokuapp.com/',
{
method: 'POST',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
},
body: `{
"url": "${url}"
}`,
}
);
const data = await response.json();
return data;
}
async setBias() {
for (let i = 0; i < this.articles.length; ++i) {
try {
this.articles[i].result = await this.getBiasedRating(
this.articles[i].url
);
} catch (e) {
throw new Error("Error fetching bias news data");
}
}
}
}
Here, when I loop through all the indices in the this.articles array and set the i'th index equal to the response I get from getBiasedRating(this.articles[i].url), I see many repeating POST 500 (INTERNAL SERVER ERROR)'s.
I have no idea why this is, since if I have the same functionality, just with the first index instead of looping through everything:
async setBias() {
this.articles[0].result = await this.getBiasedRating(this.articles[0].url)
}
This works just fine, and the data for this.articles[0] gets stored correctly. If anyone could explain why the for loop doesn't work here, it would be greatly appreciated.
HTTP ERROR 500 isn't problem in your code, it's a server problem.
I'm using React Query and I'm not sure why one of my useQuery hooks is logging undefined while the other one is logging the correct data. They're both async functions and are literally doing the same thing. I've been trying to debug this for a while but to no avail, I've hit a wall.
How can I fix it so that console.log(winnersData); also logs the correct data just like console.log(data);?
Note: Yes, fetchUploads() is returning data correctly when testing on Postman so we can rule out that nothing's being returned.
async function fetchUploads(){
const headers = {
"Accept": 'application/json',
"Authorization": `Bearer ${authToken}`
};
const {data} = await axios.get('http://localhost/api/get-user-uploads-data', {headers});
return data
}
async function fetchWinners(){
const headers = {
"Accept": 'application/json',
"Authorization": `Bearer ${authToken}`
};
const {winnersData} = await axios.get('http://localhost/api/choose-winners', {headers});
return winnersData
}
const { data } = useQuery('uploads', fetchUploads)
const { winnersData } = useQuery('winners', fetchWinners)
console.log(data); // returns correct data
console.log(winnersData); // returns undefined
return(...);
I think the confusion revolves around your first return being called data. Coincidentally, Axios requests return an object with a property called data, which you are successfully destructuring. They don't have a property called winnersData, hence const {winnersData} = await axios.get('http://localhost/api/choose-winners', {headers}); is undefined.
Edit:
async function fetchUploads(){
const headers = {
"Accept": 'application/json',
"Authorization": `Bearer ${authToken}`
};
const {data} = await axios.get('http://localhost/api/get-user-uploads-data', {headers});
return data
}
async function fetchWinners(){
const headers = {
"Accept": 'application/json',
"Authorization": `Bearer ${authToken}`
};
// const {winnersData} = await axios.get('http://localhost/api/choose-winners', {headers});
// return winnersData
// Remember, there is no winnersData on the object Axios is returning
const {data} = await axios.get('http://localhost/api/choose-winners', {headers});
return data
// There is a data property for you to destructure (as there is on every successful Axios return), so we'll return that!
}
const data = useQuery('uploads', fetchUploads)
const winnersData = useQuery('winners', fetchWinners)
// I'm not sure what the shape of the data returned by these functions is, so I took out the destructuring. Remember, you can't destructure a property that isn't there.
console.log(data); // returns correct data
console.log(winnersData); // returns undefined
return(...);
Hi i would like some help on authorize my token to json as if i use ajax where i store my api key under my storage it will work but in json due to there no storing in json so i not sure how to work
here the code for the script this dont work
<script>
$.ajax({
url:"https://ecoexchange.dscloud.me:8080/api/get",
method:"GET",
// In this case, we are going to use headers as
headers:{
// The query you're planning to call
// i.e. <query> can be UserGet(0), RecyclableGet(0), etc.
query:"RecyclableGet(0)",
// Gets the apikey from the sessionStorage
apikey:sessionStorage.getItem("apikey")
},
success:function(data,xhr,textStatus) {
const buildTable = data => {
const table = document.querySelector('.table tbody');
for (let i = 0; i < data.length; i++) {
let row = `<tr>
<td>${data[i].RecyclableID}</td>
<td>${data[i].Name}</td>
<td>${data[i].RecyclableType}</td>
</tr>`;
table.insertAdjacentHTML('beforeEnd', row);
}
};
const getData = async(url) => {
const response = await fetch(url);
const json = await response.json();
return buildTable(json);
};
getData('https://ecoexchange.dscloud.me:8080/api/get');
$(document).ready(function() {
const btns = $('.change-row');
$("tbody").on('click', 'tr', function(e) {
/* //Add this line if you want only a single row selected
$('tbody tr').not($(this)).removeClass('highlight');
*/
$(this).toggleClass('highlight');
if ($('tbody tr').hasClass('highlight')) {
btns.prop('disabled', false).removeClass('disabled dark');
} else {
btns.prop('disabled', true).addClass('disabled dark');
}
});
});
},
error:function(xhr,textStatus,err) {
console.log(err);
}
});
if use ajax code for authorize this works
$.ajax({
url:"https://ecoexchange.dscloud.me:8080/api/get",
method:"GET",
// In this case, we are going to use headers as
headers:{
// The query you're planning to call
// i.e. <query> can be UserGet(0), RecyclableGet(0), etc.
query:"RecyclableGet(0)",
// Gets the apikey from the sessionStorage
apikey:sessionStorage.getItem("apikey")
},
success:function(data,xhr,textStatus) {
console.log(data);
},
error:function(xhr,textStatus,err) {
console.log(err);
}
});
i try to store inside ajax to call but it don't work as is calling the json instead causing 401 does any know how fix this issue ?
Hi i have solve the solution by
const getData = async(url) => {
const response = await fetch(url, {
method: 'GET',
headers: {
query:"RecyclableGet(0)",
// Gets the apikey from the sessionStorage
apikey:sessionStorage.getItem("apikey")
}
});
const json = await response.json();
return buildTable(json);
};
getData('https://ecoexchange.dscloud.me:8080/api/get');
For future notice if people wan to figure out
I am trying to put fetch functions into a separated file, so I can organise these API fetch easily. However, when I try to fetch and return the data, it gives me null or an unexpected json object. Here is part of my src:
//api.js
export async function LoginAPI(username, password) {
const url = baseURL + "/login/";
var params = {username: username, password: md5.hex_md5(password)};
let response = await fetch(url, {
method: 'POST',
headers: {'Accept': 'application/json','Content-Type': 'application/x-www-form-urlencoded'},
body: JSON.stringify(params)
});
return await fetch(url, {
method: 'POST',
headers: header,
body: JSON.stringify(params)
})
.then((res) => res.text())
.then((text) => text.length ? JSON.parse(text) : {})
.catch((error) => {
throw error;
});
};
Here is the another file.
//index.js
var Login = React.createClass({
onPress_login: function() {
var value = this.refs.form.getValue();
if (value) {
result = LoginAPI(value.username, value.password);
console.log(result);
} else {
this.AlertDialog.openDialog();
}
},
render() {
return (
(...)
<Button onPress={this.onPress_login}>
<Text>Login</Text>
</Button>
The fetch is working, it is communicating with the server. However, the console log returns me this at the first place
Promise _45: 0_54: null _65: null _81: 1 __proto__: Object
I am assuming that the result log in the console at the first place is not the await result (the actual response from server, but an object of fetch response). I tried to search out methods online but I can't find any post/blog/article saying how to do fetch as a function call.
Is there any way to do like swift, LoginAPI(username, password, callback: {...}) please?
The problem is that you're are making an async function and not waiting for the response, the you see that kind of console log.
Try this:
result = await LoginAPI(value.username, value.password);
Let me know if this was your problem.