I am using metaweather.com API to build a Web Application. I want to show 6 cities on the home page; I guess I have to call the API 6 time and push the data in an array like allCitiesDetails. How I have to do that? If there is a better way, please tell me. Here is my code :
state = {
city: {
cityname: this.props.value
},
woeid: '',
todayWeather: [],
weatherDetails: [],
allCitiesDetails: []
};
getCity = (cityName) => {
var self = this;
axios
.get('https://www.metaweather.com/api/location/search/?query=' + cityName)
.then(response => {
self.setState({
woeid: response.data[0].woeid
});
self.getWeather(response.data[0].woeid);
})
.catch(function(error) {
alert('No results were found. Try changing the keyword!');
});
}
getWeather = async (woeid) => {
const { data: weatherDetails } = await axios.get(
'https://www.metaweather.com/api/location/' + woeid
);
this.setState({
weatherDetails,
todayWeather: weatherDetails.consolidated_weather[0]
});
}
You should make 6 different promises and use Promise.all to get the weather of all 6 cities in parallel. You can do this as :
const getWeatherFromWoeid = cityName => axios.get(`https://www.metaweather.com/api/location/${woeid}`);
....
const p1 = getWeatherFromWoeid(woeid1);
const p2 = getWeatherFromWoeid(woeid2);
const p3 = getWeatherFromWoeid(woeid3);
const p4 = getWeatherFromWoeid(woeid4);
const p5 = getWeatherFromWoeid(woeid5);
const p6 = getWeatherFromWoeid(woeid6);
Promise.all([p1,p2,p3,p4,p5,p6])
.then(([result1, result2, result3, result4, result5, result6]) => {
...set result in the state
})
.catch((err) => {
...handle error
})
Also, always use catch if you're using promises or async
instead of using state inside the api call...
self.setState({
woeid: response.data[0].woeid
});
you can push the values in dummy array then outside the api call u can set state.
Related
I have a question, so i'm using AsyncStorage to store the user input data as a json format. However , while i'm checking whether the data are stored correctly using console.log, it always print out undefined, so i'm curious about how to access the data i store and print it out so that i can check if the data is correct? thanks!
Here's the json formate that i want the user input to store in
////JSON FORMAT////
const MyRecipeData = [
{
name: recipeName,
video_cover: selectedVideoCover,
video_url: UploadVideo,
servings: servingSize,
channel_name: channelName,
publish_date: uploadDate,
ingredients: ingredientsInput,
directions: directionsInput,
},
];
////JSON FORMAT////
and these are the function that called after the user pressing upload button, and i try to read it using getAllinput function, but not sure i did it right or not
////------- Save all DATA --------------------////
const SaveAllInput = async () => {
await AsyncStorage.setItem("MyRecipeData", JSON.stringify(MyRecipeData))
.then(() => {
alert("your Recipe " + MyRecipeData.name + " has been saved");
})
.catch(() => {
console.log("error");
});
getAllInput();
};
////------- Save all DATA --------------------////
////------- READING THE DATA THAT UPLOAD PREVIOUSLY-------- /////
const getAllInput = async () => {
try {
const NewRecipeData = await AsyncStorage.getItem("MyRecipeData");
NewRecipeData !== null ? JSON.parse(NewRecipeData) : null;
console.log(NewRecipeData);
return NewRecipeData;
} catch {
console.log(error);
}
};
////------- READING THE DATA THAT UPLOAD PREVIOUSLY-------- /////
the console.log(NewRecipeData) print out [{}] in my terminal, seems like i did not read my data properly
i tried to use getItem to read it out, but instead i got undefined or [{}]
const NewRecipeData = await AsyncStorage.getItem("MyRecipeData");
NewRecipeData !== null ? JSON.parse(NewRecipeData) : null;
You using const and you are redefining the variable, try to console.log like this :
const NewRecipeData = await AsyncStorage.getItem("MyRecipeData");
console.log(NewRecipeData);
You are caliing getAllInput(); without await
const SaveAllInput = async () => {
const MyRecipeData = [
{
name: recipeName,
video_cover: selectedVideoCover,
video_url: UploadVideo,
servings: servingSize,
channel_name: channelName,
publish_date: uploadDate,
ingredients: ingredientsInput,
directions: directionsInput,
},
];
await AsyncStorage.setItem('MyRecipeData', JSON.stringify(MyRecipeData))
.then(() => {
alert('your Recipe ' + MyRecipeData.name + ' has been saved');
})
.catch(() => {
console.log('error');
});
await getAllInput();
};
how to consume from one api with another api.
var url_1 = 'https://dog.ceo/api/breeds/list/all';
fetch(url_1)
.then( response => response.json())
.then(data => {
const breeds = data.message;
var arr = [];
for (var b in breeds) {
arr.push({
breed : b,
subBreeds : [
breeds[b][0]
],
images : [{
url: ''
}]
})
}
I also have this other api, from where I extract the images of each breed of dog, but here you need the variable that would be the name of the dog's breed.
var url_2 = 'https://dog.ceo/api/breed/{breed_name}/images';
fetch(url_2)
.then( response => response.json())
.then(data => {
const images = data.message;
var arr_images = [];
for (var i in images) {
arr_images.push({
images : [{
url: images[i]
}]
})
}
So what I don't know, how can I join to send the name of the dog's breed to the second api to consume it?
And how can I join the arrangement of the images with the arrangement above?
it should be something like this
{ "breed": "Hound",
"subBreeds": [
"subBreedA",
"subBreedB",
"subBreedC"
],
"images":[
{"url":"http://some.url.com"},
{"url":"http://some.other.url"}
]
}
I hope I have been clear, thanks for your help, I will be very grateful.
I would split it up into separate functions so that you can focus on one part at a time. Then, combine them to get all of the data that you want. In this way, you can also re-use each function in case you want to use the data in a different way:
TS Playground
// dog.ceo API
async function fetchDogApiResult (apiPath) {
const response = await fetch(`https://dog.ceo/api/${apiPath}`);
if (!response.ok) throw new Error(`Response not OK (${response.status})`);
const data = await response.json();
if (data.status !== 'success') throw new Error('Response not successful');
return data.message;
}
async function fetchBreeds () {
return fetchDogApiResult('breeds/list/all');
}
async function fetchSubBreeds (breed) {
return fetchDogApiResult(`breed/${breed}/list`);
}
async function fetchImages (breed, subBreed) {
return fetchDogApiResult(`breed/${breed}${subBreed ? `/${subBreed}` : ''}/images`);
}
async function fetchDogData () {
const breeds = await fetchBreeds();
return Promise.all(Object.entries(breeds).map(async ([breed, subBreeds]) => ({
breed,
subBreeds,
images: (await fetchImages(breed)).map(url => ({url})),
})));
}
(async () => {
const dogData = await fetchDogData();
console.log(JSON.stringify(dogData));
})();
You can use async/await for call second api in second then of first api, after you get data from second api, you can use for loop for them. like this
var url_1 = 'https://dog.ceo/api/breeds/list/all';
fetch(url_1)
.then( response => response.json())
.then(async data => {
const breeds = data.message;
const resUrl2 = await fetch(url_2)
const dataUrl2 = await resUrl2.json()
var arr = [];
for (var b in breeds) {
arr.push({
breed : b,
subBreeds : [
breeds[b][0]
],
images : [{
url: ''
}]
})
}
const images = dataUrl2.message;
var arr_images = [];
for (var i in images) {
arr_images.push({
images : [{
url: images[i]
}]
})
}
})
I'm doing a project that fetch different types of data from SWAPI API (people, planets, etc.) using react but I have an issue with multiple Ajax request.
The problem is when I quickly request from 2 different URL for example, 'species' and 'people', and my last request is 'species' but the load time of 'people' is longer, I will get 'people' instead.
What I want is to get the data of the last clicked request, if that make sense.
How do I achieve that? All the solution I found from Google is using jQuery.
Here's a slice of my code in src/app.js (root element) :
constructor(){
super();
this.state = {
searchfield: '',
data: [],
active: 'people'
}
}
componentDidMount() {
this.getData();
}
componentDidUpdate(prevProps, prevState) {
if(this.state.active !== prevState.active) {
this.getData();
}
}
getData = async function() {
console.log(this.state.active);
this.setState({ data: [] });
let resp = await fetch(`https://swapi.co/api/${this.state.active}/`);
let data = await resp.json();
let results = data.results;
if(data.next !== null) {
do {
let nextResp = await fetch(data.next);
data = await nextResp.json();
let nextResults = data.results
results.push(nextResults);
results = results.reduce(function (a, b) { return a.concat(b) }, []);
} while (data.next);
}
this.setState({ data: results});
}
categoryChange = (e) => {
this.setState({ active: e.target.getAttribute('data-category') });
}
render() {
return (
<Header searchChange={this.searchChange} categoryChange={this.categoryChange}/>
);
}
I made a gif of the problem here.
Sorry for the bad formatting, I'm writing this on my phone.
You have to store your requests somewhere and to abandon old ones by making only one request active. Something like:
getData = async function() {
console.log(this.state.active);
this.setState({ data: [] });
// my code starts here
if (this.controller) { controller.abort() }
this.controller = new AbortController();
var signal = controller.signal;
let resp = await fetch(`https://swapi.co/api/${this.state.active}/`, { signal });
let data = await resp.json();
let results = data.results;
if(data.next !== null) {
do {
let nextResp = await fetch(data.next);
data = await nextResp.json();
let nextResults = data.results
results.push(nextResults);
results = results.reduce(function (a, b) { return a.concat(b) }, []);
} while (data.next);
}
this.setState({ data: results});
}
I am using cheerio and node to do web scraping, but I have a problem with promises. I can scrape an article list from a page but in that list, we have more links for single pages. I need to scrape single pages as well for each item on the list.
I will show you my code for the better solution.
import rp from 'request-promise'
import cheerio from 'cheerio'
import conn from './connection'
const flexJob = `https://www.flexjobs.com`
const flexJobCategory = ['account-management', 'bilingual']
class WebScraping {
//list of article e.g for page 2
results = [] // [[title], [link for page],...]
contentPage = [] //content for each page
scrapeWeb(link) {
let fullLink = `${link}/jobs/${flexJobCategory[1]}?page=2`
const options = {
uri: fullLink,
transform(body) {
return cheerio.load(body)
}
}
rp(options)
.then(($) => {
console.log(fullLink)
$('.featured-job').each((index, value) => {
//html nodes
let shortDescription = value.children[1].children[1].children[3].children[1].children[1].children[0].data
let link = value.children[1].children[1].children[1].children[1].children[1].children[0].attribs.href
let pageLink = flexJob + '' + link
let title = value.children[1].children[1].children[1].children[1].children[1].children[0].children[0].data
let place = value.children[1].children[1].children[1].children[1].children[3].children[1].data
let jobType = value.children[1].children[1].children[1].children[1].children[3].children[0].children[0].data
this.results.push([title, '', pageLink.replace(/\s/g, ''), '', shortDescription.replace(/\n/g, ''), place, jobType, 'PageContent::: '])
})
})
.then(() => {
this.results.forEach(element => {
console.log('link: ', element[2])
this.scrapePage(element[2])
});
})
.then(() => {
console.log('print content page', this.contentPage)
})
.then(() => {
//this.insertIntoDB()
console.log('insert into db')
})
.catch((err) => {
console.log(err)
})
}
/**
* It's going to scrape all pages from list of jobs
* #param {Any} pageLink
* #param {Number} count
*/
scrapePage(pageLink) {
let $this = this
//console.log('We are in ScrapePage' + pageLink + ': number' + count)
//this.results[count].push('Hello' + count)
let content = ''
const options = {
uri: pageLink,
transform(body) {
return cheerio.load(body)
}
}
rp(options)
.then(($) => {
//this.contentPage.push('Hello' + ' : ');
console.log('Heloo')
})
.catch((err) => {
console.log(err)
})
}
/**
* This method is going to insert data into Database
*/
insertIntoDB() {
conn.connect((err) => {
var sql = "INSERT INTO contact (title, department, link, salary, short_description, location, job_type, page_detail) VALUES ?"
var values = this.results
conn.query(sql, [values], function (err) {
if (err) throw err
conn.end()
})
})
}
}
let webScraping = new WebScraping()
let scrapeList = webScraping.scrapeWeb(flexJob)
So, at 'scrapeWeb' method, at second '.then', I am calling 'scrapePage' method, however, the third promise executed before promise inside 'scrapePage' method.
You need a little more control flow at that stage. You do not want that .then()'s promise to resolve until all the calls are resolved.
You could use a Promise library like bluebird to do a Promise.each or a Promise.map for all the results you want to run.
Or use async/await to set up like .then(async () => {}) and do not use .forEach.
for(let element of this.results){
console.log('link: ', element[2])
await this.scrapePage(element[2])
}
You have a race condition problem.
The first tweak you'll need is having scrapePage returning a Promise.
scrapePage(pageLink) {
let $this = this
let content = ''
const options = {
uri: pageLink,
transform(body) {
return cheerio.load(body)
}
}
return rp(options);
}
In the second than, you need to invoke all child pages scraping eg :
.then(() => {
return Promise.all(this.results.map(childPage => this.scrapePage(childPage)));
})
This will wrap all scrapes of child pages into promises and only if all of them are resolved the code will flow.
I'm new to NodeJS and I have a problem I don't understand.
In this function, I call several API one after another to retrieve some data about a movie. The result isn't always the same. Most of the time, the result is correct, but sometimes the result isn't complete.
I tried using then to try and chain the API calls but it doesn't seem to work.
Any idea why the result isn't always the same? Any help would be appreciated.
// test fetchData(456165)
function fetchData(filmid) {
let average = array => array.reduce((a, b) => a + b) / array.length
var notes = []
mdb.movieInfo({
id: filmid,
language: 'fr'
},
(err, resOmdb) => {
notes.push(parseFloat(resOmdb.vote_average))
imdb
.getById(resOmdb.imdb_id, {
apiKey: 'e9d59b68',
timeout: 3000
})
.then(
allocine.api(
'search', {
q: `${resOmdb.title}`,
filter: 'movie'
},
function(error, resAllo) {
if (error) {
return
}
allocine.api(
'movie', {
code: `${resAllo.feed.movie[0].code}`
},
function(error, result) {
if (error) {
return
}
notes.push(parseFloat(result.movie.statistics.userRating) * 2)
}
)
// doesn't seem to execute all the time
allocine.api(
'showtimelist', {
zip: 44260,
movie: resAllo.feed.movie[0].code
},
function(error, resultCin) {
if (error) {
return
}
// sometimes doesn't appear in the result
resOmdb.cinemas = resultCin
}
)
}
)
)
.then(
function(result) {
notes.push(parseFloat(result.rating))
resOmdb.vote_average = average(notes).toFixed(2)
// check the result
console.log(util.inspect(resOmdb, false, null))
},
function(error) {
return
}
)
}
)
}
First of all you should decide if you want to use Promises or not.
If you do, promisify all functions. Next thing you need to do is 'return' your promises if they are used inside a function.
In your case your first imbd api call is not returned probably.
As next thing you should check if your node version supports async await.
Then you can easily do your api calls without any distractions.
'use strict';
const Promise = require('bluebird');
const mdb = Promise.promisfyAll(require('mdb'));
const allocine = Promise.pomisifyAll(require('allocine-api'));
// test fetchData(456165)
async function fetchDate(filmId) {
const notes = [];
const resOmdb = await mdb.movieInfoAsync({ id: filmId });
notes.push(parseFloat(resOmdb.vote_average));
const imdbResult = await imdb.getByIdAsync(resOmdb.imdb_id, { apiKey: 'e9d59b68', timeout: 3000 });
const resAllo = await allocine.apiAsync('search', { q: `${resOmdb.title}`, filter: 'movie' });
// and so on ...
}
UPDATE:
To speed up your function you can do requests concurrently.
To do so, use Promise.join
const [imdbResult, allocineResult] = await Promise.join(
imdb.getByIdAsync(resOmdb.imdb_id, { apiKey: 'e9d59b68', timeout: 3000 }),
allocine.apiAsync('search', { q: `${resOmdb.title}`, filter: 'movie' });
);