Having some trouble mapping over this array to create elements. What I'm doing is fairly simple. I'm querying my database which returns and title and a URL. I then send that URL out to fetch the html from a website. I go through that response and pull out a bunch of json objects. Those objects are pushed to an array. Then I want to map through that array to post images to my screen. What I have read online is that you can't map over a constructed array. But if I'm not mistaken I'm doing that once and it works fine. But when it comes to pulling the variables from the array and using it to produce the img it doesn't work.
class Scraper extends Component {
arraystuff = [];
handleData = (data) => {
data.allSearches.map((datum) => {
// This is constructing the results array.
this.results.push({ title: datum.title, url: datum.url });
});
// Now I'm mapping over results without issue
this.results.map(async (post) => {
await fetch(
`http://localhost:4100/scrape/${encodeURIComponent(post.url)}`
)
.then((res) => res.json())
// Now I am constructing the arraystuff array. I cannot map over this in the render
.then((res) => res.map((result) => this.arraystuff.push(result)));
});
};
render() {
return (
<>
<Query query={ALL_SEARCHES_QUERY}>
{({ error, loading, data }) => {
if (error) return <p>Error!</p>;
if (loading) return <p>Loading...</p>;
this.handleData(data);
return null;
}}
</Query>
<div>
{this.arraystuff.map((search) => {
// This is what isn't working
<img
src={`http:${search.photo}`}
alt={search.id}
key={search.id}
/>;
})}
</div>
</>
);
}
}
export default Scraper;
Any help with this would be greatly appreciated. Doesn't seem like it should be that difficult. Driving me nuts
Related
I'm working on web scraping a news website to show top headlines and URLs. In my backend, I put each title and URL in an object within an array, which works fine. When I fetch this information in React, it is not working as expected.
I used the useState hook to initialize an empty array, where I would put each object containing the title and URL. Then, I map through that array and render each title and URL to the page.
When I refresh the webpage, it takes several seconds for each title and URL to pop up, and additionally, they are all the same title and URL.
It seems that my array is not being updated properly by putting in the same article information each time I set the state. However, I do not understand why it is taking so long for all the information to show up on the webpage, and do not know how to make it a faster process. I want all the information to be on the page after the user hits refresh, not appear after several seconds at a time. Could anyone help me see where I'm going wrong?
Here is my code:
import {useState} from 'react';
const News = () => {
const [articles, setArticles] = useState([])
fetch('http://localhost:8000/news')
.then(response => {return response.json()})
.then(data => {
data.forEach(article => {
setArticles([...articles, {
title: article.article_title,
url: article.article_url}])
})
})
.catch(err => console.log(err))
return (
<div>
{articles.map(article => {
return (
<div className="indv-article">
<h1 className="article-title" key={article.title}>{article.title}</h1>
<p className='article-url' key={article.url}>{article.url}</p>
</div>);
})}
</div>
)
}
export default News
Couple of things that may solve your issues.
First: In React, all side-effects (such as data fetching, for example) should be handled inside a useEffect hook.
Second: As stated in #Khorne07's answer, the key attribute should be on the root DOM node that is being returned from the map.
Third: I don't really know the purpose of looping through your data to set the state. If the reason you are doing this is because the response contains other information that you are not interested to display and you just want the title and url for each article, I suggest you to create an adapter function that will receive this data as a parameter and return just the information that you are interested in.
Additional: You can use a loading state to show a loading indicator while the data is being fetched and improve user experience.
Putting it all together:
const News = () => {
const [articles, setArticles] = useState([])
const [loading, setLoading] = useState(false)
const adaptArticles = (data) => {
return data.map(({ article_title, article_url }) => ({
title: article_title,
url: article_url
}))
}
useEffect(() => {
setLoading(true)
fetch('http://localhost:8000/news')
.then(response => {return response.json()})
.then(data => setArticles((prevArticles) => prevArticles.concat(adaptArticles(data))))
.catch(err => console.log(err))
.finally(() => {
setLoading(false)
})
}, []) //Insert the corresponding dependencies (if any) in the dependencies array so the useEffect hook gets executed when any of these dependencies change.
if(loading) //Return some loading indicator, like a Spinner for example.
return (
<div>
{articles.map(article => {
return (
<div className="indv-article" key={article.title}>
<h1 className="article-title">{article.title}</h1>
<p className='article-url'>{article.url}</p>
</div>);
})}
</div>
)
}
Edited:
You have some errors on your current code:
First of all, and, as mentioned by the accepted answer, all side effects like fetch calls should be inside a useEffect hook.
The second error is related to the way you are updating your state array. When your new state depends on the previous state value, you should use the callback function inside your setState function, in order to have your data correctly synchronized with the previous value. And in this particular example you are also calling a setState function inside a loop, which is a bad idea and can potentially drive your app into unexpected behavior. The best approach is described in the code snippet bellow.
fetch('http://localhost:8000/news')
.then(response => {return response.json()})
.then(data => {
const articlesArray = []
data.forEach(article => {
articlesArray.push({
title: article.article_title,
url: article.article_url
})
setArticles(currentArticles => [...currentArticles, ...articlesArray])
})
})
.catch(err => console.log(err))
And on the map function, the key attribute should be on the root node you are returning:
{articles.map(article => {
return (
<div className="indv-article" key={article.title}>{article.title}>
<h1 className="article-title"</h1>
<p className='article-url'</p>
</div>);
})}
To start with I'm a beginner. Any help would be appreciated.
So I'm getting my data from mongoDB atlas using node+express API. I'm successfull at getting the array to show up in console log using following code.
const [product, setProduct] = useState();
const url = "http://localhost:5000/api/items";
useEffect(() => {
axios.get(url).then((res) => {
setProduct(res.data);
// setProduct(
// JSON.stringify({
// title: setProduct.title,
// price: setProduct.price,
// image: setProduct.image,
// details: setProduct.details,
// })
// );
})
}, [url])
console.log(product)
The console log displays the array properly as collection named 'items' with content of arrays. As you can see I tried to stringify the response as the response returns JSON but again I didn't know how to map Following is the code where I tried to map the contents like id, name etc as props to component.
<div>
{product.map((product) => {
<Product name={product.title} />
})}
</div>
When I do this I get error that the map is not a function. I don't know what I'm doing wrong here. I know I'm supposed to use redux or reducer/context here but I want to get this to work before updating it with those.
[![Response from res.data][1]][1]
[1]: https://i.stack.imgur.com/auxvl.png
you didnt get yours products.
As we can see from screenshot
res.data equal to object with one property items:
res.data= {items: []}
and we need to take/access to these items
use this: setProducts(res?.data?.items || [])
const [products, setProducts] = useState();
useEffect(() => {
axios.get(url).then((res) => {
setProducts(res?.data?.items || []);
})
}, [url])
<div>
{products?.map((product) => {
<Product name={product.title} />
})}
</div>
On the first render the value for types will be undefined ( on sync code execution ), try using it as
<div>
{product?.map((product) => {
<Product name={product.name} />
})}
</div>
? will make sure to run map once value for types is there ( also make sure it is mappable ( is an array ))
I have been trying to run data fetching code in a container component and pass it to the display one to reduce using hooks and load time. I tried both await/sync and Meteor's wrapAsync Here is its Docs but I admit I dont really understand this function.
N.B: The code is pseudo code that explains the basic structure and functionality I need to apply, everything has been imported properly. The code structure and order is the same as the actual code. Thats what matters
Current Result
display component errors out as undefined because the data or t1Doc are empty
Container async/await
const Container = withTracker(({ id, ...rest }) => {
const t1Handle = Meteor.subscribe("table1");
const t2Handle = Meteor.subscribe("table2");
const isLoading = !t1Handle.ready() && !t2Handle.ready();
const fetchT1Documents = async () => {
const t1Doc = await t1Collection.findOne(query);
let t2Doc;
const fetchT2Documents = async () => {
t2Doc = await t2Collection.findOne(t1Doc.FK);
}
fetchT2Documents();
parseXmltoJs(t1Doc, (err, result) => {
fetchAjaxData(result,t2Doc)
.then((data) => {
return {
isLoading,
t1Doc,
t2Doc,
data
}
})
});
}
fetchT1Documents();
return {
isLoading,
t1,
...rest,
};
})(WithLoading(displayComponent));
Loader
const WithLoading = (Comp) => ({ isLoading, children, ...props }) => {
if (isLoading) {
return <Loader />;
} else {
return (
<Fade in={true} timeout={500}>
<div>
<Comp {...props}>{children}</Comp>
</div>
</Fade>
);
}
};
export { WithLoading };
sorry for the bad editing
A few tips on this one...
You can make a publication that publishes more than one collection/query, eg table1 and table2. It returns a single handle, that is ready when both collections have data.
No need to do an await on the collection.findOne() calls - these are calls to minimongo (client side cache), and will return either null when the subscription is loading, and will return the data otherwise.
The pattern you are using is excellent, and allows separation of display logic from the grubby data layer :)
I loop through this array like this:
{props.choosenMovie.characters.map((characters) => (
<p>{characters}</p> /* This displays the URL of course */
))}
These URL's include a name object which is what i want to display,
what is the best practice to do this?
This is how it is displayed on my application, but the desire is to display the name object from the URL's.
In useEffect, map thru your array of urls and make the api call and store the promises in an array. Use promise.all and update the state which will cause re-render.
In render method map thru the updated state and display the names.
see working demo
Code snippet
export default function App() {
const [char, setChar] = useState([
"https://swapi.dev/api/people/1/",
"https://swapi.dev/api/people/2/"
]);
const [people, setPeople] = useState([]);
useEffect(() => {
const promiseArray = [];
char.forEach(c => {
promiseArray.push(fetch(c).then(res => res.json()));
Promise.all(promiseArray).then(res => {
console.log("res", res);
setPeople(res);
});
});
}, []);
return (
<div className="App">
<h1>Hello CodeSandbox</h1>
{people.map((p, i) => {
return <p key={i}>{p.name}</p>;
})}
</div>
);
}
I was working with that API some time ago, and the way I approached it (to display the names etc) was with Promise.all
so the snipped looked like
axios.get(`https://swapi.dev/api/${this.props.match.path.split('/')[1]}/${this.props.match.params.id}/`).then((res) => {
let characters = []
// get all characters in the movie data
let characterPromises = []
res.data.characters.forEach((character) => {
characterPromises.push(axios.get(character))
})
// Create list with all characters names and link to page
Promise.all(characterPromises).then((res) => {
res.forEach((character, i) => {
characters.push(<li key={i}><Link to={`/${character.data.url.split('api/')[1]}`}>{character.data.name}</Link></li>)
})
this.setState({
characters
})
})
})
}
then I just used the characters lists (from state) in the render method
Hey I have been trying to figure out a fix for this for a while, I've searched here and all over but I've found nothing.
I have an API, it contains nested arrays, and is given through JSON.
I am able to get the contents inside the JSON to display when I use the console.log, but when I try to map the arrays, I constantly get errors stating that there's an issue with the .map function.
From what I have seen, it usually happens because it doesn't work with a string, but it's not being applied to a string from what I can tell...
There's a lot of data in the API, but I'm sure once I know how to get just one, eg the date, to display then I'll be fine with the rest.
Here is the code
export default class App extends React.Component {
constructor() {
super();
this.state = {
myItem: []
};
}
componentDidMount() {
this.getItems();
}
getItems () {
fetch('MYAPIGOESHERE')
.then(results => results.json())
//THIS BELOW WORKS
.then(results => console.log(results.date, results.location));
//THIS BELOW AND THE RENDER DOES NOT
.then(myItem => this.setState({myItem}))
}
render() {
return (
<div>
{this.state.myItem.map (myItem =>
<div> {myItem.date} </div> )}
</div>
)
})
}
}
Thank you!
{"date":"2018-09-16T11:22:00.000Z","location":"Cardiff City Stadium","teams":[{"name":"Cardiff City","homeTeam":true,"formation":"4-4-2","players":[{"playerId":"5afc0b73b481e9b536c4727b","position":"GK"},{"playerId":"5afc1377b481e9b536c4727c","position":"RB"},{"playerId":"5afc188bb481e9b536c47299","position":"CB"},{"playerId":"5afc188ab481e9b536c47297","position":"CB"},{"playerId":"5afc1872b481e9b536c4727e","position":"LB"},{"playerId":"5afc1873b481e9b536c4727f","position":"RM"},{"playerId":"5afc1874b481e9b536c47280","position":"CM"},{"playerId":"5afc1876b481e9b536c47281","position":"CM"},{"playerId":"5afc1876b481e9b536c47282","position":"LM"},{"playerId":"5afc1876b481e9b536c47283","position":"FW"},{"playerId":"5afc1877b481e9b536c47284","position":"FW"}]},{"name":"Swansea City","homeTeam":false,"formation":"4-3-3","players":[{"playerId":"5afc187ab481e9b536c4728a","position":"GK"},{"playerId":"5afc1878b481e9b536c47286","position":"RB"},{"playerId":"5afc1879b481e9b536c47289","position":"CB"},{"playerId":"5afc187ab481e9b536c4728b","position":"CB"},{"playerId":"5afc187bb481e9b536c4728c","position":"LB"},{"playerId":"5afc1879b481e9b536c47288","position":"RM"},{"playerId":"5afc1878b481e9b536c47287","position":"CM"},{"playerId":"5afc187bb481e9b536c4728d","position":"LM"},{"playerId":"5afc187cb481e9b536c47290","position":"FW"},{"playerId":"5afc187db481e9b536c47291","position":"FW"},{"playerId":"5afc187db481e9b536c47292","position":"FW"}]}]}
You can't map over this.state.myItem because it's an object.
You could do:
<div>{this.state.myItem.date}</div>
If you have an array of objects, you can do:
<div>{this.state.myItems.map(e => <div>{e.date}</div>)}</div>
for example.
If you have an object with an array in it, you can do:
const { teams } = this.state.myItem;
<div>{teams.map(t => <div>{t}</div>)}</div>
The data you have is like this: Object --> Array --> Object, so you can combine the above to display the data you want.
Try wrapping setState in braces:
.then(myItem => {
this.setState({myItem})
})
Map function works with arrays, you seem to have a single object. Try this:
return (
<div> {this.state.myItem.date} </div>
)
and remove the space between map and the opening parentheses.