We are implementing a Google Site Search for a client, and need access to all of the results for custom result output.
Currently only 10 results are returned at a time, is there a way to retrieve more than 10, preferably the entire result set.
Are you scraping the web site? If so, turn off Google Instant because that option limits you to 10 search results. In your search settings, you will be able to set the number of returned results. Obviously, you can't return the entire result set, though.
Related
For example, I have the following URL:
https://www.website.com/cf/script.php?page=1
This returns a large JSON object with 50 entries. ?page=2 returns the next 50, so on and so forth. What would be the best method to cycle through all given pages (say for example I want to go from pages 1-200) and store all the results in a separate table/database? Preferable using Javascript.
Have a look at the API documentation, as many APIs provide a way for you to specify the amount of entries in the response. E.g. limit=50, take=50. There is probably a maximum amount though.
There is also often other paging information in the response, i.e. how many more pages there are.
If that exists, you can create a loop that fetches each page, and creates/stores the results somewhere. Or you can just fetch and increment page until there are no more results.
I am using the food2fork target="_blank" API to load search results onto a page. However, I run into a problem when I try to do pagination. I can only get 30 results at a time, and I don't know how to find out the total number of search results possible either. Does anyone know how I can achieve pagination for this or if it's even possible?
I built this with angular + node, hosted on heroku, if this makes a difference.
(Right now I've got it limited so that users can search up to three pages of their desired search, but it's hardcoded into the site so it's problematic for searches that give more or less than exactly 3 pages worth of results. I could have only 'prev' and 'next' buttons, but I feel that's also limiting.)
As said in the doc:
Pages (Search Only)
Any request will return a maximum of 30 results. To get the next set of results send the same request again but with page = 2
The default if omitted is page = 1
Il you want to fetch the results ranging from 31 to 60, you need to pass page=2 in the request. It looks like the API doesn't provide the total number of results.
I don't subscribe to #Arashsoft proposal. It actually defeats the purpose of pagination which is not to load the full resultset. What would be the performances if you have thousands of recipes ?
But with this simple API, you could implements infinite scrolling for instance.
If you cannot get more than 30 results form the API, I suggest to call the API in a loop until you get all the data (30, 60, 90, ...). Then you can paginate it easily for your end user.
I have frontend AngularJS application that connects to backend API.
I'd like to implement search engine in my SPA so that I can specify
offset - the number of elements displayed on each page
page - number of current page
count - total elements
I also want to sort the results by specified attribute and filter them out as well.
What are the patterns to implement such search? What should be done on backend and what on frontend? How should API look like?
I have a lot of issues with that - what if user requested for 5th page with offset 10, and then change offset to 20, which page should be displayed then? When to fetch additional data? What if I set offset to 15 and then filter out some results, what will give me only 5 elements? Should I fetch missing by current filter?
Can you provide some behaviors of that mechanisms?
I'm using javascript and php.
Instagram's API will only return 20 results at a time. I can get a user's feed OR tags from all users but not a combination of both.
For example:
If I do a search for all the tags with "selfie", but only want results from the users #ladygaga and #justinbieber, Instagram will only return 20 results from everybody with #selfie. Since the tag "selfie" is so widely uses, it's extremely unlikely that any of those results will have #ladygaga or #justinbieber in them.
In which way can I use pagination or any other method to filter search results by tag and user.
I'm using the YouTube API to generate a page that loads YouTube videos. The stack that I'm using is HTML, CSS, and AngularJS. I want a button that will generate a random video given a search query. The way that I was planning to do this is to use the pageToken attribute.
I noticed that the token "CAEQAA" always returns the second page of search results of the query. And following that, "CAIQAA" gives the next page of search results after that. So this makes me think that these keys are independent of the search query.
However, this might be specific to my search options (one video per page of search results, safe search = strict, etc) even if it is independent of the search query. Is there a way to retrieve all the page tokens possible in a list or some form? This way, I can select a random token from this list to pick a random page of search results and thus a random video.
If I am misunderstanding how this works, please let me know as I am new to using this sort of API. Any help is appreciated. Thanks
I wrote an algorithm that can generate a pageToken for any given number in the range [0, 100000] ( you can install with npm install youtube-page-token).
With the package you can
1) fetch the first page of results
2) get a total count
3) get a random number in that total range
4) generate a token for that number
5) plug back into YouTube API