This Stackoverflow answer states that Google's Custom Search API allows to search the entire web.
I don't really care about the result order, but would like to...
On my own website, using browser JavaScript or at least NodeJS, do a image search for the whole web using filters like these and process the result in JavaScript (displaying the result on my site)
also do a reverse image search (find images similar to a specific image URL)
To be clear, I do not want to a search based on the content of my website, but instead only show the worldwide results myself.
Is that possible in a legal way (not by scraping the Google website)?
Note I'm only interested in public domain / CC0 images (Google is able to filter images by license).
If you're using Node.js (as the question is tagged), there is an npm module called fotology which does what you're asking. It even allows filtering by license:
let options = {
size: "large", // large images only
language: "fr", // French
safe: true, // force safe search on
color: "white", // white cats only please
rights: "cc_publicdomain" // Only public domain images
}
fotology("cats", options)
.then(function (imageURLs) {
for (i in imageURLs)
console.log imageURLs[i];
});
Disclaimer: I'm the author of that particular version of the module (original module didn't allow filtering by image license)
As this API is not available anymore, you can use our third-party service like SerpApi to scrape Google and get back structured JSON.
Example using the Node.js library:
var gsr = require('GoogleSearchResults')
let serp = new gsr.GoogleSearchResults("demo")
serp.json_with_images({
q: "Cats Meme",
location: "Portland"
}, (result) => {
console.log(result)
})
The only image search API that I'm aware of today (2018) is the Bing API. I used Yahoo and Google in the past, but they both discontinued operations.
You get 1000 free searches per month, and it's about $5/1000 searches thereafter. I just implemented it last night; it works quite well.
Related
The Goal and the Problem
I am working on a sharing solution in our React-based video and audio search apps. My goal is that when sharing a single item from a search result set to a social media app (e.g. Facebook, Messenger, LinkedIn, Twitter) and then clicked by someone else, the user lands within our search app/experience with only a single result showing (the shared item), which is done using query parameters in our search apps. Imagine using Google image search, and creating a little share icon on each search result so that when hovered/clicked/tapped and shared to social media then clicked by someone on said social media channel, the user lands in Google image search with just that single shared item in the result set.
We've been challenged to find a third party solution that uses custom query parameters for share links on each search result to help us achieve our goal. They all seem to use the base url structure (window.location.origin + window.location.pathname e.g. https://www.clipstock.com/audio) which won't work properly to show a single search result within our search experience which needs a query parameter as shown in the example below. Note: this url approach works fine for sharing a full page "node" with just the single video or audio result on the page, which we're also doing separately.
We've built a custom solution in our React based search apps that does achieve our goal of having the user land within our search app with just a single search result although we expect our custom solution to break if/when sharing urls in social apps update which seems to happen yearly or so. We figure a third party JavaScript library or third party social share service will fix these issues faster and cheaper without our development time if/when the sharing URLs break.
Has anyone seen a third party solution to this?
Examples
Example link for landing on a single search result in our audio search app, achieved by the search query q="Reunited" in the url: https://www.clipstock.com/audio?q=%22Reunited%22&size=n_12_n
Example share link of our custom solution for sharing the above search result on Facebook today, which seems to change by the year (note the https://www.facebook.com/sharer/sharer.php?u= part of the url, which is the part that may break from Facebook updates): https://www.facebook.com/sharer/sharer.php?u=https://www.clipstock.com/audio?q=%22Reunited%22&size=n_12_n
Example link of a single video clip node page, which works well with any third party sharing solution, although isn't the goal of this question which is to land in our search app(s) with just the single shared item in the result set which is achieved using query parameters as seen in the examples above: https://www.clipstock.com/clip/young-male-swimmer-diving-starting-block-olympic-size-pool-lanes-slow-motion
The below approach to sharing works for us, although is the custom sharing solution we created that may break over the years. In this example, on mobile, the Facebook app opens, and on non-mobile, the Facebook web app opens, which is our goal. Either of the Facebook share urls (and the share urls for each other social media app like Messenger, LinkedIn, Twitter) could change over time since they're essentially API integrations and we think it would be cheaper & faster to invest in a third party sharing solution than to monitor for these breaks and fix the breaks ourselves.
Note: some setup code in the below example is not included such as the DrupalContext. We aren't getting errors in the code, rather, we're looking for a third-party solution so that we don't have to maintain custom code.
const drupalContext = useContext(DrupalContext);
const queryString = type === "sfx" ? drupalContext.sfxQueryString : "";
const baseUrl = window.location.origin.concat(window.location.pathname);
const sharingLink =
type === "video"
? baseUrl.concat("?q=").concat(nodeId)
: baseUrl
.concat("?q=")
.concat(encodeURI('"'))
.concat(encodeURI(title))
.concat(encodeURI('"'))
.concat(queryString);
const shareViaFacebook = () => {
let shareUrl = "";
if (platform?.os?.family === ("iOS" || "Android" || "Windows Phone")) {
shareUrl = "fb://share/?link=" + sharingLink;
} else {
shareUrl = "https://www.facebook.com/sharer/sharer.php?u=" + sharingLink;
}
window.open(shareUrl);
};
We've created similar shareVia... methods for each social media app we're sharing to which currenly consists of Facebook, Messenger, LinkedIn, Twitter, Email, and "Copy Link". Email and "Copy Link" are what we're going to fall back to using if we can't find a reliable third-party solution (removing Facebook, Messenger, LinkedIn, and Twitter sharing options) since Email and "Copy Link" reliably work and we don't expect these two options to break over time.
I successfully followed Microsoft's tutorial to create an extension.
I'm trying to get all the work-items of certain sprint, but to be honest, I'm lost...
I'm not sure what to look for - I have the VSS object, with which I can require additional services (such as TFS/WorkItemTracking/Services or TFS/WorkItemTracking/RestClient).
I found some examples like this one, but couldn't find an API to retrieve or query work items.
Do I need a JS object for that, or is it accomplished via some REST call?
You are nearly there.
You need the WIT RestClient (assuming you are using Typescript):
import { WorkItemTrackingHttpClient, getClient } from "TFS/WorkItemTracking/RestClient";
With that you can do
const witClient = ((getClient()) as WorkItemTrackingHttpClient);
and then
const result = await witClient.queryByWiql({ query: query });
The WorkItemTrackingHttpClient is all you need to manipulate work items.
EDIT: You could also have a look at the new SDK and API. But unfortunately its lacking a lot on the documentation side. Although there are some samples.
To query work items, you can can also check this page for WorkItemTrackingHttpClient2_2 client API.
IPromise<Contracts.WorkItemQueryResult> queryById(id, project, team)
IPromise<Contracts.WorkItemQueryResult> queryByWiql(wiql, project, team)
This is also an example about how to get WorkItemTrackingHttpClient and to call Api on Microsoft docs site.
While taking screenshot of the current map using arcgis JavaScript, I could see that we have two ways to take the screenshots.
I could see same question asked here
using /export of url
using print task api
var printTask = new esriLoader.PrintTask(PrintTaskServiceUrl);
esriLoader.Config.defaults.io.proxyUrl = proxyUrl;
esriLoader.Config.defaults.io.alwaysUseProxy = true;
var template = new esriLoader.PrintTemplate();
template.exportOptions = {
width: 1015,
height: 633,
dpi: 96 // if 200 ,map image will get good quality
};
template.format = "JPG";
template.layout = "MAP_ONLY",
template.preserveScale = false;
template.showLabels = true;
template.showAttribution = false;
template.layoutOptions = {
"legendLayers": [], // empty array means no legend
"scalebarUnit": "Miles",
"copyrightText": "<div>xxx</div>",
}
var params = new esriLoader.PrintParameters();
params.map = map;
params.template = template;
printTask.execute(params, success, error);
So, what is the difference between these 2 methods?
Is the REST API provided by ArcGIS Server to generated map images.
Is the Javascript object with in ArcGIS JavaScript SDK.
The PrintTask will use the Export Web Map Task from REST api to generate the map image. PrintTask provides a simple way of creating images. On the other hand, if you want to use the REST API directly you can do so, by using esri\request object, but you would have to generate all the required parameters as described by the API.
Disclaimer: I just joined and don't have reputation to comment; hence I have to post it as an answer to original question, which following is not.
Is there any advantage when using REST API directly or vice versa?
As T Kambi stated, using PrintTask, if available, is easier. Accessing Export Web Map Task REST end point directly may be beneficial in some cases, for example:
Web_Map_as_JSON parameter needs to be modified before request is
sent to the Export Web Map Task Geoprocessing service. For example, resources not present in the map need to be added, layers' visibility needs to be changed, tokens for secured services need to be supplied, etc.
The request for map image is sent from outside of ArcGIS API for JavaScript environment. There may be no actual map and one only wants to use ArcGIS Server capabilities to fuse services' images and utilize a Layout Template.
The Export Web Map task may be accessed either from client or server side. In the latter case, a kind of "Print Task proxy" may help with printing secured services; for example, eliminate need for custom printing service in applications utilizing Long-lived Tokens.
Answering the original question now.
The Export Map REST end point is specific to a MapServer service and allows for producing a service image for given extent, specified layers, spatial reference, etc.
Ultimately, PrintTask sends request to Export Web Map Task REST end point. Export Web Map Task is an interface for producing a map image, possibly with multiple services present and graphics, per provided Web_Map_as_JSON parameter. It also provides an option to surround the map image with a layout, predefined as an ArcGIS template (stored on the server). For example:
https://sampleserver6.arcgisonline.com/arcgis/rest/services/Utilities/PrintingTools/GPServer/Export%20Web%20Map%20Task/execute
After executed, the value of url in provided results is reference to the map image.
Export Web Map Task interface results
Note: Web Map as JSON parameter can be observed in browser's developer tools after PrintTask request is sent by the API. In this case, it is taken from ESRI's Print Dijit example:
(cannot post another link)
I'm trying to get a full list of watched videos for a given user in my YouTube API application. I want to add up total duration of all videos.
When I get the list of videos from history playlist, the API caps it at 50 items. There's pagination but total amount of items is 50 (not just per page); I can't access more data with the API it appears.
Is there any way I can get this playlist without the data cap? I'm hoping for another method (of using the API) or a way to do it without the API. I know YouTube stores this data because I can view my entire history (far more that 50 videos).
I'm using this code:
var requestOptions = {
playlistId: playlistId,
part: 'snippet',
maxResults: 50
};
gapi.client.youtube.playlistItems.list(requestOptions);
where playlistId is the id of the history playlist I got from a gapi.client.youtube.channels.list request.
Edit (2017): I want to clarify that it was always my intention to download my own history, just out of interest to see how much time I have spent watching videos. I still have not been able to do this.
The API currently only retrieves the last two weeks of Watch History. For more information refer to the Bug Issue reported: https://code.google.com/p/gdata-issues/issues/detail?id=4642
Note:
There is a similar question on SO asked here: YouTube API v3 returns truncated watch history
I wrote a scraper(in Python 2.7(updated for 3.5) and Scrapy) for this task a while ago.
Sans official API, it uses a logged in session cookie and html parsing. Dumps to SQLite by default.
https://github.com/zvodd/Youtube-Watch-History-Scraper
How it's done: essentially it opens the url
https://www.youtube.com/feed/history'
with a valid(logged in) session cookie taken from Chrome. Scrapes all video entries for name, vid(url), channel/user, description, length. Then it finds the button at the bottom of the page with the attribute data-uix-load-more-href which contains the link to the next page, something like:
"/browse_ajax?action_continuation=1&continuation=98h32hfoasau0fu928hf2hf908h98hr%253D%253D&target_id=item-section-552363&direct_render=1"
... re-scrapes the video entries from there and dumps them all into an sqlite database; which you can search entries by any of the fields (name, length, user, description, etc).
So until they change their feed/history page, it's doable and done.
I might even update it.
While this isn't currently possible using just the YouTube API, there is an (albeit slightly involved) method to calculate your watch time):
download a list of your watch history as a JSON file using Google Takeout.
Unfortunately the JSON file doesn't include the video durations, so the next step is to extract the video IDs (the part after "watch?v=" in the "titleURL" object
Now take your list of video IDs, and send a request to the youtube API that looks something like this:
function execute() {
return gapi.client.youtube.videos.list({
"part": [
"contentDetails"
],
"id": [
"VIDEO IDs"
],
"fields": "items(contentDetails(duration))"
})
(Code created using YouTube API Explorer)
Note: You may need to break the list of video IDs into smaller lists (I had to) or the API may reject the request. As [pointed out by stvar in the comments] the ID list maximum length is 50, so this is the maximum length your lists can be. (full disclosure: I was using Python to send the requests)
Finally, just extract the duration values and add them up (though this might not be quite as easy as it sounds)
The best part of this is I don't believe this actually violates any ToS.
It seems like this is a known bug originally reported in 2013. The exact same behavior is explained on a Google Code thread: https://code.google.com/p/gdata-issues/issues/detail?id=4642
Brainstorming, never tried: Have you tried not using the API and instead parsing the https://www.youtube.com/feed/history URL?
Theoretically, the user browsing could be emulated, including the pagination. I am not aware of how hard though (probably very), since you need to deal with authentication and YouTube probably tries to verify that a human is browsing.
I was looking for some way to get the list of YouTube history.
I just found out that Google has a tool for this. In Google Takeout you have a option taht you can get the entire list of watched videos. My list went back util 2011.
To get explanation short there are two videos explaining how to do this:
https://www.youtube.com/watch?v=zlzzO1e6dws
https://www.youtube.com/watch?v=dto8jGMxHxY
I want to access Google Spreadsheets using JavaScript only (no .NET, C#, Java, etc.)
I came here and was shocked to know that there is NO API for JavaScript to access Google Sheets.
Please tell me how to access (CREATE/EDIT/DELETE) Google Sheets using JavaScript or any of its frameworks like jQuery.
I have created a simple javascript library that retrieves google spreadsheet data (if they are published) via the JSON api:
https://github.com/mikeymckay/google-spreadsheet-javascript
You can see it in action here:
http://mikeymckay.github.com/google-spreadsheet-javascript/sample.html
Jan 2018 UPDATE: When I answered this question last year, I neglected to mention a third way to access Google APIs with JavaScript, and that would be from Node.js apps using its client library, so I added it below.
It's Mar 2017, and most of the answers here are outdated -- the accepted answer now refers to a library that uses an older API version. A more current answer: you can access most Google APIs with JavaScript only. Google provides 2 (correction, 3) ways to do this today:
As mentioned in the answer by Dan Dascalescu, you can use Google Apps Script, the JavaScript-in-Google's-cloud solution. That is, non-Node server-side JS apps outside the browser that run on Google servers.
You code your apps in the Apps Script code editor, and they can access Google Sheets in two different ways:
The Spreadsheet Service (native object support; usage guide); native is easier but is generally older than...
The Google Sheets Advanced Service (directly access the latest Google Sheets REST API [see below]; usage guide)
Apps Script also powers add-ons, and you can extend Sheets UI functionality with Sheets add-ons (like these)
You can even write mobile add-ons which extend the Sheets app on Android
To learn more about using Apps Script, check out these videos I've created (most involve the use of Sheets)
You can also use the Google APIs Client Library for JavaScript to access the latest Google Sheets REST API on the client side.
Here are some generic samples of using the client library
The latest Sheets API (v4) was released at Google I/O 2016; it's much more powerful than all previous versions, giving developers programmatic access to most features found in the Sheets UI
Here is the JavaScript quickstart for the API to help you get started
Here are sample "recipes" (JSON payloads) for core API requests
If you're not "allergic" to Python (if you are, just pretend it's pseudocode ;) ), I made several videos with more "real-world" samples of using the API you can learn from and migrate to JS if desired (NOTE: even though it's Python code, most API requests have JSON & easily portable to JS):
Migrating SQL data to a Sheet (code deep dive post)
Formatting text using the Sheets API (code deep dive post)
Generating slides from spreadsheet data (code deep dive post)
Those and others in the Sheets API video library
The 3rd way to access Google APIs with JavaScript is from the Node.js client library on the server-side. It works similarly to using the JavaScript (client) client library described just above, only you'll be accessing the same API from the server-side. Here's the Node.js Quickstart example for Sheets. You may find the Python-based videos above to be even more useful as they too access the API from the server-side.
When using the REST API, you need to manage & store your source code as well as perform authorization by rolling your own auth code (see samples above). Apps Script handles this on your behalf, managing the data (reducing the "pain" as mentioned by Ape-inago in their answer), and your code is stored on Google's servers. But your functionality is restricted to what services App Script provides whereas the REST API gives developers much broader access to the API. But hey, it's good to have choices, right? In summary, to answer the OP original question, instead of zero, developers have three ways of accessing Google Sheets using JavaScript.
Here's the Gist.
You can create a spreadsheet using the Google Sheets API. There is currently no way to delete a spreadsheet using the API (read the documentation). Think of Google Docs API as the route to create and look-up documents.
You can add/remove worksheets within the spreadsheet using the worksheet based feeds.
Updating a spreadsheet is done through either list based feeds or cell based feeds.
Reading the spreadsheet can be done through either the Google Spreadsheets APIs mentioned above or, for published sheets only, by using the Google Visualization API Query Language to query the data (which can return results in CSV, JSON, or HTML table format).
Forget jQuery. jQuery is only really valuable if you're traversing the DOM. Since GAS (Google Apps Scripting) doesn't use the DOM jQuery will add no value to your code. Stick to vanilla.
I'm really surprised that nobody has provided this information in an answer yet. Not only can it be done, but it's relatively easy to do using vanilla JS. The only exception being the Google Visualization API which is relatively new (as of 2011). The Visualization API also works exclusively through a HTTP query string URI.
There's a solution that does not require one to publish the spreadsheet. However, the sheet does need to be 'Shared'. More specifically, one needs to share the sheet in a manner where anyone with the link can access the spreadsheet. Once this is done, one can use the Google Sheets HTTP API.
First up, you need an Google API key. Head here:
https://developers.google.com/places/web-service/get-api-key
NB. Please be aware of the security ramifications of having an API key made available to the public: https://support.google.com/googleapi/answer/6310037
Get all data for a spreadsheet - warning, this can be a lot of data.
https://sheets.googleapis.com/v4/spreadsheets/{spreadsheetId}/?key={yourAPIKey}&includeGridData=true
Get sheet metadata
https://sheets.googleapis.com/v4/spreadsheets/{spreadsheetId}/?key={yourAPIKey}
Get a range of cells
https://sheets.googleapis.com/v4/spreadsheets/{spreadsheetId}/values/{sheetName}!{cellRange}?key={yourAPIKey}
Now armed with this information, one can use AJAX to retrieve data and then manipulate it in JavaScript. I would recommend using axios.
var url = "https://sheets.googleapis.com/v4/spreadsheets/{spreadsheetId}/?key={yourAPIKey}&includeGridData=true";
axios.get(url)
.then(function (response) {
console.log(response);
})
.catch(function (error) {
console.log(error);
});
2016 update: The easiest way is to use the Google Apps Script API, in particular the SpreadSheet Service. This works for private sheets, unlike the other answers that require the spreadsheet to be published.
This will let you bind JavaScript code to a Google Sheet, and execute it when the sheet is opened, or when a menu item (that you can define) is selected.
Here's a Quickstart/Demo. The code looks like this:
// Let's say you have a sheet of First, Last, email and you want to return the email of the
// row the user has placed the cursor on.
function getActiveEmail() {
var activeSheet = SpreadsheetApp.getActiveSheet();
var activeRow = activeSheet.getActiveCell().getRow();
var email = activeSheet.getRange(activeRow, 3).getValue();
return email;
}
You can also publish such scripts as web apps.
edit: This was answered before the google doc's api was released. See Evan Plaice's answer and Dan Dascalescu's answer for more up-to-date
information.
It looks lke you can, but it's a pain to use. It involves using the Google data API.
http://gdatatips.blogspot.com/2008/12/using-javascript-client-library-w-non.html
"The JavaScript client library has helper methods for Calendar, Contacts, Blogger, and Google Finance. However, you can use it with just about any Google Data API to access authenticated/private feeds. This example uses the DocList API."
and an example of writing a gadget that interfaces with spreadsheets: http://code.google.com/apis/spreadsheets/gadgets/
'JavaScript accessing Google Docs' would be tedious to implement and moreover Google documentation is also not that simple to get it. I have some good links to share by which you can achieve js access to gdoc:
http://code.google.com/apis/documents/docs/3.0/developers_guide_protocol.html#UploadingDocs
http://code.google.com/apis/spreadsheets/gadgets/
http://code.google.com/apis/gdata/docs/js.html
http://www.mail-archive.com/google-help-dataapi#googlegroups.com/msg01924.html
May be these would help you out..
Sorry, this is a lousy answer. Apparently this has been an issue for almost two years so don't hold your breath.
Here is the official request that you can "star"
Probably the closest you can come is rolling your own service with Google App Engine/Python and exposing whatever subset you need with your own JS library. Though I'd love to have a better solution myself.
In this fast changing world most of these link are obsolet.
Now you can use Google Drive Web APIs:
Java
PHP
Javacript
.NET
Python
Ruby
and others...
you can do it by using Sheetsee.js and tabletop.js
example from git
another Example
You can read Google Sheets spreadsheets data in JavaScript by using the RGraph sheets connector:
https://www.rgraph.net/canvas/docs/import-data-from-google-sheets.html
Initially (a few years ago) this relied on some RGraph functions to work its magic - but now it can work standalone (ie not requiring the RGraph common library).
Some example code (this example makes an RGraph chart):
<!-- Include the sheets library -->
<script src="RGraph.common.sheets.js"></script>
<!-- Include these two RGraph libraries to make the chart -->
<script src="RGraph.common.key.js"></script>
<script src="RGraph.bar.js"></script>
<script>
// Create a new RGraph Sheets object using the spreadsheet's key and
// the callback function that creates the chart. The RGraph.Sheets object is
// passed to the callback function as an argument so it doesn't need to be
// assigned to a variable when it's created
new RGraph.Sheets('1ncvARBgXaDjzuca9i7Jyep6JTv9kms-bbIzyAxbaT0E', function (sheet)
{
// Get the labels from the spreadsheet by retrieving part of the first row
var labels = sheet.get('A2:A7');
// Use the column headers (ie the names) as the key
var key = sheet.get('B1:E1');
// Get the data from the sheet as the data for the chart
var data = [
sheet.get('B2:E2'), // January
sheet.get('B3:E3'), // February
sheet.get('B4:E4'), // March
sheet.get('B5:E5'), // April
sheet.get('B6:E6'), // May
sheet.get('B7:E7') // June
];
// Create and configure the chart; using the information retrieved above
// from the spreadsheet
var bar = new RGraph.Bar({
id: 'cvs',
data: data,
options: {
backgroundGridVlines: false,
backgroundGridBorder: false,
xaxisLabels: labels,
xaxisLabelsOffsety: 5,
colors: ['#A8E6CF','#DCEDC1','#FFD3B6','#FFAAA5'],
shadow: false,
colorsStroke: 'rgba(0,0,0,0)',
yaxis: false,
marginLeft: 40,
marginBottom: 35,
marginRight: 40,
key: key,
keyBoxed: false,
keyPosition: 'margin',
keyTextSize: 12,
textSize: 12,
textAccessible: false,
axesColor: '#aaa'
}
}).wave();
});
</script>
For this type of thing you should use Google Fusion Tables. The API is designed for that purpose.