Develop Reference

How to optimize getYoutubeViews function so as not to exceed the daily Youtube API quota?

I am trying to automate the collection of statistics on Youtube videos in google spreadsheets. To do this, I use code in spreadsheets script editor with the getYoutubeViews function, as well as with the GETURL, linkURL functions, and so on.
Here is the getYoutubeViews function sample
function getYoutubeViews(videoId){
var url = "https://www.googleapis.com/youtube/v3/videos?part=statistics&id=" + videoId;
url = url + "&key=mykey";
//Utilities.sleep(Math.random(15000))
var videoListResponse = UrlFetchApp.fetch(url);
var json = JSON.parse(videoListResponse.getContentText());
return json["items"][0]["statistics"]["viewCount"];
}
function GETURL(input) {
var range = SpreadsheetApp.getActiveSheet().getRange(input);
var url = /"(.*?)"/.exec(range.getFormulaR1C1())[1];
return url;
}
I ran into two problems.
The script starts working when the user loads the table. This starts a large number of processes, since the number of videos in the table exceeds 600 pieces. This causes the error: "Service invoked too many times in a short time: exec qps".
But fixing it with Utilities.sleep does not make sense, because there is a second problem. Google’s API key quota of 10,000 points ends after 3-4 hours of work and regular table reloads.
I tried to minimize functions and actions on list, and use Utilities.sleep, to avoid this error:
Service invoked too many times in a short time: exec qps.
Try Utilities.sleep(1000) between calls. (строка 0).
But it seems that this does not help to solve the quota problem.
It seems to me that I can somehow save data in cells, activating functions only when updating data. I tried to use change triggers for these purposes, but either I did it wrong or it didn't help.
The second assumption is that it would be possible to somehow save the previous data, so that there would be some data in the cells even in case of a script error. But I do not know how this can be done.

An approach to avoid using custom functions (which will make all requests each time), is to use an onOpen() trigger to add a menu [1] that when clicked runs the getYoutubeViews() function. This function will make the request and insert the response data (views count) in the spreadsheet. It'll take the videoIds from B column (starting from 2nd row) and set the views count in D column. I put an "If" condition so that only makes the request (an update the values) for the empty views cells.
To manipulate the data on the spreadsheet I used SpreadsheetApp class [2]
function onOpen() {
SpreadsheetApp.getUi() // Or DocumentApp or SlidesApp or FormApp.
.createMenu('Actions')
.addItem('Add views', 'getYoutubeViews')
.addToUi();
}
function getYoutubeViews(){
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheetByName("Sheet2");
var videoIdArray = sheet.getRange(2, 2, sheet.getLastRow()-1, 1).getValues();
var views = sheet.getRange(2, 4, sheet.getLastRow()-1, 1);
for(var i=0; i<videoIdArray.length ; i++) {
var videoId = videoIdArray[i][0];
var viewsCell = sheet.getRange(2 + i, 4);
if(viewsCell.getValue() == "") {
var url = "https://www.googleapis.com/youtube/v3/videos?part=statistics&id=" + videoId;
url = url + "&key=AAIzaSyAUjC5AchndLg9BRIrRBYKLuKf-fFkMC9M";
var options = {
'muteHttpExceptions' : true,
'headers': {
'Authorization': 'Bearer ' + ScriptApp.getOAuthToken(),
}
};
var videoListResponse = UrlFetchApp.fetch(url, options);
var json = JSON.parse(videoListResponse.getContentText());
Logger.log(json)
var views = json["items"][0]["statistics"]["viewCount"];
viewsCell.setValue(views);
}
}
}
You can not run the code directly with the onEdit() function because triggers have restrictions [3], among which there's one that says:
They cannot access services that require authorization.
UrlFetchApp.fetch() is a service that requires authorization from the user.
[1] https://developers.google.com/apps-script/guides/menus
[2] https://developers.google.com/apps-script/reference/spreadsheet/spreadsheet-app
[3] https://developers.google.com/apps-script/guides/triggers/#restrictions

Google Apps Script: How to set return to my page?

I'm using Google Apps Script to write values from a form to a Google Spreadsheet.
I have the form in my HTML page and its action calls the Google Apps Script to write in the sheet.
Now I'd like to go back to my site with a flag var and show a message (Error or Complete) based on the result of the function.
I know how to create and set the flag variable but I don't know how to "send" it to my page. I was only able to show my flag with something like this (that's the whole function I have in GAS)
function doPost(e){
var id = "";
var name = e.parameter['name'];
var surname = e.parameter['surname'];
var serial = e.parameter['serial'];
var eMail = e.parameter['email'];
var text = e.parameter['text'];
var date = new Date();
var ans = ""
var ctrl= "WIP";
var vals = [id, date, name, surname, serial, eMail, text, ans, flag];
var sheetObj =SpreadsheetApp.openById("myKey").getSheetByName('Requests').appendRow(vals);
return ContentService.createTextOutput(someOutput);
}
Someone knows how to do what I need?
Thanks a lot for your help!
S.

You can do something like this
return ContentService.createTextOutput("Complete").setMimeType(ContentService.MimeType.TEXT);
Or in case of exception return 'Error' from your catch block like this
return ContentService.createTextOutput("Error").setMimeType(ContentService.MimeType.TEXT);

How to send JSON data created by Python to JavaScript?

I am using Python cherrypy and Jinja to serve my web pages. I have two Python files: Main.py (handle web pages) and search.py (server-side functions).
I create a dynamic dropdown list (using JavaScript) based on a local JSON file called component.json(created by function componentSelectBar inside search.py).
I want to ask how can my JavaScript retrieve JSON data without physically storing the JSON data into my local website root's folder and still fulfil the function of dynamic dropdown list.
The componentSelectBar function inside search.py:
def componentSelectBar(self, brand, category):
args = [brand, category]
self.myCursor.callproc('findComponent', args)
for result in self.myCursor.stored_results():
component = result.fetchall()
if (len(component) == 0):
print "component not found"
return "no"
components = []
for com in component:
t = unicodedata.normalize('NFKD', com[0]).encode('ascii', 'ignore')
components.append(t)
j = json.dumps(components)
rowarraysFile = 'public/json/component.json'
f = open(rowarraysFile, 'w')
print >> f, j
print "finish component bar"
return "ok"
The selectBar.js:
$.getJSON("static/json/component.json", function (result) {
console.log("retrieve component list");
console.log("where am i");
$.each(result, function (i, word) {
$("#component").append("<option>"+word+"</option>");
});
});

store results from componentSelectBar into database
expose new api to get results from database and return json to browser
demo here:
#cherrypy.expose
def codeSearch(self, modelNumber, category, brand):
...
result = self.search.componentSelectBar(cherrypy.session['brand'], cherrypy.session['category'])
# here store result into a database, for example, brand_category_search_result
...
#cherrypy.expose
#cherrypy.tools.json_out()
def getSearchResult(self, category, brand):
# load json from that database, here is brand_category_search_result
a_json = loadSearchResult(category, brand)
return a_json
document on CherryPy, hope helps:
Encoding response
In your broswer, you need to GET /getSearchResult for json:
$.getJSON("/getSearchResult/<arguments here>", function (result) {
console.log("retrieve component list");
console.log("where am i");
$.each(result, function (i, word) {
$("#component").append("<option>"+word+"</option>");
});
});

To use that json data directly into javascript you can use
var response = JSON.parse(component);
console.log(component); //prints
OR
You already created json file.If that file is in right format then you can read json data from that file using jQuery jQuery.getJSON() For more: http://api.jquery.com/jQuery.getJSON/

You are rendering a HTML and sending it as response. If you wish to do with JSON, this has to change. You should return JSON in your main.py, whereas you will send a HTML(GET or POST) from Javascript and render it back.
def componentSelectBar(self, brand, category):
/* Your code goes here */
j = json.dumps(components)
// Code to add a persistent store here
rowarraysFile = 'public/json/component.json'
f = open(rowarraysFile, 'w')
print >> f, j
// Better to use append mode and append the contents to the file in python
return j //Instead of string ok
#cherrypy.expose
def codeSearch(self):
json_request = cherrypy.request.body.read()
import json # This should go to the top of the file
input_dict = json.loads(json_request)
modelNumber = input_dict.get("modelNumber", "")
category = input_dict.get("category", "")
brand = input_dict.get("brand", "")
/* Your code goes here */
json_response = self.search.componentSelectBar(cherrypy.session['brand'], cherrypy.session['category'])
return json_response
Here, I added only for the successful scenario. However, you should manage the failure scenarios(a JSON error response that could give as much detail as possible) in the componentSelectBar function. That will help you keep the codeSearch function as plain as possible and help in a long run(read maintaining the code).
And I would suggest you to read PEP 8 and apply it to the code as it is kind of norm for all python programmers and help any one else who touches your code.
EDIT: This is a sample javascript function that will make a post request and get the JSON response:
searchResponse: function(){
$.ajax({
url: 'http://localhost:8080/codeSearch', // Add your URL here
data: {"brand" : "Levis", "category" : "pants"}
async: False,
success: function(search_response) {
response_json = JSON.parse(search_response)
alert(response_json)
// Do what you have to do here;
// In this specific case, you have to generate table or any structure based on the response received
}
})
}

How do I get Gmail contacts using JavaScript?

What is the Ajax call that I should make to get gmail contacts using JavaScript? I already have the user OAuth Token which I got because the user signed up to my site using Google.

If you're using OAuth2 through JavaScript, you can use the Google Contacts API, but you'll need to get authorisation by sending the correct scope of permissions to Google when getting the access token, which is https://www.google.com/m8/feeds. (reference)
As you already know how to get the access token, it's as simple as calling the API with the correct query. To get all contacts for your user, it's as simple as making an asynchronous request to the API for the required info. For example, where {userEmail} is the user's email and {accessToken} is your access token, simply make a GET address to the following URI:
https://www.google.com/m8/feeds/contacts/{userEmail}/full?access_token={accessToken}&alt=json
A list of the types of queries you can send and their parameters are available here:
Google Contacts API
API Parameters

To get users' contacts using OAuth, first you need to specify the contact's scope in your request. If you're using ChromeExAuth, then you would write:
var oauth = ChromeExOAuth.initBackgroundPage({
'request_url' : 'https://www.google.com/accounts/OAuthGetRequestToken',
'authorize_url' : 'https://www.google.com/accounts/OAuthAuthorizeToken',
'access_url' : 'https://www.google.com/accounts/OAuthGetAccessToken',
'consumer_key' : 'anonymous',
'consumer_secret' : 'anonymous',
'scope' : 'https://www.googleapis.com/auth/userinfo.email https://www.googleapis.com/auth/userinfo.profile https://www.google.com/m8/feeds/',
'app_name' : 'MyApp'
});
The scope parameter above lists 3 scopes: the user's email, profile, and contacts (google.com/m8/feeds/contacts)
To get their contacts after the user authorizes the token, you would send a request like this:
var url = "http://www.google.com/m8/feeds/contacts/default/full";
oauth.sendSignedRequest(url, onContacts, {
'parameters' : {
'alt' : 'json',
'max-results' : 99999
}
});
And the callback for the request could look like this:
function onContacts(text, xhr) {
contacts = [];
var data = JSON.parse(text);
for (var i = 0, entry; entry = data.feed.entry[i]; i++) {
var contact = {
'name' : entry['title']['$t'],
'id' : entry['id']['$t'],
'emails' : []
};
if (entry['gd$email']) {
var emails = entry['gd$email'];
for (var j = 0, email; email = emails[j]; j++) {
contact['emails'].push(email['address']);
}
}
if (!contact['name']) {
contact['name'] = contact['emails'][0] || "<Unknown>";
}
contacts.push(contact);
}
};
To view the contacts array, you could just print on the console:
console.log(contacts);
You can checkout the Google OAuth tutorial here

How do I get “any valid access_token” for reading public posts?

I need to show the newest 5 posts from my facebook application, and I need to avoid running any server-side code. The following worked until this morning:
var id = myappid;
var limit = 5;
var div = document.getElementById('facebook_posts');
FB.init();
FB.api('/'+id+'/posts', {'limit': limit}, function(response) {
if (response && response.data && response.data.length) {
div.innerHTML = '';
for (var i=0, l=response.data.length; i<l; i++) {
var post = response.data[i];
if (post.message && post.link)
div.innerHTML += '' + post.message + '<br/>';
}
} else {
div.innerHTML = 'Error loading facebook posts';
}
});
I didn't change anything, but now facebook returns an "OAuthException" with the message "An access token is required to request this resource.". The API says I need "Any valid access_token to see public". So, how do I get "Any valid access_token" without showing an authendication pop-up?

The easiest thing would be to use an offline access token created for your administrator user. Look here to find out how to create one.

You can also use the token that the Facebook graph API explorer uses. But don't grab one from your personal account if you are embedding the token in your page.