I have a script that basically take info from a website for multiple users, and put these info in a google spreadsheet, with one sheet per users.
I have a function that remove values of the firstline, resize every columns, and put back the setValues:
function adjustColumnsAndIgnoreFirstLine(sheet) {
Logger.log('--- Adjust columns ---')
const range = sheet.getRange("1:1")
// save the title line
const datas = range.getValues();
// clear it
range.clearContent();
// format without the title lines
var lastColumn = sheet.getLastColumn()
sheet.autoResizeColumns(1, lastColumn);
// set width to a minimum
for (var i = 1; i < 37; i++) { // fixed number of columns
if (sheet.getColumnWidth(i) < 30) {
sheet.setColumnWidth(i, 30);
}
}
// put back titles
range.setValues(datas);
}
my problem is that the script stop executing in the middle of the function. I still have the "execution please wait" popup, but in the logs, the script stopped like there was no error (execution finished) with this as the last log:
And, on the google spreadsheet:
One thing to note is that the problem doesn't comes from the script itself, as I do not encounter this problem on any of my machines, but my client does. My client ran the script on different navigator (chrome and edge), and had the same problem, but on different users (sometimes it blocks at the before-last user, sometimes at the before-before-last user...)
So I'm kinda lost on this problem...
The problem is actually a timeout. Google app script limit the execution time of a script at ~6 minutes.
There is existing issues for this
I have been trying to download all USA and CANADA servers here on Nord VPN website: https://nordvpn.com/ovpn/
I tried to manually download it but it is time consuming to scroll down every time and identify each US related servers, so i just wrote simple Javascript that can be run on Chrome Inspect Element Console:
var servers = document.getElementsByClassName("mr-2");
var inc_servers = [];
for (var i = 0; i < servers.length; i++) {
var main_server = servers[i];
var server = servers[i].innerText;
if(server.includes("ca")){
var parent_server = main_server.parentElement.parentElement;
parent_server.querySelector(".Button.Button--primary.Button--small").click();
inc_servers.push(server);
}
}
console.log(JSON.stringify(inc_servers));
I also wrote simple python script that automatically click "save" file:
while True:
try:
app2 = pywinauto.Application().connect(title=u'Save As', found_index=0)
app2.SaveAs.Save.click()
except:
pass
It gets all the elements, it works there, however, when I let javascript click each element, maybe because of too many elements, it returns an error:
VM15575:8 Throttling navigation to prevent the browser from hanging. See https://crbug.com/1038223. Command line switch --disable-ipc-flooding-protection can be used to bypass the protection
Are there any other best alternative for my problem? Or maybe how to fix the error message above? I tried running this command in my command prompt: switch --disable-ipc-flooding-protection
but it returns also an error: 'switch' is not recognized as an internal or external command,
operable program or batch file.
I only know basic Javascript and Python. Thanks
So right off the bat, your program is simply downloading files too fast.
Adding a small delay between each file download allows your JavaScript to run.
var servers = document.getElementsByClassName("mr-2");
var inc_servers = [];
for (var i = 0; i < servers.length; i++) {
var main_server = servers[i];
var server = servers[i].innerText;
if(server.includes("ca")){
var parent_server = main_server.parentElement.parentElement;
// Add 1 second delay between each download (fast enough on my computer.. Might be too fast for yours.)
await new Promise(resolve => setTimeout(resolve, 1000));
parent_server.querySelector(".Button.Button--primary.Button--small").click();
}
}
// Remove the logging. Just tell the user that it's worked
console.log("Done downloading all files.");
This is more of a temporary solution, but this script seems like it only needs to be run once, so it'll work for now.
(your python code runs fine. Nothing to do there)
Hope this helped.
I'm trying to build a spreadsheet based around DataDT's excellent API for 1-minute Forex data. I'm trying to build a function that 1) Reads a value ("Date time") from a cell 2) Searches for that value in a given URL from the aforementioned API 3) Prints 2 other properties (open & close price) for that same date.
In other words, It would take input from rows N and O, and output the relevant values (OPEN and CLOSE from the API) in rows H and I.
(Link to current GSpreadsheet)
This spreadsheet would link macroeconomic news and historic prices and possibly reveal useful insights for Forex users.
I already managed to query data from the API effectively but I can't find a way to filter only for the datetimes I'm asking. Much less iterating for different dates! With the help from user #Cooper I got the following code that can query entire pages from the API but can't efficiently filter yet. I'd appreciate any help that you might provide.
This is the current status of the code in Appscript:
(Code.gs)
function searchOnEdit(e) {
//e.source.toast('Entry');// I use these lines for debugging
var sh=e.range.getSheet();
if(sh.getName()!='API') return;
var checkedValue='TRUE';//these are the defaults if you install the checkboxes from the Insert Menu
var uncheckedValue='FALSE';
if(e.range.columnStart==17 && e.range.rowStart>1 && e.value==checkedValue) {
e.range.setValue(uncheckedValue);//this was commented out it should not have been sorry for that Cooper
//e.source.toast('flag1');
var r=sh.getRange(e.range.rowStart,1,1,sh.getLastColumn()).getValues()[0];
var obj={datetime:r[14],url:r[13],event:e};
var dObj=getDataDT1(obj);
//Logger.log(dObj);
sh.getRange(e.range.rowStart,4).setValue(dObj.OPEN);//loading OPEN on your spreadsheet
sh.getRange(e.range.rowStart,5).setValue(dObj.CLOSE);//loadding CLOSE on your spreadsheet
}
}
//{datetime:'',url:'',event:e}
function getDataDT1(obj) {
Logger.log(JSON.stringify(obj));//I need to see this
var r=UrlFetchApp.fetch(obj.url);
var data=JSON.parse(r.getContentText("UTF-8"));
//Logger.log(data);
var pair='USDJPY';
var dat=new Date(obj.datetime);
var dtv=new Date(dat.getFullYear(),dat.getMonth(),dat.getDate(),dat.getHours(),dat.getMinutes()).valueOf();
for(var i=0;i<data.length;i++) {
var dt=data[i].DATE_TIME.split(' ');
var sd=new Date(data[i].DATE_TIME);
var sdv=new Date(sd.getFullYear(),sd.getMonth(),sd.getDate(),sd.getHours(),sd.getMinutes()).valueOf();
if(sdv==dtv) {
var d=dt[0].split('-');
var t=dt[1].split(':');
var datestring=Utilities.formatString('%s/%s/%s',d[1],d[2],d[0]);
var timestring=Utilities.formatString('%s:%s',t[0],t[1]);
var rObj={DATE:datestring,TIME:timestring,PAIR:pair,OPEN:data[i].OPEN.toFixed(3),CLOSE:data[i].CLOSE.toFixed(3)};
break;
}
}
//Logger.log(rObj);
return rObj;
}
(Appscript.json)
{
"timeZone": "America/Caracas",
"dependencies": {
},
"exceptionLogging": "STACKDRIVER",
"oauthScopes": ["https://www.googleapis.com/auth/userinfo.email","https://www.googleapis.com/auth/script.external_request","https://www.googleapis.com/auth/script.scriptapp","https://www.googleapis.com/auth/spreadsheets"]
}
(Trigger.js)
function createSearchOnEditTrigger() {
var ss=SpreadsheetApp.getActive();
ScriptApp.newTrigger('searchOnEdit').forSpreadsheet(ss.getId()).onEdit().create();
}
onEdit Search
You will need to add a column of checkboxes to column 17 and also create an installable onEdit trigger. You may use the code provided or do it manually via the Edit/Project Triggers menu. When using the trigger creation code please check to insure that only one trigger was creates as multiple triggers can cause problems.
Also, don't make the mistake of naming your installable trigger onEdit(e) because it will respond to the simple trigger and the installable trigger causing problems.
I have an animation below showing you how it operates and also you can see the spreadsheet layout as well. Please notice the hidden columns. I had to do that to make the animation as small as possible. But I didn't delete any of your columns.
It's best to wait for the the check box to get reset back to off before checking another check box. It is possible to check them so fast that script can't keep up and some searches may be missed.
I also had to add these scopes manually:
"oauthScopes":["https://www.googleapis.com/auth/userinfo.email","https://www.googleapis.com/auth/script.external_request","https://www.googleapis.com/auth/spreadsheets"]
You can put them into your appsscript.json file which is viewable using the View/Show Manifest File. Here's a reference that just barely shows you what they look like. But the basic idea is to put a comma after the last entry before the closing bracket and add the needed lines.
After you have created the trigger it's better to go into View/Current Project triggers and set the Notifications to Immediate. If you get scoping errors it will tell you which ones to add. You add them and then run a function and you can reauthorize the access with the additional scopes. You can even run a null function like function dummy(){};.
This is the onEdit function:
function searchOnEdit(e) {
//e.source.toast('Entry');// I use these lines for debugging
var sh=e.range.getSheet();
if(sh.getName()!='API') return;
var checkedValue='TRUE';//these are the defaults if you install the checkboxes from the Insert Menu
var uncheckedValue='FALSE';
if(e.range.columnStart==17 && e.range.rowStart>1 && e.value==checkedValue) {
e.range.setValue(uncheckedValue);
//e.source.toast('flag1');
var r=sh.getRange(e.range.rowStart,1,1,sh.getLastColumn()).getValues()[0];
var obj={datetime:r[14],url:r[13],event:e};//you dont really need e here
var dObj=getDataDT1(obj);
//Logger.log(dObj);
sh.getRange(e.range.rowStart,4).setValue(dObj.OPEN);//loading OPEN on your spreadsheet
sh.getRange(e.range.rowStart,5).setValue(dObj.CLOSE);//loadding CLOSE on your spreadsheet
}
}
This is the search function. I tried caching the data but it was too large. So if you can reduce the size significantly that would help speed things up for consecutive searches.
//{datetime:'',url:'',event:e}
function getDataDT1(obj) {
var r=UrlFetchApp.fetch(obj.url);
var data=JSON.parse(r.getContentText("UTF-8"));
//Logger.log(data);
var pair='USDJPY';
var dat=new Date(obj.datetime);
var dtv=new Date(dat.getFullYear(),dat.getMonth(),dat.getDate(),dat.getHours(),dat.getMinutes()).valueOf();
for(var i=0;i<data.length;i++) {
var dt=data[i].DATE_TIME.split(' ');
var sd=new Date(data[i].DATE_TIME);
var sdv=new Date(sd.getFullYear(),sd.getMonth(),sd.getDate(),sd.getHours(),sd.getMinutes()).valueOf();
if(sdv==dtv) {
var d=dt[0].split('-');
var t=dt[1].split(':');
var datestring=Utilities.formatString('%s/%s/%s',d[1],d[2],d[0]);
var timestring=Utilities.formatString('%s:%s',t[0],t[1]);
var rObj={DATE:datestring,TIME:timestring,PAIR:pair,OPEN:data[i].OPEN.toFixed(3),CLOSE:data[i].CLOSE.toFixed(3)};
break;
}
}
//Logger.log(rObj);
return rObj;
}
This is the create Trigger function. Becareful not to run this more than once and always go check to see that there is only one and set the notifications to immediate when you first turn it on so you'll get emails pretty quickly after errors occur.
function createSearchOnEditTrigger() {
var ss=SpreadsheetApp.getActive();
ScriptApp.newTrigger('searchOnEdit').forSpreadsheet(ss.getId()).onEdit().create();
}
Animation:
This is a copy of your spreadsheet with the check boxes.
If you have any difficulties feel free to come back and get some help. I know some of this stuff sounds a bit daunting but it's kind of like going metric. Once you've measured and weighed a few things it begins to sound and feel natural.
Here's my code exactly as I have it in my script which is running right now. Perhaps I have a typo in it due to editing in comments. This has some debugging lines in which run the Logger and display toasts which you should probably comment out.
function searchOnEdit(e) {
e.source.toast('Entry');
var sh=e.range.getSheet();
if(sh.getName()!='API') return;
var checkedValue='TRUE';
var uncheckedValue='FALSE';
if(e.range.columnStart==17 && e.range.rowStart>1 && e.value==checkedValue) {
e.range.setValue(uncheckedValue);
e.source.toast('flag1');
var r=sh.getRange(e.range.rowStart,1,1,sh.getLastColumn()).getValues()[0];
var obj={datetime:r[14],url:r[13],event:e};
var dObj=getDataDT1(obj);
Logger.log(dObj);
sh.getRange(e.range.rowStart,4).setValue(dObj.OPEN);
sh.getRange(e.range.rowStart,5).setValue(dObj.CLOSE);
}
}
//{datetime:'',url:''}
function getDataDT1(obj) {
var r=UrlFetchApp.fetch(obj.url);
var data=JSON.parse(r.getContentText("UTF-8"));
//Logger.log(data);
var pair='USDJPY';
var dat=new Date(obj.datetime);
var dtv=new Date(dat.getFullYear(),dat.getMonth(),dat.getDate(),dat.getHours(),dat.getMinutes()).valueOf();
for(var i=0;i<data.length;i++) {
var dt=data[i].DATE_TIME.split(' ');
var sd=new Date(data[i].DATE_TIME);
var sdv=new Date(sd.getFullYear(),sd.getMonth(),sd.getDate(),sd.getHours(),sd.getMinutes()).valueOf();
if(sdv==dtv) {
var d=dt[0].split('-');
var t=dt[1].split(':');
var datestring=Utilities.formatString('%s/%s/%s',d[1],d[2],d[0]);
var timestring=Utilities.formatString('%s:%s',t[0],t[1]);
var rObj={DATE:datestring,TIME:timestring,PAIR:pair,OPEN:data[i].OPEN.toFixed(3),CLOSE:data[i].CLOSE.toFixed(3)};
break;
}
}
//Logger.log(rObj);
return rObj;
}
function createSearchOnEditTrigger() {
var ss=SpreadsheetApp.getActive();
ScriptApp.newTrigger('searchOnEdit').forSpreadsheet(ss.getId()).onEdit().create();
}
I'm trying to understand how the WhatsApp Web front-end page was created, especially the list of contacts on the left side (<div id="pane-side">). The name of the contacts uses the class "emojitext ellipsify" and I am trying to get the name of all contacts by the code:
var contatosTemp = document.getElementsByClassName('emojitext ellipsify');
var contatos = [];
for (var i = 0; i < contatosTemp.length; i = i + 2) { contatos.push(contatosTemp[i].innerText); }
contatos
If the scroll of the list of contacts is at top, the number of contatos are 19. But if I scroll the list, the number of contatos is increased until the number of 29. If I minimize the window, the number of contatos is decreased to 15.
At resume, the number of the list of contacts caught by the JavaScript above changes when the user scrolls the list of contacts.
If you want to understand what I am doing, open your WhatsApp Web, go to browser DevTools > Console and run the JavaScript above.
You will see the size of the array contatos:
Then scroll the pane at left side of screen and execute the JavaScript again. It will show that the array size changed.
I would like to get all the contacts' names but JavaScript only gets part of it. How can I do that?
Use this script that will bring you all the contacts
function contactList(){
var user = Store.Chat.models;
var destiny = [];
user.forEach(function(obj){destiny.push(obj.__x_formattedTitle)});
return destiny }
im currently developing an extension, but im kind of lost by the moment.
Basically, what i want it to do, its kind of what "OneTab" extension does.
So my first question is, after adding the listener to the extension button, and executing the function, i want to get all the url's of the current window, and store them in an array and the show them in the html file.
So im using this:
chrome.tabs.getSelected(null,function(tab) {
var tablink = tab.url;
console.log(tablink);
});
but its not working and im not sure how it will check all the tabs one by one.
Thanks in advance.
chrome.tabs.getSelected() will only get you the current tab.
In order to get the list of all the tabs in the current window, you need to use the chrome.windows API. This API will return an object of the current window which will have the list of tab objects.
Here is the sample code:
chrome.windows.getCurrent({"populate":true}, function(currentWindow) {
var tabURLs = [];
var tabs = currentWindow.tabs;
for (var i=0; i<tabs.length; i++) {
tabURLs.push(tabs[i].url);
}
console.log(tabURLs);
});
For details check:
http://developer.chrome.com/extensions/windows.html#method-getCurrent