Develop Reference

How to continuously check if the Date has passed in a chrome extension?

I am building a Chrome Extension and I am letting the user choose a Time when they are usually turning the PC off.
If this time has passed, I want a value to be reset back to 0 and a new Date be created.
What I did
I created a function that takes a parament of a Dare ISO String, which will then be converted into a Date Object. Inside that function I am comparing between now and the end time, and if the end time is smaller or equal to now, it means the time has passed and the value should be reset. But it's not doing anything.
I call the function inside my storage.sync.get method and inside my storage.onChanged method, so I always have the correct time to work with. But that does not seem to do it.
Here's the code:
Background.js
chrome.storage.onChanged.addListener((changes, namespace) => {
if ("reset" in changes) {
const reset = changes.reset.newValue;
console.log(reset);
checkResetTimer(reset);
}
});
chrome.storage.sync.get(["reset", "amount"], (obj) => {
const reset = obj.reset;
console.log(reset);
checkResetTimer(reset);
});
function checkResetTimer(isoTime) {
const resetDate = new Date(isoTime);
const now = new Date();
if (resetDate <= now) {
chrome.storage.sync.set({ drank: 0 }, () => {
console.log("drank has been set.");
});
}
}
The time value I get from the popup, it's an input.
I am at a loss right now. I don't know how to properly have a reset timer.
You can view my whole code in this Repository: https://github.com/Braweria/TakeAGulp
I feel the problem is, that it checks only once, but it needs to check the time consistently.

A crude approach to the problem can be the following:
Background.js
// rest of your code
const resetInterval = setInterval(() => {
chrome.storage.sync.get(["reset", "amount"], (obj) => {
const reset = obj.reset;
const resetTime = new Date(reset);
const now = new Date();
if(resetTime < now) {
// past your reset time; reset the value here
// maybe clear the interval too to stop the checking
clearInterval(resetInterval);
}
});
}, 1000 * 60); // check every minute
Essentially you have to check the value of the reset timer at a given interval to make sure whether that timer has expired.

Show / Hide ad by time at the same time worldwide

The idea is like this: At 6:00 am Argentina, I want a announce (image) to be displayed that remains active for one hour, that is, that it can be visible and that when it reaches 60min it is hidden, that is , at 7:00 am hidden. That this action is repeated every 7 hours. Therefore I want it to remain hidden for 7 hours and repeat the action again. At 2:00 p.m. it appears and at 3:00 p.m. it hides. 7 hours pass. It reappears at 10pm and hides at 11pm. 7 hours pass and he appears again at 6:00 am.
I have this code created so that it recognizes the time differences and runs at the same time in all countries, that is, the ad comes out at 6:00 am Argentina and at the same time it is shown in Los Angeles even though it is 2:00 am. But it's not working. It appears at the time according to the country.
NOTE: there are two elements in the code, one is for another ad that appears at 0:00
var offset = new Date().getTimezoneOffset() / 60;
var horarios1 = [6 + offset, 14 + offset, 22 + offset];
var elemento1 = document.getElementById("panel1");
var horarios2 = [0 + offset];
var elemento2 = document.getElementById("panel2");
setInterval(function() {
var hora = new Date().getHours();
if (horarios1.includes((hora + offset) % 24)) {
elemento1.style.display = 'block';
} else {
elemento1.style.display = 'none';
}
if (horarios2.includes((hora + offset) % 24)) {
elemento2.style.display = 'block';
} else {
elemento2.style.display = 'none';
}
}, 1000);
<div id="panel1" style="display: none;">PANEL 6, 14, 22</div>
<div id="panel2" style="display: none;">PANEL 0</div>
Thank you in advance.

Your code is using javascript time. Javascript takes the time from users machine. So when you visit your website, it will show your machine's time, when I visit it'll show my machine's time. However if you want a universal time for the whole world, i.e show the ad Argentina time 06:00 all over the world, than you can apply either of the following methods.
1. USE SERVER TIME
You need a bit of a backend code here. Show the time from your server, and its fixed for the whole world. Details depend on what backend technology (php/java/python) you are using.
2. USE A THIRD PARTY API
Use api from another website. Like worldtimeapi.org/. Make an ajax call, get the time of your desired location. You can use plain javascript or use any ajax library to do that. Here I'm including two methods: 1) plain javascript and 2) using axios (a popular ajax library)
Vanilla JS
function getTime(url) {
return new Promise((resolve, reject) => {
const req = new XMLHttpRequest();
req.open("GET", url);
req.onload = () =>
req.status === 200
? resolve(req.response)
: reject(Error(req.statusText));
req.onerror = (e) => reject(Error(`Network Error: ${e}`));
req.send();
});
}
Now Use this function to make the ajax call
let url = "http://worldtimeapi.org/api/timezone/America/Argentina/Buenos_Aires";
getTime(url)
.then((response) => { //the api will send this response which is a JSON
// you must parse the JSON to get an object using JSON.parse() method
let dateObj = JSON.parse(response);
let dateTime = dateObj.datetime;
console.log(dateObj);
console.log(dateTime);
})
.catch((err) => {
console.log(err);
});
AXIOS
Add axios library to your project.
axios({
url:"http://worldtimeapi.org/api/timezone/America/Argentina/Buenos_Aires",
method: "get",
})
// Here response is an object. The api will send you a JSON. But axios automatically
// convert it to an object. So you don't need to convert it manually.
.then((response) => {
let dateObj = response.data;
let dateTime = dateObj.datetime;
console.log(dateObj);
console.log(dateTime);
})
.catch((err) => {
console.log(err);
});
(function () {
var url =
"http://worldtimeapi.org/api/timezone/America/Argentina/Buenos_Aires",
horarios1 = [6, 14, 22],
elemento1 = document.getElementById("panel1"),
horarios2 = [0],
elemento2 = document.getElementById("panel2");
function getTime(url) {
return new Promise((resolve, reject) => {
const req = new XMLHttpRequest();
req.open("GET", url);
req.onload = () =>
req.status === 200
? resolve(req.response)
: reject(Error(req.statusText));
req.onerror = (e) => reject(Error(`Network Error: ${e}`));
req.send();
});
}
setInterval(function () {
getTime(url)
.then((data) => {
var dateObj = JSON.parse(data);
var dateTime = dateObj.datetime;
var hora = Number(dateTime.slice(11, 13));
if (horarios1.includes(hora)) {
elemento1.style.display = "block";
} else {
elemento1.style.display = "none";
}
if (horarios2.includes(hora)) {
elemento2.style.display = "block";
} else {
elemento2.style.display = "none";
}
})
.catch((err) => {
console.log(err);
});
}, 1000);
})();
<div id="panel1" style="display: none;">PANEL 6, 14, 22</div>
<div id="panel2" style="display: none;">PANEL 0</div>
Hope that helps. Few Things to remember though -
1. worldtimeapi.org/ is a third party service. If they choose to terminate their service, your code will break. But if you use your server time, as long as your server is running, your code will run.
2. Because of the ajax call, this code will not work in stackoverflow. Copy paste the code in your project to make it work.
3. If still it doesn't work, it means you are facing CORS (cross origin policy) issue. Read this link, search internet/SO. You will find your solution. Happy coding :)

How to increase your limit of Github API uses per hour in Javascript

I'm trying to work pull requests, issues, and commits with repos and I have the following code:
const axios = require('axios');
var gitPullApiLink = "https://api.github.com/repos/elixir-lang/elixir/pulls";
var listOfCommits = [];
var listOfSHAs = [];
var mapOfInfoObjects = new Map();
var mapPullRequestNumberToCommits = new Map();
var mapPRNumbersToCommitObjects = new Map();
var listOfPrObjects = [];
var setOfFileObjects = new Set();
var listOfNumbersOfTargetedIssues = [];
var mapPRnumberToCloseOpenDateObjects = new Map();
class PullRequestParser {
async getListOfPullRequests(pullrequestLink) {
const message = await axios.get(pullrequestLink);
//console.log(message);
listOfPrObjects = message['data'];
}
async getCommitsForEachPullRequestAndPRinformation() {
var listOfPrNumbers = [];
var k;
// this loop will just make a list of Pull Request Numbers
for (k = 0; k < listOfPrObjects.length; k++){
var currPrNumber = listOfPrObjects[k]['number'];
listOfPrNumbers.push(currPrNumber);
}
// I created a separate list just because... I did it this way because on the github API website it seems
// like the pull request has the same number as the issue it affects. I explain how you can see this down below
listOfNumbersOfTargetedIssues = listOfPrNumbers;
// next loop will make objects that contain information about each pull request.
var n;
for (n = 0; n < listOfPrNumbers; n++){
var ApiLinkForEachPullRequest = gitPullApiLink + "/" + listOfPrNumbers[n];
const mes = await axios.get(ApiLinkForEachPullRequest);
var temp = {OpeningDate: mes['data']['created_at'],
ClosingDate: mes['data']['closed_at'],
IssueLink: mes['data']['_links']['issue']['href']};
//mapPRnumberToCloseOpenDateObjects will be a map where the key is the pull request number and the value
// is the object that stores the open date, close date, and issue link for that pull request. The reason
// why I said I think the pull request number is the same as the number of the issue it affects is because
// if you take any object from the map, say you do mapPRnumberToCloseOpenDateObjects.get(10). You'll
// get an object with a pull request number 10. Now if you take this object and look at it's "IssueLink"
// field, the very last part of the link will have the number 10, and if you look at the github API
// it says for a single issue, you do: /repos/:owner/:repo/issues/:issue_number <---- As you can see,
// the IssueLink field will have this structure and in place of the issue_number, the field will be 10
// for our example object.
mapPRnumberToCloseOpenDateObjects.set(listOfPrNumbers[n], temp);
}
//up to this point, we have the pull request numbers. we will now start getting the commits associated with
//each pull request
var j;
for (j = 0; j < listOfPrNumbers.length; j++){
var currentApiLink = gitPullApiLink + "/" + listOfPrNumbers[j] + "/commits";
const res = await axios.get(currentApiLink);
//here we map a single pull request to the information containing the commits. I'll just warn you in
// advance: there's another object called mapPRNumbersToCommitObjects. THIS MAP IS DIFFERENT! I know it's
// subtle, but I hope the language can make the distinction: mapPullRequestNumberToCommits will just
// map a pull request number to some data about the commits it's linked to. In contrast,
// mapPRNumbersToCommitObjects will be the map that actually maps pull request numbers to objects
// containing information about the commits a pull request is associated with!
mapPullRequestNumberToCommits.set(listOfPrNumbers[j], res['data']);
}
// console.log("hewoihoiewa");
}
async createCommitObjects(){
var x;
// the initial loop using x will loop over all pull requests and get the associated commits
for (x = 0; x < listOfPrObjects.length; x++){
//here we will get the commits
var currCommitObjects = mapPullRequestNumberToCommits.get(listOfPrObjects[x]['number']);
//console.log('dhsiu');
// the loop using y will iterate over all commits that we get from a single pull request
var y;
for (y = 0; y < currCommitObjects.length; y++){
var currentSHA = currCommitObjects[y]['sha'];
listOfSHAs.push(currentSHA);
var currApiLink = "https://api.github.com/repos/elixir-lang/elixir/commits/" + currentSHA;
const response = await axios.get(currApiLink,);
//console.log("up to here");
// here we start extracting some information from a single commit
var currentAuthorName = response['data']['commit']['committer']['name'];
var currentDate = response['data']['commit']['committer']['date'];
var currentFiles = response['data']['files'];
// this loop will iterate over all changed files for a single commit. Remember, every commit has a list
// of changed files, so this loop will iterate over all those files, get the necessary information
// from those files.
var z;
// we create this temporary list of file objects because for every file, we want to make an object
// that will store the necessary information for that one file. after we store all the objects for
// each file, we will add this list of file objects as a field for our bigger commit object (see down below)
var tempListOfFileObjects = [];
for (z = 0; z < currentFiles.length; z++){
var fileInConsideration = currentFiles[z];
var nameOfFile = fileInConsideration['filename'];
var numberOfAdditions = fileInConsideration['additions'];
var numberOfDeletions = fileInConsideration['deletions'];
var totalNumberOfChangesToFile = fileInConsideration['changes'];
//console.log("with file");
var tempFileObject = {fileName: nameOfFile, totalAdditions: numberOfAdditions,
totalDeletions: numberOfDeletions, numberOfChanges: totalNumberOfChangesToFile};
// we add the same file objects to both a temporary, local list and a global set. Don't be tripped
// up by this; they're doing the same thing!
setOfFileObjects.add(tempFileObject);
tempListOfFileObjects.push(tempFileObject);
}
// here we make an object that stores information for a single commit. sha, authorName, date are single
// values, but files will be a list of file objects and these file objects will store further information
// for each file.
var tempObj = {sha: currentSHA, authorName: currentAuthorName, date: currentDate, files: tempListOfFileObjects};
var currPrNumber = listOfPrObjects[x]['number'];
console.log(currPrNumber);
// here we will make a single pull request number to an object that will contain all the information for
// every single commit associated with that pull request. So for every pull request, it will map to a list
// of objects where each object stores information about a commit associated with the pull request.
mapPRNumbersToCommitObjects.set(currPrNumber, tempObj);
}
}
return mapPRNumbersToCommitObjects;
}
async startParsingPullRequests() {
this.getListOfPullRequests(gitPullApiLink + "?state=all").then(() => {
this.getCommitsForEachPullRequestAndPRinformation().then(() => {
this.createCommitObjects().then((response) => {
console.log("functions were successful");
return new mapPRNumbersToCommitObjects;
//return mapPRNumbersToCommitObjects;
}).catch((error) => {
console.log("printing first error");
console.log(error);
})
}).catch((error2) => {
console.log("printing the second error");
console.log(error2);
})
}).catch((error3) => {
console.log("printing the third error");
console.log(error3);
});
}
//adding some getter methods so they can be used to work with whatever information people may need.
//I start all of them with the this.startParsingPullRequests() method because by calling that method
it gets all
// the information for the global variables.
async getSetOfFileObjects(){
var dummyMap = await this.startParsingPullRequests();
return {files: setOfFileObjects, prMap: mapPRnumberToCloseOpenDateObjects};
}
async OpenCloseDateObjects(){
var dummyMap = await this.startParsingPullRequests();
return mapPRnumberToCloseOpenDateObjects;
}
async getNumbersOfTargetedIssues(){
var dummyMap = await this.startParsingPullRequests();
return listOfNumbersOfTargetedIssues;
}
}
var dummy = new PullRequestParser();
var dummyMap = dummy.startParsingPullRequests().then((message) => {
console.log("dummyMap is defined! :)");
console.log(dummyMap);
});
module.exports = PullRequestParser;
Whenever I run the code on the webstorm terminal though, with:
node PullRequestParser.js
I get a 403 error, followed by a bunch of error output, with the following statement:
data: {
message: "API rate limit exceeded for 138.186.17.173. (But here's the good news: Authenticated
requests get a higher rate limit. Check out the documentation for more details.)"
I looked up the documentation for this and found out that without authentication, I can make 60 requests per hour to a repo. In order to get authentication, however, the only example provided is an example they provide by using the command line. I don't think this would be enough though because I want to do some further analysis with the results I get. Does anybody know how I can increase the number of requests I can make? Where in the code would I need to make changes and what kind of changes would I need to make? Thanks!

The first line of the documentation says everything you need to know.
For API requests using Basic Authentication or OAuth, you can make up
to 5000 requests per hour.
Using Basic Authentication is pretty simple, so that may be the easiest thing to get you up and running. OAuth is more complicated, but more desirable in production.
The axios library supports basic auth requests out of the box.
async getListOfPullRequests(pullrequestLink) {
const message = await axios.get(pullrequestLink, {
auth: {
username: 'username',
password: 'password'
}
});
//console.log(message);
listOfPrObjects = message['data'];
}
You just need to supply the correct username and password information.

How to automate downloading a file from a site?

I want to download several data files from this URL: https://pselookup.vrymel.com/
The site contains a date field and a download button. I want to download data for multiple years (which would mean a lot of requests) and I want to make it automatically.
I've created a Javascript snippet, however, it keeps downloading just the same file over and over again.
$dateField = document.getElementsByClassName('csv_download_input__Input-encwx-1 dDiqPH')[2]
$dlButton = document.getElementsByClassName('csv_download_input__Button-encwx-0 KLfyv')[2]
var now = new Date();
var daysOfYear = [];
for (var d = new Date(2016, 0, 1); d <= now; d.setDate(d.getDate() + 1)) {
daysOfYear.push(new Date(d).toISOString().substring(0,10));
}
(function theLoop (i) {
setTimeout(function () {
$dlButton.click()
$dateField.value = daysOfYear[i]
if (--i) { // If i > 0, keep going
theLoop(i); // Call the loop again, and pass it the current value of i
}
}, 3000);
})(daysOfYear.length-1);
How could I download all of the files automatically?

First off, javascript in the client is probably not the best language to do this nor the best approach to make this happen. It might work, but it's better to know what is best when choosing an approach to a problem. Also, it will avoid for you clicking ~800 times in the popup accepting the download.
You can get the files in a programatically way by just learning what you browser is doing to get the file and trying to reproduce it in bunch.
After inspecting the calls you can see that it's calling an endpoint and that endpoint is returning a link which contains the file that you can download.
Well, that is going to be easy, so now you just need to make the script in any language to be able to retrieve them.
I've chosen javascript but not client side, but nodejs which means that this has to run from your computer.
You could do the same with bash, python or any other language.
To run this do the following:
Go to a new empty directory
Run npm install axios
Create a file with the code I pasted let's call it crawler.js
Run node crawler.js
This has been tested using node v8.15.0
// NOTE: Require this to make a request and save the link as file 20190813:Alevale
const axios = require('axios');
const fs = require('fs');
let now = new Date();
let daysOfYear = [];
const baseUrl = 'https://a4dzytphl9.execute-api.ap-southeast-1.amazonaws.com/prod/eod/'
for (var d = new Date(2016, 0, 1); d <= now; d.setDate(d.getDate() + 1)) {
daysOfYear.push(new Date(d).toISOString().substring(0,10));
}
const waitFor = (time) => {
return new Promise((resolve => setTimeout(resolve, time)))
}
const getUrls = async () =>{
let day
for (day of daysOfYear) {
console.log('getting day', baseUrl + day)
// NOTE: Throttle the calls to not overload the server 20190813:Alevale
await waitFor(4000)
await axios.get(baseUrl + day)
.then(response => {
console.log(response.data);
console.log(response);
if (response.data && response.data.download_url) {
return response.data.download_url
}
return Promise.reject('Could not retrieve response.data.download_url')
})
.then((url) =>{
axios({
method: 'get',
url,
responseType: 'stream'
})
.then(function (response) {
// NOTE: Save the file as 2019-08-13 20190813:Alevale
response.data.pipe(fs.createWriteStream(`${day}.csv`))
})
.catch(console.error)
})
.catch(error => {
console.log(error);
});
}
}
getUrls()

You can instead of simulating the user, get the link to download from:
https://a4dzytphl9.execute-api.ap-southeast-1.amazonaws.com/prod/eod/2019-08-07
just change the date at the end to the date of the file you want to download. And use axios to get this URL.
This will save you sometime (in case you don't really need to simulate the click of the user etc)
Then you will get a response like this:
{
download_url":"https://d3u9ukmkxau9he.cloudfront.net/eod/2019-08-07.csv?Expires=1566226156&Signature=QRUk3tstuNX5KYVPKJSWrXsSXatkWS-eFBIGUufaTEMJ~rgpVi0iPCe1AXl5pbQVdBQxOctpixCbyNz6b9ycDgYNxEdZqPr2o2pDe8cRL655d3zXdICnEGt~dU6p35iMAJkMpPSH~jbewhRSCPUwWXQBfOiEzlHwxru9lPnDfsdSnk3iI3GyR8Oc0ZP50EdUMHF7MjWSBRbCIwnu6wW4Jh0bPmZkQDQ63ms5QxehsmtuGLOgcrC6Ky1OffVQj~ihhmBt4LGhZTajjK4WO18hCP3urKt03qpC4bOvYvJ3pxvRkae0PH1f-vbTWMDkaWHHVCrzqZhkAh3FlvMTWj8D4g__&Key-Pair-Id=APKAIAXOVAEOGN2AYWNQ"
}
and then you can use axios to GET this url and download your file.

How can I make my javascript run faster and asynchronously

I'm trying to make an extension that adds some HTML to a page as it's loaded. Long story short, it gets a list of links in a table, loads those links, and grabs a specific string and adds it to the original page I loaded.
Here is the code: due to the way the webpage was coded, i couldn't get the unique ids because they didn't exist so ignore that part.
var xhr = new XMLHttpRequest();
for (i = 0; i < length; i++) {
url = rows[i].getElementsByTagName("td")[0].getElementsByTagName('a')[0].getAttribute('href');
insert = rows[i].insertCell(-1);
xhr.onreadystatechange = function() {
if (xhr.readyState == 4 && xhr.status == 200) {
//var Data = JSON.parse(xhr.responseText);
var htmlString = xhr.responseText
, parser = new DOMParser()
, doc = parser.parseFromString(htmlString,"text/html")
, test = doc.getElementById('page-wrapper').children[1]
, ip_address = doc.getElementsByClassName('col-md-5')[0].children[5].children[0].children[0].children[10].innerHTML;
insert.innerHTML = ip_address;
}
};
xhr.open("GET",url,false);
xhr.send();
}
when i call the function, it works fine, however it takes a really long time to load and it all loads at the end rather than updating as each iteration of the for loop completes. I'd like to decrease the time it takes to load but a significant amount, and possibly have ear row update instantly vs at the end. I've tried searching asycn javascript but the bit of code I tried didn't help me much. Any assistance would be appreciated.

Using modern Javascript:
rows.forEach(row => {
fetch(row.querySelector('td:first-of-type a:first-of-type').getAttribute('href'))
.then(res => res.json())
.then(data => {
// do manipulations with DOM
})
.catch(console.log)
})