I'm writing a local application running on XAMPP on Google Chrome. It interacts with IndexedDB (I use the promise library of Jake Archibald). And here is my problem.
Let's say I have an object store with 2 properties, day and salary (money made that day). I want to console.log the whole object store for let's say a report. Here is the code snippet:
//Report 4 days of work
for(Day = 1; Day <= 4; Day++) {
dbPromise.then(function(db) {
var tx = transaction("workdays", "readonly");
return tx.objectStore("workdays").get(Day);
}).then(function(val) {
console.log(val.day + '\t' + val.salary + '$\n';
})
What I expected is something like this:
1 100$
2 120$
3 90$
4 105$
But it actually gives an error, saying "can't read value day of undefined". Turns out the loop didn't wait for the dbPromise.then()... but continued asynchronously, and since the IndexDB request is slow, by the time they finished, the Day counter was already 5 and no record was matched and returned.
I struggled for a while and then found a workaround by putting a DayTemp in the loop to capture the Day like this.
//Report 4 days of work
for(Day = 1; Day <= 4; Day++) {
DayTemp = Day;
dbPromise.then(function(db) {
var tx = transaction("workdays", "readonly");
return tx.objectStore("workdays").get(DayTemp);
}).then(function(val) {
console.log(val.day + '\t' + val.salary + '$\n';
})
And it worked fine. But then it's still not. Here is the result:
1 100$
4 105$
2 120$
3 90$
I need them to be in order. What do I need to do? Thank you a lot!
Note: The situation is a little more complicated than this, so I can't use getAll() or cursor and things like that. I really have to do the looping. And also I'm interested to be enlightened in this synchronous/asynchronous subject of Javascript. I'm a beginner.
Update: I figured it out guys! First of all the result of my second attemp with DateTemp actually turn out like this:
4 105$
4 105$
4 105$
4 105$
Which is basically reflected the same problem in the beginning. The only difference this time is that Date is captured so it didn't increment past 4.
But I finally found the solution for all of this. Let me demonstrate with this trivial Javascript snippet: (1)
function dummy(day)
{
dbPromise.then(function(db) {
//Create transaction, chooose object stores, etc,..
objectStore.get(Day);
}).then(function(db) {
console.log(day + '\t' + salary + '\$n');
})
}
for(Day = 1; Day <= 4; Day++)
{
dummy(Day);
}
If I wrote the code like above, I will always get the right answers!
1 100$
2 120$
3 90$
4 105$
However if I wrote it like this: (2)
for(Day = 1; Day <= 4; Day++)
{
dbPromise.then(function(db) {
//Create transaction, chooose object stores, etc,..
return objectStore.get(Day);
}).then(function(val) {
console.log(Day + '\t' + val.salary + '\$n');
})
}
I will always get an error since the Day counter would be already 5 by the time the database requests finish the transaction creating and enter the get() method which use the Day counter! Crazy right?
Seem like Javascript have something to determine whether a statement should be waited, meaning it should executed completely before other statements behind it can begin executing. Take the (1) code snippet for example. Enter the loop with Day = 1, inside the loop body, Javascript saw that dummy() is pass Day as a parameter, and decided that "no way I'm gonna continue the loop without let the dummy() finished first, else he's gonna messed himself up". And I get a beautiful result.
In the (2) code snippet however. Javascript enter the loop and see that I called then() of dbPromise and it says "Okk executing big boy, but I don't see any reason I have to wait for you. Oh you use Day inside then() ha? I don't care about that, only look the ouside sorry. I will increment Day while you're doing your request!". And things got messy.
That's my first point. My second point is that I also found out that add() requests in indexedDB is asynchronization, even if each add() is inside an different transaction. But the get() is synchronization so it's fine.
Now want your opinions on my update answer above. Pretty sure they are incorrect in some embrassing ways. There have to be some very basics obvious things that I missed about Javascript here.
I suggest you learn about the following topics:
Asynchronous programming
Javascript function hoisting
The difference between defining a function and calling a function
As a general rule, never define a function within a loop. If you do not define a function in a loop, then you will avoid most of the complexity.
If you insist on defining a function in a loop, you can use the trick of using an immediately executed functional expression:
for(...) {
(function defined_plus_call(a, b, c) {
// operate on a and b and c here within this function's body
// do NOT use the variables arg1ToUseForA, arg2ToUseForB, etc
}(arg1ToUseForA, arg2ToUseForB, etc));
}
Or, if you are writing for only modern browsers, and are already familiar with promises and async programming, then you can use the new async/await syntax so that you can write an imperative loop:
function get_helper_function(tx, Day) {
function executor(resolve, reject) {
var request = tx.objectStore("workdays").get(Day);
request.onsuccess = function() { resolve(request.result); };
request.onerror = function() { reject(request.error); };
}
return new Promise(executor);
}
async function foo(...) {
for(Day = 1; Day <= 4; Day++) {
var promise_result = await dbPromise;
var tx = promise_result.transaction("workdays", "readonly");
var val = await get_helper_function(tx, Day);
console.log(val.day + '\t' + val.salary + '$\n';
}
}
And then, if you really wanted to write cleaner code, I would change a few other things. One, these are all basic reads that can share the same database connection and the same transaction. Two, you can use Promise.all to iterate over several promises.
function get_by_day(tx, day) {
function executor(resolve, reject) {
var request = tx.objectStore("workdays").get(day);
request.onsuccess = function() { resolve(request.result); };
request.onerror = function() { reject(request.error); };
}
return new Promise(executor);
}
function get_all(db) {
var db = await dbPromise;
var tx = db.transaction("workdays", "readonly");
var promises = [];
for(var day of days) {
var promise = get_by_day(tx, day);
promises.push(promise);
}
var all_promise = Promise.all(promises);
return all_promise);
}
get_all().then(function(resolutions) {
for(var val of resolutions) {
console.log(val.day + '\t' + val.salary + '$\n';
}
});
This way get_all resolves the individual get promises in any order, concurrently, at least in theory.
Then, go even further, if you want to try and optimize, and look at the function IDBObjectStore.prototype.getAll. Instead of explicitly getting each day, load a range of days in one function call.
Related
Script must log postback information about call detail from zvonok.com to google spreadsheets. I has write function which only append row to sreadsheet - no update or modify of any cell in code and during few manual test calls rows has been append correct, but when my client began his usual call campaign, calls and postbacks going very often one after other, values in last row began changing few times and in some cases leave strange values
I seen behavior like this first time made short video record:
https://youtu.be/0_H_mVAbp4g
here is one column with strange value
2103052006092385
2,10305E+15
210305412464544
I have found 9 cases from 248 rows.
Client has show me excel from his user cabinet, totally was maded 5649 calls, so in google spreadsheets must be 5649 rows instead 248.
function getJsonFromUrl(url) {
var query = url;
var result = {};
if (query == undefined){
return result;
}
query.split("&").forEach(function(part) {
var item = part.split("=");
result[item[0]] = decodeURIComponent(item[1]);
});
return result;
}
function doGet(e){
const ctCompl = 'ct_completed';
var doc = SpreadsheetApp.openById(SHEET_KEY);
var sheet = doc.getSheetByName(SHEET_NAME);
var row = [];
if(typeof e !== undefined){
mArr = getJsonFromUrl(e.queryString);
for (i in mArr) if( i == ctCompl) {
row.push(convTimeLong(mArr[i]));
} else
row.push(mArr[i]);
sheet.appendRow(row);
} else {
sheet.appendRow(['e undefined!']);
}
SpreadsheetApp.flush();
return handleResponse(e)
}
function convTimeLong(dateTime) {
let d = new Date();
let dt=dateTime.replace('+', 'T');
try {
var res = Utilities.formatDate(d,"GMT+2", "dd.MM.yyyy HH:mm");
return res
} catch(e){
return dateTime; }
}
executions dashboard show status "completed' everywhere, execution time longest - 1.688 s
Client has set delay 5 second between call's, right now I don't now is percent of lost postback's decreased after delay was set or not, but it still very high.
https://youtu.be/0_H_mVAbp4g
In general, using Google Sheets as a database is a bad idea. It's not designed for this so it could fail really bad. Using a proper database will make everything much, much easier. If you are using the spreadsheet to then cook the data, I'd advise to use a function that imports data like IMPORTXML (see reference).
That being said, if you insist on using Sheets, you could try using locks:
function appendRow(sheet, row) {
const lock = LockService.getScriptLock()
while (!lock.tryLock(100000)) /* Spin the lock until it gets aquired */;
try {
sheet.appendRow(row)
SpreadsheetApp.flush()
} finally {
lock.releaseLock()
}
}
To use it, you only need to pass the sheet and the values to add: sheet.appendRow(row) to appendRow(sheet, row).
It will make sure that entries don't get overridden. Note that this will slow down the code a lot and the script can time out if there are a lot of requests.
I know this question has been asked countless times, but I cant figure out for the life of me how to make this answer work in my case: wait for async javascript function to return
I'm looping through some "tv channels" in the outerloop and then looping through dates in the week in the innerloop. In the inner loop I make a ajax request to a server to fetch the data and I then store/cache it for later use like so
var dates = []; //<-- Contains a list of dates for the coming week
var baseUrl = "http://www.someserver.com";
var storedChannels = [1,2,3,4,5,6,7,8,9,10,45,23,56,34,23,67,23,567,234,67,345,465,67,34];
for(ch = 0; ch < storedChannels.length; ch++) {
var channel = storedChannels[ch];
for(d=0; d < 7; d++) {
var currentDate = dates[d];
ajax({
url: baseUrl+"?ch="+channel+"&dt=currentDate"+,
complete: function(res) {
CMLocalStore.setString('ch' + ch + "_" + scheduleDay, res);
},
});
//Want to wait here till the ajax request completes.
//Do not want to continue to next iteration.
//Do not want to fire of 50 bazillion ajax requests all at once
//Why? Very limited bandwidth scenario, plenty of channels
}
}
PS: NO JQuery please! Plain JS solutions only
Many thanks!
You want something like this. I haven't tested it, but hopefully you should get the idea.
var dates = []; //<-- Contains a list of dates for the coming week
var baseUrl = "http://www.someserver.com";
var storedChannels = [1,2,3,4,5,6,7,8,9,10,45,23,56,34,23,67,23,567,234,67,345,465,67,34];
function ProcessNext(ch, d) {
if (d < 7) {
d++;
} else {
d=0;
if (ch < storedChannels.length) {
ch++;
} else {
return;
}
}
var channel = storedChannels[ch];
var currentDate = dates[d];
ajax({
url: baseUrl+"?ch="+channel+"&dt=currentDate"+,
complete: function(res) {
CMLocalStore.setString('ch' + ch + "_" + scheduleDay, res);
ProcessNext(ch, d);
},
});
}
ProcessNext(0, 0);
You need to turn your loop into a chain of callbacks.
Instead of using a loop, you should make your callback call your original function, but with a higher parameter value.
What you are trying to do is explained in the Asynchronous Iteration Patterns tutorial by Pedro Teixeira. The examples are using Node.js but you can use the same patterns in the browser. Basically what you need to do is convert your loops to serial callbacks waiting on each other to complete, so the next AJAX request is fired from the success callback of the previous one etc. It can be done without blocking the browser but not in loops. See that tutorial.
Essentially the answer lies in using recursive calls instead of using loops. Just wanted to add this answer for anyone that might be interested in "for loop nestings" deeper than 2 levels. As you can see its easy to extend to as many "nestings" as you like.
Original credit goes to VatooVatoo implementation in Java on the DaniWeb forums.
Heres the code, tested and works (without the ajax bits of course but you can add that yourself):
<html>
<head>
<script type="text/javascript">
function loopRecurse(a, b, c)
{
if(c >= 2) {
b++;
c=0;
loopRecurse(a, b, c);
return;
}
if(b >= 2) {
a++;
b=0;
loopRecurse(a, b, c);
return;
}
if(a >= 2) return;
document.write("<div>" + a + "|" + b + "|" + c + "</div>");
c++;
loopRecurse(a, b, c);
}
loopRecurse(0, 0, 0);
</script>
</head>
<body>
<!-- output
0|0|0
0|0|1
0|1|0
0|1|1
1|0|0
1|0|1
1|1|0
1|1|1
-->
</body>
</html>
See the XMLHttpRequest documentation (linked to MDC, but shouldn't matter). Basically the condition you're looking for is request.readyState==4 - presuming that you have your ajax() return the actual XMLHttpRequest object.
new at this, please tell me if I'm leaving information out or anything like that.
The code I'm working on can be seen here: http://codepen.io/hutchisonk/pen/mVyBde and I have also pasted the relevant section of javascript below.
I'm having trouble understanding why this code is behaving as it is. Quick outline - I have defined a few variables at the top, made a function that fetches the data I need and builds it into a pretty little list. This seems to be working as planned.
With the function outlined, I then loop through each "friend" in the "friends" array, calling the function once each time. I have numbered the friends on the output to help clarify what is going on. I have tried this a number of ways, including with the "for loop" syntax that's currently implemented, as well as the "forEach" syntax that's commented out.
Two main questions:
1) The number in front of each name is the "i" in my for loop. Why is this "i" not going in order from 0 to 10? How do I get it to do so? It appears to be in a different order every time the code is run. And, it repeats the numbers it has looped through previously on each new iteration. I would like to understand why this is happening.
2) The code seems to be running out of order. The unexpected behavior can be seen in the console.log - the for loop outputs the first two lines of console.log on a loop, then jumps out and console.logs the test variable "a" and the other text below the for loop, and then jumps back into the for loop and console.logs the output from the function. I'm looking at the console in google chrome and I did read that there can be timing inconsistancies with regard to the console, but I don't understand how the loop is being split in half - the first two lines, and then the function call being logged after the later code.
What is the best way to iterate through an array? Any insights on how to call a function within a loop correctly or resources you can provide are much appreciated.
$("document").ready(function(){
var friends = ["lainzero", "freecodecamp", "storbeck", "terakilobyte", "habathcx","RobotCaleb","thomasballinger","noobs2ninjas","beohoff", "dtphase", "MedryBW"];
var html = "";
var url = "";
function getStreamingData(eachfriend, number) {
url = "https://api.twitch.tv/kraken/streams/"+eachfriend;
$.ajax({
dataType: "jsonp",
url: url,
success: function(result) {
console.log(result+", "+result.stream);
if(result.stream !== null) {
html+= "<li class='streaming'><a href='twitch.tv/"+eachfriend+"'>"+number+": "+eachfriend;
html +="<i class='fa fa-play-circle style='font-size:20px;color:green;''></i>";
} else if (result.stream === null) {
html+= "<li class='not_streaming'><a href='twitch.tv/"+eachfriend+"'>"+number+": "+eachfriend;
html +="<i class='fa fa-stop-circle' style='font-size:20px;color:red;'></i>";
}
html +="</a></li>";
$("#all ul").append(html);
}//success
});//$ajax
}//getstreamingdata function
for (var i=0;i<friends.length;i++) {
console.log(i);
console.log(friends[i]);
getStreamingData(friends[i], i);
}
//Same as for loop above, but using forEach. This produces the same results.
/*
var i=0;
friends.forEach(function(friend) {
getStreamingData(friend, i);
i++;
});
*/
var a = 4;//testing console output
console.log(a);
console.log("why is this showing up before the getStreamingData function's console output?");
console.log("but it's showing up after the console.log(i) and console.lg(friends[i]) output? So this section is interupting the for loop above");
console.log(" and why is the for loop out of order and repeating itself?");
});//doc ready
You are doing an asynchronous task in your loop. You should not expect those async tasks finish in the order that they have started.
The function getStreamingData is the one that I'm talking about.
Related: Asynchronous for cycle in JavaScript
This is one snippet that I wrote long time ago and I'm still using it in small projects. However there are many libraries out there which do the same plus many more.
Array.prototype.forEachAsync = function (cb, end) {
var _this = this;
setTimeout(function () {
var index = 0;
var next = function () {
if (this.burned) return;
this.burned = true;
index++;
if (index >= _this.length) {
if (end) end();
return;
}
cb(_this[index], next.bind({}));
}
if (_this.length == 0) {
if (end) end();
}else {
cb(_this[0], next.bind({}));
}
}, 0);
}
It is not a good practice to touch the prototype like this. But just to give you an idea how you can do this ...
After that code, you can loop over arrays asynchronously. When you are done with one element, call next.
var array = [1, 2, 3, 4]
array.forEachAsync(function (item, next) {
// do some async task
console.log(item + " started");
setTimeout(function () {
console.log(item + " done");
next();
}, 1000);
}, function () {
console.log("All done!");
});
To give you a grasp of what I mean in my title.
Take a look at this code which is before the setInterval stopped working.
var anime = function(){
_.each(db.get('','animedb'), function(site){
var ann = function(){
^ the function is in a var
for (var epid in eps) {
epid = parseInt(epid, 10);
var eptime = (new Date(eps[epid].pubDate[0])*1000)/1000;
if(eptime > site.lastbuilddate){
counter = counter+1;
if(counter < 6){
list.push(font(colors['normal'])+eps[epid].title[0] +' - ['+ utils.secondsToString((new Date() - (eptime+site.delay))/1000, 1)+' ago.]</f>');
}
}
};
^ this is the part that breaks everything after its been edited
var run = setInterval(ann, site.interval*60000);
^ here is the setInterval its at the bottom of the each
anime();
^ here is the call for the whole function that calls the setInterval
The above code is part of an anime announcement for chat rooms owned by anime sites owners using their rss feeds.
The above code works and excuse me for saying this but at this point.
I'm going to say "I have no idea why". Because i really have no idea why setInterval picks and chooses when to work.
I talked to a friend who had more knowledge than me in javascript and time based functions and he said that there are no "conditions" required for setInterval to run.
for (var epid in eps) {
epid = parseInt(epid, 10);
var eptime = (new Date(eps[epid].pubDate[0])*1000)/1000;
if(eptime > site.lastbuilddate){
counter = counter+1;
if(counter < 6){
var url = eps[epid].link.split('//')[1];
var keyword = '';
var words = url.substr(0, url.length-1).split('/').join('-').split('-');
for (var wid in words) {
keyword += words[wid].charAt(0);
}
http.get({hostname:'dev.ilp.moe', port:80, path:'/surl/yourls-api.php?username=usernameremovedforsecurity&password=passwordremovedforsecurity&format=json&action=shorturl&url='+url+'&title='+ctitle+' - '+eps[epid].title[0]+'&keyword='+keyword}, function(r) {
if(r.statusCode === 200) { //200 is success
var b = '';
r.on('data', function(c) {
b += c;
});
r.on('end', function() {
list.push(font(colors['normal'])+eps[epid].title[0] +' - ['+ utils.secondsToString((new Date() - (eptime+site.delay))/1000, 1)+' ago.] - http://dev.ilp.moe/surl/'+keyword+'</f>');
}
}
});
}
}
};
The above code is the part for creating shorturls.
Here is the json DB that is being loaded.
{"0":{"lastbuilddate":1426441081000,"delay":0,"host":"www.animerush.tv","path":"/rss.xml","chats":["animerushtv"],"interval":15},"1":{"lastbuilddate":1424068119000,"delay":28800000,"host":"dubbedanime.tv","path":"/feed/","chats":["dubbed-anime-tv"],"interval":15},"2":{"lastbuilddate":1426415086000,"delay":32400000,"host":"bestanimes.tv","path":"/feed/","chats":["bestanimestv"],"interval":15},"3":{"lastbuilddate":1426434866000,"delay":0,"host":"www.theanime.tv","path":"/feed/","chats":["timecapsule"],"interval":15}}
The recent edit to my code was supposed to implement Shortened links for each episode released using the links provided in the rss feeds from the sites in the database.
The domain http://ilp.moe is my domain.
I have console logged everywhere and tested as much as I possibly could.
At this point I do not understand why the edit is making code that used to be executed by setInterval no longer be executed.
The reason why the code wasn't executed is because the functions were assigned to a variable so they weren't run until it got to setInterval.
When they reach setInterval the errors prevent setInterval from executing (depends on the severity of the error).
after taking the function and just running it without putting it in a var or setInterval and console logging for a bit i found the error was caused by this line
var url = eps[epid].link.split('//')[1];
in this case
eps[epid].link; // === ["http://blah.com/animelolep1"]
my problem was that the var url was trying to split on a list and not a string
here is the fix
var url = eps[epid].link[0].split('//')[1]; // grabs the first item from the list and then splits
I have a list containing folders, and I'm trying to get the count of the total number of files in these folders.
I manage to retrieve a ListItemCollection containing my folders. Then it starts being... picky.
ctx is my ClientContext, and collection my ListItemCollection.
function countFiles()
{
var enumCollection = collection.getEnumerator();
while(enumCollection.moveNext())
{
currentItem = enumCollection.get_current();
var folder = currentItem.get_folder();
if (folder === 'undefined')
return;
ctx.load(folder, 'ItemCount');
ctx.executeQueryAsync(Function.createDelegate(this, function()
{
totalCount += folder.get_itemCount();
}), Function.createDelegate(this, onQueryFailed));
}
}
So it works... half of the time. If I have 6 items in my collection, I get 3 or 4 "The property or field 'ItemCount' has not been initialized" exceptions, and obviously my totalCount is wrong. I just can't seem to understand why, since the executeQueryAsync should not happen before the folder is actually loaded.
I'm very new to Javascript, so it may look horrid and be missing some essential code I didn't consider worthy of interest, feel free to ask if it is so.
Referencing closure variables (like folder in this case) from an asynchronous callback is generally a big problem. Thankfully it's easy to fix:
function countFiles()
{
function itemCounter(folder) {
return function() { totalCount += folder.get_itemCount(); };
}
var enumCollection = collection.getEnumerator();
while(enumCollection.moveNext())
{
var folder = enumCollection.getCurrent().get_folder();
if (folder === undefined) // not a string!
return;
ctx.load(folder, 'ItemCount');
ctx.executeQueryAsync(itemCounter(folder), Function.createDelegate(this, onQueryFailed));
}
}
(You don't need that .createDelegate() call because the function doesn't need this.)
Now, after that, you face the problem of knowing when that counter has been finally updated. Those asynchronous callbacks will eventually finish, but when? You could keep a separate counter, one for each query you start, and then decrement that in the callback. When it drops back to zero, then you'll know you're done.
Since SP.ClientContext.executeQueryAsync is an async function it is likely that the loop could be terminated before the first call to callback function completes, so the behavior of specified code could be unexpected.
Instead, i would recommend another and more clean approach for counting files (including files located under nested folders) using SharePoint JSOM.
How to count the total number of files in List using JSOM
The following function allows to count the number of list items in List:
function getItemsCount(listTitle, complete){
var ctx = SP.ClientContext.get_current();
var list = ctx.get_web().get_lists().getByTitle(listTitle);
var items = list.getItems(createQuery());
ctx.load(items);
ctx.executeQueryAsync(
function() {
complete(items.get_count());
},
function() {
complete(-1);
}
);
function createQuery()
{
var query = new SP.CamlQuery();
query.set_viewXml('<View Scope="RecursiveAll"><Query><Where><Eq><FieldRef Name="FSObjType" /><Value Type="Integer">0</Value></Eq></Where></Query></View>');
return query;
}
}
Usage
getItemsCount('Documents', function(itemsCount){
console.log(String.format('Total files count in Documents library: {0}',itemsCount));
});