My Chrome app has a function that asks for a file to be loaded by another function, checks that the function has set a flag signifying success (External.curFile.lodd), then attempts to process it. My problem is that the flags are not set the first time I call the function, but when I call it a second time the flags are already set.
I had a feeling this has to do with Chrome file functions being asynchronous, so I had the first function idle for a bit while the file loads. The first load never succeeds, no matter how long I wait, but the second load always does!
Calling Function:
function load_by_lines_from_cur_dir( fileName, context ){ // determine the 'meaning' of a file line by line, return last 'meaning', otherwise 'null'
var curLineMeaning = null;
var lastLineValid = true;
External.read_file_in_load_path(fileName); // 'External' load 'fileName' and reads lines, REPLacement does not see this file
// This is a dirty workaround that accounts for the fact that 'DirectoryEntry.getFile' is asynchronous, thus pre-parsing checks fail intil loaded
var counter = 0, maxLoops = 10;
nuClock();
do{
sleep(500);
counter++;
preDebug.innerText += '\r\nLoop:' + counter + " , " + time_since_last();
}while( !External.curFile.lodd && (counter < maxLoops) ); //idle and check if file loaded, 5000ms max
preDebug.innerText += '\r\nLoaded?:' + External.curFile.lodd;
preDebug.innerText += '\r\nLines?:' + External.curFile.lins;
if( External.curFile.lodd ){ // The last load operating was successful, attempt to parse and interpret each line
// parse and interpret lines, storing each meaning in 'curLineMeaning', until last line is reached
while(!External.curFile.rEOF){
curLineMeaning = meaning( s( External.readln_from_current_file() ), context);
preDebug.innerText += '\r\nNext Line?: ' + External.curFile.lnnm;
preDebug.innerText += '\r\nEOF?: ' + External.curFile.rEOF;
}
} // else, return 'null'
return curLineMeaning; // return the result of the last form
}
which calls the following:
External.read_file_in_load_path = function(nameStr){ // Read the lines of 'nameStr' into 'External.curFile.lins'
External.curPath.objt.getFile( // call 'DirectoryEntry.getFile' to fetch a file in that directory
nameStr,
{create: false},
function(fileEntry){ // action to perform on the fetched file, success
External.curFile.name = nameStr; // store the file name for later use
External.curFile.objt = fileEntry; // store the 'FileEntry' for later use
External.curFile.objt.file( function(file){ // Returns 'File' object associated with selected file. Use this to read the file's content.
var reader = new FileReader();
reader.onload = function(e){
External.curFile.lodd = true; // File load success
};
reader.onloadend = function(e){
//var contents = e.target.result;
// URL, split string into lines: http://stackoverflow.com/questions/12371970/read-text-file-using-filereader
External.curFile.lins = e.target.result.split('\n'); // split the string result into individual lines
};
reader.readAsText(file);
External.curFile.lnnm = 0; // Set current line to 0 for the newly-loaded file
External.curFile.rEOF = false; // Reset EOF flag
// let's try a message instead of a flag ...
/*chrome.runtime.sendMessage({greeting: "hello"}, function(response) {
console.log(response.farewell);
});*/
} );
},
function(e){ External.curFile.lodd = false; } // There was an error
);
};
This app is a dialect of Scheme. It's important that the app knows that the source file has been loaded or not.
I didn't read through all of your code, but you can't kick off an asynchronous activity and then busy-wait for it to complete, because JavaScript is single threaded. No matter what's happened, the asynchronous function won't be executed until the script completes its current processing. In other words, asynchronous does not imply concurrent.
Generally speaking, if task A is to be performed after asynchronous task B completes, you should execute A from the completion callback for B. That's the straightforward, safe way to do it. Any shortcut, to achieve better responsiveness or to simplify the code, is going to have dependency or race-condition problems, and will require lots of horsing around to get right. Even then, it will be hard to prove that the code operates correctly on all platforms in all circumstances.
Related
I'm writing a Tampermonkey script for a web page and trying to extract data from other pages.
I'm trying to make a function that has a loop inside that goes thru a list, llcList, and retrieves data from ajax method GET, but would like to wait for to finish one request before going to second one.
Bonus would be if I could make it wait some extra time.
What should happen:
send request for a llcList[0]
get return data, process it
wait some time
send new request for a llcList[1]
Is this possible? I tried few methods, every time loop send all requests not a second apart. :
function F_Company_LLC(){
for (i = 0; i < llcList.length;i++) {
if(llcList[i][2]=="lab"){
//run function 0
//break;
}
else if(llcList[i][2]=="shop"){
//run function 1
//break;
}
else{
F_GET_CompData(llcList, llcList[i][1],i,function(result){
console.log(result);
});
}
}}
function F_GET_CompData(F_GET_CompData_list, CompID, F_GET_CompData_row, callback){
$.ajax({
method : "GET",
url: base_link+"/company/edit_company/"+CompID,
beforeSend: function(){runningRequest++;},
success: function(data){
//data processing
runningRequest--;
},
error: function() {console.log("Get_ComData");}
});
callback(runningRequest);}
This is a common scenario. Note that it's often unnecessary to process the calls sequentially though. It's usually adequate to just send context with the ajax calls and piece everything together as it comes in semi randomly, as shown in this answer.
One way to force sequential behavior is to chain calls via the complete function. Here is fully functional code that demonstrates the process. To use, paste it into your browser console while on a Stack Overflow page. :
var listO_pages = ["q/48/", "q/27/", "q/34/", "q/69/", "badpage"];
var numPages = listO_pages.length;
getPageN (0); //-- Kick off chained fetches
function getPageN (K) {
if (K >= 0 && K < numPages) {
let targPage = listO_pages[K];
$.ajax ( {
url: "https://stackoverflow.com/" + targPage,
context: {arryIdx: K}, // Object Helps handle K==0, and other things
success: processPage,
complete: finishUpRequest,
error: logError
} );
}
}
function processPage (sData, sStatus, jqXHR) {
//-- Use DOMParser so that images and scripts don't get loaded (like jQuery methods would).
var parser = new DOMParser ();
var doc = parser.parseFromString (sData, "text/html");
var payloadTable = doc.querySelector ("title");
var pageTitle = "Not found!";
if (payloadTable) {
pageTitle = payloadTable.textContent.trim ();
}
var [tIdx, tPage] = getIdxAndPage (this); // Set by `context` property
console.log (`Processed index ${tIdx} (${tPage}). Its title was: "${pageTitle}"`);
}
function finishUpRequest (jqXHR, txtStatus) {
var nextIdx = this.arryIdx + 1;
if (nextIdx < numPages) {
var tPage = listO_pages[nextIdx];
//-- The setTimeout is seldom needed, but added here per OP's request.
setTimeout ( function () {
console.log (`Fetching index ${nextIdx} (${tPage})...`);
getPageN (nextIdx);
}, 222);
}
}
function logError (jqXHR, txtStatus, txtError) {
var [tIdx, tPage] = getIdxAndPage (this); // Set by `context` property
console.error (`Oopsie at index ${tIdx} (${tPage})!`, txtStatus, txtError, jqXHR);
}
function getIdxAndPage (contextThis) {
return [contextThis.arryIdx, listO_pages[contextThis.arryIdx] ];
}
This typically outputs:
Processed index 0 (q/48/). Its title was: "Multiple submit buttons in an HTML form - Stack Overflow"
Fetching index 1 (q/27/)...
Processed index 1 (q/27/). Its title was: "datetime - Calculate relative time in C# - Stack Overflow"
Fetching index 2 (q/34/)...
Processed index 2 (q/34/). Its title was: "flex - Unloading a ByteArray in Actionscript 3 - Stack Overflow"
Fetching index 3 (q/69/)...
Processed index 3 (q/69/). Its title was: ".net - How do I calculate someone's age in C#? - Stack Overflow"
Fetching index 4 (badpage)...
GET https://stackoverflow.com/badpage?_=1512087299126 404 ()
Oopsie at index 4 (badpage)! error Object {...
-- depending on your Stack Overflow reputation.
Important: Do not attempt to use async: false techniques. These will just: lock up your browser, occasionally crash your computer, and make debug and partial results much harder.
I am trying to stream mp3 data from my server to the client side. I am doing this using Ajax. The server sends 50 kilobytes per request. I wrote two functions: one that gets the mp3 data and one that plays them. The first function takes the 50 kilobytes, decodes them and stores the decoded data in an array then it calls itself recursively. The second function starts playing as soon as the first element in the array is filled with data. The problem is that this works for the first 50kilobytes only then it fails. What I want to do is keep my get_buffer function running until the server tells it no more data to send, and keep my play() function playing data until there is no more elements in the array.
Here is my two functions:
function buffer_seg() {
// starts a new request
buff_req = new XMLHttpRequest();
// Request attributes
var method = 'GET';
var url = '/buffer.php?request=buffer&offset=' + offset;
var async = true;
// set attributes
buff_req.open(method, url, async);
buff_req.responseType = 'arraybuffer';
// keeps loading until something is recieved
if (!loaded) {
change_icon();
buffering = true;
}
buff_req.onload = function() {
segment = buff_req.response;
// if the whole file was already buffered
if (segment.byteLength == 4) {
return true;
} else if (segment.byteLength == 3) {
return false;
}
// sets the new offset
if (offset == -1) {
offset = BUFFER_SIZE;
} else {
offset += BUFFER_SIZE;
}
//decodes mp3 data and adds it to the array
audioContext.decodeAudioData(segment, function(decoded) {
buffer.push(decoded);
debugger;
if (index == 0) {
play();
}
});
}
buff_req.send();
buff_seg();
}
Second function:
function play() {
// checks if the end of buffer has been reached
if (index == buffer.length) {
loaded = false;
change_icon();
if (buffer_seg == false) {
stop();
change_icon();
return false;
}
}
loaded = true;
change_icon();
// new buffer source
var src = audioContext.createBufferSource();
src.buffer = buffer[index++];
// connects
src.connect(audioContext.destination);
src.start(time);
time += src.buffer.duration;
src.onended = function() {
src.disconnect(audioContext.destination);
play();
}
}
The recursive call to buffer_seg is in the main body of buffer_seg, not in the callback, so it happens immediately - not, as you seem to intend, after a response is received. Second, this also means that the recursive call is unconditional when it should be based on whether the previous response indicated more data would be available. If this isn't just crashing your browser, I'm not sure why. It also means that chunks of streamed audio could be pushed into the buffer out of order.
So to start I'd look at moving the recursive call to the end of the onload handler, after the check for end of stream.
In the 2nd function, what do you intend if (buffer_seg == false) to do? This condition will never be met. Are you thinking this is a way to see the last return value from buffer_seg? That's not how it works. Perhaps you should have a variable that both functions can see, which buffer_seg can set and play can test, or something like that.
I have written a service to download files from an external partner site. There are around 1000 files of 1 MB each. My process is going out of memory every time I reach around 800 files.
How should I identify the root cause ?
var request = require('sync-request');
var fs = require('graceful-fs')
function find_starting_url(xyz_category){
feed_url = "<url>"
response = request("GET", feed_url).getBody().toString()
response = JSON.parse(response)
apiListings = response['apiGroups']['affiliate']['apiListings']
starting_url = apiListings[xyz_category]['availableVariants']['v0.1.0']['get']
return starting_url
}
function get_all_files(feed_category, count, next_url, retry_count){
var headers = {
'Id': '<my_header>',
'Token': '<my key>'
}
console.log(Date())
console.log(count)
if(next_url){
products_url = next_url
}
else{
products_url = find_starting_url(feed_category)
}
try{
var products = request("GET", products_url, {"headers": headers}).getBody().toString()
var parsed = JSON.parse(products)
var home = process.env.HOME
var fd = fs.openSync(home + "/data/abc/xyz/" + feed_category + "/" + count + ".json", 'w')
fs.writeSync(fd, products)
fs.closeSync(fd)
next_url = parsed['nextUrl']
count++;
if(next_url){
get_all_files(feed_category, count, next_url)
}
}catch(e){
if(retry_count >= 5){
console.log("TERRIBLE ENDING!!!", e)
}else{
retry_count++;
console.log("some error... retrying ..", e)
get_all_files(feed_category, count, next_url, retry_count)
}
}
}
var feed_category = process.argv[2]
get_all_files(feed_category, 1)
You're calling a synchronous function recursively so every single request you have and all the data from each request is retained in memory in your local variables until all of the requests are done and all the recursive calls can unwind and then finally free all the sets of local variables. This requires monster amounts of memory (as you have discovered).
It would be best to restructure your code so that the current request is processed, written to disk and then nothing from that request is retained when it goes onto the next request. The simplest way to do that would be to use a while loop instead of a recursive call. In pseudo code:
initialize counter
while (more to do) {
process the next item
increment counter
}
I don't understand the details of what your code is trying to do well enough to propose a rewrite, but hopefully you can see how you can replace the recursion with the type of non-recursive structure above.
It's because you are performing a recursive call to the get_all_files function and it's keeping the body variable in memory for every single execution, since every child execution needs to be completed before the memory is released.
i am trying to write a text file when suspend event is fired i.e inside winjs.application.oncheckpoint event handler.
i am writing my object as JSON text.
here is the code:
applicationData.localFolder.createFileAsync("dataFile.txt", Windows.Storage.CreationCollisionOption.replaceExisting).then(function (sampleFile) {
var stringData = "";
var i;
for (i = 0 ; i < myData.objData.length - 1 ; i++) {
stringData += '{"title":"' + myData.objData[i].title + '","challange":"' + myData.objData[i].challange + '"},\n';
}
stringData += '{"title":"' + myData.objData[i].title + '","challange":"' + myData.objData[i].challange + '"}';
stringData = "[" + stringData + "]";
return Windows.Storage.FileIO.writeTextAsync(sampleFile, stringData);
}).done(function () { });
but windows app closes before writing anything into the text file.
I called args.setPromise() and passed the above code as a function parameter but same problem persists.
PS: i don't know how to do it asynchronously correctly.
please help.
like Phil has commented, remove .done() will fix it.
This is because done() does not return anything whereas then() returns chained promise. That is the reason nothing is getting called.
It is also recommended to save your application data as soon as it changes or at some intervals and not wait for the suspend event to save everything. Suspend event gives time of 5s. otherwise, the app is terminated as per msdn documentation here.
"If an app does not return from the suspending event within 5 seconds, Windows assumes that the app has stopped responding and terminates it"
app.oncheckpoint = function (args) {
app.sessionState.history = nav.history;
args.setPromise(Windows.Storage.ApplicationData.current.localFolder.createFileAsync
("dataFile.txt", Windows.Storage.CreationCollisionOption.replaceExisting).then(function (sampleFile) {
var stringData;
// code to set stringData.
return Windows.Storage.FileIO.writeTextAsync(sampleFile, stringData);
}));
};
Just event.setPromise(applicationData.localFolder.createFileAsync ...)
Then the suspend will wait until your promise finished. Remove the .done
I know this question has been asked several times, but I couldn't seem to find a solution that worked for me in any of the previous questions. I have a variable that gets set when my HTML page is done loading, but sometimes when my code tries to access that variable, it says that it is undefined. I'm not sure why, since I believe I am waiting for everything to load properly. This exception seems to happen randomly, as most of the time all the code runs fine. Here's a simplified version of my code:
var globalVar;
function initStuff(filePath) {
// I wait till the HTML page is fully loaded before doing anything
$(document).ready(function(){
var video = document.getElementById("videoElementID");
// My parseFile() function seems to run smoothly
var arrayOfStuff = parseFile(filePath);
if (arrayOfStuff == null) {
console.error("Unable to properly parse the file.");
} else {
setGlobalVariable(arrayOfStuff);
video.addEventListener("play", updateVideoFrame, false);
}
});
}
function setGlobalVariable(arrayOfStuff) {
window.globalVar = arrayOfStuff;
}
function updateVideoFrame() {
// A bunch of other code happens first
// This is the line that fails occasionally, saying
// "window.globalVar[0].aProperty.anArray[0] is undefined"
var test = window.globalVar[0].aProperty.anArray[0].aProperty;
}
The only thing that I can think of that might be causing this problem is some sort of synchronicity issue. I don't see why that would be the case, though. Help please!
Edit:
In case the asynchronicity issue is coming from my parseFile(xmlFile) method, here is what I'm doing there. I thought it couldn't possibly be causing the issue, since I force the method to happen synchronously, but in case I'm wrong, here it is:
function parseKML(xmlFile) {
var arrayOfStuff = new Array();
// Turn the AJAX asynchronicity off for the following GET command
$.ajaxSetup( { async : false } );
// Open the XML file
$.get(xmlFile, {}, function(xml) {
var doc = $("Document", xml);
// Code for parsing the XML file is here
// arrayOfStuff() gets populated here
});
// Once I'm done processing the XML file, I turn asynchronicity back on, since that is AJAX's default state
$.ajaxSetup( { async : true } );
return arrayOfStuff;
}
The first thing you should do in your code is figure out which part of:
window.globalVar[0].aProperty.anArray[0]
is undefined.
Since you have multiple chained property references and array references, it could be many different places in the chain. I'd suggest either set a breakpoint right before your reference it examine what's in it or use several console.log() statement sto output each nested piece of the structure in order to find out where your problem is.
console.log("globalVar = " + globalVar);
console.log("globalVar[0] = " + globalVar[0]);
console.log("globalVar[0].aProperty = " + globalVar[0].aProperty);
console.log("globalVar[0].aProperty.anArray = " + globalVar[0].aProperty.anArray);
console.log("globalVar[0].aProperty.anArray[0] = " + globalVar[0].aProperty.anArray[0]);
If the problem is that globalVar isn't yet set, then you have a timing problem or an initialization problem.
If the problem is that one of the other properties isn't set, then you aren't initializing globalVar with what you think you are.
You may also want to write your code more defensibly so it fails gracefully if some of your data isn't set properly.
You need to use defensive programming.
http://www.javascriptref.com/pdf/ch23_ed2.pdf
Example:
var video = document.getElementById("videoElementID") || 0;
-
if( video && video.addEventListener ){
video.addEventListener("play", updateVideoFrame, false);
}
Here's another version of your code.
window.globalVar = globalVar || [];
function setGlobalVariable(arrayOfStuff) {
window.globalVar = arrayOfStuff;
}
function updateVideoFrame() {
// A bunch of other code happens first
// This is the line that fails occasionally, saying
// "window.globalVar[0].aProperty.anArray[0] is undefined"
if( window.globalVar ){
var g = window.globalVar || [];
var d = (g[0] || {})["aProperty"];
// etc...
}else{
console.error( "test error." );
}
}
function initStuff(filePath) {
// I wait till the HTML page is fully loaded before doing anything
$(document).ready(function () {
var video = $("#videoElementID");
// My parseFile() function seems to run smoothly
var arrayOfStuff = parseFile(filePath) || [];
if (arrayOfStuff == null || video == null ) {
console.error("Unable to properly parse the file.");
} else {
setGlobalVariable(arrayOfStuff);
video.bind("play", updateVideoFrame);
}
});
}