Develop Reference

How to loop through GET/POST calls sequentially (waiting for previous) return?

I'm writing a Tampermonkey script for a web page and trying to extract data from other pages.
I'm trying to make a function that has a loop inside that goes thru a list, llcList, and retrieves data from ajax method GET, but would like to wait for to finish one request before going to second one.
Bonus would be if I could make it wait some extra time.
What should happen:
send request for a llcList[0]
get return data, process it
wait some time
send new request for a llcList[1]
Is this possible? I tried few methods, every time loop send all requests not a second apart. :
function F_Company_LLC(){
for (i = 0; i < llcList.length;i++) {
if(llcList[i][2]=="lab"){
//run function 0
//break;
}
else if(llcList[i][2]=="shop"){
//run function 1
//break;
}
else{
F_GET_CompData(llcList, llcList[i][1],i,function(result){
console.log(result);
});
}
}}
function F_GET_CompData(F_GET_CompData_list, CompID, F_GET_CompData_row, callback){
$.ajax({
method : "GET",
url: base_link+"/company/edit_company/"+CompID,
beforeSend: function(){runningRequest++;},
success: function(data){
//data processing
runningRequest--;
},
error: function() {console.log("Get_ComData");}
});
callback(runningRequest);}

This is a common scenario. Note that it's often unnecessary to process the calls sequentially though. It's usually adequate to just send context with the ajax calls and piece everything together as it comes in semi randomly, as shown in this answer.
One way to force sequential behavior is to chain calls via the complete function. Here is fully functional code that demonstrates the process. To use, paste it into your browser console while on a Stack Overflow page. :
var listO_pages = ["q/48/", "q/27/", "q/34/", "q/69/", "badpage"];
var numPages = listO_pages.length;
getPageN (0); //-- Kick off chained fetches
function getPageN (K) {
if (K >= 0 && K < numPages) {
let targPage = listO_pages[K];
$.ajax ( {
url: "https://stackoverflow.com/" + targPage,
context: {arryIdx: K}, // Object Helps handle K==0, and other things
success: processPage,
complete: finishUpRequest,
error: logError
} );
}
}
function processPage (sData, sStatus, jqXHR) {
//-- Use DOMParser so that images and scripts don't get loaded (like jQuery methods would).
var parser = new DOMParser ();
var doc = parser.parseFromString (sData, "text/html");
var payloadTable = doc.querySelector ("title");
var pageTitle = "Not found!";
if (payloadTable) {
pageTitle = payloadTable.textContent.trim ();
}
var [tIdx, tPage] = getIdxAndPage (this); // Set by `context` property
console.log (`Processed index ${tIdx} (${tPage}). Its title was: "${pageTitle}"`);
}
function finishUpRequest (jqXHR, txtStatus) {
var nextIdx = this.arryIdx + 1;
if (nextIdx < numPages) {
var tPage = listO_pages[nextIdx];
//-- The setTimeout is seldom needed, but added here per OP's request.
setTimeout ( function () {
console.log (`Fetching index ${nextIdx} (${tPage})...`);
getPageN (nextIdx);
}, 222);
}
}
function logError (jqXHR, txtStatus, txtError) {
var [tIdx, tPage] = getIdxAndPage (this); // Set by `context` property
console.error (`Oopsie at index ${tIdx} (${tPage})!`, txtStatus, txtError, jqXHR);
}
function getIdxAndPage (contextThis) {
return [contextThis.arryIdx, listO_pages[contextThis.arryIdx] ];
}
This typically outputs:
Processed index 0 (q/48/). Its title was: "Multiple submit buttons in an HTML form - Stack Overflow"
Fetching index 1 (q/27/)...
Processed index 1 (q/27/). Its title was: "datetime - Calculate relative time in C# - Stack Overflow"
Fetching index 2 (q/34/)...
Processed index 2 (q/34/). Its title was: "flex - Unloading a ByteArray in Actionscript 3 - Stack Overflow"
Fetching index 3 (q/69/)...
Processed index 3 (q/69/). Its title was: ".net - How do I calculate someone's age in C#? - Stack Overflow"
Fetching index 4 (badpage)...
GET https://stackoverflow.com/badpage?_=1512087299126 404 ()
Oopsie at index 4 (badpage)! error Object {...
-- depending on your Stack Overflow reputation.
Important: Do not attempt to use async: false techniques. These will just: lock up your browser, occasionally crash your computer, and make debug and partial results much harder.

How to get and append most recent messages from server using jQuery and AJAX?

I'm working on my first simple chat application and this issue has me stuck. I know what I'm trying to do, but I end up overthinking it.
Basically, I have this heroku server going:
http://tiy-fee-rest.herokuapp.com/collections/blabberTalk
Whenever someone sends a message, it is added to this array.
My Issue:
I have it on a set interval so that every 2 seconds, it runs the getNewestMessages function. When this setInterval is working and someone sends a message, it will keep appending the last message they sent every 2 seconds. If I disable the setInterval and simply call the getNewestMessages function myself in a separate browser tab, this doesn't seem to happen. I want to make it so that the most recently sent message isn't constantly re-appended to the DOM when the setInterval is active.
This is the function I'm using to check for recent messages. It's pretty bloated, sorry about that:
getNewestMessages: function() {
$.ajax({
url: http://tiy-fee-rest.herokuapp.com/collections/blabberTalk,
method: 'GET',
success: function (data) {
// Finds Id of most recent message displayed in the DOM
var recentId = $('.message').last().data('id');
var prevMostRecent = 0;
var newMostRecent = [];
jQuery.each(data, function(idx,el){
if (el._id === recentId) {
// if one of the messages on the server has an Id equal to
// one of the messages in the DOM, it saves its index in a var
prevMostRecent = idx;
}
});
jQuery.each(data, function(idx,el){
if (idx < prevMostRecent) {
// if there are messages on the server with a lower index than
// the most recent message in the DOM, it pushes them to a new
// array. Basically, creates a new array of any messages newer
// than the last one displayed in the DOM.
newMostRecent.push(el);
}
});
for (var i = 0; i < newMostRecent.length; i++) {
console.log(newMostRecent[i]);
if (newMostRecent[i]._id === $('.message').last().data('id')) {
// My attempt at trying to remove the last DOM message from
// the array of newer messages. My main issue was that this
// whole function would keep appending the most recent message
// over and over again.
var result = _.without(newMostRecent, newMostRecent[i]);
console.log('MESSAGE TO BE EXCLUDED: ', newMostRecent[i]);
// If the array of newer messages contained the most recent
// DOM message, it removes it and sends it to be appended.
page.appendNewestMessages(result);
}
}
// If the array of newer messages DOESN'T contain the most recent
// DOM message, it just sends the whole array normally.
page.appendNewestMessages(newMostRecent);
},
error: function (err) {
}
});
}
Here is the append function:
appendNewestMessages: function(messagesToAppend) {
console.log(messagesToAppend.reverse());
_.each(messagesToAppend.reverse(), function(el, idx, arr) {
var newMessage = {
content: el.content,
timestamp: el.timestamp,
author: el.author,
userIcon: el.userIcon
}
$.ajax({
url: page.url,
method: 'POST',
data: newMessage,
success: function (data) {
page.addOneMessageToDOM(data);
},
error: function (err) {
console.log("error ", err);
}
});
})
}
Can anyone help me understand how to get the most recent messages from a server and append them to the DOM without any repeats? This has been driving me nuts.
Thanks for any and all help.

Unknown amount of Ajax Request inside loop

So here is my problem (and I have tried several suggestions found here at stackOverflow):
Scenario:
I am using the Gitlab API and I want to list all the "issues" of the bug tracker present on the system for a given project.
Problem:
This is all fine and good, however there is a paging system to do this since the ajax requesst is limited to 100 entries per response.
So I have to do something like this:
$.ajax({
url: "https://mygitlabURL/api/v3/projects/97/issues",
type: "GET",
data: {
private_token: "mytoken",
per_page: 100,
page: 1
}
This will give me back 100 entries. What I need to do is add these entries to a list, and check: "was there fewer than 100 entries in the response?" if so I can stop doing requests.
I need however to get the complete list before I can move on with my code, so I tried using $.when() function and do my requests in a function of its own. In this function I have tried using:
closures like in this answer
recursion like suggested in another answer (don't have the link)
while loop since, oh well, why not
The problem with all the solutions is that, beeing asynchronous, I end up receiving a response and my $.when() function executes before I have any response from the server.
Is there a way to do this?
Here is the latest code (with recursion) I have tried:
$(function () {
$("button").on("click", function () {
$.when(func1(), func2()).then(finishedFunc);
});
});
var func1 = function (pageNr) {
pageNr = pageNr || 1;
megaList = [];
// Get server values
$.ajax({
url: "https://mygitlabURL/api/v3/projects/97/issues",
type: "GET",
data: {
private_token: "myToken",
per_page: 100,
page: pageNr
},
success: function (issuesList) {
console.log("Page number: " + pageNr);
megaList = [pageNr];
if (issuesList.length < 100) {
return megaList;
}
pageNr = pageNr +1 ;
var received = func1(pageNr);
megaList = $.merge(megaList, received);
return megaList;
}
});
}
var func2 = function () {
return 20;
}
var finishedFunc = function (resp1, resp2) {
console.log("Responses were resp1: " + resp1 + " and resp2: " + resp2);
}
And I always get something like:
"Responses were resp1: undefined and resp2: 20"
And I am expecting something like:
"Responses were resp1: [1, 2, 3, 4, 5, ..., 27] and resp2: 20"
As stated before, I can't find any solutions that resolve my problem here in the forums, but if I might have overlooked something, please point me in the right way.

While reading the documentation.I came across this.
Pagination
When listing resources you can pass the following parameters:
page (default: 1) - page number
per_page (default: 20, max: 100) - number of items to list per page
Link headers are send back with each response. These have rel prev/next/first/last and contain the relevant URL. Please use these instead of generating your own URLs.
It automatically says that the Response that came back will contain rel prev/next/first/last. So you can easily check that link headers contain next rel or not and If it contain then directly call that url for more issues and If not that means it does not contain more issues.

Once you start to think in async terms the solution become pretty simple:
var megaList = [];
function loadList(page) {
page = Math.max(1, page);
$.ajax({
url: "https://mygitlabURL/api/v3/projects/97/issues",
type: "GET",
data: {
private_token: "myToken",
per_page: 100,
page: page
},
success: function (issuesList) {
console.log("Page number: " + page);
megaList = megaList.concat(issuesList);
if (issuesList.length >= 100) loadList(page+1);
}
});
}
loadList();

How can I convert background jobs to something like functions?

I am making a newsreader app and using Parse.com background jobs to collect links from RSS feed of the newspaper. I have used xmlreader.js and sax.js to parse the httpResponse and using saveAll and beforeSave, periodically update the classes in data browser.
I have multiple newspapers with multiple categories making a total of more than 30 pairs, (I would have to later include more pair as I would like to include regional newspapers). Till now I was working with one newspaper and one category - The Hindu, sports category; and it is now working fine. Making copies of these two function and create jobs wont be efficient I think.
Therefore, I wanted to ask if I can convert both these jobs and beforeSave into some kind of function so that I can just pass in either newspaper-category pair class name or its url to do the stuff automatically.
Full Code - main.js
job -
Parse.Cloud.job("job_hindu_sports", function (request, response) {
return Parse.Cloud.httpRequest({
url: 'http://www.thehindu.com/sport/?service=rss'
}).then(function(httpResponse) {
var someXml = httpResponse.text;
xmlreader.read(someXml, function (err, res){
if(err) {
response.error("Error " +err);
return console.log(err);
}
var listArray = [];
res.rss.channel.item.each(function (i, item){
var hinduSports = new HinduSports(); //#startswithaj - this part
hinduSports.set("link", item.link.text());
hinduSports.set("title", item.title.text());
hinduSports.set("pubDate", item.pubDate.text());
//console.log("pubDate - "+ item.pubDate.text());
listArray.push(hinduSports);
});
var promises = [];
Parse.Object.saveAll(listArray, {
success: function(objs) {
promises.push(objs);
console.log("SAVED ALL!");
},
error: function(error) {
console.log("ERROR WHILE SAVING - "+error);
}
});
return Parse.Promise.when(promises);
});
}).then(function() {
response.success("Saving completed successfully.");
},function(error) {
response.error("Uh oh, something went wrong.");
});
});
beforeSave -
Parse.Cloud.beforeSave("HinduSports", function(request, response) {
//console.log("in beforeSave");
var query = new Parse.Query(HinduSports);
var linkText = request.object.get("link")
var titleText = request.object.get("title");
query.equalTo("link", linkText);
query.first({
success: function(object) {
//console.log("in query");
if (object) {
//console.log("found");
if(object.get('title')!==titleText){
console.log("title not same");
object.set("title", titleText);
response.success();
}
else{
console.log("title same");
response.error();
}
} else {
console.log("not found");
response.success();
}
},
error: function(error) {
response.error();
}
});
});

In your job code you could query your datastore for all of the URLS you want to process, and then iterate through the results requesting each url and passing the httpresponse to a function that does all the work
So you would have (pseudo code)
function getDataForNewspaper(id, url){
return (function(id) {
Parse.Cloud.httpRequest({
url: url
}).then(function(httpResponse){
processDataForNewspaper(id, httpResponse)
})
})(id) //you need to have this in a closure so you can pass id to processDataFor...
function processDataforNewpaper(id, httpResponse){
someXml = httpResponse.text
//process your xml here
}
Parse.Cloud.job("get_data_for_all_newspapers", function (request, response) {
var query = new Parse.Query("Get all the newspapers").find{
success: function(list){
for each newspaper in list then
getDataForNewspaper(newspaper.id, newspaper.url)
}
}
}
It's not the best explanation but I hope this helps

With the help from #startswithaj I modified my code to save all the articles in one class. The only thing left is to add a beforeSave method. But there is still a problem. saveAll gets completed only sometimes. For eg. I ran the code first time and got this in log :
I2014-04-26T18:18:40.036Z] v93: Ran job job_get_data_for_all_newspapers with:
Input: {}
Result: Saving completed successfully.
I2014-04-26T18:18:40.926Z] Successfully retrieved 2
I2014-04-26T18:18:40.926Z] getData NEW & CAT ID - 1, 5 feedUrl http://www.thehindu.com/sport/?service=rss
I2014-04-26T18:18:40.927Z] getData NEW & CAT ID - 1, 4 feedUrl http://www.thehindu.com/news/national/?service=rss
I2014-04-26T18:18:40.927Z] promisesGetNP [object Object],[object Object]
I2014-04-26T18:18:41.479Z] processData NEW & CAT ID - 1, 5
I2014-04-26T18:18:41.622Z] listArray http://www.thehindu.com/sport/other-sports/mankirat-singh-sets-record/article5951540.ece?utm_source=RSS_Feed&utm_medium=RSS&utm_campaign=RSS_Syndication
I2014-04-26T18:18:41.628Z] promises undefined
I2014-04-26T18:18:41.629Z] promisesGetData
I2014-04-26T18:18:41.629Z] Done getData?
I2014-04-26T18:18:42.082Z] processData NEW & CAT ID - 1, 4
I2014-04-26T18:18:42.311Z] listArray http://www.thehindu.com/news/national/muslim-women-entitled-to-maintenance-even-after-divorce-supreme-court/article5951562.ece?utm_source=RSS_Feed&utm_medium=RSS&utm_campaign=RSS_Syndication
I2014-04-26T18:18:42.324Z] promises undefined
I2014-04-26T18:18:42.324Z] promisesGetData
I2014-04-26T18:18:42.324Z] Done getData?
I2014-04-26T18:18:42.324Z] done job
and second time after deleting a few useless console.log I got this. You can see there is a SAVED ALL! which is called in the success: function of the saveAll -
I2014-04-26T18:20:53.130Z] v94: Ran job job_get_data_for_all_newspapers with:
Input: {}
Result: Saving completed successfully.
I2014-04-26T18:20:53.307Z] Successfully retrieved 2
I2014-04-26T18:20:53.307Z] getData NEW & CAT ID - 1, 5 feedUrl http://www.thehindu.com/sport/?service=rss
I2014-04-26T18:20:53.307Z] getData NEW & CAT ID - 1, 4 feedUrl http://www.thehindu.com/news/national/?service=rss
I2014-04-26T18:20:53.911Z] processData NEW & CAT ID - 1, 5
I2014-04-26T18:20:53.951Z] listArray http://www.thehindu.com/sport/other-sports/mankirat-singh-sets-record/article5951540.ece?utm_source=RSS_Feed&utm_medium=RSS&utm_campaign=RSS_Syndication
I2014-04-26T18:20:53.995Z] Done getData?
I2014-04-26T18:20:54.200Z] SAVED ALL!
I2014-04-26T18:20:54.818Z] processData NEW & CAT ID - 1, 4
I2014-04-26T18:20:55.016Z] listArray http://www.thehindu.com/news/national/muslim-women-entitled-to-maintenance-even-after-divorce-supreme-court/article5951562.ece?utm_source=RSS_Feed&utm_medium=RSS&utm_campaign=RSS_Syndication
I2014-04-26T18:20:55.031Z] Done getData?
I2014-04-26T18:20:55.031Z] done job
My new code can be found here. The new code starts at Line 150.

Interrupting an looping function

I trigger the following function when a tooltip is clicked. It is an ajax poll.
There can be many tooltips on the page, and more than one can need access to the data retrieved from the server.
What I want to achieve is to have this poll running as one instance - so if the user clicks a different tooltip the polling stops, rather than being duplicated.
Would be grateful if you could help.
Thanks
function doConversationsAjaxLongPoll(tablename){
clientSubmit = new Object;
// HERE WE'RE GOING TO GET A LIST OF THE ROWIDS THAT WE NEED TO POLL FOR, MAKE AN OBJECT OUT OF THEM. DO THIS BY LOOKING AT WHICH //TOOLIPS HAVE CLASS OPEN
var tooltips = [];
$('.tooltipOpen').each(function(index){
tooltips.push($(this).data('idrow'))
})
console.log("tooltips length: " + tooltips.length)
if(tooltips.length==0){
// console.log("tooltip length is 0 so we're returning false")
return false
}
clientSubmit.OpenConversations = tooltips
clientSubmit.tablename = tablename
clientSubmit.CurrentData = $('body').data('conversations')
console.log(clientSubmit)
$.ajax({
type: 'POST',
url: '/conversations.php?loadNew=1',
data: clientSubmit,
timeout: 25000,
success: function(data){
console.log('success')
data=JSON.parse(data)
console.log(data)
$('body').data('conversations', data)
},
complete: function(status, jqXHR){
if(tooltips.length==0){
// console.log("tooltip length is 0 so we're returning false")
return false
}
else
{
doConversationsAjaxLongPoll(tablename);
}
}
});
updateConversations()
}

I don't doubt that there are faaaar better ways of doing this but I have worked around the problem by having a random number generated by the click function, stored in $('body').data('random') which is then passed to the poll function. When the poll function loops it checks if the random number it was passed matches the one in data-random and returns false if it doesn't.