Develop Reference

What is the best way to manipulate an API AJAX JSON response and pass it to another page?

I think I have a tough one for you guys. Or at least it's been tough for me. I've been searching for the best way to do this in Stack Overflow and everyone that has asked has been given a different response.
I have this code that is accessing an API and calling a maintenance list of all the vehicles in a fleet.
function getMaintenanceList() {
var settings = {
"url": "API URL HERE",
"method": "GET",
"timeout": 0,
"headers": {
"Authorization": "Bearer token here"
},
};
$.ajax(settings).done(function (response) {
// The response the API sends is a JSON object.
// It is an array.
var jsonMaintenance = response;
var parsedJson = JSON.stringify(jsonMaintenance);
//Left over code from when I was trying to
//pass the data directly into the other page
// I was unable to do so
//return jsonMaintenance;
//Left over code from when this was in a PHP file
//and I was posting the stringified response to the page
// for testing purpose
//I had to disable CORS in Google Chrome to test the response out
//console.log(jsonMaintenance);
//document.getElementById("main").innerHTML = parsedJson;
});
};
The code above works well. What I was attempting to do here was write the stringified response to a file. Save that file in the server. Call it from another page using JavaScript and save it as an object in JavaScript, parse it using JSON.parse(), and then pull the required information.
Here's an explanation as to why I'm trying to do it this way. When I call the maintenance list from the API, I'm getting the entire maintenance list from the API, but I need to be able to display only parts of the information from the list.
On one page, we'll call it vehicle-list.php, on it I have a list of all the vehicles in our fleet. They all have unit numbers assigned to them. When I click on a unit number on this page it'll take me to another page which has more information on the vehicle such as the VIN number, license plate, etc. we'll call this page vehicle-info.php. We're using this page for all the vehicles' information, in other words, when we click on different unit numbers on vehicle-list.php it'll always take us to vehicle-info.php. We're only updating the DOM when we go to the page.
I only want to include the information specific to each vehicle unit in the page along with the other info in the DOM. And I only want to call the info from the API once as I am limited to a certain amount of calls for that API. This is why I am attempting to do it this way.
I will say that what I originally wanted to do was get this JSON response once every 24 hours by using a function in vehicle-list.php save the reponse as a variable as seen above var jsonMaintenance = response; and then just access certain parts of the array every time a unit number is clicked. However, I have been unable to access the variable in any other page. I've written many test files attempting to call jsonMaintenance without success so I've been trying to to just save it as a text file to the server and I haven't been able to figure that out either.
After explaining all of the above. My questions are these:
How do I best manipulate this data to accomplish what I want to accomplish? What would be the best standard? Is the above code even the right way to call the data for what I'm trying to do?
There doesn't seem to be a set standard on accomplishing any of this when I search on Stack Overflow. I'd like to be as efficient as possible.
Thank you for your time.

there is a lot of ways how you pass your data through your website after getting it in from an api call, the best approach is to store these information in a database and call it back in which ever way you want, you can do that as far as you are using php, you can store it to sql or to access, if you don't want to store these information in a database like in sql or access, then best way is to store it to localStorage and call it back whenever you want.
I will show you briefly how you can do that, if you want better explanation post an example of your returned data.
to store an item in localstorage use,
localStorage.setItem('key', 'value');
to call an item back from localstorage use,
var somevar = localStorage.getItem('key')
to remove specific item from localstorage use,
localStorage.removeItem('key')
to clear all items saved to localstorage use,
localStorage.clear()
be aware storing the data to localStorage is only at the station you are using

I would do it somehow like this.
Call the maintenance list from the API with the server side language of your choice which seems to be PHP in your case. Lets say the script is called: get-list.php. This can be triggered by a cron job running get-list.php in intervals limited to the certain amount of calls that you are allowed to do for that API. Or if you are not able to create cron jobs then trigger the same get-list.php with an AJAX-call (eg jQuery.get('sld.tld/get-list.php') - in this case get-list.php have to figure out if its the right time to call the API or not).
Now that you have the data you can prepare it as you want and store it as a JSON-string in a text file or database of your choice. If I get you right you have a specific dataset for each vehicle, which have to be identified by an id (you named it "unit number") so your JSON would look kind of: {"unit1": { property1: "val1", property2: "val2" }, "unit2": { property1: "valXYZ", property2: "valABC" }} or alike.
Now when you link to vehicle-info.php from vehicle-list.php, you do it like so: ancor or similar as well. Of course you can also grab the data with AJAX, its just important to deliver vehicle-info.php the corresponding unit number (or id - better to say) and you are good to go.
vehicle-info.php now have all there is to render the page, which is the complete data set stored in text file or data base and the id (unit number) to know which part of the whole dataset to extract.
I wanted to give you this different approach because in my experience this should work out just so much better. If you are working server side (eg PHP) you have write permissions which is not the case for JavaScript-client side. And also performance is not so much of an issue. For instance its not an issue if you have heavy manipulating on the data set at the get-list.php-level. It can run for minutes and once its done it stores the ready-to-use-data making it staticly available without any further impact on performance.
Hope it helps!

If i ran into a similiar problem i would just store the data in a database of my own and call it from there, considering you are only (willing/abe/allowed) to request the data from the API very rarely but need to operate on the data quite frequently (whenever someone clicks on a specific vehice on your applicaiton) this seems like the best course of action.
So rather than querying the data on client side, I'd call it from server, store it on server and and have the client operate on that data.

Old data using YQL and Javascript getJSON call

After doing some searching, it seems that this type of issue affects many users out there so an answer to this question could help many users of Yahoo's YQL Platform.
I am essentially aiming to extract a semi-static CVS document stored on a webserver to then parse in Javascript. Semi-static means that the CVS document isn't getting appended to with additional entries, rather each entry is getting modified.
Using the YQL console https://developer.yahoo.com/yql/console/ gives me updated data with every call made to my semi-static CVS file stored on a remote server. I can modify the data and YQL console will successfully return the updated data. When I extract their provided rest query and simply 'paste' it into a browser window, the data provided correspond to the very first query that I made. When I embed the query as a $.getJSON request in Javascript as such:
$.getJSON("https://query.yahooapis.com/v1/public/yqlq=select%20*%20from%20csv%20where%20url%3D'mywebsite.csv'&format=json").done(function (data) {
I still get the outdated data. If I switch to a different web-browser or device, the information is still outdated which gives me the feeling that it is not a cache issue on the local machine.
I believe the problem is in one of two spots:
1) Perhaps Yahoo caches the queries and only acquires updated information from tables/files that grow dynamically
2) I am not using the YQL query correctly.
As an additional note, the exact same query structure works perfectly with Google Forms (which can export as a CVS) and also works without a hitch on a dynamically growing CVS document that I used on a now antiqued database, requiring a quick switch to the simple semi-static document.
Any thoughts or fixes that can work on my semi-static CVS document?

This sounds like a browser cache issue
Try adding a timestamp in the url which makes every request have a unique url and therefore browser won't have a cache for it
var params ={
q : "select%20*%20from%20csv%20where%20url%3D'mywebsite.csv'",
format: "json",
_v : Date.now()
};
$.getJSON("https://query.yahooapis.com/v1/public/yql", params ).done...

Serve files from Apache to Javascript

I am writing my first web application with Javascript and WebGL. For now I am running the app on localhost from Apache. The app needs to work with data that is provided instantly. Until now I worked with AJAX calls that happen during runtime which doesn't work out for my purposes anymore. So instead of serving individual files from Server to Client when asked, I want the application to load all files from the Server to Client side at initialization time (I want this to happen automatically at the start so I don't have to add every new file as a url in the html index). I understand I should do this with Server Side scripting; probably with PHP since I have a Apache localhost? I have different folders which hold my necessary resources in a uniform dataformat (.txt, .png and .json). So what I want to do is, before the Javascript app starts, look through the folder and send one object per folder that holds filenames as keys bound to filedata. Is my intuition right that I need to do that with PHP? If yes, where do I start to tell my application what to do when (first start serving files with php, then start the javascript app)? How do I do this on localhost? Should I already think about extending my toolset (e.g. using nodeJS on ServerSide(locally for now))? If so what lightweight tools do you propose for this kind of work? I feel I am missing some design principles here.
EDIT:
Keep in mind that I don't want to specifically call a single file... I am already doing that. What I need is a script that automatically serves all the files of a certain folder on the server to the client side at init time of the app before the program logic of the actual application starts.

Your question is kind of broad so I'll try my best. Why does AJAX not work for real-time data but loading all the files once does? If you're working with real time data, why not look into a websocket or at the bare minimum, AJAX queries?
If you want to pass data from the server to the client, you will need to use a HTTP request no matter what. A GET request or POST request is necessary for the client to request data from the server and receive it as a response.
You could theoretically just pass the data from PHP straight to the view of the application (which is technically done through a GET request whenever a user requests data such as .php files from the server) but this isn't as flexible as if Javascript had access to the data. You can do some hacks and 'transfer' the data from the view to Javascript with some .value methods, but this isn't ideal and can be prone to some security holes. This also means data is only being passed once.
So what would need to happen is that the data would need to be processed upon initialization and then immediately transferred to the client by use of Javascript and HTTP requests.
So if you want Javascript to have access to the data and use it in variables or manipulate it further, then you'd need to use an HTTP request such as GET or POST which is called by Javascript. Otherwise, you need to immediately pass the data to the view upon initialization (through PHP), but this means you can't work with real-time data because the data is only being passed once when there is a page refresh.
Example of scandir():
<?php
//scandir() returns filenames, not data from files
$fileArray = scandir('datafolder/') //this is a relative path reference to the folder 'datafolder'
$finalArray = [];
foreach($fileArray as $filename){
tempArray = [];
$file = fopen('datafolder/' . $filename, 'r'); //im pretty sure scandir only retrieves the filenames and not the path, so you might need to append the filepath so your script knows where to look
$tempArray = fgetcsv($file, 1024); //temp array to hold contents of each iteration of foreach loop
array_push($finalArray, $tempArray); //this will store the data for later use
}
Or the data can be used however, depending on what it is. Say, if you need to combine the data from multiple .csv files, you can read each file and append it to a single array. If you want to read multiple distinct files and preserve the independence of each file, you can create multiple arrays and then pass back a single JSON encoded object that contains each file's data as a separate attribute of the object such as:
{
'dataOne': [0,1,2,3,4,5,6...],
'dataTwo': ['new', 'burger', 'milkshake'],
'dataThree': ['Mary', 'Joe', 'Pam', 'Eric']
}
Which can be created with a PHP associative array using one of the following methods:
//assuming $arrayOne is already assigned from reading a file and storing its contents within $arrayOne
$data['dataOne'] = $arrayOne;
// or
array_push($data['dataTwo'], $arrayTwo);
// or
array_push($data, [
'dataThree' => ['Mary', 'Joe', 'Pam', 'Eric']
]);
Then $data can simply be passed back which is a single array containing all the different sets of data, if each set needs to be distinct.

Access a cookie that was set on a javascript file, but not on the HTML

I have a script that needs some external information to work with. It fetches this using Ajax requests. So far so good.
However, the script needs some of it's data right from the start. So I have been pondering a few options to supply it with that initial data at page load time:
Simplest: Just have it perform an Ajax request for the data right away. Downside of this is extra latency and more requests than strictly needed.
Ugly: Add a small script fragment at HTML render time that provides the initial data
Bad caching properties: Create the whole JS file dynamically and add the data right then.
Impossible: Something with headers... but unfortunately it seems we can't access them (see e.g. this question). Doing the extra Ajax request is not useful here as in that case we might just as well use option #1.
Something with cookies...
Not tried yet: Create a dynamic 'initial-data.js' script whose sole purpose it is to load the initial data. This would at least only send the data when needed, but it would require all users of my script to include 2 script files instead of one.... Also it will cause an extra request...
I am trying out the 4th option of using cookies to transport the initial data but so far not having any success. What I am trying to do:
When the browser requests the .js file, have the server add a Set-Cookie header with the initial data in it in the response.
In the JS file, read out the cookie.
It doesn't work. It seems I need to set the cookie on the response for the .html instead of the .js for the browser to make it available to the script... That's too bad as it would involve adding the Set-Cookie header to each page, even though it's only needed by that particular piece of JS.
I was actually very happy with the solution I thought I found because it let me send the initial data along with the request for the script only to those pages that actually use the script... Too bad!
Is there any way to do what I'm trying to do using cookies, headers or some similar mechanism?
Do you guys have any tips for this situation?
Background:
I am trying to write a semi-offline application. Semi-offline in that it should continue to work (apart from some functions that just need connectivity) when offline, but is expected to have periods with connectivity regularly. So I'm using local storage and synching with the server when possible.
To be able to have the client generate new items when offline, I am including an ID generator that gets handed out ID blocks by the server, consuming them as it generates ID's. The data I was trying to send to the script in a cookie is the initial list of ID blocks and some settings and looks like this:
/suid/suid.json:3:3:dxb,dyb,dzb
^ ^ ^ ^
url min max blocks
Where:
url = path to JSON for subsequent Ajax requests
min = minimum amount of ID blocks to keep in local storage
max = maximum amount of ID blocks to keep in local storage
blocks = comma separated list of ID blocks
The ID blocks are encoded as sort-of Base32 strings. I'm using a custom formatting schema because I want 53-bit ID's to be as short as possible in text format while still being easily human readable and write-able and URL-safe.

Force existing client web pages to reload - using only JSON (no eval)

I'm a consultant working on a web app that's basically a single page app. All it does is constantly retrieve new json data behind the scenes (like once a minute), and then display it on screen.
Our clients load this app, and leave it running 24/7, for weeks on end. If errors happen when retrieving new json data, the app ignores it and keeps running.
We're rolling out an update, and want the existing clients to either become invalidated, or reload themselves without any user interaction. This feature wasn't "built in" by anyone, and we're trying to do this after the fact.
Is there some way to make the existing clients reload without telling our end users to just reload the page?
The following conditions define the app a bit more:
The app uses jQuery 1.9.0
Runs exclusively in Chrome
Retrieves new json data frequently using jquery
Throws away any errors it finds in json responses and uses old data.
EDIT:
I've had it suggested that we could try the following:
send invalid data through the JSON responses to crash chrome (like 500 megs of data, for example)
send window.location.reload through the JSON response (which supposedly won't work due jquery protecting against this type of thing)
send "script" data in the JSON response and if it gets $.html(....) at some point, then it may run the script as well.
and am open to any suggestions on getting this to reload or kill chrome, so the client is forced to reload the page.

If you're using $.ajax to request your data, and not specifically setting your content type, then you may be able to do the following on the server:
set the content type header to "text/javascript"
respond with javascript, e.g. window.location = "http://www.yoursite.com"
jQuery may eval that, and simply run your javascript.

No it is not possible. As far as I can tell you do not execute code from the JSON response (which is a very good thing). Thus you have no way of altering your current client's behaviour. According to your own statement:
"Throws away any errors it finds in JSON responses and uses old data"
You will not be able to crash the user's browser by sending invalid JSON data as the errors will be suppressed.
You can build in automatic deployment in to future versions by sending an application version number and testing for changes or by using WebSockets (which the application seems better suited to anyway as you can ensure your clients only poll the server when the JSON has actually changed).

If I get it correctly, create a version referance page, and make the client check this page very couple seconds, when you update the file, client will reload itself with this script.
var buildNo = "1.2.0.1";//
var cV = setInterval(checkVersion,(5*1000))//Every 5 sec.
function checkVersion(){
$.ajax({
url:"checkVersion.php?v="+buildNo,
dataType:"JSON",
success:function(d){
if(d.version != buildNo){//if version is different
window.location.reload();
//chrome.runtime.reload(); //for chrome extensions
}
}
})
}
if you cant add extra page, you may just add extra variable to end of your JSON data.