JavaScript JSON Combination and Data Anylish - javascript

I am trying to add values from multiple JSON responses that are saved in a .txt file. The .txt files has about 4000 entries. They are each the same format as follows:
{"id":"8f546dcf-b66a-4c53-b3d7-7290429483b8","price":"247.96000000","size":"0.03121005","product_id":"BTC-USD","side":"sell","stp":"dc"}
{"id":"0ec4b63a-b736-42af-a0aa-b4581bf12955","price":"247.90000000","size":"0.03910014","product_id":"BTC-USD","side":"sell","stp":"dc"}
{"id":"be403848-74dc-4494-8095-bd468777c958","price":"247.89000000","size":"0.04280854","product_id":"BTC-USD","side":"sell","stp":"dc"}
{"id":"ae2ae129-e850-4d8f-b945-55e65eb68a88","price":"247.83000000","size":"0.07941840","product_id":"BTC-USD","side":"sell","stp":"dc"}
{"id":"96194be4-40d8-446d-9f7e-ce72bc84af48","price":"247.63000000","size":"0.06225897","product_id":"BTC-USD","side":"sell","stp":"dc"}
I believe I need to combine the different JSON data sets before I can loop through them with a for loop and do the summing/analysis part (size:JSONSET1 + size:JSONSET2 + .....) but I'm not sure how I should call the .txt file in javascript and have it combine the multiple json parts. Suggestions??

Do you have any control over the file with the data set? If you do, you can make the input file one big JSON string.
Run this command in a terminal to add a comma to the end of every line:
sed -i 's/$/,/' yourFile.txt
Then edit the file with a text editor and put a [ at the beginning of the first line, and replace the last line's ending comma with a ].
Then after you read the file into a string, you can parse it like so:
var dataArray = JSON.parse(dataString);
And you could access the data like this:
console.log(dataArray[0].id);
This will print "8f546dcf-b66a-4c53-b3d7-7290429483b8" to the console

I would recommend using a .json file since you are working with JSON{Objects}.
I've made a demo of the file here and I've made a demo of the data analysis here.
Note: Demo file is hosted in a personal server, so it may not work later on.
Now the file isn't proper JSON Syntax, so it needs some parsing.
entries.json
{"id":"8f546dcf-b66a-4c53-b3d7-7290429483b8","price":"247.96000000","size":"0.03121005","product_id":"BTC-USD","side":"sell","stp":"dc"}
{"id":"0ec4b63a-b736-42af-a0aa-b4581bf12955","price":"247.90000000","size":"0.03910014","product_id":"BTC-USD","side":"sell","stp":"dc"}
(..)
JavaScript
Req = new XMLHttpRequest();
Req.onload = process_entries;
Req.open("get", "http://butler.tk/entries.json", true);
Req.send();
function process_entries() {
var response = Req.responseText;
var arrayed = "[" + response
.split("}")
.join("},")
.slice(0,-1)
+ "]";
var entries = JSON.parse(arrayed);
for (var i = 0, l = entries.length; i < l; i++) {
var entry = entries[i];
//entry.id,entry.size,etc.
}
}
We fetch the file with a XMLHttpRequest().
We parse the file so that it's valid JSON Syntax, the file will now look like this.
entries.json (parsed)
[
{"id":"8f546dcf-b66a-4c53-b3d7-7290429483b8","price":"247.96000000","size":"0.03121005","product_id":"BTC-USD","side":"sell","stp":"dc"},
{"id":"0ec4b63a-b736-42af-a0aa-b4581bf12955","price":"247.90000000","size":"0.03910014","product_id":"BTC-USD","side":"sell","stp":"dc"},
(..)
]
We parse the file into a JSON{Object}
We iterate through the array of objects and access their properties.
Note: If you have control over how the data is saved, you can save the data in the array format instead of parsing it.
Hope it helps! :)

Related

Parsing JSON from a file path is not working properly

I am trying to parse a list of JSON files iteratively. Using PHP I have managed to compile the list of JSON files in a directly into a a single JSON object. Now I would like to parse each of these objects and output a property from each of them in HTML. I am sure that the JSON I am initially passing works. Any ideas? here is the error and the function.
$(document).ready(function(){
console.log("something")
for(var i = 2; i < Object.keys(jsTrips).length; i++){
var data_file = "http://localhos:8080/trips/" + jsTrips[i];
var currTrip = JSON.parse(data_file)
document.getElementsByClassName(i).innerHTML = currJSON.start_time;
}
console.log("finished")
});
ignore the missing t in localhost. The problem persists even when the typo is fixed
Thanks in advance!!
UPDATE:
The Javascript object jsTrips is formatted like this:
{2: name.json, 3:name.json, 4:name.json}
The the JSON files named in jsTrips are formatted like this:
{start_time: some start time, coords: [{lat: ##, long: ##}, {lat: ##, long: ##}...], end_time: some end time}
To address the error you see:
SyntaxError: JSON.parse: Unexpected character at Line 1 Column 1
The JSON.parse() method parses a JSON string, constructing the JavaScript value or object described by the string. You are feeding it a URL when it is expecting [ or { as the first character. So h causes a Syntax Error.
Assuming you define the Object jsTrips someplace in your code, and this is a more basic object, I would suggest the following:
$(function(){
console.log("Start jsTrips");
var i = 0;
$.each(jsTrips, function(k, v){
if(i >= 2){
$.getJSON("http://localhos:8080/trips/" + v, function(data_file){
$("." + i).html(data_file.start_time);
});
}
i++;
}
console.log("Finished");
});
This code also assumes there are HTML Elements with attributes like class="2". It would be better to update your Post with an example of the Objects and an example of the JSON being returned.
Now, if the Index of the Object is the class name, then it might look more like:
$.getJSON("http://localhos:8080/trips/" + v, function(data_file){
$("." + k).html(data_file.start_time);
}
Again, need to know what you're sending and what you expect to get back.
jQuery.getJSON() Load JSON-encoded data from the server using a GET HTTP request.
See More: https://api.jquery.com/jquery.getjson/
Update
Now using JSONP method via $.getJSON(), this will help address CORS:
$(function() {
var jsTrips = {
2: "file-1.json",
3: "file-2.json",
4: "file-3.json"
};
console.log("Start jsTrips");
$.each(jsTrips, function(k, v) {
var url = "http://localhos:8080/trips/" + v;
console.log("GET " + url);
$.getJSON(url + "?callback=?", function(json) {
$("." + k).html(json.start_time);
});
});
console.log("Finished");
});
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
As you can see, this builds the new URL from your jsTrips as expected. You can get the start_time from the JSON directly. You don't need to parse it when JSON is expected.
In regards to the new CORS issue, this is more complicated. Basically, you're not calling the same URI so the browser is protecting itself from outside code.
Cross-Origin Resource Sharing (CORS) is a mechanism that uses additional HTTP headers to tell a browser to let a web application running at one origin (domain) have permission to access selected resources from a server at a different origin. A web application makes a cross-origin HTTP request when it requests a resource that has a different origin (domain, protocol, and port) than its own origin.
An example of a cross-origin request: The frontend JavaScript code for a web application served from http://domain-a.com uses XMLHttpRequest to make a request for http://api.domain-b.com/data.json.
For security reasons, browsers restrict cross-origin HTTP requests initiated from within scripts. For example, XMLHttpRequest and the Fetch API follow the same-origin policy. This means that a web application using those APIs can only request HTTP resources from the same origin the application was loaded from, unless the response from the other origin includes the right CORS headers.
See more: https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS & https://www.sitepoint.com/jsonp-examples/
If you are unable to change the port being used for the target JSON files, which I suspect is creating the CORS issue, you can consider using JSONP method versus GET method. Comment and let me know if this is the case, and I can update the answer. Example included in update.
Hope that helps.
This will not probably be a complete answer, because I don't really understand the question. But maybe it will help.
You told us you compiled in PHP one file from several JSON files, in general, if you have object in JSON file, it will look like { ...some key-values here... }, if you have array there, it will be [ ...some key-values here... ].
So when you compile several files with objects into one, you'll get {...some key-values here...} {...some key-values here...} and JSON does not know how to parse those, it will throw error:
console.log(JSON.parse('{"key": "value"}{"key": "value"}'))
This will work just fine, only one object there:
console.log(JSON.parse('{"key": "value"}'))
So, if for some reason you really need to compile several JSON files into one, there is a solution - to make such resulting file with new lines as separators. Than in JS you can split your file by new line, and parse each line without issues.
Like so:
const arrayOfJSONs = Array(10).fill(null).map((_,i) => JSON.stringify({key: i, keyXTen: i * 10}))
// then you join them in one big file with \\n new lines as separators
const oneBigFile = arrayOfJSONs.join("\n");
console.log("oneBigFile:\n", oneBigFile)
// on the client you get your one big file, and then parse each line like so
const parsedJSONs = oneBigFile.split("\n").map(JSON.parse)
console.log("parsedJSONs\n", parsedJSONs)
JSON.parse take string input
Javacript fetch
$(document).ready(function(){
console.log("something")
for(var i = 2; i < Object.keys(jsTrips).length; i++){
var data_file = "http://localhos:8080/trips/" + jsTrips[i];
fetch(data_file).then((res) => res.json())
.then((currJSON) => {
// document.getElementsByClassName(i).innerHTML = currJSON.start_time;
// update your DOM here
})
}
console.log("finished")
});
JQuery $.getJSON
$(document).ready(function(){
console.log("something")
for(var i = 2; i < Object.keys(jsTrips).length; i++){
var data_file = "http://localhos:8080/trips/" + jsTrips[i];
$.getJSON(data_file, (currJSON) => {
// document.getElementsByClassName(i).innerHTML = currJSON.start_time;
// Update DOM here
});
}
console.log("finished")
});

Need to grab the JSON here from an external file

This code does what I need but I want to store the JSON values it uses in a separate file using standard JSON format so the same data can be used elsewhere with getJSON requests.
(I need to run this to match up data returned from a getJSON request with additional values. I use the id to do the match on returned items. This is how I build the complete HTML items I need.)
CODE SAMPLE
$(function() {
var json = {
"nwsltrs":[
{
"id":"53c57dede4b07621dafde5d1",
"nwsltrNames":"Hello",
"author":"Joe"
} // these entries will grow to a hundred or more.
]
};
var titleID = "53c57dede4b07621dafde5d1"; // in actual code getting this value from a separate getJSON call
$.each(json.nwsltrs, function(i, v) {
if (v.id === titleID ){
nwsltrName = v.nwsltrNames;
author = v.author;
console.log(nwsltrName + ":" + author);
return;
}
});
});
Want the external JSON file to look like this with and use .json extension:
{
"nwsltrs":[
{
"id":"53c57dede4b07621dafde5d1",
"nwsltrNames":"Hello",
"author":"Joe",
},
{
"id":"54b57dede4b07621dafde5d2",
"nwsltrNames":"Bye",
"author":"Mary",
} // these entries will grow to a hundred or more.
]
I'm sure there are better ways to approach this and am open to hearing them—as long as I can use an external JSON file.
Solved.
Moved getJSON request that was tried earlier before posting.
It needed to run and return the data prior to this .each.
This allows me to replace the local JSON function and use a validated external JSON file here and elsewhere as desired.

Read CSV from server and put contents into drop down list in html or javascript

I have a csv file sitting on my server (using Apache 2) which I would like to "read" when a webpage loads. After reading the contents (two columns that are comma separated), I want to have one column be the display contents of a drop down list and the other column be the "data" contents. For instance, if I had a csv where the first row was [1,"me"], I would want the "me to be displayed to the user and the 1 to be the data that I could understand (selected value).
I know very little on javascripting and html (still learning), so any simple help would be much appreciated. To recap:
How do I read CSV file off my server (currently doing localhost)?
How do I parse this CSV file and add each row as an entry of a dropdown list?
Thanks!
How do I read CSV file off my server (currently doing localhost)?
First, you'll need to fetch the data from the CSV file so you can work with it as a native JavaScript object. Using jQuery's $.get() method, you can make an asynchronous request to the server and wait for the response. It would look something like this:
$.get(csv_path, function(data) {
var csv_string = data;
//Do stuff with the CSV data ...
});
How do I parse this CSV file and add each row as an entry of a dropdown list?
This can be accomplished with a for loop and some jQuery:
(function() {
var csv_path = "data/sample.csv",
var renderCSVDropdown = function(csv) {
var dropdown = $('select#my-dropdown');
for(var i = 0; i < csv.length; i++) {
var record = csv[i];
var entry = $('<option>').attr('value', record.someProperty);
dropdown.append(entry);
}
};
$.get(csv_path, function(data) {
var csv = CSVToArray(data);
renderCSVDropdown(csv);
});
}());
Note that this code calls a CSVToArray method, which you must also define in your code. An implementation can be found here, thanks to Ben Nadel: Javascript code to parse CSV data

how to export csv file with filename

I want to export the exist data into csv file. I try to use this code:
var uriContent = "data:text/csv;charset=utf-8," + encodeURIComponent(data);
var myWindow = window.open(uriContent);
myWindow.focus();
it works but I can design filename. I can only get the dialog with name like "MVeAnkW8.csv.part" which I don't know where the name come from.
How can I assign filename in the first dialog? Thanks in advance.
update:
I am now using rails. Actually I have a method in server side names export_to_csv. In end of this method, I use code like that:
send_data(csv_string,
:type => 'text/csv; charset=utf-8;',
:filename => "myfile.csv")
It works and I can specify the file name.
For now, I want to use ajax to get more csv files(that is the reason why I want to use javascript, because a normal http request can only get one file to be downloaded).
I use js code like that:
$.post("export_to_csv",
function(data) {
var uriContent = "data:text/csv;charset=utf-8," + encodeURIComponent(data);
var myWindow = window.open(uriContent);
myWindow.focus();});
It get the data from server side and I try to transfer it into csv. I can get a file but can't specify the file name.
As I know, you can specify the filename only in chrome 14+.
Take a look at this question: Is there any way to specify a suggested filename when using data: URI?
Update!
If you want to download multiple csv files "at once", you can zip them together, or save each file on the server separately (each file now has a url that points to it - if you place them inside the 'www' folder).
Then send the file names and their path/url to the client via ajax (use a json encoded list for example).
In the ajax callback function: take the list of the files and open each file-url in a separate popup.

JMeter modifying output to file from XML Stream

I'm attempting to write a JMeter script which after receiving and XML response from a server, extracts a string from it on the fly (drops the first part of the response) and writes it to a file.
Currently I use a Save Response Data to write to ChannelData_UAT_1 (filename). All good, it writes happily.
Then I add a BSF PreProcessor BEFORE it, and use javascript to try and extract the string. It's a bunch of XML tags, I want everything from "<Markets>" onwards.
I use:
function extract_markets(str)
{
marketIndex = str.indexOf("<Markets");
__log(marketIndex);
length = str.length;
marketString = str.substring(markeIndex, length-1);
return str;
}
vars.put('ChannelData_UAT_1', extract_markets(vars.get('ChannelData_UAT_1')));
As far as I can tell, ChannelData_UAT_1 is the variable the data is in. However this is only mentioned in the Save Response Data. But I can't do it afterwards otherwise it'll have already written to the file.
The current performance is for it to receive the response and write to the file. No filtering is done - as if my javascript didn't exist.
Anything small or obvious that I've missed? Suggestions?
I believe the issue stems from the fact that ChannelData_UAT_1 is not a variable and how Save Response Data works.
ChannelData_UAT_1 is the file name, not the content of the file.
You need to modify the contents of the "Response". You can replace the value of the page response with the value of your function.
I think the code would look something like this:
import org.apache.jmeter.util.JMeterUtils;
import org.apache.jmeter.samplers.SampleResult;
prev.setResponseData(extract_markets(vars.get('ChannelData_UAT_1')));
Source:
http://www.javadocexamples.com/java_examples/org/apache/jmeter/samplers/SampleResult/

Categories

Resources