Develop Reference

Load webpage in NodeJs

I'm just wondering how I can load a webpage within nodejs, I've been searching for 2 days and I can't find anything. Either I'm using the wrong search terms or I'm just not looking for what I actually need.
I need to open a link to authenticate an account and I've retrieved the URL that I need to open but I'm not sure how. I've tried http.request but it mustn't be loading the page as when I check if the account has been verified it hasn't.
How would I go about this?
Thanks (Sorry for nooby question and bad formatting)
Buck
P.S Oops I wrote it in such a hurry I forgot to add the snippet of my code
var http = require('http');
var options = {
host: Host, //Both variables defined earlier in the code this is just a snippet of the http.request part
path: Path
};
callback = function(response) {
var string = '';
response.on('data', function (blob) {
string += blob;
});
response.on('end', function () {
console.log(string);
});
}
http.request(options, callback).end();
It just returns
['R޾�\s۝�V������T��:�I����$��v�* �*�;�* P���q�ܠ���5�E!9��I���v��r��� �CmO����q��<�>���&�趩�C��i�&��a��q�(��1a4I^XvLe�T˔�|��M�3�EA!نY\0�h�R��#r�b�a��Yr��z��1аB
Even when I try
console.log(string.toString('utf8'));

I managed to find what the problem was and fix it,
Further above in my code I had not declared an important variable properly.
http.request works perfectly

OpenLayers 3 Reload Layer(s)

I am working on a project using OL3 in which I need to be able to manually (by button press) or automatically (time based) reload vector layers IF they have been updated since the last load using HTTP conditional GETs (304 headers and such).
I found this very old post (https://gis.stackexchange.com/questions/333/how-to-dynamically-refresh-reload-a-kml-layer-in-openlayers) for KML layers, but it appears to use variables no longer found in OL3 and I am not sure that it would allow for only loading files that have been modified since the last load. At first glance it appears that a full reload is forced, even if the file has not been modified.
There does not seem to be anything in the API that resembles a reload function for the map or layer objects in OL3. Is there a way to do this?
Update 1:
I found a possible way to do this as an answer in this question: https://gis.stackexchange.com/questions/125074/openlayers3-how-to-reload-a-layer-from-geoserver-when-underlying-data-change using the code:
layer.getSource().updateParams({"time": Date.now()});
however when I run this code I get the error:
TypeError: selectedLayer.getSource(...).updateParams is not a function
Upon checking the API Reference for OL3 it appears that no such functions exist. The closest is setProperties() or setAttributions(). Neither of which work. It also appears that not all layer types implement getSource().
Update 2:
The refresh() reloads the tiles, but does not appear to be requesting them from the server. Rather, it appears they are being loaded from a cache (but not the HTTP cache). No requests are made, no HTTP 304s or anything like that. Will be trying a variant of the KML approach and posting the results soon.
Update 3:
After trying LOTS of different solutions I accidentally stumbled upon something that worked for vector layers. By calling the layer source.clear() function and then calling Map.updateSize(), the layer is automagically reloaded from it's source URL. An XHR GET request is issued and if the source file has changed, it will be reloaded from the file. If the source file has not changed, a 304 will be issued and the source will be reloaded from the cache.
Below is a function that should uses this method to reload a given layer:
function refreshLayer(selectedLayer)
{
var selectedLayerSource = selectedLayer.getSource();
if(selectedLayerSource instanceof ol.source.Vector)
{
//do vector reload
selectedLayerSource.clear();
map.updateSize();
}
else
{
//reload the entire page
window.location.reload();
}
}
However, it appears that on the first few tries (depending on the browser) the request is sent, a 200 code is sent back, but the layer does not reflect any changes. After a few tries (and reloading the page a few times) it works. Once it starts working for a layer it continues to work as often as the source file is changed. Does anyone have any idea what is going on?
Update 4:
Using an adaptation of Jonatas' answer I am getting better results. New features pop up instantly on a reload. However, old features are not removed from the map and many features that have moved locations are shown on the map twice. Below is my code:
function refreshSelectedLayer()
{
console.log("This feature is still in the process of being implemented. Refresh may not actually occur.");
var selectedLayerSource = selectedLayer.getSource();
if(selectedLayerSource instanceof ol.source.Vector)
{
var now = Date.now();
var format = selectedLayerSource.getFormat();
var url = selectedLayerSource.getUrl();
url = url + '?t=' + now;
loader = ol.featureloader.xhr(url, format);
selectedLayerSource.clear();
loader.call(selectedLayerSource, [], 1, 'EPSG:3857');
map.updateSize();
}
else if(selectedLayerSource instanceof ol.source.Tile)
{
selectedLayerSource.changed();
selectedLayerSource.refresh();
}
}
Note that the var selectedLayer is set elsewhere in the code. Any ideas why these very odd results are occuring?
Update 5:
I noticed that if I remove all other code besides the:
source.clear();
call an XHR GET request is made and the features do not disappear. Why is clearing the source not removing all of the features?
Update 6:
After discovering that ol.source.clear() was not actually removing features from a given data source/layer I replaced it using the following code:
selectedLayerSource.forEachFeature(function(feature){
selectedLayerSource.removeFeature(feature);
});
By outputting the features in the layer before and after each step, I got this:
var now = Date.now();
var format = selectedLayerSource.getFormat();
var url = selectedLayerSource.getUrl();
url = url + '?t=' + now;
console.log("time: "+now+" format: "+format+" url: "+url);
loader = ol.featureloader.xhr(url, format);
console.log(selectedLayerSource.getFeatures());
console.log("Deleting features...");
/*
Try adding code here to manually remove all features from source
*/
selectedLayerSource.forEachFeature(function(feature){
selectedLayerSource.removeFeature(feature);
});
console.log(selectedLayerSource.getFeatures());
console.log("Loading features from file...");
loader.call(selectedLayerSource, [], 1, 'EPSG:3857');
window.setTimeout(function(){
console.log(selectedLayerSource.getFeatures());
map.updateSize();
}, 500);
Which outputs into the console:
"time: 1471462410554 format: [object Object] url: http://server/file.ext?t=1471462410554" file.php:484:3
Array [ Object, Object, Object, Object, Object, Object, Object, Object, Object, Object, 1 more… ] file.php:491:3
Deleting features... file.php:492:3
Array [ ] file.php:501:3
Loading features from file... file.php:503:3
GET XHR http://server/file.ext [HTTP/1.1 200 OK 34ms]
Array [ Object, Object, Object, Object, Object, Object, Object, Object, Object, Object, 1 more… ]
After several tests with GeoJSON and KML layers I confirmed that this method works!!!
However, because the loader makes its request asynchronously, I am left with the problem of how to execute code after the loader function has been called. Obviously using setTimeout() is a horrible way to do this and was only implemented for testing purposes. A success/failure callback function would be perfect and when looking at the source of featureloader.js it appears that they are offered as parameters in ol.featureloader.loadFeaturesXhr. See below code block from featureloader.js:
/**
* #param {string|ol.FeatureUrlFunction} url Feature URL service.
* #param {ol.format.Feature} format Feature format.
* #param {function(this:ol.VectorTile, Array.<ol.Feature>, ol.proj.Projection)|function(this:ol.source.Vector, Array.<ol.Feature>)} success
* Function called with the loaded features and optionally with the data
* projection. Called with the vector tile or source as `this`.
* #param {function(this:ol.VectorTile)|function(this:ol.source.Vector)} failure
* Function called when loading failed. Called with the vector tile or
* source as `this`.
* #return {ol.FeatureLoader} The feature loader.
*/
ol.featureloader.loadFeaturesXhr = function(url, format, success, failure)
I attempted to implement these functions like so when creating the loader:
loader = ol.featureloader.xhr(url, format,
function(){
console.log(selectedLayerSource.getFeatures());
map.updateSize();
console.log("Successful load!");
},
function(){
console.log("Could not load "+selectedLayerName+" layer data from "+url);
}
);
but neither function is being called. Any suggestions? I feel like I am missing something really simple here...
Update 7:
Using the solution provided by #Jonatas Walker I adapted it to use jQuery:
var now = Date.now();
var format = selectedLayerSource.getFormat();
var url = selectedLayerSource.getUrl();
url = url + '?t=' + now;
//make AJAX request to source url
$.ajax({url: url, success: function(result){
//manually remove features from the source
selectedLayerSource.forEachFeature(function(feature){
selectedLayerSource.removeFeature(feature);
});
//create features from AJAX results
var features = format.readFeatures(result, {
featureProjection: 'EPSG:3857'
});
//add features to the source
selectedLayerSource.addFeatures(features);
},
error: function(err){
alert("Could not load features from "+selectedLayerName+" at "+url+" error code: "+err.status);
}
});
After extensive testing with GeoJSON and KML sources this has proved an extremely reliable refresh method!

Well, there are another options! Have your own loader.
Load this script - just in case someone is still using old browsers
<script src="//cdn.polyfill.io/v2/polyfill.min.js?features=fetch"></script>
Then load your JSON file and know when it's ready/loaded:
function refreshSelectedLayer(layer) {
var now = Date.now();
var source = layer.getSource();
var format = new ol.format.GeoJSON();
var url = '//your_server.net/tmp/points.json?t=' + now;
fetch(url)
.then(function(response) {
return response.json();
}).then(function(json) {
console.log('parsed json', json);
source.clear(); // if this is not enough try yours
var features = format.readFeatures(json, {
featureProjection: 'EPSG:3857'
});
source.addFeatures(features);
}).catch(function(ex) {
console.log('parsing failed', ex);
});
}

Try an adaptation of this:
function refreshSource() {
var now = Date.now();
var source = vectorLayer.getSource();
var format = new ol.format.GeoJSON();
var url = '//your_server.net/tmp/points.json?t=' + now;
var loader = ol.featureloader.xhr(url, format);
source.clear();
loader.call(source, [], 1, 'EPSG:3857');
}
The trick is to tell the browser this is a new loading by changing the url.

How can I set the Accept Header for a link on a web page?

This is mostly an ivory tower question, since I can easily just make a new URL endpoint. But basically, I'd like to be able to serve up CSV when the user has the Accept header configured to include text/csv. That's trivial on the server side, but on the client side I don't know how to set the Accept header unless I'm using XHR or some other "non-browser" client. Is there a way in HTML to set the Accept header in a link or in JS to set the Accept header when using window.location?

I figure I might as well put this here for the next thousand people looking at the post. You cannot do it.

Update:
I will leave my original answer below for posterity, but I now see that I didn't really answer the question. There isn't a way to do this "natively", the best approach I can think of would be to use a data uri (http://en.wikipedia.org/wiki/Data_URI_scheme) and have AJAX do the work for you:
// aware that you didn't mention jQuery, but you can do this with or without
var download = function(){
var mime = this.getAttribute('data-mime-type');
$.ajax({
accepts: mime,
success: function(data){
var uri = 'data:'+mime+';charset=UTF-8,' + encodeURIComponent(data);
window.location = uri;
}
})
return false;
}
With the same idea used in the example below:
Download CSV
document.querySelectorAll('a[data-mime-type]').onclick = download;
Original Answer
There is no built-in way to force an 'Accept' header on a link (via HTML or Javascript). I think you could pull this off fairly easily using a very small amount of server & client-side code though. Should be easy in any language, my example is PHP:
function get_accepted_headers() {
$headers = apache_request_headers();
if(array_key_exists('Accept', $headers)) {
$accepted = explode(',', $headers['Accept']);
return $accepted;
}
return array();
}
Add a data-accept attribute to your download links:
Download CSV
Then attach a click event handler to ensure that the user accepts the specified content type:
// will end up with something like: ["text/html", "application/xhtml+xml", "application/xml;q=0.9", "image/webp", "*/*;q=0.8"]
var user_headers = <?=stripslashes(json_encode(get_accepted_headers()))?>;
var click_header_check = function() {
var accept = this.getAttribute('data-accept');
if(user_headers.indexOf(accept) == -1) {
console.log('User does not explicitly accept content-type: %s', accept);
return false;
}
window.location = this.href;
return;
}
// attach the event listener
document.querySelector('a[data-accept]').onclick = click_header_check;
Not sure if this is what you were looking for, but hope that it helps.

For those still interested, there is a way to do this in pure javascript.
The following code uses JQuery (https://jquery.com/) and FileSaver.js (http://purl.eligrey.com/github/FileSaver.js/blob/master/FileSaver.js) though you could write the respective parts yourself:
//in case of non binary data use:
var type = 'text/xml';
var url = 'http://your_url_here/'
$.ajax({accepts:{text:type},
url:url,
processData:false,
dataType:'text',
success:function(data){
saveAs(new Blob([data], {type: type}),'filename.txt');
},
error: function(){
// Handle errors here
}
});

How can you read a file line by line in JavaScript?

I'm writing a web-app for the iPad that will be loading data from a text file. (A sample data set is around ~400 kb). I have everything set up except the file reading. The way I have set up my code, you pass an object which reads a file line by line.
How can I read a file line by line?
If there is no direct way to read a file line by line, can someone please show me an example of how to read a file into a string object? (so that I can use the split method :P)

This could work, if I understood what you want to do:
var txtFile = new XMLHttpRequest();
txtFile.open("GET", "http://website.com/file.txt", true);
txtFile.onreadystatechange = function()
{
if (txtFile.readyState === 4) { // document is ready to parse.
if (txtFile.status === 200) { // file is found
allText = txtFile.responseText;
lines = txtFile.responseText.split("\n");
}
}
}
txtFile.send(null);

Mobile Safari doesn't have the File API, so I assume you're talking about reading from a web resource. You can't do that. When you read a resource via ajax, the browser will first read it fully into memory and then pass the entire string to your ajax callback as a string.
In your callback, you can take the string and break it into lines, and wrap that up in an object that has the API that your code wants, but you're still going to have the string in memory all at once..

With jQuery:
myObject = {}; //myObject[numberline] = "textEachLine";
$.get('path/myFile.txt', function(myContentFile) {
var lines = myContentFile.split("\r\n");
for(var i in lines){
//here your code
//each line is "lines[i]"
//save in object "myObject":
myObject[i] = lines[i]
//print in console
console.log("line " + i + " :" + lines[i]);
}
}, 'text');

i dont think thats possible until you use ajax to hit some server side code.

Json problem in fetching data

<script>
function jsonfunc(){
var data ="publick="+document.getElementById("publickeyval").value+"&privatek="+document.getElementById("privatekeyval").value;
var url="http://www.remoteapiserver.com/example_api/example_adcpatchaapi.php?"+data;
alert(url);
var my_JSON_object = {};
var http_request = new XMLHttpRequest();
http_request.open( "GET", url, true );
http_request.onreadystatechange = function () {
if (http_request.readyState == 4){
alert(http_request.responseText+"#"); // showing only # //
my_JSON_object = JSON.parse( http_request.responseText );
}
};
http_request.send(null);
}
</script>
I was as asked my question as per comment i read Json and writing above code in my php page.But still this is giving problem.I am not getting fetched dada from remote server.I am getting only "#" in alert box.

I highly recommend a JavaScript Framework like jQuery for this kind of stuff. It definitely makes this a lot easier. jQuery lets you do cross-domain requests, if you use JSONP (take a look at the docs). The code would look something like this:
$.getJSON(yourUrlGoesHere, function(data) { // ready callback
// data is an object; no need to parse it manually.
});

Sometimes it't better use some library:
JQuery or Moootools: http://mootools.net/docs/core/Request/Request.JSON
Implement this in native JS is difficulty if we want use it in all browsers ;)