I have a json file containing 200000 point coordinates in geojson format and I want to generate a heat map of these coordinates. I have two ideas to do that but I have problems with both ideas:
use google map-->heatmap layer. I plan to write a html file as shown on
https://developers.google.com/maps/documentation/javascript/examples/layer-heatmap
, however, I dont know how to put the coordinates of json file into the batch of google.maps.LatLng(), any idea?
use google Fusion Table. However, it seems that a delimited text file(.csv, .tsv, or .txt), and Keyhole Markup Language file(.kml) are necessary, is that right?
I don't know whether there are some other good ways to generate a heat map. Any good idea?
I dont think that geoJson is a good choice to transport such an amount of data(I wouldn't even use it for 2000 items).
Let's take a look at a single LatLng, (lets assume 5 decimals):
to simply transport the needed data via JSON you would need e.g.:
//17 bytes, about 3.5 MB for 200000 points
[9.12345,5.43219]
in geoJson:
//95 bytes, about 18 MB for 200000 items
{"type":"Feature","geometry":{"type": "Point","coordinates":[5.43219,9.12345]},"properties":{}}
I guess I don't have to say anything....
The basic issue with #1: (I don't think it's recommendable for 200000 points, no matter which format you choose):
As you can't transport a google.maps.LatLng via JSON, you'll need to pre-process the JSON to create an array with LatLngs(will take some time for 200000 points)
I think #2 is the only option you should think about.
You may set up a script, macro, etc. which parses the geoJson when it's static and upload it to a FusionTable.
When the data are not static you may use a serverside script which uploads/updates the data automatically.
KML isn't required for points, you may simply use a CSV(lat and lng may be stored in a single field, delimited by a comma) or in separate fields.
There are two reasons why you should go for option#1.
a) Serializing the GeoJSON data points to GoogleMap-compatible format latlng is relatively easier than option #2. You can even load GeoJSON data, parse it to GoogleMap coordinate, and draw on the map without any need to convert the file type.
b) If you choose option 2, you definitely need to write a script which converts GeoJSON file to a comma-delimited (or whatever symbol) text file. When you have a new GeoJSON data, you will need to convert it again. This doesn't save your time.
So let's say you choose option #1
You can just add a script tag which refers to your GeoJSON data file like this:
<script type="text/javascript" src="data.json"></script>
Then parse it into a JSON format with your JavaScript code like this:
var geoData = JSON.parse(data);
Then you'll need to parse these geoJSON data into an array of lat-lng coordinates:
var coords = geoData.map(function mapToLatLng(c){
return new google.maps.LatLng(c.geometry.coordinates[0], c.geometry.coordinates[1])
});
With this code above, you should get coords which carries all Google Map latitude-longitude coordinates ready for rendering into a heatmap.
To render those points, you may try:
heatmap = new google.maps.visualization.HeatmapLayer({
data: new google.maps.MVCArray(coords)
});
heatmap.setMap(map);
I'm trying to render Antarctica geojson shape on a map using the HERE maps api.
The geojson is found here: https://github.com/johan/world.geo.json/blob/master/countries/ATA.geo.json
You can see github renders it nicely.
Using the same geojson on geojson.io also renders it nicely.
But somehow it seems to render the 'inverse' of Antarctica when using it in HERE maps.
It colors everything except antarctica.
see: http://imagebin.ca/v/1dZIn5vsEuFx
(I've tried making an expample using jsfiddle, but it's not able to load external json. And the HERE maps api doesn't allow you to load geoJSON from a string)
Is there an issue with the geoJSON? Is there an issue with the HERE maps api?
The API doesn't quite understand what to do with the open polygon. Because the polygon is basically just a line around the globe the API doesn't know if you shape closes over the north pole or the south pole. By default it assumes that open polygons close over the north pole. You can change this by using this flag (setNorthPoleCovering):
http://developer.here.com/javascript-apis/documentation/v3/maps/topics_api_nlp/h-map-polygon.html#h-map-polygon__setnorthpolecovering
However, actually getting to that point in the code where this can be done is a bit complicated:
// When you instantiate the geojson.Reader you can specify a function that
// receives all objects the reader parsed. It is called when objects are
// being rendered on the map. At that point we can look into the object and
// check whether it is Antarctica
var reader = new H.data.geojson.Reader('...ATA.geo.json', {
style: function(obj) {
if (obj.getData().properties.name === "Antarctica") {
//AHA! We found Antarctica!
// Since this is a multi-polygon we have a group here which contains
// all polygons. We apply the north-pole-covering flag to each of the
// polygons
obj.forEach(function(polygon) {
polygon.setNorthPoleCovering(false);
});
}
}
});
reader.parse();
map.addLayer(reader.getLayer());
Depending on what you want to accomplish in terms of dynamic behavior, if you are just looking to display or share a map with cards and other metadata about a country with some basic styling -- HERE XYZ can be used to render GeoJSON on a HERE map.
If you want to do it with JavaScript rather than an embedded iframe, the other answer may be what you are looking for.
There is an there an issue with the GeoJSON, and other mapping APIs would have the same problem. It needs to be closed at the 180th meridian, so
[178.277212,-84.472518],[180,-84.71338],[-179.942499,-84.721443]
becomes
[178.277212,-84.472518],[180,-84.71338],[180,-90],[-180,-90],[-180,-84.71338],[-179.942499,-84.721443]
..and then Google-maps "divide the waters from the waters"
Well, not in the biblical sense but..
I would like to know what options I have in order to verify if a point of [Lat, Lon] is Land or Water.
Google Maps obviously has this data (the bodies of water are blue) - but is there something in the API that I can use for that? And if not - are they not serving it because they never thought of it? Or because it is too complicated?
I have not found any info on the matter - except some similar questions here (like finding type of terrain, or elevation - but it is not exactly what I need).
Is there separated layer for that? An option? Command? Or should I go to do that manually?
The only way that I can think of how to approach this (should I need to do that manually) is to check every served tile for the exact point - and then check RGB value for that Google map hue.
This is only on theory - because in practice - I have no idea how to accomplish that, the first obstacle being that I do not know how I can convert a pixel location on a tile to [LatLon] point for example
A ready made solution would be much easier.
Note that I do not need ALL the water in the world (for example - I do not care about streams, small ponds, most rivers or your neighbor's swimming pool. I need the points where a person can venture without the aid of a floating vehicle)
EDIT I
After reading comments:
The elevation method is not reliable, there are too many places BELOW sea-level (you can see a list of the "deepest" 10 here http://geology.com/below-sea-level/ ) and there are too many land-locked water bodies ABOVE sea level (lakes).
The reverse geolocation method is not reliable because it will return a Geo-political entity, like city, or state - or ZERO many times.
I have already looked into those pseudo-solutions before asking the question - but none of them actually answered the question - those methods are bad "guessing" at best.
These are 2 different ways, you may try:
You can use Google Maps Reverse Geocoding . In result set you can determine whether it is water by checking types. In waters case the type is natural_feature. See more at this link http://code.google.com/apis/maps/documentation/geocoding/#Types.
Also you need to check the names of features, if they contain Sea, Lake, Ocean and some other words related to waters for more accuracy. For example the deserts also are natural_features.
Pros - All detection process will be done on client's machine. No need of creating own server side service.
Cons - Very inaccurate and the chances you will get "none" at waters is very high.
You can detect waters/lands by pixels, by using Google Static Maps. But for this purpose you need to create http service.
These are steps your service must perform:
Receive latitude,longitude and current zoom from client.
Send http://maps.googleapis.com/maps/api/staticmap?center={latitude,longitude}&zoom={current zoom`}&size=1x1&maptype=roadmap&sensor=false request to Google Static Map service.
Detect pixel's color of 1x1 static image.
Respond an information about detection.
You can't detect pixel's color in client side. Yes , you can load static image on client's machine and draw image on canvas element. But you can't use getImageData of canvas's context for getting pixel's color. This is restricted by cross domain policy.
Prons - Highly accurate detection
Cons - Use of own server resources for detection
It doesn't seem possible with any current Google service.
But there are other services, like Koordinates Vector JSON Query service! You simply query the data in the URL, and you get back a JSON/XML response.
Example request: http://api.koordinates.com/api/vectorQuery.json?key=YOUR_GEODATA_KEY&layer=1298&x=-159.9609375&y=13.239945499286312&max_results=3&radius=10000&geometry=true&with_field_names=true
You have to register and supply your key and selected layer number. You can search all their repository of available layers. Most of the layers are only regional, but you can find global also, like the World Coastline:
When you select a layer, you click on the "Services" tab, you get the example request URL. I believe you just need to register and that's it!
And now the best:
You can upload your layer!
It is not available right away, hey have to process it somehow, but it should work! The layer repository actually looks like people uploaded them as they needed.
There is a free web API that solves exactly this problem called onwater.io. It isn't something built into Google maps, but given a latitude and longitude it will accurately return true or false via a get request.
Example on water:
https://api.onwater.io/api/v1/results/23.92323,-66.3
{
lat: 23.92323,
lon: -66.3,
water: true
}
Example on land:
https://api.onwater.io/api/v1/results/42.35,-71.1
{
lat: 42.35,
lon: -71.1,
water: false
}
Full disclosure I work at Dockwa.com, the company behind onwater. We built onwater to solve this problem ourselves and help the community. It is free to use (paid for high volume) and we wanted to share :)
I thought it was more interesting to do this query locally, so I can be more self-reliant: let's say I want to generate 25000 random land coordinates at once, I would rather want to avoid calls to possibly costly external APIs. Here is my shot at this in python, using the python example mentionned by TomSchober. Basically it looks up the coordinates on a pre-made 350MB file containing all land coordinates, and if the coordinates exist in there, it prints them.
import ogr
from IPython import embed
import sys
drv = ogr.GetDriverByName('ESRI Shapefile') #We will load a shape file
ds_in = drv.Open("land_polygons.shp") #Get the contents of the shape file
lyr_in = ds_in.GetLayer(0) #Get the shape file's first layer
#Put the title of the field you are interested in here
idx_reg = lyr_in.GetLayerDefn().GetFieldIndex("P_Loc_Nm")
#If the latitude/longitude we're going to use is not in the projection
#of the shapefile, then we will get erroneous results.
#The following assumes that the latitude longitude is in WGS84
#This is identified by the number "4236", as in "EPSG:4326"
#We will create a transformation between this and the shapefile's
#project, whatever it may be
geo_ref = lyr_in.GetSpatialRef()
point_ref=ogr.osr.SpatialReference()
point_ref.ImportFromEPSG(4326)
ctran=ogr.osr.CoordinateTransformation(point_ref,geo_ref)
def check(lon, lat):
#Transform incoming longitude/latitude to the shapefile's projection
[lon,lat,z]=ctran.TransformPoint(lon,lat)
#Create a point
pt = ogr.Geometry(ogr.wkbPoint)
pt.SetPoint_2D(0, lon, lat)
#Set up a spatial filter such that the only features we see when we
#loop through "lyr_in" are those which overlap the point defined above
lyr_in.SetSpatialFilter(pt)
#Loop through the overlapped features and display the field of interest
for feat_in in lyr_in:
# success!
print lon, lat
check(-95,47)
I tried a dozen coordinates, it works wonderfully. The "land_polygons.shp" file can be downloaded here, compliments of OpenStreetMaps. (I used the first WGS84 download link myself, maybe the second works as well)
This what I use and it is working not too bad... you can improve the test if you have more cpu to waste by adding pixels.
function isItWatter($lat,$lng) {
$GMAPStaticUrl = "https://maps.googleapis.com/maps/api/staticmap?center=".$lat.",".$lng."&size=40x40&maptype=roadmap&sensor=false&zoom=12&key=YOURAPIKEY";
//echo $GMAPStaticUrl;
$chuid = curl_init();
curl_setopt($chuid, CURLOPT_URL, $GMAPStaticUrl);
curl_setopt($chuid, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($chuid, CURLOPT_SSL_VERIFYPEER, FALSE);
$data = trim(curl_exec($chuid));
curl_close($chuid);
$image = imagecreatefromstring($data);
// this is for debug to print the image
ob_start();
imagepng($image);
$contents = ob_get_contents();
ob_end_clean();
echo "<img src='data:image/png;base64,".base64_encode($contents)."' />";
// here is the test : I only test 3 pixels ( enough to avoid rivers ... )
$hexaColor = imagecolorat($image,0,0);
$color_tran = imagecolorsforindex($image, $hexaColor);
$hexaColor2 = imagecolorat($image,0,1);
$color_tran2 = imagecolorsforindex($image, $hexaColor2);
$hexaColor3 = imagecolorat($image,0,2);
$color_tran3 = imagecolorsforindex($image, $hexaColor3);
$red = $color_tran['red'] + $color_tran2['red'] + $color_tran3['red'];
$green = $color_tran['green'] + $color_tran2['green'] + $color_tran3['green'];
$blue = $color_tran['blue'] + $color_tran2['blue'] + $color_tran3['blue'];
imagedestroy($image);
var_dump($red,$green,$blue);
//int(492) int(570) int(660)
if($red == 492 && $green == 570 && $blue == 660)
return 1;
else
return 0;
}
Checkout this article. It accurately detects if something is on the water without needing a server. It's a hack that relies on the custom styling feature in Google Maps.
In addition to the reverse geocoding -- as Dr Molle has pointed out, it may return ZERO_RESULTS -- you could use the Elevation service. If you get zero results by reverse geocoding, get the elevation of the location. Generally, the sea gets a negative number as the seabed is below sea level. There's a fully-worked example of the elevation service.
Bear in mind that as Google don't make this information available any other method is just a guess and guesses are inherently inaccurate. However using the type returned by reverse geocoding, or the elevation if type is not available, will cover most eventualities.
This method is totally unreliable.
In fact, the returned data will totally depend on what part of the world you are working with.
For example, I am working in France.
If I click on the sea on the coast of France, Google will return the nearest LAND location it can "guess" at.
When I requested information from Google for this same question, they answered that they are unable to accurately return that the point requested in on a water mass.
Not a very satisfactory answer, I know.
This is quite frustrating, especially for those of us who provide the user with the ability to click on the map to define a marker position.
If all else fails you could always try checking the elevation at the point and for some distance about - not many things other than water tend to be completely flat.
Unfortunately this answer isn't within the Google Maps API and the referenced resource is not free, but there's a web service provided by DynamicGeometry that exposes an operation GetWaterOrLand which accepts a latitude/longitude pair (you can see a demo here).
My understanding of how this is implemented is by using water body shape files. How exactly these shape files are used with the Google Maps API, but you might be able to get some insight from the linked demo.
Hope that helps in some way.
Here's another example in pure JavaScript: http://jsfiddle.net/eUwMf/
As you can see, the ideia is basically the same as rebe100x, getting the image from Google static map API, and read the first pixel:
$("#xGps, #yGps").change(function() {
var img = document.getElementById('mapImg');
// Bypass the security issue : drawing a canvas from an external URL.
img.crossOrigin='anonymous';
var xGps = $("#xGps").val();
var yGps = $("#yGps").val();
var mapUrl = "http://maps.googleapis.com/maps/api/staticmap?center=" + xGps + "," + yGps +
"&zoom=14&size=20x20&maptype=roadmap&sensor=false";
// mapUrl += "&key=" + key;
$(img).attr("src", mapUrl);
var canvas = $('<canvas/>')[0];
canvas.width = img.width;
canvas.height = img.height;
canvas.getContext('2d').drawImage(img, 0, 0, img.width, img.height);
var pixelData = canvas.getContext('2d').getImageData(1, 1, 1, 1).data;
if (pixelData[0] == 164 &&
pixelData[1] == 190 &&
pixelData[2] == 220) {
$("#result").html("Water");
} else {
$("#result").html("Not water");
}
});
See the answer I gave to a similar question - it uses "HIT_TEST_TERRAIN" from the Earth Api to achieve the function.
There is a working example of the idea I put together here: http://www.msa.mmu.ac.uk/~fraser/ge/coord/
If List<Address> address returns 0 , you can assume this location as ocean or Natural Resources.Just add Below Code in Your response Method of Google Places API Response.
Initialize Below List as mentioned
List<Address> addresses = geocoder.getFromLocation(latLng.latitude, latLng.longitude, 1);
if (addresses.size()==0)
{
Toast.MakeText(getApplicationContext,"Ocean or Natural Resources selected",Toast.LENGTH_SHORT).show();
}else{
}
I would recommend rolling your own here. You can use tools like GDAL to query the contents under a point in a shapefile. You can get shapefiles for US geography from many sources including the US Census Bureau.
This can be done via GDAL binaries, the source C, or via swig in Java, Python, and more.
Census Maps
GDAL Information
Point Query Example in Python
Here is a simple solution
Because Google does not provide reliable results with regards to coordinates that lay on either ocean or inland bodies of water you need to use another backup service, such as Yandex, to help provide that critical information when it is missing. You most likely would not want to use Yandex as your primary geocoder because Google is far superior in the reliability and completeness of the worlds data, however Yandex can be very useful for the purpose of retrieving data when it relates to coordinates over bodies of water, so use both.
Yandex Documentation: https://api.yandex.com.tr/maps/doc/geocoder/desc/concepts/input_params.xml
The steps to retrieve Ocean name:
1.) Use Google first to reverse geocode the coordinate.
2.) If Google returns zero results, it is 99% likely the coordinate lies over an ocean. Now make a secondary reverse geocoding request with the same coordinates to Yandex. Yandex will return a JSON response with for the exact coordinates, within this response will be two "key":"value" pairs of importance
["GeoObject"]["metaDataProperty"]["GeocoderMetaData"]["kind"]
and
["GeoObject"]["name"]
Check the kind key, if it == "hydro" you know you are over a body of water, and because Google returned zero results it is 99.99% likely this body of water is an ocean. The name of the ocean will be the above "name" key.
Here is an example of how I use this strategy written in Ruby
if result.data["GeoObject"]["metaDataProperty"]["GeocoderMetaData"]["kind"] == "hydro"
ocean = result.data["GeoObject"]["name"]
end
The steps to retrieve an Inland Body of Water name:
For this example assume our coordinate lies in a lake somewhere:
1.) Use Google first to reverse geocode the coordinate.
2.) Google will most likely return a result that is a prominent default address on land nearby. In this result it supplies the coordinates of the address it returned, this coordinate will not match the one you provided. Measure the distance between the coordinate you supplied and the one returned with the result, if it is significantly different (for example 100 yards) then perform a secondary backup request with Yandex and check to see the value of the "kind" key, if it is "hydro" then you know the coordinate lies on water. Because Google returned a result as opposed to the example above, it is 99.99% likely this is an inland body of water so now you can get the name. If "kind" does not == "hydro" then use the Google geocoded object.
["GeoObject"]["metaDataProperty"]["GeocoderMetaData"]["kind"]
and
["GeoObject"]["name"]
Here is the same code written in Ruby to get inland_body_of_water
if result.data["GeoObject"]["metaDataProperty"]["GeocoderMetaData"]["kind"] == "hydro"
inland_body_of_water = result.data["GeoObject"]["name"]
end
A note about Licensing: As far as I know Google does not allow you to use their data to display on any other maps other than those Google offers. Yandex however has very flexible licensing, and you can use their data to be displayed on Google maps.
Also Yandex has a a high rate limit of 50,000 request / day free of charge, and with no required API key.
I managed to get quite close by using the Google Elevation API. Here's an image of the results:
You see the hexagons pretty much stay on land even though a rectangular perimeter is defined that goes partly over water. In this case I did a quick check from Google Maps itself and the minimum elevation on land was about 8-9m so that was my threshold. The code is mostly copy/pasted from Google documentation and Stack Overflow, here's the full gist:
https://gist.github.com/dvas0004/fd541a0502528ebfb825
As a complete novice to Python I couldn't get SylvainB's solution to work with the python script that checks if coordinates are on land. I managed to figure it out however, by downloading OSGeo4W (https://trac.osgeo.org/osgeo4w/) and then installed everything I needed pip, Ipython, and checked that all the imports specified were there. I saved the following code as a .py file.
Code to check if coordinates are on land
###make sure you check these are there and working separately before using the .py file
import ogr
from IPython import embed
from osgeo import osr
import osgeo
import random
#####generate a 1000 random coordinates
ran1= [random.uniform(-180,180) for x in range(1,1001)]
ran2= [random.uniform(-180,180) for x in range(1,1001)]
drv = ogr.GetDriverByName('ESRI Shapefile') #We will load a shape file
ds_in = drv.Open("D:\Downloads\land-polygons-complete-4326\land-polygons-complete-4326\land_polygons.shp") #Get the contents of the shape file
lyr_in = ds_in.GetLayer(0) #Get the shape file's first layer
#Put the title of the field you are interested in here
idx_reg = lyr_in.GetLayerDefn().GetFieldIndex("P_Loc_Nm")
#If the latitude/longitude we're going to use is not in the projection
#of the shapefile, then we will get erroneous results.
#The following assumes that the latitude longitude is in WGS84
#This is identified by the number "4236", as in "EPSG:4326"
#We will create a transformation between this and the shapefile's
#project, whatever it may be
geo_ref = lyr_in.GetSpatialRef()
point_ref=osgeo.osr.SpatialReference()
point_ref.ImportFromEPSG(4326)
ctran=osgeo.osr.CoordinateTransformation(point_ref,geo_ref)
###check if the random coordinates are on land
def check(runs):
lon=ran1[runs]
lat=ran2[runs]
#Transform incoming longitude/latitude to the shapefile's projection
[lon,lat,z]=ctran.TransformPoint(lon,lat)
#Create a point
pt = ogr.Geometry(ogr.wkbPoint)
pt.SetPoint_2D(0, lon, lat)
#Set up a spatial filter such that the only features we see when we
#loop through "lyr_in" are those which overlap the point defined above
lyr_in.SetSpatialFilter(pt)
#Loop through the overlapped features and display the field of interest
for feat_in in lyr_in:
return(lon, lat)
###give it a try
result = [check(x) for x in range(1,11)] ###checks first 10 coordinates
I tried to get it to work in R but I had a nightmare trying to get all the packages you need to install so stuck to python.
Here's a typed async function that returns true or false if a lat/lng is water or not. No need to pay for external api's. You must enable static maps on google cloud though.
async function isLatLngWater(lat: number, lng: number) {
return new Promise<boolean>((resolve) => {
const img = new Image();
img.crossOrigin = "Anonymous";
img.onload = () => {
const canvas = document.createElement("canvas");
const ctx = canvas.getContext("2d");
ctx!.drawImage(img, 0, 0);
const { data } = ctx!.getImageData(10, 10, 1, 1);
if (data[0] == 156 && data[1] == 192 && data[2] == 249) {
canvas.remove();
resolve(true);
} else {
canvas.remove();
resolve(false);
}
};
img.src =
"https://maps.googleapis.com/maps/api/staticmap?center=" +
lat +
"," +
lng +
"&size=40x40&maptype=roadmap&sensor=false&zoom=20&key=" +
import.meta.env.VITE_GM_MSTAT;
});
}
There an API service called IsItWater.com which will let you check:
Request
curl 'https://isitwater-com.p.rapidapi.com/?latitude=41.9029192&longitude=-70.2652276&rapidapi-key=YOUR-X-RAPIDAPI-KEY'
Response
{
"water": true,
"latitude": 41.9029192,
"longitude": -70.2652276
}
I have a different solution here.
In current google map implementation, it does not calculate direction/distance from a water location to land location and vice versa. Why dont we use this logic to determine if the point is land or water.
For example lets take this example
if we want to determine, if a point x is land or water, then
let us check the direction between point x and a known point y which is land. If it determines the direction/distance then point x is land or else it is water.