I'm following along Mike Bostock's example for creating a geoJSON projection by downloading and converting shapefiles to geoJSON. I've downloaded the files correctly, and they're online here:
https://bl.ocks.org/KingOfCramers/2c5ceb2e7526a8370d6926958654cf21
This works fine (obviously in the future I'll simplify the shape file to get it to run faster on a browser). Right now, I want to be able to replicate this process quickly for other shapefiles. I've downloaded many from Natural Earth and have converted them successfully into JSON files for use in geoJSON and topoJSON, but I am unsure how to determine which projection to use on them.
Is there a way to quickly examine a .shp file (or after it's been converted JSON) to determine which D3 projection to use, which "translate" values to use, and any other presets for my projection? Or, if I'm going to use geoproject prior to even mapping the file, how I do I find the values to plug in? Here's Mike Bostock's example:
geoproject 'd3.geoConicEqualArea().parallels([34, 40.5]).rotate([120, 0]).fitSize([960, 960], d)' < ca.json > ca-albers.json
How does he know the rotate value? How does he know which parameters to feed into this function?
For an un-finished example, here's my bl.ock of the current earth, but the projection breaks the JSON, because obviously my projection settings are not right:
http://blockbuilder.org/KingOfCramers/16be1bf014683572086511c6a8bd7470
-- or --
https://bl.ocks.org/KingOfCramers/16be1bf014683572086511c6a8bd7470
I can drop this JSON file into mapshaper, which projects it quickly and flawlessly. I want to be able to do this in D3, or at the very least convert the file before mapping it. I'm assuming that information is stored somewhere in the JSON file? Or can be accessed somehow using the JSON projection converter that Mike Bostock recommends, geoproject? Thanks for any help you can provide!
Key Issue
D3 assumes the file to be projected requires projection - that is to say, it assumes the file has not already been projected. This applies if pre-projecting your files from the command lie so that you can display them without a d3 projection. If using projected features, you will not get the results you want - you must unproject your features first.
If using a standard d3 projection such as d3.geoAlbers, data must be unprojected and contain latitude longitude pairs.
Unprojected vs Projected
Unprojected features are those which have latitude and longitude coordinates, they are points located on a three dimensional globe. To display these we need a projection function (most simply: lat = y, long = x, a plate carree projection).
Projected features are those which have Cartesian x,y coordinates. They are the product of some projection function which as a consequence introduces distortion of some or all of: shape, area, distance, or direction.
Signs of Using Projected Data
Upside Down Features
Upside down features are an easy indicator that your features are already projected. Projected geographic data generally features and origin at the bottom left of the features, as one moves north, y values increase. SVG coordinate space is the opposite, as one moves south y values increase.
When displaying your data in mapshaper if you include a shapefiles .prj file, mapshaper will project your data according to this. This will ensure that north is true. When displaying this data with d3, there is no flip on the y axis unless you bake that into the projection function.
Projection File
Secondly, the prj file that comes with every shapefile (or the vast majority) will tell you if features are projected or not. If your prj file lists anything like Albers, Conic, etc, then you have projected data. You need to have your data "projected" using the WGS84 datum or unprojected (also using WGS84). Data using this coordinate space has the EPSG number of 4326, and the prj file should look something like:
GEOGCS["GCS_WGS_1984",DATUM["D_WGS_1984",SPHEROID["WGS_1984",6378137,298.257223563]],PRIMEM["Greenwich",0],UNIT["Degree",0.017453292519943295]]
Coordinate Domain
Lastly, if in mapshaper (or any other GIS utility that handles geojson) you were to export the data as geojson, if you see coordinates in excess of [+/-180,+/-90] you are probably dealing with projected data, which often uses a unit of measure like meters.
If you include a file and a projection function I can provide some more specific signs rather than these generalizations.
Easy Solution
If you don't want to modify the projection the data came in, you can use an identity projection:
d3.geoIdentity().reflectY(true).fitSize([w, h], geojson)
This will not modify the input projection, essentially it is only scaling and flipping the input features to match your intended svg/canvas dimensions.
The downside is you can't take features that are already projected as an Albers Equal Area and convert it straight into a Azimuthal Equidistant projection. Also, this approach may make it difficult to overlay locations with geographic coordinates on top of your pre-projected features - for that you will need to re-create the projection the features originally came in.
The upside is the simplicity, it is fine for choropleths or visualizations where nothing geographic is overlain on the projected features.
More Flexible Solution
Unproject your data first, in mapshaper you can do this, assuming you imported the prj files by using the console window and typing:
proj wgs84
Now you can reproject or preproject for d3. Other tools exist for the command line, while programs like QGIS can help convert data quickly too.
The advantage to this is that you can easily re-apply the projection you used on the command line to any points you want to overlay on top, and of course you can modify the projection easily.
What Project Parameters To Choose
If following the 2nd approach or overlaying geographic coordinates on top of features displayed using the 1st approach, the question of what projection parameters to choose becomes relevant again.
The projection parameters chosen are chosen very specifically and often taken straight from standard projections. The .prj file of a shapefile contains everything you need to re-create the projection used in the shapefile. This answer goes into how to emulate a prj file with a d3 projection.
SpatialReference.org is a great reference for finding parameters to different projections. There is a good chance that the California Albers example was based on a standard projection that you can find on this site, probably this one. Of course though, when Mike Bostock used this projection, he applied it to unprojected data.
Related
I am using react-d3-map-bubble library to plot US map.
react lib- https://github.com/react-d3/react-d3-map-bubble
US map json file used - https://d3js.org/us-10m.v1.json
My target is to plot points on that map.
In current js file, counties object uses arcs and plots different points on US map. See here for pictorial discription - https://bl.ocks.org/mbostock/9943478
I have lat and longs instead of arcs/polygons.
So, in json file I will be changing 'countries' object with below sample :
{
"type": "GeometryCollection",
"geometries": [
{
"type": "Point",
"properties": {
"name": "Some place in New Mexico",
"population": 54590
},
"arcs": [
[]
],
"coordinates":[33.500142,-111.929818]
}
]
}
With given transform function in US json I am not able to put given coordinates to the correct place on the Map.
Transform function
"transform": {
"scale": [0.09996701564084985, 0.058373467440357915],
"translate": [-56.77775821661018, 12.469025989284091]
}
So,
How could I get correct transform function for my coordinates which uses topojson?
OR
How could I get all coordinates converted into arcs/polygons, which could be directly placed in "counties" object?
Topojson and coordinates
I think there is some confusion about what topojson does: "I have lat and longs instead of arcs/polygons".
Topojson saves space by applying a "quantized delta-encoding for integer coordinates", but the real space savings can come from the use of arcs (geojson would contain the mutual boundary of two features twice, once for each feature). But d3 doesn't actually draw topojson, the data is converted back into geojson (as seen in your linked example):
topojson.feature(us, us.objects.counties).features
If moving from geojson to topojson and back again, your original coordinate system remains unchanged. So if you start with lat longs, you'll finish with lat longs. Arcs and polygons aren't alternative coordinate systems, they are shapes built with coordinates in any given coordinate system.
Problem
The problem is your topojson has encoded coordinates that aren't latitude and longitude pairs - in this case they are pixel values so that the feature is drawn on a coordinate plane that stretches from [0,0] to [960,600] in svg coordinate space. Why would they be in this odd coordinate space that isn't latitude longitude? So that you don't need to use a projection to draw them, this is faster in the browser and simpler for an example like what Mike Bostock might be trying to demonstrate in the linked block.
The topojson uses a projected coordinate space (which is on a 2d Cartesian plane). Your latitude longitude uses geographic coordinate space (which are points on a three dimensional ellipsoid).
Solutions
To convert your points to projected coordinate space you need to apply the same projection if you want features to be aligned.
Alternatively, you can get unprojected data for the US and project that and your latitude longitude points with the same projection.
Apply the same projection used for the US to each point
Normally you are not be able to easily figure out what projection a geojson or topojson (or shapefile for that matter if it is without a .prj file) uses if it is already projected. In this case we know it is a composite d3.geoAlbersUsa() projection, but we don't know the projection's parameters so we can't use it, unless we had access to additional data about the file, or if we made it ourselves and as a consequence knew what parameters we used.
Even if we had this information, we may find it cumbersome to scale the map for different svg/canvas sizes, as we would need to scale both projected data and unprojected data differently since we would be working with two different coordinate systems.
Get unprojected data for the US
It is not too difficult to find geographic data for the US states/counties/etc online. If you find shapefiles, they are easy to convert in tools such as mapshaper.org, though geojson or topojson files shouldn't be hard to find either. You just need to make sure that the coordinates use latitude longitude pairs (with the order of [long,lat], the example coordinate in your question uses [lat,long] which won't work.) rather than any other coordinate system.
Once you have this you need to make sure you apply the same projection to all features.
Why Scale and Translate Don't Help
The scale and translate of the topojson also don't refer to the projection parameters, but the quantized delta encoding used in the topojson.
I have a set of lat/long 37.786453, -122.490402 that when mapped correctly maps to San Francisco. However, in my code. It is mapping to western Utah. I am able to manipulate the location of the point by altering this section of code that has been unchanged from the original block I utilized:
projection
.scale(1000)
.center([-106, 37.5])
What correct value(s) need to be placed here? If I remove the section entirely it places the point in eastern Oregon.
Here is the current fiddle: https://jsfiddle.net/Lv5knc0n/
This is a follow-up question regarding plotting lat/long on a map using d3.js. Link to the first post here.
Your fiddle is showing features with two different projections:
A null projection for the US topojson
A Mercator projection for the point
For the topojson shown with a null projection:
The US topojson referenced has come up in many other posts, it is a composite Albers projection of the US (which incorporates multiple projections in one frame). The data in it is already projected, hence the use of a null projection. The null projection simply takes the x,y coordinate of each point and translates it to svg coordinates with no transform (in mapshaper.org or any geographic software, it will appear upside down as svg coordinates start at the top, while geographic coordinates start at the equator, below your map).
Ultimately you are matching an Albers to a Mercator - and when you make changes to the Mercator projection:
projection
.scale(1000)
.center([-106, 37.5])
You are not changing the topojson since the path generator for it doesn't use a projection. You are only changing the point's projection. And as you are using two different projections, the same geographic point will be represented differently in each projection - making alignment of multiple points problematic if not impossible (depending on the points and projections).
There is a relatively straightforward solution, use unprojected data for your US Counties (where the spatial data consists of lat long pairs) and project it with the same projection as your points. This will allow you to use one projection for both points and paths, or any other features, allowing you to scale and zoom all features at once (otherwise, you will need to use a geoTransform to manipulate the counties data while trying to match those changes with changes to the projection, not an ideal solution).
Try using the json in this block (here).
So I started with a really small example that is working well. I used QGIS to draw a line and then converted it to topojson with mapshaper. Then I use D3.js to load and visualize it. This is the bigger but not working example:
jsfiddle.net/kwoxer/kpL1uyy2/2/
As you can see it not showing just one line, it is showing crazy lines as if the convert gone wrong. Already tested different browsers.
But as I said I already did a small line before with QGIS, converted it, and everything was fine. So is this an issue of the size of the line? Or by the converter?
Here a picture from QGIS how it should like in the browser: http://i.imgur.com/s1FPn2P.png
So what is a good way to create an (huge) own map and using it in D3.js?
I looked at your Fiddle, and I think I see the problem. You are using d3's Equirectangular map projection, which expects lat-long coordinates with x values in the range [-180, 180] and y values in the range [-90, 90]. The actual x and y ranges of your dataset are [-991.4407052281722, 6787.6906928973385] and [-4789.571699454693, -155.32649155239142]. When d3 encounters coordinates outside the expected range, it wraps them -- this is why the lines look "crazy."
Based on your coordinate values, I'm guessing that you're using a projected coordinate system in your QGIS project. To display the data using d3, you can either export your shapes in lat-long coordinates (e.g. by selecting "WGS 84" as the CRS when saving a QGIS layer), or you can use d3 with projected data (see Drawing already projected geoJSON map in d3.js)
I have a geojson object defining Neighborhoods in Los Angeles using lon/lat polygons. In my web application, the client has to process a live stream of spatial events, basically a list of lon/lat coordinates. How can I classify these coordinates into neighborhoods using Javascript on the client (in the browser)?
I am willing to assume neighborhoods are exclusive. So once a coordinate as been classified as neighborhood X, there is no need to further test it for other neighborhoods.
There's a great set of answers here on how to solve the general problem of determining whether a point is contained by a polygon. The two options there that sound the most interesting in your case:
As #Bubbles mentioned, do a bounding box check first. This is very fast, and I believe should work fine with either projected or unprotected coordinates. If you have SVG paths for the neighborhoods, you can use the native .getBBox() method to quickly get the bounding box.
the next thing I'd try for complex polygons, especially if you can use D3 v3, is rendering to an off-screen canvas and checking pixel color. D3 v3 offers a geo path helper that can produce canvas paths as well as SVG paths, and I suspect if you can pre-render the neighborhoods this could be very fast indeed.
Update: I thought this was an interesting problem, so I came up with a generalized raster-based plugin here: http://bl.ocks.org/4246925
This works with D3 and a canvas element to do raster-based geocoding. Once the features are drawn to the canvas, the actual geocoding is O(1), so it should be very fast - a quick in-browser test could geocode 1000 points in ~0.5 sec. If you were using this in practice, you'd need to deal with edge-cases better than I do here.
If you're not working in a browser, you may still be able to do this with node-canvas.
I've seen a few libraries out there that do this, but most of them are canvas libraries that may rely on approximations more than you'd want, and might be hard to adapt to a project which has no direct need to rely on them for intersections.
The only other half-decent option I can think of is implementing ray casting in javascript. This algorithm isn't technically perfect since it's for Euclidean geometry and lat/long coordinates are not (as they denote points on a curved surface), but for areas as small as a neighbourhood in a city I doubt this will matter.
Here's a google maps extension that essentially does this algorithm. You'd have to adapt it a bit, but the principles are quite similar. The big thing is you'd have to preprocess your coordinates into paths of just two coordinates, but that should be doable.*
This is by no means cheap - for every point you have to classify, you must test every line segment in the neighborhood polygons. If you expect a user to be reusing the same coordinates over and over between sessions, I'd be tempted to store their neighborhood as part of it's data. Otherwise, if you are testing against many, many neighborhoods, there are a few simple timesavers you can implement. For example, you can preprocess every neighborhoods extreme coordinates (get their northmost, eastmost, southmost, and westmost points), and use these to define a rectangle that inscribes the town. Then, you can first check the points for candidate neighborhoods by checking if it lies inside the rectangle, then run the full ray casting algorithm.
*If you decide to go this route and have any trouble adapting this code, I'd be happy to help
I'm trying to plot MGRS lines over a map in an overlay using OpenLayers (JavaScript). Where I'm really having problems is identifying the strange squares (non-100kmx100km grids). Does anyone know where I might find an algorithm for plotting these?
In particular, the information I have or can find is:
Convert a Lat/Long to MGRS
Convert a full MGRS string (i.e., 17SLA123678 but not 17SLA) to Lat/Lon
Convert Lat/Lon to screen pixel and vice-versa
Thanks!
I don't know the MGRS system, but the Proj4JS library may be useful. This is used to transform between coordinate systems, so you can set it up to convert from a coordsys to pixels, or more commonly from one coordsys to another.
it is open source, and broadly based on the well known proj.4 library, and interfaces with OpenLayers. Actually, OpenLayers uses proj4js to transform between different coordinate systems.
I just added some of that functionality to https://github.com/jaycrossler/js-maptools (uses Leaflet instead of OpenLayers). It will draw a polygon over the USNG/MGRS cell that the mouse is over.