I'm looking into some options for a mapping project (web, probably HTML5 Canvas based). Ideally, we are creating a navigational map (i.e. Google Maps) that would have our own very simple imagery (not satellite), but we are looking into ways of getting state / city lines, and maybe other important features. Does anyone know of a good database or project that contains this data correlated to longitude and latitude? Or are we looking at potentially drawing these lines ourselves?
There is a huge amount of choice available, but it really does depend on exactly what your looking for, what level of detail and which countries.
One of the first places you'll want to look is open geo:
http://opengeo.org/
These guys have links to all the open source geographic stuff that's available more or less.
second to that is the OS-Geo consortium:
http://www.osgeo.org/
Again these guys maintain pre-made kits and links to the majority of data & applications available.
Once you go beyond here, then you likely will need to start looking at country organisations like the USGS:
http://www.usgs.gov/pubprod/
and
http://egsc.usgs.gov/isb/pubs/factsheets/worldmaps.html
you'll most likely also want to take a look at some of the country specific data sets available such as the UK's ordnance survey:
http://www.ordnancesurvey.co.uk/oswebsite/products/os-opendata.html
and the USA Tiger data sets:
http://www.census.gov/geo/www/tiger/
finally, you could take a look at the likes of "Open street map" (And other similar products)
http://www.openstreetmap.org/
Open street map originally set out to cover the UK, but seems to be going at least Europe wide now (Maybe even world wide)
This is all I can remember off the top of my head at the moment, but one other tip I'll give, sites belonging to the top GIS software providers such as ARC-GIS & Mapinfo all have lists of mapping data providers on their websites, and some even have free data available too, an afternoons browsing should uncover you a lot of sources.
http://openstreetmap.org should have what you need.
Related
I am currently looking at an efficient way to visualise a lot of data in javascript. The data is geospatial and I have approximately 2 million data points.
Now I know that I cannot give that many datapoint to the browser directly otherwise it would just crash most of the time (or the response time will be very slow anyway).
I was thinking of having a javascript window communicating with a python which would do all the operations on the data and stream json data back to the javascript app.
My idea was to have the javascript window send in real time the bounding box of the map (lat and lng of north east and south west Point) so that the python script could go through all the entries before sending the json of only viewable objects.
I just did a very simple script that could do that which basically
Reads the whole CSV and store data in a list with lat, lng, and other attributes (2 or 3)
A naive implementation to check whether points are within the bounding box sent by the javascript.
Currently, going through all the datapoints takes approximately 15 seconds... Which is way too long, since I also have to then transform them into a geojson object before streaming them to my javascript application.
Now of course, I could first of all sort my points in ascending order of lat and lng so that the function checking if a point is within the javascript sent bounding box would be an order of magnitude faster. However, the processing time would still be too slow.
But even admitting that it is not, I still have the problem that at very low zoom levels, I would get too many points. Constraining the min_zoom_level is not really an option for me. So I was thinking that I should probably try and cluster data points.
My question is therefore do you think that this approach is the right one? If so, how does one compute the clusters... It seems to me that I would have to generate a lot of possible clusters (different zoom levels, different places on the map...) and I am not sure if this is an efficient and smart way to do that.
I would very much like to have your input on that, with possible adjustments or completely different solutions if you have some.
This is almost language agnostic, but I will tag as python since currently my server is running python script and I believe that python is quite efficient for large datasets.
Final note:
I know that it is possible to pre-compute tiles that I could just feed my javascript visualization but as I want to have interactive control over what is being displayed, this is not really an option for me.
Edit:
I know that, for instance, mapbox provides the clustering of data point to facilitate displaying something like a million data point.
However, I think (and this is related to an open question here
) while I can easily display clusters of points, I cannot possibly make a data-driven style for my cluster.
For instance, if we take the now famous example of ethnicity maps, if I use mapbox to cluster data points and a cluster is giving me 50 people per cluster, I cannot make the cluster the color of the most represented ethnicity in the sample of 50 people that it gathers.
Edit 2:
Also learned about supercluster, but I am quite unsure whether this tool could support multiple million data points without crashing either.
I'm looking for a solution to get data to create a simple roads-only map.
Within a small area specified by lat/long I need data that allows me to draw custom lines between given street coordinates (start and endpoints are basically enough and only from major streets). Coordinates from intersections are also fine to connect those with my custom lines.
Another approaches would be to get all the major street names within the specified area via reverse geocoding and then - somehow - get coordinates for each of this streets.
There was a similar request a couple years ago:
Get street graph for a game using Google Maps API and I'm wondering is it still a big deal to achieve this kind of technical information in a simple way?
EDIT:
I researched the topic but no result gave me the satisfaction I was looking for. For instance Google's Directions/Roads API are both depending on routes. OSM/Google Maps API are filled with information around a specified geo location but without any technical data I need. To be simple: An Array of intersection/street-point coordinates (are they even called coordinates?!).
I think I'm lost - or even stuck - in terms of my own search phrases. I'm not looking for a copy & paste snippet more like a direction or some hints where to focus next or how it is done.
With the clarification of geocodezip's comment I've found the following related question:
How to get all roads around a given location in OpenStreetMap?
This fully suits my demand to get specific road information within a certain location using OSM instead of Google Maps API.
Many thanks
I would like to create an commercial Android GPS app that allows to route a way depending on a speed limit and a vehicle restrictions such us its height and width. I picked GraphHooper as a routing engine and because it does not provide informations about the vehicle's size, I want to add some of the Truck QA Map tags (http://maxheight.bplaced.net/overpass/map.html) and apply it to the GraphHooper's algorithm.
The problem is that GraphHooper is written in Java and from what I see on the GitHub (https://github.com/mmd-osm/osm-maxheight-map), QA Map is based on javascript, php and css.
My question is, is it possible to apply javascript, php and css code to the android project? At this moment I'm considering different solutions and I wonder if it could be done in general. If yes, how it can be done?
There seems to be a misunderstanding.
GraphHopper is a routing engine which is able to process OSM data. Likewise the OSM Truck QA Map is just using OSM data, too (especially the maxheight tag, the OSM wiki has more information about this map).
If GraphHopper doesn't support the maxheight tag yet then all you have to do is add it to GraphHopper. There is absolutely no need to extract it from the QA map because it is already contained in the data used by GraphHopper. You just need to tell GraphHopper to additionally check this tag when calculating its route the same way it already checks lots of other routing tags (like maxspeed, access and so on).
The main purpose of the OSM Truck QA Map is to help you spot any missing maxheight tags in the OpenStreetMap data or visualize already existing tags for trucks, such as maxheight, maxweight, etc. All the data comes in fact from the fabulous Overpass API, so there's not much point trying to extract the data form OSM Truck QA Map. It is really just a visualization tool and works completely in your browser.
Assuming you already have a OSM planet or some smaller extract available for testing, all those maxheight, maxweight, ... tags already exist in that data. You just need to include them in GraphHopper processing (see Scai's reply).
... and if some of the routing results look odd, you might want to check back on OSM Truck QA Map what the actual tagging looks like in the OSM data - or even check what it looked like some time back in the past.
Is it possible to query a UK Postcode Database that is available offline and gather Geo Coordinates with JavaScript?
I am under the impression that the database is too vast to do this without online/local database management.
Has anyone found a solution for this problem?
Thanks.
EDIT:
The data will need to be used by a private organisation and does not need to be either hugely accurate nor updated regularly. (Basically the user needs to find the nearest location to their specific postcode - < 10 locations). I Think my concern is seeing the 550+mb databases.
It also needs to work purely in the browser offline using JavaScript. JSON format would be preferred.
This is possible and there are few choices but first a few questions.
To what level of accuracy do you need the coordinates
The more exact, the more it will cost
How much time and effort do you want to put into long term maintenance
The data is regularly updated so make sure you have a business process in place
Are you building a solution for an existing company, or trying something out as a startup?
It is a large amount of data, but is manageable. The practicalities depend more upon the amount of time and money you want to invest.
There are two approaches:
1) Build your own
If needing premise level coordinates, you'll need to get a copy of the postcode file from the Royal Mail here and depending upon your level of accuracy you'll then need to also grab a supply of coordinate information from someone such as Ordnance Survey.
Load and link together the supplies into some sort of database or other store you are comfortable working in and make sure this process is easily repeatable once a month when the data comes through.
For just postcode level, I think the Code-point Open data should be good for you. If you are not worried about individual premises and postcode level is good enough. That comes to about 150MB in CSV files but you can get it a lot smaller than that by just storing the data you need.
The next step and develop a look up mechanism. The quality of your look up will depend upon your use case. If you are just after a rough location from the postcode this will be simpler than if you are after it down to the nearest cm and need a full premise address.
2) Go third party.
There are many companies around that can do this for you. My company - Experian Data Quality - provide tools to allow you to do this offline as a library or SOAP based service.
Once integrated, they will allow your users to quickly and easily find their correct address and then in the same call enhance the address with location (and many other types) of information.
This will cope with typos, data updates etc. Fill in the form or give us a call for a demo if sounds useful http://www.qas.co.uk/solutions/data-quality-software/data-capture/address-checker.htm
There is sample integration code available that will allow you to hook it into your website with JavaScript.
You can buy the database from the Royal Mail:
http://www.poweredbypaf.com/end-user/products/data-products/paf-raw-data/
This is the official database from the Royal Mail and can be downloaded and used offline.
I am working on a quote calculator that will generate a quote based on mileage between various locations (amongst other conditionals). Up until two days ago, I had planned to use Google's Distance Matrix service until I discovered:
Display of a Google Map
Use of the Distance Matrix service must relate to the display of information on a Google Map; for example, to determine origin-destination pairs that fall within a specific driving time from one another, before requesting and displaying those destinations on a map. Use of the service in an application that doesn't display a Google map is prohibited.
I had hoped to use only the services that I require:
- Distance by Road Measurement between up to three different locations
- Address Autocomplete Service usable on an input box
- Accurate, reliable service that can provide multiple different routes to create an average distance
I know there are other methods available for this, but I doubt many can be as accurate and reliable as Google, I've found it challenging to find anything comparable to Google Maps for the purposes I require.
So, unless you guys can point me to something that I can use, my only option is to use a Google Map where I don't need it, adding additional loading time and altering the UX design I had planned.
Are there any free services available for what I require (preferably with a JS API)?
On a slightly different note
If I do use a Google map, would it have to be displayed immediately, or could I hide it and add an option to 'Show On Map', and have it .slideToggle revealed?
Unfortunately for the Distance Matrix API, Google strictly says you NEED to display the map in your application:
Use of the Distance Matrix API must relate to the display of information on a Google Map; for example, to determine origin-destination pairs that fall within a specific driving time from one another, before requesting and displaying those destinations on a map. Use of the service in an application that doesn't display a Google map is prohibited.
http://developers.google.com/maps/documentation/distancematrix/#Limits
However, what I think is more useful for your need is Google Directions API. The directions API allows you to cover your requirements.
The total distance is returned in the JSON object from the request.
You can select upto as many different locations to find distances between using the Waypoints in your search request. The distances between these locations are then returned in each "leg".
You can obtain the average distance from multiple different routes to your destination by specifying the alternatives parameter in your search request to true. See: http://developers.google.com/maps/documentation/directions/#RequestParameters
Best of all, there is no requirement from Google to display the Google Map in your application when using this service.
I should also mention the drawbacks to this service, if you choose to use it.
The request time it takes to process your request will be slightly longer than if you were to use the Distance Matrix API.
You'll have a lot of unneeded data in the return object, for instance the individual "steps" of the route in the returned json object is not necessary based on your application requirements.
Given the drawbacks, I'd still highly recommend looking into the Directions API for your application.
I don't know if Google Static Maps count as a map, but it should, since it's a Map and from Google.
You could calculate the route and then show it as an image from Static Maps. No extra map loading times required. Only one image.
https://developers.google.com/maps/documentation/staticmaps/#Paths
Many developers have been able to do this with the Bing Maps REST routing service http://msdn.microsoft.com/en-us/library/ff701705.aspx. It requires a bit more development but works well. Here is an example: http://code.msdn.microsoft.com/Bing-Maps-trip-optimizer-c4e037f7
Streetmap and arcserver can solve a vehicle routing problem but it's not free. Read more here: http://www.esri.com/data/streetmap.