diff --git a/.travis.yml b/.travis.yml index f642106070e28720590bd6fa0040ef049ebb6dfd..460484cd92e691c512288eced943091a07cf11df 100644 --- a/.travis.yml +++ b/.travis.yml @@ -2,6 +2,7 @@ rvm: - 1.8.7 - 1.9.2 - 1.9.3 + - jruby-19mode gemfile: - Gemfile - gemfiles/Gemfile.mongoid-2.4.x @@ -17,3 +18,6 @@ matrix: - rvm: 1.9.3 gemfile: gemfiles/Gemfile.mongoid-2.4.x env: SSL_CERT_DIR=/etc/ssl/certs + - rvm: jruby-19mode + gemfile: gemfiles/Gemfile.mongoid-2.4.x + env: SSL_CERT_DIR=/etc/ssl/certs diff --git a/CHANGELOG.rdoc b/CHANGELOG.rdoc index 3591852cad61e7f66fbce8f5c0f8703c1d3f2b9a..e299403d35e8e0a0c392cc1d595842c8f4b69d12 100644 --- a/CHANGELOG.rdoc +++ b/CHANGELOG.rdoc @@ -2,11 +2,26 @@ Per-release changes to Geocoder. -== 1.2.0 (???) +== 1.1.5 (2012 Nov 9) -* Add support for setting arbitrary params in geocoding request URL. +* Replace support for old Yahoo Placefinder with Yahoo BOSS (thanks github.com/pwoltman). +* Add support for actual Mapquest API (was previously just a proxy for Nominatim), including the paid service (thanks github.com/jedschneider). +* Add support for :select => :id_only option to near scope. +* Treat a given query as blank (don't do a lookup) if coordinates are given but latitude or longitude is nil. +* Speed up 'near' queries by adding bounding box condition (thanks github.com/mlandauer). +* Fix: don't redefine Object#hash in Yahoo result object (thanks github.com/m0thman). + +== 1.1.4 (2012 Oct 2) + +* Deprecate Geocoder::Result::Nominatim#class and #type methods. Use #place_class and #place_type instead. +* Add support for setting arbitrary parameters in geocoding request URL. * Add support for Google's :bounds parameter (thanks to github.com/rosscooperman and github.com/peterjm for submitting suggestions). -* Code refactoring and cleanup (most notably, added Geocoder::Query class). +* Add support for :select => :geo_only option to near scope (thanks github.com/gugl). +* Add ability to omit ORDER BY clause from .near scope (pass option :order => false). +* Fix: error on Yahoo lookup due to API change (thanks github.com/kynesun). +* Fix: problem with Mongoid field aliases not being respected. +* Fix: :exclude option to .near scope when primary key != :id (thanks github.com/smisml). +* Much code refactoring (added Geocoder::Query class and Geocoder::Sql module). == 1.1.3 (2012 Aug 26) diff --git a/Gemfile b/Gemfile index b114997e63766aaffb45ad5066df769b59516390..85c71ce03310e9930678f9c4b5bd51b719aa3a9e 100644 --- a/Gemfile +++ b/Gemfile @@ -1,10 +1,8 @@ source "http://rubygems.org" -gemspec - group :development, :test do gem 'rake' - gem 'mongoid' + gem 'mongoid', '3.0.13' gem 'bson_ext', :platforms => :ruby gem 'rails' @@ -13,3 +11,5 @@ group :development, :test do gem 'jruby-openssl' end end + +gemspec diff --git a/README.md b/README.md index b2e6b739e93936b42c0316e399bd7873bdcbf718..1cabb3549ae18830f9630c733006bd13c2e9aea7 100644 --- a/README.md +++ b/README.md @@ -81,6 +81,10 @@ Reverse geocoding is similar: reverse_geocoded_by :coordinates after_validation :reverse_geocode # auto-fetch address +Once you've set up your model you'll need to create the necessary spatial indices in your database: + + rake db:mongoid:create_indexes + Be sure to read _Latitude/Longitude Order_ in the _Notes on MongoDB_ section below on how to properly retrieve latitude/longitude coordinates from your objects. ### MongoMapper @@ -126,9 +130,11 @@ To find objects by location, use the following scopes: With geocoded objects you can do things like this: - obj.nearbys(30) # other objects within 30 miles - obj.distance_from([40.714,-100.234]) # distance from arbitrary point to object - obj.bearing_to("Paris, France") # direction from object to arbitrary point + if obj.geocoded? + obj.nearbys(30) # other objects within 30 miles + obj.distance_from([40.714,-100.234]) # distance from arbitrary point to object + obj.bearing_to("Paris, France") # direction from object to arbitrary point + end Some utility methods are also available: @@ -263,7 +269,7 @@ By default Geocoder uses Google's geocoding API to fetch coordinates and street Geocoder.configure do |config| # geocoding service (see below for supported options): - config.lookup = :yahoo + config.lookup = :yandex # to use an API key: config.api_key = "..." @@ -304,17 +310,19 @@ The following is a comparison of the supported geocoding APIs. The "Limitations" * **Limitations**: "You must not use or display the Content without a corresponding Google map, unless you are explicitly permitted to do so in the Maps APIs Documentation, or through written permission from Google." "You must not pre-fetch, cache, or store any Content, except that you may store: (i) limited amounts of Content for the purpose of improving the performance of your Maps API Implementation..." * **Notes**: To use Google Premier set `Geocoder::Configuration.lookup = :google_premier` and `Geocoder::Configuration.api_key = [key, client, channel]`. -#### Yahoo (`:yahoo`) +#### Yahoo BOSS (`:yahoo`) + +Yahoo BOSS is **not a free service**. As of November 17, 2012 Yahoo no longer offers a free geocoding API. -* **API key**: optional in development (required for production apps) -* **Key signup**: https://developer.apps.yahoo.com/wsregapp -* **Quota**: 50,000 requests/day, more available by special arrangement +* **API key**: requires OAuth consumer key and secret (set `Geocoder::Configuration.api_key = [key, secret]`) +* **Key signup**: http://developer.yahoo.com/boss/geo/ +* **Quota**: unlimited, but subject to usage fees * **Region**: world * **SSL support**: no -* **Languages**: ? -* **Documentation**: http://developer.yahoo.com/geo/placefinder/guide/responses.html -* **Terms of Service**: http://info.yahoo.com/legal/us/yahoo/maps/mapsapi/mapsapi-2141.html -* **Limitations**: "YOU SHALL NOT... (viii) store or allow end users to store map imagery, map data or geocoded location information from the Yahoo! Maps APIs for any future use; (ix) use the stand-alone geocoder for any use other than displaying Yahoo! Maps or displaying points on Yahoo! Maps;" +* **Languages**: en, fr, de, it, es, pt, nl, zh, ja, ko +* **Documentation**: http://developer.yahoo.com/boss/geo/docs/index.html +* **Terms of Service**: http://info.yahoo.com/legal/us/yahoo/boss/tou/?pir=ucJPcJ1ibUn.h.d.lVmlcbcEkoHjwJ_PvxG9SLK9VIbIQAw1XFrnDqY- +* **Limitations**: No mass downloads, no commercial map production based on the data, no storage of data except for caching. #### Bing (`:bing`) @@ -363,8 +371,11 @@ The following is a comparison of the supported geocoding APIs. The "Limitations" #### Mapquest (`:mapquest`) -* **API key**: none +* **API key**: required for the licensed API, do not use for open tier * **Quota**: ? +* **HTTP Headers**: in order to use the licensed API you can configure the http_headers to include a referer as so: + `Geocoder::Configuration.http_headers = { "Referer" => "http://foo.com" }` + You can also allow a blank referer from the API management console via mapquest but it is potentially a security risk that someone else could use your API key from another domain. * **Region**: world * **SSL support**: no * **Languages**: English @@ -502,7 +513,6 @@ When you install the Geocoder gem it adds a `geocode` command to your shell. You There are also a number of options for setting the geocoding API, key, and language, viewing the raw JSON reponse, and more. Please run `geocode -h` for details. - Notes on MongoDB ---------------- @@ -524,6 +534,22 @@ Calling `obj.coordinates` directly returns the internal representation of the co For consistency with the rest of Geocoder, always use the `to_coordinates` method instead. +Notes on Non-Rails Frameworks +----------------------------- + +If you are using Geocoder with ActiveRecord and a framework other than Rails (like Sinatra or Padrino) you will need to add this in your model before calling Geocoder methods: + + extend Geocoder::Model::ActiveRecord + +Optimisation of Distance Queries +-------------------------------- + +In MySQL and Postgres the finding of objects near a given point is speeded up by using a bounding box to limit the number of points over which a full distance calculation needs to be done. + +To take advantage of this optimisation you need to add a composite index on latitude and longitude. In your Rails migration: + + add_index :table, [:latitude, :longitude] + Distance Queries in SQLite -------------------------- @@ -566,6 +592,26 @@ You can also do this to raise all exceptions: See `lib/geocoder/exceptions.rb` for a list of raise-able exceptions. +Troubleshooting +--------------- + +### Mongoid + +If you get one of these errors: + + uninitialized constant Geocoder::Model::Mongoid + uninitialized constant Geocoder::Model::Mongoid::Mongo + +you should check your Gemfile to make sure the Mongoid gem is listed _before_ Geocoder. If Mongoid isn't loaded when Geocoder is initialized, Geocoder will not load support for Mongoid. + +### ActiveRecord + +A lot of debugging time can be saved by understanding how Geocoder works with ActiveRecord. When you use the `near` scope or the `nearbys` method of a geocoded object, Geocoder creates an ActiveModel::Relation object which adds some attributes (eg: distance, bearing) to the SELECT clause. It also adds a condition to the WHERE clause to check that distance is within the given radius. Because the SELECT clause is modified, anything else that modifies the SELECT clause may produce strange results, for example: + +* using the `pluck` method (selects only a single column) +* specifying another model through `includes` (selects columns from other tables) + + Known Issue ----------- diff --git a/lib/geocoder/configuration.rb b/lib/geocoder/configuration.rb index c2df6d3eb2817afc7d29eaa7febde5df061e3403..dd426e0b1b7ad1aa1e74fe2b59a51fb4590e89b6 100644 --- a/lib/geocoder/configuration.rb +++ b/lib/geocoder/configuration.rb @@ -23,7 +23,7 @@ module Geocoder # # Geocoder.configure do |config| # config.timeout = 5 - # config.lookup = :yahoo + # config.lookup = :yandex # config.api_key = "2a9fsa983jaslfj982fjasd" # config.units = :km # end diff --git a/lib/geocoder/lookup.rb b/lib/geocoder/lookup.rb index 11d0a8a20d5fac110f3516c2791479aced6b176b..4e58888d09ab9f46a4e5287883a4a23075f92e4e 100644 --- a/lib/geocoder/lookup.rb +++ b/lib/geocoder/lookup.rb @@ -1,31 +1,42 @@ module Geocoder module Lookup + extend self ## # Array of valid Lookup service names. # - def self.all_services + def all_services street_services + ip_services end ## # Array of valid Lookup service names, excluding :test. # - def self.all_services_except_test + def all_services_except_test all_services - [:test] end ## # All street address lookup services, default first. # - def self.street_services - [:google, :google_premier, :yahoo, :bing, :geocoder_ca, :yandex, :nominatim, :mapquest, :test] + def street_services + [ + :google, + :google_premier, + :yahoo, + :bing, + :geocoder_ca, + :yandex, + :nominatim, + :mapquest, + :test + ] end ## # All IP address lookup services, default first. # - def self.ip_services + def ip_services [:freegeoip] end @@ -34,7 +45,7 @@ module Geocoder # Use this instead of Geocoder::Lookup::X.new to get an # already-configured Lookup object. # - def self.get(name) + def get(name) @services = {} unless defined?(@services) @services[name] = spawn(name) unless @services.include?(name) @services[name] @@ -46,17 +57,25 @@ module Geocoder ## # Spawn a Lookup of the given name. # - def self.spawn(name) + def spawn(name) if all_services.include?(name) - name = name.to_s - require "geocoder/lookups/#{name}" - klass = name.split("_").map{ |i| i[0...1].upcase + i[1..-1] }.join - Geocoder::Lookup.const_get(klass).new + Geocoder::Lookup.const_get(classify_name(name)).new else valids = all_services.map(&:inspect).join(", ") raise ConfigurationError, "Please specify a valid lookup for Geocoder " + "(#{name.inspect} is not one of: #{valids})." end end + + ## + # Convert an "underscore" version of a name into a "class" version. + # + def classify_name(filename) + filename.to_s.split("_").map{ |i| i[0...1].upcase + i[1..-1] }.join + end end end + +Geocoder::Lookup.all_services.each do |name| + require "geocoder/lookups/#{name}" +end diff --git a/lib/geocoder/lookups/base.rb b/lib/geocoder/lookups/base.rb index 3ef2d2d19454d0614976df10affd7c26b852de7f..db42313b35ba3a9ae22420746f6d7bb82673af1a 100644 --- a/lib/geocoder/lookups/base.rb +++ b/lib/geocoder/lookups/base.rb @@ -16,6 +16,13 @@ module Geocoder class Base + ## + # Human-readable name of the geocoding API. + # + def name + fail + end + ## # Query the geocoding API and return a Geocoder::Result object. # Returns +nil+ on timeout or error. @@ -43,6 +50,14 @@ module Geocoder nil end + ## + # Array containing string descriptions of keys required by the API. + # Empty array if keys are optional or not required. + # + def required_api_key_parts + [] + end + private # ------------------------------------------------------------- @@ -97,6 +112,16 @@ module Geocoder fail end + ## + # Key to use for caching a geocoding result. Usually this will be the + # request URL, but in cases where OAuth is used and the nonce, + # timestamp, etc varies from one request to another, we need to use + # something else (like the URL before OAuth encoding). + # + def cache_key(query) + query_url(query) + end + ## # Class of the result objects # @@ -151,25 +176,45 @@ module Geocoder end ## - # Fetches a raw search result (JSON string). + # Fetch a raw geocoding result (JSON string). + # The result might or might not be cached. # def fetch_raw_data(query) - timeout(configuration.timeout) do - url = query_url(query) - uri = URI.parse(url) - if cache and body = cache[url] - @cache_hit = true - else - client = http_client.new(uri.host, uri.port) - client.use_ssl = true if configuration.use_https - response = client.get(uri.request_uri, configuration.http_headers) - body = response.body - if cache and (200..399).include?(response.code.to_i) - cache[url] = body - end - @cache_hit = false + key = cache_key(query) + if cache and body = cache[key] + @cache_hit = true + else + check_api_key_configuration!(query) + response = make_api_request(query) + body = response.body + if cache and (200..399).include?(response.code.to_i) + cache[key] = body end - body + @cache_hit = false + end + body + end + + ## + # Make an HTTP(S) request to a geocoding API and + # return the response object. + # + def make_api_request(query) + timeout(configuration.timeout) do + uri = URI.parse(query_url(query)) + client = http_client.new(uri.host, uri.port) + client.use_ssl = true if Geocoder::Configuration.use_https + client.get(uri.request_uri, Geocoder::Configuration.http_headers) + end + end + + def check_api_key_configuration!(query) + key_parts = query.lookup.required_api_key_parts + if key_parts.size > Array(Geocoder::Configuration.api_key).size + parts_string = key_parts.size == 1 ? key_parts.first : key_parts + raise Geocoder::ConfigurationError, + "The #{query.lookup.name} API requires a key to be configured: " + + parts_string.inspect end end diff --git a/lib/geocoder/lookups/bing.rb b/lib/geocoder/lookups/bing.rb index b89d820012d1368b86ac6967eabe2b079782cd22..c4ce5453a674a83d04aacbeb70c8a4a4c3d2f1a3 100644 --- a/lib/geocoder/lookups/bing.rb +++ b/lib/geocoder/lookups/bing.rb @@ -4,10 +4,18 @@ require "geocoder/results/bing" module Geocoder::Lookup class Bing < Base + def name + "Bing" + end + def map_link_url(coordinates) "http://www.bing.com/maps/default.aspx?cp=#{coordinates.join('~')}" end + def required_api_key_parts + ["key"] + end + private # --------------------------------------------------------------- def results(query) @@ -29,7 +37,7 @@ module Geocoder::Lookup end def query_url(query) - "http://dev.virtualearth.net/REST/v1/Locations" + + "#{protocol}://dev.virtualearth.net/REST/v1/Locations" + (query.reverse_geocode? ? "/#{query.sanitized_text}?" : "?") + url_query_string(query) end diff --git a/lib/geocoder/lookups/freegeoip.rb b/lib/geocoder/lookups/freegeoip.rb index 621d8ed638699b05bb695c370f8130cf22660151..b94946a60a9f046ccfff54788cbaeabd4311247b 100644 --- a/lib/geocoder/lookups/freegeoip.rb +++ b/lib/geocoder/lookups/freegeoip.rb @@ -4,6 +4,10 @@ require 'geocoder/results/freegeoip' module Geocoder::Lookup class Freegeoip < Base + def name + "FreeGeoIP" + end + private # --------------------------------------------------------------- def parse_raw_data(raw_data) @@ -37,7 +41,7 @@ module Geocoder::Lookup end def query_url(query) - "http://freegeoip.net/json/#{query.sanitized_text}" + "#{protocol}://freegeoip.net/json/#{query.sanitized_text}" end end end diff --git a/lib/geocoder/lookups/geocoder_ca.rb b/lib/geocoder/lookups/geocoder_ca.rb index 2355bbf863864b6ac367a86d0e4e293296e666a2..6c6bbf0f698d640e6b36397788372f65c2cb5a0a 100644 --- a/lib/geocoder/lookups/geocoder_ca.rb +++ b/lib/geocoder/lookups/geocoder_ca.rb @@ -4,6 +4,14 @@ require "geocoder/results/geocoder_ca" module Geocoder::Lookup class GeocoderCa < Base + def name + "Geocoder.ca" + end + + def required_api_key_parts + ["key"] + end + private # --------------------------------------------------------------- def results(query) @@ -39,7 +47,7 @@ module Geocoder::Lookup end def query_url(query) - "http://geocoder.ca/?" + url_query_string(query) + "#{protocol}://geocoder.ca/?" + url_query_string(query) end def parse_raw_data(raw_data) diff --git a/lib/geocoder/lookups/google.rb b/lib/geocoder/lookups/google.rb index e2a19cee4c2e249b137a9702dc3486818f2c3c03..43adc15cf7241a0e41ea0c59d2b018d131b32f56 100644 --- a/lib/geocoder/lookups/google.rb +++ b/lib/geocoder/lookups/google.rb @@ -4,6 +4,10 @@ require "geocoder/results/google" module Geocoder::Lookup class Google < Base + def name + "Google" + end + def map_link_url(coordinates) "http://maps.google.com/maps?q=#{coordinates.join(',')}" end diff --git a/lib/geocoder/lookups/google_premier.rb b/lib/geocoder/lookups/google_premier.rb index 90a6ef2833fe53b2052c539c299c22677fa276b9..2bf63dc9d3b5098372fbf7bd7d512f6e43fbc598 100644 --- a/lib/geocoder/lookups/google_premier.rb +++ b/lib/geocoder/lookups/google_premier.rb @@ -6,6 +6,14 @@ require 'geocoder/results/google_premier' module Geocoder::Lookup class GooglePremier < Google + def name + "Google Premier" + end + + def required_api_key_parts + ["private key", "client", "channel"] + end + private # --------------------------------------------------------------- def query_url_params(query) diff --git a/lib/geocoder/lookups/mapquest.rb b/lib/geocoder/lookups/mapquest.rb index 404bda22eef80bfc0e79593a39df3345f6ef5717..fd5ae8c643c0228158ed8e01bf8727f869ae5904 100644 --- a/lib/geocoder/lookups/mapquest.rb +++ b/lib/geocoder/lookups/mapquest.rb @@ -1,15 +1,44 @@ +require 'cgi' require 'geocoder/lookups/base' -require "geocoder/lookups/nominatim" require "geocoder/results/mapquest" module Geocoder::Lookup - class Mapquest < Nominatim + class Mapquest < Base + + def name + "Mapquest" + end + + def required_api_key_parts + ["key"] + end private # --------------------------------------------------------------- def query_url(query) - method = query.reverse_geocode? ? "reverse" : "search" - "http://open.mapquestapi.com/#{method}?" + url_query_string(query) + key = Geocoder::Configuration.api_key + domain = key ? "www" : "open" + url = "#{protocol}://#{domain}.mapquestapi.com/geocoding/v1/#{search_type(query)}?" + url + url_query_string(query) + end + + def search_type(query) + query.reverse_geocode? ? "reverse" : "address" end + + def query_url_params(query) + key = Geocoder::Configuration.api_key + params = { :location => query.sanitized_text } + if key + params[:key] = CGI.unescape(key) + end + super.merge(params) + end + + def results(query) + return [] unless doc = fetch_data(query) + doc["results"][0]['locations'] + end + end end diff --git a/lib/geocoder/lookups/nominatim.rb b/lib/geocoder/lookups/nominatim.rb index 803dd0b3b190b7d296286992df13e135059f192a..b5275080491910fcd8ef992de6a58007a0014929 100644 --- a/lib/geocoder/lookups/nominatim.rb +++ b/lib/geocoder/lookups/nominatim.rb @@ -4,6 +4,10 @@ require "geocoder/results/nominatim" module Geocoder::Lookup class Nominatim < Base + def name + "Nominatim" + end + def map_link_url(coordinates) "http://www.openstreetmap.org/?lat=#{coordinates[0]}&lon=#{coordinates[1]}&zoom=15&layers=M" end @@ -34,7 +38,7 @@ module Geocoder::Lookup def query_url(query) method = query.reverse_geocode? ? "reverse" : "search" - "http://nominatim.openstreetmap.org/#{method}?" + url_query_string(query) + "#{protocol}://nominatim.openstreetmap.org/#{method}?" + url_query_string(query) end end end diff --git a/lib/geocoder/lookups/test.rb b/lib/geocoder/lookups/test.rb index 89aa97f4445d7d65c75caea6b0fe6d67c3d75dbe..ebf7b5d76a2d201aeb39599826d0b1100ce22fd0 100644 --- a/lib/geocoder/lookups/test.rb +++ b/lib/geocoder/lookups/test.rb @@ -5,6 +5,10 @@ module Geocoder module Lookup class Test < Base + def name + "Test" + end + def self.add_stub(query_text, results) stubs[query_text] = results end diff --git a/lib/geocoder/lookups/yahoo.rb b/lib/geocoder/lookups/yahoo.rb index b42f399551796b1814a19f98b41a445ecd641344..c34765a401e32274385bbfe21f36e0c7ee218493 100644 --- a/lib/geocoder/lookups/yahoo.rb +++ b/lib/geocoder/lookups/yahoo.rb @@ -1,21 +1,35 @@ require 'geocoder/lookups/base' require "geocoder/results/yahoo" +require 'oauth_util' module Geocoder::Lookup class Yahoo < Base + def name + "Yahoo BOSS" + end + def map_link_url(coordinates) "http://maps.yahoo.com/#lat=#{coordinates[0]}&lon=#{coordinates[1]}" end + def required_api_key_parts + ["consumer key", "consumer secret"] + end + private # --------------------------------------------------------------- def results(query) return [] unless doc = fetch_data(query) - if doc = doc['ResultSet'] and doc['Error'] == 0 - return doc['Found'] > 0 ? doc['Results'] : [] + doc = doc['bossresponse'] + if doc['responsecode'].to_i == 200 + if doc['placefinder']['count'].to_i > 0 + return doc['placefinder']['results'] + else + return [] + end else - warn "Yahoo Geocoding API error: #{doc['Error']} (#{doc['ErrorMessage']})." + warn "Yahoo Geocoding API error: #{doc['responsecode']} (#{doc['reason']})." return [] end end @@ -30,8 +44,24 @@ module Geocoder::Lookup ) end + def cache_key(query) + raw_url(query) + end + + def base_url + "#{protocol}://yboss.yahooapis.com/geo/placefinder?" + end + + def raw_url(query) + base_url + url_query_string(query) + end + def query_url(query) - "http://where.yahooapis.com/geocode?" + url_query_string(query) + parsed_url = URI.parse(raw_url(query)) + o = OauthUtil.new + o.consumer_key = Geocoder::Configuration.api_key[0] + o.consumer_secret = Geocoder::Configuration.api_key[1] + base_url + o.sign(parsed_url).query_string end end end diff --git a/lib/geocoder/lookups/yandex.rb b/lib/geocoder/lookups/yandex.rb index 382d71971828e4b9585891c15f9d275e4d7ac310..4b306e1ef68d0eb596283c43083a60aec5ffda74 100644 --- a/lib/geocoder/lookups/yandex.rb +++ b/lib/geocoder/lookups/yandex.rb @@ -4,10 +4,18 @@ require "geocoder/results/yandex" module Geocoder::Lookup class Yandex < Base + def name + "Yandex" + end + def map_link_url(coordinates) "http://maps.yandex.ru/?ll=#{coordinates.reverse.join(',')}" end + def required_api_key_parts + ["key"] + end + private # --------------------------------------------------------------- def results(query) @@ -40,7 +48,7 @@ module Geocoder::Lookup end def query_url(query) - "http://geocode-maps.yandex.ru/1.x/?" + url_query_string(query) + "#{protocol}://geocode-maps.yandex.ru/1.x/?" + url_query_string(query) end end end diff --git a/lib/geocoder/models/mongoid.rb b/lib/geocoder/models/mongoid.rb index 9e0b9fd832127dcf2439a06786d78658d7fc9e40..b1497327f32f583839502e73fca90ba7b7f75bc2 100644 --- a/lib/geocoder/models/mongoid.rb +++ b/lib/geocoder/models/mongoid.rb @@ -18,7 +18,7 @@ module Geocoder super(options) if options[:skip_index] == false # create 2d index - if (::Mongoid::VERSION >= "3") + if defined?(::Mongoid::VERSION) && ::Mongoid::VERSION >= "3" index({ geocoder_options[:coordinates].to_sym => '2d' }, {:min => -180, :max => 180}) else diff --git a/lib/geocoder/request.rb b/lib/geocoder/request.rb index 5f71d46bc86455f9be4b2771b545a16bf5384653..3e1dbc823a7d7d769a289f5dbed9adeadb982e07 100644 --- a/lib/geocoder/request.rb +++ b/lib/geocoder/request.rb @@ -5,7 +5,13 @@ module Geocoder def location unless defined?(@location) - @location = Geocoder.search(ip).first + if env.has_key?('HTTP_X_REAL_IP') + @location = Geocoder.search(env['HTTP_X_REAL_IP']).first + elsif env.has_key?('HTTP_X_FORWARDED_FOR') + @location = Geocoder.search(env['HTTP_X_FORWARDED_FOR']).first + else + @location = Geocoder.search(ip).first + end end @location end diff --git a/lib/geocoder/results/base.rb b/lib/geocoder/results/base.rb index 8a42413f229dff258629fb64785518eee212f3e2..a798ced6852dea49b1eaa8f23fe75298bd07021d 100644 --- a/lib/geocoder/results/base.rb +++ b/lib/geocoder/results/base.rb @@ -1,10 +1,16 @@ module Geocoder module Result class Base - attr_accessor :data, :cache_hit + + # data (hash) fetched from geocoding service + attr_accessor :data + + # true if result came from cache, false if from request to geocoding + # service; nil if cache is not configured + attr_accessor :cache_hit ## - # Takes a hash of result data from a parsed Google result document. + # Takes a hash of data from a parsed geocoding service response. # def initialize(data) @data = data diff --git a/lib/geocoder/results/google.rb b/lib/geocoder/results/google.rb index 73e2289db58c0027b006f28d4520107e753e10b1..e293c0ba12b6bea6683ec43ec7a7ff0af3c6f956 100644 --- a/lib/geocoder/results/google.rb +++ b/lib/geocoder/results/google.rb @@ -53,6 +53,12 @@ module Geocoder::Result end end + def route + if route = address_components_of_type(:route).first + route['long_name'] + end + end + def types @data['types'] end diff --git a/lib/geocoder/results/mapquest.rb b/lib/geocoder/results/mapquest.rb index f977a6b5f5afb5e26aa8303e29723140daaae04b..354d8e61e516edc0107beaf6553711470a306fe2 100644 --- a/lib/geocoder/results/mapquest.rb +++ b/lib/geocoder/results/mapquest.rb @@ -1,7 +1,51 @@ require 'geocoder/results/base' -require 'geocoder/results/nominatim' module Geocoder::Result - class Mapquest < Nominatim + class Mapquest < Base + def latitude + @data["latLng"]["lat"] + end + + def longitude + @data["latLng"]["lng"] + end + + def coordinates + [latitude, longitude] + end + + def city + @data['adminArea5'] + end + + def street + @data['street'] + end + + def state + @data['adminArea3'] + end + + alias_method :state_code, :state + + #FIXME: these might not be right, unclear with MQ documentation + alias_method :provinice, :state + alias_method :province_code, :state + + def postal_code + @data['postalCode'].to_s + end + + def country + @data['adminArea1'] + end + + def country_code + country + end + + def address + [street, city, state, postal_code, country].reject{|s| s.length == 0 }.join(", ") + end end end diff --git a/lib/geocoder/results/yahoo.rb b/lib/geocoder/results/yahoo.rb index fab775c545deb39f0826d1223d45d5310802ccef..cb0fed3369a02f71d081c103685f41ab7d01ccbc 100644 --- a/lib/geocoder/results/yahoo.rb +++ b/lib/geocoder/results/yahoo.rb @@ -31,17 +31,24 @@ module Geocoder::Result @data['postal'] end + def address_hash + @data['hash'] + end + def self.response_attributes %w[quality offsetlat offsetlon radius boundingbox name line1 line2 line3 line4 cross house street xstreet unittype unit + city state statecode country countrycode postal neighborhood county countycode level0 level1 level2 level3 level4 level0code level1code level2code timezone areacode uzip hash woeid woetype] end response_attributes.each do |a| - define_method a do - @data[a] + unless method_defined?(a) + define_method a do + @data[a] + end end end end diff --git a/lib/geocoder/sql.rb b/lib/geocoder/sql.rb index 69c146a9e2122419a4e513958df732db07edea1a..af0b4983a84cd6275b56da4de70fbe8514282a8e 100644 --- a/lib/geocoder/sql.rb +++ b/lib/geocoder/sql.rb @@ -11,12 +11,13 @@ module Geocoder # http://www.scribd.com/doc/2569355/Geo-Distance-Search-with-MySQL # def full_distance(latitude, longitude, lat_attr, lon_attr, options = {}) - earth = Geocoder::Calculations.earth_radius(options[:units] || :mi) + units = options[:units] || Geocoder::Configuration.units + earth = Geocoder::Calculations.earth_radius(units) "#{earth} * 2 * ASIN(SQRT(" + - "POWER(SIN((#{latitude} - #{lat_attr}) * PI() / 180 / 2), 2) + " + - "COS(#{latitude} * PI() / 180) * COS(#{lat_attr} * PI() / 180) * " + - "POWER(SIN((#{longitude} - #{lon_attr}) * PI() / 180 / 2), 2)" + + "POWER(SIN((#{latitude.to_f} - #{lat_attr}) * PI() / 180 / 2), 2) + " + + "COS(#{latitude.to_f} * PI() / 180) * COS(#{lat_attr} * PI() / 180) * " + + "POWER(SIN((#{longitude.to_f} - #{lon_attr}) * PI() / 180 / 2), 2)" + "))" end @@ -31,14 +32,15 @@ module Geocoder # are not intended for use in production! # def approx_distance(latitude, longitude, lat_attr, lon_attr, options = {}) - dx = Geocoder::Calculations.longitude_degree_distance(30, options[:units] || :mi) - dy = Geocoder::Calculations.latitude_degree_distance(options[:units] || :mi) + units = options[:units] || Geocoder::Configuration.units + dx = Geocoder::Calculations.longitude_degree_distance(30, units) + dy = Geocoder::Calculations.latitude_degree_distance(units) # sin of 45 degrees = average x or y component of vector factor = Math.sin(Math::PI / 4) - "(#{dy} * ABS(#{lat_attr} - #{latitude}) * #{factor}) + " + - "(#{dx} * ABS(#{lon_attr} - #{longitude}) * #{factor})" + "(#{dy} * ABS(#{lat_attr} - #{latitude.to_f}) * #{factor}) + " + + "(#{dx} * ABS(#{lon_attr} - #{longitude.to_f}) * #{factor})" end def within_bounding_box(sw_lat, sw_lng, ne_lat, ne_lng, lat_attr, lon_attr) @@ -61,23 +63,23 @@ module Geocoder # http://www.beginningspatial.com/calculating_bearing_one_point_another # def full_bearing(latitude, longitude, lat_attr, lon_attr, options = {}) - case options[:bearing] + case options[:bearing] || Geocoder::Configuration.distances when :linear "CAST(" + "DEGREES(ATAN2( " + - "RADIANS(#{lon_attr} - #{longitude}), " + - "RADIANS(#{lat_attr} - #{latitude})" + + "RADIANS(#{lon_attr} - #{longitude.to_f}), " + + "RADIANS(#{lat_attr} - #{latitude.to_f})" + ")) + 360 " + "AS decimal) % 360" when :spherical "CAST(" + "DEGREES(ATAN2( " + - "SIN(RADIANS(#{lon_attr} - #{longitude})) * " + + "SIN(RADIANS(#{lon_attr} - #{longitude.to_f})) * " + "COS(RADIANS(#{lat_attr})), (" + - "COS(RADIANS(#{latitude})) * SIN(RADIANS(#{lat_attr}))" + + "COS(RADIANS(#{latitude.to_f})) * SIN(RADIANS(#{lat_attr}))" + ") - (" + - "SIN(RADIANS(#{latitude})) * COS(RADIANS(#{lat_attr})) * " + - "COS(RADIANS(#{lon_attr} - #{longitude}))" + + "SIN(RADIANS(#{latitude.to_f})) * COS(RADIANS(#{lat_attr})) * " + + "COS(RADIANS(#{lon_attr} - #{longitude.to_f}))" + ")" + ")) + 360 " + "AS decimal) % 360" @@ -90,14 +92,14 @@ module Geocoder # def approx_bearing(latitude, longitude, lat_attr, lon_attr, options = {}) "CASE " + - "WHEN (#{lat_attr} >= #{latitude} AND " + - "#{lon_attr} >= #{longitude}) THEN 45.0 " + - "WHEN (#{lat_attr} < #{latitude} AND " + - "#{lon_attr} >= #{longitude}) THEN 135.0 " + - "WHEN (#{lat_attr} < #{latitude} AND " + - "#{lon_attr} < #{longitude}) THEN 225.0 " + - "WHEN (#{lat_attr} >= #{latitude} AND " + - "#{lon_attr} < #{longitude}) THEN 315.0 " + + "WHEN (#{lat_attr} >= #{latitude.to_f} AND " + + "#{lon_attr} >= #{longitude.to_f}) THEN 45.0 " + + "WHEN (#{lat_attr} < #{latitude.to_f} AND " + + "#{lon_attr} >= #{longitude.to_f}) THEN 135.0 " + + "WHEN (#{lat_attr} < #{latitude.to_f} AND " + + "#{lon_attr} < #{longitude.to_f}) THEN 225.0 " + + "WHEN (#{lat_attr} >= #{latitude.to_f} AND " + + "#{lon_attr} < #{longitude.to_f}) THEN 315.0 " + "END" end end diff --git a/lib/geocoder/stores/active_record.rb b/lib/geocoder/stores/active_record.rb index 4d4acfd26d036e664dc8a6b004bfea0b2727cb30..b088089f0a445bbecb85a158c8da1798d9062a56 100644 --- a/lib/geocoder/stores/active_record.rb +++ b/lib/geocoder/stores/active_record.rb @@ -94,22 +94,26 @@ module Geocoder::Store # set to false for no bearing calculation. # See Geocoder::Configuration to know how configure default method. # * +:select+ - string with the SELECT SQL fragment (e.g. “id, nameâ€) - # * +:order+ - column(s) for ORDER BY SQL clause; default is distance + # * +:order+ - column(s) for ORDER BY SQL clause; default is distance; + # set to false or nil to omit the ORDER BY clause # * +:exclude+ - an object to exclude (used by the +nearbys+ method) # def near_scope_options(latitude, longitude, radius = 20, options = {}) options[:units] ||= (geocoder_options[:units] || Geocoder::Configuration.units) bearing = bearing_sql(latitude, longitude, options) distance = distance_sql(latitude, longitude, options) + + b = Geocoder::Calculations.bounding_box([latitude, longitude], radius, options) + args = b + [ + full_column_name(geocoder_options[:latitude]), + full_column_name(geocoder_options[:longitude]) + ] + bounding_box_conditions = Geocoder::Sql.within_bounding_box(*args) + if using_sqlite? - b = Geocoder::Calculations.bounding_box([latitude, longitude], radius, options) - args = b + [ - full_column_name(geocoder_options[:latitude]), - full_column_name(geocoder_options[:longitude]) - ] - conditions = Geocoder::Sql.within_bounding_box(*args) + conditions = bounding_box_conditions else - conditions = ["#{distance} <= ?", radius] + conditions = [bounding_box_conditions + " AND #{distance} <= ?", radius] end { :select => select_clause(options[:select], distance, bearing), @@ -156,8 +160,10 @@ module Geocoder::Store ## # Generate the SELECT clause. # - def select_clause(columns, distance, bearing = nil) - if columns == :geo_only + def select_clause(columns, distance = nil, bearing = nil) + if columns == :id_only + return full_column_name(primary_key) + elsif columns == :geo_only clause = "" else clause = (columns || full_column_name("*")) + ", " diff --git a/lib/geocoder/stores/base.rb b/lib/geocoder/stores/base.rb index a9d578f4f55ab521aa0074342f912bb694d9477f..bccdf008f4d97810f64ca151b8da06067d04f013 100644 --- a/lib/geocoder/stores/base.rb +++ b/lib/geocoder/stores/base.rb @@ -58,9 +58,10 @@ module Geocoder ## # Get nearby geocoded objects. # Takes the same options hash as the near class method (scope). + # Returns nil if the object is not geocoded. # def nearbys(radius = 20, options = {}) - return [] unless geocoded? + return nil unless geocoded? options.merge!(:exclude => self) self.class.near(self, radius, options) end diff --git a/lib/geocoder/stores/mongo_base.rb b/lib/geocoder/stores/mongo_base.rb index 13ae12fd02dc5eebd08378a14b28a0440e958c43..62a4d98f987f8a83c3a533a353c545ce2ebc0308 100644 --- a/lib/geocoder/stores/mongo_base.rb +++ b/lib/geocoder/stores/mongo_base.rb @@ -59,7 +59,7 @@ module Geocoder::Store do_lookup(false) do |o,rs| if r = rs.first unless r.coordinates.nil? - o.send :write_attribute, self.class.geocoder_options[:coordinates], r.coordinates.reverse + o.__send__ "#{self.class.geocoder_options[:coordinates]}=", r.coordinates.reverse end r.coordinates end @@ -74,7 +74,7 @@ module Geocoder::Store do_lookup(true) do |o,rs| if r = rs.first unless r.address.nil? - o.send :write_attribute, self.class.geocoder_options[:fetched_address], r.address + o.__send__ "#{self.class.geocoder_options[:fetched_address]}=", r.address end r.address end diff --git a/lib/geocoder/version.rb b/lib/geocoder/version.rb index c68b238138799e2f48624754ff23a20080c0524e..03c1136552da9346d8b8e7745d8271892d00c09a 100644 --- a/lib/geocoder/version.rb +++ b/lib/geocoder/version.rb @@ -1,3 +1,3 @@ module Geocoder - VERSION = "1.1.4" + VERSION = "1.1.5" end diff --git a/lib/oauth_util.rb b/lib/oauth_util.rb new file mode 100644 index 0000000000000000000000000000000000000000..a98cb686e7adf815c13195b3a65179deaec70571 --- /dev/null +++ b/lib/oauth_util.rb @@ -0,0 +1,106 @@ +# A utility for signing an url using OAuth in a way that's convenient for debugging +# Note: the standard Ruby OAuth lib is here http://github.com/mojodna/oauth +# Source: http://gist.github.com/383159 +# License: http://gist.github.com/375593 +# Usage: see example.rb below + +require 'uri' +require 'cgi' +require 'openssl' +require 'base64' + +class OauthUtil + + attr_accessor :consumer_key, :consumer_secret, :token, :token_secret, :req_method, + :sig_method, :oauth_version, :callback_url, :params, :req_url, :base_str + + def initialize + @consumer_key = '' + @consumer_secret = '' + @token = '' + @token_secret = '' + @req_method = 'GET' + @sig_method = 'HMAC-SHA1' + @oauth_version = '1.0' + @callback_url = '' + end + + # openssl::random_bytes returns non-word chars, which need to be removed. using alt method to get length + # ref http://snippets.dzone.com/posts/show/491 + def nonce + Array.new( 5 ) { rand(256) }.pack('C*').unpack('H*').first + end + + def percent_encode( string ) + + # ref http://snippets.dzone.com/posts/show/1260 + return URI.escape( string, Regexp.new("[^#{URI::PATTERN::UNRESERVED}]") ).gsub('*', '%2A') + end + + # @ref http://oauth.net/core/1.0/#rfc.section.9.2 + def signature + key = percent_encode( @consumer_secret ) + '&' + percent_encode( @token_secret ) + + # ref: http://blog.nathanielbibler.com/post/63031273/openssl-hmac-vs-ruby-hmac-benchmarks + digest = OpenSSL::Digest::Digest.new( 'sha1' ) + hmac = OpenSSL::HMAC.digest( digest, key, @base_str ) + + # ref http://groups.google.com/group/oauth-ruby/browse_thread/thread/9110ed8c8f3cae81 + Base64.encode64( hmac ).chomp.gsub( /\n/, '' ) + end + + # sort (very important as it affects the signature), concat, and percent encode + # @ref http://oauth.net/core/1.0/#rfc.section.9.1.1 + # @ref http://oauth.net/core/1.0/#9.2.1 + # @ref http://oauth.net/core/1.0/#rfc.section.A.5.1 + def query_string + pairs = [] + @params.sort.each { | key, val | + pairs.push( "#{ percent_encode( key ) }=#{ percent_encode( val.to_s ) }" ) + } + pairs.join '&' + end + + # organize params & create signature + def sign( parsed_url ) + + @params = { + 'oauth_consumer_key' => @consumer_key, + 'oauth_nonce' => nonce, + 'oauth_signature_method' => @sig_method, + 'oauth_timestamp' => Time.now.to_i.to_s, + 'oauth_version' => @oauth_version + } + + # if url has query, merge key/values into params obj overwriting defaults + if parsed_url.query + #@params.merge! CGI.parse( parsed_url.query ) + CGI.parse( parsed_url.query ).each do |k,v| + if v.is_a?(Array) && v.count == 1 + @params[k] = v.first + else + @params[k] = v + end + end + end + + # @ref http://oauth.net/core/1.0/#rfc.section.9.1.2 + @req_url = parsed_url.scheme + '://' + parsed_url.host + parsed_url.path + + # create base str. make it an object attr for ez debugging + # ref http://oauth.net/core/1.0/#anchor14 + @base_str = [ + @req_method, + percent_encode( req_url ), + + # normalization is just x-www-form-urlencoded + percent_encode( query_string ) + + ].join( '&' ) + + # add signature + @params[ 'oauth_signature' ] = signature + + return self + end +end diff --git a/test/cache_test.rb b/test/cache_test.rb new file mode 100644 index 0000000000000000000000000000000000000000..f505b805db0863bbc9c6fc8168b7cb6457b0785a --- /dev/null +++ b/test/cache_test.rb @@ -0,0 +1,19 @@ +# encoding: utf-8 +require 'test_helper' + +class CacheTest < Test::Unit::TestCase + + def test_second_occurrence_of_request_is_cache_hit + Geocoder::Configuration.cache = {} + Geocoder::Lookup.all_services_except_test.each do |l| + Geocoder::Configuration.lookup = l + set_api_key!(l) + results = Geocoder.search("Madison Square Garden") + assert !results.first.cache_hit, + "Lookup #{l} returned erroneously cached result." + results = Geocoder.search("Madison Square Garden") + assert results.first.cache_hit, + "Lookup #{l} did not return cached result." + end + end +end diff --git a/test/error_handling_test.rb b/test/error_handling_test.rb index 369613e69540d3c33a039fd8ccf35800ee2445a3..01f5f73305a22c39954bd2e984d427496229adc5 100644 --- a/test/error_handling_test.rb +++ b/test/error_handling_test.rb @@ -12,8 +12,10 @@ class ErrorHandlingTest < Test::Unit::TestCase orig = $VERBOSE; $VERBOSE = nil Geocoder::Lookup.all_services_except_test.each do |l| Geocoder::Configuration.lookup = l + set_api_key!(l) assert_nothing_raised { Geocoder.search("timeout") } end + ensure $VERBOSE = orig end @@ -21,6 +23,7 @@ class ErrorHandlingTest < Test::Unit::TestCase Geocoder::Configuration.always_raise = [TimeoutError] Geocoder::Lookup.all_services_except_test.each do |l| lookup = Geocoder::Lookup.get(l) + set_api_key!(l) assert_raises TimeoutError do lookup.send(:results, Geocoder::Query.new("timeout")) end @@ -31,6 +34,7 @@ class ErrorHandlingTest < Test::Unit::TestCase Geocoder::Configuration.always_raise = [SocketError] Geocoder::Lookup.all_services_except_test.each do |l| lookup = Geocoder::Lookup.get(l) + set_api_key!(l) assert_raises SocketError do lookup.send(:results, Geocoder::Query.new("socket_error")) end diff --git a/test/fixtures/mapquest_madison_square_garden.json b/test/fixtures/mapquest_madison_square_garden.json index 1c8d3b31746fefaf645edadcf8cc805780e8607e..86c135bac459db715783f28a07b793cc8820ff19 100644 --- a/test/fixtures/mapquest_madison_square_garden.json +++ b/test/fixtures/mapquest_madison_square_garden.json @@ -1,27 +1,52 @@ -[ - { - "place_id":"2177656031", - "licence":"Data Copyright OpenStreetMap Contributors, Some Rights Reserved. CC-BY-SA 2.0.", - "osm_type":"way", - "osm_id":"138141251", - "boundingbox":["40.7498588562012","40.751163482666","-73.9944381713867","-73.9925842285156"], - "polygonpoints":[["-73.9944367","40.7505417"],["-73.9940278","40.7511034"],["-73.9939442","40.7510658"],["-73.9938776","40.7510941"],["-73.9937734","40.7511298"],["-73.9936562","40.7511561"],["-73.993619","40.7511624"],["-73.9935537","40.7510862"],["-73.9935336","40.7510885"],["-73.9934248","40.7510898"],["-73.9933248","40.7510806"],["-73.9932268","40.7510614"],["-73.9931334","40.7510327"],["-73.9930378","40.7509909"],["-73.9929554","40.7509421"],["-73.9928865","40.7508886"],["-73.992821","40.7508216"],["-73.9927742","40.7507572"],["-73.9926591","40.7507581"],["-73.9926036","40.750603"],["-73.992704","40.7505536"],["-73.9927029","40.7505065"],["-73.9925855","40.7505009"],["-73.9925989","40.7503952"],["-73.9926442","40.7504003"],["-73.9926722","40.7503155"],["-73.9927117","40.7502402"],["-73.9927617","40.7501715"],["-73.992824","40.7501067"],["-73.9928991","40.7500484"],["-73.992869","40.7500159"],["-73.9929742","40.749956"],["-73.9930375","40.7500318"],["-73.9931229","40.7499938"],["-73.9931273","40.7499005"],["-73.9933201","40.7498624"],["-73.9933853","40.7499355"],["-73.993402","40.7499336"],["-73.9935038","40.749932"],["-73.9936041","40.7499407"],["-73.9936962","40.7499579"],["-73.9937875","40.7499846"],["-73.9938783","40.7500221"],["-73.9939639","40.7500701"],["-73.9940328","40.7501206"],["-73.9940991","40.7501842"],["-73.9941506","40.7502504"],["-73.9941562","40.7502603"],["-73.9942791","40.7502628"],["-73.9942969","40.7503035"],["-73.9943271","40.7503844"],["-73.9943435","40.7504689"],["-73.9943454","40.7505049"],["-73.9944367","40.7505417"]], - "lat":"40.7505206016777", - "lon":"-73.993490694181", - "display_name":"Madison Square Garden, 46, West 31st Street, Chelsea, New York City, New York, United States of America", - "class":"leisure", - "type":"stadium", - "address":{ - "stadium":"Madison Square Garden", - "house_number":"46", - "road":"West 31st Street", - "suburb":"Chelsea", - "city":"New York City", - "county":"New York", - "state":"New York", - "postcode":"10119", - "country":"United States of America", - "country_code":"us" +{ + "results": [ + { + "locations": [ + { + "latLng": { + "lng": -73.994637, + "lat": 40.720409 + }, + "adminArea4": "New York County", + "adminArea5Type": "City", + "adminArea4Type": "County", + "adminArea5": "New York", + "street": "46 West 31st Street", + "adminArea1": "US", + "adminArea3": "NY", + "type": "s", + "displayLatLng": { + "lng": -73.994637, + "lat": 40.720409 + }, + "linkId": 0, + "postalCode": "10001", + "sideOfStreet": "N", + "dragPoint": false, + "adminArea1Type": "Country", + "geocodeQuality": "CITY", + "geocodeQualityCode": "A5XAX", + "mapUrl": "http://www.mapquestapi.com/staticmap/v3/getmap?type=map&size=225,160&pois=purple-1,40.720409,-73.994637,0,0|¢er=40.720409,-73.994637&zoom=9&key=Gmjtd|luua2hu2nd,7x=o5-lz8lg&rand=604519389", + "adminArea3Type": "State" + } + ], + "providedLocation": { + "location": "Madison Square Garden, New York, NY" + } } + ], + "options": { + "ignoreLatLngInput": false, + "maxResults": -1, + "thumbMaps": true + }, + "info": { + "copyright": { + "text": "© 2012 MapQuest, Inc.", + "imageUrl": "http://api.mqcdn.com/res/mqlogo.gif", + "imageAltText": "© 2012 MapQuest, Inc." + }, + "statuscode": 0, + "messages": [] } -] +} diff --git a/test/fixtures/mapquest_no_results.json b/test/fixtures/mapquest_no_results.json index fe51488c7066f6687ef680d6bfaa4f7768ef205c..0cfc3e76e385c41ecb2f9cfcea589e3f44ba38ab 100644 --- a/test/fixtures/mapquest_no_results.json +++ b/test/fixtures/mapquest_no_results.json @@ -1 +1,7 @@ -[] +{ + "results": [ + { + "locations": [] + } + ] +} diff --git a/test/fixtures/yahoo_error.json b/test/fixtures/yahoo_error.json new file mode 100644 index 0000000000000000000000000000000000000000..7dbfdfeabeaa611d00af6ef756118795619703a1 --- /dev/null +++ b/test/fixtures/yahoo_error.json @@ -0,0 +1 @@ +{"bossresponse":{"responsecode":"6000","reason":"internal error"}} diff --git a/test/fixtures/yahoo_garbage.json b/test/fixtures/yahoo_garbage.json deleted file mode 100644 index 8e8970fd26048791ff42b17e0543ff1c4beb0710..0000000000000000000000000000000000000000 --- a/test/fixtures/yahoo_garbage.json +++ /dev/null @@ -1,50 +0,0 @@ -{ - "ResultSet":{ - "version":"1.0", - "Error":0, - "ErrorMessage":"No error", - "Locale":"us_US", - "Quality":87, - "Found":1, - "Results":[{ - "quality":9, - "latitude":"55.008390", - "longitude":"-5.822485", - "offsetlat":"54.314072", - "offsetlon":"-2.230010", - "radius":1145100, - "boundingbox":{ - "north":"60.854691", - "south":"49.162090", - "east":"1.768960", - "west":"-13.413930" - }, - "name":"", - "line1":"", - "line2":"", - "line3":"", - "line4":"United Kingdom", - "cross":"", - "house":"", - "street":"", - "xstreet":"", - "unittype":"", - "unit":"", - "postal":"", - "neighborhood":"", - "city":"", - "county":"", - "state":"", - "country":"United Kingdom", - "countrycode":"GB", - "statecode":"", - "countycode":"", - "timezone":"Europe/London", - "areacode":"", - "uzip":"", - "hash":"", - "woeid":23424975, - "woetype":12 - }] - } -} diff --git a/test/fixtures/yahoo_madison_square_garden.json b/test/fixtures/yahoo_madison_square_garden.json index 6e54b8e7269281ca7dcf02663361b158f6edd7eb..24161f60b57f58f99c68ec9e99f648c89ee08097 100644 --- a/test/fixtures/yahoo_madison_square_garden.json +++ b/test/fixtures/yahoo_madison_square_garden.json @@ -1,46 +1,52 @@ { - "ResultSet":{ - "version":"1.0", - "Error":0, - "ErrorMessage":"No error", - "Locale":"us_US", - "Quality":90, - "Found":1, - "Results":[{ - "quality":90, - "latitude":"40.750381", - "longitude":"-73.993988", - "offsetlat":"40.750381", - "offsetlon":"-73.993988", - "radius":100, - "name":"Madison Square Garden", - "line1":"Madison Square Garden", - "line2":"New York, NY 10001", - "line3":"", - "line4":"United States", - "house":"", - "street":"", - "xstreet":"", - "unittype":"", - "unit":"", - "postal":"10001", - "neighborhood":"", - "city":"New York", - "county":"New York County", - "state":"New York", - "country":"United States", - "countrycode":"US", - "statecode":"NY", - "countycode":"", - "uzip":"10001", - "hash":"", - "woeid":23617041, - "woetype":20, - "cross":"", - "timezone":"America/New_York", - "neighborhood":"Garment District|Midtown|Midtown West|Manhattan", - "areacode":"212", - "boundingbox":{"north":"40.750832","south":"40.749931","east":"-73.993393","west":"-73.994591"} - }] + "bossresponse": { + "responsecode": "200", + "placefinder": { + "start": "0", + "count": "1", + "request": "flags=JXTSR&location=Madison%20Square%20Garden%2C%20NY%2C%20NY&%unsafe%appid=%5B%22dj0yJmk9ZmZ5NXFrNGhNcEthJmQ9WVdrOVFUSlhPV2x1TjJVbWNHbzlORE0wT0RFME9UWXkmcz1jb25zdW1lcnNlY3JldCZ4PTAy%22%2C%20%22b57b1b98eb21f171231f5b441cba505261d6c9bb%22%5D&gflags=AC&locale=en_US", + "results": [ + { + "quality": "90", + "latitude": "40.750381", + "longitude": "-73.993988", + "offsetlat": "40.750381", + "offsetlon": "-73.993988", + "radius": "400", + "boundingbox": { + "north": "40.750832", + "south": "40.749931", + "east": "-73.993393", + "west": "-73.994591" + }, + "name": "Madison Square Garden", + "line1": "Madison Square Garden", + "line2": "New York, NY 10001", + "line3": "", + "line4": "United States", + "cross": "", + "house": "", + "street": "", + "xstreet": "", + "unittype": "", + "unit": "", + "postal": "10001", + "neighborhood": "Garment District|Midtown|Midtown West|Manhattan", + "city": "New York", + "county": "New York County", + "state": "New York", + "country": "United States", + "countrycode": "US", + "statecode": "NY", + "countycode": "", + "timezone": "America/New_York", + "areacode": "212", + "uzip": "10001", + "hash": "", + "woeid": "23617041", + "woetype": "20" + } + ] + } } } diff --git a/test/fixtures/yahoo_no_results.json b/test/fixtures/yahoo_no_results.json index e97865dfcf60fb4e970caac77d2187f3eadf640d..0c65fddc4a9dd4aa3791c7ec19ef75ba977837e8 100644 --- a/test/fixtures/yahoo_no_results.json +++ b/test/fixtures/yahoo_no_results.json @@ -1,10 +1,10 @@ { - "ResultSet":{ - "version":"1.0", - "Error":0, - "ErrorMessage":"No error", - "Locale":"us_US", - "Quality":10, - "Found":0 + "bossresponse": { + "responsecode": "200", + "placefinder": { + "start": "0", + "count": "0", + "request": "flags=JXTSR&location=asdfasdf28394782sdfj2983&%unsafe%appid=%5B%22dj0yJmk9ZmZ5NXFrNGhNcEthJmQ9WVdrOVFUSlhPV2x1TjJVbWNHbzlORE0wT0RFME9UWXkmcz1jb25zdW1lcnNlY3JldCZ4PTAy%22%2C%20%22b57b1b98eb21f171231f5b441cba505261d6c9bb%22%5D&gflags=AC&locale=en_US" + } } } diff --git a/test/lookup_test.rb b/test/lookup_test.rb index 5b4550c6c5045705395fb7bc687a6eda16bb7803..4c9a00478a879694992d7abf1b6f84a9086f75f6 100644 --- a/test/lookup_test.rb +++ b/test/lookup_test.rb @@ -6,6 +6,7 @@ class LookupTest < Test::Unit::TestCase def test_search_returns_empty_array_when_no_results Geocoder::Lookup.all_services_except_test.each do |l| lookup = Geocoder::Lookup.get(l) + set_api_key!(l) assert_equal [], lookup.send(:results, Geocoder::Query.new("no results")), "Lookup #{l} does not return empty array when no results." end @@ -29,16 +30,17 @@ class LookupTest < Test::Unit::TestCase assert_match "key=MY_KEY", g.send(:query_url, Geocoder::Query.new("Madison Square Garden, New York, NY 10001, United States")) end - def test_yahoo_app_id - Geocoder::Configuration.api_key = "MY_KEY" - g = Geocoder::Lookup::Yahoo.new - assert_match "appid=MY_KEY", g.send(:query_url, Geocoder::Query.new("Madison Square Garden, New York, NY 10001, United States")) - end - def test_geocoder_ca_showpostal Geocoder::Configuration.api_key = "MY_KEY" g = Geocoder::Lookup::GeocoderCa.new assert_match "showpostal=1", g.send(:query_url, Geocoder::Query.new("Madison Square Garden, New York, NY 10001, United States")) end + def test_raises_configuration_error_on_missing_key + assert_raises Geocoder::ConfigurationError do + Geocoder::Configuration.lookup = :bing + Geocoder::Configuration.api_key = nil + Geocoder.search("Madison Square Garden, New York, NY 10001, United States") + end + end end diff --git a/test/near_test.rb b/test/near_test.rb new file mode 100644 index 0000000000000000000000000000000000000000..359f2011423d0b38d3b83cbbb10eb2bfeed49abe --- /dev/null +++ b/test/near_test.rb @@ -0,0 +1,11 @@ +require 'test_helper' + +class NearTest < Test::Unit::TestCase + + def test_near_scope_options_without_sqlite_includes_bounding_box_condition + result = Event.send(:near_scope_options, 1.0, 2.0, 5) + + assert_match /test_table_name.latitude BETWEEN 0.9276\d* AND 1.0723\d* AND test_table_name.longitude BETWEEN 1.9276\d* AND 2.0723\d* AND /, + result[:conditions][0] + end +end diff --git a/test/query_test.rb b/test/query_test.rb index ee22e4a994c204f4dc3f6517ff83262fae967b45..52da81fc03c98bc8fe66899fc7f2ca1a32be08a5 100644 --- a/test/query_test.rb +++ b/test/query_test.rb @@ -23,6 +23,11 @@ class QueryTest < Test::Unit::TestCase assert !Geocoder::Query.new(nil, :params => {:woeid => 1234567}).blank? end + def test_blank_query_detection_for_coordinates + assert Geocoder::Query.new([nil,nil]).blank? + assert Geocoder::Query.new([87,nil]).blank? + end + def test_coordinates_detection assert Geocoder::Query.new("51.178844,5").coordinates? assert Geocoder::Query.new("51.178844, -1.826189").coordinates? diff --git a/test/result_test.rb b/test/result_test.rb index ab207c5fb6f0bd4834cf88b3764708d2db91ebee..f826648dd743cde17c0c2a8d29d87f0d39406029 100644 --- a/test/result_test.rb +++ b/test/result_test.rb @@ -6,6 +6,7 @@ class ResultTest < Test::Unit::TestCase def test_result_has_required_attributes Geocoder::Lookup.all_services_except_test.each do |l| Geocoder::Configuration.lookup = l + set_api_key!(l) result = Geocoder.search([45.423733, -75.676333]).first assert_result_has_required_attributes(result) end diff --git a/test/services_test.rb b/test/services_test.rb index 07cb694c728458b88f1ac04ab7654aa11bcf05ac..d33a648f16f6e8cab7a1eea80f3e19e825f5b2fb 100644 --- a/test/services_test.rb +++ b/test/services_test.rb @@ -6,12 +6,13 @@ class ServicesTest < Test::Unit::TestCase def test_query_url_contains_values_in_params_hash Geocoder::Lookup.all_services_except_test.each do |l| - next if l == :google_premier # TODO: need to set keys to test next if l == :freegeoip # does not use query string + set_api_key!(l) url = Geocoder::Lookup.get(l).send(:query_url, Geocoder::Query.new( "test", :params => {:one_in_the_hand => "two in the bush"} )) - assert_match /one_in_the_hand=two\+in\+the\+bush/, url, + # should be "+"s for all lookups except Yahoo + assert_match /one_in_the_hand=two(%20|\+)in(%20|\+)the(%20|\+)bush/, url, "Lookup #{l} does not appear to support arbitrary params in URL" end end @@ -24,6 +25,12 @@ class ServicesTest < Test::Unit::TestCase result.address_components_of_type(:sublocality).first['long_name'] end + def test_google_result_components_contains_route + result = Geocoder.search("Madison Square Garden, New York, NY").first + assert_equal "Penn Plaza", + result.address_components_of_type(:route).first['long_name'] + end + def test_google_returns_city_when_no_locality_in_result result = Geocoder.search("no locality").first assert_equal "Haram", result.city @@ -53,6 +60,7 @@ class ServicesTest < Test::Unit::TestCase def test_google_premier_result_components Geocoder::Configuration.lookup = :google_premier + set_api_key!(:google_premier) result = Geocoder.search("Madison Square Garden, New York, NY").first assert_equal "Manhattan", result.address_components_of_type(:sublocality).first['long_name'] @@ -67,17 +75,34 @@ class ServicesTest < Test::Unit::TestCase # --- Yahoo --- + def test_yahoo_no_results + Geocoder::Configuration.lookup = :yahoo + set_api_key!(:yahoo) + assert_equal [], Geocoder.search("no results") + end + + def test_yahoo_error + Geocoder::Configuration.lookup = :yahoo + set_api_key!(:yahoo) + # keep test output clean: suppress timeout warning + orig = $VERBOSE; $VERBOSE = nil + assert_equal [], Geocoder.search("error") + ensure + $VERBOSE = orig + end + def test_yahoo_result_components Geocoder::Configuration.lookup = :yahoo - result = Geocoder.search("Madison Square Garden, New York, NY").first + set_api_key!(:yahoo) + result = Geocoder.search("madison square garden").first assert_equal "10001", result.postal_code end def test_yahoo_address_formatting Geocoder::Configuration.lookup = :yahoo - result = Geocoder.search("Madison Square Garden, New York, NY").first - assert_equal "Madison Square Garden, New York, NY 10001, United States", - result.address + set_api_key!(:yahoo) + result = Geocoder.search("madison square garden").first + assert_equal "Madison Square Garden, New York, NY 10001, United States", result.address end @@ -87,7 +112,9 @@ class ServicesTest < Test::Unit::TestCase # keep test output clean: suppress timeout warning orig = $VERBOSE; $VERBOSE = nil Geocoder::Configuration.lookup = :yandex + set_api_key!(:yandex) assert_equal [], Geocoder.search("invalid key") + ensure $VERBOSE = orig end @@ -96,6 +123,7 @@ class ServicesTest < Test::Unit::TestCase def test_geocoder_ca_result_components Geocoder::Configuration.lookup = :geocoder_ca + set_api_key!(:geocoder_ca) result = Geocoder.search([45.423733, -75.676333]).first assert_equal "CA", result.country_code assert_equal "289 Somerset ST E, Ottawa, ON K1N6W1, Canada", result.address @@ -119,6 +147,7 @@ class ServicesTest < Test::Unit::TestCase def test_bing_result_components Geocoder::Configuration.lookup = :bing + set_api_key!(:bing) result = Geocoder.search("Madison Square Garden, New York, NY").first assert_equal "Madison Square Garden, NY", result.address assert_equal "NY", result.state @@ -127,22 +156,52 @@ class ServicesTest < Test::Unit::TestCase def test_bing_no_results Geocoder::Configuration.lookup = :bing + set_api_key!(:bing) results = Geocoder.search("no results") assert_equal 0, results.length end # --- Nominatim --- - def test_nominatim_result_components + def test_nominatim_result_components Geocoder::Configuration.lookup = :nominatim + set_api_key!(:nominatim) result = Geocoder.search("Madison Square Garden, New York, NY").first assert_equal "10001", result.postal_code end def test_nominatim_address_formatting Geocoder::Configuration.lookup = :nominatim + set_api_key!(:nominatim) result = Geocoder.search("Madison Square Garden, New York, NY").first assert_equal "Madison Square Garden, West 31st Street, Long Island City, New York City, New York, 10001, United States of America", result.address end + + # --- MapQuest --- + + def test_api_route + Geocoder::Configuration.lookup = :mapquest + Geocoder::Configuration.api_key = "abc123" + lookup = Geocoder::Lookup::Mapquest.new + query = Geocoder::Query.new("Bluffton, SC") + res = lookup.send(:query_url, query) + assert_equal "http://www.mapquestapi.com/geocoding/v1/address?key=abc123&location=Bluffton%2C+SC", + res + end + + def test_mapquest_result_components + Geocoder::Configuration.lookup = :mapquest + set_api_key!(:mapquest) + result = Geocoder.search("Madison Square Garden, New York, NY").first + assert_equal "10001", result.postal_code + end + + def test_mapquest_address_formatting + Geocoder::Configuration.lookup = :mapquest + set_api_key!(:mapquest) + result = Geocoder.search("Madison Square Garden, New York, NY").first + assert_equal "46 West 31st Street, New York, NY, 10001, US", + result.address + end end diff --git a/test/test_helper.rb b/test/test_helper.rb index c77b44c19adb6ddd72a6a317295cf9084800d754..bff1219907fc5343f6426f114a9290135e195a05 100644 --- a/test/test_helper.rb +++ b/test/test_helper.rb @@ -4,6 +4,12 @@ require 'test/unit' $LOAD_PATH.unshift(File.dirname(__FILE__)) $LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) +class MysqlConnection + def adapter_name + "mysql" + end +end + ## # Simulate enough of ActiveRecord::Base that objects can be used for testing. # @@ -28,6 +34,10 @@ module ActiveRecord def self.scope(*args); end + def self.connection + MysqlConnection.new + end + def method_missing(name, *args, &block) if name.to_s[-1..-1] == "=" write_attribute name.to_s[0...-1], *args @@ -65,13 +75,19 @@ module Geocoder class Base private #----------------------------------------------------------------- def read_fixture(file) - File.read(File.join("test", "fixtures", file)).strip.gsub(/\n\s*/, "") + filepath = File.join("test", "fixtures", file) + s = File.read(filepath).strip.gsub(/\n\s*/, "") + s.instance_eval do + def body; self; end + def code; "200"; end + end + s end end class Google < Base private #----------------------------------------------------------------- - def fetch_raw_data(query) + def make_api_request(query) raise TimeoutError if query.text == "timeout" raise SocketError if query.text == "socket_error" file = case query.text @@ -89,12 +105,13 @@ module Geocoder class Yahoo < Base private #----------------------------------------------------------------- - def fetch_raw_data(query) + def make_api_request(query) raise TimeoutError if query.text == "timeout" raise SocketError if query.text == "socket_error" file = case query.text - when "no results"; :no_results - else :madison_square_garden + when "no results"; :no_results + when "error"; :error + else :madison_square_garden end read_fixture "yahoo_#{file}.json" end @@ -102,7 +119,7 @@ module Geocoder class Yandex < Base private #----------------------------------------------------------------- - def fetch_raw_data(query) + def make_api_request(query) raise TimeoutError if query.text == "timeout" raise SocketError if query.text == "socket_error" file = case query.text @@ -116,7 +133,7 @@ module Geocoder class GeocoderCa < Base private #----------------------------------------------------------------- - def fetch_raw_data(query) + def make_api_request(query) raise TimeoutError if query.text == "timeout" raise SocketError if query.text == "socket_error" if query.reverse_geocode? @@ -133,7 +150,7 @@ module Geocoder class Freegeoip < Base private #----------------------------------------------------------------- - def fetch_raw_data(query) + def make_api_request(query) raise TimeoutError if query.text == "timeout" raise SocketError if query.text == "socket_error" file = case query.text @@ -146,7 +163,7 @@ module Geocoder class Bing < Base private #----------------------------------------------------------------- - def fetch_raw_data(query) + def make_api_request(query) raise TimeoutError if query.text == "timeout" raise SocketError if query.text == "socket_error" if query.reverse_geocode? @@ -163,7 +180,7 @@ module Geocoder class Nominatim < Base private #----------------------------------------------------------------- - def fetch_raw_data(query) + def make_api_request(query) raise TimeoutError if query.text == "timeout" raise SocketError if query.text == "socket_error" file = case query.text @@ -174,9 +191,9 @@ module Geocoder end end - class Mapquest < Nominatim + class Mapquest < Base private #----------------------------------------------------------------- - def fetch_raw_data(query) + def make_api_request(query) raise TimeoutError if query.text == "timeout" raise SocketError if query.text == "socket_error" file = case query.text @@ -307,5 +324,16 @@ class Test::Unit::TestCase return false unless coordinates.size == 2 # Should have dimension 2 coordinates[0].nan? && coordinates[1].nan? # Both coordinates should be NaN end -end + def set_api_key!(lookup_name) + lookup = Geocoder::Lookup.get(lookup_name) + if lookup.required_api_key_parts.size == 1 + key = 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' + elsif lookup.required_api_key_parts.size > 1 + key = lookup.required_api_key_parts + else + key = nil + end + Geocoder::Configuration.api_key = key + end +end diff --git a/test/test_mode_test.rb b/test/test_mode_test.rb index 32dcd54660c764dcf1b8e6db0287b5bfcf3cf9c2..0bece98b0935edd9a3a717cfa10904601c219f75 100644 --- a/test/test_mode_test.rb +++ b/test/test_mode_test.rb @@ -1,5 +1,4 @@ require 'test_helper' -require 'geocoder/lookups/test' class TestModeTest < Test::Unit::TestCase