README.md

Geocoder

A complete geocoding solution for Ruby.

Key features:

• Forward and reverse geocoding, and IP address geocoding.
• Connects to more than 40 APIs worldwide.
• Performance-enhancing feaures like caching.
• Advanced configuration allows different parameters and APIs to be used in different conditions.
• Integrates with ActiveRecord and Mongoid.
• Basic geospatial queries: search within radius (or rectangle, or ring).

Compatibility:

• Supports multiple Ruby versions: Ruby 1.9.3, 2.x, and JRuby.
• Supports multiple databases: MySQL, PostgreSQL, SQLite, and MongoDB (1.7.0 and higher).
• Supports Rails 3, 4, and 5. If you need to use it with Rails 2 please see the rails2 branch (no longer maintained, limited feature set).
• Works very well outside of Rails, you just need to install either the json (for MRI) or json_pure (for JRuby) gem.

Table of Contents

Basic Features:

• Basic Search
• Geocoding Objects
• Geospatial Database Queries
• Geocoding HTTP Requests
• Geocoding Service ("Lookup") Configuration

Advanced Features:

• Performance and Optimization
• Advanced Model Configuration
• Advanced Database Queries
• Batch Geocoding
• Testing
• Error Handling
• Command Line Interface

The Rest:

• Technical Discussions
• Troubleshooting
• Known Issues
• Reporting Issues
• Contributing

See Also:

• Guide to Geocoding APIs (formerly part of this README)

Basic Search

In its simplest form, Geocoder takes an address and searches for its latitude/longitude coordinates:

results = Geocoder.search("Paris")
results.first.coordinates
=> [48.856614, 2.3522219]  # latitude and longitude

The reverse is possible too. Given coordinates, it finds an address:

results = Geocoder.search([48.856614, 2.3522219])
results.first.address
=> "Hôtel de Ville, 75004 Paris, France"

You can also look up the location of an IP addresses:

results = Geocoder.search("172.56.21.89")
results.first.coordinates
=> [30.267153, -97.7430608]
results.first.country
=> "United States"

The success and accuracy of geocoding depends entirely on the API being used to do these lookups. Most queries work fairly well with the default configuration, but every application has different needs and every API has its particular strengths and weaknesses. If you need better coverage for your application you'll want to get familiar with the large number of supported APIs, listed in the API Guide.

Geocoding Objects

To automatically geocode your objects:

1. Your model must provide a method that returns an address to geocode. This can be a single attribute, but it can also be a method that returns a string assembled from different attributes (eg: city, state, and country). For example, if your model has street, city, state, and country attributes you might do something like this:

def address
  [street, city, state, country].compact.join(', ')
end

2. Your model must have a way to store latitude/longitude coordinates. With ActiveRecord, add two attributes/columns (of type float or decimal) called latitude and longitude. For MongoDB, use a single field (of type Array) called coordinates (i.e., field :coordinates, type: Array). (See Advanced Model Configuration for using different attribute names.)

3. In your model, tell geocoder where to find the object's address:

geocoded_by :address

This adds a geocode method which you can invoke via callback:

after_validation :geocode

Reverse geocoding (given lat/lon coordinates, find an address) is similar:

reverse_geocoded_by :latitude, :longitude
after_validation :reverse_geocode

With any geocoded objects, you can do the following:

obj.distance_to([43.9,-98.6])  # distance from obj to point
obj.bearing_to([43.9,-98.6])   # bearing from obj to point
obj.bearing_from(obj2)         # bearing from obj2 to obj

The bearing_from/to methods take a single argument which can be: a [lat,lon] array, a geocoded object, or a geocodable address (string). The distance_from/to methods also take a units argument (:mi, :km, or :nm for nautical miles). See Distance and Bearing below for more info.

One More Thing for MongoDB!

Before you can call geocoded_by you'll need to include the necessary module using one of the following:

include Geocoder::Model::Mongoid
include Geocoder::Model::MongoMapper

Latitude/Longitude Order in MongoDB

Everywhere coordinates are passed to methods as two-element arrays, Geocoder expects them to be in the order: [lat, lon]. However, as per the GeoJSON spec, MongoDB requires that coordinates be stored longitude-first ([lon, lat]), so internally they are stored "backwards." Geocoder's methods attempt to hide this, so calling obj.to_coordinates (a method added to the object by Geocoder via geocoded_by) returns coordinates in the conventional order:

obj.to_coordinates  # => [37.7941013, -122.3951096] # [lat, lon]

whereas calling the object's coordinates attribute directly (obj.coordinates by default) returns the internal representation which is probably the reverse of what you want:

obj.coordinates     # => [-122.3951096, 37.7941013] # [lon, lat]

So, be careful.

Use Outside of Rails

To use Geocoder with ActiveRecord and a framework other than Rails (like Sinatra or Padrino), you will need to add this in your model before calling Geocoder methods:

extend Geocoder::Model::ActiveRecord

Geospatial Database Queries

For ActiveRecord models:

To find objects by location, use the following scopes:

Venue.near('Omaha, NE, US')                   # venues within 20 miles of Omaha
Venue.near([40.71, -100.23], 50)              # venues within 50 miles of a point
Venue.near([40.71, -100.23], 50, units: :km)  # venues within 50 kilometres of a point
Venue.geocoded                                # venues with coordinates
Venue.not_geocoded                            # venues without coordinates

by default, objects are ordered by distance. To remove the ORDER BY clause use the following:

Venue.near('Omaha', 20, order: false)

With geocoded objects you can do things like this:

if obj.geocoded?
  obj.nearbys(30)                       # other objects within 30 miles
  obj.distance_from([40.714,-100.234])  # distance from arbitrary point to object
  obj.bearing_to("Paris, France")       # direction from object to arbitrary point
end

Some utility methods are also available:

# look up coordinates of some location
Geocoder.coordinates("25 Main St, Cooperstown, NY")
 => [42.700149, -74.922767]

# distance between Eiffel Tower and Empire State Building
Geocoder::Calculations.distance_between([47.858205,2.294359], [40.748433,-73.985655])
 => 3619.77359999382 # in configured units (default miles)

# find the geographic center (aka center of gravity) of objects or points
Geocoder::Calculations.geographic_center([city1, city2, [40.22,-73.99], city4])
 => [35.14968, -90.048929]

Please see the code for more methods and detailed information about arguments (eg, working with kilometers).

For MongoDB-backed models:

Please do not use Geocoder's near method. Instead use MongoDB's built-in geospatial query language, which is faster. Mongoid also provides a DSL for geospatial queries.

Geocoding HTTP Requests

Geocoder adds location and safe_location methods to the standard Rack::Request object so you can easily look up the location of any HTTP request by IP address. For example, in a Rails controller or a Sinatra app:

# returns Geocoder::Result object
result = request.location

The location method is vulnerable to trivial IP address spoofing via HTTP headers. If that's a problem for your application, use safe_location instead, but be aware that safe_location will not try to trace a request's originating IP through proxy headers; you will instead get the location of the last proxy the request passed through, if any (excepting any proxies you have explicitly whitelisted in your Rack config).

Note that these methods will usually return nil in test and development environments because things like "localhost" and "0.0.0.0" are not geocodable IP addresses.

Geocoding Service ("Lookup") Configuration

Geocoder supports a variety of street and IP address geocoding services. The default lookups are :google for street addresses and :ipinfo_io for IP addresses. Please see the API Guide for details on specific geocoding services (not all settings are supported by all services).

To create a Rails initializer with sample configuration:

rails generate geocoder:config

Some common options are:

# config/initializers/geocoder.rb
Geocoder.configure(

  # street address geocoding service (default :google)
  lookup: :yandex,

  # IP address geocoding service (default :ipinfo_io)
  ip_lookup: :maxmind,

  # to use an API key:
  api_key: "...",

  # geocoding service request timeout, in seconds (default 3):
  timeout: 5,

  # set default units to kilometers:
  units: :km,

  # caching (see [below](#caching) for details):
  cache: Redis.new,
  cache_prefix: "..."

)

Please see lib/geocoder/configuration.rb for a complete list of configuration options. Additionally, some lookups have their own special configuration options which are directly supported by Geocoder. For example, to specify a value for Google's bounds parameter:

# with Google:
Geocoder.search("Paris", bounds: [[32.1,-95.9], [33.9,-94.3]])

Please see the source code for each lookup to learn about directly supported parameters. Parameters which are not directly supported can be specified using the :params option, which appends options to the query string of the geocoding request. For example:

# Nominatim's `countrycodes` parameter:
Geocoder.search("Paris", params: {countrycodes: "gb,de,fr,es,us"})

# Google's `region` parameter:
Geocoder.search("Paris", params: {region: "..."})

Configuring Multiple Services

You can configure multiple geocoding services at once by using the service's name as a key for a sub-configuration hash, like this:

Geocoder.configure(

  timeout: 2,
  cache: Redis.new,

  yandex: {
    api_key: "...",
    timeout: 5
  },

  baidu: {
    api_key: "..."
  },

  maxmind: {
    api_key: "...",
    service: :omni
  }

)

Lookup-specific settings override global settings so, in this example, the timeout for all lookups is 2 seconds, except for Yandex which is 5.

Performance and Optimization

Database Indices

In MySQL and Postgres, queries use a bounding box to limit the number of points over which a more precise distance calculation needs to be done. To take advantage of this optimisation, you need to add a composite index on latitude and longitude. In your Rails migration:

add_index :table, [:latitude, :longitude]

In MongoDB, by default, the methods geocoded_by and reverse_geocoded_by create a geospatial index. You can avoid index creation with the :skip_index option, for example:

include Geocoder::Model::Mongoid
geocoded_by :address, skip_index: true

Avoiding Unnecessary API Requests

Geocoding only needs to be performed under certain conditions. To avoid unnecessary work (and quota usage) you will probably want to geocode an object only when:

• an address is present
• the address has been changed since last save (or it has never been saved)

The exact code will vary depending on the method you use for your geocodable string, but it would be something like this:

after_validation :geocode, if: ->(obj){ obj.address.present? and obj.address_changed? }

Caching

When relying on any external service, it's always a good idea to cache retrieved data. When implemented correctly, it improves your app's response time and stability. It's easy to cache geocoding results with Geocoder -- just configure a cache store:

Geocoder.configure(cache: Redis.new)

This example uses Redis, but the cache store can be any object that supports these methods:

• store#[](key) or #get or #read - retrieves a value
• store#[]=(key, value) or #set or #write - stores a value
• store#del(url) - deletes a value
• store#keys - (Optional) Returns array of keys. Used if you wish to expire the entire cache (see below).

Even a plain Ruby hash will work, though it's not a great choice (cleared out when app is restarted, not shared between app instances, etc).

You can also set a custom prefix to be used for cache keys:

Geocoder.configure(cache_prefix: "...")

By default the prefix is geocoder:

If you need to expire cached content:

Geocoder::Lookup.get(Geocoder.config[:lookup]).cache.expire(:all)  # expire cached results for current Lookup
Geocoder::Lookup.get(:google).cache.expire("http://...")           # expire cached result for a specific URL
Geocoder::Lookup.get(:google).cache.expire(:all)                   # expire cached results for Google Lookup
# expire all cached results for all Lookups.
# Be aware that this methods spawns a new Lookup object for each Service
Geocoder::Lookup.all_services.each{|service| Geocoder::Lookup.get(service).cache.expire(:all)}

Do not include the prefix when passing a URL to be expired. Expiring :all will only expire keys with the configured prefix -- it will not expire every entry in your key/value store.

For an example of a cache store with URL expiry, please see examples/autoexpire_cache.rb

Before you implement caching in your app please be sure that doing so does not violate the Terms of Service for your geocoding service.

Advanced Model Configuration

You are not stuck with the latitude and longitude database column names (with ActiveRecord) or the coordinates array (Mongo) for storing coordinates. For example:

geocoded_by :address, latitude: :lat, longitude: :lon  # ActiveRecord
geocoded_by :address, coordinates: :coords             # MongoDB

For reverse geocoding, you can specify the attribute where the address will be stored. For example:

reverse_geocoded_by :latitude, :longitude, address: :loc    # ActiveRecord
reverse_geocoded_by :coordinates, address: :street_address  # MongoDB

To specify geocoding parameters in your model:

geocoded_by :address, params: {region: "..."}

Supported parameters: :lookup, :ip_lookup, :language, and :params. You can specify an anonymous function if you want to set these on a per-request basis. For example, to use different lookups for objects in different regions:

geocoded_by :address, lookup: lambda{ |obj| obj.geocoder_lookup }

def geocoder_lookup
  if country_code == "RU"
    :yandex
  elsif country_code == "CN"
    :baidu
  else
    :google
  end
end

Custom Result Handling

So far we have seen examples where geocoding results are assigned automatically to predefined object attributes. However, you can skip the auto-assignment by providing a block which handles the parsed geocoding results any way you like, for example:

reverse_geocoded_by :latitude, :longitude do |obj,results|
  if geo = results.first
    obj.city    = geo.city
    obj.zipcode = geo.postal_code
    obj.country = geo.country_code
  end
end
after_validation :reverse_geocode

Every Geocoder::Result object, result, provides the following data:

• result.latitude - float
• result.longitude - float
• result.coordinates - array of the above two in the form of [lat,lon]
• result.address - string
• result.city - string
• result.state - string
• result.state_code - string
• result.postal_code - string
• result.country - string
• result.country_code - string

Most APIs return other data in addition to these globally-supported attributes. To directly access the full response, call the #data method of any Geocoder::Result object. See the API Guide for links to documentation for all geocoding services.

Forward and Reverse Geocoding in the Same Model

You can apply both forward and reverse geocoding to the same model (i.e. users can supply an address or coordinates and Geocoder fills in whatever's missing) but you'll need to provide two different address methods:

• one for storing the fetched address (when reverse geocoding)
• one for providing an address to use when fetching coordinates (forward geocoding)

For example:

class Venue

  # build an address from street, city, and state attributes
  geocoded_by :address_from_components

  # store the fetched address in the full_address attribute
  reverse_geocoded_by :latitude, :longitude, address: :full_address
end

The same goes for latitude/longitude. However, for purposes of querying the database, there can be only one authoritative set of latitude/longitude attributes for use in database queries. This is whichever you specify last. For example, here the attributes without the fetched_ prefix will be authoritative:

class Venue
  geocoded_by :address,
    latitude: :fetched_latitude,
    longitude: :fetched_longitude
  reverse_geocoded_by :latitude, :longitude
end

Advanced Database Queries

The following apply to ActiveRecord only. For MongoDB, please use the built-in geospatial features.

The default near search looks for objects within a circle. To search within a doughnut or ring use the :min_radius option:

Venue.near("Austin, TX", 200, min_radius: 40)

To search within a rectangle (note that results will not include distance and bearing attributes):

sw_corner = [40.71, 100.23]
ne_corner = [36.12, 88.65]
Venue.within_bounding_box(sw_corner, ne_corner)

To search for objects near a certain point where each object has a different distance requirement (which is defined in the database), you can pass a column name for the radius:

Venue.near([40.71, 99.23], :effective_radius)

If you store multiple sets of coordinates for each object, you can specify latitude and longitude columns to use for a search:

Venue.near("Paris", 50, latitude: :secondary_latitude, longitude: :secondary_longitude)

Distance and Bearing

When you run a geospatial query, the returned objects have two attributes added:

• obj.distance - number of miles from the search point to this object
• obj.bearing - direction from the search point to this object

Results are automatically sorted by distance from the search point, closest to farthest. Bearing is given as a number of degrees clockwise from due north, for example:

• 0 - due north
• 180 - due south
• 90 - due east
• 270 - due west
• 230.1 - southwest
• 359.9 - almost due north

You can convert these to compass point names via provided utility method:

Geocoder::Calculations.compass_point(355) # => "N"
Geocoder::Calculations.compass_point(45)  # => "NE"
Geocoder::Calculations.compass_point(208) # => "SW"

Note: when running queries on SQLite, distance and bearing are provided for consistency only. They are not very accurate.

Batch Geocoding

If you have just added geocoding to an existing application with a lot of objects, you can use this Rake task to geocode them all:

rake geocode:all CLASS=YourModel