際際滷

際際滷Share a Scribd company logo
PostGIS on Rails




                                    Matt Nemenman
                           matt@apartmentlist.com
                                     @quarterdome

Friday, April 19,                                   1
About me


                    Programmer

                    Love maps

                    Building map based rental search
                    engine @ Apartment List




Friday, April 19,                                      2
Location Aware Apps



                    Where am I?

                    What is around me?




Friday, April 19,                         3
Where am I?

                    Latitude and Longitude

                      HTML5 geolocation or GPS

                    Address

                      Geocoding

                      Reverse geocoding


Friday, April 19,                                4
Where am I?

                    What if you need to know ...

                      Neighborhood

                      School district

                      National park

                      Earthquake safety zone



Friday, April 19,                                  5
What is around me?


                    Yelp and Google Places API

                      Restaurants, bars, etc.

                      Points of interest




Friday, April 19,                                6
What is around me?

                    What if you want to know ...

                      What are three neighborhoods
                      closest to me?

                      Average rent for 1 bedroom in
                      Lower Pacific Heights

                      Crime rate on my city block

                      Who is around me?

Friday, April 19,                                     7
When 3rd party APIs
                      fall short ...


                    Get your own data set

                    Build your own solution




Friday, April 19,                             8
Spatial Systems

                    MongoDB

                    Solr

                    MySQL

                    Oracle / DB2

                    PostGIS



Friday, April 19,                       9
PostGIS
                     Geospatial extension to Postgres

                     New datatypes

                     Functions to work with those
                     datatypes

                     Spatial indices using GiST


                    create extension postgis;

Friday, April 19,                                       10
A Simple Example




Friday, April 19,                      11
A simple example

                    Yosemite Park Ranger

                    Tracking bears equipped with GPS
                    transmitters

                    Want to show all the bears on Google
                    Maps




Friday, April 19,                                          12
Bears (database
                         migration)
                    create_table :bears do |t|
                      t.column :lat, :float
                      t.column :lon, :float
                    end

                    add_index :bears, :lat
                    add_index :bears, :lon
                    add_index :bears, [:lat, :lon]


Friday, April 19,                                    13
Bears (model)
                    class Bear < ActiveRecord::Base

                    def self.bbox(sw_lon, sw_lat,
                                  ne_lon, ne_lat)
                      self
                        .where( :lon => sw_lon..ne_lon )
                        .where( :lat => sw_lat..ne_lat )
                    end

                    end


Friday, April 19,                                          14
A PostGIS example
                        (migration)
                    create_table :bears do |t|
                      t.column :coordinates,
                        :geometry,
                        :srid => 4326
                    end

                    add_index :bears,
                      :coordinates,
                      :spatial => true

Friday, April 19,                                15
A PostGIS example
                           (model)
                    def self.box(sw_lon, sw_lat, ne_lon, ne_lat)

                      factory = Rails.application.spatial_factory

                      sw   =   factory.point(sw_lon,   sw_lat)
                      nw   =   factory.point(sw_lon,   ne_lat)
                      ne   =   factory.point(ne_lon,   ne_lat)
                      se   =   factory.point(ne_lon,   sw_lat)

                      ring = factory.linear_ring([sw, nw, ne, se])
                      bbox = factory.polygon(ring)

                      self
                        .where('ST_Intersects(coordinates, :bbox)',
                               :bbox => bbox)
                    end


Friday, April 19,                                                     16
A PostGIS example


                    1_000_000.times do
                      Bear.create!( ...)
                    end




Friday, April 19,                          17
1,000,000 bears



                    PostGIS is 1.5x to 50x faster




Friday, April 19,                                   18
Active Record
                       PostGIS Adapter
                    Migration support

                    Automatic conversion of PostGIS
                    datatypes to Ruby (RGeo) objects and
                    back

             gem 'pg'
             gem 'rgeo'
             gem 'activerecord-postgis-adapter'

Friday, April 19,                                          19
Rails Migrations

                    add_column :bears,
                      :coordinates,
                      :geometry, :srid => 4326

                    add_index :bears,
                      :coordinates,
                      :spatial => true


Friday, April 19,                                20
Postgres Table

                    => d bears
                                     Table "public.bears"
                        Column    |            Type
                    -------------+-----------------------------
                     id           | integer
                     coordinates | geometry(Geometry,4326)
                    Indexes:
                         "pins_pkey" PRIMARY KEY, btree (id)
                         "index_pins_on_coordinates" gist (coordinates)




Friday, April 19,                                                         21
PostGIS Data

             => select id, coordinates from bears limit 4;
              id |                    coordinates
             ----+----------------------------------------------------
               1 | 0101000020E61000002A8A632C341F664021805D8DDBEB4BC0
               2 | 0101000020E61000004DF900A54A5866C0A2BAB6AC827D50C0
               3 | 0101000020E61000002450EA628F5259C01C789C77C2883040
               4 | 0101000020E610000038760C7B85443C4013206005DC2C48C0
             (4 rows)




Friday, April 19,                                                        22
RGeo

             ##> bear=Bear.first
              => #<Bear id: 1, coordinates:
             #<RGeo::Geos::CAPIPointImpl:0x3fd52ab501b4 "POINT
             (176.9751188224921 -55.84263770165877)">>

             ##> bear.coordinates.x
              => 176.9751188224921

             ##> bear.coordinates.y
              => -55.84263770165877




Friday, April 19,                                                23
More examples
             # d parks
                       Column        |            Type
             ------------------------+-----------------------------
              id                     | integer
              name                   | character varying(255)
              boundary               | geometry(Geometry,4326)

             Indexes:
                 "parks_pkey" PRIMARY KEY, btree (id)
                 "index_parks_on_name" btree (name)
                 "index_parks_on_boundary" gist (polygon)

             # select id, name, boundary from parks limit 2;
                id   |    name     |      boundary
             --------+-------------+------------------------
                   1 | Yosemite    |0103000020E6100000010000...
                   2 | Yellowstone |0103000020E6100000010000...



Friday, April 19,                                                     24
How many bears are
                     in Yosemite now?

          ##> park = Park.find_by_name(Yosemite)

          ##> bears = Bear.where(ST_Intersects(coordinates, :bounds),
                         :bounds => park.boundary)

          ##> bear_count = bears.count




Friday, April 19,                                                         25
How Many Bears in
                    Yosemite and Yellowstone
                             (Ruby)?

          ##> yosemite = Park.find_by_name(Yosemite)

          ##> yellowstone = Park.find_by_name(Yellowstone)

          ##> bounds = yosemite.boundary + yellowstone.boundary

          ##> bears = Bear.where(ST_Intersects(coordinates, :bounds),
                         :bounds => bounds)

          ##> bear_count = bears.count




Friday, April 19,                                                         26
How Many Bears in
                    Yosemite and Yellowstone
                             (SQL)?


          select count(*) from bears
            inner join parks
              on ST_Intersects(bears.coordinates,
                               parks.boundary)
            where parks.name in (Yosemite,
                                 Yellowstone);




Friday, April 19,                                   27
Three parks closest
                          to me?
                    Distance operator (KNN) is a feature
                    of Postgres 9.1 and above


          select id, name,
               boundary <-> ST_Point(37.775, -122.44) as distance
          from parks
          order by distance
          limit 3;




Friday, April 19,                                                   28
What else is
                     possible?



Friday, April 19,                  29
Geometry
                       Simplification

                    ST_Simplify

                    ST_ConvexHull

                    ST_ConcaveHull




Friday, April 19,                       30
Spatial
                        Relationships
                    ST_Centroid

                    ST_Contained

                    ST_Area

                    ST_Perimeter

                    ST_DWithin



Friday, April 19,                       31
Format Conversions



                    ST_AsGeoJSON

                    ST_AsText




Friday, April 19,                        32
Do Try It at Home

                    Heroku Postgres
                      https://devcenter.heroku.com/articles/heroku-
                      postgres-extensions-postgis-full-text-search


                    Postgres.app
                      http://postgresapp.com/


                    select postgis_full_version();



Friday, April 19,                                                     33
Data Sources
                             (free)

                    US Census Data (Tiger)
                      http://www.census.gov/geo/maps-data/data/tiger.html


                    Zillow Neighborhoods
                      http://www.zillow.com/howto/api/neighborhood-
                      boundaries.htm




Friday, April 19,                                                           34
Data Sources
                         (commercial)
                    Maponics

                      http://www.maponics.com/

                    Urban Mapping

                      http://www.urbanmapping.com/

                    Onboard Informatics

                      http://www.onboardinformatics.com/


Friday, April 19,                                          35
Links
                    PostGIS

                      http://postgis.net/

                    Active Record PostGIS Adapter

                      https://github.com/dazuma/
                      activerecord-postgis-adapter

                    RGeo

                      https://github.com/dazuma/rgeo


Friday, April 19,                                      36
Q & A




                    We are hiring @apartmentlist!
Friday, April 19,                                   37

More Related Content

PostGIS on Rails

  • 1. PostGIS on Rails Matt Nemenman matt@apartmentlist.com @quarterdome Friday, April 19, 1
  • 2. About me Programmer Love maps Building map based rental search engine @ Apartment List Friday, April 19, 2
  • 3. Location Aware Apps Where am I? What is around me? Friday, April 19, 3
  • 4. Where am I? Latitude and Longitude HTML5 geolocation or GPS Address Geocoding Reverse geocoding Friday, April 19, 4
  • 5. Where am I? What if you need to know ... Neighborhood School district National park Earthquake safety zone Friday, April 19, 5
  • 6. What is around me? Yelp and Google Places API Restaurants, bars, etc. Points of interest Friday, April 19, 6
  • 7. What is around me? What if you want to know ... What are three neighborhoods closest to me? Average rent for 1 bedroom in Lower Pacific Heights Crime rate on my city block Who is around me? Friday, April 19, 7
  • 8. When 3rd party APIs fall short ... Get your own data set Build your own solution Friday, April 19, 8
  • 9. Spatial Systems MongoDB Solr MySQL Oracle / DB2 PostGIS Friday, April 19, 9
  • 10. PostGIS Geospatial extension to Postgres New datatypes Functions to work with those datatypes Spatial indices using GiST create extension postgis; Friday, April 19, 10
  • 11. A Simple Example Friday, April 19, 11
  • 12. A simple example Yosemite Park Ranger Tracking bears equipped with GPS transmitters Want to show all the bears on Google Maps Friday, April 19, 12
  • 13. Bears (database migration) create_table :bears do |t| t.column :lat, :float t.column :lon, :float end add_index :bears, :lat add_index :bears, :lon add_index :bears, [:lat, :lon] Friday, April 19, 13
  • 14. Bears (model) class Bear < ActiveRecord::Base def self.bbox(sw_lon, sw_lat, ne_lon, ne_lat) self .where( :lon => sw_lon..ne_lon ) .where( :lat => sw_lat..ne_lat ) end end Friday, April 19, 14
  • 15. A PostGIS example (migration) create_table :bears do |t| t.column :coordinates, :geometry, :srid => 4326 end add_index :bears, :coordinates, :spatial => true Friday, April 19, 15
  • 16. A PostGIS example (model) def self.box(sw_lon, sw_lat, ne_lon, ne_lat) factory = Rails.application.spatial_factory sw = factory.point(sw_lon, sw_lat) nw = factory.point(sw_lon, ne_lat) ne = factory.point(ne_lon, ne_lat) se = factory.point(ne_lon, sw_lat) ring = factory.linear_ring([sw, nw, ne, se]) bbox = factory.polygon(ring) self .where('ST_Intersects(coordinates, :bbox)', :bbox => bbox) end Friday, April 19, 16
  • 17. A PostGIS example 1_000_000.times do Bear.create!( ...) end Friday, April 19, 17
  • 18. 1,000,000 bears PostGIS is 1.5x to 50x faster Friday, April 19, 18
  • 19. Active Record PostGIS Adapter Migration support Automatic conversion of PostGIS datatypes to Ruby (RGeo) objects and back gem 'pg' gem 'rgeo' gem 'activerecord-postgis-adapter' Friday, April 19, 19
  • 20. Rails Migrations add_column :bears, :coordinates, :geometry, :srid => 4326 add_index :bears, :coordinates, :spatial => true Friday, April 19, 20
  • 21. Postgres Table => d bears Table "public.bears" Column | Type -------------+----------------------------- id | integer coordinates | geometry(Geometry,4326) Indexes: "pins_pkey" PRIMARY KEY, btree (id) "index_pins_on_coordinates" gist (coordinates) Friday, April 19, 21
  • 22. PostGIS Data => select id, coordinates from bears limit 4; id | coordinates ----+---------------------------------------------------- 1 | 0101000020E61000002A8A632C341F664021805D8DDBEB4BC0 2 | 0101000020E61000004DF900A54A5866C0A2BAB6AC827D50C0 3 | 0101000020E61000002450EA628F5259C01C789C77C2883040 4 | 0101000020E610000038760C7B85443C4013206005DC2C48C0 (4 rows) Friday, April 19, 22
  • 23. RGeo ##> bear=Bear.first => #<Bear id: 1, coordinates: #<RGeo::Geos::CAPIPointImpl:0x3fd52ab501b4 "POINT (176.9751188224921 -55.84263770165877)">> ##> bear.coordinates.x => 176.9751188224921 ##> bear.coordinates.y => -55.84263770165877 Friday, April 19, 23
  • 24. More examples # d parks Column | Type ------------------------+----------------------------- id | integer name | character varying(255) boundary | geometry(Geometry,4326) Indexes: "parks_pkey" PRIMARY KEY, btree (id) "index_parks_on_name" btree (name) "index_parks_on_boundary" gist (polygon) # select id, name, boundary from parks limit 2; id | name | boundary --------+-------------+------------------------ 1 | Yosemite |0103000020E6100000010000... 2 | Yellowstone |0103000020E6100000010000... Friday, April 19, 24
  • 25. How many bears are in Yosemite now? ##> park = Park.find_by_name(Yosemite) ##> bears = Bear.where(ST_Intersects(coordinates, :bounds), :bounds => park.boundary) ##> bear_count = bears.count Friday, April 19, 25
  • 26. How Many Bears in Yosemite and Yellowstone (Ruby)? ##> yosemite = Park.find_by_name(Yosemite) ##> yellowstone = Park.find_by_name(Yellowstone) ##> bounds = yosemite.boundary + yellowstone.boundary ##> bears = Bear.where(ST_Intersects(coordinates, :bounds), :bounds => bounds) ##> bear_count = bears.count Friday, April 19, 26
  • 27. How Many Bears in Yosemite and Yellowstone (SQL)? select count(*) from bears inner join parks on ST_Intersects(bears.coordinates, parks.boundary) where parks.name in (Yosemite, Yellowstone); Friday, April 19, 27
  • 28. Three parks closest to me? Distance operator (KNN) is a feature of Postgres 9.1 and above select id, name, boundary <-> ST_Point(37.775, -122.44) as distance from parks order by distance limit 3; Friday, April 19, 28
  • 29. What else is possible? Friday, April 19, 29
  • 30. Geometry Simplification ST_Simplify ST_ConvexHull ST_ConcaveHull Friday, April 19, 30
  • 31. Spatial Relationships ST_Centroid ST_Contained ST_Area ST_Perimeter ST_DWithin Friday, April 19, 31
  • 32. Format Conversions ST_AsGeoJSON ST_AsText Friday, April 19, 32
  • 33. Do Try It at Home Heroku Postgres https://devcenter.heroku.com/articles/heroku- postgres-extensions-postgis-full-text-search Postgres.app http://postgresapp.com/ select postgis_full_version(); Friday, April 19, 33
  • 34. Data Sources (free) US Census Data (Tiger) http://www.census.gov/geo/maps-data/data/tiger.html Zillow Neighborhoods http://www.zillow.com/howto/api/neighborhood- boundaries.htm Friday, April 19, 34
  • 35. Data Sources (commercial) Maponics http://www.maponics.com/ Urban Mapping http://www.urbanmapping.com/ Onboard Informatics http://www.onboardinformatics.com/ Friday, April 19, 35
  • 36. Links PostGIS http://postgis.net/ Active Record PostGIS Adapter https://github.com/dazuma/ activerecord-postgis-adapter RGeo https://github.com/dazuma/rgeo Friday, April 19, 36
  • 37. Q & A We are hiring @apartmentlist! Friday, April 19, 37