More Related Content
Similar to Making the entire world accessible via a single API - the OpenCage Geocoder (20)
More from Ed Freyfogle (7)
Recently uploaded (20)
![DT Presentation[1].pptxeffsffewfwefewfewefefeef](https://cdn.slidesharecdn.com/ss_thumbnails/dtpresentation1-250404175210-289fff92-thumbnail.jpg?width=560&fit=bounds)
Making the entire world accessible via a single API - the OpenCage Geocoder
- 1. Making the entire world accessible via a single API Ed Freyfogle Barcelona Data Engineering - 30 Nov. 2017 http://opencagedata.com @opencagedata
- 2. - Some background - Technical overview - Build versus Buy? - Questions Overview
- 3. $whoami
- 4. $whoami - Co-founder of OpenCage - Before that, co-founded a real estate metasearch engine called Nestoria. 10 years, 20 employees, 4.5M in revenue - Way, way before that, first developer at yahoo.de - Moved to BCN to live the good life
- 9. Barcelona
- 12. Barcelona IronHack BCN 08002 Carrer de Pamplona Poblenou
- 13. Barcelona IronHack BCN 08002 Carrer de Pamplona Poblenou Europe
- 14. Barcelona IronHack BCN 08002 Carrer de Pamplona Poblenou Europe Spain / Rep. of Catalonia NEW!
- 16. 41.39778,2.19011
- 21. Why Geocode? - Vehicle tracking - Routing - Data enrichment - Put points on a map - Image processing
- 23. Yes!
- 24. Yes! but T&Cs expensive at volume
- 25. OpenStreetMap
- 26. Yes!
- 27. Yes! but not for high volume use variable coverage
- 29. Yes!
- 30. Yes! and one API, many geocoders free as in freedom enterprise level reliability
- 31. - Some background - Technical overview - Build versus Buy? - Questions Overview
- 34. OpenStreetMap - Crowdsourced geodata. Anyone can edit. 4 million reg. users - Bad name. Is a database, not a map. - 3 million edits per day. And growing. - Operations run by volunteers. - Master DB is hosted at a university in London - Data is under ODbL License - Many, many different tools for editing, querying, visualizing the data
- 36. Example $ curl https://api.opencagedata.com/geocode/v1/json?q=41.39778,2.19011&key=...&pretty=1 "components": { "ISO_3166-1_alpha-2": "ES", "_type": "pub", "city": "Barcelona", "city_district": "Sant Mart鱈", "country": "Spain", "country_code": "es", "county": "BCN", "house_number": "96", "postcode": "08018", "pub": "Sonora Sport Tavern", "road": "Carrer de Pamplona", "state": "Catalonia", "suburb": "el Parc i la Llacuna del Poblenou" }, "formatted": "Sonora Sport Tavern, Carrer de Pamplona, 96, 08018 Barcelona, Spain",
- 37. https://geocoder.opencagedata.com/code Libraries for - Perl, PHP, Python, Ruby, R, Java, .NET, elixir, Haskell, Go, Erlang, nodejs, Stata, perl6, iOS - Leaflet, OpenLayers TMTOWTDI
- 39. Architecture HTTP request LB LB - HAProxy - Two different physical locations
- 40. Architecture HTTP request LB LB HTTP request LB meta - ubuntu - apache - mod_perl
- 41. Architecture HTTP request LB LB HTTP request LB meta Kong HTTP request - authentication - validity checking - routing
- 42. Architecture HTTP request LB LB HTTP request LB meta Kong HTTP request geocoder geocoder geocoder
- 43. Architecture HTTP request LB LB HTTP request LB meta Kong HTTP request geocoder geocoder geocoder OSM Continually updating HTTP request
- 44. Architecture HTTP request LB LB HTTP request LB meta Kong HTTP request geocoder geocoder geocoder OSM Continually updating - accept response, error check - standardize format - rank - add annotations - send to client in XML, JSON, etc
- 45. - timezones - geohash, MGRS, Maidenhead, DMS - what3words - calling codes, currency - and more ... Annotations
- 46. Formatting $ curl https://api.opencagedata.com/geocode/v1/json?q=41.39778,2.19011&key=...&pretty=1 "components": { "ISO_3166-1_alpha-2": "ES", "_type": "pub", "city": "Barcelona", "city_district": "Sant Mart鱈", "country": "Spain", "country_code": "es", "county": "BCN", "house_number": "96", "postcode": "08018", "pub": "Sonora Sport Tavern", "road": "Carrer de Pamplona", "state": "Catalonia", "suburb": "el Parc i la Llacuna del Poblenou" }, "formatted": "Sonora Sport Tavern, Carrer de Pamplona, 96, 08018 Barcelona, Spain",
- 47. 115 New Cavendish Street, London W1T 5DU, United Kingdom 223 William Street, Melbourne VIC 3000, Australia Rosenthaler Strae 1, 10119 Berlin, Germany Via Pisacane, 13, 76121 Barletta BT, Italy 3 Upper Alma Road, Rosebank, Cape Town, 7700, South Africa Formatting
- 48. Architecture HTTP request LB LB HTTP request LB meta Kong HTTP request geocoder geocoder geocoder OSM Continually updating - logging - monitoring - emails metrics scp
- 50. - Dev/testing process - Deployment - User registration / onboarding - Billing Not covered
- 51. - Some background - Technical overview - Build versus Buy? - Questions Overview
- 52. - Unable to buy - Lack of finance - Lack of affordable, reliable vendor When to build
- 53. - Unable to buy - Core to your service When to build
- 54. - Unable to buy - Core to your service - Critical differentiator - Privacy / data protection issues - Legal requirement When to build
- 55. - Unable to build When to buy
- 56. - Unable to build - Need to move fast When to buy
- 57. - Unable to build - Need to move fast - Is non-core When to buy
- 58. - Unable to build - Need to move fast - Is non-core - Unable to maintain When to buy
- 59. - Financial cost - Opportunity costs - Staffing costs Cost analysis
- 60. - Industry trend has been towards buying - Servers (AWS, Dropbox, etc) - Login / Identity (FB, google, etc) - Billing (Stripe) - 1000s of SaaS businesses The trend
- 61. - Engineers love to build MAIN POINT
- 62. - Engineers love to build - Building is (comparatively) easy MAIN POINT
- 63. - Engineers love to build - Building is (comparatively) easy - MAINTENANCE IS REALLY HARD!!!! MAIN POINT
- 64. - Some background - Technical overview - Build versus Buy? - Questions Overview
- 66. Three more things 1. Lifetime 10% discount use code: XXXXXX
- 67. Three more things 1. Lifetime 10% discount use code: XXXXXX 2. Happy to pay anyone who wants to write a wrapper library: F#, Scala, Lua, etc
- 68. Three more things My friends at AVUXI are hiring http://www.avuxi.com
- 69. Ed Freyfogle Barcelona Data Engineering - 30 Nov. 2017 https://opencagedata.com @opencagedata Thanks! Questions?