際際滷

際際滷Share a Scribd company logo
araport.org@araport
Integrate JBrowse REST API
with ADAMA
Vivek Krishnakumar
12/03/2015
araport.org@araport
Outline
 JBrowse REST API spec
 ADAMA endpoints
 Mapping JBrowse to ADAMA endpoints
 Extending JBrowse REST feature store
 Handling Auth in JBrowse 1.x
 Use Case
 Integrate Chromatin States data into JBrowse via
ADAMA
 Summary
 Challenges
araport.org@araport
JBrowse REST API spec
 Javascript based portable genome browser tool
developed by GMOD community
 Ships with REST feature and names store adapter
 REST feature store requires access to following
endpoints via HTTP GET:
 (base)/stats/global
 (base)/stats/region/(refseq)?start=123&end=456
 (base)/stats/regionFeatureDensities/(refseq)?start=123&end=456&basesP
erBin=20000
 (base)/features/(refseq)?start=234&end=5678
 Can provide feature, sequence, and quantitative
data for display
 Accepted response type: application/json
http://gmod.org/wiki/JBrowse_Configuration_Guide#Writing_JBrowse-compatible_Web_Services
araport.org@araport
ADAMA endpoints
 Araport Data And Microservices API
 Developed by WalterM and MattV
 Encapsulates python code into reusable
docker containers
 Exposes internal logic as API endpoints:
 query, generic adapters via HTTP GET
 (base)/search
 (base)/list
 passthrough adapters via HTTP GET and POST
 (base)/access
 Implements token based Auth via iPlant
OAuth2 https://github.com/Arabidopsis-Information-Portal/adama
araport.org@araport
Mapping JBrowse to ADAMA endpoints
 ADAMA (currently) doesnt allow defining
custom endpoints
 So, we have to map the 4 JBrowse
endpoints to the 2 ADAMA endpoints
 Proposed endpoint mapping:
JBrowse ADAMA
(base)/stats/global (base)/search?q=global
(base)/stats/region (base)/search?q=region
(base)/stats/regionFeatureDensities (base)/search?q=regionFeatureDensities
(base)/features (base)/search?q=features
(base)/list?q=makeTrackListJson
araport.org@araport
Extending JBrowse REST feature store
 JBrowse/Store/SeqFeature/REST handles
requests from the browser to the
configured API service
 Understands the 4 pre-defined endpoints
 REST feature store extended based on
proposed endpoint mapping
 Utilize JBrowse Plugin architecture to
develop Araport plugin:
Araport/Store/SeqFeature/REST
http://gmod.org/wiki/JBrowse_Configuration_Guide#Writing_JBrowse_Plugins
araport.org@araport
Handling Auth in JBrowse 1.x
 ADAMA (currently) requires all requests to be
accompanied by an Authorization Bearer $TOKEN
 JBrowse 1.x does not support OAuth2 implicit flow,
thus cannot generate tokens
 Solution:
 OAuth2 $TOKEN created with unlimited validity (-1)
 $TOKEN generated by Araport anonymous user
 $TOKEN scope limited to ADAMA services
 Araport/Store/SeqFeature/RESTAuth hard codes
$TOKEN
 Araport/Store/SeqFeature/REST inherits from
RESTAuth
araport.org@araport
Use Case
Integrate Chromatin States data into JBrowse via ADAMA
 Published dataset representing a landscape
of Arabidopsis thaliana chromatin states
using 16 features, including DNA sequence,
CG methylation, histone variants, and
modifications
 Data represents chromosomal regions
(start/stop coordinates) corresponding with 9
distinct chromatin states
 Published with manuscript as supplementary
dataset (in XLS format)Sequeira-Mendes, J., Arag端ez, I., Peir坦, R., Mendez-Giraldez, R., Zhang, X., Jacobsen, S. E., ... & Gutierrez, C. (2014). The functional
topography of the Arabidopsis genome is organized in a reduced number of linear motifs of chromatin states. The Plant Cell, 26(6),
2351-2366.
araport.org@araport
 Develop ADAMA adapter to dynamically query data from
desired region and represent as JBrowse compatible
JSON
 Code: https://github.com/Arabidopsis-Information-
Portal/chromatin_states_to_jbrowse
 Implementation logic:
 Data transformed from 9 XLS worksheets into individual GFF3 files
 Pythonic parser developed using gffutils, allowing range querying (via intermediary
sqlite database)
 ADAMA metadata.yml written which describes dependencies, endpoints and
provenance
 Data hosted with code (due to small size, well within
GitHub file size limits)
 ADAMA namespace created and adapter registered
 Available in API explorer (search for chromatin)
Use Case
Integrate Chromatin States data into JBrowse via ADAMA
https://jira.araport.org/browse/MINE-1078
araport.org@araport
Integrate Chromatin States data into JBrowse via ADAMA
 JBrowse tracks configured to talk to registered ADAMA endpoint
 End result visible in screenshot above
 Live view:
https://apps.araport.org/jbrowse/?data=arabidopsis&loc=Chr3:11355449..11407858&tracks=chromatin_state_1,chr
omatin_state_2,chromatin_state_3,chromatin_state_4,chromatin_state_5,chromatin_state_6,chromatin_state_7,chr
omatin_state_8,chromatin_state_9,Araport11_gene_models
araport.org@araport
Summary
 Extended JBrowse to integrate with
ADAMA endpoints
 Exposed a published community dataset
via ADAMA as an API
 Exposed track configuration via ADAMA
API
 Visualized dataset in JBrowse via ADAMA
API
araport.org@araport
Challenges
 Making web service calls to ADAMA
requires Authorization $TOKEN
 JBrowse needs to hard code the
Authorization $TOKEN
 Track configuration needs to be manually
added to JBrowse
 Scaling to larger datasets not feasible via
GitHub route
araport.org@araport
Acknowledgements
 JCVI Araport Team
 Chia-Yi Cheng
 Agnes Chan
 Chris Town
 TACC Araport Team
 Walter Moreira
 Matthew Vaughn
 Prof. Crisanto Gutierrez (CBMSO)

More Related Content

Integrate JBrowse REST API Framework with Adama Federation Architecture

  • 1. araport.org@araport Integrate JBrowse REST API with ADAMA Vivek Krishnakumar 12/03/2015
  • 2. araport.org@araport Outline JBrowse REST API spec ADAMA endpoints Mapping JBrowse to ADAMA endpoints Extending JBrowse REST feature store Handling Auth in JBrowse 1.x Use Case Integrate Chromatin States data into JBrowse via ADAMA Summary Challenges
  • 3. araport.org@araport JBrowse REST API spec Javascript based portable genome browser tool developed by GMOD community Ships with REST feature and names store adapter REST feature store requires access to following endpoints via HTTP GET: (base)/stats/global (base)/stats/region/(refseq)?start=123&end=456 (base)/stats/regionFeatureDensities/(refseq)?start=123&end=456&basesP erBin=20000 (base)/features/(refseq)?start=234&end=5678 Can provide feature, sequence, and quantitative data for display Accepted response type: application/json http://gmod.org/wiki/JBrowse_Configuration_Guide#Writing_JBrowse-compatible_Web_Services
  • 4. araport.org@araport ADAMA endpoints Araport Data And Microservices API Developed by WalterM and MattV Encapsulates python code into reusable docker containers Exposes internal logic as API endpoints: query, generic adapters via HTTP GET (base)/search (base)/list passthrough adapters via HTTP GET and POST (base)/access Implements token based Auth via iPlant OAuth2 https://github.com/Arabidopsis-Information-Portal/adama
  • 5. araport.org@araport Mapping JBrowse to ADAMA endpoints ADAMA (currently) doesnt allow defining custom endpoints So, we have to map the 4 JBrowse endpoints to the 2 ADAMA endpoints Proposed endpoint mapping: JBrowse ADAMA (base)/stats/global (base)/search?q=global (base)/stats/region (base)/search?q=region (base)/stats/regionFeatureDensities (base)/search?q=regionFeatureDensities (base)/features (base)/search?q=features (base)/list?q=makeTrackListJson
  • 6. araport.org@araport Extending JBrowse REST feature store JBrowse/Store/SeqFeature/REST handles requests from the browser to the configured API service Understands the 4 pre-defined endpoints REST feature store extended based on proposed endpoint mapping Utilize JBrowse Plugin architecture to develop Araport plugin: Araport/Store/SeqFeature/REST http://gmod.org/wiki/JBrowse_Configuration_Guide#Writing_JBrowse_Plugins
  • 7. araport.org@araport Handling Auth in JBrowse 1.x ADAMA (currently) requires all requests to be accompanied by an Authorization Bearer $TOKEN JBrowse 1.x does not support OAuth2 implicit flow, thus cannot generate tokens Solution: OAuth2 $TOKEN created with unlimited validity (-1) $TOKEN generated by Araport anonymous user $TOKEN scope limited to ADAMA services Araport/Store/SeqFeature/RESTAuth hard codes $TOKEN Araport/Store/SeqFeature/REST inherits from RESTAuth
  • 8. araport.org@araport Use Case Integrate Chromatin States data into JBrowse via ADAMA Published dataset representing a landscape of Arabidopsis thaliana chromatin states using 16 features, including DNA sequence, CG methylation, histone variants, and modifications Data represents chromosomal regions (start/stop coordinates) corresponding with 9 distinct chromatin states Published with manuscript as supplementary dataset (in XLS format)Sequeira-Mendes, J., Arag端ez, I., Peir坦, R., Mendez-Giraldez, R., Zhang, X., Jacobsen, S. E., ... & Gutierrez, C. (2014). The functional topography of the Arabidopsis genome is organized in a reduced number of linear motifs of chromatin states. The Plant Cell, 26(6), 2351-2366.
  • 9. araport.org@araport Develop ADAMA adapter to dynamically query data from desired region and represent as JBrowse compatible JSON Code: https://github.com/Arabidopsis-Information- Portal/chromatin_states_to_jbrowse Implementation logic: Data transformed from 9 XLS worksheets into individual GFF3 files Pythonic parser developed using gffutils, allowing range querying (via intermediary sqlite database) ADAMA metadata.yml written which describes dependencies, endpoints and provenance Data hosted with code (due to small size, well within GitHub file size limits) ADAMA namespace created and adapter registered Available in API explorer (search for chromatin) Use Case Integrate Chromatin States data into JBrowse via ADAMA https://jira.araport.org/browse/MINE-1078
  • 10. araport.org@araport Integrate Chromatin States data into JBrowse via ADAMA JBrowse tracks configured to talk to registered ADAMA endpoint End result visible in screenshot above Live view: https://apps.araport.org/jbrowse/?data=arabidopsis&loc=Chr3:11355449..11407858&tracks=chromatin_state_1,chr omatin_state_2,chromatin_state_3,chromatin_state_4,chromatin_state_5,chromatin_state_6,chromatin_state_7,chr omatin_state_8,chromatin_state_9,Araport11_gene_models
  • 11. araport.org@araport Summary Extended JBrowse to integrate with ADAMA endpoints Exposed a published community dataset via ADAMA as an API Exposed track configuration via ADAMA API Visualized dataset in JBrowse via ADAMA API
  • 12. araport.org@araport Challenges Making web service calls to ADAMA requires Authorization $TOKEN JBrowse needs to hard code the Authorization $TOKEN Track configuration needs to be manually added to JBrowse Scaling to larger datasets not feasible via GitHub route
  • 13. araport.org@araport Acknowledgements JCVI Araport Team Chia-Yi Cheng Agnes Chan Chris Town TACC Araport Team Walter Moreira Matthew Vaughn Prof. Crisanto Gutierrez (CBMSO)