Execution Environments
0 likes433 views
Talk from GA4GH meeting at ASHG/San Diego meeting about support for efficiently executing code on top of the GA4GH APIs.
1 of 4
Download to read offline
Recommended
Goodbye Flash, Hello OpenFL: Next Generation Cross-Platform Game Development 
Goodbye Flash, Hello OpenFL: Next Generation Cross-Platform Game Development Jessica Tams
油
Delivered at Casual Connect Tel Aviv | Haxe is an open source language with syntax similar to Actionscript, with some major improvements. Games written in Haxe can target many environments including Flash Player, HTML5, iOS and Android. OpenFL is an open source library built in Haxe, which provides a Flash-like API. The combination of Haxe and OpenFL is a natural fit for developers looking to move away from Actionscript/Flash. This talk will show you how. Data-Center Replication with Apache Accumulo
Data-Center Replication with Apache AccumuloJosh Elser
油
This document describes the implementation of data replication in Apache Accumulo. It discusses justifying the need for replication to handle failures, describes how replication is implemented using write-ahead logs, and outlines future work including replicating to other systems and improving consistency.Introduction to hubot
Introduction to hubotTencent
油
Hubot is a customizable robot assistant created with Node.js and CoffeeScript. It can interact with chat platforms like Campfire and IRC. Developers can build scripts to give Hubot new abilities like unlocking doors, finding apartment rentals, getting weather forecasts, and more. The document demonstrates how to configure Hubot and add scripts to extend its functionality.Affective domain
Affective domainjksofy
油
This document discusses interests and how they develop in children. It notes that interests have both subjective and objective aspects, with the subjective focusing on feelings and the objective on observable behaviors. Interests are generated through three types of learning - trial and error, identification with admired people, and guidance from others. A child's interests can be identified by observing their activities, questions, conversations, reading materials, drawings, wishes, and self reports. The document also states that all individuals have both inborn and acquired interests that show individual differences.Scalable Genome Analysis with ADAM
Scalable Genome Analysis with ADAMfnothaft
油
This document discusses scalable genome analysis using ADAM (Apache Spark-based framework). It begins by describing genomes and the goal of analyzing genetic variations. The document then discusses challenges like the large size of genomes and complexity of linking variations to traits. It proposes using ADAM's schema, optimized storage and algorithms to accelerate common access patterns like overlap joins. The document also emphasizes applying biological knowledge like protein grammars to make sense of non-coding variations. Finally, it acknowledges contributions from various institutions that have helped develop ADAM and its ability to enable genome analysis at scale.Rethinking Data-Intensive Science Using Scalable Analytics Systems 
Rethinking Data-Intensive Science Using Scalable Analytics Systems fnothaft
油
Presentation from SIGMOD 2015. With Matt Massie, Timothy Danford, Zhao Zhang, Uri Laserson, Carl Yeksigian, Jey Kottalam, Arun Ahuja, Jeff Hammerbacher, Michael Linderman, Michael J. Franklin, Anthony D. Joseph, David A. Patterson. Paper at http://dl.acm.org/citation.cfm?id=2742787.Scalable Genome Analysis With ADAM
Scalable Genome Analysis With ADAMfnothaft
油
This document provides a summary of the Scalable Genome Analysis with ADAM project. ADAM is an open-source, high-performance, distributed platform for genomic analysis that defines a data schema, data layout on disk, and programming interface for distributed processing of genomic data using Spark and Scala. The goal of ADAM is to integrate across terabyte and petabyte-scale datasets to enable the discovery of low frequency genetic variants linked to traits and diseases.Fast Variant Calling with ADAM and avocado
Fast Variant Calling with ADAM and avocadofnothaft
油
際際滷s from the BioBankCloud Hadoop/NGS Workshop at KTH Kista on 2/19/2015. Discusses early work on variant calling using ADAM and avocado.Scaling Genomic Analyses
Scaling Genomic Analysesfnothaft
油
ADAM is an open source, high performance, distributed platform for genomic analysis that defines a data schema and layout on disk using Parquet and Avro, integrates with Spark's Scala and Java APIs, and provides a command line interface. ADAM achieves linear scalability out to 128 nodes for most tasks and provides a 2-4x performance improvement over other tools like GATK and samtools. The platform includes various tools like avocado for efficient local variant calling via de Bruijn graph reassembly of sequencing reads.Scaling up genomic analysis with ADAM
Scaling up genomic analysis with ADAMfnothaft
油
際際滷s from DNAnexus about engineering/system design principles from ADAM, as well as emerging work in long-read OLC assembly.Scaling up genomic analysis with ADAM
Scaling up genomic analysis with ADAMfnothaft
油
The document discusses ADAM, a new framework for scalable genomic data analysis. It aims to make genomic pipelines horizontally scalable by using a columnar data format and in-memory computing. This avoids disk I/O bottlenecks. The framework represents genomic data as schemas and stores data in Parquet for efficient column-based access. It has been shown to reduce genome analysis pipeline times from 100 hours to 1 hour by enabling analysis on large datasets in parallel across many nodes.Reproducible Emulation of Analog Behavioral Models
Reproducible Emulation of Analog Behavioral Modelsfnothaft
油
1) Analog behavioral models are abstracted using SystemVerilog real numbers to allow simulation in digital emulation environments with higher throughput.
2) Key challenges to emulating analog models include converting floating-point implementations to fixed-point and handling high sampling rates in filters.
3) The document describes techniques used by Broadcom to synthesize analog behavioral models for emulation, including pragmas for sensitivity analysis and parallelizing filters.Scalable up genomic analysis with ADAM
Scalable up genomic analysis with ADAMfnothaft
油
ADAM is an open source platform for scalable genomic analysis that defines a data schema, Scala API, and command line interface. It uses Apache Spark for efficient parallel and distributed processing of large genomic datasets stored in Parquet format. Key features of ADAM include its ability to perform iterative analysis on whole genome datasets while minimizing data movement through Spark. The document also describes using ADAM and PacMin for long read assembly through techniques like minhashing for fast read overlapping and building consensus sequences on read graphs.CS176: Genome Assembly
CS176: Genome Assemblyfnothaft
油
The document discusses genome assembly from sequencing reads. It describes how reads can be aligned to a reference genome if available, but for a new genome the reads must be assembled without a reference. Two main assembly approaches are described: overlap-layout-consensus which builds an overlap graph, and de Brujin graph assembly which constructs a de Brujin graph from k-mers. Both approaches aim to find contiguous sequences (contigs) from the reads but face challenges from computational complexity and sequencing errors in the reads.PacMin @ AMPLab All-Hands
PacMin @ AMPLab All-Handsfnothaft
油
Describes early work on PacMin, a ploidy aware overlap/string based approach for assembling genomes from long reads.Design for Scalability in ADAM
Design for Scalability in ADAMfnothaft
油
ADAM is an open source, high performance, distributed platform for genomic analysis built on Apache Spark. It defines a Scala API and data schema using Avro and Parquet to store data in a columnar format, addressing the I/O bottleneck in genomics pipelines. ADAM implements common genomics algorithms as data or graph parallel computations and minimizes data movement by sending code to the data using Spark. It is designed to scale to processing whole human genomes across distributed file systems and cloud infrastructure.Adam bosc-071114
Adam bosc-071114fnothaft
油
ADAM is an open source, scalable genome analysis platform developed by researchers at UC Berkeley and other institutions. It includes tools for processing, analyzing and accessing large genomic datasets using Apache Spark. ADAM provides efficient data formats, rich APIs, and scalable algorithms to allow genome analysis to be performed on clusters and clouds. The goal is to enable fast, distributed analysis of genomic data across platforms while enhancing data access and flexibility.ADAMSpark Summit, 2014
ADAMSpark Summit, 2014fnothaft
油
ADAM is a scalable genome analysis platform that uses a column-oriented file format called Parquet to efficiently store and access large genomic datasets across distributed systems. It provides APIs and tools for transforming, analyzing, and querying genomic data in a scalable way using Apache Spark. Some key goals of ADAM include enabling efficient processing of genomes using clusters/clouds, providing a data format for parallel data access, and enhancing data semantics to allow more flexible access patterns.
INVESTIGATING TARDIGRADES RESISTANCE AS A MODEL FOR LIFE IN EXTREME SPACE ENV...
INVESTIGATING TARDIGRADES RESISTANCE AS A MODEL FOR LIFE IN EXTREME SPACE ENV...S辿rgio Sacani
油
Introduction: Tardigrades, microscopic extremophiles renowned for their exceptional resilience to hostile environments, have emerged as a pivotal model for astrobiological research and the exploration of life's potential beyond Earth. These organisms exhibit remarkable adaptability, surviving extreme conditions such as temperatures ranging from -271属C to over 150属C, pressures exceeding 1,200 times atmospheric levels, desiccation, and intense ionizing radiation. Their unique biological traits pose fundamental questions about the molecular and cellular mechanisms underpinning such resilience. Central to this adaptability are specific proteins, such as Dsup (Damage Suppressor), which mitigates radiation-induced DNA damage by forming a protective shield around genetic material, reducing double-strand breaks and preserving genomic integrity. 
Poison Apparatus and biting Mechanism of snakes.pptx
Poison Apparatus and biting Mechanism of snakes.pptxDhing College
油
Poison apparatus and biting mechanism of snakes 
Electronics in Physics Logic Gates and Rectification.pptx
Electronics in Physics Logic Gates and Rectification.pptxjhoyvan
油
Electronics in Physics Logic Gates and Rectification
IMMUNOMODULATORS: IMMUNOSTIMULATION AND IMMUNOSUPPRESSION .pptx
IMMUNOMODULATORS: IMMUNOSTIMULATION AND IMMUNOSUPPRESSION .pptxkarishmaduhijod1
油
Immunomodulators are natural or synthetic substances that help to regulate or normalize the immune system.
Natural Sciences Grade 9 learner textbook part b
Natural Sciences Grade 9 learner textbook part bWandaMbem
油
Natural Sciences Grade 9 learner textbook part bhpi homoeopathic pharmacopoeia of india...........
hpi homoeopathic pharmacopoeia of india...........Dr. Atul Rajgurav
油
It is the supreme (standard) authoritative book, published by the authority of government of any country, which deals with the scientific facts and logical attitude of the rules and regulations of standardisation of drug substances, containing directions for collecting drug materials from different sources, ...More Related Content
More from fnothaft (10)
Scaling Genomic Analyses
Scaling Genomic Analysesfnothaft
油
ADAM is an open source, high performance, distributed platform for genomic analysis that defines a data schema and layout on disk using Parquet and Avro, integrates with Spark's Scala and Java APIs, and provides a command line interface. ADAM achieves linear scalability out to 128 nodes for most tasks and provides a 2-4x performance improvement over other tools like GATK and samtools. The platform includes various tools like avocado for efficient local variant calling via de Bruijn graph reassembly of sequencing reads.Scaling up genomic analysis with ADAM
Scaling up genomic analysis with ADAMfnothaft
油
際際滷s from DNAnexus about engineering/system design principles from ADAM, as well as emerging work in long-read OLC assembly.Scaling up genomic analysis with ADAM
Scaling up genomic analysis with ADAMfnothaft
油
The document discusses ADAM, a new framework for scalable genomic data analysis. It aims to make genomic pipelines horizontally scalable by using a columnar data format and in-memory computing. This avoids disk I/O bottlenecks. The framework represents genomic data as schemas and stores data in Parquet for efficient column-based access. It has been shown to reduce genome analysis pipeline times from 100 hours to 1 hour by enabling analysis on large datasets in parallel across many nodes.Reproducible Emulation of Analog Behavioral Models
Reproducible Emulation of Analog Behavioral Modelsfnothaft
油
1) Analog behavioral models are abstracted using SystemVerilog real numbers to allow simulation in digital emulation environments with higher throughput.
2) Key challenges to emulating analog models include converting floating-point implementations to fixed-point and handling high sampling rates in filters.
3) The document describes techniques used by Broadcom to synthesize analog behavioral models for emulation, including pragmas for sensitivity analysis and parallelizing filters.Scalable up genomic analysis with ADAM
Scalable up genomic analysis with ADAMfnothaft
油
ADAM is an open source platform for scalable genomic analysis that defines a data schema, Scala API, and command line interface. It uses Apache Spark for efficient parallel and distributed processing of large genomic datasets stored in Parquet format. Key features of ADAM include its ability to perform iterative analysis on whole genome datasets while minimizing data movement through Spark. The document also describes using ADAM and PacMin for long read assembly through techniques like minhashing for fast read overlapping and building consensus sequences on read graphs.CS176: Genome Assembly
CS176: Genome Assemblyfnothaft
油
The document discusses genome assembly from sequencing reads. It describes how reads can be aligned to a reference genome if available, but for a new genome the reads must be assembled without a reference. Two main assembly approaches are described: overlap-layout-consensus which builds an overlap graph, and de Brujin graph assembly which constructs a de Brujin graph from k-mers. Both approaches aim to find contiguous sequences (contigs) from the reads but face challenges from computational complexity and sequencing errors in the reads.PacMin @ AMPLab All-Hands
PacMin @ AMPLab All-Handsfnothaft
油
Describes early work on PacMin, a ploidy aware overlap/string based approach for assembling genomes from long reads.Design for Scalability in ADAM
Design for Scalability in ADAMfnothaft
油
ADAM is an open source, high performance, distributed platform for genomic analysis built on Apache Spark. It defines a Scala API and data schema using Avro and Parquet to store data in a columnar format, addressing the I/O bottleneck in genomics pipelines. ADAM implements common genomics algorithms as data or graph parallel computations and minimizes data movement by sending code to the data using Spark. It is designed to scale to processing whole human genomes across distributed file systems and cloud infrastructure.Adam bosc-071114
Adam bosc-071114fnothaft
油
ADAM is an open source, scalable genome analysis platform developed by researchers at UC Berkeley and other institutions. It includes tools for processing, analyzing and accessing large genomic datasets using Apache Spark. ADAM provides efficient data formats, rich APIs, and scalable algorithms to allow genome analysis to be performed on clusters and clouds. The goal is to enable fast, distributed analysis of genomic data across platforms while enhancing data access and flexibility.ADAMSpark Summit, 2014
ADAMSpark Summit, 2014fnothaft
油
ADAM is a scalable genome analysis platform that uses a column-oriented file format called Parquet to efficiently store and access large genomic datasets across distributed systems. It provides APIs and tools for transforming, analyzing, and querying genomic data in a scalable way using Apache Spark. Some key goals of ADAM include enabling efficient processing of genomes using clusters/clouds, providing a data format for parallel data access, and enhancing data semantics to allow more flexible access patterns.Recently uploaded (20)
INVESTIGATING TARDIGRADES RESISTANCE AS A MODEL FOR LIFE IN EXTREME SPACE ENV...
INVESTIGATING TARDIGRADES RESISTANCE AS A MODEL FOR LIFE IN EXTREME SPACE ENV...S辿rgio Sacani
油
Introduction: Tardigrades, microscopic extremophiles renowned for their exceptional resilience to hostile environments, have emerged as a pivotal model for astrobiological research and the exploration of life's potential beyond Earth. These organisms exhibit remarkable adaptability, surviving extreme conditions such as temperatures ranging from -271属C to over 150属C, pressures exceeding 1,200 times atmospheric levels, desiccation, and intense ionizing radiation. Their unique biological traits pose fundamental questions about the molecular and cellular mechanisms underpinning such resilience. Central to this adaptability are specific proteins, such as Dsup (Damage Suppressor), which mitigates radiation-induced DNA damage by forming a protective shield around genetic material, reducing double-strand breaks and preserving genomic integrity. Poison Apparatus and biting Mechanism of snakes.pptx
Poison Apparatus and biting Mechanism of snakes.pptxDhing College
油
Poison apparatus and biting mechanism of snakes Electronics in Physics Logic Gates and Rectification.pptx
Electronics in Physics Logic Gates and Rectification.pptxjhoyvan
油
Electronics in Physics Logic Gates and RectificationIMMUNOMODULATORS: IMMUNOSTIMULATION AND IMMUNOSUPPRESSION .pptx
IMMUNOMODULATORS: IMMUNOSTIMULATION AND IMMUNOSUPPRESSION .pptxkarishmaduhijod1
油
Immunomodulators are natural or synthetic substances that help to regulate or normalize the immune system.Natural Sciences Grade 9 learner textbook part b
Natural Sciences Grade 9 learner textbook part bWandaMbem
油
Natural Sciences Grade 9 learner textbook part bhpi homoeopathic pharmacopoeia of india...........
hpi homoeopathic pharmacopoeia of india...........Dr. Atul Rajgurav
油
It is the supreme (standard) authoritative book, published by the authority of government of any country, which deals with the scientific facts and logical attitude of the rules and regulations of standardisation of drug substances, containing directions for collecting drug materials from different sources, ...
The JWST-NIRCamViewofSagittarius C. II. Evidence for Magnetically Dominated H...
The JWST-NIRCamViewofSagittarius C. II. Evidence for Magnetically Dominated H...S辿rgio Sacani
油
We present JWST-NIRCam narrowband, 4.05 亮mBr留 images of the Sgr C H II region, located in the central molecular zone (CMZ) of the Galaxy. Unlike any H II region in the solar vicinity, the Sgr C plasma is dominated by filamentary structure in both Br 留 and the radio continuum. Some bright filaments, which form a fractured arc with a radius of about 1.85 pc centered on the Sgr C star-forming molecular clump, likely trace ionization fronts. The brightest filaments form a -shaped structure in the center of the H II region. Fainter filaments radiate away from the surface of the Sgr C molecular cloud. The filaments are emitting optically thin freefree emission, as revealed by spectral index measurements from 1.28 GHz (MeerKAT) to 97GHz (Atacama Large Millimeter/ submillimeter Array). But, the negative in-band 1 to 2 GHz spectral index in the MeerKAT data alone reveals the presence of a nonthermal component across the entire Sgr C H II region. We argue that the plasma flow in Sgr C is controlled by magnetic fields, which confine the plasma to ropelike filaments or sheets. This results in the measured nonthermal component of low-frequency radio emission plasma, as well as a plasma 硫 (thermal pressure divided by magnetic pressure) below 1, even in the densest regions. We speculate that all mature H II regions in the CMZ, and galactic nuclei in general, evolve in a magnetically dominated, low plasma 硫 regime. Unified Astronomy Thesaurus concepts: Emission nebulae (461)
Data and Computing Infrastructure for the Life Sciences
Data and Computing Infrastructure for the Life SciencesChris Dwan
油
My slides from the 2025 Bio-IT World Expo.
I tried to lift above the churn to find constants that an architect or strategist could use to make well informed and durable technology choices.Accelerated Multi-Objective Alloy Discovery through Efficient Bayesian Method...
Accelerated Multi-Objective Alloy Discovery through Efficient Bayesian Method...Raymundo Arroyave
油
In this talk, I talk about BIRDSHOT, an integrated Bayesian materials discovery framework designed to efficiently explore complex compositional spaces while optimizing multiple material properties. We applied this framework to the CoCrFeNiVAl FCC high entropy alloy (HEA) system, targeting three key performance objectives: ultimate tensile strength/yield strength ratio, hardness, and strain rate sensitivity. The experimental campaign employed an integrated cyber-physical approach that combined vacuum arc melting (VAM) for alloy synthesis with advanced mechanical testing, including tensile and high-strain-rate nanoindentation testing. By incorporating batch Bayesian optimization schemes that allowed the parallel exploration of the alloy space, we completed five iterative design-make-test-learn loops, identifying a non-trivial three-objective Pareto set in a high-dimensional alloy space. Notably, this was achieved by exploring only 0.15% of the feasible design space, representing a significant acceleration in discovery rate relative to traditional methods. This work demonstrates the capability of BIRDSHOT to navigate complex, multi-objective optimization challenges and highlights its potential for broader application in accelerating materials discovery.Responsible Use of Research Metrics Module Launch
Responsible Use of Research Metrics Module Launchdri_ireland
油
Presentation by Dr Michelle Doran, National Open research Coordinator of the National Open Research Forum at the official launch of the Responsible Use of Research Metrics Module on 31 March 2025, at the Museum of Literature Ireland.Execution Environments
- 1. Execution Environments Frank Austin Nothaft
- 2. v0.5 API Support data access to reads/variants over REST Most existing applications using API are interactive
- 3. Batch Processing Is REST the correct approach? API is consistent for both local & remote data But, has overhead (perf + admin) for local data Approaches moving forward: Shims to current file formats Native interface to Hadoop ecosystem?
- 4. Common Workflow Language Pain point: how do we build reproducible pipelines of tasks? A group has started building a common workflow description language for bioinformatics: https://groups.google.com/forum/#!forum/ common-workflow-language Should the GA4GH take this task on?