ºÝºÝߣshows by User: kaxil / http://www.slideshare.net/images/logo.gif ºÝºÝߣshows by User: kaxil / Tue, 11 Jun 2024 12:08:00 GMT ºÝºÝߣShare feed for ºÝºÝߣshows by User: kaxil Orchestrating the Future: Navigating Today's Data Workflow Challenges with Airflow and Beyond | Budapest Data + ML Forum 2024 /slideshow/orchestrating-the-future-navigating-today-s-data-workflow-challenges-with-airflow-and-beyond-budapest-data-ml-forum-2024/269623476 orchestratingthefuturenavigatingtodaysdataworkflowchallengeswithairflowandbeyondbudapestdatamlforum2-240611120800-c7351736
Navigating today's data landscape isn't just about managing workflows; it's about strategically propelling your business forward. Apache Airflow has stood out as the benchmark in this arena, driving data orchestration forward since its early days. As we dive into the complexities of our current data-rich environment, where the sheer volume of information and its timely, accurate processing are crucial for AI and ML applications, the role of Airflow has never been more critical. In my journey as the Senior Engineering Director and a pivotal member of Apache Airflow's Project Management Committee (PMC), I've witnessed Airflow transform data handling, making agility and insight the norm in an ever-evolving digital space. At Astronomer, our collaboration with leading AI & ML teams worldwide has not only tested but also proven Airflow's mettle in delivering data reliably and efficiently—data that now powers not just insights but core business functions. This session is a deep dive into the essence of Airflow's success. We'll trace its evolution from a budding project to the backbone of data orchestration it is today, constantly adapting to meet the next wave of data challenges, including those brought on by Generative AI. It's this forward-thinking adaptability that keeps Airflow at the forefront of innovation, ready for whatever comes next. The ever-growing demands of AI and ML applications have ushered in an era where sophisticated data management isn't a luxury—it's a necessity. Airflow's innate flexibility and scalability are what makes it indispensable in managing the intricate workflows of today, especially those involving Large Language Models (LLMs). This talk isn't just a rundown of Airflow's features; it's about harnessing these capabilities to turn your data workflows into a strategic asset. Together, we'll explore how Airflow remains at the cutting edge of data orchestration, ensuring your organization is not just keeping pace but setting the pace in a data-driven future. Session in https://budapestdata.hu/2024/04/kaxil-naik-astronomer-io/ | https://dataml24.sessionize.com/session/667627]]>
Navigating today's data landscape isn't just about managing workflows; it's about strategically propelling your business forward. Apache Airflow has stood out as the benchmark in this arena, driving data orchestration forward since its early days. As we dive into the complexities of our current data-rich environment, where the sheer volume of information and its timely, accurate processing are crucial for AI and ML applications, the role of Airflow has never been more critical. In my journey as the Senior Engineering Director and a pivotal member of Apache Airflow's Project Management Committee (PMC), I've witnessed Airflow transform data handling, making agility and insight the norm in an ever-evolving digital space. At Astronomer, our collaboration with leading AI & ML teams worldwide has not only tested but also proven Airflow's mettle in delivering data reliably and efficiently—data that now powers not just insights but core business functions. This session is a deep dive into the essence of Airflow's success. We'll trace its evolution from a budding project to the backbone of data orchestration it is today, constantly adapting to meet the next wave of data challenges, including those brought on by Generative AI. It's this forward-thinking adaptability that keeps Airflow at the forefront of innovation, ready for whatever comes next. The ever-growing demands of AI and ML applications have ushered in an era where sophisticated data management isn't a luxury—it's a necessity. Airflow's innate flexibility and scalability are what makes it indispensable in managing the intricate workflows of today, especially those involving Large Language Models (LLMs). This talk isn't just a rundown of Airflow's features; it's about harnessing these capabilities to turn your data workflows into a strategic asset. Together, we'll explore how Airflow remains at the cutting edge of data orchestration, ensuring your organization is not just keeping pace but setting the pace in a data-driven future. Session in https://budapestdata.hu/2024/04/kaxil-naik-astronomer-io/ | https://dataml24.sessionize.com/session/667627]]> Tue, 11 Jun 2024 12:08:00 GMT /slideshow/orchestrating-the-future-navigating-today-s-data-workflow-challenges-with-airflow-and-beyond-budapest-data-ml-forum-2024/269623476 kaxil@slideshare.net(kaxil) Orchestrating the Future: Navigating Today's Data Workflow Challenges with Airflow and Beyond | Budapest Data + ML Forum 2024 kaxil Navigating today's data landscape isn't just about managing workflows; it's about strategically propelling your business forward. Apache Airflow has stood out as the benchmark in this arena, driving data orchestration forward since its early days. As we dive into the complexities of our current data-rich environment, where the sheer volume of information and its timely, accurate processing are crucial for AI and ML applications, the role of Airflow has never been more critical. In my journey as the Senior Engineering Director and a pivotal member of Apache Airflow's Project Management Committee (PMC), I've witnessed Airflow transform data handling, making agility and insight the norm in an ever-evolving digital space. At Astronomer, our collaboration with leading AI & ML teams worldwide has not only tested but also proven Airflow's mettle in delivering data reliably and efficiently—data that now powers not just insights but core business functions. This session is a deep dive into the essence of Airflow's success. We'll trace its evolution from a budding project to the backbone of data orchestration it is today, constantly adapting to meet the next wave of data challenges, including those brought on by Generative AI. It's this forward-thinking adaptability that keeps Airflow at the forefront of innovation, ready for whatever comes next. The ever-growing demands of AI and ML applications have ushered in an era where sophisticated data management isn't a luxury—it's a necessity. Airflow's innate flexibility and scalability are what makes it indispensable in managing the intricate workflows of today, especially those involving Large Language Models (LLMs). This talk isn't just a rundown of Airflow's features; it's about harnessing these capabilities to turn your data workflows into a strategic asset. Together, we'll explore how Airflow remains at the cutting edge of data orchestration, ensuring your organization is not just keeping pace but setting the pace in a data-driven future. Session in https://budapestdata.hu/2024/04/kaxil-naik-astronomer-io/ | https://dataml24.sessionize.com/session/667627 <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/orchestratingthefuturenavigatingtodaysdataworkflowchallengeswithairflowandbeyondbudapestdatamlforum2-240611120800-c7351736-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> Navigating today&#39;s data landscape isn&#39;t just about managing workflows; it&#39;s about strategically propelling your business forward. Apache Airflow has stood out as the benchmark in this arena, driving data orchestration forward since its early days. As we dive into the complexities of our current data-rich environment, where the sheer volume of information and its timely, accurate processing are crucial for AI and ML applications, the role of Airflow has never been more critical. In my journey as the Senior Engineering Director and a pivotal member of Apache Airflow&#39;s Project Management Committee (PMC), I&#39;ve witnessed Airflow transform data handling, making agility and insight the norm in an ever-evolving digital space. At Astronomer, our collaboration with leading AI &amp; ML teams worldwide has not only tested but also proven Airflow&#39;s mettle in delivering data reliably and efficiently—data that now powers not just insights but core business functions. This session is a deep dive into the essence of Airflow&#39;s success. We&#39;ll trace its evolution from a budding project to the backbone of data orchestration it is today, constantly adapting to meet the next wave of data challenges, including those brought on by Generative AI. It&#39;s this forward-thinking adaptability that keeps Airflow at the forefront of innovation, ready for whatever comes next. The ever-growing demands of AI and ML applications have ushered in an era where sophisticated data management isn&#39;t a luxury—it&#39;s a necessity. Airflow&#39;s innate flexibility and scalability are what makes it indispensable in managing the intricate workflows of today, especially those involving Large Language Models (LLMs). This talk isn&#39;t just a rundown of Airflow&#39;s features; it&#39;s about harnessing these capabilities to turn your data workflows into a strategic asset. Together, we&#39;ll explore how Airflow remains at the cutting edge of data orchestration, ensuring your organization is not just keeping pace but setting the pace in a data-driven future. Session in https://budapestdata.hu/2024/04/kaxil-naik-astronomer-io/ | https://dataml24.sessionize.com/session/667627
Orchestrating the Future: Navigating Today's Data Workflow Challenges with Airflow and Beyond | Budapest Data + ML Forum 2024 from Kaxil Naik
]]> 40 0 https://cdn.slidesharecdn.com/ss_thumbnails/orchestratingthefuturenavigatingtodaysdataworkflowchallengeswithairflowandbeyondbudapestdatamlforum2-240611120800-c7351736-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 Building and deploying LLM applications with Apache Airflow /slideshow/building-and-deploying-llm-applications-with-apache-airflow/261368477 buildinganddeployingllmapplicationswithapacheairflowairflowsummit2023-230924160626-a0406c2f
Behind the growing interest in Generate AI and LLM-based enterprise applications lies an expanded set of requirements for data integrations and ML orchestration. Enterprises want to use proprietary data to power LLM-based applications that create new business value, but they face challenges in moving beyond experimentation. The pipelines that power these models need to run reliably at scale, bringing together data from many sources and reacting continuously to changing conditions. This talk focuses on the design patterns for using Apache Airflow to support LLM applications created using private enterprise data. We’ll go through a real-world example of what this looks like, as well as a proposal to improve Airflow and to add additional Airflow Providers to make it easier to interact with LLMs such as the ones from OpenAI (such as GPT4) and the ones on HuggingFace, while working with both structured and unstructured data. In short, this shows how these Airflow patterns enable reliable, traceable, and scalable LLM applications within the enterprise. https://airflowsummit.org/sessions/2023/keynote-llm/ ]]>
Behind the growing interest in Generate AI and LLM-based enterprise applications lies an expanded set of requirements for data integrations and ML orchestration. Enterprises want to use proprietary data to power LLM-based applications that create new business value, but they face challenges in moving beyond experimentation. The pipelines that power these models need to run reliably at scale, bringing together data from many sources and reacting continuously to changing conditions. This talk focuses on the design patterns for using Apache Airflow to support LLM applications created using private enterprise data. We’ll go through a real-world example of what this looks like, as well as a proposal to improve Airflow and to add additional Airflow Providers to make it easier to interact with LLMs such as the ones from OpenAI (such as GPT4) and the ones on HuggingFace, while working with both structured and unstructured data. In short, this shows how these Airflow patterns enable reliable, traceable, and scalable LLM applications within the enterprise. https://airflowsummit.org/sessions/2023/keynote-llm/ ]]> Sun, 24 Sep 2023 16:06:26 GMT /slideshow/building-and-deploying-llm-applications-with-apache-airflow/261368477 kaxil@slideshare.net(kaxil) Building and deploying LLM applications with Apache Airflow kaxil Behind the growing interest in Generate AI and LLM-based enterprise applications lies an expanded set of requirements for data integrations and ML orchestration. Enterprises want to use proprietary data to power LLM-based applications that create new business value, but they face challenges in moving beyond experimentation. The pipelines that power these models need to run reliably at scale, bringing together data from many sources and reacting continuously to changing conditions. This talk focuses on the design patterns for using Apache Airflow to support LLM applications created using private enterprise data. We’ll go through a real-world example of what this looks like, as well as a proposal to improve Airflow and to add additional Airflow Providers to make it easier to interact with LLMs such as the ones from OpenAI (such as GPT4) and the ones on HuggingFace, while working with both structured and unstructured data. In short, this shows how these Airflow patterns enable reliable, traceable, and scalable LLM applications within the enterprise. https://airflowsummit.org/sessions/2023/keynote-llm/ <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/buildinganddeployingllmapplicationswithapacheairflowairflowsummit2023-230924160626-a0406c2f-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> Behind the growing interest in Generate AI and LLM-based enterprise applications lies an expanded set of requirements for data integrations and ML orchestration. Enterprises want to use proprietary data to power LLM-based applications that create new business value, but they face challenges in moving beyond experimentation. The pipelines that power these models need to run reliably at scale, bringing together data from many sources and reacting continuously to changing conditions. This talk focuses on the design patterns for using Apache Airflow to support LLM applications created using private enterprise data. We’ll go through a real-world example of what this looks like, as well as a proposal to improve Airflow and to add additional Airflow Providers to make it easier to interact with LLMs such as the ones from OpenAI (such as GPT4) and the ones on HuggingFace, while working with both structured and unstructured data. In short, this shows how these Airflow patterns enable reliable, traceable, and scalable LLM applications within the enterprise. https://airflowsummit.org/sessions/2023/keynote-llm/
Building and deploying LLM applications with Apache Airflow from Kaxil Naik
]]> 421 0 https://cdn.slidesharecdn.com/ss_thumbnails/buildinganddeployingllmapplicationswithapacheairflowairflowsummit2023-230924160626-a0406c2f-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 Introducing airflowctl: A CLI to streamline getting started with Airflow - Airflow Summit 2023 /slideshow/introducing-airflowctl-a-cli-to-streamline-getting-started-with-airflow-airflow-summit-2023/261368355 airflowctl-airflowsummit2023-230924160325-d0ea7e3c
New users starting with Airflow frequently encounter several challenges, ranging from the complexities of Containers and virtual environments to the Python dependency hell. Moreover, their familiarity with tools such as Docker, docker-compose, and Helm might be somewhat limited and even overkill. In contrast, seasoned Airflow users encounter their problems, encompassing configuration conflics with ongoing Airflow projects and intricacies stemming from Docker and docker-compose configurations and lack of visibility into all the projects. With airflowctl, users can install & setup Airflow using a single command. For existing users, they can use it to manage multiple Airflow projects with different Airflow versions on the same machine. This allows creating & debugging DAGs in an IDE seamlessly. https://airflowsummit.org/sessions/2023/introducing-airflowctl/ ]]>
New users starting with Airflow frequently encounter several challenges, ranging from the complexities of Containers and virtual environments to the Python dependency hell. Moreover, their familiarity with tools such as Docker, docker-compose, and Helm might be somewhat limited and even overkill. In contrast, seasoned Airflow users encounter their problems, encompassing configuration conflics with ongoing Airflow projects and intricacies stemming from Docker and docker-compose configurations and lack of visibility into all the projects. With airflowctl, users can install & setup Airflow using a single command. For existing users, they can use it to manage multiple Airflow projects with different Airflow versions on the same machine. This allows creating & debugging DAGs in an IDE seamlessly. https://airflowsummit.org/sessions/2023/introducing-airflowctl/ ]]> Sun, 24 Sep 2023 16:03:25 GMT /slideshow/introducing-airflowctl-a-cli-to-streamline-getting-started-with-airflow-airflow-summit-2023/261368355 kaxil@slideshare.net(kaxil) Introducing airflowctl: A CLI to streamline getting started with Airflow - Airflow Summit 2023 kaxil New users starting with Airflow frequently encounter several challenges, ranging from the complexities of Containers and virtual environments to the Python dependency hell. Moreover, their familiarity with tools such as Docker, docker-compose, and Helm might be somewhat limited and even overkill. In contrast, seasoned Airflow users encounter their problems, encompassing configuration conflics with ongoing Airflow projects and intricacies stemming from Docker and docker-compose configurations and lack of visibility into all the projects. With airflowctl, users can install & setup Airflow using a single command. For existing users, they can use it to manage multiple Airflow projects with different Airflow versions on the same machine. This allows creating & debugging DAGs in an IDE seamlessly. https://airflowsummit.org/sessions/2023/introducing-airflowctl/ <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/airflowctl-airflowsummit2023-230924160325-d0ea7e3c-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> New users starting with Airflow frequently encounter several challenges, ranging from the complexities of Containers and virtual environments to the Python dependency hell. Moreover, their familiarity with tools such as Docker, docker-compose, and Helm might be somewhat limited and even overkill. In contrast, seasoned Airflow users encounter their problems, encompassing configuration conflics with ongoing Airflow projects and intricacies stemming from Docker and docker-compose configurations and lack of visibility into all the projects. With airflowctl, users can install &amp; setup Airflow using a single command. For existing users, they can use it to manage multiple Airflow projects with different Airflow versions on the same machine. This allows creating &amp; debugging DAGs in an IDE seamlessly. https://airflowsummit.org/sessions/2023/introducing-airflowctl/
Introducing airflowctl: A CLI to streamline getting started with Airflow - Airflow Summit 2023 from Kaxil Naik
]]> 54 0 https://cdn.slidesharecdn.com/ss_thumbnails/airflowctl-airflowsummit2023-230924160325-d0ea7e3c-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 Airflow: Save Tons of Money by Using Deferrable Operators /slideshow/airflow-save-tons-of-money-by-using-deferrable-operators/252969083 airflowsavetonsofmoneybyusingdeferrableoperators1-220914113247-d31d2acd
This talk is from Open Source Summit 2022 Apache Airflow 2.2 introduced the concept of Deferrable Tasks that uses Python's async feature. All the Airflow sensors and poll-based operators can be hugely optimized to save tons of money by freeing up worker slots when polling. This session will cover the following topics: - Introduction to the concept of deferrable operator - Why do we need them? - When to use them? - How does it work? - Writing Custom deferrable operators & Sensors]]>
This talk is from Open Source Summit 2022 Apache Airflow 2.2 introduced the concept of Deferrable Tasks that uses Python's async feature. All the Airflow sensors and poll-based operators can be hugely optimized to save tons of money by freeing up worker slots when polling. This session will cover the following topics: - Introduction to the concept of deferrable operator - Why do we need them? - When to use them? - How does it work? - Writing Custom deferrable operators & Sensors]]> Wed, 14 Sep 2022 11:32:47 GMT /slideshow/airflow-save-tons-of-money-by-using-deferrable-operators/252969083 kaxil@slideshare.net(kaxil) Airflow: Save Tons of Money by Using Deferrable Operators kaxil This talk is from Open Source Summit 2022 Apache Airflow 2.2 introduced the concept of Deferrable Tasks that uses Python's async feature. All the Airflow sensors and poll-based operators can be hugely optimized to save tons of money by freeing up worker slots when polling. This session will cover the following topics: - Introduction to the concept of deferrable operator - Why do we need them? - When to use them? - How does it work? - Writing Custom deferrable operators & Sensors <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/airflowsavetonsofmoneybyusingdeferrableoperators1-220914113247-d31d2acd-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> This talk is from Open Source Summit 2022 Apache Airflow 2.2 introduced the concept of Deferrable Tasks that uses Python&#39;s async feature. All the Airflow sensors and poll-based operators can be hugely optimized to save tons of money by freeing up worker slots when polling. This session will cover the following topics: - Introduction to the concept of deferrable operator - Why do we need them? - When to use them? - How does it work? - Writing Custom deferrable operators &amp; Sensors
Airflow: Save Tons of Money by Using Deferrable Operators from Kaxil Naik
]]> 908 0 https://cdn.slidesharecdn.com/ss_thumbnails/airflowsavetonsofmoneybyusingdeferrableoperators1-220914113247-d31d2acd-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 Why Airflow? & What's new in Airflow 2.3? /slideshow/why-airflow-whats-new-in-airflow-23/251999483 whatsnewinairflow2-220616123143-8bb23fea
Talk: https://odsc.com/speakers/whats-new-in-apache-airflow-2-3/ This session talks about Why to use Apache Airflow & the awesome new features the community has built that were recently released in Apache Airflow 2.3. Highlights: - Dynamic Task Mapping - First-class support for DB Downgrades - Pruning old DB records (No need of using Maintenance DAGs anymore) - Building Connections using JSON - UI Improvements The talk will also cover the growth of Airflow Community over years and why Airflow is still the defacto tool for Workflow Orchestration.]]>
Talk: https://odsc.com/speakers/whats-new-in-apache-airflow-2-3/ This session talks about Why to use Apache Airflow & the awesome new features the community has built that were recently released in Apache Airflow 2.3. Highlights: - Dynamic Task Mapping - First-class support for DB Downgrades - Pruning old DB records (No need of using Maintenance DAGs anymore) - Building Connections using JSON - UI Improvements The talk will also cover the growth of Airflow Community over years and why Airflow is still the defacto tool for Workflow Orchestration.]]> Thu, 16 Jun 2022 12:31:42 GMT /slideshow/why-airflow-whats-new-in-airflow-23/251999483 kaxil@slideshare.net(kaxil) Why Airflow? & What's new in Airflow 2.3? kaxil Talk: https://odsc.com/speakers/whats-new-in-apache-airflow-2-3/ This session talks about Why to use Apache Airflow & the awesome new features the community has built that were recently released in Apache Airflow 2.3. Highlights: - Dynamic Task Mapping - First-class support for DB Downgrades - Pruning old DB records (No need of using Maintenance DAGs anymore) - Building Connections using JSON - UI Improvements The talk will also cover the growth of Airflow Community over years and why Airflow is still the defacto tool for Workflow Orchestration. <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/whatsnewinairflow2-220616123143-8bb23fea-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> Talk: https://odsc.com/speakers/whats-new-in-apache-airflow-2-3/ This session talks about Why to use Apache Airflow &amp; the awesome new features the community has built that were recently released in Apache Airflow 2.3. Highlights: - Dynamic Task Mapping - First-class support for DB Downgrades - Pruning old DB records (No need of using Maintenance DAGs anymore) - Building Connections using JSON - UI Improvements The talk will also cover the growth of Airflow Community over years and why Airflow is still the defacto tool for Workflow Orchestration.
Why Airflow? & What's new in Airflow 2.3? from Kaxil Naik
]]> 19 0 https://cdn.slidesharecdn.com/ss_thumbnails/whatsnewinairflow2-220616123143-8bb23fea-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 What's new in Airflow 2.3? /slideshow/whats-new-in-airflow-23/251870442 whatsnewinairflow2-220527175520-fd0c7f47
This session will talk about the awesome new features the community has built that are part of Apache Airflow 2.3. Highlights: - Dynamic Task Mapping - DB. Downgrades - Pruning old DB records - Connections using JSON - UI Improvements ]]>
This session will talk about the awesome new features the community has built that are part of Apache Airflow 2.3. Highlights: - Dynamic Task Mapping - DB. Downgrades - Pruning old DB records - Connections using JSON - UI Improvements ]]> Fri, 27 May 2022 17:55:20 GMT /slideshow/whats-new-in-airflow-23/251870442 kaxil@slideshare.net(kaxil) What's new in Airflow 2.3? kaxil This session will talk about the awesome new features the community has built that are part of Apache Airflow 2.3. Highlights: - Dynamic Task Mapping - DB. Downgrades - Pruning old DB records - Connections using JSON - UI Improvements <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/whatsnewinairflow2-220527175520-fd0c7f47-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> This session will talk about the awesome new features the community has built that are part of Apache Airflow 2.3. Highlights: - Dynamic Task Mapping - DB. Downgrades - Pruning old DB records - Connections using JSON - UI Improvements
What's new in Airflow 2.3? from Kaxil Naik
]]> 89 0 https://cdn.slidesharecdn.com/ss_thumbnails/whatsnewinairflow2-220527175520-fd0c7f47-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 Upgrading to Apache Airflow 2 | Airflow Summit 2021 /slideshow/upgrading-to-apache-airflow-2-airflow-summit-2021/249738942 upgradingtoapacheairflow2-210715011741
Airflow 2.0 was a big milestone for the Airflow community. However, companies and enterprises are still facing difficulties in upgrading to 2.0. In this talk, I would like to focus and highlight the ideal upgrade path and talk about - upgrade_check CLI tool - separation of providers - registering connections types - DB Migration - deprecated feature around Airflow Plugins https://airflowsummit.org/sessions/2021/upgrading-to-apache-airflow-2/ ]]>
Airflow 2.0 was a big milestone for the Airflow community. However, companies and enterprises are still facing difficulties in upgrading to 2.0. In this talk, I would like to focus and highlight the ideal upgrade path and talk about - upgrade_check CLI tool - separation of providers - registering connections types - DB Migration - deprecated feature around Airflow Plugins https://airflowsummit.org/sessions/2021/upgrading-to-apache-airflow-2/ ]]> Thu, 15 Jul 2021 01:17:41 GMT /slideshow/upgrading-to-apache-airflow-2-airflow-summit-2021/249738942 kaxil@slideshare.net(kaxil) Upgrading to Apache Airflow 2 | Airflow Summit 2021 kaxil Airflow 2.0 was a big milestone for the Airflow community. However, companies and enterprises are still facing difficulties in upgrading to 2.0. In this talk, I would like to focus and highlight the ideal upgrade path and talk about - upgrade_check CLI tool - separation of providers - registering connections types - DB Migration - deprecated feature around Airflow Plugins https://airflowsummit.org/sessions/2021/upgrading-to-apache-airflow-2/ <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/upgradingtoapacheairflow2-210715011741-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> Airflow 2.0 was a big milestone for the Airflow community. However, companies and enterprises are still facing difficulties in upgrading to 2.0. In this talk, I would like to focus and highlight the ideal upgrade path and talk about - upgrade_check CLI tool - separation of providers - registering connections types - DB Migration - deprecated feature around Airflow Plugins https://airflowsummit.org/sessions/2021/upgrading-to-apache-airflow-2/
Upgrading to Apache Airflow 2 | Airflow Summit 2021 from Kaxil Naik
]]> 102 0 https://cdn.slidesharecdn.com/ss_thumbnails/upgradingtoapacheairflow2-210715011741-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 Contributing to Apache Airflow | Journey to becoming Airflow's leading contributor /slideshow/contributing-to-apache-airflow-journey-to-becoming-airflows-leading-contributor/249687414 contributingtoapacheairflowjourneytobecomingairflowsleadingcontributor1-210711171604
From not knowing Python (let alone Airflow), and from submitting the first PR that fixes typo to becoming Airflow Committer, PMC Member, Release Manager, and #1 Committer this year, this talk walks through Kaxil’s journey in the Airflow World. The second part of this talk explains: how you can also start your OSS journey by contributing to Airflow Expanding familiarity with a different part of the Airflow codebase Continue committing regularly & steadily to become Airflow Committer. (including talking about current Guidelines of becoming a Committer) Different mediums of communication (Dev list, users list, Slack channel, Github Discussions etc)]]>
From not knowing Python (let alone Airflow), and from submitting the first PR that fixes typo to becoming Airflow Committer, PMC Member, Release Manager, and #1 Committer this year, this talk walks through Kaxil’s journey in the Airflow World. The second part of this talk explains: how you can also start your OSS journey by contributing to Airflow Expanding familiarity with a different part of the Airflow codebase Continue committing regularly & steadily to become Airflow Committer. (including talking about current Guidelines of becoming a Committer) Different mediums of communication (Dev list, users list, Slack channel, Github Discussions etc)]]> Sun, 11 Jul 2021 17:16:04 GMT /slideshow/contributing-to-apache-airflow-journey-to-becoming-airflows-leading-contributor/249687414 kaxil@slideshare.net(kaxil) Contributing to Apache Airflow | Journey to becoming Airflow's leading contributor kaxil From not knowing Python (let alone Airflow), and from submitting the first PR that fixes typo to becoming Airflow Committer, PMC Member, Release Manager, and #1 Committer this year, this talk walks through Kaxil’s journey in the Airflow World. The second part of this talk explains: how you can also start your OSS journey by contributing to Airflow Expanding familiarity with a different part of the Airflow codebase Continue committing regularly & steadily to become Airflow Committer. (including talking about current Guidelines of becoming a Committer) Different mediums of communication (Dev list, users list, Slack channel, Github Discussions etc) <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/contributingtoapacheairflowjourneytobecomingairflowsleadingcontributor1-210711171604-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> From not knowing Python (let alone Airflow), and from submitting the first PR that fixes typo to becoming Airflow Committer, PMC Member, Release Manager, and #1 Committer this year, this talk walks through Kaxil’s journey in the Airflow World. The second part of this talk explains: how you can also start your OSS journey by contributing to Airflow Expanding familiarity with a different part of the Airflow codebase Continue committing regularly &amp; steadily to become Airflow Committer. (including talking about current Guidelines of becoming a Committer) Different mediums of communication (Dev list, users list, Slack channel, Github Discussions etc)
Contributing to Apache Airflow | Journey to becoming Airflow's leading contributor from Kaxil Naik
]]> 125 0 https://cdn.slidesharecdn.com/ss_thumbnails/contributingtoapacheairflowjourneytobecomingairflowsleadingcontributor1-210711171604-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 Upcoming features in Airflow 2 /slideshow/upcoming-features-in-airflow-2/237553385 whatsnewinairflow2-kaxilnaik-200804190632
Speaker: Kaxil Naik Conference: PiterPy Date: 4 August 2020 Title: What's upcoming in Airflow 2?]]>
Speaker: Kaxil Naik Conference: PiterPy Date: 4 August 2020 Title: What's upcoming in Airflow 2?]]> Tue, 04 Aug 2020 19:06:32 GMT /slideshow/upcoming-features-in-airflow-2/237553385 kaxil@slideshare.net(kaxil) Upcoming features in Airflow 2 kaxil Speaker: Kaxil Naik Conference: PiterPy Date: 4 August 2020 Title: What's upcoming in Airflow 2? <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/whatsnewinairflow2-kaxilnaik-200804190632-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> Speaker: Kaxil Naik Conference: PiterPy Date: 4 August 2020 Title: What&#39;s upcoming in Airflow 2?
Upcoming features in Airflow 2 from Kaxil Naik
]]> 144 0 https://cdn.slidesharecdn.com/ss_thumbnails/whatsnewinairflow2-kaxilnaik-200804190632-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 What's coming in Airflow 2.0? - NYC Apache Airflow Meetup /slideshow/whats-coming-in-airflow-20/233838798 whatscominginairflow2-200513181159
Group of Airflow core committers talking about what's coming with Airflow 2.0! Speakers: Ash Berlin-Taylor, Kaxil Naik, Kamil Breguła Jarek Potiuk, Daniel Imberman and Tomasz Urbaszek.]]>
Group of Airflow core committers talking about what's coming with Airflow 2.0! Speakers: Ash Berlin-Taylor, Kaxil Naik, Kamil Breguła Jarek Potiuk, Daniel Imberman and Tomasz Urbaszek.]]> Wed, 13 May 2020 18:11:59 GMT /slideshow/whats-coming-in-airflow-20/233838798 kaxil@slideshare.net(kaxil) What's coming in Airflow 2.0? - NYC Apache Airflow Meetup kaxil Group of Airflow core committers talking about what's coming with Airflow 2.0! Speakers: Ash Berlin-Taylor, Kaxil Naik, Kamil Breguła Jarek Potiuk, Daniel Imberman and Tomasz Urbaszek. <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/whatscominginairflow2-200513181159-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> Group of Airflow core committers talking about what&#39;s coming with Airflow 2.0! Speakers: Ash Berlin-Taylor, Kaxil Naik, Kamil Breguła Jarek Potiuk, Daniel Imberman and Tomasz Urbaszek.
What's coming in Airflow 2.0? - NYC Apache Airflow Meetup from Kaxil Naik
]]> 202 0 https://cdn.slidesharecdn.com/ss_thumbnails/whatscominginairflow2-200513181159-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 Airflow Best Practises & Roadmap to Airflow 2.0 /slideshow/airflow-best-practises-roadmap-to-airflow-20/233521197 talk1-airflowbestpracticesroadmapto2-200510190455
- Best Practises on writing DAG for Apache Airflow. - New features in Airflow 1.10.8 and 1.10.9 - Roadmap to Airflow 2.0]]>
- Best Practises on writing DAG for Apache Airflow. - New features in Airflow 1.10.8 and 1.10.9 - Roadmap to Airflow 2.0]]> Sun, 10 May 2020 19:04:55 GMT /slideshow/airflow-best-practises-roadmap-to-airflow-20/233521197 kaxil@slideshare.net(kaxil) Airflow Best Practises & Roadmap to Airflow 2.0 kaxil - Best Practises on writing DAG for Apache Airflow. - New features in Airflow 1.10.8 and 1.10.9 - Roadmap to Airflow 2.0 <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/talk1-airflowbestpracticesroadmapto2-200510190455-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> - Best Practises on writing DAG for Apache Airflow. - New features in Airflow 1.10.8 and 1.10.9 - Roadmap to Airflow 2.0
Airflow Best Practises & Roadmap to Airflow 2.0 from Kaxil Naik
]]> 475 0 https://cdn.slidesharecdn.com/ss_thumbnails/talk1-airflowbestpracticesroadmapto2-200510190455-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 Apache Airflow in the Cloud: Programmatically orchestrating workloads with Python - PyData London 2018 /slideshow/apache-airflow-in-the-cloud-programmatically-orchestrating-workloads-with-python-pydata-london-2018-95391267/95391267 airflowpydata-180428232516
Introducing the basics of Airflow and how to orchestrate workloads on Google Cloud Platform (GCP).]]>
Introducing the basics of Airflow and how to orchestrate workloads on Google Cloud Platform (GCP).]]> Sat, 28 Apr 2018 23:25:15 GMT /slideshow/apache-airflow-in-the-cloud-programmatically-orchestrating-workloads-with-python-pydata-london-2018-95391267/95391267 kaxil@slideshare.net(kaxil) Apache Airflow in the Cloud: Programmatically orchestrating workloads with Python - PyData London 2018 kaxil Introducing the basics of Airflow and how to orchestrate workloads on Google Cloud Platform (GCP). <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/airflowpydata-180428232516-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> Introducing the basics of Airflow and how to orchestrate workloads on Google Cloud Platform (GCP).
Apache Airflow in the Cloud: Programmatically orchestrating workloads with Python - PyData London 2018 from Kaxil Naik
]]> 3009 3 https://cdn.slidesharecdn.com/ss_thumbnails/airflowpydata-180428232516-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 https://cdn.slidesharecdn.com/profile-photo-kaxil-48x48.jpg?cb=1718107641 Committer & PMC Member, Apache Airflow. Open-source advocate. https://cdn.slidesharecdn.com/ss_thumbnails/orchestratingthefuturenavigatingtodaysdataworkflowchallengeswithairflowandbeyondbudapestdatamlforum2-240611120800-c7351736-thumbnail.jpg?width=320&height=320&fit=bounds slideshow/orchestrating-the-future-navigating-today-s-data-workflow-challenges-with-airflow-and-beyond-budapest-data-ml-forum-2024/269623476 Orchestrating the Futu... https://cdn.slidesharecdn.com/ss_thumbnails/buildinganddeployingllmapplicationswithapacheairflowairflowsummit2023-230924160626-a0406c2f-thumbnail.jpg?width=320&height=320&fit=bounds slideshow/building-and-deploying-llm-applications-with-apache-airflow/261368477 Building and deploying... https://cdn.slidesharecdn.com/ss_thumbnails/airflowctl-airflowsummit2023-230924160325-d0ea7e3c-thumbnail.jpg?width=320&height=320&fit=bounds slideshow/introducing-airflowctl-a-cli-to-streamline-getting-started-with-airflow-airflow-summit-2023/261368355 Introducing airflowctl...