devopstools

Sep 12, 20150 likes365 views

This document discusses the evolution of DevOps tools at AppsFlyer. It notes that AppsFlyer processes around 4.9 billion events per day across 120 services running on elastic computing resources. It has a single development team of 25 people that builds tools to meet operational needs, extend existing tools, learn new technologies, and improve collaboration between development and operations teams. Two examples of internally developed tools that will be demonstrated are Sniffer, a mapping tool, and Sensu Reports, which aggregates reports from the Sensu monitoring system.

Today's Menu
●
AppsFlyer in numbers
●
Rnd structure in AppsFlyer
●
Our DevOps philosophy
●
Motivation for creating DevOps tools
●
Setting requirements and project assignment
●
Example for tools we built internally
●
Live demo and the developer angle

AppsFlyer in numbers
●
~ 4.9 billions events per day
●
~ 120 services
●
Elastic hundreds to 1K instances
●
Dozens of deployments per day
●
25 people in development
●
1 team

Motivations
●
Actual operational needs
●
Extend existing tools
●
Learn new technologies
●
Familiarize the production
environment for new Rnd
members
●
Blur the line between Dev
and Ops
●
Improve the bonding inside
the team

Considerations
●
Needs, why do we need this project?
●
Usability – How do I see myself using the project?
●
The target audience.
●
The developer's background (Front, Mobile, etc)
●
Will the developer be better familiar with production environment?
●
How are we going to maintain the project?
●
What are the time constraints?
●
Can we roll it out in stages?
●
Can we release it outside?

Today's Examples
●
Sniffer: Mapping tool
(Artyom Shein)
●
Sensu Reports:
Aggregated reports of
Sensu
(Avner Schwartz)
●
Bouncer: Authentication
and Authorization
(Reshef Mann)

The document discusses tools and workflows for improving collaboration between designers and developers, highlighting Sketch for its scalability and community plugins, and InVision for its prototyping and collaboration features. It also mentions Framer for coding interactions and emphasizes the importance of establishing ownership and communication throughout the design process. Key steps include building a style guide, creating prototypes, and maintaining up-to-date assets.

Build, deploy and scale: Django, GraphQL and SPA (DjangoCon EU 2021)Dhilipsiva DS

��

The document outlines a developer's experience with Django, GraphQL, and SPA technologies, highlighting benefits like precise queries and standardized documentation. It discusses specific tools such as React and Relay for their compatibility with GraphQL and the project's structure for Django applications. Additionally, it details deployment methods using Docker and AWS, emphasizing cost-effectiveness and scalability.

[WSO2Con USA 2018] Chipping Away at Technical Debt with WSO2WSO2

��

The document outlines Micron Technology's transition from monolithic client/server applications to an API-first strategy using the WSO2 API Gateway. It details the steps taken to modernize their application infrastructure, the challenges faced, and the lessons learned during implementation, including security measures and performance management. The transition aims to improve developer experience and achieve high availability across global deployments through effective training and process automation.

Agile Development: Key to smart software developmentJerlyn Manohar

��

The document discusses key aspects of adopting an agile development workflow including having daily stand up meetings to discuss progress, taking an iterative development approach with working software delivered each sprint, and conducting retrospectives at the end of each sprint to improve. Adopting agile practices helps ensure working software is delivered frequently for stakeholder feedback, allows skills and the product to evolve together, and prevents waste through continuous testing.

t3kit - kickstart your TYPO3 projects in minutesJozef Spisiak

��

Pixelant, a full-service agency with 10 years of experience in TYPO3, offers a standardized deployment package aimed at enhancing development efficiency for agencies and clients. Their approach includes a range of tools such as Docker, automated testing, and a variety of preconfigured extensions to facilitate website projects of all sizes. Future plans involve developing a more modular theme, focusing on editor experience, and enhancing community contributions.

Continuous delivery journey at LesFurets.comRaphaël Brugier

��

The document details a continuous delivery journey of a Richmond, VA-based insurance comparison website launched in 2012, highlighting its shift from a monolithic architecture to a more agile delivery process. With 4 feature teams and an emphasis on rapid builds and releases, the development workflow includes automated deployment and thorough validation processes. Key metrics and tools such as Jenkins, Selenium, and Git flow are utilized to enhance reliability and speed in delivering valuable software to customers.

(Sprint 1) besser childress logerquist-swainKristadeSwain

��

The document summarizes the goals and achievements of Sprint 1 of a project to create an employee suggestion system mobile app. The team created login and registration pages, employee and reviewer views, and pulled ideas from a database to display dynamically. They aimed to complete the app framework but did not finish reviewing or commenting on ideas. The team encountered challenges with the Android development workflow, documentation, implementing mockups, and version control.

El Director de Proyectos ÁgilJose Barato

��

This document discusses agile project management. It begins with an introduction of Jose Barato and his background. It then contrasts the traditional waterfall approach with the agile approach. Some key aspects of agile covered include product backlogs, the agile paradigm of being value-driven and adaptive. The document also references the agile manifesto and common agile methodologies like Scrum. It provides overviews of the agile product life cycle and roles in agile project management like the product owner and Scrum master.

How do you agile your global team to contribute to openstackAlexis Monville

��

This document discusses how the company eNovance uses agile practices to manage their global team working on OpenStack. Key points include: - Using Scrum values and framework to manage distributed teams across multiple time zones. Communication tools include instant messaging, video conferencing, wikis. - Onboarding for remote workers includes assigning a buddy, self-training programs, and 2-4 weeks in an office. An Agile Guild shares skills across teams. - Contributions to OpenStack must be merged upstream before deployment. Real open source involvement is emphasized. - Agile practices like 2-week sprints and limiting work in progress are applied to both internal development and customer projects involving discovery workshops and iterative design

CUHK CSCI 4140 2015 Spring Guest Lecture - Agile DevelopmentWong Hoi Sing Edison

��

The document summarizes a guest lecture on agile development given by Edison Wong. It defines agile development and describes popular agile methods like Scrum and Kanban. It compares Scrum and Kanban and outlines why agile development is preferable to traditional waterfall methods. The lecture also included a 15 minute tutorial on using JIRA for agile projects and covered advanced topics like branching strategies, code reviews, and continuous integration.

Career_ModelRocco Cutrupi

��

This document outlines a typical career progression model for software developers and graphic designers. For software developers, the progression usually moves from trainee developer to junior developer after 1-2 years, to software developer after 3-4 years, to senior developer after 6-8 years, and potentially to project manager thereafter. For graphic designers, the progression typically goes from trainee graphic designer to junior graphic designer after 2 years, to graphic designer after 4-6 years, and to senior graphic designer thereafter.

(Open Hack Night Fall 2014) OverviewJames Griffin

��

This document outlines an open hack night hosted by Lafayette College Libraries. It describes the objectives to resolve bugs through development and propose improvements through user experience design. Teams will be formed using Scrum methodology and roles assigned as team manager, product manager, designers, and developers. Participants will use Git and GitHub for version control and collaboration. The night will involve hacking the site structure as developers and hacking the site design as designers. Achievements will be rewarded with points for bug resolutions, documentation, and design deliverables.

Agile Testing Days 2015 - PIMP MY SCRUM – DOCUMENTATION FUN FOR AGILE TEAMSAnis Ben Hamidene

��

The document provides an overview of best practices for creating effective documentation in agile environments, focusing on defining clear goals, sustainable documentation strategies, and integrating documentation into the development lifecycle. It emphasizes the importance of collaboration among team members and the need for living documentation that adapts through examples and automation. Additionally, it discusses various tools and frameworks to enhance documentation efforts and ensure quality throughout the project.

BITS 2015: The Beauty of DrupalGerald Villorente

��

The document discusses the popularity and demand for Drupal, an open source content management system (CMS). It notes that Drupal is used by many large organizations and provides flexibility, security, and scalability. There is a growing demand for Drupal engineers as businesses increasingly adopt open source solutions. Drupal can be used to build various types of websites and applications, providing opportunities for career growth and high salaries within a large supportive community.

Building awesome sample apps that actually provide valueLauren Hayward Schaefer

��

The document outlines a guide for building effective sample applications that add value, focusing on empowering developers and showcasing best practices. It details a step-by-step approach including identifying project goals, creating scenarios, defining MVP requirements, implementing projects, and engaging the developer community. Additionally, it emphasizes the importance of measuring success and avoiding common pitfalls in the development process.

Lessons learned from 6 month project with india based software houseNathan Krasney

��

The document outlines lessons learned from a six-month project transitioning a Point of Sale system from Windows to Android. Key tips include finding a competitive software house in India, maintaining daily communication, using short milestones, breaking tasks into smaller segments, and managing design independently. Overall, effective management and clear communication were emphasized for successful project outcomes.

Yuriy Golikov, Wrike: Team of teams how to organize effective development of...Anton Anokhin

��

The document outlines the transformation of a product development framework at �ݺ�ߣwrike, from functional teams to cross-functional teams using Scrum starting in 2016. This shift aimed to expedite feature development, enhance scalability, and increase transparency within teams. Results showed high employee satisfaction and engagement post-implementation, with a significant portion of the workforce expressing a desire to continue working at the company.

Running an Agile Project with OdooOdoo

��

This document summarizes an presentation about running agile projects within Odoo. It discusses the benefits of using agile methods like Scrum and Kanban such as better quality, higher customer satisfaction, and increased collaboration. It compares Scrum and Kanban approaches, highlighting that Scrum uses sprints while Kanban is more continuous. It also introduces the brain-tec backlog module for managing backlogs in Odoo projects using agile principles.

Beyond the web: Mobile apps using Drupal & Ionic 2 - Drupal Dev Days Seville ...La Drupalera

��

The document outlines a workshop focused on creating mobile applications using Drupal and Ionic 2, emphasizing hands-on learning experiences. Participants will learn about hybrid applications, Drupal's RESTful capabilities, and will build an app to view the DrupalDevDays schedule with additional features like event calendar integration and push notifications. The creators, experienced developers in frontend and QA fields, encourage collaboration and enjoyment during the learning process.

General presentation - Bitcraft Kamila Katyal

��

BitCraft is a software development company with over 30 years of combined experience. Their mission is to create innovative solutions through a team of professionals including programmers, designers, and project managers. They use modern tools and processes like Scrum methodology, Git, Docker, and Jenkins in their development. BitCraft provides custom software, mobile apps, and dedicated development teams to startups, companies, agencies, and individuals.

2018-10-23 3C - Lean, Scrum and low code approach of SharePoint and O365 proj...aOS Community

��

The document discusses an approach to SharePoint and Office 365 projects using Lean, Scrum and low code methods. It describes a project for a banking sector to rebuild a deal decision management system using these approaches. Key aspects covered include prioritizing features, sticking to standards, challenging assumptions to reduce scope, and delivering in iterations to get early results. The project was delivered on time and under budget with satisfied stakeholders.

DevOps OverviewOmri Spector

��

The document discusses the concept of DevOps as a cultural shift that enhances collaboration between development and operations teams while emphasizing the need for automation, continuous integration, and reliable deployment practices. It highlights key principles such as treating all operations as code and prioritizing a common goal to enable business efficiency. Additionally, it outlines practical tools and methodologies that facilitate the implementation of DevOps within organizations, particularly in software as a service (SaaS) environments.

DevOps Days OhioKelly Looney

��

This document discusses the efforts of a €600M online gaming company to adopt DevOps practices across their organization. They had previously merged two gaming platforms with different architectures and teams worked in silos. To improve, they have undertaken an agile transformation, encouraged cultural changes in developers to take more ownership, adopted new tools like Git, Jenkins, Puppet and AppDynamics for monitoring, and are moving to containerize applications. Monitoring in particular has helped improve quality by providing visibility. They are also working to improve testing environments to better simulate production. The company aims to continuously deploy changes through proliferating environments and automating infrastructure as code.

Gartner Infrastructure and Operations Summit Berlin 2015 - DevOps JourneyKelly Looney

��

The document discusses the DevOps journey of a large online gaming company with separate sports betting and poker/casino divisions that were merged. It outlines the challenges of integrating different code bases and cultures. Key steps taken include adopting Agile, moving to Git/Jenkins, implementing monitoring with AppDynamics, and moving to containers. Automating testing environments and adopting continuous delivery principles helped improve quality and allow smaller, more frequent changes. Monitoring provided visibility and helped identify issues and refactoring needs. The changes helped bring development and operations teams together.

Devops Mindset EssentialsWilly-Peter Schaub

��

The document outlines best practices for DevOps, emphasizing the importance of continuous integration and delivery, effective monitoring, and user-centric innovations to enhance application delivery outcomes. It describes the critical metrics for assessing performance, such as deployment frequency and change failure rate, and stresses the need for a collaborative culture focused on customer value. By adopting agile methodologies and automated processes, teams can achieve faster feedback, increased deployment velocity, and improved overall satisfaction for both customers and employees.

SaltConf14 - Justin Carmony, Deseret Digital Media - Teaching Devs About DevOpsSaltStack

��

Justin Carmony discusses the journey of integrating DevOps practices at Deseret Digital Media, emphasizing the importance of collaboration between developers and operations teams. He outlines challenges faced, such as a fragmented infrastructure, and the company's strategy to promote from within and spread DevOps knowledge among developers. The presentation also highlights the need for enhanced metrics, automated deployments, and a culture of continuous learning and responsibility for code performance.

Implementing Azure DevOps with your Testing ProjectRTTS

��

The document outlines the role of Azure DevOps in enhancing testing and quality assurance in software development, highlighting its integration with agile methodologies and DevOps culture. It emphasizes collaborative approaches, constant adjustments, and the importance of automated testing for delivering high-quality software efficiently. Additionally, it presents RTTS as a leading firm in test automation solutions since 1996, serving over 700 clients, including Fortune 500 companies.

DevOps and ToolsMohammed Fazuluddin

��

DevOps Powered by SplunkSplunk

��

The document outlines the complexities of DevOps workflows powered by Splunk, detailing processes from planning and coding to testing and monitoring. It emphasizes the integration of various tools for continuous integration and delivery, as well as the importance of visibility and speed in development and operations. Additionally, it discusses practical use cases and the role of Splunk in transforming DevOps practices for successful business outcomes.

WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOpsWinOps Conf

��

The document discusses the challenges and scenarios associated with implementing DevOps, emphasizing the importance of collaboration, infrastructure as code, and testing in production. It outlines practical scenarios like handling database schema changes, utilizing feature flags, and A/B testing, while advocating for a build-measure-learn approach to product development. Ultimately, measuring user engagement and collecting actionable insights are crucial for successful DevOps adoption.

More Related Content

What's hot (13)

How do you agile your global team to contribute to openstackAlexis Monville

��

CUHK CSCI 4140 2015 Spring Guest Lecture - Agile DevelopmentWong Hoi Sing Edison

��

Career_ModelRocco Cutrupi

��

(Open Hack Night Fall 2014) OverviewJames Griffin

��

Agile Testing Days 2015 - PIMP MY SCRUM – DOCUMENTATION FUN FOR AGILE TEAMSAnis Ben Hamidene

��

BITS 2015: The Beauty of DrupalGerald Villorente

��

Building awesome sample apps that actually provide valueLauren Hayward Schaefer

��

Lessons learned from 6 month project with india based software houseNathan Krasney

��

Yuriy Golikov, Wrike: Team of teams how to organize effective development of...Anton Anokhin

��

Running an Agile Project with OdooOdoo

��

Beyond the web: Mobile apps using Drupal & Ionic 2 - Drupal Dev Days Seville ...La Drupalera

��

General presentation - Bitcraft Kamila Katyal

��

2018-10-23 3C - Lean, Scrum and low code approach of SharePoint and O365 proj...aOS Community

��

How do you agile your global team to contribute to openstackAlexis Monville

��

CUHK CSCI 4140 2015 Spring Guest Lecture - Agile DevelopmentWong Hoi Sing Edison

��

Career_ModelRocco Cutrupi

��

(Open Hack Night Fall 2014) OverviewJames Griffin

��

Agile Testing Days 2015 - PIMP MY SCRUM – DOCUMENTATION FUN FOR AGILE TEAMSAnis Ben Hamidene

��

BITS 2015: The Beauty of DrupalGerald Villorente

��

Building awesome sample apps that actually provide valueLauren Hayward Schaefer

��

Lessons learned from 6 month project with india based software houseNathan Krasney

��

Yuriy Golikov, Wrike: Team of teams how to organize effective development of...Anton Anokhin

��

Running an Agile Project with OdooOdoo

��

Beyond the web: Mobile apps using Drupal & Ionic 2 - Drupal Dev Days Seville ...La Drupalera

��

General presentation - Bitcraft Kamila Katyal

��

2018-10-23 3C - Lean, Scrum and low code approach of SharePoint and O365 proj...aOS Community

��

Similar to devopstools (20)

DevOps OverviewOmri Spector

��

DevOps Days OhioKelly Looney

��

Gartner Infrastructure and Operations Summit Berlin 2015 - DevOps JourneyKelly Looney

��

Devops Mindset EssentialsWilly-Peter Schaub

��

SaltConf14 - Justin Carmony, Deseret Digital Media - Teaching Devs About DevOpsSaltStack

��

Implementing Azure DevOps with your Testing ProjectRTTS

��

DevOps and ToolsMohammed Fazuluddin

��

DevOps Powered by SplunkSplunk

��

WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOpsWinOps Conf

��

Dev opsEman Abdelmohsen

��

The document discusses DevOps principles, goals, advantages, disadvantages and tools. It summarizes that DevOps is a culture that encourages collaboration between development and operations teams to build and maintain software quickly while improving quality and reliability. Traditional processes make modern architectures hard to manage at scale while DevOps aims to remove barriers and bottlenecks through continuous delivery, automation, collaboration and feedback loops.

DevOpsGehad Elsayed

��

The document discusses DevOps as a cultural movement aimed at enhancing collaboration between development and operations to improve software delivery quality and speed. It outlines the principles, goals, advantages, and disadvantages of adopting DevOps practices, emphasizing the importance of integration with Agile methodologies. Benefits include faster time-to-market and improved operational stability, while challenges involve reliance on new tools and external services.

DevTestOpsPaul Mateos

��

This document provides an overview of DevTestOps. It begins with defining DevTestOps as bringing together DevOps principles with a focus on quality assurance. It discusses key aspects like risk appetites, strategic objectives, paths of communication, and testing practices at different stages. These include unit, integration, end-to-end testing as well as monitoring, logging and testing in production. It emphasizes implementing testing practices throughout the DevOps pipeline and describes different types of tests within a DevOps process like orchestration testing. The document aims to help understand where testing fits in DevOps and how to become a competent DevTestOps team member.

Devops On Cloud Powerpoint Template �ݺ�ߣs Powerpoint Presentation �ݺ�ߣs�ݺ�ߣTeam

��

The document is a comprehensive PowerPoint template on DevOps on Cloud, covering topics such as the definition of DevOps, its importance, differences from traditional IT and Agile, use cases in various sectors, DevOps lifecycle, roles and responsibilities of engineers, and future trends. It also includes detailed information on cloud computing, its benefits, risks, deployment models, and a roadmap for integrating DevOps and cloud solutions within organizations. Additionally, the document outlines a 30-60-90 day planning framework for implementing DevOps and cloud strategies.

ADC 2017 - DevOps by examples part I – forward pipeline Giulio Vian

��

DevOps is a practice that emphasizes collaboration between software developers and IT operations professionals. It aims to unify development and operations teams to automate and streamline the process of software delivery and infrastructure changes. The presentation introduces DevOps concepts and practices like infrastructure as code, continuous integration and delivery, dynamic configuration, and emphasizing people and culture. It demonstrates how to set up infrastructure hosting for an application using tools like Azure, VSTS, Docker, and Kubernetes and discusses considerations for deploying applications, managing secrets and configurations, and monitoring operational data.

AWS DevOps Guide and Best Practices Presentation.pdfArslan594776

��

The document provides an overview of DevOps, highlighting its role as a combination of cultural philosophies, practices, and tools to enhance software delivery and reliability within organizations. It discusses key concepts such as automation, continuous integration, and the importance of collaboration among teams, as well as the benefits of infrastructure as code and containerization. Additionally, it outlines best practices for DevOps implementation and emphasizes the significance of measuring progress through various DevOps maturity signals.

BlueHat v18 || Go build a tool - best practices for building a robust & e...BlueHat Security Conference

��

The document discusses a talk aimed at security experts focused on developing effective security tools, emphasizing best practices and personal experiences. It covers tool types, importance for developer integration, the need for simplicity and clear documentation, and considerations for false positives and onboarding. Additionally, it highlights specific tools used at Microsoft, such as SARIF and Binskim, and underscores the importance of scaling and developer trust in tool adoption.

DevOps 101 Qualitest

��

The document discusses the evolution and principles of DevOps within the context of Agile software development, emphasizing the need for collaboration between developers and operations. It outlines various frameworks and methodologies, the benefits of implementing DevOps, and the importance of continuous testing. The document also highlights challenges such as skill gaps and the necessity for a shift in mindset to improve efficiency and quality in software delivery.

Devconf - Moving 65000 Microsofties to DevOps with Visual Studio Team ServicesWilly-Peter Schaub

��

The document outlines Microsoft's transition to a DevOps model utilizing Visual Studio Team Services (VSTS), focusing on delivering value and improving customer engagement since 2010. It highlights five key habits learned throughout this journey, emphasizing customer focus, data-driven decision-making, team autonomy, and a shift towards streamlined processes to enhance software quality. The transformation illustrates significant shifts from traditional practices to modern agile methodologies, including shorter sprint cycles and continuous learning principles.

Moving 75,000 Microsofties to DevOps with Visual Studio Team ServicesVSTS Community MSFT

��

The document outlines Microsoft's transition of 65,000 employees to DevOps using Visual Studio Team Services (VSTS) to enhance value flow and improve cycle times since 2010. Key lessons learned emphasize customer focus, team autonomy, continuous learning, and a production-first mindset, promoting transparency and actionable alerting for live site incidents. The DevOps transformation has led to agile, self-managing teams that deliver features every sprint, contrasting with previous rigid and hierarchical structures.

DevOps Tools to Look Out for in 2025Amnic

��

The DevOps field is changing quickly, reshaping how companies create, launch, and manage software. As we move toward 2025, combining development and operations has become essential for businesses wanting to stay competitive online. This change is mainly driven by new DevOps tools that make processes smoother and improve teamwork. DevOps is a cultural change that removes the barriers between development and operations teams. This teamwork focuses on: - Automated workflows - Faster deployment cycles - Continuous integration and delivery - Better monitoring and feedback The right DevOps tools can greatly help a company by: - Speeding up development cycles by 30-50% - Cutting deployment failures by up to 60% - Improving recovery times by 70% - Boosting team efficiency by 40% As we look to 2025, the DevOps toolset keeps growing with new solutions using AI, machine learning, and cloud technologies. This presentation explores the top DevOps tools across various categories, helping teams select the right solutions to enhance their workflows and stay ahead in the evolving tech landscape.

DevOps OverviewOmri Spector

��

DevOps Days OhioKelly Looney

��

Gartner Infrastructure and Operations Summit Berlin 2015 - DevOps JourneyKelly Looney

��

Devops Mindset EssentialsWilly-Peter Schaub

��

SaltConf14 - Justin Carmony, Deseret Digital Media - Teaching Devs About DevOpsSaltStack

��

Implementing Azure DevOps with your Testing ProjectRTTS

��

DevOps and ToolsMohammed Fazuluddin

��

DevOps Powered by SplunkSplunk

��

WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOpsWinOps Conf

��

Dev opsEman Abdelmohsen

��

DevOpsGehad Elsayed

��

DevTestOpsPaul Mateos

��

Devops On Cloud Powerpoint Template �ݺ�ߣs Powerpoint Presentation �ݺ�ߣs�ݺ�ߣTeam

��

ADC 2017 - DevOps by examples part I – forward pipeline Giulio Vian

��

AWS DevOps Guide and Best Practices Presentation.pdfArslan594776

��

BlueHat v18 || Go build a tool - best practices for building a robust & e...BlueHat Security Conference

��

DevOps 101 Qualitest

��

Devconf - Moving 65000 Microsofties to DevOps with Visual Studio Team ServicesWilly-Peter Schaub

��

Moving 75,000 Microsofties to DevOps with Visual Studio Team ServicesVSTS Community MSFT

��

DevOps Tools to Look Out for in 2025Amnic

��

More from Ariel Moskovich (13)

Consul scaleAriel Moskovich

��

The document summarizes the growth of a Consul cluster from a few dozen agents to thousands of agents across multiple data centers. As the cluster scaled up, it began experiencing performance and stability issues like IO wait, slow key-value updates, and DNS query failures. The operators addressed these issues by upgrading Consul versions, improving monitoring, stabilizing join/leave operations, and using DNS caching. They also detailed their process for a successful large-scale Consul upgrade with minimal downtime.

Kafka ops-newAriel Moskovich

��

This document provides an overview of Kafka including basic concepts like topics, brokers, and partitions. It discusses how to install and run Kafka, configure topics and clusters, and monitor and troubleshoot Kafka. It also demonstrates producing, consuming, and loading test scenarios. Key lessons learned are around balancing replication, partitions, and leaders across brokers while ensuring adequate disk space, IOPS, and retention periods. Automating cluster changes and backing up messages is also recommended.

Docker appsflyerAriel Moskovich

��

This document discusses why Docker was chosen for an organization's infrastructure, how it is being used and controlled, and the security measures in place. Docker was selected because it provides isolated environments through immutable images, has an easy to use API, a large community, and facilitates faster deployment and portability. The organization controls Docker services through its deployment system which defines who can deploy, what operations are allowed, includes flow control and auditing. Security measures implemented include MFA, image scanning, private registries, vault for secrets, VPC configuration, and AWS security groups.

Advanced Code Flow, Notes From the FieldAriel Moskovich

��

This document discusses the use of CI/CD (continuous integration and continuous delivery) pipelines to safely and quickly deliver code from development to production. It outlines the tools used, including BitBucket, Jenkins, Docker, and Mesos. It describes build conventions and how the build infrastructure is scaled horizontally. The deployment flow and types of operations like deploy, start, stop, and restart are explained. Auto healing, saving money, and the need for a PAAS platform are also covered. Build and deployment demos are included to demonstrate the workflows.

Practical Monitoring TechniquesAriel Moskovich

��

The document discusses practical monitoring techniques including current tools used like Collectd, Sensu, New Relic, Grafana, and PagerDuty. It covers increasing coverage by automatically collecting metrics and alerts for new instances. Other topics include on-call schedules, automatic self-healing, bots and alerts channels, an events dashboard, accessibility of monitoring tools, and best practices like automating monitoring, sharing responsibilities, and automating healing when possible.

ConsulAriel Moskovich

��

Consul is a service discovery tool that allows services to register themselves and discover other services via a distributed key-value store. It provides service discovery, health checking, and distributed configuration management. Consul works by having Consul agents that gossip and use the Raft consensus protocol to maintain a consistent key-value store across datacenters. Clients can access services via HTTP endpoints to register services, discover health checks, and access the key-value store.

sensuAriel Moskovich

��

This document provides an overview of Sensu, an open source monitoring tool. It discusses what needs to be monitored at AppsFlyer, why Sensu was chosen, Sensu's architecture and configurations, defining alerts and checks, the metrics flow, the Sensu API, and the Uchiwa dashboard. Sensu allows AppsFlyer to monitor hundreds of instances and services for liveliness and metrics, scales properly for their needs, and provides a simple and flexible way to monitor their growing infrastructure and applications.

kafkaAriel Moskovich

��

This document discusses the evolution of Kafka clusters at AppsFlyer over time. The initial cluster had 4 brokers and handled hundreds of millions of messages with low partitioning and replication. A new cluster was designed with more brokers, replication across availability zones, and higher partitioning to support billions of messages. However, this led to issues like uneven leader distribution and failures. Various solutions were implemented like increasing brokers, splitting topics, and hardware upgrades. Ongoing testing and monitoring helped identify more problems and improvements around replication, partitioning, and automation. Key lessons learned included balancing replication and leaders, supporting dynamic changes, and thorough testing of failure scenarios.

BouncerAriel Moskovich

��

DevopstoolsAriel Moskovich

��

This document discusses the evolution of DevOps tools at AppsFlyer and provides examples. It notes that AppsFlyer processes around 4.9 billion events per day across 120 services running on elastic infrastructure with dozens of daily deployments and a team of 25 developers. The one team philosophy motivates building internal tools to meet operational needs, extend existing tools, learn new technologies, and improve collaboration between development and operations. Considerations for new tools include needs, usability, audience, maintenance, and time constraints. Examples given are a mapping tool called Sniffer, aggregated monitoring reports from Sensu, and an authentication system called Bouncer.

Kafka awsAriel Moskovich

��

This document discusses building a fault-tolerant Kafka cluster on AWS to handle 2.5 billion requests per day. It covers choosing AWS instance types and broker counts, spreading brokers across availability zones, configuring replication and partitioning, automating fault tolerance, adding metrics and alerts, and testing the cluster's resilience. Key decisions include broker placement, topic partitioning, Zookeeper ensemble sizing, and automation to dynamically reassign partitions and change configurations in response to failures or added capacity.

Docker in prodAriel Moskovich

��

The document discusses Docker usage in production at a large company with high traffic and fast growth. It outlines the tools used to deploy Docker containers at scale, including an internal deployment tool called Santa, Consul for configuration, Mesos for clustering, Marathon as the Mesos API, and custom AWS Docker images. It also describes internal tools developed for logging, metrics, and protection. The presentation will demonstrate the deployment flow from code to production using these tools and discuss best practices, failures encountered, and how they were addressed.

Docker tlvAriel Moskovich

��

This document discusses how a company uses Docker, Docker private registries, Consul for service discovery, and Mesos/Marathon to deploy and manage applications at scale. It outlines the challenges of high traffic, data integrity, and fast growth that necessitated these tools. It provides an overview of how Dockerfiles are created and managed, how Consul integrates with DNS to enable service discovery, and how Mesos and Marathon abstract compute resources and enable deploying Docker containers across a cluster.

Consul scaleAriel Moskovich

��

Kafka ops-newAriel Moskovich

��

Docker appsflyerAriel Moskovich

��

Advanced Code Flow, Notes From the FieldAriel Moskovich

��

Practical Monitoring TechniquesAriel Moskovich

��

ConsulAriel Moskovich

��

sensuAriel Moskovich

��

kafkaAriel Moskovich

��

BouncerAriel Moskovich

��

DevopstoolsAriel Moskovich

��

Kafka awsAriel Moskovich

��

Docker in prodAriel Moskovich

��

Docker tlvAriel Moskovich

��

devopstools

1. DevOps Tools Evolution

2. Today's Menu ● AppsFlyer in numbers ● Rnd structure in AppsFlyer ● Our DevOps philosophy ● Motivation for creating DevOps tools ● Setting requirements and project assignment ● Example for tools we built internally ● Live demo and the developer angle

4. AppsFlyer in numbers ● ~ 4.9 billions events per day ● ~ 120 services ● Elastic hundreds to 1K instances ● Dozens of deployments per day ● 25 people in development ● 1 team

5. One team philosophy

6. That's me

9. Motivations ● Actual operational needs ● Extend existing tools ● Learn new technologies ● Familiarize the production environment for new Rnd members ● Blur the line between Dev and Ops ● Improve the bonding inside the team

10. Considerations ● Needs, why do we need this project? ● Usability – How do I see myself using the project? ● The target audience. ● The developer's background (Front, Mobile, etc) ● Will the developer be better familiar with production environment? ● How are we going to maintain the project? ● What are the time constraints? ● Can we roll it out in stages? ● Can we release it outside?

11. Today's Examples ● Sniffer: Mapping tool (Artyom Shein) ● Sensu Reports: Aggregated reports of Sensu (Avner Schwartz) ● Bouncer: Authentication and Authorization (Reshef Mann)

12. Live Demo

13. And... Rock n Roll

14. Questions?

�ݺ�ߣ

devopstools

Recommended

More Related Content

What's hot (13)

Similar to devopstools (20)

More from Ariel Moskovich (13)

devopstools