This slide show highlights some of the deficiencies in the current attribute mapping functionality of Keystone (IceHouse release) and proposes a modification for Juno based on adding trusted attributes and simplified mapping rules
Data Access 2.0? Please welcome, Spring Data!Oliver Gierke
油
This document discusses Spring Data, a framework that provides a consistent programming model for data access while supporting different data store technologies. It summarizes that Spring Data aims to abstract custom traits of different data stores without over-abstracting. It also highlights key Spring Data features like mapping, templates, repositories, and support for both relational and non-relational databases.
Spring Data JPA - Repositories done rightOliver Gierke
油
Spring Data JPA provides JPA repositories that allow for easy data access and management. It offers declarative CRUD and query methods, specifications for complex queries, integration with Querydsl for type-safe queries, auditing functionality, and the ability to add custom code when needed. The project is open source and available on GitHub along with code examples to demonstrate its features for simplifying data access with JPA.
Retailing Dictionary A To Z Retail BusinessAnoopsinghMba
油
This document provides definitions for over 100 common retail business terms starting with A through G. Some of the key terms defined include:
- Anchor store: A major store used to drive customers to smaller retailers in a shopping center.
- Brand: A name, symbol or mark associated with a seller's goods/services that distinguishes them from competitors.
- Brick and mortar: Refers to physical retail stores located in a building rather than online.
- Category killer store: A large specialty store with an enormous selection in its product category and low prices that draws customers from a wide area.
- Department store: A large retail unit organized into departments selling a wide variety of goods and services
This document provides an overview of statistical computing for big data. It introduces MapReduce and Hadoop as distributed computing frameworks for scaling computations to large datasets. The document discusses some of the statistical challenges posed by big data, including scale, dimensionality, streaming data, and experiments. It also compares different distributed computing methods like multithreading, GPUs, MPI, and MapReduce in terms of scalability, fault tolerance, and suitability for iterative processes.
Spark Summit EU 2015: Combining the Strengths of MLlib, scikit-learn, and RDatabricks
油
This talk discusses integrating common data science tools like Python pandas, scikit-learn, and R with MLlib, Sparks distributed Machine Learning (ML) library. Integration is simple; migration to distributed ML can be done lazily; and scaling to big data can significantly improve accuracy. We demonstrate integration with a simple data science workflow. Data scientists often encounter scaling bottlenecks with single-machine ML tools. Yet the overhead in migrating to a distributed workflow can seem daunting. In this talk, we demonstrate such a migration, taking advantage of Spark and MLlibs integration with common ML libraries. We begin with a small dataset which runs on a single machine. Increasing the size, we hit bottlenecks in various parts of the workflow: hyperparameter tuning, then ETL, and eventually the core learning algorithm. As we hit each bottleneck, we parallelize that part of the workflow using Spark and MLlib. As we increase the dataset and model size, we can see significant gains in accuracy. We end with results demonstrating the impressive scalability of MLlib algorithms. With accuracy comparable to traditional ML libraries, combined with state-of-the-art distributed scalability, MLlib is a valuable new tool for the modern data scientist.
Web-Scale Graph Analytics with Apache Spark with Tim HunterDatabricks
油
Graph analytics has a wide range of applications, from information propagation and network flow optimization to fraud and anomaly detection. The rise of social networks and the Internet of Things has given us complex web-scale graphs with billions of vertices and edges. However, in order to extract the hidden gems of understanding and information within those graphs, you need tools to analyze the graphs easily and efficiently.
At Spark Summit 2016, Databricks introduced GraphFrames, which implements graph queries and pattern matching on top of Spark SQL to simplify graph analytics. In this talk, well discuss the work that has made graph algorithms in GraphFrames faster and more scalable. For example, new implementations of connected components have received algorithm improvements based on recent research, as well as performance improvements from Spark DataFrames. Discover lessons learned from scaling the implementation from millions to billions of nodes; see its performance in the context of other popular graph libraries; and hear about real-world applications.
Relational data modeling trends for transactional applicationsIke Ellis
油
This document provides a summary of Ike Ellis's presentation on data modeling priorities and design patterns for transactional applications. The presentation discusses how data modeling priorities have changed from focusing on writes and normalization to emphasizing reads, flexibility, and performance. It outlines several current design priorities including optimizing the schema for reads, making it easy to change and discoverable, and designing for the network instead of the disk. The presentation concludes with practicing modeling data for example transactional applications like a blog, online store, and refrigeration trucks.
Deep Dive : Spark Data Frames, SQL and Catalyst OptimizerSachin Aggarwal
油
RDD recap
Spark SQL library
Architecture of Spark SQL
Comparison with Pig and Hive Pipeline
DataFrames
Definition of a DataFrames API
DataFrames Operations
DataFrames features
Data cleansing
Diagram for logical plan container
Plan Optimization & Execution
Catalyst Analyzer
Catalyst Optimizer
Generating Physical Plan
Code Generation
Extensions
Accelerating Data Science with Better Data Engineering on DatabricksDatabricks
油
Whether youre processing IoT data from millions of sensors or building a recommendation engine to provide a more engaging customer experience, the ability to derive actionable insights from massive volumes of diverse data is critical to success. MediaMath, a leading adtech company, relies on Apache Spark to process billions of data points ranging from ads, user cookies, impressions, clicks, and more translating to several terabytes of data per day. To support the needs of the data science teams, data engineering must build data pipelines for both ETL and feature engineering that are scalable, performant, and reliable.
Join this webinar to learn how MediaMath leverages Databricks to simplify mission-critical data engineering tasks that surface data directly to clients and drive actionable business outcomes. This webinar will cover:
- Transforming TBs of data with RDDs and PySpark responsibly
- Using the JDBC connector to write results to production databases seamlessly
- Comparisons with a similar approach using Hive
This document proposes fast single-pass k-means clustering algorithms to allow for fast nearest neighbor search on large datasets. It discusses the rationale for using k-means clustering, describes algorithms like ball k-means and surrogate methods that can perform clustering in a single pass. It covers implementations using techniques like locality sensitive hashing and projection search to speed up vector searches. Evaluation on synthetic and real datasets shows the algorithms can achieve the same or better accuracy as traditional k-means 10x faster, enabling applications like fast nearest neighbor search on massive datasets for applications like customer modeling.
This document summarizes a project on implementing and evaluating parallel algorithms for connected components labeling on graphs using CPU (OpenMP) and GPU (CUDA). It studied different graph types and architectures. It proposed a simple autotuning approach to choose the best technique for a given graph by characterizing graphs based on features and employing the best algorithm. It discussed motivations, definitions, basic algorithms, optimizations, experiments on datasets, and future work including more sophisticated autotuning and heterogeneous algorithms.
Building Identity Graphs over Heterogeneous DataDatabricks
油
In todays world, customers and service providers (e.g., Social networks, ad targeting, retail, etc.) interact in a variety of modes and channels such as browsers, apps, devices, etc. In each such interaction, users are identified using a token (possibly different token for each mode/channel). Examples of such identity tokens include cookies, app IDs etc. As the user engages more with these services, linkages are generated between tokens belonging to the same user; linkages connect multiple identity tokens together.
Lambda Data Grid: An Agile Optical Platform for Grid Computing and Data-inten...Tal Lavian Ph.D.
油
Lambda Data Grid
An Agile Optical Platform for Grid Computing
and Data-intensive Applications
Focus on BIRN Mouse application.
Great vision
LambdaGrid is one step towards this concepts
LambdaGrid
A novel service architecture
Lambda as a Scheduled Service
Lambda as a prime resource - like storage and computation
Change our current systems assumptions
Potentially opens new horizon
How does the Cloud Foundry Diego Project Run at Scale, and Updates on .NET Su...Amit Gupta
油
The Cloud Foundry Diego team at Pivotal has been hard at work for the past few months exploring and improving Diego's performance at scale and under stress. This talk covers the goals, tools, and results of the experiments to date, as well as a glimpse of what's next.
And finally, a brief teaser about the current state of .NET support in Diego
How does the Cloud Foundry Diego Project Run at Scale?VMware Tanzu
油
From Pivotal's Amit Gupta on July 9, 2015, a look at how the Cloud Foundry Diego project runs at scale, and what it took to get there. Offering a look into the Diego project scheduler and the performance testing efforts, all the tools necessary to ensure that Cloud Foundry can scale quickly and effortlessly.
To learn more, visit pivotal.io/platform-as-a-service/pivotal-cloud-foundry
This document provides an overview of data science work at Zillow. It discusses Zillow's use of machine learning models like the Zestimate and Rent Zestimate to analyze housing data. It describes Zillow's technology stack, which heavily leverages Python, R, and SQL. Specific examples are provided on automated waterfront determination using GIS data and discovering home street features. The document also discusses how tools like Dato and Scikit-Learn are used for tasks like fraud detection, property matching, and data modeling. In closing, current job openings at Zillow are listed.
Bayes Nets meetup talk, October 22nd at Mathys and Squire, The Shard. Ralf Herbrich, machine learning director at Amazon, gave a talk on graphical models, used at Microsoft, Facebook and Amazon.
Apache Kylin: OLAP Engine on Hadoop - Tech Deep DiveXu Jiang
油
Kylin is an open source Distributed Analytics Engine from eBay Inc. that provides SQL interface and multi-dimensional analysis (OLAP) on Hadoop supporting extremely large datasets.
If you want to do multi-dimension analysis on large data sets (billion+ rows) with low query latency (sub-seconds), Kylin is a good option. Kylin also provides seamless integration with existing BI tools (e.g Tableau).
This document discusses different data models used to describe database structures, including the relational, entity-relationship, object-based, and semi-structured models. It focuses on explaining the entity-relationship model and its key concepts such as entities, attributes, relationships, cardinalities that define the number of relationships between entities, and participation constraints on entity involvement in relationships.
Machine Learning Essentials Demystified part1 | Big Data DemystifiedOmid Vahdaty
油
Machine Learning Essentials Abstract:
Machine Learning (ML) is one of the hottest topics in the IT world today. But what is it really all about?
In this session we will talk about what ML actually is and in which cases it is useful.
We will talk about a few common algorithms for creating ML models and demonstrate their use with Python. We will also take a peek at Deep Learning (DL) and Artificial Neural Networks and explain how they work (without too much math) and demonstrate DL model with Python.
The target audience are developers, data engineers and DBAs that do not have prior experience with ML and want to know how it actually works.
Lc3 beijing-june262018-sahdev zala-guangyaSahdev Zala
油
Our slides deck, used at the LinuxCon+ContainerCon+CLOUDOPEN China 2018, on Kubernetes cluster design considerations and our journey to 1000+ node single cluster with IBM Cloud.
Challenging Web-Scale Graph Analytics with Apache SparkDatabricks
油
Graph analytics has a wide range of applications, from information propagation and network flow optimization to fraud and anomaly detection. The rise of social networks and the Internet of Things has given us complex web-scale graphs with billions of vertices and edges. However, in order to extract the hidden gems within those graphs, you need tools to analyze the graphs easily and efficiently.
At Spark Summit 2016, Databricks introduced GraphFrames, which implemented graph queries and pattern matching on top of Spark SQL to simplify graph analytics. In this talk, youll learn about work that has made graph algorithms in GraphFrames faster and more scalable. For example, new implementations like connected components have received algorithm improvements based on recent research, as well as performance improvements from Spark DataFrames. Discover lessons learned from scaling the implementation from millions to billions of nodes; compare its performance with other popular graph libraries; and hear about real-world applications.
Challenging Web-Scale Graph Analytics with Apache Spark with Xiangrui MengDatabricks
油
Graph analytics has a wide range of applications, from information propagation and network flow optimization to fraud and anomaly detection. The rise of social networks and the Internet of Things has given us complex web-scale graphs with billions of vertices and edges. However, in order to extract the hidden gems within those graphs, you need tools to analyze the graphs easily and efficiently.
At Spark Summit 2016, Databricks introduced GraphFrames, which implemented graph queries and pattern matching on top of Spark SQL to simplify graph analytics. In this talk, you'll learn about work that has made graph algorithms in GraphFrames faster and more scalable. For example, new implementations like connected components have received algorithm improvements based on recent research, as well as performance improvements from Spark DataFrames. Discover lessons learned from scaling the implementation from millions to billions of nodes; compare its performance with other popular graph libraries; and hear about real-world applications.
Strata Conference + Hadoop World NY 2016: Lessons learned building a scalable...Sumeet Singh
油
This document discusses lessons learned from building a scalable, self-serve, real-time, multi-tenant monitoring service at Yahoo. It describes transitioning from a classical architecture to one based on real-time big data technologies like Storm and Kafka. Key lessons include properly handling producer-consumer problems at scale, challenges of debugging skewed data, strategically managing multi-tenancy and resources, issues optimizing asynchronous systems, and not neglecting assumptions outside the application.
Abstract:
Cassandra is a new kind of database: it is more than a single-machine system. It naturally runs in a High-Availability configuration. All nodes in the system are symmetric; there is no single point of failure. As you add machines, failure becomes routine, and Cassandra is built to tolerate that with no interruptions.
Cassandra is linearly scalable with good performance characteristics for very small and very large data stores. Unlike earlier efforts, Cassandra is more than just a key-value store; it is a structured data store which can facilitate complex use cases and queries. Cassandra allows for random access to your data organized into rows and columns.
Cassandra is different, and exciting. This presentation will discuss the pros and cons of using Cassandra, and why it has seen such amazing adoption in the past year.
Bio:
Ben Coverston is Director of Operations at DataStax (formerly knows as Riptano), a provider of software, support, services, training, resources and help for Cassandra. He has been involved in enterprise software his entire career. Working in the airline industry, he helped to build some of the highest volume online booking sites in the world. He saw first hand the consequences of trying to solve real world scalability problems at the limit of what traditional relational databases are capable of.
Infrastructure API Lightning Talk by Jeremy Pollard of box.comDevOps4Networks
油
This document describes how Box automated their network configuration by building an Infrastructure API. The API generates all configuration details for switches mathematically based on identifiers for datacenter, pod, cabinet, and host type. It handles tasks like IP and hostname assignment, DNS registration, and cable mapping. New switches can now be automatically provisioned by downloading scripts that call the API to generate configs and register the switch. This reduces human errors and saves significant time in network provisioning and management. Potential issues include incorrectly allocated identifiers or cabling errors, but the system aims to make the network "smarter than humans."
The document discusses performance analysis of the BOUT++ code. It notes that improving HPC performance has economic and scientific benefits. The goals of performance analysis are to identify bottlenecks and suggest improvements to optimize code performance. Profilers are used to measure performance and identify issues such as poor scaling, load imbalance, communication overhead, and memory bandwidth sensitivity. Analysis of BOUT++ shows good scaling up to 8,192 cores but decreased performance at higher concurrencies potentially due to increased computational work in ghost cells as the grid points per processor decrease with increasing concurrency.
Lessons Learned from Using Spark for Evaluating Road Detection at BMW Autonom...Databricks
油
Getting cars to drive autonomously is one of the most exciting problems these days. One of the key challenges is making them drive safely, which requires processing large amounts of data. In our talk we would like to focus on only one task of a self-driving car, namely road detection. Road detection is a software component which needs to be safe for being able to keep the car in the current lane. In order to track the progress of such a software component, a well-designed KPI (key performance indicators) evaluation pipeline is required. In this presentation we would like to show you how we incorporate Spark in our pipeline to deal with huge amounts of data and operate under strict scalability constraints for gathering relevant KPIs. Additionally, we would like to mention several lessons learned from using Spark in this environment.
Learn what satellite communication is, how it functions, and its importance in enabling data transmission via satellites, transponders, and ground stations.
Mastering SEO: Build a Winning Strategy from the Ground Upthedigicenter
油
Want to drive more traffic and rank higher on Google? This presentation breaks down the essential steps to craft an effective SEO strategy from scratch. Whether you're a beginner or a marketing pro looking to refresh your skills, discover practical tips, on-page & off-page techniques, keyword research methods, content strategies, and performance tracking tools to boost your websites visibility and search engine performance. Ideal for bloggers, business owners, and digital marketers!
More Related Content
Similar to Scalable and usable attribute mappings in Keystone - Openstack (20)
Accelerating Data Science with Better Data Engineering on DatabricksDatabricks
油
Whether youre processing IoT data from millions of sensors or building a recommendation engine to provide a more engaging customer experience, the ability to derive actionable insights from massive volumes of diverse data is critical to success. MediaMath, a leading adtech company, relies on Apache Spark to process billions of data points ranging from ads, user cookies, impressions, clicks, and more translating to several terabytes of data per day. To support the needs of the data science teams, data engineering must build data pipelines for both ETL and feature engineering that are scalable, performant, and reliable.
Join this webinar to learn how MediaMath leverages Databricks to simplify mission-critical data engineering tasks that surface data directly to clients and drive actionable business outcomes. This webinar will cover:
- Transforming TBs of data with RDDs and PySpark responsibly
- Using the JDBC connector to write results to production databases seamlessly
- Comparisons with a similar approach using Hive
This document proposes fast single-pass k-means clustering algorithms to allow for fast nearest neighbor search on large datasets. It discusses the rationale for using k-means clustering, describes algorithms like ball k-means and surrogate methods that can perform clustering in a single pass. It covers implementations using techniques like locality sensitive hashing and projection search to speed up vector searches. Evaluation on synthetic and real datasets shows the algorithms can achieve the same or better accuracy as traditional k-means 10x faster, enabling applications like fast nearest neighbor search on massive datasets for applications like customer modeling.
This document summarizes a project on implementing and evaluating parallel algorithms for connected components labeling on graphs using CPU (OpenMP) and GPU (CUDA). It studied different graph types and architectures. It proposed a simple autotuning approach to choose the best technique for a given graph by characterizing graphs based on features and employing the best algorithm. It discussed motivations, definitions, basic algorithms, optimizations, experiments on datasets, and future work including more sophisticated autotuning and heterogeneous algorithms.
Building Identity Graphs over Heterogeneous DataDatabricks
油
In todays world, customers and service providers (e.g., Social networks, ad targeting, retail, etc.) interact in a variety of modes and channels such as browsers, apps, devices, etc. In each such interaction, users are identified using a token (possibly different token for each mode/channel). Examples of such identity tokens include cookies, app IDs etc. As the user engages more with these services, linkages are generated between tokens belonging to the same user; linkages connect multiple identity tokens together.
Lambda Data Grid: An Agile Optical Platform for Grid Computing and Data-inten...Tal Lavian Ph.D.
油
Lambda Data Grid
An Agile Optical Platform for Grid Computing
and Data-intensive Applications
Focus on BIRN Mouse application.
Great vision
LambdaGrid is one step towards this concepts
LambdaGrid
A novel service architecture
Lambda as a Scheduled Service
Lambda as a prime resource - like storage and computation
Change our current systems assumptions
Potentially opens new horizon
How does the Cloud Foundry Diego Project Run at Scale, and Updates on .NET Su...Amit Gupta
油
The Cloud Foundry Diego team at Pivotal has been hard at work for the past few months exploring and improving Diego's performance at scale and under stress. This talk covers the goals, tools, and results of the experiments to date, as well as a glimpse of what's next.
And finally, a brief teaser about the current state of .NET support in Diego
How does the Cloud Foundry Diego Project Run at Scale?VMware Tanzu
油
From Pivotal's Amit Gupta on July 9, 2015, a look at how the Cloud Foundry Diego project runs at scale, and what it took to get there. Offering a look into the Diego project scheduler and the performance testing efforts, all the tools necessary to ensure that Cloud Foundry can scale quickly and effortlessly.
To learn more, visit pivotal.io/platform-as-a-service/pivotal-cloud-foundry
This document provides an overview of data science work at Zillow. It discusses Zillow's use of machine learning models like the Zestimate and Rent Zestimate to analyze housing data. It describes Zillow's technology stack, which heavily leverages Python, R, and SQL. Specific examples are provided on automated waterfront determination using GIS data and discovering home street features. The document also discusses how tools like Dato and Scikit-Learn are used for tasks like fraud detection, property matching, and data modeling. In closing, current job openings at Zillow are listed.
Bayes Nets meetup talk, October 22nd at Mathys and Squire, The Shard. Ralf Herbrich, machine learning director at Amazon, gave a talk on graphical models, used at Microsoft, Facebook and Amazon.
Apache Kylin: OLAP Engine on Hadoop - Tech Deep DiveXu Jiang
油
Kylin is an open source Distributed Analytics Engine from eBay Inc. that provides SQL interface and multi-dimensional analysis (OLAP) on Hadoop supporting extremely large datasets.
If you want to do multi-dimension analysis on large data sets (billion+ rows) with low query latency (sub-seconds), Kylin is a good option. Kylin also provides seamless integration with existing BI tools (e.g Tableau).
This document discusses different data models used to describe database structures, including the relational, entity-relationship, object-based, and semi-structured models. It focuses on explaining the entity-relationship model and its key concepts such as entities, attributes, relationships, cardinalities that define the number of relationships between entities, and participation constraints on entity involvement in relationships.
Machine Learning Essentials Demystified part1 | Big Data DemystifiedOmid Vahdaty
油
Machine Learning Essentials Abstract:
Machine Learning (ML) is one of the hottest topics in the IT world today. But what is it really all about?
In this session we will talk about what ML actually is and in which cases it is useful.
We will talk about a few common algorithms for creating ML models and demonstrate their use with Python. We will also take a peek at Deep Learning (DL) and Artificial Neural Networks and explain how they work (without too much math) and demonstrate DL model with Python.
The target audience are developers, data engineers and DBAs that do not have prior experience with ML and want to know how it actually works.
Lc3 beijing-june262018-sahdev zala-guangyaSahdev Zala
油
Our slides deck, used at the LinuxCon+ContainerCon+CLOUDOPEN China 2018, on Kubernetes cluster design considerations and our journey to 1000+ node single cluster with IBM Cloud.
Challenging Web-Scale Graph Analytics with Apache SparkDatabricks
油
Graph analytics has a wide range of applications, from information propagation and network flow optimization to fraud and anomaly detection. The rise of social networks and the Internet of Things has given us complex web-scale graphs with billions of vertices and edges. However, in order to extract the hidden gems within those graphs, you need tools to analyze the graphs easily and efficiently.
At Spark Summit 2016, Databricks introduced GraphFrames, which implemented graph queries and pattern matching on top of Spark SQL to simplify graph analytics. In this talk, youll learn about work that has made graph algorithms in GraphFrames faster and more scalable. For example, new implementations like connected components have received algorithm improvements based on recent research, as well as performance improvements from Spark DataFrames. Discover lessons learned from scaling the implementation from millions to billions of nodes; compare its performance with other popular graph libraries; and hear about real-world applications.
Challenging Web-Scale Graph Analytics with Apache Spark with Xiangrui MengDatabricks
油
Graph analytics has a wide range of applications, from information propagation and network flow optimization to fraud and anomaly detection. The rise of social networks and the Internet of Things has given us complex web-scale graphs with billions of vertices and edges. However, in order to extract the hidden gems within those graphs, you need tools to analyze the graphs easily and efficiently.
At Spark Summit 2016, Databricks introduced GraphFrames, which implemented graph queries and pattern matching on top of Spark SQL to simplify graph analytics. In this talk, you'll learn about work that has made graph algorithms in GraphFrames faster and more scalable. For example, new implementations like connected components have received algorithm improvements based on recent research, as well as performance improvements from Spark DataFrames. Discover lessons learned from scaling the implementation from millions to billions of nodes; compare its performance with other popular graph libraries; and hear about real-world applications.
Strata Conference + Hadoop World NY 2016: Lessons learned building a scalable...Sumeet Singh
油
This document discusses lessons learned from building a scalable, self-serve, real-time, multi-tenant monitoring service at Yahoo. It describes transitioning from a classical architecture to one based on real-time big data technologies like Storm and Kafka. Key lessons include properly handling producer-consumer problems at scale, challenges of debugging skewed data, strategically managing multi-tenancy and resources, issues optimizing asynchronous systems, and not neglecting assumptions outside the application.
Abstract:
Cassandra is a new kind of database: it is more than a single-machine system. It naturally runs in a High-Availability configuration. All nodes in the system are symmetric; there is no single point of failure. As you add machines, failure becomes routine, and Cassandra is built to tolerate that with no interruptions.
Cassandra is linearly scalable with good performance characteristics for very small and very large data stores. Unlike earlier efforts, Cassandra is more than just a key-value store; it is a structured data store which can facilitate complex use cases and queries. Cassandra allows for random access to your data organized into rows and columns.
Cassandra is different, and exciting. This presentation will discuss the pros and cons of using Cassandra, and why it has seen such amazing adoption in the past year.
Bio:
Ben Coverston is Director of Operations at DataStax (formerly knows as Riptano), a provider of software, support, services, training, resources and help for Cassandra. He has been involved in enterprise software his entire career. Working in the airline industry, he helped to build some of the highest volume online booking sites in the world. He saw first hand the consequences of trying to solve real world scalability problems at the limit of what traditional relational databases are capable of.
Infrastructure API Lightning Talk by Jeremy Pollard of box.comDevOps4Networks
油
This document describes how Box automated their network configuration by building an Infrastructure API. The API generates all configuration details for switches mathematically based on identifiers for datacenter, pod, cabinet, and host type. It handles tasks like IP and hostname assignment, DNS registration, and cable mapping. New switches can now be automatically provisioned by downloading scripts that call the API to generate configs and register the switch. This reduces human errors and saves significant time in network provisioning and management. Potential issues include incorrectly allocated identifiers or cabling errors, but the system aims to make the network "smarter than humans."
The document discusses performance analysis of the BOUT++ code. It notes that improving HPC performance has economic and scientific benefits. The goals of performance analysis are to identify bottlenecks and suggest improvements to optimize code performance. Profilers are used to measure performance and identify issues such as poor scaling, load imbalance, communication overhead, and memory bandwidth sensitivity. Analysis of BOUT++ shows good scaling up to 8,192 cores but decreased performance at higher concurrencies potentially due to increased computational work in ghost cells as the grid points per processor decrease with increasing concurrency.
Lessons Learned from Using Spark for Evaluating Road Detection at BMW Autonom...Databricks
油
Getting cars to drive autonomously is one of the most exciting problems these days. One of the key challenges is making them drive safely, which requires processing large amounts of data. In our talk we would like to focus on only one task of a self-driving car, namely road detection. Road detection is a software component which needs to be safe for being able to keep the car in the current lane. In order to track the progress of such a software component, a well-designed KPI (key performance indicators) evaluation pipeline is required. In this presentation we would like to show you how we incorporate Spark in our pipeline to deal with huge amounts of data and operate under strict scalability constraints for gathering relevant KPIs. Additionally, we would like to mention several lessons learned from using Spark in this environment.
Learn what satellite communication is, how it functions, and its importance in enabling data transmission via satellites, transponders, and ground stations.
Mastering SEO: Build a Winning Strategy from the Ground Upthedigicenter
油
Want to drive more traffic and rank higher on Google? This presentation breaks down the essential steps to craft an effective SEO strategy from scratch. Whether you're a beginner or a marketing pro looking to refresh your skills, discover practical tips, on-page & off-page techniques, keyword research methods, content strategies, and performance tracking tools to boost your websites visibility and search engine performance. Ideal for bloggers, business owners, and digital marketers!
Amazon Sidewalk: A Global Wake-Up Call for the Telecom IndustryDavid Swift
油
咋腫 駒告 咋介瑞駒稲 告瑞基介 腫 諮介腫瑞呉and nobody noticed. $腫 介介 巨 介告稲腫.
No spectrum auctions.
No cell towers.
No billion-dollar rollouts.
Here's the story... Amazon has quietly launched the largest IoT network in the United States, covering over 90% of the population. This network, known as Amazon Sidewalk, bypasses traditional telecom infrastructure, leverages consumer devices, and utilizes unlicensed spectrum to deliver pervasive, low-bandwidth connectivity. This white paper explores the global implications of Amazon's approach, outlines strategic risks and opportunities for telecom operators, and provides actionable insights for future-proofing telco business models in the face of tech-driven disruption.
E3 MDF Manufacturing Facility in Kashipur, Uttarakhand, sets new industry standards with state-of-the-art European machinery for wood chipping, fiber refinement, and continuous pressing. Our advanced system produces 300 cubic meters daily, supplemented by multi-daylight presses generating 250 cubic meters. This allows us to achieve an impressive total of 550 cubic meters of high-quality MDF boards daily. We are committed to delivering excellence, ensuring that every board meets with the highest quality standards of strength, durability and finish. Choose E3 MDF boards for your projects, and experience the perfect blend of innovation, quality, and reliability. Trust us as your MDF board manufacturer to elevate your projects to new heights.
Revolutionizing Tomorrow: The Power of AI
Artificial Intelligence (AI) is no longer just a futuristic concept; it is rapidly becoming an integral part of our daily lives, reshaping industries, economies, and the way we live. As AI technology continues to evolve at an unprecedented rate, its impact is felt in almost every sector, from healthcare and finance to entertainment and transportation. This transformative force is not only changing the way businesses operate but also challenging our understanding of intelligence itself.
At its core, AI is the simulation of human intelligence in machines. It involves the creation of algorithms and systems that can analyze data, recognize patterns, and make decisions autonomously. This ability to learn from experience and adapt to new situations is what sets AI apart from traditional software systems. Unlike conventional programs that follow predefined instructions, AI systems can improve over time, becoming more efficient and effective in their tasks.
One of the most significant advancements in AI is machine learning (ML), a subset of AI that enables machines to learn from data without explicit programming. ML algorithms can process vast amounts of data, identify hidden patterns, and make predictions or decisions based on that data. This has led to breakthroughs in areas such as natural language processing (NLP), computer vision, and speech recognition, allowing AI systems to understand and interact with the world in ways that were once thought to be exclusive to humans.
In healthcare, AI is revolutionizing diagnostics and treatment. Machine learning models are being used to analyze medical images, detect diseases like cancer at early stages, and predict patient outcomes with remarkable accuracy. AI-powered tools are also assisting doctors in developing personalized treatment plans, ensuring that patients receive the most effective care based on their individual genetic profiles and medical histories.
The financial sector is also benefiting from AI, particularly in areas like fraud detection, algorithmic trading, and customer service. AI systems can analyze financial data in real-time, identifying unusual patterns that may indicate fraudulent activity. In trading, AI algorithms can process market data and execute trades at lightning speed, maximizing profits and minimizing risks. Chatbots and virtual assistants powered by AI are transforming customer service, providing instant support and solving complex queries with human-like precision.
Transportation is another industry being transformed by AI. Autonomous vehicles, which rely heavily on AI algorithms, are set to revolutionize how we travel. Self-driving cars, trucks, and drones are already being tested on roads and in the skies, promising to reduce traffic accidents, lower emissions, and improve overall efficiency in transportation networks. AI-powered traffic management systems are also being developed to optimize traffic flow in cities, reducing
BGP Best Practices, presented by Imtiaz SajidAPNIC
油
Imtiaz Sajid, Network Analyst / Technical Trainer at APNIC, delivered a remote presentation on 'BGP Best Practices' for MMNOG 7 held Yangon, Myanmar from 19 to 22 March 2025.
DOWNLOAD LINK
https://ncracked.com/7961-2/
Free Download MiniTool Power Data Recovery Business 12.3 + WinPE ISO - Edition: Edition: Standard, Deluxe, Enterprise, Technician.Free Download MiniTool Power Data Recovery Business 12.3 + WinPE ISO - Edition: Edition: Standard, Deluxe, Enterprise, Technician.MiniTool Power Data Recovery is read-only File and data recovery software. It can help you recover all your data, whether it was lost by accidental deletion, format, re-partition, system crash, or virus attack. Power Data Recovery can scan your device sector by sector to recover whatever is left on it. It provides a deep scan module to scan and analyze raw data, including Microsoft Office files, digital graphics/pictures, and audio/video file
Frontier Internet Availability Expanding Access to Rural Communities.pdfInternet Bundle Now
油
Explore how frontier internet service from Internet Bundle Now is transforming rural communities with high-speed access and responsive frontier Customer Service support.
APNIC's Senior Regional Advisor, Membership and Policy, Sunny Chendi, presented an introduction to APNIC and the policy development process at APIGA India 2025 held in Delhi, India on 21 and 22 March 2025.
2. How Many Are There?
Millions of potential IdPs (cf LDAP directories)
Billions of potential identity attributes (cf
different variants of LDAP attributes)
How to manage this in a simple, scalable, easy
to use, trustworthy way?
4. Net Effect
Gives the administrator must better and finer
control
Much easier to specify mapping rules (which
can be complex)
Dont need to worry about unknown
attributes and regex matches on them
Simpler and less complex mapping rules
5. Simple Example
Suppose you trust 3 IdPs
Trust IdP 1 to issue attributes a and b
Trust IdP 2 to issue attributes a and c
Trust IdP 3 to issue attributes a, b and c
With existing scheme you need 3 largish mapping rules
IdP 1: a maps to g1, b maps to g2
IdP 2: a maps to g1, c maps to g3
IdP 3: a maps to g1, b maps to g2, c maps to g3
With proposed scheme you need 3 simpler rules
a maps to g1
b maps to g2
c maps to g3
Along with 3 trusted attribute rules
Trust IdP1 to issue a and b
Trust IdP2 to issue a and c
Trust IdP3 to issue a, b and c
6. What about Large Federations?
The current mapping rules are even worse.
E.g. UK AMF has over 100 IdPs who all issue the same set of EduPerson
Schema attributes
This will require >100 almost identical rules
IdP 1: a maps to g1, b maps to g2
IdP 2: a maps to g1, b maps to g2
..
IdP 100: a maps to g1, b maps to g2
Instead of
a maps to g1
b maps to g2
Trust IdP1 to issue a and b
Trust IdP2 to issue a and b
Trust IdP 100 to issue a and b
We may want to allow Trust All IdPs to issue a and b
7. Implication
All attributes must be syntactically and semantically unique
in a federation
But what if attribute a from IdP 1 is syntactically the same
as but semantically different to attribute a from IdP 2?
E.g. VP in IdP1 means senior manager, lots of them
VP in IdP2 means 2nd in command, only 1 of them
Solution Optionally qualify identity attribute in mapping
rule with issuing IdP
IdP1.a maps to g1
IdP2.a maps to g2
b maps to g2
c maps to g3
If IdP is missing it means all IdPs