Sprezzatura - Roelof van Zwol - May 2018
3 likes1,229 views
The document discusses the application of machine learning in content promotion on Netflix, emphasizing the importance of understanding correlation versus causation and the need for incrementality in advertising strategies. It highlights the challenges and methodologies involved in optimizing content visibility on the platform, including the use of reinforcement learning and incremental policies to maximize viewer engagement. Ultimately, the text underscores the significance of experimental data in developing effective promotional strategies that account for user behavior and preferences.
1 of 35
Downloaded 25 times
![Incrementality-based policy
Goal: Select title for promotion that benefits most from
being shown in billboard
t = argmax [ P(ym,t
| xm,t
, T, b=1) - P(ym,t
| xm,t
, T, b=0) ]
Where b is an indicator for the treatment of a title being shown in billboard (b=1),
versus not being shown in billboard (b=0)](https://image.slidesharecdn.com/sprezzatura-roelofvanzwol-may2018-180509214049/85/Sprezzatura-Roelof-van-Zwol-May-2018-28-320.jpg)
![Offline evaluation: Replay [Li et al, 2010]
Relies upon uniform exploration data.
For every record in the uniform exploration log
{context, title k shown, reward, list of candidates}
For every record:
Evaluate the trained model for all the titles in the candidate pool.
Pick the winning title k
Keep the record in history if k = k (the title impressed in the logged
data) else discard it.
Compute the metrics from the history.](https://image.slidesharecdn.com/sprezzatura-roelofvanzwol-may2018-180509214049/85/Sprezzatura-Roelof-van-Zwol-May-2018-29-320.jpg)
![Offline evaluation: Replay [Li et al, 2010]
Uniform Exploration Data - Unbiased evaluation
Evaluation
Data
Train Data
Trained
Model
Reveal context x
Use reward only if k = k
Winner title k
context,title,reward
context,title,reward
context,title,reward
Take Rate = # Plays
# Matches](https://image.slidesharecdn.com/sprezzatura-roelofvanzwol-may2018-180509214049/85/Sprezzatura-Roelof-van-Zwol-May-2018-30-320.jpg)
Ad
Recommended
Marketplace in motion - AdKDD keynote - 2020
Marketplace in motion - AdKDD keynote - 2020 Roelof van Zwol
油
This document discusses Pinterest's ads marketplace and optimization strategies. It provides an overview of Pinterest's ads delivery funnel including ranking, auction, and retrieval. It then discusses predicting relevance and engagement through human labels, deep learning models, and multi-task learning. It also covers auction design principles and candidate retrieval using a two-tower deep learning approach. The goal is to maximize long-term value for users, advertisers, and Pinterest across different surfaces and ad formats.Correlation, causation and incrementally recommendation problems at netflix ...
Correlation, causation and incrementally recommendation problems at netflix ...Roelof van Zwol
油
The document discusses the importance of understanding correlations versus causation in the context of promoting content at Netflix, emphasizing that traditional correlational models can lead to ineffective advertising. It introduces concepts of incrementality and explores how reinforcement learning and 竜-greedy policies can help optimize promotional strategies on the Netflix homepage to maximize user engagement. The document also highlights the need for causal modeling to accurately measure the effects of promotional actions and improve decision-making.A Multi-Armed Bandit Framework For Recommendations at Netflix
A Multi-Armed Bandit Framework For Recommendations at NetflixJaya Kawale
油
The document discusses the application of a multi-armed bandit framework for improving content recommendations at Netflix, focusing on case studies for artwork optimization and billboard recommendation. It highlights challenges faced by traditional recommendation methods, the exploration-exploitation tradeoff, and various bandit algorithms used for enhancing user engagement. The findings suggest that personalized recommendations, particularly for lesser-known titles, significantly improve user interaction compared to random selections.Sequential Decision Making in Recommendations
Sequential Decision Making in RecommendationsJaya Kawale
油
The document discusses sequential decision-making in recommendation systems, particularly in the context of Netflix's personalized content recommendations for its 158 million subscribers across ~200 countries. It highlights challenges such as unknown state environments, rewards, and transitions, and explores multi-armed bandit strategies and reinforcement learning techniques for maximizing user satisfaction over time. Various methods and considerations for optimizing recommendations, including exploration versus exploitation and dealing with high-dimensional action spaces, are also presented.Reinforcement Learning in Practice: Contextual Bandits
Reinforcement Learning in Practice: Contextual BanditsMax Pagels
油
The document discusses the application of reinforcement learning (RL) and contextual bandits in real-world problems, particularly in recommendation systems. It highlights how business logic can bias data generation and hinder optimal supervised learning, proposing that random exploration through contextual bandits could offer a solution. The presentation includes case studies showing successful implementations in various sectors, emphasizing the ability to evaluate new policies effectively before deployment.Intro to Reinforcement Learning
Intro to Reinforcement LearningUtkarsh Garg
油
The document provides an introduction to reinforcement learning. It discusses how reinforcement learning allows agents to learn behaviors through trial-and-error interactions with an environment. The agent receives rewards or punishments that modify the likelihood of behaviors to maximize rewards over time. Examples are given of how dogs can be trained and how babies learn behaviors through reinforcement. Grid worlds are presented as a simple example problem to introduce key concepts before discussing more complex applications in domains like robotics, games, and self-driving cars.Causality without headaches
Causality without headachesBeno樽t Rostykus
油
This document discusses causal inference techniques for machine learning, including:
- Correlation does not imply causation, and observational data can be biased by confounding variables. Randomization and counterfactual modeling are introduced as alternatives.
- Inverse propensity scoring is presented as a method for estimating treatment effects from observational data by reweighting samples based on their propensity to receive treatment.
- Instrumental variable regression is discussed as another technique, using variables that influence the treatment but not the outcome except through treatment. Scalable methods for instrumental variable regression on large datasets are proposed.
- Challenges with weak instruments are noted, as instrumental variable estimates can become more biased than purely correlational models when instruments are weakFrom Practice to Theory in Learning from Massive Data by Charles Elkan at Big...
From Practice to Theory in Learning from Massive Data by Charles Elkan at Big...BigMine
油
The document discusses the importance of detecting anomalies in streaming data and making real-time predictions using Spark, along with Amazon's key recommendation algorithms. It emphasizes the shift from merely correlating user behavior to understanding causation through uplift modeling, which seeks to influence user decisions. Examples of optimizing treatments for customers based on past purchasing behavior illustrate the need for tailored marketing strategies to increase efficacy.Artwork Personalization at Netflix
Artwork Personalization at NetflixJustin Basilico
油
This document summarizes an presentation about personalizing artwork selection on Netflix using multi-armed bandit algorithms. Bandit algorithms were applied to choose representative, informative and engaging artwork for each title to maximize member satisfaction and retention. Contextual bandits were used to personalize artwork selection based on member preferences and context. Netflix deployed a system that precomputes personalized artwork using bandit models and caches the results to serve images quickly at scale. The system was able to lift engagement metrics based on A/B tests of the personalized artwork selection models.10 Lessons Learned from Building Machine Learning Systems
10 Lessons Learned from Building Machine Learning SystemsXavier Amatriain
油
The document outlines 10 lessons learned from building machine learning systems at Netflix, emphasizing the trade-off between data quantity and model complexity. It highlights the importance of thoughtful training data selection, evaluation techniques, and the interplay between user interface and algorithms. Key recommendations include optimizing hyperparameters wisely, understanding model dependencies, and making data-driven product decisions.AI to open more doors in Personal Finance Management (PFM)
AI to open more doors in Personal Finance Management (PFM)SK Reddy
油
AI and machine learning techniques are increasingly being used in personal finance management and credit scoring applications. Some key points from the document:
1. AI and machine learning can help open more doors by using non-traditional data sources like mobile phone metadata to assess creditworthiness for those without credit histories.
2. Various machine learning classifiers like random forests, SVMs, and neural networks are being applied to tasks like credit fraud detection, predicting bankruptcies, and nowcasting recessions.
3. Emerging techniques like uplift modeling and autoencoders are helping financial institutions better understand customer behaviors and tailor marketing campaigns.MachineLearning_Unit-I.pptx.pdtegfdxcdsfxf
MachineLearning_Unit-I.pptx.pdtegfdxcdsfxf22eg105n49
油
This document provides an overview of machine learning, defining it as the ability for computers to learn from experience without explicit programming. It details the evolution of machine learning from the 1950s to present, discussing various types of learning algorithms, including supervised, unsupervised, and reinforcement learning. Additionally, it outlines the machine learning process, key applications, and challenges such as overfitting and underfitting.Rl chapter 1 introduction
Rl chapter 1 introductionConnorShorten2
油
The document introduces reinforcement learning (RL) by discussing its core components: policies, rewards, value functions, and optional models. It highlights key challenges such as exploration versus exploitation and the importance of estimating values for effective decision-making. Examples of RL applications, including games and adaptive systems, illustrate the complexity and utility of RL in real-world scenarios.Scott Clark, Software Engineer, Yelp at MLconf SF
Scott Clark, Software Engineer, Yelp at MLconf SFMLconf
油
The document discusses optimal learning methodologies such as multi-armed bandits and Bayesian global optimization, focusing on efficiently collecting information to maximize payouts in experimental settings like A/B testing. It introduces the Metric Optimization Engine (MOE) as a solution for optimizing parameters by leveraging past performance data to suggest new, optimal configurations. MOE is currently utilized in production for optimizing various metrics and is open source, providing implementations of multi-armed bandit policies and global optimization techniques.Past, present, and future of Recommender Systems: an industry perspective
Past, present, and future of Recommender Systems: an industry perspectiveXavier Amatriain
油
The document discusses the evolution and challenges of recommender systems, focusing on traditional utility measurement, the Netflix Prize competition, and the significance of accuracy in predicting user preferences. It emphasizes the importance of incorporating various factors such as context, social recommendations, and the balance between exploration and exploitation in crafting effective recommendations. Additionally, it highlights lessons learned regarding the value of implicit feedback, careful training data selection, and the potential impact of user interface changes on model efficacy.Actionable Machine Learning
Actionable Machine LearningMeir Maor
油
This document discusses using machine learning to optimize future outcomes rather than just predict them. It explains that randomized studies are needed to accurately predict the effects of actions, but sometimes this is not possible with observational data alone. The document proposes techniques like transfer learning, common support analysis, and generative adversarial networks to help evaluate strategies without randomized trials by expanding the available data.Learning for exploration-exploitation in reinforcement learning. The dusk of ...
Learning for exploration-exploitation in reinforcement learning. The dusk of ...Universit辿 de Li竪ge (ULg)
油
The document discusses reinforcement learning and the exploration-exploitation tradeoff that agents face. It proposes learning exploration-exploitation strategies rather than relying on predefined formulas. The approach defines training problems, candidate strategies parameterized by formulas, a performance criterion, and optimizes strategies on training problems using an estimation of distribution algorithm. Simulation results show learned strategies outperform strategies from common formulas on matching test problems.Reinforcement Learning
Reinforcement LearningCloudxLab
油
Reinforcement learning (RL) involves an agent learning optimal actions to maximize rewards through trial and error in various environments, such as playing games or controlling robots. Key topics include policy generation, exploring action spaces, and addressing the credit assignment problem, where agents must determine which actions led to received rewards. Tools like OpenAI Gym provide simulation environments for RL training, while algorithms such as policy gradients and Markov decision processes are used to optimize learning and decision-making.Machine Learning Techniques all units .ppt
Machine Learning Techniques all units .pptvidhyav58
油
The document provides a comprehensive overview of machine learning (ML), defining it as a branch of artificial intelligence that allows computers to learn from data and improve over time. It covers fundamental concepts, types of learning (supervised, unsupervised, reinforcement), and the importance of data understanding, representation, and visualization. Additionally, it discusses challenges like the curse of dimensionality and the significance of simplicity in model design, emphasizing trends and applications across various fields.Mlintro 120730222641-phpapp01-210624192524
Mlintro 120730222641-phpapp01-210624192524Scott Domes
油
- Prof. Lior Rokach introduces machine learning and gives an overview of his background and research interests.
- Machine learning aims to develop systems that can learn from experience to improve their performance on some task. A key aspect is that the system improves its performance based on experience.
- Examples of machine learning applications include spam filtering, data mining, handwriting recognition, and more. Popular algorithms discussed include decision trees, neural networks, nearest neighbors, and ensemble methods.
- Challenges in machine learning like overfitting, dimensionality, and finding meaningful patterns are also covered.Applied Data Science for monetization: pitfalls, common misconceptions, and n...
Applied Data Science for monetization: pitfalls, common misconceptions, and n...DevGAMM Conference
油
The document discusses applied data science techniques for improving retention in e-commerce and online services, addressing common pitfalls and misconceptions. It emphasizes the importance of decision-making models like uplift modeling and multi-armed bandits over traditional predictive approaches for optimizing marketing strategies and updating game mechanics. Additionally, it highlights the significance of proper metric analysis using bootstrap methods to avoid misleading conclusions about performance metrics like ARPU.Causal inference in practice
Causal inference in practiceAmit Sharma
油
Amit Sharma discusses the concept of causality across various disciplines, emphasizing its importance in building recommender systems in social networks and estimating causal effects from observational data. The document highlights numerous studies demonstrating causal inference in fields such as economics, political science, and human behavior, and presents various methods for causal estimation including graphical models and counterfactual reasoning. The findings stress the challenges of estimating causal effects, particularly in the context of online recommendations and treatments, while advocating for an advanced understanding of causality to improve practical applications.MachineLearning_Unit-I.pptxScrum.pptxAgile Model.pptxAgile Model.pptxAgile Mo...
MachineLearning_Unit-I.pptxScrum.pptxAgile Model.pptxAgile Model.pptxAgile Mo...22eg105n11
油
The document provides a comprehensive overview of machine learning, defining it as a field enabling computers to learn from experience without explicit programming. It covers the evolution of machine learning, types of algorithms such as supervised, unsupervised, and reinforcement learning, and their various applications in real-world scenarios. Additionally, the document discusses steps in the machine learning process, data preparation, model training, and challenges like overfitting and underfitting.anintroductiontoreinforcementlearning-180912151720.pdf
anintroductiontoreinforcementlearning-180912151720.pdfssuseradaf5f
油
Dr. Subrat Panda gave an overview of reinforcement learning. He defined reinforcement learning as dealing with agents that must sense and act upon their environment to receive delayed scalar feedback in the form of rewards. The goal is to learn an optimal policy that maps states to actions to maximize total future discounted reward. Q-learning is introduced as a way to estimate state-action values directly without needing a model of the environment. Q-learning updates estimates based on new observations and prior estimates in a process called bootstrapping. Exploration must be balanced with exploitation of current knowledge. Real-world applications of deep reinforcement learning were discussed.An introduction to reinforcement learning
An introduction to reinforcement learningSubrat Panda, PhD
油
Dr. Subrat Panda gave an introduction to reinforcement learning. He defined reinforcement learning as dealing with agents that must sense and act upon their environment to receive delayed scalar feedback in the form of rewards. He described key concepts like the Markov decision process framework, value functions, Q-functions, exploration vs exploitation, and extensions like deep reinforcement learning. He listed several real-world applications of reinforcement learning and resources for learning more.On the Dynamics of Machine Learning Algorithms and Behavioral Game Theory
On the Dynamics of Machine Learning Algorithms and Behavioral Game TheoryRikiya Takahashi
油
The document discusses the integration of machine learning algorithms with behavioral game theory to improve decision-making in multi-agent environments, emphasizing the complexities of human behavior influenced by irrationality and uncertainty. It explores the limitations of traditional rational models, introducing concepts like hyperbolic discounting, multiple equilibria, and the necessity for better modeling of human decision processes. Additionally, it addresses the implications of these theories for social sciences and market dynamics, advocating for a focus on transitional states rather than static equilibria.Optimal Learning for Fun and Profit with MOE
Optimal Learning for Fun and Profit with MOEYelp Engineering
油
The document discusses optimal learning strategies, focusing on multi-armed bandits and Bayesian global optimization for efficient information collection in expensive settings. It introduces the Metric Optimization Engine (MOE), which optimizes parameters for improved performance in A/B testing and other complex systems. MOE is now live and open source, designed to handle various applications from click-through rates to machine learning parameter tuning.Chapter01.ppt
Chapter01.pptbutest
油
The document provides an overview of a machine learning and knowledge discovery course. It outlines the course objectives, components, and topics that will be covered, including machine learning algorithms, experimental methodology, and two research papers. It also discusses what machine learning and knowledge discovery are, and provides examples of typical tasks like predicting customer behavior or medical outcomes.Gas Exchange in Insects and structures 01
Gas Exchange in Insects and structures 01PhoebeAkinyi1
油
Description of structures involved in gaseous exchangeMOLD -GENERAL CHARACTERISTICS AND CLASSIFICATION
MOLD -GENERAL CHARACTERISTICS AND CLASSIFICATIONaparnamp966
油
This is a brief note on types of organism on food with special focus on molds. This includes their general characteristics, spores; their classification, mycotoxins, and how the molds affect food. More Related Content
Similar to Sprezzatura - Roelof van Zwol - May 2018 (20)
Artwork Personalization at Netflix
Artwork Personalization at NetflixJustin Basilico
油
This document summarizes an presentation about personalizing artwork selection on Netflix using multi-armed bandit algorithms. Bandit algorithms were applied to choose representative, informative and engaging artwork for each title to maximize member satisfaction and retention. Contextual bandits were used to personalize artwork selection based on member preferences and context. Netflix deployed a system that precomputes personalized artwork using bandit models and caches the results to serve images quickly at scale. The system was able to lift engagement metrics based on A/B tests of the personalized artwork selection models.10 Lessons Learned from Building Machine Learning Systems
10 Lessons Learned from Building Machine Learning SystemsXavier Amatriain
油
The document outlines 10 lessons learned from building machine learning systems at Netflix, emphasizing the trade-off between data quantity and model complexity. It highlights the importance of thoughtful training data selection, evaluation techniques, and the interplay between user interface and algorithms. Key recommendations include optimizing hyperparameters wisely, understanding model dependencies, and making data-driven product decisions.AI to open more doors in Personal Finance Management (PFM)
AI to open more doors in Personal Finance Management (PFM)SK Reddy
油
AI and machine learning techniques are increasingly being used in personal finance management and credit scoring applications. Some key points from the document:
1. AI and machine learning can help open more doors by using non-traditional data sources like mobile phone metadata to assess creditworthiness for those without credit histories.
2. Various machine learning classifiers like random forests, SVMs, and neural networks are being applied to tasks like credit fraud detection, predicting bankruptcies, and nowcasting recessions.
3. Emerging techniques like uplift modeling and autoencoders are helping financial institutions better understand customer behaviors and tailor marketing campaigns.MachineLearning_Unit-I.pptx.pdtegfdxcdsfxf
MachineLearning_Unit-I.pptx.pdtegfdxcdsfxf22eg105n49
油
This document provides an overview of machine learning, defining it as the ability for computers to learn from experience without explicit programming. It details the evolution of machine learning from the 1950s to present, discussing various types of learning algorithms, including supervised, unsupervised, and reinforcement learning. Additionally, it outlines the machine learning process, key applications, and challenges such as overfitting and underfitting.Rl chapter 1 introduction
Rl chapter 1 introductionConnorShorten2
油
The document introduces reinforcement learning (RL) by discussing its core components: policies, rewards, value functions, and optional models. It highlights key challenges such as exploration versus exploitation and the importance of estimating values for effective decision-making. Examples of RL applications, including games and adaptive systems, illustrate the complexity and utility of RL in real-world scenarios.Scott Clark, Software Engineer, Yelp at MLconf SF
Scott Clark, Software Engineer, Yelp at MLconf SFMLconf
油
The document discusses optimal learning methodologies such as multi-armed bandits and Bayesian global optimization, focusing on efficiently collecting information to maximize payouts in experimental settings like A/B testing. It introduces the Metric Optimization Engine (MOE) as a solution for optimizing parameters by leveraging past performance data to suggest new, optimal configurations. MOE is currently utilized in production for optimizing various metrics and is open source, providing implementations of multi-armed bandit policies and global optimization techniques.Past, present, and future of Recommender Systems: an industry perspective
Past, present, and future of Recommender Systems: an industry perspectiveXavier Amatriain
油
The document discusses the evolution and challenges of recommender systems, focusing on traditional utility measurement, the Netflix Prize competition, and the significance of accuracy in predicting user preferences. It emphasizes the importance of incorporating various factors such as context, social recommendations, and the balance between exploration and exploitation in crafting effective recommendations. Additionally, it highlights lessons learned regarding the value of implicit feedback, careful training data selection, and the potential impact of user interface changes on model efficacy.Actionable Machine Learning
Actionable Machine LearningMeir Maor
油
This document discusses using machine learning to optimize future outcomes rather than just predict them. It explains that randomized studies are needed to accurately predict the effects of actions, but sometimes this is not possible with observational data alone. The document proposes techniques like transfer learning, common support analysis, and generative adversarial networks to help evaluate strategies without randomized trials by expanding the available data.Learning for exploration-exploitation in reinforcement learning. The dusk of ...
Learning for exploration-exploitation in reinforcement learning. The dusk of ...Universit辿 de Li竪ge (ULg)
油
The document discusses reinforcement learning and the exploration-exploitation tradeoff that agents face. It proposes learning exploration-exploitation strategies rather than relying on predefined formulas. The approach defines training problems, candidate strategies parameterized by formulas, a performance criterion, and optimizes strategies on training problems using an estimation of distribution algorithm. Simulation results show learned strategies outperform strategies from common formulas on matching test problems.Reinforcement Learning
Reinforcement LearningCloudxLab
油
Reinforcement learning (RL) involves an agent learning optimal actions to maximize rewards through trial and error in various environments, such as playing games or controlling robots. Key topics include policy generation, exploring action spaces, and addressing the credit assignment problem, where agents must determine which actions led to received rewards. Tools like OpenAI Gym provide simulation environments for RL training, while algorithms such as policy gradients and Markov decision processes are used to optimize learning and decision-making.Machine Learning Techniques all units .ppt
Machine Learning Techniques all units .pptvidhyav58
油
The document provides a comprehensive overview of machine learning (ML), defining it as a branch of artificial intelligence that allows computers to learn from data and improve over time. It covers fundamental concepts, types of learning (supervised, unsupervised, reinforcement), and the importance of data understanding, representation, and visualization. Additionally, it discusses challenges like the curse of dimensionality and the significance of simplicity in model design, emphasizing trends and applications across various fields.Mlintro 120730222641-phpapp01-210624192524
Mlintro 120730222641-phpapp01-210624192524Scott Domes
油
- Prof. Lior Rokach introduces machine learning and gives an overview of his background and research interests.
- Machine learning aims to develop systems that can learn from experience to improve their performance on some task. A key aspect is that the system improves its performance based on experience.
- Examples of machine learning applications include spam filtering, data mining, handwriting recognition, and more. Popular algorithms discussed include decision trees, neural networks, nearest neighbors, and ensemble methods.
- Challenges in machine learning like overfitting, dimensionality, and finding meaningful patterns are also covered.Applied Data Science for monetization: pitfalls, common misconceptions, and n...
Applied Data Science for monetization: pitfalls, common misconceptions, and n...DevGAMM Conference
油
The document discusses applied data science techniques for improving retention in e-commerce and online services, addressing common pitfalls and misconceptions. It emphasizes the importance of decision-making models like uplift modeling and multi-armed bandits over traditional predictive approaches for optimizing marketing strategies and updating game mechanics. Additionally, it highlights the significance of proper metric analysis using bootstrap methods to avoid misleading conclusions about performance metrics like ARPU.Causal inference in practice
Causal inference in practiceAmit Sharma
油
Amit Sharma discusses the concept of causality across various disciplines, emphasizing its importance in building recommender systems in social networks and estimating causal effects from observational data. The document highlights numerous studies demonstrating causal inference in fields such as economics, political science, and human behavior, and presents various methods for causal estimation including graphical models and counterfactual reasoning. The findings stress the challenges of estimating causal effects, particularly in the context of online recommendations and treatments, while advocating for an advanced understanding of causality to improve practical applications.MachineLearning_Unit-I.pptxScrum.pptxAgile Model.pptxAgile Model.pptxAgile Mo...
MachineLearning_Unit-I.pptxScrum.pptxAgile Model.pptxAgile Model.pptxAgile Mo...22eg105n11
油
The document provides a comprehensive overview of machine learning, defining it as a field enabling computers to learn from experience without explicit programming. It covers the evolution of machine learning, types of algorithms such as supervised, unsupervised, and reinforcement learning, and their various applications in real-world scenarios. Additionally, the document discusses steps in the machine learning process, data preparation, model training, and challenges like overfitting and underfitting.anintroductiontoreinforcementlearning-180912151720.pdf
anintroductiontoreinforcementlearning-180912151720.pdfssuseradaf5f
油
Dr. Subrat Panda gave an overview of reinforcement learning. He defined reinforcement learning as dealing with agents that must sense and act upon their environment to receive delayed scalar feedback in the form of rewards. The goal is to learn an optimal policy that maps states to actions to maximize total future discounted reward. Q-learning is introduced as a way to estimate state-action values directly without needing a model of the environment. Q-learning updates estimates based on new observations and prior estimates in a process called bootstrapping. Exploration must be balanced with exploitation of current knowledge. Real-world applications of deep reinforcement learning were discussed.An introduction to reinforcement learning
An introduction to reinforcement learningSubrat Panda, PhD
油
Dr. Subrat Panda gave an introduction to reinforcement learning. He defined reinforcement learning as dealing with agents that must sense and act upon their environment to receive delayed scalar feedback in the form of rewards. He described key concepts like the Markov decision process framework, value functions, Q-functions, exploration vs exploitation, and extensions like deep reinforcement learning. He listed several real-world applications of reinforcement learning and resources for learning more.On the Dynamics of Machine Learning Algorithms and Behavioral Game Theory
On the Dynamics of Machine Learning Algorithms and Behavioral Game TheoryRikiya Takahashi
油
The document discusses the integration of machine learning algorithms with behavioral game theory to improve decision-making in multi-agent environments, emphasizing the complexities of human behavior influenced by irrationality and uncertainty. It explores the limitations of traditional rational models, introducing concepts like hyperbolic discounting, multiple equilibria, and the necessity for better modeling of human decision processes. Additionally, it addresses the implications of these theories for social sciences and market dynamics, advocating for a focus on transitional states rather than static equilibria.Optimal Learning for Fun and Profit with MOE
Optimal Learning for Fun and Profit with MOEYelp Engineering
油
The document discusses optimal learning strategies, focusing on multi-armed bandits and Bayesian global optimization for efficient information collection in expensive settings. It introduces the Metric Optimization Engine (MOE), which optimizes parameters for improved performance in A/B testing and other complex systems. MOE is now live and open source, designed to handle various applications from click-through rates to machine learning parameter tuning.Chapter01.ppt
Chapter01.pptbutest
油
The document provides an overview of a machine learning and knowledge discovery course. It outlines the course objectives, components, and topics that will be covered, including machine learning algorithms, experimental methodology, and two research papers. It also discusses what machine learning and knowledge discovery are, and provides examples of typical tasks like predicting customer behavior or medical outcomes.Learning for exploration-exploitation in reinforcement learning. The dusk of ...
Learning for exploration-exploitation in reinforcement learning. The dusk of ...Universit辿 de Li竪ge (ULg)
油
Recently uploaded (20)
Gas Exchange in Insects and structures 01
Gas Exchange in Insects and structures 01PhoebeAkinyi1
油
Description of structures involved in gaseous exchangeMOLD -GENERAL CHARACTERISTICS AND CLASSIFICATION
MOLD -GENERAL CHARACTERISTICS AND CLASSIFICATIONaparnamp966
油
This is a brief note on types of organism on food with special focus on molds. This includes their general characteristics, spores; their classification, mycotoxins, and how the molds affect food. GBSN_Unit 3 - Medical and surgical Asepsis
GBSN_Unit 3 - Medical and surgical AsepsisAreesha Ahmad
油
Microbiology for Nursing students - According to New PNC course curriculum - 2025
GBSN__Unit 2 - Control of Microorganisms
GBSN__Unit 2 - Control of MicroorganismsAreesha Ahmad
油
Microbiology for Nursing students - According to New PNC course curriculum - 2025
TISSUE TRANSPLANTATTION and IT'S IMPORTANCE IS DISCUSSED
TISSUE TRANSPLANTATTION and IT'S IMPORTANCE IS DISCUSSEDPhoebeAkinyi1
油
Tissue transplant in immunologyHOW TO FACE THREATS FROM THE FORCES OF NATURE EXISTING ON PLANET EARTH.pdf
HOW TO FACE THREATS FROM THE FORCES OF NATURE EXISTING ON PLANET EARTH.pdfFaga1939
油
This article aims to present how to deal with localized or global threats to human beings caused by the forces of nature that exist on planet Earth. The threats to human beings caused by the forces of nature that exist on planet Earth include earthquakes and tsunamis that can affect specific areas of the planet, volcanic eruptions that can affect specific areas of the planet or have a global impact, the cooling of the Earth's core and the inversion of the Earth's magnetic poles, both of which have a global impact. Specific measures are proposed to deal with each of the threats, but in addition to these, it is necessary to create a global structure, a World Organization for Defense Against Natural Disasters with global scope linked to the UN (United Nations) that has the capacity to technically coordinate the actions of countries around the world in dealing with these threats. The creation of this body is absolutely necessary because most of the threats from the forces of nature that exist on planet Earth have a global impact. To carry out its functions, this body must have financial resources from a global fund against natural disasters of global scope to be maintained by all countries on the planet and administered by the UN.Abzymes mimickers in catalytic reactions at nanoscales
Abzymes mimickers in catalytic reactions at nanoscalesOrchideaMariaLecian
油
Title: Abzymes mimickers in
catalytic reactions at nanoscales
Speaker: Orchidea Maria Lecian
Authors: Orchidea Maria Lecian, Sergey Suchkov
Talk presented at 4th International Conference on
Advanced Nanomaterials and Nanotechnology
12-13 June 2025, Rome, Italy on 12 June 2025.Science Holiday Homework (interesting slide )
Science Holiday Homework (interesting slide )aryanxkohli88
油
science holiday homework
good for childrenThe Emergence of Signatures of AGI: The Physics of Learning
The Emergence of Signatures of AGI: The Physics of LearningCharles Martin
油
A talk for the Cybernetic Society
How Psychology Can Power Product Decisions: A Human-Centered Blueprint- Shray...
How Psychology Can Power Product Decisions: A Human-Centered Blueprint- Shray...ShrayasiRoy
油
In an era where users are bombarded with endless choices, capturing attention and driving meaningful engagement isn't just about building features it's about understanding what drives human behavior at its core. This presentation offers a deep dive into how psychological principles can inform smarter, more intuitive product decisions. Its not just theory its a hands-on blueprint for applying human-centered thinking at every stage of product development.
Grounded in behavioral science, consumer psychology, and cognitive design, this deck unpacks the key psychological drivers behind user motivation, decision-making, emotional engagement, and habit formation. From attention economics and dopamine-driven interactions to trust cues, loss aversion, and the paradox of choice you'll see how to harness what the mind naturally does to build digital products people dont just use they return to, talk about, and even advocate for.
Well explore:
Cognitive biases that shape user perception and choices and how to use them to your advantage without crossing ethical lines.
Emotional design principles that build trust, trigger desire, and turn features into feelings.
User behavior loops that build stickiness, deepen retention, and create emotional investments.
Psychological friction when to reduce it for conversion, and when to add it for intentionality.
Dark patterns vs. ethical persuasion the thin line between influence and manipulation.
A/B testing with a psychological lens learning not just what works, but why it works.
This is more than UX research or product marketing fluff. Its a call to build with empathy, backed by science, and sharpened by real-world product thinking. Whether youre shaping onboarding flows, gamifying engagement, designing pricing models, or rethinking retention strategies this deck will arm you with a new lens: the human mind.
Perfect for product managers, growth strategists, UX designers, behavioral scientists, and anyone serious about building products that resonate, retain, and inspire action.
Because at the heart of every great product is a human.
Lets start building for the brain not just the screen.THE CIRCULATORY SYSTEM GRADE 9 SCIENCE.pptx
THE CIRCULATORY SYSTEM GRADE 9 SCIENCE.pptxroselyncatacutan
油
The Circulatory System Grade 9 Science Quarter 1 Week 1GBSN_ Unit 1 - Introduction to Microbiology
GBSN_ Unit 1 - Introduction to MicrobiologyAreesha Ahmad
油
Microbiology for Nursing students - According to New PNC course curriculum - 2025What is Skeleton system.pptx by rizwan bashir
What is Skeleton system.pptx by rizwan bashirbhatbashir421
油
science, any system of knowledge that is concerned with the physical world and its phenomena and that entails unbiased observations and systematic experimentation. In general, a science involves a pursuit of knowledge covering general truths or the operations of fundamental laws.
Science can be divided into different branches based on the subject of study. The physical sciences study the inorganic world and comprise the fields of astronomy, physics, chemistry, and the Earth sciences. The biological sciences such as biology and medicine study the organic world of life and its processes. Social sciences like anthropology and economics study the social and cultural aspects of human behaviour.
Science is further treated in a number of articles. For the history of Western and Eastern science, see science, history of. For the conceptualization of science and its interrelationships with culture, see science, philosophy of. For the basic aspects of the scientific approach, see physical science, principles of; and scientific method.
Model of a molecule. Atom, Biology, Molecular Structure, Science, Science and Technology. Homepage 2010 arts and entertainment, history and society
Britannica Quiz
Science Quiz
The Editors of Encyclopaedia Britannica
This article was most recently revised and updated by Encyclopaedia Britannica.
physics
Table of Contents
Introduction & Top Questions
The scope of physics
The methodology of physics
Relations between physics and other disciplines and society
References & Edit History
Related Topics
Images & Videos
Bernoulli model of gas pressureillustration of Robert Hooke's law of elasticity of materialsLaser Interferometer Space Antenna (LISA)curved space-timetemperature scalesBrownian particleElectromagnetic radiationDiffraction gratingMillikan oil-drop experiment
Robert Millikan's method for measuring the charge of an electron
For Students
Diffraction grating
physics summary
Quizzes
Michael Faraday (L) English physicist and chemist (electromagnetism) and John Frederic Daniell (R) British chemist and meteorologist who invented the Daniell cell.
Faces of Science
Italian-born physicist Dr. Enrico Fermi draws a diagram at a blackboard with mathematical equations. circa 1950.
Physics and Natural Law
Italian physicist Guglielmo Marconi at work in the wireless room of his yacht Electra, c. 1920.
All About Physics Quiz
atom. Orange and green illustration of protons and neutrons creating the nucleus of an atom.
Another Physics Quiz
Encyclopaedia Britannica thistle graphic to be used with a Mendel/Consumer quiz in place of a photograph.
How Much Do You Know About Physics?
Related Questions
Why does physics work in SI units?
Read Next
Highway Night Traffic Portland, drive, driving, car, automobile.
What's the Difference Between Speed and Velocity?
Galaxy clusters like Abell 2744 can act as a natural cosmic lens, magnifying light from more distant, background objects through gravity. Chromatography 際際滷s for the course of Introduction to Biology and Chemistry...
Chromatography 際際滷s for the course of Introduction to Biology and Chemistry...Md. Arif Shahriar
油
This presentation was made as a coursework of "Introduction to Biology and Chemistry for Computation" Course under Fatema Tuj Johora ma'am at Daffodil International University.Ad
Sprezzatura - Roelof van Zwol - May 2018
- 2. Sprez.za.tura It is an art which does not seem to be an art. One must avoid affectation and practice in all things. A certain sprezzatura, disdain or carelessness, so as to conceal art, and make whatever is done or said appear to be without effort and almost without any thought about it ... obvious effort is the antithesis of grace. Baldassare Castiglione (1478-1529)
- 3. Is machine learning an art?
- 4. When done well, recommendations are perceived a natural extension of the service 98% Match
- 7. Introducing new content Who will watch the show? How many members will watch the show? Which canvas to use? When to promote?
- 8. Overview Correlation Causation Online-learning Incrementality
- 10. Should you stop buying margarine, to save your marriage?
- 11. Correlation (X,Y) is high, does it mean X causes Y? Y causes X?
- 12. Correlation (X,Y) is high, does it mean X causes Y? Y causes X? In general, neither! Most common reason: unobserved confounder X Y Unobserved Observed Observed C Omited variable bias
- 13. Advertising W1 W2 W3 W4 W5 Probability of buying: Advertise?$ $ $ $
- 14. Advertising High probability of conversion the day before weekly groceries irrespective of adverts shown Effect of Pampers ads is null in this case. Traditional (correlational) machine learning will fail and waste $ on useless ads W1 W2 W3 W4 W5 Probability of buying: Advertise?$ $ $ $ in practice, Cost-Per-Incremental-Acquisition can be > 100x Cost-Per-Acquisition (!!!!!)
- 15. Netflix Promotions Netflix homepage is an expensive real-estate (opportunity cost): - so many titles to promote - so few opportunities to win a moment of truth D1 D2 D3 D4 D5 Promote?
- 16. Netflix Promotions Netflix homepage is an expensive real-estate (opportunity cost): - so many titles to promote - so few opportunities to win a moment of truth Traditional (correlational) ML systems: - take action if probability of positive reward is high, irrespective of reward base rate - dont model incremental effect of taking action D1 D2 D3 D4 D5 Promote?
- 17. Surely we can do better!
- 18. CASE STUDY: Content promotion through Billboard 98% Match
- 19. Online Learning
- 20. Background and notation Title t belongs to the pool of candidate titles T, eligible for promotion in Billboard when member m visits the homepage Let xm,t be a context vector for member m and title t Let ym,t be the label indicating a play of title t by member m from the homepage, after having seen a billboard.
- 21. What (sequence of) actions will maximize the cumulative reward? Reinforcement Learning Multi-Armed Bandits Acknowledge the need for balancing exploration and exploitation Allow sub-optimal actions, to collect unbiased treatment effects and learn the probability distributions over the space of possible actions. B B7 7 7B 7 77 ? R3 R2 R1
- 22. 狼-greedy policy Explore Collect experimental data With 狼 probability, select at random a title for promotion in Billboard Log context (xm,t ) Causal observations of play-feedback (ym,t ) Exploit Train on the experimental data With (1-狼) probability, select the optimal title for promotion Alternatives: UCB, Thompson Sampling
- 23. Greedy exploit model Learn a model per title to predict likelihood of play P(ym,t | xm,t ,T) = ( f(xm,t , ) ) Pick winning title: t = argmax P(ym,t | xm,t ,T) Various models can be used to predict probability of play, such as logistic regression, GBDT, neural networks
- 24. Considerations for 狼-greedy policy Explore Bandwidth allocation and cost of exploration New vs existing titles Exploit Model synchronisation Title availability (group censoring) Observation window Frequency of model update Incremental updates vs batch training Stationarity of title popularities ? ? ? ? ?? ?
- 25. Online learning works great for title cold start scenarios, but... MABs are greedy, not lift-based!
- 26. Incrementality
- 27. Incrementality-based policy Goal: Select title for promotion that benefits most from being shown in billboard Member can play title from other sections on the homepage or search Popular titles likely to appear on homepage anyway: Trending Now Better utilize most expensive real-estate on the homepage! Define policy to be incremental with respect to probability of play
- 28. Incrementality-based policy Goal: Select title for promotion that benefits most from being shown in billboard t = argmax [ P(ym,t | xm,t , T, b=1) - P(ym,t | xm,t , T, b=0) ] Where b is an indicator for the treatment of a title being shown in billboard (b=1), versus not being shown in billboard (b=0)
- 29. Offline evaluation: Replay [Li et al, 2010] Relies upon uniform exploration data. For every record in the uniform exploration log {context, title k shown, reward, list of candidates} For every record: Evaluate the trained model for all the titles in the candidate pool. Pick the winning title k Keep the record in history if k = k (the title impressed in the logged data) else discard it. Compute the metrics from the history.
- 30. Offline evaluation: Replay [Li et al, 2010] Uniform Exploration Data - Unbiased evaluation Evaluation Data Train Data Trained Model Reveal context x Use reward only if k = k Winner title k context,title,reward context,title,reward context,title,reward Take Rate = # Plays # Matches
- 31. Offline replay Greedy exploit has higher replay take rate than incrementality based model. Incrementality Based Policy sacrifices replay by selecting a lesser known title that would benefit from being shown on the Billboard. Lift in Replay in the various algorithms as compared to the Random baseline
- 32. Which titles benefit from Billboard promotion? Title A has a low baseline probability of play, however when the billboard is shown the probability of play increases substantially! Title C has higher baseline probability and may not benefit as much from being shown on the Billboard. Scatter plot of incremental vs baseline probability of play for various members.
- 33. Online observations Online take rates for take rates follow the offline patterns. Our implementation of incrementality is able to shift engagement within the candidate pool.
- 34. In Summary
- 35. Correlation, causation, and incrementality Most ML algorithms are correlational, e.g. based on observational data In this context, the Explore-exploit models are causal E.g. we train models based on experimental data, where we are in control of the randomization Incrementality can be defined as the causal lift in a metric of interest For instance, the change in probability of play for a title in a session, when a billboard is shown for that title to a member