Classification metrics
1 like764 views
The document discusses various evaluation metrics that can be used for binary classification and click prediction, including AUC, RIG, LogLoss, precision, recall, and F1. It notes that AUC ignores predicted probabilities and considers type 1 and type 2 errors equally. RIG is bad for comparing models with different data distributions but can be used to compare multiple models trained on the same data. The document also provides a reference for more information on offline and online predictive model performance evaluations.
1 of 15
Downloaded 11 times
Ad
Recommended
Alpaydin - Chapter 2
Alpaydin - Chapter 2butest
油
This document introduces machine learning and supervised learning. It discusses learning a classifier from labeled examples to predict a target variable. The key points covered are:
- Supervised learning involves learning a function that maps inputs to outputs from example input-output pairs.
- The goal is to learn a hypothesis h that has low error on the training set and generalizes well to new examples.
- The version space is the set of all hypotheses consistent with the training data.
- Controlling the complexity of the hypothesis class H via measures like VC dimension can improve generalization.
- For classification, multiple target classes are handled by learning one hypothesis per class. Regression learns a real-valued target function.
- There is a tradeDimensionality Reduction
Dimensionality ReductionSaad Elbeleidy
油
A brief overview of dimensionality reduction; why it's needed, different types, and some sample methods.AI 7 | Constraint Satisfaction Problem
AI 7 | Constraint Satisfaction ProblemMohammad Imam Hossain
油
The document provides an overview of constraint satisfaction problems (CSPs). It defines a CSP as consisting of variables with domains of possible values, and constraints specifying allowed value combinations. CSPs can represent many problems using variables and constraints rather than explicit state representations. Backtracking search is commonly used to solve CSPs by trying value assignments and backtracking when constraints are violated.Hadoop File system (HDFS)
Hadoop File system (HDFS)Prashant Gupta
油
HDFS is a Java-based file system that provides scalable and reliable data storage, and it was designed to span large clusters of commodity servers. HDFS has demonstrated production scalability of up to 200 PB of storage and a single cluster of 4500 servers, supporting close to a billion files and blocks.Fuzzy c means manual work
Fuzzy c means manual workDr.E.N.Sathishkumar
油
Fuzzy C-means is an extension of k-means clustering that allows data points to belong to multiple clusters simultaneously. It assigns a membership value between 0 and 1 to each data point for each cluster, indicating the likelihood of membership. The example demonstrates fuzzy C-means clustering on a dataset with 6 data points and 2 clusters, calculating the membership values and distances over multiple iterations until the cluster centroids stabilize.Fuzzy Clustering(C-means, K-means)
Fuzzy Clustering(C-means, K-means)UMBC
油
Clustering
Goals of Clustering
Clustering Application
K-means
C-means
Fuzzy Clustering Application
Iris dataset clustering
SKFCM
Randomized algorithms ver 1.0
Randomized algorithms ver 1.0Dr. C.V. Suresh Babu
油
This document discusses randomized algorithms. It begins by listing different categories of algorithms, including randomized algorithms. Randomized algorithms introduce randomness into the algorithm to avoid worst-case behavior and find efficient approximate solutions. Quicksort is presented as an example randomized algorithm, where randomness improves its average runtime from quadratic to linear. The document also discusses the randomized closest pair algorithm and a randomized algorithm for primality testing. Both introduce randomness to improve efficiency compared to deterministic algorithms for the same problems.Data streaming fundamentals
Data streaming fundamentalsMohammed Fazuluddin
油
This document provides an overview of data streaming fundamentals and tools. It discusses how data streaming processes unbounded, continuous data streams in real-time as opposed to static datasets. The key aspects covered include data streaming architecture, specifically the lambda architecture, and popular open source data streaming tools like Apache Spark, Apache Flink, Apache Samza, Apache Storm, Apache Kafka, Apache Flume, Apache NiFi, Apache Ignite and Apache Apex.daa-unit-3-greedy method
daa-unit-3-greedy methodhodcsencet
油
The document discusses the greedy method algorithmic approach. It provides an overview of greedy algorithms including that they make locally optimal choices at each step to find a global optimal solution. The document also provides examples of problems that can be solved using greedy methods like job sequencing, the knapsack problem, finding minimum spanning trees, and single source shortest paths. It summarizes control flow and applications of greedy algorithms.convex hull
convex hullravikirankalal
油
The document discusses the convex hull algorithm. It begins by defining a convex hull as the shape a rubber band would take if stretched around pins on a board. It then provides explanations of extreme points, edges, and applications of convex hulls. Various algorithms for finding convex hulls are presented, including divide and conquer in O(n log n) time and Jarvis march in O(n^2) time in the worst case.Birch Algorithm With Solved Example
Birch Algorithm With Solved Examplekailash shaw
油
BIRCH油(balanced iterative reducing and clustering using hierarchies) is an unsupervised油data-mining油algorithm used to perform油hierarchical clustering油over, particularly large data sets.
Recurrent neural networks rnn
Recurrent neural networks rnnKuppusamy P
油
The document discusses recurrent neural networks (RNNs) and long short-term memory (LSTM) networks. It provides details on the architecture of RNNs including forward and back propagation. LSTMs are described as a type of RNN that can learn long-term dependencies using forget, input and output gates to control the cell state. Examples of applications for RNNs and LSTMs include language modeling, machine translation, speech recognition, and generating image descriptions.Conversion from infix to prefix using stack
Conversion from infix to prefix using stackHaqnawaz Ch
油
This document is a data structures assignment that contains two tasks:
1) Converting an infix expression to postfix notation using a stack. The example infix expression is converted to postfix as -+-AB*C+DE+FG.
2) Evaluating the postfix expression from the first task by assigning values to variables and using a stack to calculate the result, which is 33.Logical Agents
Logical AgentsYasir Khan
油
This document summarizes a chapter about logical agents from an artificial intelligence textbook. It discusses how logical agents can represent knowledge and reason using that knowledge. Key points include: logical agents use a knowledge base to store facts and make inferences; propositional logic and inference rules like modus ponens allow agents to derive new knowledge; the Wumpus World example demonstrates logical reasoning to solve problems based on partial observations.Big Data Visualization
Big Data VisualizationRaffael Marty
油
Raffael Marty gave a presentation on big data visualization. He discussed using visualization to discover patterns in large datasets and presenting security information on dashboards. Effective dashboards provide context, highlight important comparisons and metrics, and use aesthetically pleasing designs. Integration with security information management systems requires parsing and formatting data and providing interfaces for querying and analysis. Marty is working on tools for big data analytics, custom visualization workflows, and hunting for anomalies. He invited attendees to join an online community for discussing security visualization.Harris corner detector and face recognition
Harris corner detector and face recognitionShih Wei Huang
油
際際滷s taken from: Matching with Invariant Features, Darya
Frolova, Denis Simakov, The Weizmann Institute of Science,
March 2004Syntax Directed Definition and its applications
Syntax Directed Definition and its applicationsShivanandManjaragi2
油
A syntax-directed definition (SDD) is a context-free grammar with attributes and semantic rules. Attributes are associated with grammar symbols and rules are associated with productions. An SDD can be evaluated on a parse tree to compute attribute values at each node. There are two types of attributes: synthesized attributes depend on child nodes, while inherited attributes depend on parent or sibling nodes. The order of evaluation is determined by a dependency graph showing the flow of information between attribute instances.Data preprocessing using Machine Learning 
Data preprocessing using Machine Learning Gopal Sakarkar
油
Data Preprocessing plays an important role in data cleaning .
Good data preparation is key to producing valid and reliable
Machine Learning models. Introduction to prolog
Introduction to prologHarry Potter
油
Prolog is a declarative logic programming language where programs consist of facts and rules. Facts are terms that are always true, while rules define relationships between terms using logic notation "if-then". A Prolog program is run by asking queries of the program's database. Variables must start with an uppercase letter and are used to represent unknown values, while atoms are constants that represent known values.Hadoop MapReduce Fundamentals
Hadoop MapReduce FundamentalsLynn Langit
油
This document provides an overview of the Hadoop MapReduce Fundamentals course. It discusses what Hadoop is, why it is used, common business problems it can address, and companies that use Hadoop. It also outlines the core parts of Hadoop distributions and the Hadoop ecosystem. Additionally, it covers common MapReduce concepts like HDFS, the MapReduce programming model, and Hadoop distributions. The document includes several code examples and screenshots related to Hadoop and MapReduce.Convex Hull Algorithms
Convex Hull AlgorithmsKasun Ranga Wijeweera
油
This document discusses convex hull algorithms. It defines a convex set as one where any line segment between two points in the set is also contained in the set. The convex hull of a set of points is the smallest convex set containing those points. Intuitively, in 2D the convex hull is the shape formed by stretching a rubber band around nails at each point, and in 3D it is the shape formed by stretching plastic wrap tightly around the points. The document then lists and describes several existing convex hull algorithms and provides an overview of an interior points algorithm that identifies non-extreme points based on whether they lie within triangles formed by other points.Fractional knapsack class 13
Fractional knapsack class 13Kumar
油
The document discusses two types of knapsack problems - the 0-1 knapsack problem and the fractional knapsack problem. The 0-1 knapsack problem uses dynamic programming to determine how to fill a knapsack to maximize the total value of items without exceeding the knapsack's weight limit, where each item is either fully included or not included. The fractional knapsack problem allows partial inclusion of items and can be solved greedily by always including a fraction of the highest value per unit weight item until the knapsack is full.5.2 mining time series data
5.2 mining time series dataKrish_ver2
油
This document discusses time-series data and methods for analyzing it. Time-series data consists of sequential values measured over time that can be analyzed to identify patterns, trends, and outliers. Key methods discussed include trend analysis to identify long-term movements, seasonal variations, and irregular components; similarity search to find similar sequences; and dimensionality reduction and transformation techniques to reduce data size before analysis or indexing.SINGLE SOURCE SHORTEST PATH.ppt
SINGLE SOURCE SHORTEST PATH.pptshanthishyam
油
This document discusses several algorithms for finding single-source shortest paths in graphs:
1) Bellman-Ford algorithm can handle graphs with negative edge weights by detecting negative cycles. It runs in O(VE) time.
2) For directed acyclic graphs (DAGs), topological sorting followed by relaxation yields the shortest paths from a single source in O(V+E) time.
3) Dijkstra's algorithm uses a greedy approach to find single-source shortest paths in graphs with non-negative edge weights.Midpoint circle algo
Midpoint circle algoMohd Arif
油
The midpoint circle algorithm is similar to Bresenham's circle algorithm and uses the midpoint between pixels to determine whether the pixel is inside or outside a circle. It defines a decision parameter pi based on the midpoint and updates pi by integer amounts at each step to determine the next pixel along the circle. The initial value of pi is set to 5/4 - r when r is an integer to determine the first pixel.K Nearest Neighbors
K Nearest NeighborsTilani Gunawardena PhD(UNIBAS), BSc(Pera), FHEA(UK), CEng, MIESL
油
The document discusses the K-nearest neighbors (KNN) algorithm, a simple machine learning algorithm used for classification problems. KNN works by finding the K training examples that are closest in distance to a new data point, and assigning the most common class among those K examples as the prediction for the new data point. The document covers how KNN calculates distances between data points, how to choose the K value, techniques for handling different data types, and the strengths and weaknesses of the KNN algorithm.K Means Clustering Algorithm | K Means Clustering Example | Machine Learning ...
K Means Clustering Algorithm | K Means Clustering Example | Machine Learning ...Simplilearn
油
This K-Means clustering algorithm presentation will take you through the machine learning introduction, types of clustering algorithms, k-means clustering, how does K-Means clustering work and at least explains K-Means clustering by taking a real life use case. This Machine Learning algorithm tutorial video is ideal for beginners to learn how K-Means clustering work.
Below topics are covered in this K-Means Clustering Algorithm presentation:
1. Types of Machine Learning?
2. What is K-Means Clustering?
3. Applications of K-Means Clustering
4. Common distance measure
5. How does K-Means Clustering work?
6. K-Means Clustering Algorithm
7. Demo: k-Means Clustering
8. Use case: Color compression
- - - - - - - -
About Simplilearn Machine Learning course:
A form of artificial intelligence, Machine Learning is revolutionizing the world of computing as well as all peoples digital interactions. Machine Learning powers such innovative automated technologies as recommendation engines, facial recognition, fraud protection and even self-driving cars.This Machine Learning course prepares engineers, data scientists and other professionals with knowledge and hands-on skills required for certification and job competency in Machine Learning.
- - - - - - -
Why learn Machine Learning?
Machine Learning is taking over the world- and with that, there is a growing need among companies for professionals to know the ins and outs of Machine Learning
The Machine Learning market size is expected to grow from USD 1.03 Billion in 2016 to USD 8.81 Billion by 2022, at a Compound Annual Growth Rate (CAGR) of 44.1% during the forecast period.
- - - - - -
What skills will you learn from this Machine Learning course?
By the end of this Machine Learning course, you will be able to:
1. Master the concepts of supervised, unsupervised and reinforcement learning concepts and modeling.
2. Gain practical mastery over principles, algorithms, and applications of Machine Learning through a hands-on approach which includes working on 28 projects and one capstone project.
3. Acquire thorough knowledge of the mathematical and heuristic aspects of Machine Learning.
4. Understand the concepts and operation of support vector machines, kernel SVM, naive bayes, decision tree classifier, random forest classifier, logistic regression, K-nearest neighbors, K-means clustering and more.
5. Be able to model a wide variety of robust Machine Learning algorithms including deep learning, clustering, and recommendation systems
- - - - - - -
K means clustering
K means clusteringkeshav goyal
油
K-means clustering is an algorithm that groups data points into k clusters based on their attributes and distances from initial cluster center points. It works by first randomly selecting k data points as initial centroids, then assigning all other points to the closest centroid and recalculating the centroids. This process repeats until the centroids are stable or a maximum number of iterations is reached. K-means clustering is widely used for machine learning applications like image segmentation and speech recognition due to its efficiency, but it is sensitive to initialization and assumes spherical clusters of similar size and density.Diabetic Retinopathy Detection
Diabetic Retinopathy DetectionSPb_Data_Science
油
This document discusses a competition to detect diabetic retinopathy from eye images. It provides details on the competition such as the goal of identifying signs of the disease, the number and size of training and test images. It then describes the solution including data preprocessing, network configuration using convolutional and max pooling layers, and optimization techniques. Special layers and losses were used to handle the ordinal rating scale. Microaneurysm detection is also discussed as the earliest clinical sign of the disease.
Ad
More Related Content
What's hot (20)
daa-unit-3-greedy method
daa-unit-3-greedy methodhodcsencet
油
The document discusses the greedy method algorithmic approach. It provides an overview of greedy algorithms including that they make locally optimal choices at each step to find a global optimal solution. The document also provides examples of problems that can be solved using greedy methods like job sequencing, the knapsack problem, finding minimum spanning trees, and single source shortest paths. It summarizes control flow and applications of greedy algorithms.convex hull
convex hullravikirankalal
油
The document discusses the convex hull algorithm. It begins by defining a convex hull as the shape a rubber band would take if stretched around pins on a board. It then provides explanations of extreme points, edges, and applications of convex hulls. Various algorithms for finding convex hulls are presented, including divide and conquer in O(n log n) time and Jarvis march in O(n^2) time in the worst case.Birch Algorithm With Solved Example
Birch Algorithm With Solved Examplekailash shaw
油
BIRCH油(balanced iterative reducing and clustering using hierarchies) is an unsupervised油data-mining油algorithm used to perform油hierarchical clustering油over, particularly large data sets.
Recurrent neural networks rnn
Recurrent neural networks rnnKuppusamy P
油
The document discusses recurrent neural networks (RNNs) and long short-term memory (LSTM) networks. It provides details on the architecture of RNNs including forward and back propagation. LSTMs are described as a type of RNN that can learn long-term dependencies using forget, input and output gates to control the cell state. Examples of applications for RNNs and LSTMs include language modeling, machine translation, speech recognition, and generating image descriptions.Conversion from infix to prefix using stack
Conversion from infix to prefix using stackHaqnawaz Ch
油
This document is a data structures assignment that contains two tasks:
1) Converting an infix expression to postfix notation using a stack. The example infix expression is converted to postfix as -+-AB*C+DE+FG.
2) Evaluating the postfix expression from the first task by assigning values to variables and using a stack to calculate the result, which is 33.Logical Agents
Logical AgentsYasir Khan
油
This document summarizes a chapter about logical agents from an artificial intelligence textbook. It discusses how logical agents can represent knowledge and reason using that knowledge. Key points include: logical agents use a knowledge base to store facts and make inferences; propositional logic and inference rules like modus ponens allow agents to derive new knowledge; the Wumpus World example demonstrates logical reasoning to solve problems based on partial observations.Big Data Visualization
Big Data VisualizationRaffael Marty
油
Raffael Marty gave a presentation on big data visualization. He discussed using visualization to discover patterns in large datasets and presenting security information on dashboards. Effective dashboards provide context, highlight important comparisons and metrics, and use aesthetically pleasing designs. Integration with security information management systems requires parsing and formatting data and providing interfaces for querying and analysis. Marty is working on tools for big data analytics, custom visualization workflows, and hunting for anomalies. He invited attendees to join an online community for discussing security visualization.Harris corner detector and face recognition
Harris corner detector and face recognitionShih Wei Huang
油
際際滷s taken from: Matching with Invariant Features, Darya
Frolova, Denis Simakov, The Weizmann Institute of Science,
March 2004Syntax Directed Definition and its applications
Syntax Directed Definition and its applicationsShivanandManjaragi2
油
A syntax-directed definition (SDD) is a context-free grammar with attributes and semantic rules. Attributes are associated with grammar symbols and rules are associated with productions. An SDD can be evaluated on a parse tree to compute attribute values at each node. There are two types of attributes: synthesized attributes depend on child nodes, while inherited attributes depend on parent or sibling nodes. The order of evaluation is determined by a dependency graph showing the flow of information between attribute instances.Data preprocessing using Machine Learning
Data preprocessing using Machine Learning Gopal Sakarkar
油
Data Preprocessing plays an important role in data cleaning .
Good data preparation is key to producing valid and reliable
Machine Learning models. Introduction to prolog
Introduction to prologHarry Potter
油
Prolog is a declarative logic programming language where programs consist of facts and rules. Facts are terms that are always true, while rules define relationships between terms using logic notation "if-then". A Prolog program is run by asking queries of the program's database. Variables must start with an uppercase letter and are used to represent unknown values, while atoms are constants that represent known values.Hadoop MapReduce Fundamentals
Hadoop MapReduce FundamentalsLynn Langit
油
This document provides an overview of the Hadoop MapReduce Fundamentals course. It discusses what Hadoop is, why it is used, common business problems it can address, and companies that use Hadoop. It also outlines the core parts of Hadoop distributions and the Hadoop ecosystem. Additionally, it covers common MapReduce concepts like HDFS, the MapReduce programming model, and Hadoop distributions. The document includes several code examples and screenshots related to Hadoop and MapReduce.Convex Hull Algorithms
Convex Hull AlgorithmsKasun Ranga Wijeweera
油
This document discusses convex hull algorithms. It defines a convex set as one where any line segment between two points in the set is also contained in the set. The convex hull of a set of points is the smallest convex set containing those points. Intuitively, in 2D the convex hull is the shape formed by stretching a rubber band around nails at each point, and in 3D it is the shape formed by stretching plastic wrap tightly around the points. The document then lists and describes several existing convex hull algorithms and provides an overview of an interior points algorithm that identifies non-extreme points based on whether they lie within triangles formed by other points.Fractional knapsack class 13
Fractional knapsack class 13Kumar
油
The document discusses two types of knapsack problems - the 0-1 knapsack problem and the fractional knapsack problem. The 0-1 knapsack problem uses dynamic programming to determine how to fill a knapsack to maximize the total value of items without exceeding the knapsack's weight limit, where each item is either fully included or not included. The fractional knapsack problem allows partial inclusion of items and can be solved greedily by always including a fraction of the highest value per unit weight item until the knapsack is full.5.2 mining time series data
5.2 mining time series dataKrish_ver2
油
This document discusses time-series data and methods for analyzing it. Time-series data consists of sequential values measured over time that can be analyzed to identify patterns, trends, and outliers. Key methods discussed include trend analysis to identify long-term movements, seasonal variations, and irregular components; similarity search to find similar sequences; and dimensionality reduction and transformation techniques to reduce data size before analysis or indexing.SINGLE SOURCE SHORTEST PATH.ppt
SINGLE SOURCE SHORTEST PATH.pptshanthishyam
油
This document discusses several algorithms for finding single-source shortest paths in graphs:
1) Bellman-Ford algorithm can handle graphs with negative edge weights by detecting negative cycles. It runs in O(VE) time.
2) For directed acyclic graphs (DAGs), topological sorting followed by relaxation yields the shortest paths from a single source in O(V+E) time.
3) Dijkstra's algorithm uses a greedy approach to find single-source shortest paths in graphs with non-negative edge weights.Midpoint circle algo
Midpoint circle algoMohd Arif
油
The midpoint circle algorithm is similar to Bresenham's circle algorithm and uses the midpoint between pixels to determine whether the pixel is inside or outside a circle. It defines a decision parameter pi based on the midpoint and updates pi by integer amounts at each step to determine the next pixel along the circle. The initial value of pi is set to 5/4 - r when r is an integer to determine the first pixel.K Nearest Neighbors
K Nearest NeighborsTilani Gunawardena PhD(UNIBAS), BSc(Pera), FHEA(UK), CEng, MIESL
油
The document discusses the K-nearest neighbors (KNN) algorithm, a simple machine learning algorithm used for classification problems. KNN works by finding the K training examples that are closest in distance to a new data point, and assigning the most common class among those K examples as the prediction for the new data point. The document covers how KNN calculates distances between data points, how to choose the K value, techniques for handling different data types, and the strengths and weaknesses of the KNN algorithm.K Means Clustering Algorithm | K Means Clustering Example | Machine Learning ...
K Means Clustering Algorithm | K Means Clustering Example | Machine Learning ...Simplilearn
油
This K-Means clustering algorithm presentation will take you through the machine learning introduction, types of clustering algorithms, k-means clustering, how does K-Means clustering work and at least explains K-Means clustering by taking a real life use case. This Machine Learning algorithm tutorial video is ideal for beginners to learn how K-Means clustering work.
Below topics are covered in this K-Means Clustering Algorithm presentation:
1. Types of Machine Learning?
2. What is K-Means Clustering?
3. Applications of K-Means Clustering
4. Common distance measure
5. How does K-Means Clustering work?
6. K-Means Clustering Algorithm
7. Demo: k-Means Clustering
8. Use case: Color compression
- - - - - - - -
About Simplilearn Machine Learning course:
A form of artificial intelligence, Machine Learning is revolutionizing the world of computing as well as all peoples digital interactions. Machine Learning powers such innovative automated technologies as recommendation engines, facial recognition, fraud protection and even self-driving cars.This Machine Learning course prepares engineers, data scientists and other professionals with knowledge and hands-on skills required for certification and job competency in Machine Learning.
- - - - - - -
Why learn Machine Learning?
Machine Learning is taking over the world- and with that, there is a growing need among companies for professionals to know the ins and outs of Machine Learning
The Machine Learning market size is expected to grow from USD 1.03 Billion in 2016 to USD 8.81 Billion by 2022, at a Compound Annual Growth Rate (CAGR) of 44.1% during the forecast period.
- - - - - -
What skills will you learn from this Machine Learning course?
By the end of this Machine Learning course, you will be able to:
1. Master the concepts of supervised, unsupervised and reinforcement learning concepts and modeling.
2. Gain practical mastery over principles, algorithms, and applications of Machine Learning through a hands-on approach which includes working on 28 projects and one capstone project.
3. Acquire thorough knowledge of the mathematical and heuristic aspects of Machine Learning.
4. Understand the concepts and operation of support vector machines, kernel SVM, naive bayes, decision tree classifier, random forest classifier, logistic regression, K-nearest neighbors, K-means clustering and more.
5. Be able to model a wide variety of robust Machine Learning algorithms including deep learning, clustering, and recommendation systems
- - - - - - -
K means clustering
K means clusteringkeshav goyal
油
K-means clustering is an algorithm that groups data points into k clusters based on their attributes and distances from initial cluster center points. It works by first randomly selecting k data points as initial centroids, then assigning all other points to the closest centroid and recalculating the centroids. This process repeats until the centroids are stable or a maximum number of iterations is reached. K-means clustering is widely used for machine learning applications like image segmentation and speech recognition due to its efficiency, but it is sensitive to initialization and assumes spherical clusters of similar size and density.More from SPb_Data_Science (11)
Diabetic Retinopathy Detection
Diabetic Retinopathy DetectionSPb_Data_Science
油
This document discusses a competition to detect diabetic retinopathy from eye images. It provides details on the competition such as the goal of identifying signs of the disease, the number and size of training and test images. It then describes the solution including data preprocessing, network configuration using convolutional and max pooling layers, and optimization techniques. Special layers and losses were used to handle the ordinal rating scale. Microaneurysm detection is also discussed as the earliest clinical sign of the disease.Meetup#4, Apache Spark as SQL Engine 
Meetup#4, Apache Spark as SQL Engine SPb_Data_Science
油
This document discusses Apache Spark and Spark SQL. It provides an overview of Apache Spark, noting that it can run programs faster than Hadoop in memory or on disk and supports SQL, streaming, and complex analytics. It then discusses Spark SQL, Spark's module for working with structured data. Spark SQL allows users to mix SQL queries with Spark programs and connect to data sources like Hive, Avro and Parquet. It also supports a server mode that connects to Spark SQL via JDBC. The document promotes Spark SQL as a distributed SQL engine that can integrate with BI tools.Trending Topics in Recommender Systems
Trending Topics in Recommender SystemsSPb_Data_Science
油
This document summarizes RecSys 2015, a conference on recommender systems. It discusses trends in academia versus industry, including that academia focuses on complex models and small datasets while industry uses simpler models like SVD on large "Big Data" with metrics like click-through rate and retention. It also summarizes several papers presented on topics like using social networks, cross-domain recommendations, and addressing the cold start problem. Industry sessions were held by companies like LinkedIn, Netflix, and Amazon discussing their recommendation architectures and focus on user experience over complex models.Meetup#4, Smart.Data@OK.ru
Meetup#4, Smart.Data@OK.ruSPb_Data_Science
油
The document discusses OK.ru's use of smart data and big data analytics. It details how OK.ru analyzes over 1 petabyte of user data across music, communities, video, news feed, and other areas to improve user experience and engagement. OK.ru employs techniques like collaborative filtering, temporal analysis, and deep learning models to perform personalized recommendations, spam detection, and business intelligence. The document also describes OK.ru's open data initiatives and invites readers to access datasets and participate in challenges to help further their smart data work.
Benfords law & fraud detection slides
Benfords law & fraud detection slidesSPb_Data_Science
油
The document summarizes a meetup about Benford's law and its applications in detecting fraud. Benford's law states that in many real-life datasets, the digits 1 through 9 occur with different frequencies as first significant digits, with 1 appearing more often than 2. The meetup discussed how analyzing the frequency of first significant digits can help detect accounting fraud, election irregularities, scientific misconduct, and fake social media accounts. Examples were given showing how Benford's law identified over 90% of Russian Twitter bots by their deviation from the expected digit distribution.Meetup#2. Intro to Factorization Machines
Meetup#2. Intro to Factorization MachinesSPb_Data_Science
油
This document summarizes Factorization Machines, which were proposed in a 2010 paper by Steffen Rendle. Factorization Machines address the independence of interaction parameters in linear models and polynomial feature-based models by "breaking" this independence using factorization. They model pairwise interactions between variables via factor vectors, learning a factor vector for each variable. This reduces the number of parameters compared to linear models and allows modeling of interactions without explicit interaction terms. Factorization Machines have been shown to work well for sparse, high-dimensional predictive analytics problems. Recent extensions include Field-Aware Factorization Machines, which incorporate field information to better model features.Meetup#2. Introduction to Algorithmic Trading
Meetup#2. Introduction to Algorithmic TradingSPb_Data_Science
油
Talk by Evgeniy Zhurin (RuTarget). Meetup#2.Meetup #1. Trends, talks, cool stuff.
Meetup #1. Trends, talks, cool stuff.SPb_Data_Science
油
This document discusses trends in data science and outlines topics for further discussion. It notes that rich data sources require complex representations and data science needs stronger connections to business needs. Additionally, networking was described as "magic". The document suggests collecting more data from all available sources and encourages working on personally interesting projects. It promotes an open approach to data science and provides contact information for the organization DM Labs.Meetup #1. Building a CNN in Kaggle Data Science Bowl
Meetup #1. Building a CNN in Kaggle Data Science BowlSPb_Data_Science
油
Marat and Alexey sharing their experience building a convolutional neural network in https://www.kaggle.com/c/datasciencebowl.Ad
Classification metrics
- 1. EVALUATION METRICS FOR CLICK PREDICTION Evgeniy Zhurin, RuTarget
- 2. Binary Classification Error Measurement 1) AUC 2) RIG 3) LogLoss 4) Precision/Recall 5) F1 6) PE, MSE, MAE
- 8. AUC 1) ignores the predicted probability values 2) usually we are interested in parts of roc curve 3) considers Type 1 error and Type 2 error weights equivalently 4) dependent on the underlying distribution of data
- 12. RIG 1) bad to compare two model performances with different distributions 2) can be used to compare the relative performance of multiple models trained and tested on the same data 3) is not informative, because score also depends on the data distribution
- 14. OR WRITE A SIMULATOR
- 15. Thanks! J. Yi, Y. Chen, J. Li, S. Sett, and T. W. Yan. Predictive model performance: Offline and online evaluations. In KDD, pages 12941302, 2013. http://chbrown.github.io/kdd-2013- usb/kdd/p1294.pdf