ºÝºÝߣ

ºÝºÝߣShare a Scribd company logo
Lasso and Bayesian Lasso
Liangqun Lu @Bayesian Inference 2017 Spring
Biology Department
University of Memphis
Outline
? Lasso
? Bayesian Lasso Regression
? Application Example
LASSO
? Least Absolute Shrinkage and Selection Operator
? Usage: Variable Selection
¡ð Variables are greater than observations (p > n)
¡ð Shrinkage = loss function + penalty function
Lasso (Tibshirani, 1996)
Courtesy http://www.stat.wisc.edu/~shao/stat992/pres1-1.pdf
The loss function || y - X¦Â||2
= (¦Â ? ¦Â) T X T X(¦Â ? ¦Â) + constant
Like when p = 2
|¦Â1
| + |¦Â2
| ¡Ü t
Some ¦Â are set to 0
Lasso solution is the touching place
?? ( 2 )
Bayesian Lasso Regression (CHRIS HAN, 2009)
Laplacian Priors Laplace(0, ¦Ë)
Posterior
(orthant-wise Gaussian)
The orthogonalized Gibbs sampler
Diabetes example
Training:
192 X 8
Testing:
250 X 8
? Compare the error between non- bayes and bayes
? T = 1000 times
? orthogonalized Gibbs sampler
? Calculate the posterior probability
(The sampling results can be seen from R monomvn package)
Coefficients
Lambda
Variance
Prostate example
(97 X 9)
Outcome: cancer volume
Variables: prostate weight; age; benign prostatic hyperplasia amount; seminal vesicle
invasion; capsular penetration; Gleason score; percentage Gleason scores 4 or 5; prostate
specific antigen
Coefficients from Lasso regularization
Optimization from Cross Validation
¦Ë = 0.01899013
Log¦Ë = -3.963836
> coef(cvfit, s = "lambda.min")
(Intercept) -1.642167273
lweight -0.009419681
age 0.016850891
lbph -0.066278770
svi .
lcp 0.317658663
gleason 0.116595770
pgg45 -0.003493425
lpsa 0.529322761

More Related Content

What's hot (20)

Deep Dive into Hyperparameter Tuning
Deep Dive into Hyperparameter TuningDeep Dive into Hyperparameter Tuning
Deep Dive into Hyperparameter Tuning
Shubhmay Potdar
?
Naive Bayes
Naive BayesNaive Bayes
Naive Bayes
Abdullah al Mamun
?
Uninformed search
Uninformed searchUninformed search
Uninformed search
Bablu Shofi
?
Time complexity
Time complexityTime complexity
Time complexity
Katang Isip
?
NAIVE BAYES CLASSIFIER
NAIVE BAYES CLASSIFIERNAIVE BAYES CLASSIFIER
NAIVE BAYES CLASSIFIER
Knoldus Inc.
?
An overview of gradient descent optimization algorithms
An overview of gradient descent optimization algorithms An overview of gradient descent optimization algorithms
An overview of gradient descent optimization algorithms
Hakky St
?
Decision Tree - C4.5&CART
Decision Tree - C4.5&CARTDecision Tree - C4.5&CART
Decision Tree - C4.5&CART
Xueping Peng
?
Artificial Neural Networks (ANNs) - XOR - Step-By-Step
Artificial Neural Networks (ANNs) - XOR - Step-By-StepArtificial Neural Networks (ANNs) - XOR - Step-By-Step
Artificial Neural Networks (ANNs) - XOR - Step-By-Step
Ahmed Gad
?
Recurrent Neural Network
Recurrent Neural NetworkRecurrent Neural Network
Recurrent Neural Network
Mohammad Sabouri
?
Decision trees & random forests
Decision trees & random forestsDecision trees & random forests
Decision trees & random forests
SC5.io
?
TabuSearch FINAL
TabuSearch  FINALTabuSearch  FINAL
TabuSearch FINAL
Mustafa RASHID
?
Mask R-CNN
Mask R-CNNMask R-CNN
Mask R-CNN
Jaehyun Jun
?
Deep Learning for Time Series Data
Deep Learning for Time Series DataDeep Learning for Time Series Data
Deep Learning for Time Series Data
Arun Kejariwal
?
Feature selection concepts and methods
Feature selection concepts and methodsFeature selection concepts and methods
Feature selection concepts and methods
Reza Ramezani
?
A star algorithms
A star algorithmsA star algorithms
A star algorithms
sandeep54552
?
Knowledge Engineering in FOL.
Knowledge Engineering in FOL.Knowledge Engineering in FOL.
Knowledge Engineering in FOL.
Megha Sharma
?
algorithm Unit 3
algorithm Unit 3algorithm Unit 3
algorithm Unit 3
Monika Choudhery
?
Time series data mining techniques
Time series data mining techniquesTime series data mining techniques
Time series data mining techniques
Shanmukha S. Potti
?
07 approximate inference in bn
07 approximate inference in bn07 approximate inference in bn
07 approximate inference in bn
IEEE International Conference on Intelligent Information Hiding and Multimedia Signal Processing
?
Introduction to capsule networks
Introduction to capsule networksIntroduction to capsule networks
Introduction to capsule networks
Andrii Babii
?

More from Liangqun Lu (13)

NFL_intros.pptx
NFL_intros.pptxNFL_intros.pptx
NFL_intros.pptx
Liangqun Lu
?
BERT: Bidirectional Encoder Representations from Transformers
BERT: Bidirectional Encoder Representations from TransformersBERT: Bidirectional Encoder Representations from Transformers
BERT: Bidirectional Encoder Representations from Transformers
Liangqun Lu
?
Gan summary
Gan summaryGan summary
Gan summary
Liangqun Lu
?
Data integration lab_meeting
Data integration lab_meetingData integration lab_meeting
Data integration lab_meeting
Liangqun Lu
?
NLP DLforDS
NLP DLforDSNLP DLforDS
NLP DLforDS
Liangqun Lu
?
Irgan
IrganIrgan
Irgan
Liangqun Lu
?
Deep Learning Application in Biology
Deep Learning Application in BiologyDeep Learning Application in Biology
Deep Learning Application in Biology
Liangqun Lu
?
Liangqun ms defense.pptx
Liangqun ms defense.pptxLiangqun ms defense.pptx
Liangqun ms defense.pptx
Liangqun Lu
?
Thesis ms llq
Thesis ms llqThesis ms llq
Thesis ms llq
Liangqun Lu
?
Liangqun lu 1st_gss_version2
Liangqun lu 1st_gss_version2Liangqun lu 1st_gss_version2
Liangqun lu 1st_gss_version2
Liangqun Lu
?
Presentation orientation
Presentation orientationPresentation orientation
Presentation orientation
Liangqun Lu
?
Journal club.pptx
Journal club.pptxJournal club.pptx
Journal club.pptx
Liangqun Lu
?
Final.project
Final.projectFinal.project
Final.project
Liangqun Lu
?
BERT: Bidirectional Encoder Representations from Transformers
BERT: Bidirectional Encoder Representations from TransformersBERT: Bidirectional Encoder Representations from Transformers
BERT: Bidirectional Encoder Representations from Transformers
Liangqun Lu
?
Data integration lab_meeting
Data integration lab_meetingData integration lab_meeting
Data integration lab_meeting
Liangqun Lu
?
Deep Learning Application in Biology
Deep Learning Application in BiologyDeep Learning Application in Biology
Deep Learning Application in Biology
Liangqun Lu
?
Liangqun ms defense.pptx
Liangqun ms defense.pptxLiangqun ms defense.pptx
Liangqun ms defense.pptx
Liangqun Lu
?
Liangqun lu 1st_gss_version2
Liangqun lu 1st_gss_version2Liangqun lu 1st_gss_version2
Liangqun lu 1st_gss_version2
Liangqun Lu
?
Presentation orientation
Presentation orientationPresentation orientation
Presentation orientation
Liangqun Lu
?

Recently uploaded (20)

entomology notes ppt 2 insect-dominance.pptx
entomology notes  ppt 2 insect-dominance.pptxentomology notes  ppt 2 insect-dominance.pptx
entomology notes ppt 2 insect-dominance.pptx
ANKUSHBISEN3
?
Scientific Pig Farming Manual for Pig Farmers
Scientific Pig Farming Manual for Pig FarmersScientific Pig Farming Manual for Pig Farmers
Scientific Pig Farming Manual for Pig Farmers
Dr. Subhrajit Das
?
Excretion in Humans | Cambridge IGCSE Biology
Excretion in Humans | Cambridge IGCSE BiologyExcretion in Humans | Cambridge IGCSE Biology
Excretion in Humans | Cambridge IGCSE Biology
Blessing Ndazie
?
2025-03-03-Data-related-Ethics Issues in Technologies for Professional Learni...
2025-03-03-Data-related-Ethics Issues in Technologies for Professional Learni...2025-03-03-Data-related-Ethics Issues in Technologies for Professional Learni...
2025-03-03-Data-related-Ethics Issues in Technologies for Professional Learni...
Graz University of Technology & Know-Center
?
AUTOSOMES , ALLOSOMES AND SEX RATIO IN HUMAN POPULATION
AUTOSOMES , ALLOSOMES AND SEX RATIO IN HUMAN POPULATIONAUTOSOMES , ALLOSOMES AND SEX RATIO IN HUMAN POPULATION
AUTOSOMES , ALLOSOMES AND SEX RATIO IN HUMAN POPULATION
Nistarini College, Purulia (W.B) India
?
Coordination and Response: The Nervous System | IGCSE Biology
Coordination and Response: The Nervous System | IGCSE BiologyCoordination and Response: The Nervous System | IGCSE Biology
Coordination and Response: The Nervous System | IGCSE Biology
Blessing Ndazie
?
QUANTITATIVE GENETICS PART 2.pdf agriculture
QUANTITATIVE GENETICS PART 2.pdf agricultureQUANTITATIVE GENETICS PART 2.pdf agriculture
QUANTITATIVE GENETICS PART 2.pdf agriculture
KushiBhatia
?
Simple Phenomena of Magnetism | IGCSE Physics
Simple Phenomena of Magnetism | IGCSE PhysicsSimple Phenomena of Magnetism | IGCSE Physics
Simple Phenomena of Magnetism | IGCSE Physics
Blessing Ndazie
?
Leafcurl viral disease presentation.pptx
Leafcurl viral disease presentation.pptxLeafcurl viral disease presentation.pptx
Leafcurl viral disease presentation.pptx
Mir Ali M
?
Detection of ferrihydrite in Martian red dust records ancient cold and wet co...
Detection of ferrihydrite in Martian red dust records ancient cold and wet co...Detection of ferrihydrite in Martian red dust records ancient cold and wet co...
Detection of ferrihydrite in Martian red dust records ancient cold and wet co...
S¨¦rgio Sacani
?
Neumann, Franz. - Behemoth [ocr] [1942].pdf
Neumann, Franz. - Behemoth [ocr] [1942].pdfNeumann, Franz. - Behemoth [ocr] [1942].pdf
Neumann, Franz. - Behemoth [ocr] [1942].pdf
Francisco Sandoval Mart¨ªnez
?
Improving the Perturbation-Based Explanation of Deepfake Detectors Through th...
Improving the Perturbation-Based Explanation of Deepfake Detectors Through th...Improving the Perturbation-Based Explanation of Deepfake Detectors Through th...
Improving the Perturbation-Based Explanation of Deepfake Detectors Through th...
VasileiosMezaris
?
Preparing Ultrasound Imaging Data for Artificial Intelligence Tasks: Anonymis...
Preparing Ultrasound Imaging Data for Artificial Intelligence Tasks: Anonymis...Preparing Ultrasound Imaging Data for Artificial Intelligence Tasks: Anonymis...
Preparing Ultrasound Imaging Data for Artificial Intelligence Tasks: Anonymis...
ThrombUS+ Project
?
(Chapter Summary) Molecular Biology of the Cell, Chapter 18: Cell Death
(Chapter Summary) Molecular Biology of the Cell, Chapter 18: Cell Death(Chapter Summary) Molecular Biology of the Cell, Chapter 18: Cell Death
(Chapter Summary) Molecular Biology of the Cell, Chapter 18: Cell Death
David Podorefsky, PhD
?
(Journal Club) Focused ultrasound excites neurons via mechanosensitive calciu...
(Journal Club) Focused ultrasound excites neurons via mechanosensitive calciu...(Journal Club) Focused ultrasound excites neurons via mechanosensitive calciu...
(Journal Club) Focused ultrasound excites neurons via mechanosensitive calciu...
David Podorefsky, PhD
?
Parasitology Practical Book .pdf Biomedical science
Parasitology Practical Book .pdf Biomedical scienceParasitology Practical Book .pdf Biomedical science
Parasitology Practical Book .pdf Biomedical science
saihetharan
?
Hormones and the Endocrine System | IGCSE Biology
Hormones and the Endocrine System | IGCSE BiologyHormones and the Endocrine System | IGCSE Biology
Hormones and the Endocrine System | IGCSE Biology
Blessing Ndazie
?
(Journal Club) Folding DNA to create nanoscale shapes and patterns
(Journal Club) Folding DNA to create nanoscale shapes and patterns(Journal Club) Folding DNA to create nanoscale shapes and patterns
(Journal Club) Folding DNA to create nanoscale shapes and patterns
David Podorefsky, PhD
?
B-FPGM: Lightweight Face Detection via Bayesian-Optimized Soft FPGM Pruning
B-FPGM: Lightweight Face Detection via Bayesian-Optimized Soft FPGM PruningB-FPGM: Lightweight Face Detection via Bayesian-Optimized Soft FPGM Pruning
B-FPGM: Lightweight Face Detection via Bayesian-Optimized Soft FPGM Pruning
VasileiosMezaris
?
Drug evaluation¨C Organoleptic, Microscopic, Chemical, Biological
Drug evaluation¨C Organoleptic, Microscopic, Chemical, BiologicalDrug evaluation¨C Organoleptic, Microscopic, Chemical, Biological
Drug evaluation¨C Organoleptic, Microscopic, Chemical, Biological
Nistarini College, Purulia (W.B) India
?
entomology notes ppt 2 insect-dominance.pptx
entomology notes  ppt 2 insect-dominance.pptxentomology notes  ppt 2 insect-dominance.pptx
entomology notes ppt 2 insect-dominance.pptx
ANKUSHBISEN3
?
Scientific Pig Farming Manual for Pig Farmers
Scientific Pig Farming Manual for Pig FarmersScientific Pig Farming Manual for Pig Farmers
Scientific Pig Farming Manual for Pig Farmers
Dr. Subhrajit Das
?
Excretion in Humans | Cambridge IGCSE Biology
Excretion in Humans | Cambridge IGCSE BiologyExcretion in Humans | Cambridge IGCSE Biology
Excretion in Humans | Cambridge IGCSE Biology
Blessing Ndazie
?
Coordination and Response: The Nervous System | IGCSE Biology
Coordination and Response: The Nervous System | IGCSE BiologyCoordination and Response: The Nervous System | IGCSE Biology
Coordination and Response: The Nervous System | IGCSE Biology
Blessing Ndazie
?
QUANTITATIVE GENETICS PART 2.pdf agriculture
QUANTITATIVE GENETICS PART 2.pdf agricultureQUANTITATIVE GENETICS PART 2.pdf agriculture
QUANTITATIVE GENETICS PART 2.pdf agriculture
KushiBhatia
?
Simple Phenomena of Magnetism | IGCSE Physics
Simple Phenomena of Magnetism | IGCSE PhysicsSimple Phenomena of Magnetism | IGCSE Physics
Simple Phenomena of Magnetism | IGCSE Physics
Blessing Ndazie
?
Leafcurl viral disease presentation.pptx
Leafcurl viral disease presentation.pptxLeafcurl viral disease presentation.pptx
Leafcurl viral disease presentation.pptx
Mir Ali M
?
Detection of ferrihydrite in Martian red dust records ancient cold and wet co...
Detection of ferrihydrite in Martian red dust records ancient cold and wet co...Detection of ferrihydrite in Martian red dust records ancient cold and wet co...
Detection of ferrihydrite in Martian red dust records ancient cold and wet co...
S¨¦rgio Sacani
?
Improving the Perturbation-Based Explanation of Deepfake Detectors Through th...
Improving the Perturbation-Based Explanation of Deepfake Detectors Through th...Improving the Perturbation-Based Explanation of Deepfake Detectors Through th...
Improving the Perturbation-Based Explanation of Deepfake Detectors Through th...
VasileiosMezaris
?
Preparing Ultrasound Imaging Data for Artificial Intelligence Tasks: Anonymis...
Preparing Ultrasound Imaging Data for Artificial Intelligence Tasks: Anonymis...Preparing Ultrasound Imaging Data for Artificial Intelligence Tasks: Anonymis...
Preparing Ultrasound Imaging Data for Artificial Intelligence Tasks: Anonymis...
ThrombUS+ Project
?
(Chapter Summary) Molecular Biology of the Cell, Chapter 18: Cell Death
(Chapter Summary) Molecular Biology of the Cell, Chapter 18: Cell Death(Chapter Summary) Molecular Biology of the Cell, Chapter 18: Cell Death
(Chapter Summary) Molecular Biology of the Cell, Chapter 18: Cell Death
David Podorefsky, PhD
?
(Journal Club) Focused ultrasound excites neurons via mechanosensitive calciu...
(Journal Club) Focused ultrasound excites neurons via mechanosensitive calciu...(Journal Club) Focused ultrasound excites neurons via mechanosensitive calciu...
(Journal Club) Focused ultrasound excites neurons via mechanosensitive calciu...
David Podorefsky, PhD
?
Parasitology Practical Book .pdf Biomedical science
Parasitology Practical Book .pdf Biomedical scienceParasitology Practical Book .pdf Biomedical science
Parasitology Practical Book .pdf Biomedical science
saihetharan
?
Hormones and the Endocrine System | IGCSE Biology
Hormones and the Endocrine System | IGCSE BiologyHormones and the Endocrine System | IGCSE Biology
Hormones and the Endocrine System | IGCSE Biology
Blessing Ndazie
?
(Journal Club) Folding DNA to create nanoscale shapes and patterns
(Journal Club) Folding DNA to create nanoscale shapes and patterns(Journal Club) Folding DNA to create nanoscale shapes and patterns
(Journal Club) Folding DNA to create nanoscale shapes and patterns
David Podorefsky, PhD
?
B-FPGM: Lightweight Face Detection via Bayesian-Optimized Soft FPGM Pruning
B-FPGM: Lightweight Face Detection via Bayesian-Optimized Soft FPGM PruningB-FPGM: Lightweight Face Detection via Bayesian-Optimized Soft FPGM Pruning
B-FPGM: Lightweight Face Detection via Bayesian-Optimized Soft FPGM Pruning
VasileiosMezaris
?

Lasso