際際滷

際際滷Share a Scribd company logo

Using Market Segmentation to Track Program Success_ADwelley

Opinion Dynamics
Opinion Dynamics

This document discusses using market segmentation to track the success of energy efficiency programs. It argues that segmenting utility customer populations can provide more meaningful insights for program design, evaluation, and planning compared to treating all customers equally. While program implementers and evaluators already analyze some results by customer groups, this is often not done in a consistent way. The document advocates integrating segmentation more cohesively into evaluation to deliver insights that can help programs improve and determine whether programs should be tailored to different customer segments. It provides examples of meaningful segmentation approaches like using demographic, energy usage, and participation data to define segments that may have different opportunities, barriers, and preferences.

Convert to study materialsBETA

Transform any presentation into ready-made study materialselect from outputs like summaries, definitions, and practice questions.

1 of 17
Downloaded 11 times
Using Market Segmentation to Track Program Success Amanda Dwelley AESP EM&V Online Conference December 4, 2013
About Opinion Dynamics Established in 1987 Leader in market research for utilities Offices in Massachusetts, California & Wisconsin Energy Efficiency Evaluation Energy Advising Smart Grid, DR, and Behavior Market Research Custom approach We work with utilities and implementers to use all available data to develop tailored solutions AESP EM&V Online Conference 2
Key Points There are many ways to segment utility customer populations Some are more meaningful than others for program design, portfolio planning and/or EM&V Implementers are already using segmentation to improve program targeting (and uptake) The EM&V community (us!) does analyze results by customer group/segment But often not in a cohesive or consistent way Consistently integrating segmentation in to EM&V will: Deliver insights that help programs improve faster Get stakeholders thinking about (a) how results can be used/extrapolated, and (b) if/how programs should be tailored/targeted to different segments AESP EM&V Online Conference 3
Program implementers use segmentation all the time Segmentation defines and divides a large population into identifiable groups based on similar characteristics Summer kWh 25% 20% 15% 10% 5% 0% High summer usage targeted for HVAC rebate High annual usage targeted for behavioral programs ExperianMosaicSegment Multi-family middle-income targeted for audits / weatherization 1 AESP EM&V Online Conference Urbanites targeted for HEMS / IHD
Historical approach of equal access to programs, and undifferentiated marketing, hasnt yielded equal impacts For this utility, theres a strong relationship between wealth quintile (measured three ways) and long-term EE program participation: 10% 8% 6% 4% 2% 0% 1 2 3 4 Income Quintile 5 12% Cumulative EE Participation vs. Assessed Home Value (among the 50% of customers with assessor data) EE Participation Rate 12% Cumulative EE Participation vs. Pct of Neighborhood with Income >$75k (from secondary data) EE Participation Rate EE Participation Rate Cumulative EE Participation vs. Per Capita Income as % Poverty Line (modeled value) 10% 8% 6% 4% 2% 0% 1 2 3 4 Income Quintile 5 16% 14% 12% 10% 8% 6% 4% 2% 0% 1 2 3 4 5 Home Value Quintile What were the drivers of these differences? Targeted marketing? Awareness/knowledge? Qualification criteria? Interest? AESP EM&V Online Conference 5
Were leaving opportunity on the table, but dont know where or how much Our customers are unique So we cant reach statewide goals Three-Year Plan vs. Statewide Goals 3.5% 3.0% 2.5% 2.50% 2.55% 2.60% PY 2013 PY 2014 Segmented program evaluation and opportunity studies can uncover how/why: Moderate income status? House type (SF/MF)? Seasonal/vacation homes? Channel preferences vs. implementation channels? Baseline efficiencies already high? PY2015 2.0% 1.5% 1.0% 0.5% 0.0% AESP EM&V Online Conference 6
Evaluators do report on differences by customer group, but sometimes we only look within a program Annual Percent Savings 2.5% Annual Percent Savings by Consumption Tertile 2.0% 1.5% 1.6% 1.8% 1.2% 1.0% 0.5% 0.0% Low Medium High Consumption Consumption Consumption Top 2030% Top 1020% Misleading to report, because the program targeted high users! Difficult for planners/evaluators to understand how to use findings Top 10% Make sure segment membership we report is relative to the customer population; use the same data source AESP EM&V Online Conference 7
Segment-level insights are useful across the program lifecycle Metrics Measurement Opportunities Awareness / Knowledge General population and non-part surveys Intention Inquiries, leads, incomplete applications that link to customer database by account # Qualification Ex ante: Filter database by qualifying criteria Ex post: Program qualification rates Participation Program participation rates Portfolio-level participation: What % of all segment members have participated in any EE? Engagement Online / HEMS / IHD device tracking Participant surveys Impacts Realization rates by segment Savings depth by segment (% savings) Measure mix by segment AESP EM&V Online Conference 8
Using Market Segmentation to Track Program Success_ADwelley
So, what segmentation is good for EM&V purposes? 1 1. Segment membership must be identifiable ex ante for all customers Rate code (Low income, SF/MF, Small/large commercial) Psychographic lifestyle segment available through data providers (e.g., Experian) Usage characteristics (L/M/H; summer load; load shape) 2 1. Segments should distinguish between meaningful differences that affect program outcomes Energy opportunity Barriers to participation (own/rent; income) Motivation to participate Channel/communication preferences (on-bill, web, phone) Impacts! 3 1. Segments should be consumable by readers/regulators: Easy to understand / well-named Manageable number AESPEM&VOnlineConference 10
1 Identification: Tracking by segment requires defining segments based on readily-available data And we have a lot! Secondary demographic/ housing data e.g., age, income, home value Past program participation DSM and non-DSM TOU Account Rate A B C Energy Audit Ref. Rebate Customer characteristics from CIS data e.g., rate class, time-as-customer New Customer engagement e.g., online activity, payment preferences Energy indicators e.g., seasonal usage, load shape 1-4 yrs 5-9 yrs 10-19 20+ yrs yrs 0 2 4 6 8 10 12 14 16 18 20 22 AESP EM&V Online Conference 11
2 Meaningful Differences: Segment membership should correlate with savings opportunities, program propensity, barriers and preferences Demographically-Based Lifestyle Segmentation Custom Psychographi Segmentation Energy Usage Patterns Past Participation Highest Medium Dim. 1 Lowest Dim. 2 May correlate well with: Ability to participate Channel/ marketing affinity Heterogeneous in terms of: Savings opportunities May correlate well with: Ability to participate Motivation Heterogeneous in terms of: Savings opportunities Channel/ marketing affinity 0 2 4 6 8 10 12 14 16 18 20 22 May correlate well with: Savings opportunities Heterogeneous in terms of: Ability to participate Channel/ marketing affinity
Have AMI data? Clustering customers into Load Shape Segments could enable long-term impact tracking Best target for DR and conservation programs? clustersimilar patterns Relatively high baseload - many EE/Wx opportunities WholeHouseLoadShapes 4000 3500 HighPeak/LowBaseload 3500 3000 3000 2500 2500 ExtendedPeak HighBaseload LowUsers Non-HVAC EE and behavioral interventions 2000 2000 1500 1500 1000 1000 500 500 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 0 0 1 2 3 4 5 6 7 8 Low-cost conservation and 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23behavior Identify highest-impact equipment, envelope and behavioral opportunities for each segment AESP EM&V Online Conference 13
3 Consumable segments: Easy to explain and interpret; manageable number Single dimensions (single-family / multi-family) or 2X2 matrices have merit But they leave a lot of heterogeneity undescribed Complex segmentation schemes quickly go un-used Reviewers dont have background/knowledge of approach Imagine 70 Experian lifestyle segments! Cost implications to what we choose Segment quotas AESP EM&V Online Conference 14
We can start by reporting savings at a segment level Segment Percent of Customers Percent of Wx Participants Wx Savings per Household (kWh) Wx Savings Total (MWh) A 25% 28% 180 81.0 B 15% 14% 150 33.8 C 40% 34% 100 56.0 D 20% 24% 80 32 Total 100% 100% 124 202.8 AESP EM&V Online Conference 15
End game: Identify and track program opportunities and success metrics specific to each segment Participationrate amongencouraged Savingsdepthor realizationrate n Targeted for Wx Wx Uptake (among those targeted) Wx Savings per Household (kWh) Wx Opportunity per Household (kWh) % of Opportunity Achieved 28% 5,000 9% 180 200 90% 15% 14% 3,000 7.5% 150 300 50% C 40% 34% 8,000 7% 100 150 75% D 20% 24% 4,000 10% 80 100 80% Total 100% 100% 20,000 8.2% 124 172 72% Segment Percent of Customers Percent of Wx Participants A 25% B AESP EM&V Online Conference 16
Thank You! Amanda Dwelley Associate Director 617-301-4629 adwelley@opiniondynamics.com Visit us at www.opiniondynamics.com AESP EM&V Online Conference 17

More Related Content

Using Market Segmentation to Track Program Success_ADwelley

  • 1. Using Market Segmentation to Track Program Success Amanda Dwelley AESP EM&V Online Conference December 4, 2013
  • 2. About Opinion Dynamics Established in 1987 Leader in market research for utilities Offices in Massachusetts, California & Wisconsin Energy Efficiency Evaluation Energy Advising Smart Grid, DR, and Behavior Market Research Custom approach We work with utilities and implementers to use all available data to develop tailored solutions AESP EM&V Online Conference 2
  • 3. Key Points There are many ways to segment utility customer populations Some are more meaningful than others for program design, portfolio planning and/or EM&V Implementers are already using segmentation to improve program targeting (and uptake) The EM&V community (us!) does analyze results by customer group/segment But often not in a cohesive or consistent way Consistently integrating segmentation in to EM&V will: Deliver insights that help programs improve faster Get stakeholders thinking about (a) how results can be used/extrapolated, and (b) if/how programs should be tailored/targeted to different segments AESP EM&V Online Conference 3
  • 4. Program implementers use segmentation all the time Segmentation defines and divides a large population into identifiable groups based on similar characteristics Summer kWh 25% 20% 15% 10% 5% 0% High summer usage targeted for HVAC rebate High annual usage targeted for behavioral programs ExperianMosaicSegment Multi-family middle-income targeted for audits / weatherization 1 AESP EM&V Online Conference Urbanites targeted for HEMS / IHD
  • 5. Historical approach of equal access to programs, and undifferentiated marketing, hasnt yielded equal impacts For this utility, theres a strong relationship between wealth quintile (measured three ways) and long-term EE program participation: 10% 8% 6% 4% 2% 0% 1 2 3 4 Income Quintile 5 12% Cumulative EE Participation vs. Assessed Home Value (among the 50% of customers with assessor data) EE Participation Rate 12% Cumulative EE Participation vs. Pct of Neighborhood with Income >$75k (from secondary data) EE Participation Rate EE Participation Rate Cumulative EE Participation vs. Per Capita Income as % Poverty Line (modeled value) 10% 8% 6% 4% 2% 0% 1 2 3 4 Income Quintile 5 16% 14% 12% 10% 8% 6% 4% 2% 0% 1 2 3 4 5 Home Value Quintile What were the drivers of these differences? Targeted marketing? Awareness/knowledge? Qualification criteria? Interest? AESP EM&V Online Conference 5
  • 6. Were leaving opportunity on the table, but dont know where or how much Our customers are unique So we cant reach statewide goals Three-Year Plan vs. Statewide Goals 3.5% 3.0% 2.5% 2.50% 2.55% 2.60% PY 2013 PY 2014 Segmented program evaluation and opportunity studies can uncover how/why: Moderate income status? House type (SF/MF)? Seasonal/vacation homes? Channel preferences vs. implementation channels? Baseline efficiencies already high? PY2015 2.0% 1.5% 1.0% 0.5% 0.0% AESP EM&V Online Conference 6
  • 7. Evaluators do report on differences by customer group, but sometimes we only look within a program Annual Percent Savings 2.5% Annual Percent Savings by Consumption Tertile 2.0% 1.5% 1.6% 1.8% 1.2% 1.0% 0.5% 0.0% Low Medium High Consumption Consumption Consumption Top 2030% Top 1020% Misleading to report, because the program targeted high users! Difficult for planners/evaluators to understand how to use findings Top 10% Make sure segment membership we report is relative to the customer population; use the same data source AESP EM&V Online Conference 7
  • 8. Segment-level insights are useful across the program lifecycle Metrics Measurement Opportunities Awareness / Knowledge General population and non-part surveys Intention Inquiries, leads, incomplete applications that link to customer database by account # Qualification Ex ante: Filter database by qualifying criteria Ex post: Program qualification rates Participation Program participation rates Portfolio-level participation: What % of all segment members have participated in any EE? Engagement Online / HEMS / IHD device tracking Participant surveys Impacts Realization rates by segment Savings depth by segment (% savings) Measure mix by segment AESP EM&V Online Conference 8
  • 10. So, what segmentation is good for EM&V purposes? 1 1. Segment membership must be identifiable ex ante for all customers Rate code (Low income, SF/MF, Small/large commercial) Psychographic lifestyle segment available through data providers (e.g., Experian) Usage characteristics (L/M/H; summer load; load shape) 2 1. Segments should distinguish between meaningful differences that affect program outcomes Energy opportunity Barriers to participation (own/rent; income) Motivation to participate Channel/communication preferences (on-bill, web, phone) Impacts! 3 1. Segments should be consumable by readers/regulators: Easy to understand / well-named Manageable number AESPEM&VOnlineConference 10
  • 11. 1 Identification: Tracking by segment requires defining segments based on readily-available data And we have a lot! Secondary demographic/ housing data e.g., age, income, home value Past program participation DSM and non-DSM TOU Account Rate A B C Energy Audit Ref. Rebate Customer characteristics from CIS data e.g., rate class, time-as-customer New Customer engagement e.g., online activity, payment preferences Energy indicators e.g., seasonal usage, load shape 1-4 yrs 5-9 yrs 10-19 20+ yrs yrs 0 2 4 6 8 10 12 14 16 18 20 22 AESP EM&V Online Conference 11
  • 12. 2 Meaningful Differences: Segment membership should correlate with savings opportunities, program propensity, barriers and preferences Demographically-Based Lifestyle Segmentation Custom Psychographi Segmentation Energy Usage Patterns Past Participation Highest Medium Dim. 1 Lowest Dim. 2 May correlate well with: Ability to participate Channel/ marketing affinity Heterogeneous in terms of: Savings opportunities May correlate well with: Ability to participate Motivation Heterogeneous in terms of: Savings opportunities Channel/ marketing affinity 0 2 4 6 8 10 12 14 16 18 20 22 May correlate well with: Savings opportunities Heterogeneous in terms of: Ability to participate Channel/ marketing affinity
  • 13. Have AMI data? Clustering customers into Load Shape Segments could enable long-term impact tracking Best target for DR and conservation programs? clustersimilar patterns Relatively high baseload - many EE/Wx opportunities WholeHouseLoadShapes 4000 3500 HighPeak/LowBaseload 3500 3000 3000 2500 2500 ExtendedPeak HighBaseload LowUsers Non-HVAC EE and behavioral interventions 2000 2000 1500 1500 1000 1000 500 500 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 0 0 1 2 3 4 5 6 7 8 Low-cost conservation and 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23behavior Identify highest-impact equipment, envelope and behavioral opportunities for each segment AESP EM&V Online Conference 13
  • 14. 3 Consumable segments: Easy to explain and interpret; manageable number Single dimensions (single-family / multi-family) or 2X2 matrices have merit But they leave a lot of heterogeneity undescribed Complex segmentation schemes quickly go un-used Reviewers dont have background/knowledge of approach Imagine 70 Experian lifestyle segments! Cost implications to what we choose Segment quotas AESP EM&V Online Conference 14
  • 15. We can start by reporting savings at a segment level Segment Percent of Customers Percent of Wx Participants Wx Savings per Household (kWh) Wx Savings Total (MWh) A 25% 28% 180 81.0 B 15% 14% 150 33.8 C 40% 34% 100 56.0 D 20% 24% 80 32 Total 100% 100% 124 202.8 AESP EM&V Online Conference 15
  • 16. End game: Identify and track program opportunities and success metrics specific to each segment Participationrate amongencouraged Savingsdepthor realizationrate n Targeted for Wx Wx Uptake (among those targeted) Wx Savings per Household (kWh) Wx Opportunity per Household (kWh) % of Opportunity Achieved 28% 5,000 9% 180 200 90% 15% 14% 3,000 7.5% 150 300 50% C 40% 34% 8,000 7% 100 150 75% D 20% 24% 4,000 10% 80 100 80% Total 100% 100% 20,000 8.2% 124 172 72% Segment Percent of Customers Percent of Wx Participants A 25% B AESP EM&V Online Conference 16
  • 17. Thank You! Amanda Dwelley Associate Director 617-301-4629 adwelley@opiniondynamics.com Visit us at www.opiniondynamics.com AESP EM&V Online Conference 17