際際滷

際際滷Share a Scribd company logo

Bertrand Fontaine - Deep Learning for driver/passenger detection of car trips

H
Hendrik D'Oosterlinck

Sentiance's Bertrand Fontaine gives a few concrete use-cases for deep learning on sensor data: transport classification, driver/passenger classification, and driver behavior characterization.

1 of 23
From mobile data to smart life.
Which data do we collect and what information do we extract Transport classifier Driving behaviour characterisation Driver passenger classification Overview
Our clients integrate our SDK in their app SDK detects stationaries: sends GPS fix, start/end timestamps SDK detects transports: one GPS fix per minute, accelerometer and gyroscope sensor data All processing done on AWS cloud Data collection
Low-level DATA (GPS+sensors)
Driving behaviour profiling Aggregate User profile For each trip
Transport Classification
One waypoint per minute Timestamps Coordinates (GPS+cell tower+wifi) Speed Accuracy Waypoints Sensors segment trip and classify Accelerometer Gyroscope BIKING WALKING IDLE Transport data
Transport Classification
Sensor processing for driving behaviour
Phone orientation alignment What we want: longitudinal and lateral car accelerations, angular velocity x z x z x z y z xy Straight up Mounted In pocket On passenger seat What we have: a phone with axes in arbitrary direction Align the referential of the phone with the referential of the car = find rotation matrix z Gravity pull x y x Gravity pull x z y z xyKalman filter Gravity pull x y x y Gradient descent
Phone orientation alignment z rotation matrix longitudinal lateral Compared externally with high-end inertial measurement units As good as 100k $ hardware accelerations brakes left turn right turn
Driver passenger classification
Driver vs Passenger - Context based Home Station Station Hotel Office car car car car train train car DRIVER PASSENGER Time line chunks with transports between two home stationaries Find consistent car cycles = closed loops where each trip starts where the previous one ended If home in cycle -> likely driver, if not passenger
Driver specific model Unsupervised approach utilising the derived car signals Patterns in signals relate to how one drives Learn a model specific for each user characterising her driving style Feature extractor must characterise driving style but invariant to road type, traffic, weather One feature extractor for all user Sensors from many trips Feature vectors User-specific model Feature extraction
Feature extraction: transfer learning We dont want to engineer the features -> deep learning approach No big labelled car data set annotated with driver or passenger Transfer learning: train a network for a task, use if for another Classify trips from 1000 users (1000 classes) take the output of a dense layer as feature vector = projection to metric space
Feature extraction: deep learning model User finger print: Acceleration, brake, turn events: short time scale Temporal relationship between events Beyond 30 seconds: road layout and traffic more prominent Data augmentation (sign, additive and multiplicative noise) 2M parameters 1D convolutions input 128x1x3 3 channels
Driver vs passenger: visualising the embeddings Users with ground truth (not used in the DL training) Assumption: most of the time driver Passenger are at the edge of the distribution and clustered together Build user-specific models using those feature vectors Blue: driver Red: passenger User #1 User #2 Sensors Feature vectorsNetwork
Driver vs passenger: anomaly detection Consider passenger trips as outliers: they differ a lot from the users average profile User model = isolation forest characterising the 50 dimensional feature space Use past trips to learn model and distribution (save user features) precision recall support passenger 0.66 0.84 25 driver 0.91 0.91 129 avg/total 0.92 0.90 154
Driver vs passenger: universal background model Learn a gaussian mixture model for a big population = universal background model Learn user model: for each new trip, update the universal model Compare log likelihood of both models precision recall support passenger 0.72 0.75 24 driver 0.96 0.95 136 avg/total 0.92 0.92 180
Population results and limitations Both approaches works best for different users For some users, models take longer (need more data) to stabilise Combine both approaches: recall=0.73 and precision=0.76 on passenger prediction How good is the assumption Number of passenger trips received at the beginning of the user model learning Harder if passenger trips in the same car Harder if driver trips in different cars Unsupervised driver/passenger classification
Meet the Sentiance team
Bertrand Fontaine - Deep Learning for driver/passenger detection of car trips
www.sentiance.com 息 2016

More Related Content

Bertrand Fontaine - Deep Learning for driver/passenger detection of car trips

  • 1. From mobile data to smart life.
  • 2. Which data do we collect and what information do we extract Transport classifier Driving behaviour characterisation Driver passenger classification Overview
  • 3. Our clients integrate our SDK in their app SDK detects stationaries: sends GPS fix, start/end timestamps SDK detects transports: one GPS fix per minute, accelerometer and gyroscope sensor data All processing done on AWS cloud Data collection
  • 7. One waypoint per minute Timestamps Coordinates (GPS+cell tower+wifi) Speed Accuracy Waypoints Sensors segment trip and classify Accelerometer Gyroscope BIKING WALKING IDLE Transport data
  • 10. Phone orientation alignment What we want: longitudinal and lateral car accelerations, angular velocity x z x z x z y z xy Straight up Mounted In pocket On passenger seat What we have: a phone with axes in arbitrary direction Align the referential of the phone with the referential of the car = find rotation matrix z Gravity pull x y x Gravity pull x z y z xyKalman filter Gravity pull x y x y Gradient descent
  • 11. Phone orientation alignment z rotation matrix longitudinal lateral Compared externally with high-end inertial measurement units As good as 100k $ hardware accelerations brakes left turn right turn
  • 13. Driver vs Passenger - Context based Home Station Station Hotel Office car car car car train train car DRIVER PASSENGER Time line chunks with transports between two home stationaries Find consistent car cycles = closed loops where each trip starts where the previous one ended If home in cycle -> likely driver, if not passenger
  • 14. Driver specific model Unsupervised approach utilising the derived car signals Patterns in signals relate to how one drives Learn a model specific for each user characterising her driving style Feature extractor must characterise driving style but invariant to road type, traffic, weather One feature extractor for all user Sensors from many trips Feature vectors User-specific model Feature extraction
  • 15. Feature extraction: transfer learning We dont want to engineer the features -> deep learning approach No big labelled car data set annotated with driver or passenger Transfer learning: train a network for a task, use if for another Classify trips from 1000 users (1000 classes) take the output of a dense layer as feature vector = projection to metric space
  • 16. Feature extraction: deep learning model User finger print: Acceleration, brake, turn events: short time scale Temporal relationship between events Beyond 30 seconds: road layout and traffic more prominent Data augmentation (sign, additive and multiplicative noise) 2M parameters 1D convolutions input 128x1x3 3 channels
  • 17. Driver vs passenger: visualising the embeddings Users with ground truth (not used in the DL training) Assumption: most of the time driver Passenger are at the edge of the distribution and clustered together Build user-specific models using those feature vectors Blue: driver Red: passenger User #1 User #2 Sensors Feature vectorsNetwork
  • 18. Driver vs passenger: anomaly detection Consider passenger trips as outliers: they differ a lot from the users average profile User model = isolation forest characterising the 50 dimensional feature space Use past trips to learn model and distribution (save user features) precision recall support passenger 0.66 0.84 25 driver 0.91 0.91 129 avg/total 0.92 0.90 154
  • 19. Driver vs passenger: universal background model Learn a gaussian mixture model for a big population = universal background model Learn user model: for each new trip, update the universal model Compare log likelihood of both models precision recall support passenger 0.72 0.75 24 driver 0.96 0.95 136 avg/total 0.92 0.92 180
  • 20. Population results and limitations Both approaches works best for different users For some users, models take longer (need more data) to stabilise Combine both approaches: recall=0.73 and precision=0.76 on passenger prediction How good is the assumption Number of passenger trips received at the beginning of the user model learning Harder if passenger trips in the same car Harder if driver trips in different cars Unsupervised driver/passenger classification