ºÝºÝߣ

ºÝºÝߣShare a Scribd company logo

Analytics in IoT

?
?
wesley Dias
wesley Dias

The document discusses analytics for Internet of Things (IoT) data from trucks. It describes an architecture that uses technologies like Kafka and Storm for real-time streaming of sensor data, HDFS for storage, Elasticsearch for retrieval, and Spark and machine learning tools for predictive analytics on the data to discover patterns related to violations. A web app with dashboards and alerts in ActiveMQ would display insights and messages based on the captured and analyzed truck event data.

1 of 11
Analytics in Internet of Things - IoT
Introduction
Introduction ? The Internet of Things (IoT) is a scenario in which objects, animals or people are provided with unique identifiers and the ability to transfer data over a network without requiring human-to-human or human-to-computer interaction. ? Thus, the internet of things creates an opportunity to measure, collect and analyze an ever-increasing variety of behavioral statistics. Cross-correlation of this data could revolutionize the targeted marketing of products and services. ? As per Forbes, IoT analytics will be hot, with a five-year CAGR of 30% ? The Internet of Things (IoT) will be the next critical focus for data/analytics services. While the IoT trend has focused on the data generation and production (sensors) side of the equation, the ¡°Analytics¡± of Things is a particular form of big data analytics that often involves anomaly detection and ¡°bringing the data to the analytics.¡± ? The Internet of Things (IoT) has a potential transformational effect on the data center market, its customers, technology providers, technologies, and sales and marketing models, according to Gartner, Inc. Gartner estimates that the IoT will include 26 billion units installed by 2020, and by that time, IoT product and service suppliers will generate incremental revenue exceeding $300 billion, mostly in services.
Business Case
Business Case - Scenario Scenario ? A truck generates millions of event for a given route. ? An event could be a normal event: Starting/Stopping a vehicle ? An event could be violation event: Speeding, excessive acceleration and braking. ? Company uses application to monitor location and violation of truck/driver at real time.
Architecture
HDFS (Truck Event) High Level Architecture YARN ElasticSearch Active MQ Kafka SPOUT ES BOLTHDFS BOLTMonitoring BOLT T(1 ) T(2 ) T(N) Truck Event Topic Events Table Hadoop ELASTICSEARCH STORM STREAM PROCESSING Web App Inbound Messaging KafkaTruck Streaming Data
Architecture Highlights ? Use of Real ¨C time streaming application to capture truck sensor data like Kafka and Storm ? Use of ElasticSearch for Retrieving the data present in HDFS ? Web App for displaying Dashboard and Insights processed using in-memory tools like Spark ? Use of machine learning tools like Spark Mllib/H2O for predictive analysis to discovering patterns in data related to violations like speeding and excessive acceleration/braking, sudden deceleration ? Use of Active MQ to display alerts and messages based on data captured
Technologies Used / Infrastructure
Hadoop [HDFS] IoT ¨C Device Data APIs P U S H AD HOC External Sources Real-Time user interface ? Alert and Event(ActiveMQ) Truck Sensor Stream Processing (STORM) APIs Inbound Messaging (Kafka) Interactive query (Spark Core/SQL) P U S H Real ¨Ctime Serving (ELASTICSEARCH) Q U E R Y G E T / P O S T A P I HTTP GET P U S H SQL
Technology Used Technology Used ? Hadoop ? Hive ? ElasticSearch ? SpringBoot/Java ? Storm ? Spark Core ? Spark Sql ? Spark Mllib/H2o

More Related Content

Analytics in IoT

  • 1. Analytics in Internet of Things - IoT
  • 3. Introduction ? The Internet of Things (IoT) is a scenario in which objects, animals or people are provided with unique identifiers and the ability to transfer data over a network without requiring human-to-human or human-to-computer interaction. ? Thus, the internet of things creates an opportunity to measure, collect and analyze an ever-increasing variety of behavioral statistics. Cross-correlation of this data could revolutionize the targeted marketing of products and services. ? As per Forbes, IoT analytics will be hot, with a five-year CAGR of 30% ? The Internet of Things (IoT) will be the next critical focus for data/analytics services. While the IoT trend has focused on the data generation and production (sensors) side of the equation, the ¡°Analytics¡± of Things is a particular form of big data analytics that often involves anomaly detection and ¡°bringing the data to the analytics.¡± ? The Internet of Things (IoT) has a potential transformational effect on the data center market, its customers, technology providers, technologies, and sales and marketing models, according to Gartner, Inc. Gartner estimates that the IoT will include 26 billion units installed by 2020, and by that time, IoT product and service suppliers will generate incremental revenue exceeding $300 billion, mostly in services.
  • 5. Business Case - Scenario Scenario ? A truck generates millions of event for a given route. ? An event could be a normal event: Starting/Stopping a vehicle ? An event could be violation event: Speeding, excessive acceleration and braking. ? Company uses application to monitor location and violation of truck/driver at real time.
  • 7. HDFS (Truck Event) High Level Architecture YARN ElasticSearch Active MQ Kafka SPOUT ES BOLTHDFS BOLTMonitoring BOLT T(1 ) T(2 ) T(N) Truck Event Topic Events Table Hadoop ELASTICSEARCH STORM STREAM PROCESSING Web App Inbound Messaging KafkaTruck Streaming Data
  • 8. Architecture Highlights ? Use of Real ¨C time streaming application to capture truck sensor data like Kafka and Storm ? Use of ElasticSearch for Retrieving the data present in HDFS ? Web App for displaying Dashboard and Insights processed using in-memory tools like Spark ? Use of machine learning tools like Spark Mllib/H2O for predictive analysis to discovering patterns in data related to violations like speeding and excessive acceleration/braking, sudden deceleration ? Use of Active MQ to display alerts and messages based on data captured
  • 9. Technologies Used / Infrastructure
  • 10. Hadoop [HDFS] IoT ¨C Device Data APIs P U S H AD HOC External Sources Real-Time user interface ? Alert and Event(ActiveMQ) Truck Sensor Stream Processing (STORM) APIs Inbound Messaging (Kafka) Interactive query (Spark Core/SQL) P U S H Real ¨Ctime Serving (ELASTICSEARCH) Q U E R Y G E T / P O S T A P I HTTP GET P U S H SQL
  • 11. Technology Used Technology Used ? Hadoop ? Hive ? ElasticSearch ? SpringBoot/Java ? Storm ? Spark Core ? Spark Sql ? Spark Mllib/H2o