際際滷

際際滷Share a Scribd company logo

Securing Solr Search Data in the Cloud

?
?
Sameer Maggon
Sameer Maggon

Apache Solr is a highly scalable, open source search technology that makes it easy for organizations to deliver core search, recommendations or business intelligence applications. In Solr today, if you have to store and provide search against confidential data, you are on your own on trying to figure out how to protect that data from being encrypted at rest, during transport or to prevent a rogue employee or third party having access to that sensitive data. In this talk, you will learn about how you can keep the search index secure and provide key based encryption that ensures that your sensitive data is protected at all levels and your Solr application meets compliance requirements.

1 of 20
Downloaded 25 times
Securing Search Data in the Cloud Sameer Maggon Measured Search Harry Ochiai Hitachi Solutions New York Enterprise Cloud Meetup Jan 25, 2017
2 Agenda ? About the Speakers ? About Measured Search & Hitachi Solutions ? What is Apache Solr? ? Where is Apache Solr used? ? How Search Data is stored ? Data Security Challenge in the Cloud ? Protecting Con?dential Search Data ? Challenges of Encrypted Search Index ? Encryption Solution ? Demo ? Q&A ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
3 About the Speakers Harry Ochiai ? Senior Business Development Manager of Hitachi Solutions ? Worked on networking, cyber security, and storage ? Focus on cloud encryption solutions since 2013 ? New Yorker Sameer Maggon ? Founder / Technologist at Measured Search ? Been working in Open Source Search since 2001 (Lucene/Solr/Elastic) ? USC Engineering Alumni ? Works and Lives in Los Angeles, CA ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
4 About Hitachi Solutions Global IT Solutions Company ? A Hitachi Company ? Japan(HQ), North America, Europe, China, India and Southeast Asia ? 12,000 Employees Leading security solution provider in Japan ? Innovator and leading provider of encryption technology for over 20 years ? HIBUN: 40% market share in the endpoint encryption segment in Japan ? Launched new security solution Credeon globally in 2013 ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
5 About Measured Search Measured Search? enables companies to elevate the experience of Search based applications faster and with more con?dence. Managed Services & Support SearchStax? Platform as a Service On-Demand Expertise & Consulting ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
6 by Measured SearchSearchStax? SearchStax? Solr Cloud Manager SearchStax? Pulse SearchStax? Analytics Comprehensive Solr Monitoring & Alerting with service level reporting to proactively manage your clusters. Realtime feedback & user insights to help optimize your Search Experience Easiest way to run & manage Solr in the cloud - saves time, money and reduces risk. ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
7 What is Apache Solr? Solr is the popular, blazing-fast, open source enterprise search platform built on Apache Lucene? ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
8 Where is Apache Solr used? Government eCommerce Education Life Sciences Entertainment Healthcare Financial Services High Tech ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
9 Where is Apache Solr used? ?nding tickets ?nding job ?nding restaurant/services Enterprise Search Media Search Retail Customer Search Fraud Analytics Publishing RecruitingTravelResearch Business Intelligence ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
10 Search Index Search platforms maintains internal indices of terms and properties of each indexed document in plaintext. Plain Search Index Encrypted Search Index ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
11 Data Security Challenges in the Cloud Solr / Search Cluster Backups Threats Managed Service Provider (MSP) Rogue EmployeeManaged Service Provider Rogue Employee Hacker Accidental Data Access ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
12 Challenges of Encrypted Search Index Challenges ? To search through encrypted data, data must be decrypted ? Decryption slows down the process ? Encryption limits usability Goals ? Maintain encrypted state without sacri?cing security level ? Encrypt using your own key ? Maintain search performance and usability ? Protect against unauthorized users and rogue system administrators ? Regulatory compliance Solution ? Searchable Encryption technology ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
13 Protecting Con?dential Search Data Solr/Lucene text Key word result value Solr/Lucene text Key word result File System Encryption value Solr/Lucene text Key word result value Simple Encryption Solr/Lucene text Key word result value Searchable Encryption text Key word result value Searchable Encryption Storage Storage Storage Storage Storage Client Server App Server OS No Encryption OS Encryption Simple Encryption with Solr Plugin Client-Side Searchable Encryption Searchable Encryption with Solr Plugin Low Security High Security No Security Decryption at storage layer X Difficult to separate key Decrypt first and match X Very slow X plaintextin memory Match first and decrypt O Key separation O High Performance(1) O Semantically Secure(2) X plaintextin memory Client Client Client Client Client Decryption at client-side O Key separation O High Performance(1) O Semantically Secure(2) O no plaintexton server Solr/Lucene (1) Use of Symmetric Key (2) Probabilistic Encryption Scheme ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
14 Search Encryption 101 Searchable Encryption is an encryption technology for searching data in an encrypted state. ? Fully Homomorphic Encryption ? Homomorphic Encryption ? Functional Encryption ? Hitachi Searchable Encryption Practical CKA CPA Tokenization 103 10610010-3 Hitachi Searchable Encryption Functional Encryption Homomorphic Encryption Fully Homomorphic Encryption Security Performance (Search / sec) EncryptedPlain CKA: Chosen Keyword Attack CPA: Chosen Phrase Attack ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
15 Search Encryption 101 Encryption Schemes ? Deterministic ? Constant value ? Vulnerable to statistical attacks ? Probabilistic ? Random value ? Semantically secure Encryption Key Exchange ? Symmetric ? Asymmetric / PKI ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
16 Solution: Searchable Encryption with Solr Plugin Hitachi Credeon Secure Full-Text Search ? Searchable Encryption plugin for Apache Solr and Elasticsearch ? Deterministic Encryption Scheme ? 128 bit randomization ? AES 256, FIPS 140-2 ? Symmetric Key ? Real-time search (15%+ overhead) ? Key Management System, Java KeyStore ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
17 Solution: Client Side Searchable Encryption Credeon Secure Document Solution for SharePoint Online ? Client-side encryption for search index and data ? Searchable encryption on Solr ? Search Engine and Key Management are independent of Microsoft Search Server SharePoint Server Key Management Server Search Engine SharePoint Online Client PC 1. Get a key 2. Index the file contents and encrypt index 4. Upload encrypted file 4. Upload encrypted index 3. Encrypt file ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
18 Demo: Securing Solr Search in the Cloud SearchStax with Credeon ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
19 Q&A ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
20 Contact Info Sameer Maggon @maggon sameer@measuredsearch.com https://www.measuredsearch.com Harry Ochiai @credeon hochiai@hitachi-solutions.com https://psg.hitachi-solutions.com/credeon/overview ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.

More Related Content

Securing Solr Search Data in the Cloud

  • 1. Securing Search Data in the Cloud Sameer Maggon Measured Search Harry Ochiai Hitachi Solutions New York Enterprise Cloud Meetup Jan 25, 2017
  • 2. 2 Agenda ? About the Speakers ? About Measured Search & Hitachi Solutions ? What is Apache Solr? ? Where is Apache Solr used? ? How Search Data is stored ? Data Security Challenge in the Cloud ? Protecting Con?dential Search Data ? Challenges of Encrypted Search Index ? Encryption Solution ? Demo ? Q&A ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
  • 3. 3 About the Speakers Harry Ochiai ? Senior Business Development Manager of Hitachi Solutions ? Worked on networking, cyber security, and storage ? Focus on cloud encryption solutions since 2013 ? New Yorker Sameer Maggon ? Founder / Technologist at Measured Search ? Been working in Open Source Search since 2001 (Lucene/Solr/Elastic) ? USC Engineering Alumni ? Works and Lives in Los Angeles, CA ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
  • 4. 4 About Hitachi Solutions Global IT Solutions Company ? A Hitachi Company ? Japan(HQ), North America, Europe, China, India and Southeast Asia ? 12,000 Employees Leading security solution provider in Japan ? Innovator and leading provider of encryption technology for over 20 years ? HIBUN: 40% market share in the endpoint encryption segment in Japan ? Launched new security solution Credeon globally in 2013 ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
  • 5. 5 About Measured Search Measured Search? enables companies to elevate the experience of Search based applications faster and with more con?dence. Managed Services & Support SearchStax? Platform as a Service On-Demand Expertise & Consulting ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
  • 6. 6 by Measured SearchSearchStax? SearchStax? Solr Cloud Manager SearchStax? Pulse SearchStax? Analytics Comprehensive Solr Monitoring & Alerting with service level reporting to proactively manage your clusters. Realtime feedback & user insights to help optimize your Search Experience Easiest way to run & manage Solr in the cloud - saves time, money and reduces risk. ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
  • 7. 7 What is Apache Solr? Solr is the popular, blazing-fast, open source enterprise search platform built on Apache Lucene? ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
  • 8. 8 Where is Apache Solr used? Government eCommerce Education Life Sciences Entertainment Healthcare Financial Services High Tech ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
  • 9. 9 Where is Apache Solr used? ?nding tickets ?nding job ?nding restaurant/services Enterprise Search Media Search Retail Customer Search Fraud Analytics Publishing RecruitingTravelResearch Business Intelligence ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
  • 10. 10 Search Index Search platforms maintains internal indices of terms and properties of each indexed document in plaintext. Plain Search Index Encrypted Search Index ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
  • 11. 11 Data Security Challenges in the Cloud Solr / Search Cluster Backups Threats Managed Service Provider (MSP) Rogue EmployeeManaged Service Provider Rogue Employee Hacker Accidental Data Access ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
  • 12. 12 Challenges of Encrypted Search Index Challenges ? To search through encrypted data, data must be decrypted ? Decryption slows down the process ? Encryption limits usability Goals ? Maintain encrypted state without sacri?cing security level ? Encrypt using your own key ? Maintain search performance and usability ? Protect against unauthorized users and rogue system administrators ? Regulatory compliance Solution ? Searchable Encryption technology ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
  • 13. 13 Protecting Con?dential Search Data Solr/Lucene text Key word result value Solr/Lucene text Key word result File System Encryption value Solr/Lucene text Key word result value Simple Encryption Solr/Lucene text Key word result value Searchable Encryption text Key word result value Searchable Encryption Storage Storage Storage Storage Storage Client Server App Server OS No Encryption OS Encryption Simple Encryption with Solr Plugin Client-Side Searchable Encryption Searchable Encryption with Solr Plugin Low Security High Security No Security Decryption at storage layer X Difficult to separate key Decrypt first and match X Very slow X plaintextin memory Match first and decrypt O Key separation O High Performance(1) O Semantically Secure(2) X plaintextin memory Client Client Client Client Client Decryption at client-side O Key separation O High Performance(1) O Semantically Secure(2) O no plaintexton server Solr/Lucene (1) Use of Symmetric Key (2) Probabilistic Encryption Scheme ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
  • 14. 14 Search Encryption 101 Searchable Encryption is an encryption technology for searching data in an encrypted state. ? Fully Homomorphic Encryption ? Homomorphic Encryption ? Functional Encryption ? Hitachi Searchable Encryption Practical CKA CPA Tokenization 103 10610010-3 Hitachi Searchable Encryption Functional Encryption Homomorphic Encryption Fully Homomorphic Encryption Security Performance (Search / sec) EncryptedPlain CKA: Chosen Keyword Attack CPA: Chosen Phrase Attack ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
  • 15. 15 Search Encryption 101 Encryption Schemes ? Deterministic ? Constant value ? Vulnerable to statistical attacks ? Probabilistic ? Random value ? Semantically secure Encryption Key Exchange ? Symmetric ? Asymmetric / PKI ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
  • 16. 16 Solution: Searchable Encryption with Solr Plugin Hitachi Credeon Secure Full-Text Search ? Searchable Encryption plugin for Apache Solr and Elasticsearch ? Deterministic Encryption Scheme ? 128 bit randomization ? AES 256, FIPS 140-2 ? Symmetric Key ? Real-time search (15%+ overhead) ? Key Management System, Java KeyStore ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
  • 17. 17 Solution: Client Side Searchable Encryption Credeon Secure Document Solution for SharePoint Online ? Client-side encryption for search index and data ? Searchable encryption on Solr ? Search Engine and Key Management are independent of Microsoft Search Server SharePoint Server Key Management Server Search Engine SharePoint Online Client PC 1. Get a key 2. Index the file contents and encrypt index 4. Upload encrypted file 4. Upload encrypted index 3. Encrypt file ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
  • 18. 18 Demo: Securing Solr Search in the Cloud SearchStax with Credeon ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
  • 19. 19 Q&A ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
  • 20. 20 Contact Info Sameer Maggon @maggon sameer@measuredsearch.com https://www.measuredsearch.com Harry Ochiai @credeon hochiai@hitachi-solutions.com https://psg.hitachi-solutions.com/credeon/overview ? Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.