狠狠撸

狠狠撸Share a Scribd company logo

Splunk bangalore user group 2020 07-06

?Download as PPTX, PDF?
?
N
NiketNilay

Attendees Introduction Session 1 on Splunk Add On Builder Session 2 on Splunk Custom Visualization API

1 of 65
Download to read offline
Bengaluru User Group WELCOME 6th July 2020 ?????? ?????? ?????? ?????? ??? ?????????????? ??????? ????? ?? ?????? ?? ??????? ????? ?????? ????? ????? ?????
Housekeeping Join #splunk_bengaluru_usergroup on Slack http://splk.it/slack Use #splunk_bengaluru_usergroup for Q&A during the session Please keep your lines muted when not speaking 狠狠撸s, recording & feedback form will be posted to the Events page Splunk Bengaluru User Group https://usergroups.splunk.com/bengaluru-splunk-user-group/
? 2020 SPLUNK INC. Session 1
Splunk Add On Builder Sr. Sales Engineer (Splunk) Architect/Consultant Blackbelt in Unarmed Combat Astrophotography Enthusiast DIY Enthusiast Sandeep Nayak
? 2020 SPLUNK INC. Q&A Raise hand to be unmuted Post questions in WebEx Chat Join Slack for Q&A http://splk.it/slack
? 2020 SPLUNK INC. Community Resources Splunk Learning & Development Docs: https://docs.splunk.com/ Blogs: https://docs.splunk.com/ Dev: https://docs.splunk.com/ Lantern: https://lantern.splunk.com/hc/en-us Education: https://education.splunk.com/free Github: https://github.com/splunk Splunkbase: https://splunkbase.splunk.com/ Splunk Community Sessions User Groups: https://usergroups.splunk.com/ Events: https://www.splunk.com/en_us/about- us/events.html Live: https://www.splunk.com/en_us/events/splunklive.html .Conf: https://conf.splunk.com/watch/conf-online.html Splunk queries Answers: https://commmunity.splunk.com/ Slack Splunk User Group: http://splk.it/slack Support & Services: https://www.splunk.com/en_us/support-and-services.html Enhancement Requests Ideas: https://ideas.splunk.com/
? 2020 SPLUNK INC. Session 2
Splunk Custom Visualization API Niket Nilay Sales Engineer (Splunk) Splunk Trust MVP 2018,2019,2020 Splunk Answers karma leader with over 50K points
? 2020 SPLUNK INC. Q&A Raise hand to be unmuted Post questions in WebEx Chat Join Slack for Q&A http://splk.it/slack
? 2020 SPLUNK INC. Community Resources Splunk Learning & Development Docs: https://docs.splunk.com/ Blogs: https://docs.splunk.com/ Dev: https://docs.splunk.com/ Lantern: https://lantern.splunk.com/hc/en-us Education: https://education.splunk.com/free Github: https://github.com/splunk Splunkbase: https://splunkbase.splunk.com/ Splunk Community Sessions User Groups: https://usergroups.splunk.com/ Events: https://www.splunk.com/en_us/about- us/events.html Live: https://www.splunk.com/en_us/events/splunklive.html .Conf: https://conf.splunk.com/watch/conf-online.html Splunk queries Answers: https://commmunity.splunk.com/ Slack Splunk User Group: http://splk.it/slack Support & Services: https://www.splunk.com/en_us/support-and-services.html Enhancement Requests Ideas: https://ideas.splunk.com/
We plan to meet 1st Monday of every month at 6:00 PM IST. Please provide feedback for : ? Sessions and improvements. ? Topics to be covered in future sessions. ? Let us know if you are interested in presenting in User Group. Keep the comradery through Slack and Splunk Answers> What’s Next http://splk.it/slack http://community.splunk.com
Splunk Add-On Builder Sandeep Nayak Sr. Sales Engineer (Splunk)
During the course of this presentation, we may make forward-looking statements regarding future events or the expected performance of the company. We caution you that such statements reflect our current expectations and estimates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward-looking statements, please review our filings with the SEC. The forward-looking statements made in this presentation are being made as of the time and date of its live presentation. If reviewed after its live presentation, this presentation may not contain current or accurate information. We do not assume any obligation to update any forward-looking statements we may make. In addition, any information about our roadmap outlines our general product direction and is subject to change at any time without notice. It is for informational purposes only and shall not be incorporated into any contract or other commitment. Splunk undertakes no obligation either to develop the features or functionality described or to include any such feature or functionality in a future release. Splunk, Splunk>, Listen to Your Data, The Engine for Machine Data, Splunk Cloud, Splunk Light and SPL are trademarks and registered trademarks of Splunk Inc. in the United States and other countries. All other brand names, product names, or trademarks belong to their respective owners. ? 2019 Splunk Inc. All rights reserved. Forward- Looking Statements
Splunk The only “Data to Everything” Platform Any Data Structure Any Source Any Timescale Any Insight Any Action Apply data across IT, security, application development, and business questions, decisions, and actions. Any Insight
Splunk supports a variety of data sources Security Outside the Datacenter Applications Email server Web server App server Log4J, JMS, JMX Networking Routers SNMP Netflow configurations Databases Configurations Audit/query logs Tables Schemas Virtualization & Cloud Hypervisor Guest OS, Apps Cloud Linux/Unix Configurations syslog File system ps, iostat, top Windows Registry Event logs File system sysinternals Logfiles Configs Messages Traps Alerts Metrics Scripts TicketsChanges Authentication Firewall Intrusion prevention Anti-malware Data Loss prevention Vulnerability scanner Manufacturing, logistics… Power consumption RFID data GPS data
? The Splunk Add-on Builder is a Splunk app that helps you build and validate Technology add-ons (TAs) for your Splunk Enterprise deployment. ? The goals of the Splunk Add-on Builder are to: ? Guide you through all of the necessary steps of creating an add-on ? Build alert actions and adaptive response actions for Splunk Enterprise Security ? Reduce development and testing time ? Follow best practices and naming conventions ? Maintain CIM compliance ? Maintain quality of add-ons ? Validate and test the add-on, helping you to check for certification readiness and to identify any limitations such as compatibilities and dependencies ? Maintain a consistent look and feel while still making it easy for you to add branding Splunk Add On builder https://splunkbase.splunk.com/app/2962/
Add On Builder Easily create add-ons: ? To collect data ? To create alert actions ? Map to Data Models ? Field Extractions ? Validate and Package for distribution Screenshot from Splunk Dashboard Examples App
Demo
Add-On Builder Steps Splunk Docs : https://docs.splunk.com/Documenta tion/AddonBuilder/latest/UserGuide/ Overview
Add-On Builder Steps ** Latest d3 installed instead of specific 3.5 version. Currently 5.16.0 Configure Data Collection: ? REST API Call ? Run a Shell command ? Python Code
Add-On Builder Steps Create the Setup Page for the Add-On 1. Configure Server URL to connect to 2. Configure credentials to use 3. This would all be setup as the main configuration for the app
Add-On Builder Steps Create the Inputs 1. Configure what information to query if the REST API supports passing parameters.
Add-On Builder Steps Test and Debug your Script/REST API 1. Interface to check outputs and event creation 2. Python basic error outputs
Add-On Builder Steps Field Extractions: 1. Guided mechanism to extract fields. 2. Add-on builder will create transforms.conf for you 3. Add sample data to test
Add-On Builder Steps Map to the Data Model 1. Very useful when mapping data to fit an existing data model like CIM or ITSI data models 2. Provides mechanisms to map to the data model and create several Eval conditions
Add-On Builder Steps Validate and package the app 1. Makes the app ready for certification! 2. Package and Download the app for distribution
Further Resources https://splunkbase.splunk.com/app/2962/ Some good online tutorials available here: ? http://dev.splunk.com/view/addon-builder/SP-CAAAE83 ? http://dev.splunk.com/view/addon-builder/SP-CAAAE9F ? https://www.splunk.com/blog/2016/04/04/building-add-ons- has-never-been-easier/ ? https://www.splunk.com/blog/2016/10/12/building-add-ons- just-got-2-0-times-easier.html ? https://www.splunk.com/blog/2016/10/24/creating-mcafee- epo-alert-and-arf-actions-with-add-on-builder/ ? Splunk .Conf Sessions ? Splunk Blogs ? Splunk Dev ? Splunk User Groups on Slack Join #add-on-builder channel ? Splunk Community
Thank You
Splunk Custom Visualization API Niket Nilay Sales Engineer (Splunk)
During the course of this presentation, we may make forward-looking statements regarding future events or the expected performance of the company. We caution you that such statements reflect our current expectations and estimates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward-looking statements, please review our filings with the SEC. The forward-looking statements made in this presentation are being made as of the time and date of its live presentation. If reviewed after its live presentation, this presentation may not contain current or accurate information. We do not assume any obligation to update any forward-looking statements we may make. In addition, any information about our roadmap outlines our general product direction and is subject to change at any time without notice. It is for informational purposes only and shall not be incorporated into any contract or other commitment. Splunk undertakes no obligation either to develop the features or functionality described or to include any such feature or functionality in a future release. Splunk, Splunk>, Listen to Your Data, The Engine for Machine Data, Splunk Cloud, Splunk Light and SPL are trademarks and registered trademarks of Splunk Inc. in the United States and other countries. All other brand names, product names, or trademarks belong to their respective owners. ? 2019 Splunk Inc. All rights reserved. Forward- Looking Statements
Splunk The only “Data to Everything” Platform Any Data Structure Any Source Any Timescale Any Insight Any Action Apply data across IT, security, application development, and business questions, decisions, and actions. Any Insight
Built-in Visualizations Core visualizations Line, Area, Column, Bar, Pie, Scatter, Bubble, Single Value, Radial Gauge, Filler Gauge, Marker Gauge, Cluster Map, Choropleth Pros ?Customization through UI and Simple XML. ?Supports drilldown and trellis. ?Extension through CSS and JS. ?Available for Connected Experience – Mobile, TV, AR, VR. ?Available in new Dashboard
? Looking Good ≠ Easily Comprehendible ? Each chart solves different purpose depending on correlation and use case ? Know power of each visualization as well as their weaknesses ? Reduce chart-junk and increase data-to-ink ratio in dashboard ? While configuring follow precedence – UI > SimpleXML > CSS Extension > SplunkJS ? Built in visualization features are extended with every release. .Conf 2019 Talk – IT1523 : The Top 10 Glasstable Design Principles to Boost your Career and your Business Splunk Built-in Visualizations “Not all charts are created equal”
1. Get Custom Visualizations from Splunkbase ? Built on top of Splunk Custom Visualization API. Both Splunk built and Community built. ? May not support Drilldown or Trellis out of the box. 2. Covert Simple XML to HTML ? Allows importing and integrating JS based custom visualization as in any HTML app with JS scripting. ? May loose several features and ease of Simple XML. 3. Simple XML JS and CSS Extension ? Similar as above but allows to retain Simple XML features, ease of configuration and editing. ? No framework to integrate and may loose reusability, configuration and extensions to the visualization. 4. Build yourself using Custom Visualization API ? Use Splunk Custom Visualization API to build custom visualization based on preferred viz and use case. ? Requires development time as any other chart library integration would take. 5. Build Custom visualization using New Dashboard Framework Custom Visualizations Oooooo shiny visualization... How do I get it in Splunk?
Custom Visualizations Custom Visualizations Visualizations based on JS Libraries like D3 imported to Splunk. Pros ?Ability to import any visualization. ?Extension through CSS and JS. ?Splunk built and community built. Cons ?May not support drilldown and trellis. ?Not available for Connected Experience. ?May not work with new Dashboard Screenshot from Splunk Dashboard Examples App
Splunkbase Custom Visualizations Custom Visualizations on Splunkbase
Visualization Libraries Custom Visualizations on Splunkbase Choose from famous Charting Libraries ? D3 JS ? Highcharts ? Chart.js ? … And many more.
App directory and file structure appname appserver static visualizations <visualization_name> src <visualization_source>.js webpack.config.js visualization.js visualization.css formatter.html package.json preview.png default visualizations.conf savedsearches.conf metadata default.meta README savedsearches.conf.spec static appIcon.png, appIconAlt.png, appLogo.png appIcon_2x.png, appIconAlt_2x.png, appLogo_2x.png Development mode settings For working with static files in development mode without bump/refresh/restart through “web.conf” configuration file in etc/system/local. npm package file. Webpack configuration file SplunkVisualizationBase must override the following: getInitialDataParams – Data handling updateView – Visualization handling SplunkVisualizationUtils from security requirement. <splunk-control-group> <splunk-select> <splunk-radio-input> <option> <splunk-text-area> <splunk-text-input> <splunk-color-picker> <splunk-color> Custom Visualization API Splunk Docs – Custom Visualizations PNG image file used in the Visualization Picker user interface Settings for loading the visualization and making it available in Splunk Web (Including Drilldown and Trellis options) visualization property default values export visualizations to the system declare visualization properties static images for App logos
Demo
Build a Custom Visualization Tutorial https://docs.splunk.com/Documentation/Splunk/latest/AdvancedDev/CustomVizTu torial ? Windows*** steps may vary like setting environment variables ? Some steps with latest version of the dependent app would need correction. ? For ex: Vulnerability fix for jQuery requires latest version. ? package.json changes to support modules instead of root and change in path. ? d3 function changes for selected sample viz in <visualization_source>.js. ? The sample App covers basic structure for learning and not complete end-to-end feature development. ? We will add drilldown, trellis, tooltip and dark mode on top of the tutorial app, add custom App Icon, Visualization Icon. ? Package App and validate through Splunk App Inspection before publishing on Splunkbase. *** While Windows config is not covered in the session or directly in the Build a Custom Visualization Splunk documentation, it is possible. Splunk community resources https://docs.splunk.com, https://community.splunk.com would be right place to find the details. Unless already on #splunk_bengaluru_user_group Slack channel in the Splunk User group on Slack, use the following URL to get access and contact us for discussion and assistance: http://splk.it/slack
Custom Visualization Build Steps Splunk Docs Build a Custom Visualization: https://docs.splunk.com/Documenta tion/Splunk/latest/AdvancedDev/Cu stomVizTutorial 1. Download the custom visualization app template. 2. Unzip and rename the template as my_radial_meter. 3. Copy the template app to $SPLUNK_HOME/etc/apps/ folder. Set SPLUNK_HOME environment variable (steps are different for PC and Mac). 4. Rename $SPLUNK_HOME/etc/apps/my_radial_meter/appserver/st atic/visualizations/standin folder to my_radial_meter 5. Navigate to $SPLUNK_HOME/etc/apps/my_radial_meter/appserver/st atic/visualizations/my_radial_meter 6. Rename visualization_source.js as my_radial_meter_source.js
Custom Visualization Build Steps 7. npm install 8. sudo npm install -g npm 9. npm install --save d3 10.npm install --save underscore 11.npm install --save jquery 12.npm audit ? Fix Issues found during auditing 13.npm install jquery@3.5.1 14.npm install --save-dev webpack@4.43.0 ? Ensure no audit issues/vulnerabilities ** Latest d3 installed instead of specific 3.5 version. Currently 5.16.0
package.json 15.Modify package.json Following changes are made: 1. name changed to my_radial_meter 2. description added 3. main changed to my_radial_meter,js
webpack.config.js 16.Modify webpack.config.js Following changes are made: 1. Entry changed to my_radial_meter_source.js 2. root changed to modules 3. Path to source and node modules changed 4. Output path changed 5. Output filename changed to my_radial_meter,js
app.conf 17.Modify app.conf with required details
Visualization Source JS Bare-metal Setup Code 18.Get bare-minimum Visualization Source in the my_radial_meter_source.js file. https://docs.splunk.com/Documentation/Splunk/latest/AdvancedDev/CustomVizTutorial#Set_up_the_visualization_source_code
Bare Minimal Update View Code 19.Get updateView code. https://docs.splunk.com/Documentation/Splunk/latest/AdvancedDev/CustomVizTutorial#Add_the_updateView_function Following changes are made: 1. Change d3.scale.linear() with d3.scaleLinear() 2. Change d3.svg.arc() with d3.arc() 3. Change data handle in update view // Return if no data if (!data) { return; }
visualization.css 20.Apply template CSS https://docs.splunk.com/Documentation/Splunk/latest/AdvancedDev/CustomVizTutorial#Add_CSS
visualizations.conf 21.Register visualization from my_radial_meter/default/visualizations.conf
default.meta 22.Export the visualization through default.meta
Handle Data Format Errors 23. Validates and processes raw data, through formatData section. Get raw data object from splunkd and returns an object formatted for rendering Validate and Process data ? Check for empty data ? Check for invalid data ? Throw helpful errors ? Sanitize/secure data through utilities like escapeHtml() and makeSafeURL()
Build bare minimal 24. Run npm run build and choose yes for webpack-cli install.
Validate viz in UI 25. Basic viz does not have: Drilldown, Tooltip, Trellis, Dark Theme and Icons.
Set Icons for Visualization 26. Application Icons ? Place icon preview.png under folder <app_name>/appserver/static/visualizations/<viz_name> ? Place PNG icons appIcon, appIconAlt and appLogo and corresponding _2X png icons under folder <app_name>/static Validate Icons: 1. Splunk App Icon, 2. Alternate Icon, 3. Dashboard App Bar Icon, 4. Visualization picker icon
Enable Viz Formatting 27. Add formatting options in savedsearches.conf, savedsearches.conf.spec and formatter.html Visualization source JS file uses getPropertyNamespaceInfo() for reading viz properties (inside updateView() method).
Enable Trellis 28. Trellis Mode configuration supports_trellis = True Needs to be set in visualizations.conf as per previous step. May or may not work for all Custom visualizations.
Enable Drilldown 29. Trellis Mode configuration supports_drilldown = True Needs to be set in visualizations.conf as per previous step.
Enable Tooltip 30. Change visualization source JS file to handle tooltip.
Dark Mode 31. Dark Mode configuration In our example background color for svg will be turned transparent (instead of White color provided in the template), so that it picks panel color. Dark Mode used for Tooltip.
Final Output 32. Rebuild and validate the viz.
Further Resources Package app using Splunk Packaging Toolkit Validate App quality using Splunk App Inspect FN1735 – What’s next for Geo in Splunk DEV1141 – Building Applications with Splunk UI and Splunk React Visualizations DEV2178 – Building Your Own Custom Data Visualization for Dashboards? Splunk .Conf Sessions ? Splunk Blogs ? Splunk Dev ? Splunk User Groups on Slack Join #visualizations channel ? Splunk Community
Appendix
Template Code and Build Steps For Demo Link to GitHub repository for demo template: https://github.com/niketnilay/bengal uru_user_group/tree/master/2020- 07-06/my_radial_meter UPDATED BUILD STEPS for my_radial_meter_template 1. Extract my_radial_meter_template.zip to my_radial_meter_template folder. 2. export SPLUNK_HOME=/Applications/Splunk (MAC/*nix) set SPLUNK_HOME=<path> (on Windows) 3. Copy my_radial_meter folder to your $SPLUNK_HOME/etc/apps/ folder. 4. Navigate to $SPLUNK_HOME/etc/apps/my_radial_meter/appserver/static/visualizations/my _radial_meter [ONLY FOR WINDOWS] Get package.json file and update in above folder. PS: This is for path $SPLUNK_HOME to be used as %SPLUNK_HOME% in Windows. For Mac with Xcode Editor, to avoid following npm install build error (try step 5 before step 6) build error gyp ERR! stack Error: `make` failed with exit code: 69
Template Code and Build Steps For Demo Link to GitHub repository for demo template: https://github.com/niketnilay/bengal uru_user_group/tree/master/2020- 07-06/my_radial_meter Continued… 5. sudo xcodebuild -license (for Mac with XCode) Hit Enter (to open license agreement) Press q (quit) Type agree (to agree to the license) 6. sudo npm install 7. npm install --save d3 8. npm install --save underscore 9. npm install --save jquery 10. npm run build (Splunk Restart and Browser history cleanup may be required) 11. Validate the visualization in UI 12. Get my_radial_meter_source.js updated code with Drilldown, Tooltip, Dark Mode and replace the content of existing JS from Github or Google Drive. 13. Build code and validate UI (Splunk Restart and Browser history cleanup may be required)
Thank You

More Related Content

Splunk bangalore user group 2020 07-06

  • 1. Bengaluru User Group WELCOME 6th July 2020 ?????? ?????? ?????? ?????? ??? ?????????????? ??????? ????? ?? ?????? ?? ??????? ????? ?????? ????? ????? ?????
  • 2. Housekeeping Join #splunk_bengaluru_usergroup on Slack http://splk.it/slack Use #splunk_bengaluru_usergroup for Q&A during the session Please keep your lines muted when not speaking 狠狠撸s, recording & feedback form will be posted to the Events page Splunk Bengaluru User Group https://usergroups.splunk.com/bengaluru-splunk-user-group/
  • 3. ? 2020 SPLUNK INC. Session 1
  • 4. Splunk Add On Builder Sr. Sales Engineer (Splunk) Architect/Consultant Blackbelt in Unarmed Combat Astrophotography Enthusiast DIY Enthusiast Sandeep Nayak
  • 5. ? 2020 SPLUNK INC. Q&A Raise hand to be unmuted Post questions in WebEx Chat Join Slack for Q&A http://splk.it/slack
  • 6. ? 2020 SPLUNK INC. Community Resources Splunk Learning & Development Docs: https://docs.splunk.com/ Blogs: https://docs.splunk.com/ Dev: https://docs.splunk.com/ Lantern: https://lantern.splunk.com/hc/en-us Education: https://education.splunk.com/free Github: https://github.com/splunk Splunkbase: https://splunkbase.splunk.com/ Splunk Community Sessions User Groups: https://usergroups.splunk.com/ Events: https://www.splunk.com/en_us/about- us/events.html Live: https://www.splunk.com/en_us/events/splunklive.html .Conf: https://conf.splunk.com/watch/conf-online.html Splunk queries Answers: https://commmunity.splunk.com/ Slack Splunk User Group: http://splk.it/slack Support & Services: https://www.splunk.com/en_us/support-and-services.html Enhancement Requests Ideas: https://ideas.splunk.com/
  • 7. ? 2020 SPLUNK INC. Session 2
  • 8. Splunk Custom Visualization API Niket Nilay Sales Engineer (Splunk) Splunk Trust MVP 2018,2019,2020 Splunk Answers karma leader with over 50K points
  • 9. ? 2020 SPLUNK INC. Q&A Raise hand to be unmuted Post questions in WebEx Chat Join Slack for Q&A http://splk.it/slack
  • 10. ? 2020 SPLUNK INC. Community Resources Splunk Learning & Development Docs: https://docs.splunk.com/ Blogs: https://docs.splunk.com/ Dev: https://docs.splunk.com/ Lantern: https://lantern.splunk.com/hc/en-us Education: https://education.splunk.com/free Github: https://github.com/splunk Splunkbase: https://splunkbase.splunk.com/ Splunk Community Sessions User Groups: https://usergroups.splunk.com/ Events: https://www.splunk.com/en_us/about- us/events.html Live: https://www.splunk.com/en_us/events/splunklive.html .Conf: https://conf.splunk.com/watch/conf-online.html Splunk queries Answers: https://commmunity.splunk.com/ Slack Splunk User Group: http://splk.it/slack Support & Services: https://www.splunk.com/en_us/support-and-services.html Enhancement Requests Ideas: https://ideas.splunk.com/
  • 11. We plan to meet 1st Monday of every month at 6:00 PM IST. Please provide feedback for : ? Sessions and improvements. ? Topics to be covered in future sessions. ? Let us know if you are interested in presenting in User Group. Keep the comradery through Slack and Splunk Answers> What’s Next http://splk.it/slack http://community.splunk.com
  • 12. Splunk Add-On Builder Sandeep Nayak Sr. Sales Engineer (Splunk)
  • 13. During the course of this presentation, we may make forward-looking statements regarding future events or the expected performance of the company. We caution you that such statements reflect our current expectations and estimates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward-looking statements, please review our filings with the SEC. The forward-looking statements made in this presentation are being made as of the time and date of its live presentation. If reviewed after its live presentation, this presentation may not contain current or accurate information. We do not assume any obligation to update any forward-looking statements we may make. In addition, any information about our roadmap outlines our general product direction and is subject to change at any time without notice. It is for informational purposes only and shall not be incorporated into any contract or other commitment. Splunk undertakes no obligation either to develop the features or functionality described or to include any such feature or functionality in a future release. Splunk, Splunk>, Listen to Your Data, The Engine for Machine Data, Splunk Cloud, Splunk Light and SPL are trademarks and registered trademarks of Splunk Inc. in the United States and other countries. All other brand names, product names, or trademarks belong to their respective owners. ? 2019 Splunk Inc. All rights reserved. Forward- Looking Statements
  • 14. Splunk The only “Data to Everything” Platform Any Data Structure Any Source Any Timescale Any Insight Any Action Apply data across IT, security, application development, and business questions, decisions, and actions. Any Insight
  • 15. Splunk supports a variety of data sources Security Outside the Datacenter Applications Email server Web server App server Log4J, JMS, JMX Networking Routers SNMP Netflow configurations Databases Configurations Audit/query logs Tables Schemas Virtualization & Cloud Hypervisor Guest OS, Apps Cloud Linux/Unix Configurations syslog File system ps, iostat, top Windows Registry Event logs File system sysinternals Logfiles Configs Messages Traps Alerts Metrics Scripts TicketsChanges Authentication Firewall Intrusion prevention Anti-malware Data Loss prevention Vulnerability scanner Manufacturing, logistics… Power consumption RFID data GPS data
  • 16. ? The Splunk Add-on Builder is a Splunk app that helps you build and validate Technology add-ons (TAs) for your Splunk Enterprise deployment. ? The goals of the Splunk Add-on Builder are to: ? Guide you through all of the necessary steps of creating an add-on ? Build alert actions and adaptive response actions for Splunk Enterprise Security ? Reduce development and testing time ? Follow best practices and naming conventions ? Maintain CIM compliance ? Maintain quality of add-ons ? Validate and test the add-on, helping you to check for certification readiness and to identify any limitations such as compatibilities and dependencies ? Maintain a consistent look and feel while still making it easy for you to add branding Splunk Add On builder https://splunkbase.splunk.com/app/2962/
  • 17. Add On Builder Easily create add-ons: ? To collect data ? To create alert actions ? Map to Data Models ? Field Extractions ? Validate and Package for distribution Screenshot from Splunk Dashboard Examples App
  • 18. Demo
  • 19. Add-On Builder Steps Splunk Docs : https://docs.splunk.com/Documenta tion/AddonBuilder/latest/UserGuide/ Overview
  • 20. Add-On Builder Steps ** Latest d3 installed instead of specific 3.5 version. Currently 5.16.0 Configure Data Collection: ? REST API Call ? Run a Shell command ? Python Code
  • 21. Add-On Builder Steps Create the Setup Page for the Add-On 1. Configure Server URL to connect to 2. Configure credentials to use 3. This would all be setup as the main configuration for the app
  • 22. Add-On Builder Steps Create the Inputs 1. Configure what information to query if the REST API supports passing parameters.
  • 23. Add-On Builder Steps Test and Debug your Script/REST API 1. Interface to check outputs and event creation 2. Python basic error outputs
  • 24. Add-On Builder Steps Field Extractions: 1. Guided mechanism to extract fields. 2. Add-on builder will create transforms.conf for you 3. Add sample data to test
  • 25. Add-On Builder Steps Map to the Data Model 1. Very useful when mapping data to fit an existing data model like CIM or ITSI data models 2. Provides mechanisms to map to the data model and create several Eval conditions
  • 26. Add-On Builder Steps Validate and package the app 1. Makes the app ready for certification! 2. Package and Download the app for distribution
  • 27. Further Resources https://splunkbase.splunk.com/app/2962/ Some good online tutorials available here: ? http://dev.splunk.com/view/addon-builder/SP-CAAAE83 ? http://dev.splunk.com/view/addon-builder/SP-CAAAE9F ? https://www.splunk.com/blog/2016/04/04/building-add-ons- has-never-been-easier/ ? https://www.splunk.com/blog/2016/10/12/building-add-ons- just-got-2-0-times-easier.html ? https://www.splunk.com/blog/2016/10/24/creating-mcafee- epo-alert-and-arf-actions-with-add-on-builder/ ? Splunk .Conf Sessions ? Splunk Blogs ? Splunk Dev ? Splunk User Groups on Slack Join #add-on-builder channel ? Splunk Community
  • 29. Splunk Custom Visualization API Niket Nilay Sales Engineer (Splunk)
  • 30. During the course of this presentation, we may make forward-looking statements regarding future events or the expected performance of the company. We caution you that such statements reflect our current expectations and estimates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward-looking statements, please review our filings with the SEC. The forward-looking statements made in this presentation are being made as of the time and date of its live presentation. If reviewed after its live presentation, this presentation may not contain current or accurate information. We do not assume any obligation to update any forward-looking statements we may make. In addition, any information about our roadmap outlines our general product direction and is subject to change at any time without notice. It is for informational purposes only and shall not be incorporated into any contract or other commitment. Splunk undertakes no obligation either to develop the features or functionality described or to include any such feature or functionality in a future release. Splunk, Splunk>, Listen to Your Data, The Engine for Machine Data, Splunk Cloud, Splunk Light and SPL are trademarks and registered trademarks of Splunk Inc. in the United States and other countries. All other brand names, product names, or trademarks belong to their respective owners. ? 2019 Splunk Inc. All rights reserved. Forward- Looking Statements
  • 31. Splunk The only “Data to Everything” Platform Any Data Structure Any Source Any Timescale Any Insight Any Action Apply data across IT, security, application development, and business questions, decisions, and actions. Any Insight
  • 32. Built-in Visualizations Core visualizations Line, Area, Column, Bar, Pie, Scatter, Bubble, Single Value, Radial Gauge, Filler Gauge, Marker Gauge, Cluster Map, Choropleth Pros ?Customization through UI and Simple XML. ?Supports drilldown and trellis. ?Extension through CSS and JS. ?Available for Connected Experience – Mobile, TV, AR, VR. ?Available in new Dashboard
  • 33. ? Looking Good ≠ Easily Comprehendible ? Each chart solves different purpose depending on correlation and use case ? Know power of each visualization as well as their weaknesses ? Reduce chart-junk and increase data-to-ink ratio in dashboard ? While configuring follow precedence – UI > SimpleXML > CSS Extension > SplunkJS ? Built in visualization features are extended with every release. .Conf 2019 Talk – IT1523 : The Top 10 Glasstable Design Principles to Boost your Career and your Business Splunk Built-in Visualizations “Not all charts are created equal”
  • 34. 1. Get Custom Visualizations from Splunkbase ? Built on top of Splunk Custom Visualization API. Both Splunk built and Community built. ? May not support Drilldown or Trellis out of the box. 2. Covert Simple XML to HTML ? Allows importing and integrating JS based custom visualization as in any HTML app with JS scripting. ? May loose several features and ease of Simple XML. 3. Simple XML JS and CSS Extension ? Similar as above but allows to retain Simple XML features, ease of configuration and editing. ? No framework to integrate and may loose reusability, configuration and extensions to the visualization. 4. Build yourself using Custom Visualization API ? Use Splunk Custom Visualization API to build custom visualization based on preferred viz and use case. ? Requires development time as any other chart library integration would take. 5. Build Custom visualization using New Dashboard Framework Custom Visualizations Oooooo shiny visualization... How do I get it in Splunk?
  • 35. Custom Visualizations Custom Visualizations Visualizations based on JS Libraries like D3 imported to Splunk. Pros ?Ability to import any visualization. ?Extension through CSS and JS. ?Splunk built and community built. Cons ?May not support drilldown and trellis. ?Not available for Connected Experience. ?May not work with new Dashboard Screenshot from Splunk Dashboard Examples App
  • 37. Visualization Libraries Custom Visualizations on Splunkbase Choose from famous Charting Libraries ? D3 JS ? Highcharts ? Chart.js ? … And many more.
  • 38. App directory and file structure appname appserver static visualizations <visualization_name> src <visualization_source>.js webpack.config.js visualization.js visualization.css formatter.html package.json preview.png default visualizations.conf savedsearches.conf metadata default.meta README savedsearches.conf.spec static appIcon.png, appIconAlt.png, appLogo.png appIcon_2x.png, appIconAlt_2x.png, appLogo_2x.png Development mode settings For working with static files in development mode without bump/refresh/restart through “web.conf” configuration file in etc/system/local. npm package file. Webpack configuration file SplunkVisualizationBase must override the following: getInitialDataParams – Data handling updateView – Visualization handling SplunkVisualizationUtils from security requirement. <splunk-control-group> <splunk-select> <splunk-radio-input> <option> <splunk-text-area> <splunk-text-input> <splunk-color-picker> <splunk-color> Custom Visualization API Splunk Docs – Custom Visualizations PNG image file used in the Visualization Picker user interface Settings for loading the visualization and making it available in Splunk Web (Including Drilldown and Trellis options) visualization property default values export visualizations to the system declare visualization properties static images for App logos
  • 39. Demo
  • 40. Build a Custom Visualization Tutorial https://docs.splunk.com/Documentation/Splunk/latest/AdvancedDev/CustomVizTu torial ? Windows*** steps may vary like setting environment variables ? Some steps with latest version of the dependent app would need correction. ? For ex: Vulnerability fix for jQuery requires latest version. ? package.json changes to support modules instead of root and change in path. ? d3 function changes for selected sample viz in <visualization_source>.js. ? The sample App covers basic structure for learning and not complete end-to-end feature development. ? We will add drilldown, trellis, tooltip and dark mode on top of the tutorial app, add custom App Icon, Visualization Icon. ? Package App and validate through Splunk App Inspection before publishing on Splunkbase. *** While Windows config is not covered in the session or directly in the Build a Custom Visualization Splunk documentation, it is possible. Splunk community resources https://docs.splunk.com, https://community.splunk.com would be right place to find the details. Unless already on #splunk_bengaluru_user_group Slack channel in the Splunk User group on Slack, use the following URL to get access and contact us for discussion and assistance: http://splk.it/slack
  • 41. Custom Visualization Build Steps Splunk Docs Build a Custom Visualization: https://docs.splunk.com/Documenta tion/Splunk/latest/AdvancedDev/Cu stomVizTutorial 1. Download the custom visualization app template. 2. Unzip and rename the template as my_radial_meter. 3. Copy the template app to $SPLUNK_HOME/etc/apps/ folder. Set SPLUNK_HOME environment variable (steps are different for PC and Mac). 4. Rename $SPLUNK_HOME/etc/apps/my_radial_meter/appserver/st atic/visualizations/standin folder to my_radial_meter 5. Navigate to $SPLUNK_HOME/etc/apps/my_radial_meter/appserver/st atic/visualizations/my_radial_meter 6. Rename visualization_source.js as my_radial_meter_source.js
  • 42. Custom Visualization Build Steps 7. npm install 8. sudo npm install -g npm 9. npm install --save d3 10.npm install --save underscore 11.npm install --save jquery 12.npm audit ? Fix Issues found during auditing 13.npm install jquery@3.5.1 14.npm install --save-dev webpack@4.43.0 ? Ensure no audit issues/vulnerabilities ** Latest d3 installed instead of specific 3.5 version. Currently 5.16.0
  • 43. package.json 15.Modify package.json Following changes are made: 1. name changed to my_radial_meter 2. description added 3. main changed to my_radial_meter,js
  • 44. webpack.config.js 16.Modify webpack.config.js Following changes are made: 1. Entry changed to my_radial_meter_source.js 2. root changed to modules 3. Path to source and node modules changed 4. Output path changed 5. Output filename changed to my_radial_meter,js
  • 46. Visualization Source JS Bare-metal Setup Code 18.Get bare-minimum Visualization Source in the my_radial_meter_source.js file. https://docs.splunk.com/Documentation/Splunk/latest/AdvancedDev/CustomVizTutorial#Set_up_the_visualization_source_code
  • 47. Bare Minimal Update View Code 19.Get updateView code. https://docs.splunk.com/Documentation/Splunk/latest/AdvancedDev/CustomVizTutorial#Add_the_updateView_function Following changes are made: 1. Change d3.scale.linear() with d3.scaleLinear() 2. Change d3.svg.arc() with d3.arc() 3. Change data handle in update view // Return if no data if (!data) { return; }
  • 51. Handle Data Format Errors 23. Validates and processes raw data, through formatData section. Get raw data object from splunkd and returns an object formatted for rendering Validate and Process data ? Check for empty data ? Check for invalid data ? Throw helpful errors ? Sanitize/secure data through utilities like escapeHtml() and makeSafeURL()
  • 52. Build bare minimal 24. Run npm run build and choose yes for webpack-cli install.
  • 53. Validate viz in UI 25. Basic viz does not have: Drilldown, Tooltip, Trellis, Dark Theme and Icons.
  • 54. Set Icons for Visualization 26. Application Icons ? Place icon preview.png under folder <app_name>/appserver/static/visualizations/<viz_name> ? Place PNG icons appIcon, appIconAlt and appLogo and corresponding _2X png icons under folder <app_name>/static Validate Icons: 1. Splunk App Icon, 2. Alternate Icon, 3. Dashboard App Bar Icon, 4. Visualization picker icon
  • 55. Enable Viz Formatting 27. Add formatting options in savedsearches.conf, savedsearches.conf.spec and formatter.html Visualization source JS file uses getPropertyNamespaceInfo() for reading viz properties (inside updateView() method).
  • 56. Enable Trellis 28. Trellis Mode configuration supports_trellis = True Needs to be set in visualizations.conf as per previous step. May or may not work for all Custom visualizations.
  • 57. Enable Drilldown 29. Trellis Mode configuration supports_drilldown = True Needs to be set in visualizations.conf as per previous step.
  • 58. Enable Tooltip 30. Change visualization source JS file to handle tooltip.
  • 59. Dark Mode 31. Dark Mode configuration In our example background color for svg will be turned transparent (instead of White color provided in the template), so that it picks panel color. Dark Mode used for Tooltip.
  • 60. Final Output 32. Rebuild and validate the viz.
  • 61. Further Resources Package app using Splunk Packaging Toolkit Validate App quality using Splunk App Inspect FN1735 – What’s next for Geo in Splunk DEV1141 – Building Applications with Splunk UI and Splunk React Visualizations DEV2178 – Building Your Own Custom Data Visualization for Dashboards? Splunk .Conf Sessions ? Splunk Blogs ? Splunk Dev ? Splunk User Groups on Slack Join #visualizations channel ? Splunk Community
  • 63. Template Code and Build Steps For Demo Link to GitHub repository for demo template: https://github.com/niketnilay/bengal uru_user_group/tree/master/2020- 07-06/my_radial_meter UPDATED BUILD STEPS for my_radial_meter_template 1. Extract my_radial_meter_template.zip to my_radial_meter_template folder. 2. export SPLUNK_HOME=/Applications/Splunk (MAC/*nix) set SPLUNK_HOME=<path> (on Windows) 3. Copy my_radial_meter folder to your $SPLUNK_HOME/etc/apps/ folder. 4. Navigate to $SPLUNK_HOME/etc/apps/my_radial_meter/appserver/static/visualizations/my _radial_meter [ONLY FOR WINDOWS] Get package.json file and update in above folder. PS: This is for path $SPLUNK_HOME to be used as %SPLUNK_HOME% in Windows. For Mac with Xcode Editor, to avoid following npm install build error (try step 5 before step 6) build error gyp ERR! stack Error: `make` failed with exit code: 69
  • 64. Template Code and Build Steps For Demo Link to GitHub repository for demo template: https://github.com/niketnilay/bengal uru_user_group/tree/master/2020- 07-06/my_radial_meter Continued… 5. sudo xcodebuild -license (for Mac with XCode) Hit Enter (to open license agreement) Press q (quit) Type agree (to agree to the license) 6. sudo npm install 7. npm install --save d3 8. npm install --save underscore 9. npm install --save jquery 10. npm run build (Splunk Restart and Browser history cleanup may be required) 11. Validate the visualization in UI 12. Get my_radial_meter_source.js updated code with Drilldown, Tooltip, Dark Mode and replace the content of existing JS from Github or Google Drive. 13. Build code and validate UI (Splunk Restart and Browser history cleanup may be required)