![30Page:
Notebook Display and Charting Code Snippets
> filter(got)
> val got = sqlContext.sql("select * from got")
> got.limit(10).collect()
> import org.apache.spark.sql.functions._
> val allegiancesCleanupUDF = udf[String, String]
(_.toLowerCase().replace("house ", ""))
> val isDeathUDF = udf{ deathYear: Integer => if(deathYear != null) 1 else 0}
> val gotCleaned = got.filter("Allegiances !=
"None"").withColumn("Allegiances",
allegiancesCleanupUDF($"Allegiances")).withColumn("isDeath",
isDeathUDF($"Death Year"))
> display(gotCleaned)](https://image.slidesharecdn.com/databricks-community-cloud-161009203811/85/Databricks-Community-Cloud-Overview-30-320.jpg)
Databricks Community Cloud Overview
- 1. Databricks Community Cloud By: Robert Sanders
- 2. 2Page: Databricks Community Cloud Free/Paid Standalone Spark Cluster Online Notebook Python R Scala SQL Tutorials and Guides Shareable Notebooks
- 3. 3Page: Why is it useful? Learning about Spark Testing different versions of Spark Rapid Prototyping Data Analysis Saved Code Others
- 8. 8Page: Create a Cluster - Steps 1. From the Active Clusters page, click the + Create Cluster button 2. Fill in the cluster name 3. Select the version of Apache Spark 4. Click Create Cluster 5. Wait for the Cluster to start up and be in a Running state
- 11. 11Page: Active Clusters Spark Cluster UI - Master
- 13. 13Page: Create a Notebook - Steps 1. Right click within a Workspace and click Create -> Notebook 2. Fill in the Name 3. Select the programming language 4. Select the running cluster youve created that you want to attach to the Notebook 5. Click the Create button
- 15. 15Page: Notebook
- 17. 17Page: Using the Notebook Code Snippets > sc > sc.parallelize(1 to 5).collect()
- 18. 18Page: Using the Notebook - Shortcuts Short Cut Action Shift + Enter Run Selected Cell and Move to next Cell Ctrl + Enter Run Selected Cell Option + Enter Run Selected Cell and Insert Cell Bellow Ctrl + Alt + P Create Cell Above Current Cell Ctrl + Alt + N Create Cell Bellow Selected Cell
- 19. 19Page: Tables
- 20. 20Page: Create a Table - Steps 1. From the Tables section, click + Create Table 2. Select the Data Source (bellow steps assume youre using File as the Data Source) 3. Upload a file from your local file system 1. Supported file types: CSV, JSON, Avro, Parquet 4. Click Preview Table 5. Fill in the Table Name 6. Select the File Type and other Options depending on the File Type 7. Change Column Names and Types as desired 8. Click Create Table
- 21. 21Page: Create a Table Upload File
- 22. 22Page: Create a Table Configure Table
- 23. 23Page: Create a Table Review Table
- 24. 24Page: Notebook Access Table
- 25. 25Page: Notebook Access Table Code Snippets > sqlContext > sqlContext.sql("show tables").collect() > val got = sqlContext.sql("select * from got") > got.limit(10).collect()
- 27. 27Page: Notebook Data Cleaning for Charting
- 28. 28Page: Notebook Plot Options
- 30. 30Page: Notebook Display and Charting Code Snippets > filter(got) > val got = sqlContext.sql("select * from got") > got.limit(10).collect() > import org.apache.spark.sql.functions._ > val allegiancesCleanupUDF = udf[String, String] (_.toLowerCase().replace("house ", "")) > val isDeathUDF = udf{ deathYear: Integer => if(deathYear != null) 1 else 0} > val gotCleaned = got.filter("Allegiances != "None"").withColumn("Allegiances", allegiancesCleanupUDF($"Allegiances")).withColumn("isDeath", isDeathUDF($"Death Year")) > display(gotCleaned)
- 31. 31Page: Publish Notebook - Steps 1. While in a Notebook, click Publish on the top right 2. Click Publish on the pop up 3. Copy the link and send it out