This commit is contained in:
Markus Cozowicz 2018-11-19 22:03:30 +01:00
Родитель 73e07c3fc3 b322e4b7f0
Коммит e42442230e
1 изменённых файлов: 9 добавлений и 4 удалений

Просмотреть файл

@ -105,14 +105,13 @@ spark.sql.crossJoin.enabled true
1. Navigate to your workspace 1. Navigate to your workspace
2. Create library 2. Create library
3. Under 'Source' select 'Maven Coordinate' 3. Under 'Source' select 'Maven Coordinate'
4. Enter 'eisber:sarplus:0.2.2' 4. Enter 'eisber:sarplus:0.2.4'
5. Hit 'Create Library' 5. Hit 'Create Library'
6. Attach to your cluster 6. Attach to your cluster
7. Create 2nd library 7. Create 2nd library
8. Under 'Source' select 'Upload Python Egg or PyPI' 8. Under 'Source' select 'Upload Python Egg or PyPI'
9. Enter 'pysarplus' 9. Enter 'pysarplus'
10. Hit 'Create Library' 10. Hit 'Create Library'
11. Enter 'pysarplus'
This will install C++, Python and Scala code on your cluster. This will install C++, Python and Scala code on your cluster.
@ -135,11 +134,17 @@ You'll also have to mount shared storage
```pyspark ```pyspark
dbutils.fs.mount( dbutils.fs.mount(
source = "wasbs://sarpluscache@<yourcontainer>.blob.core.windows.net", source = "wasbs://sarpluscache@<accountname>.blob.core.windows.net",
mount_point = "/mnt/sarpluscache", mount_point = "/mnt/sarpluscache",
extra_configs = {"fs.azure.account.key.<yourcontainer>.blob.core.windows.net":dbutils.secrets.get(scope = "all", key = "sarpluscache")}) extra_configs = {"fs.azure.account.key.<accountname>.blob.core.windows.net":dbutils.secrets.get(scope = "all", key = "sarpluscache")})
``` ```
Disable annoying logging
```pyspark
import logging
logging.getLogger("py4j").setLevel(logging.ERROR)
```
# Packaging # Packaging