This commit is contained in:
Markus Cozowicz 2018-11-19 22:03:30 +01:00
Родитель 73e07c3fc3 b322e4b7f0
Коммит e42442230e
1 изменённых файлов: 9 добавлений и 4 удалений

Просмотреть файл

@ -105,14 +105,13 @@ spark.sql.crossJoin.enabled true
1. Navigate to your workspace
2. Create library
3. Under 'Source' select 'Maven Coordinate'
4. Enter 'eisber:sarplus:0.2.2'
4. Enter 'eisber:sarplus:0.2.4'
5. Hit 'Create Library'
6. Attach to your cluster
7. Create 2nd library
8. Under 'Source' select 'Upload Python Egg or PyPI'
9. Enter 'pysarplus'
10. Hit 'Create Library'
11. Enter 'pysarplus'
This will install C++, Python and Scala code on your cluster.
@ -135,11 +134,17 @@ You'll also have to mount shared storage
```pyspark
dbutils.fs.mount(
source = "wasbs://sarpluscache@<yourcontainer>.blob.core.windows.net",
source = "wasbs://sarpluscache@<accountname>.blob.core.windows.net",
mount_point = "/mnt/sarpluscache",
extra_configs = {"fs.azure.account.key.<yourcontainer>.blob.core.windows.net":dbutils.secrets.get(scope = "all", key = "sarpluscache")})
extra_configs = {"fs.azure.account.key.<accountname>.blob.core.windows.net":dbutils.secrets.get(scope = "all", key = "sarpluscache")})
```
Disable annoying logging
```pyspark
import logging
logging.getLogger("py4j").setLevel(logging.ERROR)
```
# Packaging