зеркало из https://github.com/microsoft/sarplus.git
Merge branch 'master' of https://github.com/eisber/sarplus
This commit is contained in:
Коммит
e42442230e
13
README.md
13
README.md
|
@ -105,14 +105,13 @@ spark.sql.crossJoin.enabled true
|
||||||
1. Navigate to your workspace
|
1. Navigate to your workspace
|
||||||
2. Create library
|
2. Create library
|
||||||
3. Under 'Source' select 'Maven Coordinate'
|
3. Under 'Source' select 'Maven Coordinate'
|
||||||
4. Enter 'eisber:sarplus:0.2.2'
|
4. Enter 'eisber:sarplus:0.2.4'
|
||||||
5. Hit 'Create Library'
|
5. Hit 'Create Library'
|
||||||
6. Attach to your cluster
|
6. Attach to your cluster
|
||||||
7. Create 2nd library
|
7. Create 2nd library
|
||||||
8. Under 'Source' select 'Upload Python Egg or PyPI'
|
8. Under 'Source' select 'Upload Python Egg or PyPI'
|
||||||
9. Enter 'pysarplus'
|
9. Enter 'pysarplus'
|
||||||
10. Hit 'Create Library'
|
10. Hit 'Create Library'
|
||||||
11. Enter 'pysarplus'
|
|
||||||
|
|
||||||
This will install C++, Python and Scala code on your cluster.
|
This will install C++, Python and Scala code on your cluster.
|
||||||
|
|
||||||
|
@ -135,11 +134,17 @@ You'll also have to mount shared storage
|
||||||
|
|
||||||
```pyspark
|
```pyspark
|
||||||
dbutils.fs.mount(
|
dbutils.fs.mount(
|
||||||
source = "wasbs://sarpluscache@<yourcontainer>.blob.core.windows.net",
|
source = "wasbs://sarpluscache@<accountname>.blob.core.windows.net",
|
||||||
mount_point = "/mnt/sarpluscache",
|
mount_point = "/mnt/sarpluscache",
|
||||||
extra_configs = {"fs.azure.account.key.<yourcontainer>.blob.core.windows.net":dbutils.secrets.get(scope = "all", key = "sarpluscache")})
|
extra_configs = {"fs.azure.account.key.<accountname>.blob.core.windows.net":dbutils.secrets.get(scope = "all", key = "sarpluscache")})
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Disable annoying logging
|
||||||
|
|
||||||
|
```pyspark
|
||||||
|
import logging
|
||||||
|
logging.getLogger("py4j").setLevel(logging.ERROR)
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
# Packaging
|
# Packaging
|
||||||
|
|
Загрузка…
Ссылка в новой задаче