This commit is contained in:
yueguoguo 2018-11-01 07:43:26 +00:00
Родитель 31d7e58cdc
Коммит 8645513540
1 изменённых файлов: 8 добавлений и 8 удалений

Просмотреть файл

@ -2,7 +2,8 @@
In this guide we show how to setup all the dependencies to run the notebooks of this repo.
We have three different environments for our algorithms:
Three environments are supported to run the notebooks in the repo:
* Python CPU
* Python GPU
* PySpark
@ -10,9 +11,9 @@ We have three different environments for our algorithms:
## Requirements
- [Anaconda Python 3.6](https://conda.io/miniconda.html)
- The library dependencies are generated depending on the environment we choose and can be found in this [script](scripts/generate_conda_file.sh).
- Machine with GPU (optional).
- Machine with Spark (optional).
- The Python library dependencies can be found in this [script](scripts/generate_conda_file.sh).
- Machine with Spark (optional for Python environment but mandatory for PySpark environment).
- Machine with GPU (optional but desirable for computing acceleration).
## Conda environments
@ -22,11 +23,11 @@ As a pre-requisite, we may want to make sure that Conda is up-to-date:
We provided a script to [generate a conda file](scripts/generate_conda_file.sh), depending of the environment we want to use.
To install each environment, first we need to generate a conda yml file and then install the environment. We can choose the environment name with the input `-n`. In the following examples, we provide a name example.
To install each environment, first we need to generate a conda yml file and then install the environment. We can specify the environment name with the input `-n`. In the following examples, we provide a name example.
### Python CPU environment
To install the Python CPU environment:
Assuming the repo is cloned as `Recommenders` in the local system, to install the Python CPU environment:
cd Recommenders
./scripts/generate_conda_file.sh
@ -46,10 +47,9 @@ To install the PySpark environment, which by default installs the CPU environmen
cd Recommenders
./scripts/generate_conda_file.sh --pyspark
conda env create -n reco_pyspark -f conda_pyspark.yaml
For this environment, we need to set the environment variables `PYSPARK_PYTHON` and `PYSPARK_DRIVER_PYTHON` to point to the conda python executable.
**NOTE** for this environment, we need to set the environment variables `PYSPARK_PYTHON` and `PYSPARK_DRIVER_PYTHON` to point to the conda python executable.
For setting these variables every time the environment is activated, we can follow the steps of this [guide](https://conda.io/docs/user-guide/tasks/manage-environments.html#macos-and-linux). Assuming that we have installed the environment in `/anaconda/envs/reco_pyspark`, we create the file `/anaconda/envs/reco_pyspark/activate.d/env_vars.sh` and add: