Add section about venv in README

This commit is contained in:
Andreas Argyriou 2021-07-12 09:33:56 +00:00
Родитель a1529e57a9
Коммит b39362fe61
1 изменённых файлов: 32 добавлений и 0 удалений

Просмотреть файл

@ -14,6 +14,7 @@ This document describes how to setup all the dependencies to run the notebooks i
- [Requirements](#requirements)
- [Dependencies setup](#dependencies-setup)
- [Register the environment as a kernel in Jupyter](#register-the-environment-as-a-kernel-in-jupyter)
- [Using a virtual environment](#using-a-virtual-environment)
- [Troubleshooting for the DSVM](#troubleshooting-for-the-dsvm)
- [Setup guide for Azure Databricks](#setup-guide-for-azure-databricks)
- [Requirements of Azure Databricks](#requirements-1)
@ -148,6 +149,37 @@ We can register our conda or virtual environment to appear as a kernel in the Ju
If you are using the DSVM, you can [connect to JupyterHub](https://docs.microsoft.com/en-us/azure/machine-learning/data-science-virtual-machine/dsvm-ubuntu-intro#jupyterhub-and-jupyterlab) by browsing to `https://your-vm-ip:8000`.
### Using a virtual environment
It is straightforward to install the recommenders package within a [virtual environment](https://docs.python.org/3/library/venv.html). However, setting up CUDA for use with a GPU can be cumbersome. We thus
recommend running the virtual environment within an Nvidia docker container as the most convenient way to do this.
sudo nvidia-docker run -v /usr:/host_usr -it 2d6e4efb9d8d
Within container:
apt-get -y update
apt-get -y install python3.7
apt-get -y install python3-pip
apt-get -y install python3.7-venv
apt-get -y install libpython3.7-dev
python3.7 -m venv --system-site-packages /venv
export JAVA_HOME=/host_usr/lib/jvm/java-8-openjdk-amd64
export PATH=$PATH:/host_usr/local/bin
source /venv/bin/activate
pip install --upgrade pip
pip install --upgrade setuptools
export PYSPARK_DRIVER_PYTHON=/venv/bin/python
export PYSPARK_PYTHON=/venv/bin/python
pip install ms-recommenders
### Troubleshooting for the DSVM
* We found that there can be problems if the Spark version of the machine is not the same as the one in the [conda file](conda.md). You can use the option `--pyspark-version` to address this issue.