Adding sphinx docs (#29)

* Adding sphinx docs
* Updating readme

.readthedocs.yaml

@@ -0,0 +1,13 @@
+version: 2
+formats: all
+build:
+  image: stable
+python:
+  version: 3.8
+  install:
+    - requirements: docs/requirements.txt
+    - method: pip
+      path: .
+sphinx:
+  builder: html
+  configuration: docs/source/conf.py

README.md

@@ -24,18 +24,24 @@
 
 The Python SDK would allow you to the following:
 
-- Parse Synthetic datasets generated with Unity.
-- SOLO support out of the box
-- Dataset iterables & converters.
+- Parse SOLO datasets generated with Unity Perception.
+- Dataset iterables.
+- Clients to access remote datasets
+  - Unity Computer Vision Datasets (UCVD)
 
 ## Pre-Requisites
-- Install [protoc](http://google.github.io/proto-lens/installing-protoc.html)
 - Install [Anaconda](https://docs.anaconda.com/anaconda/install/) or [Miniconda](https://docs.conda.io/en/latest/miniconda.html) (recommended). Install [pre-commit](https://pre-commit.com/).
 - Make sure `pip` is installed.
-- Create your API key in the Unity Dashboard. [Instructions]()
-- Save your API key somewhere safe, it will be used later.
 
-N.B. We use `proto3` and do not support `proto2` messages.
+## Using Cloud datasets from UCVD
+- Create a Unity account if you don't at [Unity Services](https://dashboard.unity3d.com/)
+- Navigate to Projects and create a project if you don't have one.
+- Navigate to Projects>Service Account in the dashboard to create a new one.
+![Service Account](images/sa.png)
+- Generate your API key.
+- Save your API key somewhere safe, it will be used later.
+- Navigate to Projects>Select the project to use and record the `Project ID`.
+- Navigate to Projects>Orgaization Settings and record the `Organization ID`.
 
 
 ## Installation
@@ -48,13 +54,21 @@ pip install unity_vision --index-url=https://artifactory.prd.it.unity3d.com/arti
 
 ** The package lives in the internal PyPi repo for now, so to install please follow -
 
-## Authentication
 
----
 
-There are multiple ways to generate, and access Unity generated synthetic datasets -
+#### SOLO Dataset
 
-### Generated in Unity Computer Vision Dataset service
+##### Local Datasets
+
+```python
+from unity_vision.consumers.solo.parser import Solo
+
+solo = Solo(path="<<dataset-path>>")
+```
+
+##### Cloud Datasets
+
+Generated in Unity Computer Vision Dataset service
 
 ```python
 from unity_vision.clients import UCVDClient
@@ -64,30 +78,15 @@ client = UCVDClient(
     sa_key="sa-key",
     api_secret="api-secret"
 )
+
+client.download_dataset_archives("<<dataset-id>>")
+
 ```
 
-<!--
+### Data Models
 
-### Generated locally with Unity Perception package
+Please refer - [models](unity_vision/core/models.py) for the data models used by SOLO.
 
-```python
-from unity_vision.clients import LocalClient
-client = LocalClient(
-    data_path="dataset-path"
-)
-``` -->
-
-<!-- [//]: # (TODO: Should this be in documentation ?)
-### Consume datasets working with Unity -->
-
-
-
-## Dataset Formats
-
-### SOLO
-
-SOLO is a dataset format being from Unity for synthetic datasets.
-Refer - [Protobuf](unity-vision/protos/solo.proto)
 
 ## Additional Resources
 

docs/Makefile

@@ -0,0 +1,29 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SPHINXAPIDOC  ?= sphinx-apidoc
+SOURCEDIR     = source
+BUILDDIR      = build
+
+.PHONY: help clean apidoc html
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+
+clean:
+	rm -rf $(BUILDDIR)
+
+apidoc:
+	sphinx-apidoc -o "$(SOURCEDIR)" -d 4 "../unity_vision/"
+
+
+html:
+	sphinx-build -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+	@echo
+	@echo "Build finished. The HTML doumentation pages are in $(BUILDDIR)/html."

docs/make.bat

@@ -0,0 +1,35 @@
+@ECHO OFF
+
+pushd %~dp0
+
+REM Command file for Sphinx documentation
+
+if "%SPHINXBUILD%" == "" (
+	set SPHINXBUILD=sphinx-build
+)
+set SOURCEDIR=source
+set BUILDDIR=build
+
+%SPHINXBUILD% >NUL 2>NUL
+if errorlevel 9009 (
+	echo.
+	echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
+	echo.installed, then set the SPHINXBUILD environment variable to point
+	echo.to the full path of the 'sphinx-build' executable. Alternatively you
+	echo.may add the Sphinx directory to PATH.
+	echo.
+	echo.If you don't have Sphinx installed, grab it from
+	echo.https://www.sphinx-doc.org/
+	exit /b 1
+)
+
+if "%1" == "" goto help
+
+%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+goto end
+
+:help
+%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+
+:end
+popd

docs/source/conf.py

@@ -0,0 +1,80 @@
+# Configuration file for the Sphinx documentation builder.
+#
+# This file only contains a selection of the most common options. For a full
+# list see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Path setup --------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#
+import os
+import sys
+
+import pkg_resources
+
+sys.path.insert(0, os.path.abspath('../..'))
+
+
+# -- Project information -----------------------------------------------------
+
+project = 'unity_vision'
+copyright = '2022, Unity Technologies'
+author = 'Unity Technologies'
+
+
+# The full version, including alpha/beta/rc tags
+release = pkg_resources.get_distribution(project).version
+napoleon_google_docstring = True
+
+
+# -- General configuration ---------------------------------------------------
+
+master_doc = "index"
+
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [
+    'sphinx.ext.autodoc',
+    'sphinx.ext.coverage',
+    'sphinx.ext.napoleon',
+    "sphinx_rtd_theme",
+    "sphinx.ext.autosectionlabel"
+]
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = []
+
+source_suffix = {
+    ".rst": "restructuredtext",
+    ".txt": "markdown",
+    ".md": "markdown",
+}
+
+
+# autodoc settings
+autodoc_default_options = {
+    'members': True,
+    'undoc-members': True,
+}
+
+# -- Options for HTML output -------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+#
+html_theme = "sphinx_rtd_theme"
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']

docs/source/index.rst

@@ -0,0 +1,27 @@
+Unity Vision
+============
+
+Unity Vision is a python package for operating on Unity Synthetic
+datasets.
+
+Installation
+============
+
+.. code-block:: bash
+
+   pip install unity_vision --index-url=https://artifactory.prd.it.unity3d.com/artifactory/api/pypi/pypi/simple
+
+** The package lives in the internal PyPi repo for now.
+
+.. toctree::
+   :maxdepth: 4
+
+   unity_vision
+
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`

docs/source/modules.rst

@@ -0,0 +1,7 @@
+unity_vision
+============
+
+.. toctree::
+   :maxdepth: 4
+
+   unity_vision

docs/source/unity_vision.clients.rst

@@ -0,0 +1,21 @@
+unity\_vision.clients package
+=============================
+
+Submodules
+----------
+
+unity\_vision.clients.ucvd\_client module
+-----------------------------------------
+
+.. automodule:: unity_vision.clients.ucvd_client
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+Module contents
+---------------
+
+.. automodule:: unity_vision.clients
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/source/unity_vision.consumers.rst

@@ -0,0 +1,18 @@
+unity\_vision.consumers package
+===============================
+
+Subpackages
+-----------
+
+.. toctree::
+   :maxdepth: 4
+
+   unity_vision.consumers.solo
+
+Module contents
+---------------
+
+.. automodule:: unity_vision.consumers
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/source/unity_vision.consumers.solo.rst

@@ -0,0 +1,29 @@
+unity\_vision.consumers.solo package
+====================================
+
+Submodules
+----------
+
+unity\_vision.consumers.solo.parser module
+------------------------------------------
+
+.. automodule:: unity_vision.consumers.solo.parser
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+unity\_vision.consumers.solo.parser\_pb module
+----------------------------------------------
+
+.. automodule:: unity_vision.consumers.solo.parser_pb
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+Module contents
+---------------
+
+.. automodule:: unity_vision.consumers.solo
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/source/unity_vision.core.rst

@@ -0,0 +1,29 @@
+unity\_vision.core package
+==========================
+
+Submodules
+----------
+
+unity\_vision.core.exceptions module
+------------------------------------
+
+.. automodule:: unity_vision.core.exceptions
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+unity\_vision.core.models module
+--------------------------------
+
+.. automodule:: unity_vision.core.models
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+Module contents
+---------------
+
+.. automodule:: unity_vision.core
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/source/unity_vision.protos.rst

@@ -0,0 +1,21 @@
+unity\_vision.protos package
+============================
+
+Submodules
+----------
+
+unity\_vision.protos.solo\_pb2 module
+-------------------------------------
+
+.. automodule:: unity_vision.protos.solo_pb2
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+Module contents
+---------------
+
+.. automodule:: unity_vision.protos
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/source/unity_vision.rst

@@ -0,0 +1,21 @@
+unity\_vision package
+=====================
+
+Subpackages
+-----------
+
+.. toctree::
+   :maxdepth: 4
+
+   unity_vision.clients
+   unity_vision.consumers
+   unity_vision.core
+   unity_vision.protos
+
+Module contents
+---------------
+
+.. automodule:: unity_vision
+   :members:
+   :undoc-members:
+   :show-inheritance:

requirements.txt

@@ -9,3 +9,5 @@ ratelimit~=2.2.1
 pytest>=6.2.2
 dataclasses-json~=0.5.7
 pandas>=1.2.1
+sphinx-rtd-theme>=1.0
+sphinx>=4.1.2

unity_vision/consumers/solo/parser_pb.py

@@ -1,19 +1,5 @@
 #!/usr/bin/env python
 
-"""
-Provides a base parser to import solo datasets.
-
-Solo:
-    Returns an iterator to read datasets which have 1 frame per sequence. This essentially flattens
-    all the sequences and gives an iterator to loop through all the frames in the dataset along with
-     the corresponding sensors and their annotations.
-
-SoloSequence:
-    Returns iterator to read all steps in a sequence.
-
-"""
-
-
 import glob
 import json
 import time
@@ -154,13 +140,6 @@ class Solo(SoloBase):
         self.path = path
         self.frame_idx = start
 
-        """
-        Default metadata location is expected at root/metadata.json but
-        if an annotation_file path is provided that is used as the annotation
-
-        Metadata can be in one of two locations, depending if it was a part of a singular build,
-        or if it was a part of a distributed build.
-        """
         metadata_f = self.__open_metadata__(annotation_file)
         pre = time.time()
         metadata = json.load(metadata_f)