зеркало из https://github.com/microsoft/qlib.git
Fix the Warnings in rst files when building Qlib's documentation (#1349)
* Fix docs/advanced/alpha.rst * Fix docs/reference/api.rst * Fix docs/component/strategy.rst * Fix docs/start/integration.rst * Fix docs/component/report.rst * Fix docs/component/data.rst * Fix docs/component/rl/framework.rst * Fix docs/introduction/quick.rst * Fix docs/advanced/task_management.rst * Fix CHANGES.rst * Fix docs/developer/code_standard_and_dev_guide.rst * Fix docs/hidden/client.rst * Fix docs/component/online.rst * Fix docs/start/getdata.rst * Add docs/hidden to exclude patterns * Add docs/developer/code_standard_and_dev_guide.rst to index.rst * Change docs/developer/code_standard_and_dev_guide.rst place in index.rst
This commit is contained in:
Maxim Smolskiy
GitHub
Родитель
4001a5d157
Коммит
82afd6a67a
12
CHANGES.rst
12
CHANGES.rst
|
|
@ -85,7 +85,7 @@ Version 0.4.0
|
|||
-------------
|
||||
- Add `data` package that holds all data-related codes
|
||||
- Reform the data provider structure
|
||||
- Create a server for data centralized management `qlib-server<https://amc-msra.visualstudio.com/trading-algo/_git/qlib-server>`_
|
||||
- Create a server for data centralized management `qlib-server <https://amc-msra.visualstudio.com/trading-algo/_git/qlib-server>`_
|
||||
- Add a `ClientProvider` to work with server
|
||||
- Add a pluggable cache mechanism
|
||||
- Add a recursive backtracking algorithm to inspect the furthest reference date for an expression
|
||||
|
|
@ -166,12 +166,12 @@ Version 0.8.0
|
|||
- Nested decision execution framework is supported
|
||||
- There are lots of changes for daily trading, it is hard to list all of them. But a few important changes could be noticed
|
||||
- The trading limitation is more accurate;
|
||||
- In `previous version <https://github.com/microsoft/qlib/blob/v0.7.2/qlib/contrib/backtest/exchange.py#L160>`_, longing and shorting actions share the same action.
|
||||
- In `current version <https://github.com/microsoft/qlib/blob/7c31012b507a3823117bddcc693fc64899460b2a/qlib/backtest/exchange.py#L304>`_, the trading limitation is different between logging and shorting action.
|
||||
- In `previous version <https://github.com/microsoft/qlib/blob/v0.7.2/qlib/contrib/backtest/exchange.py#L160>`__, longing and shorting actions share the same action.
|
||||
- In `current version <https://github.com/microsoft/qlib/blob/7c31012b507a3823117bddcc693fc64899460b2a/qlib/backtest/exchange.py#L304>`__, the trading limitation is different between logging and shorting action.
|
||||
- The constant is different when calculating annualized metrics.
|
||||
- `Current version <https://github.com/microsoft/qlib/blob/7c31012b507a3823117bddcc693fc64899460b2a/qlib/contrib/evaluate.py#L42>`_ uses more accurate constant than `previous version <https://github.com/microsoft/qlib/blob/v0.7.2/qlib/contrib/evaluate.py#L22>`_
|
||||
- `A new version <https://github.com/microsoft/qlib/blob/7c31012b507a3823117bddcc693fc64899460b2a/qlib/tests/data.py#L17>`_ of data is released. Due to the unstability of Yahoo data source, the data may be different after downloading data again.
|
||||
- Users could check out the backtesting results between `Current version <https://github.com/microsoft/qlib/tree/7c31012b507a3823117bddcc693fc64899460b2a/examples/benchmarks>`_ and `previous version <https://github.com/microsoft/qlib/tree/v0.7.2/examples/benchmarks>`_
|
||||
- `Current version <https://github.com/microsoft/qlib/blob/7c31012b507a3823117bddcc693fc64899460b2a/qlib/contrib/evaluate.py#L42>`_ uses more accurate constant than `previous version <https://github.com/microsoft/qlib/blob/v0.7.2/qlib/contrib/evaluate.py#L22>`__
|
||||
- `A new version <https://github.com/microsoft/qlib/blob/7c31012b507a3823117bddcc693fc64899460b2a/qlib/tests/data.py#L17>`__ of data is released. Due to the unstability of Yahoo data source, the data may be different after downloading data again.
|
||||
- Users could check out the backtesting results between `Current version <https://github.com/microsoft/qlib/tree/7c31012b507a3823117bddcc693fc64899460b2a/examples/benchmarks>`__ and `previous version <https://github.com/microsoft/qlib/tree/v0.7.2/examples/benchmarks>`__
|
||||
|
||||
|
||||
Other Versions
|
||||
|
|
|
|||
|
|
@ -38,7 +38,7 @@ Example
|
|||
|
||||
DIF = \frac{EMA(CLOSE, 12) - EMA(CLOSE, 26)}{CLOSE}
|
||||
|
||||
`DEA`means a 9-period EMA of the DIF.
|
||||
`DEA` means a 9-period EMA of the DIF.
|
||||
|
||||
.. math::
|
||||
|
||||
|
|
|
|||
|
|
@ -18,7 +18,7 @@ With this module, users can run their ``task`` automatically at different period
|
|||
|
||||
This whole process can be used in `Online Serving <../component/online.html>`_.
|
||||
|
||||
An example of the entire process is shown `here <https://github.com/microsoft/qlib/tree/main/examples/model_rolling/task_manager_rolling.py>`_.
|
||||
An example of the entire process is shown `here <https://github.com/microsoft/qlib/tree/main/examples/model_rolling/task_manager_rolling.py>`__.
|
||||
|
||||
Task Generating
|
||||
===============
|
||||
|
|
@ -33,7 +33,7 @@ Here is the base class of ``TaskGen``:
|
|||
:members:
|
||||
|
||||
``Qlib`` provides a class `RollingGen <https://github.com/microsoft/qlib/tree/main/qlib/workflow/task/gen.py>`_ to generate a list of ``task`` of the dataset in different date segments.
|
||||
This class allows users to verify the effect of data from different periods on the model in one experiment. More information is `here <../reference/api.html#TaskGen>`_.
|
||||
This class allows users to verify the effect of data from different periods on the model in one experiment. More information is `here <../reference/api.html#TaskGen>`__.
|
||||
|
||||
Task Storing
|
||||
============
|
||||
|
|
@ -54,7 +54,7 @@ Users need to provide the MongoDB URL and database name for using ``TaskManager`
|
|||
.. autoclass:: qlib.workflow.task.manage.TaskManager
|
||||
:members:
|
||||
|
||||
More information of ``Task Manager`` can be found in `here <../reference/api.html#TaskManager>`_.
|
||||
More information of ``Task Manager`` can be found in `here <../reference/api.html#TaskManager>`__.
|
||||
|
||||
Task Training
|
||||
=============
|
||||
|
|
|
|||
|
|
@ -24,8 +24,8 @@ The introduction of ``Data Layer`` includes the following parts.
|
|||
Here is a typical example of Qlib data workflow
|
||||
|
||||
- Users download data and converting data into Qlib format(with filename suffix `.bin`). In this step, typically only some basic data are stored on disk(such as OHLCV).
|
||||
- Creating some basic features based on Qlib's expression Engine(e.g. "Ref($close, 60) / $close", the return of last 60 trading days). Supported operators in the expression engine can be found `here <https://github.com/microsoft/qlib/blob/main/qlib/data/ops.py>`_. This step is typically implemented in Qlib's `Data Loader <https://qlib.readthedocs.io/en/latest/component/data.html#data-loader>`_ which is a component of `Data Handler <https://qlib.readthedocs.io/en/latest/component/data.html#data-handler>`_ .
|
||||
- If users require more complicated data processing (e.g. data normalization), `Data Handler <https://qlib.readthedocs.io/en/latest/component/data.html#data-handler>`_ support user-customized processors to process data(some predefined processors can be found `here <https://github.com/microsoft/qlib/blob/main/qlib/data/dataset/processor.py>`_). The processors are different from operators in expression engine. It is designed for some complicated data processing methods which is hard to supported in operators in expression engine.
|
||||
- Creating some basic features based on Qlib's expression Engine(e.g. "Ref($close, 60) / $close", the return of last 60 trading days). Supported operators in the expression engine can be found `here <https://github.com/microsoft/qlib/blob/main/qlib/data/ops.py>`__. This step is typically implemented in Qlib's `Data Loader <https://qlib.readthedocs.io/en/latest/component/data.html#data-loader>`_ which is a component of `Data Handler <https://qlib.readthedocs.io/en/latest/component/data.html#data-handler>`_ .
|
||||
- If users require more complicated data processing (e.g. data normalization), `Data Handler <https://qlib.readthedocs.io/en/latest/component/data.html#data-handler>`_ support user-customized processors to process data(some predefined processors can be found `here <https://github.com/microsoft/qlib/blob/main/qlib/data/dataset/processor.py>`__). The processors are different from operators in expression engine. It is designed for some complicated data processing methods which is hard to supported in operators in expression engine.
|
||||
- At last, `Dataset <https://qlib.readthedocs.io/en/latest/component/data.html#dataset>`_ is responsible to prepare model-specific dataset from the processed data of Data Handler
|
||||
|
||||
Data Preparation
|
||||
|
|
@ -37,7 +37,7 @@ Qlib Format Data
|
|||
We've specially designed a data structure to manage financial data, please refer to the `File storage design section in Qlib paper <https://arxiv.org/abs/2009.11189>`_ for detailed information.
|
||||
Such data will be stored with filename suffix `.bin` (We'll call them `.bin` file, `.bin` format, or qlib format). `.bin` file is designed for scientific computing on finance data.
|
||||
|
||||
``Qlib`` provides two different off-the-shelf datasets, which can be accessed through this `link <https://github.com/microsoft/qlib/blob/main/qlib/contrib/data/handler.py>`_:
|
||||
``Qlib`` provides two different off-the-shelf datasets, which can be accessed through this `link <https://github.com/microsoft/qlib/blob/main/qlib/contrib/data/handler.py>`__:
|
||||
|
||||
======================== ================= ================
|
||||
Dataset US Market China Market
|
||||
|
|
@ -47,7 +47,7 @@ Alpha360 √ √
|
|||
Alpha158 √ √
|
||||
======================== ================= ================
|
||||
|
||||
Also, ``Qlib`` provides a high-frequency dataset. Users can run a high-frequency dataset example through this `link <https://github.com/microsoft/qlib/tree/main/examples/highfreq>`_.
|
||||
Also, ``Qlib`` provides a high-frequency dataset. Users can run a high-frequency dataset example through this `link <https://github.com/microsoft/qlib/tree/main/examples/highfreq>`__.
|
||||
|
||||
Qlib Format Dataset
|
||||
-------------------
|
||||
|
|
@ -512,7 +512,7 @@ Data and Cache File Structure
|
|||
|
||||
We've specially designed a file structure to manage data and cache, please refer to the `File storage design section in Qlib paper <https://arxiv.org/abs/2009.11189>`_ for detailed information. The file structure of data and cache is listed as follows.
|
||||
|
||||
.. code-block:: json
|
||||
.. code-block::
|
||||
|
||||
- data/
|
||||
[raw data] updated by data providers
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
.. _online:
|
||||
.. _online_serving:
|
||||
|
||||
==============
|
||||
Online Serving
|
||||
|
|
|
|||
|
|
@ -174,6 +174,7 @@ Graphical Result
|
|||
The `Information Ratio` without cost.
|
||||
- `excess_return_with_cost`
|
||||
The `Information Ratio` with cost.
|
||||
|
||||
To know more about `Information Ratio`, please refer to `Information Ratio – IR <https://www.investopedia.com/terms/i/informationratio.asp>`_.
|
||||
- `max_drawdown`
|
||||
- `excess_return_without_cost`
|
||||
|
|
|
|||
|
|
@ -28,7 +28,7 @@ In QlibRL, EnvWrapper is a subclass of gym.Env, so it implements all necessary i
|
|||
|
||||
EnvWrapper will organically organize these components. Such decomposition allows for better flexibility in development. For example, if the developers want to train multiple types of policies in the same environment, they only need to design one simulator and design different state interpreters/action interpreters/reward functions for different types of policies.
|
||||
|
||||
QlibRL has well-defined base classes for all these 4 components. All the developers need to do is define their own components by inheriting the base classes and then implementing all interfaces required by the base classes. The API for the above base components can be found `here <../../reference/api.html#module-qlib.rl>`_.
|
||||
QlibRL has well-defined base classes for all these 4 components. All the developers need to do is define their own components by inheriting the base classes and then implementing all interfaces required by the base classes. The API for the above base components can be found `here <../../reference/api.html#module-qlib.rl>`__.
|
||||
|
||||
Policy
|
||||
------------
|
||||
|
|
@ -42,4 +42,4 @@ As you may have noticed, a training vessel itself holds all the required compone
|
|||
|
||||
With a training vessel, the trainer could finally launch the training pipeline by simple, Scikit-learn-like interfaces (i.e., ``trainer.fit()``).
|
||||
|
||||
The API for Trainer and TrainingVessel and can be found `here <../../reference/api.html#module-qlib.rl.trainer>`_.
|
||||
The API for Trainer and TrainingVessel and can be found `here <../../reference/api.html#module-qlib.rl.trainer>`__.
|
||||
|
|
@ -80,6 +80,7 @@ TopkDropoutStrategy
|
|||
In most cases, ``TopkDrop`` algorithm sells and buys `Drop` stocks every trading day, which yields a turnover rate of 2$\times$`Drop`/$K$.
|
||||
|
||||
The following images illustrate a typical scenario.
|
||||
|
||||
.. image:: ../_static/img/topk_drop.png
|
||||
:alt: Topk-Drop
|
||||
|
||||
|
|
|
|||
|
|
@ -77,7 +77,7 @@ language = "en_US"
|
|||
# List of patterns, relative to source directory, that match files and
|
||||
# directories to ignore when looking for source files.
|
||||
# This patterns also effect to html_static_path and html_extra_path
|
||||
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
|
||||
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store", "hidden"]
|
||||
|
||||
# The name of the Pygments (syntax highlighting) style to use.
|
||||
pygments_style = "sphinx"
|
||||
|
|
|
|||
|
|
@ -15,7 +15,8 @@ Continuous Integration (CI) tools help you stick to the quality standards by run
|
|||
When you submit a PR request, you can check whether your code passes the CI tests in the "check" section at the bottom of the web page.
|
||||
|
||||
1. Qlib will check the code format with black. The PR will raise error if your code does not align to the standard of Qlib(e.g. a common error is the mixed use of space and tab).
|
||||
You can fix the bug by inputing the following code in the command line.
|
||||
|
||||
You can fix the bug by inputing the following code in the command line.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
|
|
@ -32,7 +33,8 @@ When you submit a PR request, you can check whether your code passes the CI test
|
|||
|
||||
|
||||
3. Qlib will check your code style flake8. The checking command is implemented in [github action workflow](https://github.com/microsoft/qlib/blob/0e8b94a552f1c457cfa6cd2c1bb3b87ebb3fb279/.github/workflows/test.yml#L73).
|
||||
You can fix the bug by inputing the following code in the command line.
|
||||
|
||||
You can fix the bug by inputing the following code in the command line.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
|
|
@ -40,7 +42,8 @@ When you submit a PR request, you can check whether your code passes the CI test
|
|||
|
||||
|
||||
4. Qlib has integrated pre-commit, which will make it easier for developers to format their code.
|
||||
Just run the following two commands, and the code will be automatically formatted using black and flake8 when the git commit command is executed.
|
||||
|
||||
Just run the following two commands, and the code will be automatically formatted using black and flake8 when the git commit command is executed.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
|
|
|
|||
|
|
@ -81,6 +81,7 @@ If running on Windows, open **NFS** features and write correct **mount_path**, i
|
|||
* Open ``Programs and Features``.
|
||||
* Click ``Turn Windows features on or off``.
|
||||
* Scroll down and check the option ``Services for NFS``, then click OK
|
||||
|
||||
Reference address: https://graspingtech.com/mount-nfs-share-windows-10/
|
||||
2.config correct mount_path
|
||||
* In windows, mount path must be not exist path and root path,
|
||||
|
|
@ -161,7 +162,7 @@ Limitations
|
|||
API
|
||||
***
|
||||
|
||||
The client is based on `python-socketio<https://python-socketio.readthedocs.io>`_ which is a framework that supports WebSocket client for Python language. The client can only propose requests and receive results, which do not include any calculating procedure.
|
||||
The client is based on `python-socketio <https://python-socketio.readthedocs.io>`_ which is a framework that supports WebSocket client for Python language. The client can only propose requests and receive results, which do not include any calculating procedure.
|
||||
|
||||
Class
|
||||
-----
|
||||
|
|
|
|||
|
|
@ -56,6 +56,12 @@ Document Structure
|
|||
Task Management <advanced/task_management.rst>
|
||||
Point-In-Time database <advanced/PIT.rst>
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 3
|
||||
:caption: FOR DEVELOPERS:
|
||||
|
||||
Code Standard & Development Guidance <developer/code_standard_and_dev_guide.rst>
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 3
|
||||
:caption: REFERENCE:
|
||||
|
|
|
|||
|
|
@ -21,6 +21,7 @@ Users can easily intsall ``Qlib`` according to the following steps:
|
|||
- Before installing ``Qlib`` from source, users need to install some dependencies:
|
||||
|
||||
.. code-block::
|
||||
|
||||
pip install numpy
|
||||
pip install --upgrade cython
|
||||
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
.. _api:
|
||||
|
||||
=============
|
||||
API Reference
|
||||
=============
|
||||
|
|
|
|||
|
|
@ -83,15 +83,14 @@ Load features of certain instruments in a given time range:
|
|||
>> from qlib.data import D
|
||||
>> instruments = ['SH600000']
|
||||
>> fields = ['$close', '$volume', 'Ref($close, 1)', 'Mean($close, 3)', '$high-$low']
|
||||
>> D.features(instruments, fields, start_time='2010-01-01', end_time='2017-12-31', freq='day').head()
|
||||
|
||||
$close $volume Ref($close, 1) Mean($close, 3) $high-$low
|
||||
instrument datetime
|
||||
SH600000 2010-01-04 86.778313 16162960.0 88.825928 88.061483 2.907631
|
||||
2010-01-05 87.433578 28117442.0 86.778313 87.679273 3.235252
|
||||
2010-01-06 85.713585 23632884.0 87.433578 86.641825 1.720009
|
||||
2010-01-07 83.788803 20813402.0 85.713585 85.645322 3.030487
|
||||
2010-01-08 84.730675 16044853.0 83.788803 84.744354 2.047623
|
||||
>> D.features(instruments, fields, start_time='2010-01-01', end_time='2017-12-31', freq='day').head().to_string()
|
||||
' $close $volume Ref($close, 1) Mean($close, 3) $high-$low
|
||||
... instrument datetime
|
||||
... SH600000 2010-01-04 86.778313 16162960.0 88.825928 88.061483 2.907631
|
||||
... 2010-01-05 87.433578 28117442.0 86.778313 87.679273 3.235252
|
||||
... 2010-01-06 85.713585 23632884.0 87.433578 86.641825 1.720009
|
||||
... 2010-01-07 83.788803 20813402.0 85.713585 85.645322 3.030487
|
||||
... 2010-01-08 84.730675 16044853.0 83.788803 84.744354 2.047623'
|
||||
|
||||
Load features of certain stock pool in a given time range:
|
||||
|
||||
|
|
@ -105,15 +104,14 @@ Load features of certain stock pool in a given time range:
|
|||
>> expressionDFilter = ExpressionDFilter(rule_expression='$close>Ref($close,1)')
|
||||
>> instruments = D.instruments(market='csi300', filter_pipe=[nameDFilter, expressionDFilter])
|
||||
>> fields = ['$close', '$volume', 'Ref($close, 1)', 'Mean($close, 3)', '$high-$low']
|
||||
>> D.features(instruments, fields, start_time='2010-01-01', end_time='2017-12-31', freq='day').head()
|
||||
|
||||
$close $volume Ref($close, 1) Mean($close, 3) $high-$low
|
||||
instrument datetime
|
||||
SH600655 2010-01-04 2699.567383 158193.328125 2619.070312 2626.097738 124.580566
|
||||
2010-01-08 2612.359619 77501.406250 2584.567627 2623.220133 83.373047
|
||||
2010-01-11 2712.982422 160852.390625 2612.359619 2636.636556 146.621582
|
||||
2010-01-12 2788.688232 164587.937500 2712.982422 2704.676758 128.413818
|
||||
2010-01-13 2790.604004 145460.453125 2788.688232 2764.091553 128.413818
|
||||
>> D.features(instruments, fields, start_time='2010-01-01', end_time='2017-12-31', freq='day').head().to_string()
|
||||
' $close $volume Ref($close, 1) Mean($close, 3) $high-$low
|
||||
... instrument datetime
|
||||
... SH600655 2010-01-04 2699.567383 158193.328125 2619.070312 2626.097738 124.580566
|
||||
... 2010-01-08 2612.359619 77501.406250 2584.567627 2623.220133 83.373047
|
||||
... 2010-01-11 2712.982422 160852.390625 2612.359619 2636.636556 146.621582
|
||||
... 2010-01-12 2788.688232 164587.937500 2712.982422 2704.676758 128.413818
|
||||
... 2010-01-13 2790.604004 145460.453125 2788.688232 2764.091553 128.413818'
|
||||
|
||||
|
||||
For more details about features, please refer `Feature API <../component/data.html>`_.
|
||||
|
|
|
|||
|
|
@ -21,84 +21,88 @@ The Custom models need to inherit `qlib.model.base.Model <../reference/api.html#
|
|||
- ``Qlib`` passes the initialized parameters to the \_\_init\_\_ method.
|
||||
- The hyperparameters of model in the configuration must be consistent with those defined in the `__init__` method.
|
||||
- Code Example: In the following example, the hyperparameters of model in the configuration file should contain parameters such as `loss:mse`.
|
||||
.. code-block:: Python
|
||||
|
||||
def __init__(self, loss='mse', **kwargs):
|
||||
if loss not in {'mse', 'binary'}:
|
||||
raise NotImplementedError
|
||||
self._scorer = mean_squared_error if loss == 'mse' else roc_auc_score
|
||||
self._params.update(objective=loss, **kwargs)
|
||||
self._model = None
|
||||
.. code-block:: Python
|
||||
|
||||
def __init__(self, loss='mse', **kwargs):
|
||||
if loss not in {'mse', 'binary'}:
|
||||
raise NotImplementedError
|
||||
self._scorer = mean_squared_error if loss == 'mse' else roc_auc_score
|
||||
self._params.update(objective=loss, **kwargs)
|
||||
self._model = None
|
||||
|
||||
- Override the `fit` method
|
||||
- ``Qlib`` calls the fit method to train the model.
|
||||
- The parameters must include training feature `dataset`, which is designed in the interface.
|
||||
- The parameters could include some `optional` parameters with default values, such as `num_boost_round = 1000` for `GBDT`.
|
||||
- Code Example: In the following example, `num_boost_round = 1000` is an optional parameter.
|
||||
.. code-block:: Python
|
||||
|
||||
def fit(self, dataset: DatasetH, num_boost_round = 1000, **kwargs):
|
||||
.. code-block:: Python
|
||||
|
||||
# prepare dataset for lgb training and evaluation
|
||||
df_train, df_valid = dataset.prepare(
|
||||
["train", "valid"], col_set=["feature", "label"], data_key=DataHandlerLP.DK_L
|
||||
)
|
||||
x_train, y_train = df_train["feature"], df_train["label"]
|
||||
x_valid, y_valid = df_valid["feature"], df_valid["label"]
|
||||
def fit(self, dataset: DatasetH, num_boost_round = 1000, **kwargs):
|
||||
|
||||
# Lightgbm need 1D array as its label
|
||||
if y_train.values.ndim == 2 and y_train.values.shape[1] == 1:
|
||||
y_train, y_valid = np.squeeze(y_train.values), np.squeeze(y_valid.values)
|
||||
else:
|
||||
raise ValueError("LightGBM doesn't support multi-label training")
|
||||
# prepare dataset for lgb training and evaluation
|
||||
df_train, df_valid = dataset.prepare(
|
||||
["train", "valid"], col_set=["feature", "label"], data_key=DataHandlerLP.DK_L
|
||||
)
|
||||
x_train, y_train = df_train["feature"], df_train["label"]
|
||||
x_valid, y_valid = df_valid["feature"], df_valid["label"]
|
||||
|
||||
dtrain = lgb.Dataset(x_train.values, label=y_train)
|
||||
dvalid = lgb.Dataset(x_valid.values, label=y_valid)
|
||||
# Lightgbm need 1D array as its label
|
||||
if y_train.values.ndim == 2 and y_train.values.shape[1] == 1:
|
||||
y_train, y_valid = np.squeeze(y_train.values), np.squeeze(y_valid.values)
|
||||
else:
|
||||
raise ValueError("LightGBM doesn't support multi-label training")
|
||||
|
||||
# fit the model
|
||||
self.model = lgb.train(
|
||||
self.params,
|
||||
dtrain,
|
||||
num_boost_round=num_boost_round,
|
||||
valid_sets=[dtrain, dvalid],
|
||||
valid_names=["train", "valid"],
|
||||
early_stopping_rounds=early_stopping_rounds,
|
||||
verbose_eval=verbose_eval,
|
||||
evals_result=evals_result,
|
||||
**kwargs
|
||||
)
|
||||
dtrain = lgb.Dataset(x_train.values, label=y_train)
|
||||
dvalid = lgb.Dataset(x_valid.values, label=y_valid)
|
||||
|
||||
# fit the model
|
||||
self.model = lgb.train(
|
||||
self.params,
|
||||
dtrain,
|
||||
num_boost_round=num_boost_round,
|
||||
valid_sets=[dtrain, dvalid],
|
||||
valid_names=["train", "valid"],
|
||||
early_stopping_rounds=early_stopping_rounds,
|
||||
verbose_eval=verbose_eval,
|
||||
evals_result=evals_result,
|
||||
**kwargs
|
||||
)
|
||||
|
||||
- Override the `predict` method
|
||||
- The parameters must include the parameter `dataset`, which will be userd to get the test dataset.
|
||||
- Return the `prediction score`.
|
||||
- Please refer to `Model API <../reference/api.html#module-qlib.model.base>`_ for the parameter types of the fit method.
|
||||
- Code Example: In the following example, users need to use `LightGBM` to predict the label(such as `preds`) of test data `x_test` and return it.
|
||||
.. code-block:: Python
|
||||
|
||||
def predict(self, dataset: DatasetH, **kwargs)-> pandas.Series:
|
||||
if self.model is None:
|
||||
raise ValueError("model is not fitted yet!")
|
||||
x_test = dataset.prepare("test", col_set="feature", data_key=DataHandlerLP.DK_I)
|
||||
return pd.Series(self.model.predict(x_test.values), index=x_test.index)
|
||||
.. code-block:: Python
|
||||
|
||||
def predict(self, dataset: DatasetH, **kwargs)-> pandas.Series:
|
||||
if self.model is None:
|
||||
raise ValueError("model is not fitted yet!")
|
||||
x_test = dataset.prepare("test", col_set="feature", data_key=DataHandlerLP.DK_I)
|
||||
return pd.Series(self.model.predict(x_test.values), index=x_test.index)
|
||||
|
||||
- Override the `finetune` method (Optional)
|
||||
- This method is optional to the users. When users want to use this method on their own models, they should inherit the ``ModelFT`` base class, which includes the interface of `finetune`.
|
||||
- The parameters must include the parameter `dataset`.
|
||||
- Code Example: In the following example, users will use `LightGBM` as the model and finetune it.
|
||||
.. code-block:: Python
|
||||
|
||||
def finetune(self, dataset: DatasetH, num_boost_round=10, verbose_eval=20):
|
||||
# Based on existing model and finetune by train more rounds
|
||||
dtrain, _ = self._prepare_data(dataset)
|
||||
self.model = lgb.train(
|
||||
self.params,
|
||||
dtrain,
|
||||
num_boost_round=num_boost_round,
|
||||
init_model=self.model,
|
||||
valid_sets=[dtrain],
|
||||
valid_names=["train"],
|
||||
verbose_eval=verbose_eval,
|
||||
)
|
||||
.. code-block:: Python
|
||||
|
||||
def finetune(self, dataset: DatasetH, num_boost_round=10, verbose_eval=20):
|
||||
# Based on existing model and finetune by train more rounds
|
||||
dtrain, _ = self._prepare_data(dataset)
|
||||
self.model = lgb.train(
|
||||
self.params,
|
||||
dtrain,
|
||||
num_boost_round=num_boost_round,
|
||||
init_model=self.model,
|
||||
valid_sets=[dtrain],
|
||||
valid_names=["train"],
|
||||
verbose_eval=verbose_eval,
|
||||
)
|
||||
|
||||
Configuration File
|
||||
==================
|
||||
|
|
@ -107,21 +111,21 @@ The configuration file is described in detail in the `Workflow <../component/wor
|
|||
|
||||
- Example: The following example describes the `model` field of configuration file about the custom lightgbm model mentioned above, where `module_path` is the module path, `class` is the class name, and `args` is the hyperparameter passed into the __init__ method. All parameters in the field is passed to `self._params` by `\*\*kwargs` in `__init__` except `loss = mse`.
|
||||
|
||||
.. code-block:: YAML
|
||||
.. code-block:: YAML
|
||||
|
||||
model:
|
||||
class: LGBModel
|
||||
module_path: qlib.contrib.model.gbdt
|
||||
args:
|
||||
loss: mse
|
||||
colsample_bytree: 0.8879
|
||||
learning_rate: 0.0421
|
||||
subsample: 0.8789
|
||||
lambda_l1: 205.6999
|
||||
lambda_l2: 580.9768
|
||||
max_depth: 8
|
||||
num_leaves: 210
|
||||
num_threads: 20
|
||||
model:
|
||||
class: LGBModel
|
||||
module_path: qlib.contrib.model.gbdt
|
||||
args:
|
||||
loss: mse
|
||||
colsample_bytree: 0.8879
|
||||
learning_rate: 0.0421
|
||||
subsample: 0.8789
|
||||
lambda_l1: 205.6999
|
||||
lambda_l2: 580.9768
|
||||
max_depth: 8
|
||||
num_leaves: 210
|
||||
num_threads: 20
|
||||
|
||||
Users could find configuration file of the baselines of the ``Model`` in ``examples/benchmarks``. All the configurations of different models are listed under the corresponding model folder.
|
||||
|
||||
|
|
|
|||
Загрузка…
Ссылка в новой задаче