This tutorial will demonstrate the basic workflow.
Train a tree ensemble model using XGBoost
The first step is to train a tree ensemble model using XGBoost
(dmlc/xgboost).
Disclaimer: Treelite does NOT depend on the XGBoost package in any way.
XGBoost was used here only to provide a working example.
import xgboost
dtrain = xgboost.DMatrix(X, label=y)
params = {'max_depth':3, 'eta':1, 'silent':1, 'objective':'reg:linear',
'eval_metric':'rmse'}
bst = xgboost.train(params, dtrain, 20, [(dtrain, 'train')])
Pass XGBoost model into treelite
Next, we feed the trained model into treelite. If you used XGBoost to
train the model, it takes only one line of code:
model = treelite.Model.from_xgboost(bst)
Note
Using other packages to train decision trees
With additional work, you can use models trained with other machine learning
packages. See this page for instructions.
Generate shared library
Given a tree ensemble model, treelite will produce a prediction subroutine
(internally represented as a C program). To use
the subroutine for prediction task, we package it as a dynamic shared
library,
which exports the prediction subroutine for other programs to use.
Before proceeding, you should decide which of the following compilers is
available on your system and set the variable toolchain
appropriately:
gcc
clang
msvc
(Microsoft Visual C++)
toolchain = 'clang' # change this value as necessary
The choice of toolchain will be used to compile the prediction
subroutine into native code.
Now we are ready to generate the library.
model.export_lib(toolchain=toolchain, libpath='./mymodel.dylib', verbose=True)
# ^^^^^
# set correct file extension here; see the following paragraph
Note
File extension for shared library
Make sure to use the correct file extension for the library,
depending on the operating system:
- Windows:
.dll
- Mac OS X:
.dylib
- Linux / Other UNIX:
.so
Note
Want to deploy the model to another machine?
This tutorial assumes that predictions will be made on the same machine that
is running treelite. If you’d like to deploy your model to another machine
(that may not have treelite installed), see the page Deploying models.
Note
Reducing compilation time for large models
For large models, export_lib()
may take a long time
to finish. To reduce compilation time, enable the parallel_comp
option by
writing
model.export_lib(toolchain=toolchain, libpath='./mymodel.dylib',
params={'parallel_comp': 32}, verbose=True)
which splits the prediction subroutine into 32 source files that gets compiled
in parallel. Adjust this number according to the number of cores on your
machine.
Use the shared library to make predictions
Once the shared library has been generated, we feed it into a separate
module (treelite.runtime
) known as the runtime. The
optimized prediction subroutine is exposed through the
Predictor
class:
import treelite.runtime # runtime module
predictor = treelite.runtime.Predictor('./mymodel.dylib', verbose=True)
We decide on which of the houses in X
we should make predictions
for. Say, from 10th house to 20th:
batch = treelite.runtime.Batch.from_npy2d(X, rbegin=10, rend=20)
We used the method from_npy2d()
because the matrix X
was a dense NumPy array (numpy.ndarray
).
If X
were a sparse matrix (scipy.sparse.csr_matrix
), we would
have used the method from_csr()
instead.
out_pred = predictor.predict(batch, verbose=True)
print(out_pred)