Skip to main content

Introduction

This guide will show you how to import, extend and implement necessary methods of Tru Serve Module, so you will be able to load your custom model and do prediction through Tru AI Model API.

Configuration​

To start using the TruServe module you need to first import the package 

from gjirafatech.truai.serve import TruServe, TruRequestModel

TruRequestModel​

TruRequestModel is the base class that you need to extend and define the request body that will be used for prediction.

Example of extending TruRequestModel
class MyRequest(TruRequestModel):
    CryoSleep: bool
    Age: float
    VIP: bool
Json representation of the above TruRequestModel
{
  "CryoSleep": true,
  "Age": 0,
  "VIP": true
}

TruServe​

TruServe is the base class that you need to extend and implement load and predict methods.
The load method is triggered once through the lifetime of the application, and should be used to load the model and make it ready for prediction.

Example of load method
def load(self):
    storage = TruStorageClient()
    content = storage.get_object("My_Models/titanic_model.pickle")
    self.model = pickle.loads(content)

The predict method is triggered once for every request, accepts request model as argument, and should handle parsing and prediction from the model. Returned value will be provided as response body on the endpoint.

Example of predict method
def predict(self, request:MyRequest)->List[int]:
    body = pd.DataFrame.from_dict(request.dict(), orient='index').T
    return self.model.predict(body).tolist()

TruServe Multiple Endpoints​

In order to create custom endpoints other than the predict endpoint, you need to use the decorator method serve which has a parameter to specify the path of the endpoint. Apply that decorator to all the additional methods you implement in the TruServe class.

Example of using the decorator method
@TruServe.serve("path")
def get_feature_names(self):
    return self.model.feature_names_in_

Example​

from gjirafatech.truai.serve import TruServe, TruRequestModel
from gjirafatech.truai.storage import TruStorageClient
from typing import List
import pickle
import pandas as pd

class MyRequest(TruRequestModel):
    CryoSleep: bool
    Age: float
    VIP: bool


class MyTruServe(TruServe):
    def __init__(self):
        self.model = None
    
    def load(self):
        storage = TruStorageClient()
        content = storage.get_object("My_Models/titanic_model.pickle")
        self.model = pickle.loads(content)
    
    def predict(self, request:MyRequest)->List[int]:
        body = pd.DataFrame.from_dict(request.dict(), orient='index').T
        return self.model.predict(body).tolist()

    @TruServe.serve("path")
    def get_model_properties(self)->dict:
        return self.model.__dict__