You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
 
 
dependabot-preview[bot] de2dce0388 Update prometheus-client requirement in /python 3 months ago
.dependabot set user as default reviewer 4 months ago
.github/ISSUE_TEMPLATE Add issue templates 9 months ago
ci add pr build comment 9 months ago
components rebase 4 months ago
core-builder core-builder version 0.15 4 months ago
doc rebase 3 months ago
engine Add make test target 3 months ago
examples handle env vars like opeartor does 3 months ago
executor test for defaulting log url 3 months ago
fbs Update flatbuffer compile 2 years ago
hack update examples to 1.1.1-rc 4 months ago
helm-charts rebase 3 months ago
incubating fix the loop length from parmeters count 3 months ago
integrations update to 1.1.1-rc 4 months ago
kafka Use print() function in both Python 2 and Python 3 8 months ago
kubectl-proxy kubectl-proxy added 3 years ago
marketplaces/redhat updates from testing certified images 4 months ago
monitoring update docs 6 months ago
notebooks factor out minio setup instructions into single notebook 3 months ago
openapi Swagger UI with Embedded Seldon Deployment Values (#1744) 4 months ago
operator rebase 3 months ago
persistence persistence scripts added 3 years ago
proto feat(proto): Add custom proto handling 5 months ago
python Update prometheus-client requirement in /python 3 months ago
python-builder Fix check to link to correct website 1 year ago
redis-memonly changes to allow engine version used to be set in helm 2 years ago
seldon-controller update local private build scripts for controller 1 year ago
servers increase failureThreshold in mlflow server sample 3 months ago
testing Merge pull request #1803 from RafalSkolasinski/issue/1728-graph-level-metadata 4 months ago
util/loadtester Undefined name: 'r' --> 'response' 7 months ago
wrappers/s2i/python fix script pushing wrapper images to docker hub 4 months ago
zookeeper-k8s changed start scripts to use apply 3 years ago
.gitignore simplify centralised logging example 3 months ago
.pre-commit-config.yaml RedHat Community Operator (#1519) 5 months ago
.readthedocs.yml update readthedocs.yml 1 year ago
CHANGELOG.md version 1.1.1-SNAPSHOT 5 months ago
CODE_OF_CONDUCT.md change email for code of conduct 2 years ago
CONTRIBUTING.md Add page for integration tests 8 months ago
LICENSE Updated license file 2 years ago
Makefile update core-builder version 0.15 references 4 months ago
Makefile.ci Update engine and top level makefiles 6 months ago
OWNERS Updated OWNERS file 5 months ago
OWNERS_ALIASES Update OWNER_ALIASES 6 months ago
README.md Swagger UI with Embedded Seldon Deployment Values (#1744) 4 months ago
ci_build_and_push_images.sh update to 1.1.1-rc 4 months ago
create-changelog add script to create a CHANGELOG.md file 2 years ago
jenkins-x-integration.yml update core-builder version 0.15 references 4 months ago
jenkins-x-lint.yml update core-builder version 0.15 references 4 months ago
jenkins-x-notebooks.yml update core-builder version 0.15 references 4 months ago
jenkins-x.yml update core-builder version 0.15 references 4 months ago
release update core-builder version 0.15 references 4 months ago
release.py update version.py when release.py is run 3 months ago
start-seldon-core-private-repo remove api frontend references 1 year ago
stop-seldon-core-private-repo update local private build scripts for controller 1 year ago
version.txt update to 1.1.1-rc 4 months ago

README.md

Seldon Core: Blazing Fast, Industry-Ready ML

An open source platform to deploy your machine learning models on Kubernetes at massive scale.

Overview

Seldon core converts your ML models (Tensorflow, Pytorch, H2o, etc.) or language wrappers (Python, Java, etc.) into production REST/GRPC microservices.

Seldon handles scaling to thousands of production machine learning models and provides advanced machine learning capabilities out of the box including Advanced Metrics, Request Logging, Explainers, Outlier Detectors, A/B Tests, Canaries and more.

High Level Features

With over 2M installs, Seldon Core is used across organisations to manage large scale deployment of machine learning models, and key benefits include:

  • Easy way to containerise ML models using our language wrappers or pre-packaged inference servers.
  • Out of the box endpoints which can be tested through Swagger UI, Seldon Python Client or Curl / GRPCurl
  • Cloud agnostic and tested on AWS EKS, Azure AKS, Google GKE, Alicloud, Digital Ocean and Openshift.
  • Powerful and rich inference graphs made out of predictors, transformers, routers, combiners, and more.
  • A standardised serving layer across models from heterogeneous toolkits and languages.
  • Advanced and customisable metrics with integration to Prometheus and Grafana.
  • Full auditability through model input-output request logging integration with Elasticsearch.
  • Microservice tracing through integration to Jaeger for insights on latency across microservice hops.

Getting Started

Deploying your models using Seldon Core is simplified through our pre-packaged inference servers and language wrappers. Below you can see how you can deploy our "hello world Iris" example. You can see more details on these workflows in our Documentation Quickstart.

Install Seldon Core

Quick install using Helm 3 (you can also use Kustomize):

kubectl create namespace seldon-system

helm install seldon-core seldon-core-operator \
    --repo https://storage.googleapis.com/seldon-charts \
    --set usageMetrics.enabled=true \
    --namespace seldon-system \
    --set istio.enabled=true
    # You can set ambassador instead with --set ambassador.enabled=true

Deploy your model using pre-packaged model servers

We provide optimized model servers for some of the most popular Deep Learning and Machine Learning frameworks that allow you to deploy your trained model binaries/weights without having to containerize or modify them.

You only have to upload your model binaries into your preferred object store, in this case we have a trained scikit-learn iris model in a Google bucket:

gs://seldon-models/sklearn/iris/model.pickle

We then can deploy this model with Seldon Core to our Kubernetes cluster using the pre-packaged model server for scikit-learn (SKLEARN_SERVER) by running the kubectl apply command below:

$ kubectl apply -f - << END
apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
  name: iris-model
  namespace: model-namespace
spec:
  name: iris
  predictors:
  - graph:
      implementation: SKLEARN_SERVER
      modelUri: gs://seldon-models/sklearn/iris
      name: classifier
    name: default
    replicas: 1
END

Send API requests to your deployed model

Every model deployed exposes a standardised User Interface to send requests using our OpenAPI schema.

This can be accessed through the endpoint http://<ingress_url>/seldon/<namespace>/<model-name>/api/v1.0/doc/ which will allow you to send requests directly through your browser.

Or alternatively you can send requests programmatically using our Seldon Python Client or another Linux CLI:

$ curl -X POST http://<ingress>/seldon/model-namespace/iris-model/api/v1.0/predictions \
    -H 'Content-Type: application/json' \
    -d '{ "data": { "ndarray": [1,2,3,4] } }' | json_pp

{
   "meta" : {},
   "data" : {
      "names" : [
         "t:0",
         "t:1",
         "t:2"
      ],
      "ndarray" : [
         [
            0.000698519453116284,
            0.00366803903943576,
            0.995633441507448
         ]
      ]
   }
}

Deploy your custom model using language wrappers

For more custom deep learning and machine learning use-cases which have custom dependencies (such as 3rd party libraries, operating system binaries or even external systems), we can use any of the Seldon Core language wrappers.

You only have to write a class wrapper that exposes the logic of your model; for example in Python we can create a file Model.py:

import pickle
class Model:
    def __init__(self):
        self._model = pickle.loads( open("model.pickle", "rb") )

    def predict(self, X):
        output = self._model(X)
        return output

We can now containerize our class file using the Seldon Core s2i utils to produce the sklearn_iris image:

s2i build . seldonio/seldon-core-s2i-python3:0.18 sklearn_iris:0.1

And we now deploy it to our Seldon Core Kubernetes Cluster:

$ kubectl apply -f - << END
apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
  name: iris-model
  namespace: model-namespace
spec:
  name: iris
  predictors:
  - componentSpecs:
    - spec:
      containers:
      - name: classifier
        image: sklearn_iris:0.1
  - graph:
      name: classifier
    name: default
    replicas: 1
END

Send API requests to your deployed model

Every model deployed exposes a standardised User Interface to send requests using our OpenAPI schema.

This can be accessed through the endpoint http://<ingress_url>/seldon/<namespace>/<model-name>/api/v1.0/doc/ which will allow you to send requests directly through your browser.

Or alternatively you can send requests programmatically using our Seldon Python Client or another Linux CLI:

$ curl -X POST http://<ingress>/seldon/model-namespace/iris-model/api/v1.0/predictions \
    -H 'Content-Type: application/json' \
    -d '{ "data": { "ndarray": [1,2,3,4] } }' | json_pp

{
   "meta" : {},
   "data" : {
      "names" : [
         "t:0",
         "t:1",
         "t:2"
      ],
      "ndarray" : [
         [
            0.000698519453116284,
            0.00366803903943576,
            0.995633441507448
         ]
      ]
   }
}

Dive into the Advanced Production ML Integrations

Any model that is deployed and orchestrated with Seldon Core provides out of the box machine learning insights for monitoring, managing, scaling and debugging.

Below are some of the core components together with link to the logs that provide further insights on how to set them up.


Standard and custom metrics with prometheus


Full audit trails with ELK request logging


Explainers for Machine Learning Interpretability


Outlier and Adversarial Detectors for Monitoring


CI/CD for MLOps at Massive Scale


Distributed tracing for performance monitoring

Where to go from here

Getting Started

Seldon Core Deep Dive

Pre-Packaged Inference Servers

Language Wrappers (Production)

Language Wrappers (Incubating)

Ingress

Production

Advanced Inference

Examples

Reference

Developer

About the name "Seldon Core"

The name Seldon (ˈSɛldən) Core was inspired from the Foundation Series (Scifi Novel) where it's premise consists of a mathematician called "Hari Seldon" who spends his life developing a theory of Psychohistory, a new and effective mathematical sociology which allows for the future to be predicted extremely accurate through long periods of time (across hundreds of thousands of years).