fix(docs): first draft of the securing endpoints

[Rajakavitha1]

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
2 out of 3 committers have signed the CLA.

✅ Rajakavitha1
✅ paulb-seldon
❌ Rakavitha Kodhandapani

---

Rakavitha Kodhandapani seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[paulb-seldon]

Have made an edit to add context to the beginning of the page.

Another suggestion I have is to add a bit more content about what we are doing ahead of each of the steps. Instead of saying 'to secure model endpoints' ahead of the steps, I think we need to expand a bit more (I think it's fine in the context / intro, but when we go more into the details, we should explain exactly what we are doing). I.e. what is the auth policy doing, what is the authentication policy as shown doing.

Also ahead of each step can we call out what the placeholders are maybe we just need that for $MESH_IP?

[Rajakavitha1]

Hi @paulb-seldon I just rephrased the paragraph to adhere to the writing guidelines.

[lc525]

We should probably show how to do this in a general way, not for one model. In particular, I propose adding a VirtualService example that forwards the REST/gRPC endpoints for all models in a namespace (in service-meshes/istio.md)

[Rajakavitha1]

Hi @lc525 . Yes, this is added as a prerequisite * [Create a virtual service to expose the REST and gRPC endpoints](/kubernetes/service-meshes/istio.md)

[lc525]

I think that perhaps I was unclear in my comment. All the examples in service-meshes/istio.md target exposing REST/gRPC endpoints for a particular model. I was suggesting an example where we expose all the models loaded into a namespace via one virtual service.

This is so that for people having hundreds of models and want to add auth, we don't suggest "well, add hundreds of virtual services, and keep them updated as you add or remove models".

[Rajakavitha1]

Thank you @lc525 . I will work with Chandra to get an example and update it.

[Rajakavitha1]

Hi @lc525 I just updated the example. Please let me know if the yaml file for virtual service is correct.

[lc525]

@Rajakavitha1 I don't think that example can work, and even if it did, it doesn't solve the initial point I've raised. It's possible I wasn't clear in what I was suggesting:

What I had in mind was to get towards something closer to this (untested!):

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: all-models-route
  namespace: seldon-mesh
spec:
  gateways:
  - istio-system/seldon-gateway
  hosts:
  - '*'
  http:
  - match:
    - uri:
        prefix: /v2
    name: models-http
    route:
    - destination:
        host: seldon-mesh.seldon-mesh.svc.cluster.local
  - match:
    - uri:
        prefix: /inference.GRPCInferenceService
    name: models-grpc
    route:
    - destination:
        host: seldon-mesh.seldon-mesh.svc.cluster.local

This is would be a catch-all VirtualService redirecting all REST requests coming with a /v2 path prefix (irrespective of the model) to our backend load balancer (named seldon-mesh), and similar for gRPC. It adds an additional requirement for the requests to already have the correct headers set.

Could you, perhaps together with Chandra where help is needed, test that all the examples in this PR work as expected (or have they already been tested?)