-
Notifications
You must be signed in to change notification settings - Fork 10.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support using the ring hash balancer without xDS #33356
Comments
Mark, would you take a look at this? |
I've discussed this briefly with @ejona86 and @dfawley, and I think we'd be open to changing the ring_hash policy itself to provide a way to compute the hash for a request. We don't think it really makes sense to have a separate mechanism to define the hash in gRPC the way that it's done in xDS. I don't think we have time to work on this right now, but if you want to put together a gRFC and are willing to contribute an implementation, we'd be open to reviewing it. |
@markdroth I have a question related to this. A lot of our consistent hashing use cases make use of additional address metadata, instead of only the endpoint IP address as currently supported by gRPC. Envoy supports customizing the host hash key with the So in order to make the ring hash balancer to be truly useful to us, we would need:
I'm thinking that all of those may make sense to bundle as a single gRFC. Do you have a preference for the granularity of gRFCs? I see that they tend to bundle multiple related but somewhat independent features together. |
Bundling all of this into a single gRFC seems fine. I think the mechanism for setting the endpoint's hash key will be fairly straightforward; we can basically just define a resolver attribute for that. And I also don't see any problem with supporting an endpoint metadata key in EDS to set that attribute. However, I am a little concerned about the complexity involved in exposing an interface to programmatically set the request's hash. When we discussed this earlier, we were thinking that this would probably just be something simple like "use a specific request header", which could just be built into the ring_hash policy. Exposing an API to allow applications to do this programmatically is a much broader API surface, and before going down that road, I'd like to be convinced that there's no simpler way to do this. Is there a way that we can handle your use-case without that complexity? |
I think that your suggestion of extracting the key from a header covers our use case, which is to programatically set the hash key in the application, which can be done through the metadata interface available in all languages. So we could use a pre-defined We may also want to define hashing |
Sounds like we're aligned on using a header for the request hash. I'd be fine with adding a field to the ring_hash config to configure the header name to use. The I'm looking forward to seeing the gRFC for this! |
One last question regarding EDS LbMetadata. There are two options:
Option 1 minimizes the amount of information exposed, so any change in the details in the future would be a breaking change. 2 allows custom LB policies to take EDS endpoint metadata into consideration when picking an endpoint. I am asking this because we have Envoy based features that make use of this metadata through the Envoy LB subsets, and we would eventually like to support them in gRPC+xDS (note that Envoy LB subsets, somewhat confusingly, doesn't have much to do with subsets as proposed and discussed in the open A68). Today there is no way to even experiment with a custom LB policy that does this. This is because, at least in Go, EDS LbEndpoint metadata in gRPC are not extracted from EDS responses. Option 2. would be a good first step to support LB policies that use EDS endpoint metadata, and perhaps eventually supporting LB subset metdata. I initiated a discussion on this a few months ago in this grpc-io mailing list thread, which provides more context. I would prefer option 2, since we have a use case for it. I realize the fact that this issue mentions removing dependency on xDS, and now I'm talking about adding xDS specific features. This is because we are currently working on supporting gRPC in our xDS control plane but all existing usage use a custom resolver, so we have both use cases. |
We've discussed this a bit internally, and our consensus is that the Envoy-style subsetting (as opposed to the kind of subsetting discussed in A68) won't really work for gRPC, at least not without a tremendous amount of overhead. The reason Envoy-style subsetting works for Envoy is that Envoy is designed to completely decouple the LB policy from connection management: for each request, the LB policy is passed a list of endpoints to pick from, the pick is done independently of the current connectivity state of the endpoints, and then after the pick is done, it will look for a connection to the chosen endpoint, creating a new one if needed. Because the list of endpoints to pick from is an argument to the pick request, the subsetting policy can trivially filter the list of endpoints to pick from on each request. However, in gRPC, the LB policy itself manages endpoint connections, which allows the LB policy to never pick an endpoint for which a connection has not already been established, thus minimizing request latency. But this means that the pick request does not pass down the list of endpoints to choose from; the LB policy is expected to already know the list of endpoints it is choosing from, which it is given whenever the resolver returns a new result. This would mean that in order to implement xDS-style subsetting, the subsetting policy would basically have to create a duplicate child policy for each subset. In most gRPC implementations, this would result in a lot of duplicate connections to the same endpoint, and it would also use a lot of memory. We believe that the extra overhead here would be prohibitive in the general case. For the use-case described in the grpc.io thread that you started, I think that you will need to stick with the property that each subset is a separate xDS cluster, so that the problem can be modeled as finding a way to dynamically choose the cluster to use for a given request. There are some options we can explore for that, and we may even have a canned solution that you can use, but let's discuss that separately. Getting back to the ring_hash issue, I think it's fine to define a general-purpose resolver attribute for the hash key that can be used by any hash-based LB policy. I don't think we need to restrict it to be gettable only from the ring_hash policy; although we don't currently have any other policies that would use this attribute, I could see it possibly being useful for other hash-based policies in the future (either in a custom policy or some future gRPC-supported policy). For xDS, we can set that resolver attribute from the EDS |
I opened grpc/proposal#412 for the changes we discussed. I will follow up regarding LB metadata, but ideally not in this issue. |
It is currently not possible to use the
ring_hash
balancer without xDS. As far as I can tell (from the Go code and the A42 gRFC), there is no strong ties between xDS and thering_hash
balancer: the only missing piece to use it directly via service config is that there is no public interface to set the hash code in the request context (or equivalent, e.g.CallConfig
in c-core andCallOption
in Java).So the main thing to define would be the interface to set the hash on a per-request basis, across language. That may warrant another gRFC.
The alternative for gRPC users that want to use consistent hashing is to either build an xDS infrastructure, or copy paste the
ring_hash
balancer code just to expose a way to set the hash, which is not ideal.The text was updated successfully, but these errors were encountered: