Strimzi is unable to deploy Kafka with LoadBalancer

[TimothySutton81]

Environmental Info:
K3s Version:
$ k3s --version
k3s version v1.28.4+k3s2 (6ba6c1b)
go version go1.20.11

Node(s) CPU architecture, OS, and Version:
uname -a
Linux 5.4.17-2136.321.4.1.el7uek.x86_64 #2 SMP Wed Jul 19 14:51:35 PDT 2023 x86_64 x86_64 x86_64 GNU/Linux

Cluster Configuration:
3 Servers, 6 Agents

Describe the bug:
The bug is a strimzi loadbalancer integration issue with the k3s loadbalancer.
When you configure strimzi to have a loadbalancer installed in k3s because of the way the LoadBalancer works we end up with a loadbalancer stuck in the pending state always. This has been brought up in strimzi converstion here strimzi/strimzi-kafka-operator#2381

Not sure what the best solution for resolving this or deconflicting LoadBalancers requesting the same port is.

[brandond]

Not sure what the best solution for resolving this or deconflicting LoadBalancers requesting the same port is.

Either request different ports, create multiple loadbalancer pools, or don't use servicelb.

https://docs.k3s.io/networking/networking-services#creating-servicelb-node-pools