Skip to content

2.2.0 / 2021-12-17
Compare
Choose a tag to compare
@github-actions github-actions released this 17 Dec 22:08
· 152 commits to master since this release
v2.2.0
d0d9e9d

Changelog

  • d0d9e9d Make E2E service accept a prometheus registerer
  • 2b50411 use right nil type to fix partition reassignments
  • 236d68c Validate number of specified brokers
  • 49300f0 Order imports
  • 63d501b Handle other protocolTypes than consumer properly
  • 461321b Lower produce request timeout to 3s
  • 5fdb775 Eagerly initialize failed metrics
  • ffa4211 Fix race condition in message tracker
  • a082b4e Fix typo in enum variable name
  • 059131a [Auto Commit] Package helm chart
  • 5adc178 Release v0.8.2
  • 8e169a7 Fix indenting
  • 0236e2e [Auto Commit] Package helm chart
  • 114cdb8 Release v0.8.1
  • 7ccca22 Fix broken format
  • f94e25b [Auto Commit] Package helm chart
  • d6a8030 Bump chart version
  • 41127e4 Add readiness probe for newer KMinion versions
  • fcd8c0c Only add consume config for appropriate scrapeMode
  • c1e922e Move comment into switch cases
  • fb5f4d4 Remove line break to improve readability
  • 62b1b51 Fix error handling for topic metadata response
  • 9960755 Merge pull request #129 from Scandiravian/handle_UnknownTopicOrPartition_error
  • 776cc2d Use typed error
  • c0f7ac6 Create topic instead of panic, when error is UnknownTopicOrPartition
  • 9162d2f Consider errors in metadata response for topic check
  • 26b2314 Merge pull request #128 from alenkacz/av/send-msg-immediately
  • 7c2e82e Send first e2e init message immediately
  • 21e8a6b Update to franz-go v1.2.2
  • d2b9046 Merge pull request #127 from alenkacz/av/helm-install
  • 357f8e5 Drop --name in helm install readme
  • 64d41d5 Merge pull request #124 from amuraru/pod-target-labels
  • d3016f8 Propage kminion pod custom labels as metrics labels
  • 8a393eb [Auto Commit] Package helm chart
  • 2ed9d5a Release version 0.7.0
  • e0d6dc5 Merge pull request #122 from stoader/metric_relabeling
  • a09adb0 Default value for serviceMonitor.relabelings
  • 0223754 Support metric relabeling
  • f5c6af4 [Auto Commit] Package helm chart
  • 877e893 Release chart version 0.6.0
  • 1dd4368 Merge pull request #119 from mrandreyeff/add_daemonset
  • 1dd0306 Merge pull request #105 from amuraru/issue-103
  • ed79c19 update according to changes in values
  • 4518838 change daemonset values structure
  • 3f84442 Merge pull request #120 from rashid4lyf/feature/add_scrape_timeout_interval
  • 89ea1c6 Merge remote-tracking branch 'origin/master' into feature/add_scrape_timeout_interval
  • 430d647 update values to 10s and 15s
  • af4fb49 Merge pull request #118 from mrandreyeff/add_targetLabels
  • 00eb355 Extract consumer group member_count as a separate metric
  • 7cbbc7d fix indent
  • 66e3435 support for dynamic scrapeTimeout and interval
  • caba0ad add daemonset template
  • 5536165 add serviceMonitor.targetLabels
  • dff9857 Merge pull request #106 from amuraru/issue-104
  • 4e87969 Implemented review comments
  • 320627b Export number of members and assigned partitions for each topic in a consumer group
  • 30f84f5 Fixes #111
  • 1e3a1d5 Introduce a state property that tracks whether a msg has been produced successfully or not
  • 7b6bd7e Fix partition reassignment and creation
  • a3558e0 Change default for commitSla to 5s
  • c4e5e82 Move to new cache lib
  • 02dbc3e Update franz-go to v0.10.2
  • f10fa9e Remove unused template literal
  • 01ff757 Use oneliner to access the Kafka message keys
  • 6901e62 Do not try to create or modify the topic if management is disabled
  • d38052f Fix offset commit metrics
  • ba20914 Use named loggers everywhere
  • 1545c20 Add clarifying log message during consumer init
  • f6ab00e Add timeout for consumer initialization
  • 6f76df2 Fix consumer initialization process for empty topics
  • 020c680 Make all message metrics vectors with partition id as label
  • 08dff9b Don't manage topics if not enabled
  • 7e882f2 Wait for a fully initialized consumer before starting the producer
  • 326337f Consume from newest offset
  • f6b7b3f Add messages to messagetracker before getting produce response
  • 3d59e90 Update franz-go to v0.9.1
  • d452033 Add partitionID in labels
  • 5e11446 Fix bug where we marked successfully received messages as lost
  • 634347d Rename metrics variable names
  • ded85bf Update to latest franz-go version
  • 4285a76 Add timeout for producing records
  • d0348bd Add timeout for producing records
  • e699e83 Add counter for lost messages
  • 244567c Move end to end message struct into it's own file
  • a2daa17 Refactor produce methods so that it is no longer sequential
  • 778e87c Remove TODO in regards to a metric tracking the failed produces
  • 1c46b56 Add metrics for produced messages
  • c0b7922 [Auto Commit] Package helm chart
  • 2d3d03f Release version 0.5.0
  • 01843a9 Add PodDisruptionBudget resource
  • 7381f6b Merge pull request #98 from amuraru/hlm
  • ec2c5de Tidy go mod
  • 52bb5f2 Merge pull request #109 from amuraru/update-deps
  • eddb2b8 Use consistent naming convention for log fields
  • d6048fd Update franz-go library to 0.9.0
  • aeb3b12 [helm-chart] Append customLabels to all resources if defined
  • a234a98 Document ensureCompatibility method
  • 5a7a20a Merge pull request #101 from amuraru/cached-groups
  • f96bc3a Introduce /ready endpoint that can be used in a readiness or startup probe
  • b6b6b4c Use cached consumer groups when listing offsets via Admin API
  • 63cba76 Merge pull request #97 from amuraru/moretopicconfigs
  • 032d501 Update minion/config_topic_config.go
  • a8a95ac Update minion/config_topic_config.go
  • 4cbfe01 Update minion/config_topic_config.go
  • 77249c3 Allow to export configurable set of topic configuration keys
  • 1a777bf Refactor some end-to-end functionality
  • 8157df5 Update to franz-go v0.8.0
  • 1ffd02b Print aggregated error summary
  • 3c7cfad Add partitions per broker default of 1
  • 01bedfc Fix reference config and change some e2e defaults
  • ec31cfd Merge branch 'master' of https://github.com/cloudhut/kminion
  • ce0423c update reference config
  • b435183 update reference config
  • 20bd38d Clarify what validateManagementTopic() does
  • 8d8c683 Use key from generated kmsg instead of comparing key name
  • 423e1c0 Merge branch 'endtoend'
  • 17ec4dd Update franz-go to v0.8.0
  • 0c33e52 Merge pull request #95 from Trackunit/feature/document-scrapemode
  • be3686c Also mention the increase in CPU usage when using scrapemode=offsetsTopic
  • 4305998 Document the difference between scrapemodes
  • 4948761 Fix formatting of e2e monitoring metrics
  • 5982e2c Fix some typos and reword a few things
  • ea47f26 Merge branch 'master' of https://github.com/cloudhut/kminion
  • 23a4d9a Update franz-go to v0.7.9
  • 10ed356 add e2e feature documentation, updated readme with link
  • ec428c0 fix issues when with logging string pointers
  • 390cf6f multiple changes: - remove rebalancingProtocol config, since there will never be any rebalancing in kminions consumer groups (since every kminion instance has exactly one, unique, consumer group that it uses) - change type of 'requiredAcks' from int to string (now takes 'all' or 'leader') - lots of functions have been refactored to use "early return" and/or split into logical steps so there aren't any huge functions - 'createEndToEndRecord' doesn't return an error anymore and instead panics. if serialization fails, there must be an issue that prevents kminion from running anyway - actually set 'ProduceRequestTimeout' to 'ackSla' config property - topic.go: complete rewrite: properly validate partition count, partition assignments, replication factor; considering potentially changing broker/partition count and replicationFactor; also now error checking for all potential errors in kafka responses (including nested errors, that are for example local to only one partition)
  • e262749 Merge pull request #90 from JoschaLaubach/fix-link-in-charts-docs
  • 405dbc5 docs: Fix link in Helm chart readme
  • 192bf30 message tracker, better comments, use time.Now() everywhere (no point in dealing with millis)
  • 3754be9 custom partitioner to send messages to predetermined partitions
  • 716f3e3 rename metric 'messages_commited_total' to 'commits_total' (since we're not comitting messages, but our "read offsets" per partition)
  • 0546a23 use hooks to track which broker sent a response to our offset commit
  • 6f7eb6d use new error reporting pattern in Produce after franz-go update, report broker_id instead of address in metrics
  • 9db7976 update packages, update to latest franz-go version, fix breaking changes
  • c2df9de rename "image-on-push" to "image-on-push-to-master", and create a new workflow that only builds images for pushes to any other (non-master) branch and tagging the image as "branchName-gitSha"
  • e1a8788 put deleting old consumer groups into its own file (group_tracker.go)
  • e36eb71 each kminion instance uses its own consumer group for end-to-end now so the instances don't "steal" messages from each other
  • d5a116c remove bytesSent/Received metrics from end to end
  • b4a08c5 more refactoring, seperate client hooks for each package
  • 1349666 move end-to-end into its own package called e2e
  • cc9a4e9 - remove SLA gauge metrics, they can be inferred from the 'inifinite' bucket in the latency histograms - methods to handle metric events (onAck, onRoundtrip, onOffsetCommit); will also soon be used to extract e2e into its own package - add _total and _seconds suffixes to metrics for best practices
  • a1ca198 update e2e topic config
  • 9813ad2 remove old methods; extract 'processMessage' for a small cleanup
  • c8e360a many changes: - new metrics (in 3 groups): low level, SLA indicators, latency histograms - don't put metrics results in a cache, instead directly increment/observe/set on promauto objects - only generate minionID once, don't create a new one for every message - actually check minionID to filter messages from other kminion instances - check timestamp to discard old messages and establish roundtrip latency - keep track of last received message timestamp, so we can detect/skip cases where we receive older messages for whatever reason - don't call storage.markRecordConsumed since that's only for offset_consumer
  • 4ae6088 log which partitions are lagging
  • b8c324d update franz-go to v0.6.14
  • be77e13 gitignore __debug_bin
  • 9bc99a0 typo in ReconciliationInterval
  • 9c14fca Merge pull request #86 from rjmasikome/master
  • 2cb857b Config adjustment
  • 7276c6e [Auto Commit] Package helm chart
  • f362bea Release v0.4.0
  • 2ed9e87 Merge pull request #85 from amuraru/ns
  • 4f4804b Add namespace metadata to all k8s resource definitions
  • 4aa5344 Message Loss Rate and config validations
  • 72a3df7 More robust checking on end to end topic, config validation, manual consumer commit, better label and help on metrics, and key on produce
  • c0b0695 Better commit latency metrics
  • 12bc74d EnToEnd Consumer Availability
  • 1069e39 Validate topic and commit metrics
  • a3befc7 EnToEnd Producer Availability
  • 3bac213 EndToEnd Latencies
  • f1f9ff5 Adding skeleton for endtoend feature
Assets 8
Loading