Repository navigation
[qa] Replicate Kafka test topics so that they survive a broker restart - #7894
Conversation
A test topic had one replica, so whenever the CI cluster restarted the broker that held it, the topic had no leader and the test's producer flush timed out. Topics now take the broker's default replication factor, 3 on the CI cluster and 1 on a single broker. The tests' producers become idempotent, because the retries that follow a leader change can otherwise write a record twice, and the tests count records. Fixes #7550 Signed-off-by: Leonid Ryzhyk <ryzhyk@gmail.com>
|
What I ran
missing (medium): one Kafka test still creates topics with a single replica. - "replication_factor": 1,
+ "replication_factor": BROKER_DEFAULT_REPLICATION_FACTOR,(with Gates: The PR only changes tests, so there is nothing for docs, breaking changes or dependencies to cover. The commit message is good. The manual test plan is empty, which is acceptable here because a broker restart can't easily be reproduced by hand. Otherwise LGTM. |
Variant 1 of TestKafkaAvro still created its topics with one replica, which a broker restart leaves without a leader (#7550). CI deselects the test for now, so this matters once it runs again. Signed-off-by: Leonid Ryzhyk <ryzhyk@gmail.com>
The setup read the variant id where it expected a topic's creation date, so it deleted every kafka_avro topic on the cluster, including those of runs still in progress against it, such as the other architecture's job. It now reads the date that each topic name carries. Signed-off-by: Leonid Ryzhyk <ryzhyk@gmail.com>
A test topic had one replica, so whenever the CI cluster restarted the broker that held it, the topic had no leader and the test's producer flush timed out.
Topics now take the broker's default replication factor, 3 on the CI cluster and 1 on a single broker. The tests' producers become idempotent, because the retries that follow a leader change can otherwise write a record twice, and the tests count records.
Linked Issue
Fixes #7550
(the original CI failure was apparently cause by a strimzi bug, but it's nevertheless a good idea to harden tests against broker failures)
Describe Manual Test Plan
Checklist
Breaking Changes?
Mark if you think the answer is yes for any of these components:
Describe Incompatible Changes