[improve][pip] PIP-409: support producer configuration for retry/dead…

… letter topic producer (apache#24022)

pip/pip-409.md

@@ -0,0 +1,67 @@
+
+# PIP-409: support producer configuration for retry/dead letter topic producer
+
+# Background knowledge
+
+Retry topic is a topic that stores messages that have failed to be processed by the consumer, or the consumer can't process the message at the moment. 
+The consumer can retry the message after a certain period of time. After several retries, the message will be moved to the dead letter topic.
+
+There is a retry topic producer in consumer instance. Each time the consumer call `consumer.reconsumeLater(msg, 3, TimeUnit.SECONDS);` 
+to retry the message after 3 seconds, the hidden producer will send the corresponding message to retry topic, and ack the message in original topic.
+
+After several retries, the message will be sent to dead letter topic instead of retry topic.
+
+
+# Motivation
+
+Currently, we don't support configure the producer of retry/dead letter topic. But enable the chunk message feature
+and disable the batch configuration in hard code, which can't handle many situations. For example, when the throughput 
+of message of retry topic become considerable, the resource consumed by the un-batched messages is pretty large. 
+There is no reason that we disable the batch message feature.
+
+For better control for the retry/dead letter topic feature, we can support configuration for the producer of 
+retry/dead letter topic.
+
+# Goals
+
+## In Scope
+
+- Support configuration for the producer of retry/dead letter topic.
+
+
+# Detailed Design
+
+## Design & Implementation Details
+
+- Add two new configurations in `DeadLetterPolicy`:
+```java
+public class DeadLetterPolicy implements Serializable {
+  /**
+   * Function to build the producer for the retry letter topic.
+   * The input parameter is the topic name.
+   */
+  private Function<String, ProducerBuilder<byte[]>> retryLetterProducerBuilder;
+
+  /**
+   * Function to build the producer for the dead letter topic.
+   * The input parameter is the topic name.
+   */
+  private Function<String, ProducerBuilder<byte[]>> deadLetterProducerBuilder;
+}
+```
+
+- use the `retryLetterProducerBuilder` to build the producer for retry topic, and use the
+`deadLetterProducerBuilder` to build the producer for dead letter topic.
+
+
+# Backward & Forward Compatibility
+
+Fully compatible.
+
+# Links
+
+<!--
+Updated afterwards
+-->
+* Mailing List discussion thread: https://lists.apache.org/thread/h6jjf9wn2h4zmpjw5zjtnl5ds1r4nknq
+* Mailing List voting thread: https://lists.apache.org/thread/6wgxovk0f72d10zdgsnto6bkh6pwfzj1