Skip to content

[Bug] Dubbo ThreadPoolRejectMetric & ErrorCodeMetric cannot be exported to Prometheu #16148

@huisman6

Description

Pre-check

  • I am sure that all the content I provide is in English.

Search before asking

  • I had searched in the issues and found no similar issues.

Apache Dubbo Component

Java SDK (apache/dubbo)

Dubbo Version

Dubbo 3.3.6
SpringBoot 3.4.13
micrometer-registry-prometheus 1.14.14

Steps to reproduce this issue

Description

Subclasses of MetricsNameCountSampler, including ThreadRejectMetricsCountSampler and ErrorCodeSampler, do not support exporting and registering metrics that are newly added during runtime to Prometheus.

Currently, Dubbo's MetricsReporter periodically exports metrics by polling resetIfSamplesChanged. When new metrics are dynamically generated, they are usually re-registered to the CompositeMeterRegistry.

For the Dubbo Provider's ThreadPoolExhaustedListener and error codes, the metric data is generally empty during the application startup phase.

After the application runs for a while, corresponding events are generated, and MetricsCountSampler.inc is called to record the relevant metrics.

However, when these new metrics are added, the samplesChanged field in MetricsNameCountSampler is not updated, which fails to trigger the re-registration of the metrics to the PrometheusRegistry.

Steps to reproduce are as follows:

  1. Create a Dubbo 3.3.6 project.
  2. Adjust the configuration:
   dubbo.metrics.enable-threadpool: true
   dubbo.metrics.use-global-registry: true
   dubbo.protocol.threads: 2
  1. Implement a Dubbo Service using the Triple protocol, with an internal sleep of 60 seconds.
  2. After the Dubbo Provider starts successfully, wait for 3 minutes, then make an HTTP call to the Dubbo Service to trigger thread pool exhaustion.
  3. Access the Spring Boot management endpoint /actuator/prometheus and search for the metric: dubbo_thread_pool_reject_thread_count , but it cannot be found

What you expected to happen

Expected result:
Dynamically added metrics in MetricsNameCountSampler can be correctly exported to the PrometheusRegistry , such as dubbo_thread_pool_reject_thread_count and dubbo_error_code_count_total.

Anything else

No response

Do you have a (mini) reproduction demo?

  • Yes, I have a minimal reproduction demo to help resolve this issue more effectively!

Are you willing to submit a pull request to fix on your own?

  • Yes I am willing to submit a pull request on my own!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Status

    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions