Observability with Prometheus and Grafana

Understand Edge Cases and Special Labels

When using relabeling in Prometheus, it's important to understand several edge cases and special internal labels that affect scraping and target behavior.

Common Edge Cases

• Missing __address__: If the __address__ label is missing, Prometheus will drop the target and no metrics will be scraped.

• Missing Port in __address__: If __address__ lacks a port, Prometheus automatically appends one based on the __scheme__ value. For http or an empty scheme, the port defaults to 80; for https, it defaults to 443.

• __address__ Contains a Slash: If the value of __address__ contains / (e.g., localhost:9090/), the target will be dropped because it’s not a valid host:port pair.

• Automatic Removal of Internal Labels: After user-defined relabeling rules are applied, Prometheus automatically removes all labels that start with double underscores (__). This includes labels such as __address__, __metrics_path__, and __scheme__, which are used internally and should not appear in the final metric set.

Special Labels Reference