Reference: PodMonitor CRD
A PodMonitor describes how to scrape Prometheus metrics directly from pods. It
selects pods by label and lists the named container ports to scrape, along with
the path, interval, authentication and relabeling rules for each port. Use a
PodMonitor when your pods are not behind a Service, or when you want to
scrape a port that the service does not expose. If your application already has
a Service, a ServiceMonitor is usually simpler.
- API version:
monitoring.coreos.com/v1
- Kind:
PodMonitor
- Scope: Namespaced
- Short name:
pmon
This page documents Prometheus Operator v0.95.1.
Example
The following example scrapes the /metrics endpoint on the http container
port of all pods labeled app: your-application every 30 seconds. The port name
matches the container port in the Deployment from the getting started
guide.
apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
name: your-application
namespace: your-application-namespace
spec:
selector:
matchLabels:
app: your-application
podTargetLabels:
- app
podMetricsEndpoints:
- port: http
path: /metrics
interval: 30s
scrapeTimeout: 10s
Specification (spec)
| Field |
Type |
Description |
selector |
LabelSelector |
Required. Selects the Pod objects to scrape. See spec.selector. |
podMetricsEndpoints |
[]object |
Optional. The pod ports to scrape and how to scrape them. See spec.podMetricsEndpoints[]. |
namespaceSelector |
object |
Optional. The namespaces in which to discover pods. Defaults to the namespace of the PodMonitor. See spec.namespaceSelector. |
jobLabel |
string |
Optional. The name of a label on the Pod whose value becomes the job label of all metrics. If you leave it empty, job defaults to <namespace>/<name> of the PodMonitor. |
podTargetLabels |
[]string |
Optional. Labels to copy from the Pod onto the ingested metrics. |
attachMetadata |
object |
Optional. Additional metadata to attach to discovered targets. See spec.attachMetadata. |
sampleLimit |
integer |
Optional. Per-scrape limit on the number of samples accepted. Minimum 0. |
targetLimit |
integer |
Optional. Limit on the number of scraped targets accepted. Minimum 0. |
labelLimit |
integer |
Optional. Per-scrape limit on the number of labels accepted for a sample. Minimum 0. |
labelNameLengthLimit |
integer |
Optional. Per-scrape limit on the length of label names accepted for a sample. Minimum 0. |
labelValueLengthLimit |
integer |
Optional. Per-scrape limit on the length of label values accepted for a sample. Minimum 0. |
bodySizeLimit |
string |
Optional. Job-level limit on the uncompressed size of a scrape response, for example 10MiB. 0 means no limit. |
keepDroppedTargets |
integer |
Optional. Per-scrape limit on the number of targets dropped by relabeling that are kept in memory. 0 means no limit. |
scrapeProtocols |
[]string |
Optional. Protocols to negotiate during a scrape, in order of preference. Allowed values: PrometheusProto, OpenMetricsText0.0.1, OpenMetricsText1.0.0, PrometheusText0.0.4, PrometheusText1.0.0. If unset, the scraper uses its default. |
fallbackScrapeProtocol |
string |
Optional. Protocol to use if a scrape returns a blank, unparseable or otherwise invalid Content-Type. Allowed values are the same as for scrapeProtocols. |
scrapeNativeHistograms |
boolean |
Optional. Enables scraping of native histograms. |
scrapeClassicHistograms |
boolean |
Optional. Scrapes the classic version of a histogram that is also exposed as a native histogram. |
convertClassicHistogramsToNHCB |
boolean |
Optional. Converts all scraped classic histograms into native histograms with custom buckets. |
nativeHistogramBucketLimit |
integer |
Optional. Maximum number of buckets in a native histogram. Buckets are merged to stay within the limit. Minimum 0. |
nativeHistogramMinBucketFactor |
int-or-string |
Optional. Minimum growth factor from one native histogram bucket to the next. Buckets are merged to reach the factor. |
extraScrapeMetrics |
boolean |
Optional. Stores the additional series scrape_timeout_seconds, scrape_sample_limit and scrape_body_size_bytes for each scrape, so you can see how close targets are to their limits. |
selectorMechanism |
string |
Optional. How pods are selected. Allowed values: RelabelConfig (default, filter discovered targets with relabeling) and RoleSelector (filter at discovery, which can be more efficient in large clusters). |
scrapeClass |
string |
Optional. The name of a scrape class to apply. Scrape classes are defined by the operator of the scraping instance. |
spec.selector
A standard Kubernetes LabelSelector with matchLabels
and matchExpressions. The selector matches labels on the Pod objects.
spec.namespaceSelector
| Field |
Type |
Description |
any |
boolean |
Optional. If true, discovers pods in all namespaces. |
matchNames |
[]string |
Optional. The list of namespace names in which to discover pods. |
| Field |
Type |
Description |
node |
boolean |
Optional. If true, attaches node metadata to discovered targets as __meta_kubernetes_node_* labels. These labels are not added to metrics automatically. Copy them with relabeling. Requires list and watch permissions on Node objects for the scraper. |
spec.podMetricsEndpoints[]
Each entry describes one port to scrape on the selected pods.
| Field |
Type |
Description |
port |
string |
Optional. The name of the container port to scrape. If no pod exposes a port with this name, no targets are discovered. Takes precedence over portNumber. |
portNumber |
integer |
Optional. The number of the container port to scrape. The pod must declare the port in its spec, or the target is dropped. Minimum 1. |
path |
string |
Optional. The HTTP path to scrape. Defaults to /metrics. |
scheme |
string |
Optional. The HTTP scheme. Allowed values: http, https (upper case is also accepted). |
params |
map[string][]string |
Optional. HTTP URL query parameters to add to the scrape request. |
interval |
string |
Optional. Scrape interval as a duration, for example 30s. Defaults to the global scrape interval. |
scrapeTimeout |
string |
Optional. Timeout after which the scrape fails, for example 10s. Defaults to the global scrape timeout, or to interval if that is shorter. Must not be greater than interval, or the resource is rejected. |
honorLabels |
boolean |
Optional. If true, keeps the metric's own labels when they collide with target labels. |
honorTimestamps |
boolean |
Optional. If true, keeps timestamps exposed by the target. |
trackTimestampsStaleness |
boolean |
Optional. Tracks staleness of metrics that carry an explicit timestamp. Has no effect if honorTimestamps is false. |
filterRunning |
boolean |
Optional. Drops pods that are not running (for example Failed or Succeeded) during discovery. Enabled when unset. |
followRedirects |
boolean |
Optional. Follows HTTP 3xx redirects. |
enableHttp2 |
boolean |
Optional. Set to false to disable HTTP/2. |
authorization |
object |
Optional. Credentials for the Authorization header. See spec.podMetricsEndpoints[].authorization. Mutually exclusive with basicAuth and oauth2. |
basicAuth |
object |
Optional. Basic authentication credentials. See spec.podMetricsEndpoints[].basicAuth. Mutually exclusive with authorization and oauth2. |
oauth2 |
object |
Optional. OAuth2 client credentials. See spec.podMetricsEndpoints[].oauth2. Mutually exclusive with authorization and basicAuth. |
tlsConfig |
object |
Optional. TLS settings for the scrape. See spec.podMetricsEndpoints[].tlsConfig. |
relabelings |
[]object |
Optional. Relabeling rules applied to target metadata before the scrape. See relabeling. The original job name is available in the __tmp_prometheus_job_name label. |
metricRelabelings |
[]object |
Optional. Relabeling rules applied to samples before ingestion. Use these to drop or rename series. See relabeling. |
proxyUrl |
string |
Optional. HTTP proxy to use. Must match ^(http\|https\|socks5)://.+$. |
noProxy |
string |
Optional. Comma-separated IPs, CIDRs and domain names that bypass the proxy. |
proxyFromEnvironment |
boolean |
Optional. Uses the HTTP_PROXY, HTTPS_PROXY and NO_PROXY environment variables. |
proxyConnectHeader |
map[string][]SecretKeySelector |
Optional. Headers to send to the proxy during CONNECT requests. Each value references keys in Secrets. |
spec.podMetricsEndpoints[].authorization
| Field |
Type |
Description |
type |
string |
Optional. The authentication type, case-insensitive. Defaults to Bearer. Basic is not supported, use basicAuth instead. |
credentials |
SecretKeySelector |
Optional. A key in a Secret in the same namespace that holds the credentials. See Secret and ConfigMap references. |
spec.podMetricsEndpoints[].basicAuth
| Field |
Type |
Description |
username |
SecretKeySelector |
Optional. A key in a Secret that holds the username. |
password |
SecretKeySelector |
Optional. A key in a Secret that holds the password. |
spec.podMetricsEndpoints[].oauth2
| Field |
Type |
Description |
tokenUrl |
string |
Required. The URL to fetch the token from. Must start with http:// or https://. |
clientId |
SecretOrConfigMap |
Required. A key in a Secret or ConfigMap that holds the client ID. |
clientSecret |
SecretKeySelector |
Required. A key in a Secret that holds the client secret. |
scopes |
[]string |
Optional. OAuth2 scopes for the token request. |
endpointParams |
map[string]string |
Optional. HTTP parameters to append to the token URL. |
tlsConfig |
object |
Optional. TLS settings for the connection to the token server. Same fields as spec.podMetricsEndpoints[].tlsConfig. |
proxyUrl |
string |
Optional. HTTP proxy for the token request. |
noProxy |
string |
Optional. Comma-separated IPs, CIDRs and domain names that bypass the proxy. |
proxyFromEnvironment |
boolean |
Optional. Uses the proxy environment variables. |
proxyConnectHeader |
map[string][]SecretKeySelector |
Optional. Headers to send to the proxy during CONNECT requests. |
spec.podMetricsEndpoints[].tlsConfig
| Field |
Type |
Description |
ca |
SecretOrConfigMap |
Optional. The certificate authority used to verify the target's certificate. |
cert |
SecretOrConfigMap |
Optional. The client certificate to present for client authentication. |
keySecret |
SecretKeySelector |
Optional. A key in a Secret that holds the client private key. |
serverName |
string |
Optional. The hostname used to verify the target's certificate. |
insecureSkipVerify |
boolean |
Optional. Disables verification of the target's certificate. |
minVersion |
string |
Optional. Minimum TLS version. Allowed values: TLS10, TLS11, TLS12, TLS13. |
maxVersion |
string |
Optional. Maximum TLS version. Allowed values: TLS10, TLS11, TLS12, TLS13. |
spec.podMetricsEndpoints[].relabelings[] and spec.podMetricsEndpoints[].metricRelabelings[]
Both lists use the Prometheus relabel_config format.
relabelings run on target metadata before the scrape, and metricRelabelings
run on each sample before ingestion.
| Field |
Type |
Description |
sourceLabels |
[]string |
Optional. Labels whose values are concatenated with separator and matched against regex. |
separator |
string |
Optional. The string placed between concatenated source label values. Prometheus defaults to ;. |
regex |
string |
Optional. The regular expression matched against the concatenated value. Prometheus defaults to (.*). |
targetLabel |
string |
Optional. The label to write the result to. Required for the replace, hashmod, lowercase, uppercase, keepequal and dropequal actions. Capture groups are available. |
replacement |
string |
Optional. The value written by a replace action if regex matches. Capture groups are available. Prometheus defaults to $1. |
modulus |
integer |
Optional. The modulus for the hashmod action. Minimum 0. |
action |
string |
Optional. The action to perform. Allowed values: replace, keep, drop, hashmod, labelmap, labeldrop, labelkeep, lowercase, uppercase, keepequal, dropequal. Capitalized forms such as Replace are also accepted. Defaults to replace. |
Secret and ConfigMap References
Credential fields use two reference types. Both must point to objects in the
same namespace as the PodMonitor.
SecretKeySelector has the fields name (the Secret name), key
(required, the key in the Secret) and optional (whether the Secret or
key may be missing). It is the same type as secretKeyRef in the
Kubernetes Pod API.
SecretOrConfigMap has two optional fields, secret (a
SecretKeySelector) and configMap (a ConfigMapKeySelector with the same
name, key and optional fields). Set one of them.
Deprecated Fields
The following fields are deprecated upstream and not documented above:
spec.podMetricsEndpoints[].targetPort. Use port or portNumber instead.
spec.podMetricsEndpoints[].bearerTokenSecret. Use authorization instead.
Status (status)
The Prometheus Operator writes status only when it runs with the
StatusForConfigurationResources feature gate enabled, which is off by
default upstream. The field can be empty.
| Field |
Type |
Description |
bindings[].resource |
string |
The type of workload that selected this resource, for example prometheuses or prometheusagents. |
bindings[].name |
string |
The name of the workload that selected this resource. |
bindings[].namespace |
string |
The namespace of that workload. |
bindings[].conditions[].type |
string |
The condition type. Only Accepted is supported. |
bindings[].conditions[].status |
string |
The condition status, for example True or False. |
bindings[].conditions[].reason |
string |
A short reason for the last transition. |
bindings[].conditions[].message |
string |
A human-readable explanation, for example why the operator rejected the resource. |
bindings[].conditions[].observedGeneration |
integer |
The metadata.generation the condition is based on. If it is lower than the current generation, the condition is out of date. |
- You can create
PodMonitor resources in your own namespaces.
- As an alternative to a
PodMonitor, you can annotate your pods with
prometheus.io/scrape, prometheus.io/path and prometheus.io/port, as
described in the getting started guide.
Further Reading