Reference: ServiceMonitor CRD¶
A ServiceMonitor describes how to scrape Prometheus metrics from the pods
behind one or more Kubernetes Service objects. It selects services by label
and lists the named service ports to scrape, along with the path, interval,
authentication and relabeling rules for each port. Use a ServiceMonitor when
your application already has a Service. If it does not, use a
PodMonitor instead.
- API version:
monitoring.coreos.com/v1 - Kind:
ServiceMonitor - Scope: Namespaced
- Short name:
smon
This page documents Prometheus Operator v0.95.1.
Example¶
The following example scrapes the /metrics endpoint on the http port of the
your-application service every 30 seconds. It also copies the app label from
the service onto every scraped series and drops Go runtime metrics before
ingestion.
apiVersion: v1
kind: Service
metadata:
name: your-application
namespace: your-application-namespace
labels:
app: your-application
spec:
selector:
app: your-application
ports:
- name: http
port: 8080
targetPort: http
---
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: your-application
namespace: your-application-namespace
spec:
selector:
matchLabels:
app: your-application
targetLabels:
- app
endpoints:
- port: http
path: /metrics
interval: 30s
scrapeTimeout: 10s
metricRelabelings:
- sourceLabels: [__name__]
regex: "go_.*"
action: drop
Specification (spec)¶
| Field | Type | Description |
|---|---|---|
selector |
LabelSelector |
Required. Selects the Service objects whose endpoints are scraped. See spec.selector. |
endpoints |
[]object |
Required. The service ports to scrape and how to scrape them. See spec.endpoints[]. |
namespaceSelector |
object |
Optional. The namespaces in which to discover services. Defaults to the namespace of the ServiceMonitor. See spec.namespaceSelector. |
jobLabel |
string |
Optional. The name of a label on the Service whose value becomes the job label of all metrics. If you leave it empty, or the service does not have the label, job defaults to the service name. |
targetLabels |
[]string |
Optional. Labels to copy from the Service onto the ingested metrics. |
podTargetLabels |
[]string |
Optional. Labels to copy from the Pod onto the ingested metrics. |
attachMetadata |
object |
Optional. Additional metadata to attach to discovered targets. See spec.attachMetadata. |
sampleLimit |
integer |
Optional. Per-scrape limit on the number of samples accepted. Minimum 0. |
targetLimit |
integer |
Optional. Limit on the number of scraped targets accepted. Minimum 0. |
labelLimit |
integer |
Optional. Per-scrape limit on the number of labels accepted for a sample. Minimum 0. |
labelNameLengthLimit |
integer |
Optional. Per-scrape limit on the length of label names accepted for a sample. Minimum 0. |
labelValueLengthLimit |
integer |
Optional. Per-scrape limit on the length of label values accepted for a sample. Minimum 0. |
bodySizeLimit |
string |
Optional. Job-level limit on the uncompressed size of a scrape response, for example 10MiB. 0 means no limit. |
keepDroppedTargets |
integer |
Optional. Per-scrape limit on the number of targets dropped by relabeling that are kept in memory. 0 means no limit. |
scrapeProtocols |
[]string |
Optional. Protocols to negotiate during a scrape, in order of preference. Allowed values: PrometheusProto, OpenMetricsText0.0.1, OpenMetricsText1.0.0, PrometheusText0.0.4, PrometheusText1.0.0. If unset, the scraper uses its default. |
fallbackScrapeProtocol |
string |
Optional. Protocol to use if a scrape returns a blank, unparseable or otherwise invalid Content-Type. Allowed values are the same as for scrapeProtocols. |
scrapeNativeHistograms |
boolean |
Optional. Enables scraping of native histograms. |
scrapeClassicHistograms |
boolean |
Optional. Scrapes the classic version of a histogram that is also exposed as a native histogram. |
convertClassicHistogramsToNHCB |
boolean |
Optional. Converts all scraped classic histograms into native histograms with custom buckets. |
nativeHistogramBucketLimit |
integer |
Optional. Maximum number of buckets in a native histogram. Buckets are merged to stay within the limit. Minimum 0. |
nativeHistogramMinBucketFactor |
int-or-string |
Optional. Minimum growth factor from one native histogram bucket to the next. Buckets are merged to reach the factor. |
extraScrapeMetrics |
boolean |
Optional. Stores the additional series scrape_timeout_seconds, scrape_sample_limit and scrape_body_size_bytes for each scrape, so you can see how close targets are to their limits. |
serviceDiscoveryRole |
string |
Optional. The Kubernetes service discovery role. Allowed values: Endpoints, EndpointSlice. Defaults to the value set on the scraping instance. |
selectorMechanism |
string |
Optional. How endpoints are selected. Allowed values: RelabelConfig (default, filter discovered targets with relabeling) and RoleSelector (filter at discovery, which can be more efficient in large clusters). |
scrapeClass |
string |
Optional. The name of a scrape class to apply. Scrape classes are defined by the operator of the scraping instance. |
spec.selector¶
A standard Kubernetes LabelSelector with matchLabels
and matchExpressions. The selector matches labels on the Service object, not
on the pods.
spec.namespaceSelector¶
| Field | Type | Description |
|---|---|---|
any |
boolean |
Optional. If true, discovers services in all namespaces. |
matchNames |
[]string |
Optional. The list of namespace names in which to discover services. |
spec.attachMetadata¶
| Field | Type | Description |
|---|---|---|
node |
boolean |
Optional. If true, attaches node metadata to discovered targets as __meta_kubernetes_node_* labels. These labels are not added to metrics automatically. Copy them with relabeling. Requires list and watch permissions on Node objects for the scraper. |
spec.endpoints[]¶
Each entry describes one port to scrape on the selected services.
| Field | Type | Description |
|---|---|---|
port |
string |
Optional. The name of the service port to scrape (.spec.ports[].name on the Service). Takes precedence over targetPort. |
targetPort |
int-or-string |
Optional. The name or number of a container port on the pods behind the service. A name matches .spec.containers[].ports[].name, a number matches .spec.containers[].ports[].containerPort. Endpoints that are not backed by pods are never scraped when you use this field. |
path |
string |
Optional. The HTTP path to scrape. Defaults to /metrics. |
scheme |
string |
Optional. The HTTP scheme. Allowed values: http, https (upper case is also accepted). |
params |
map[string][]string |
Optional. HTTP URL query parameters to add to the scrape request. |
interval |
string |
Optional. Scrape interval as a duration, for example 30s. Defaults to the global scrape interval. |
scrapeTimeout |
string |
Optional. Timeout after which the scrape fails, for example 10s. Defaults to the global scrape timeout, or to interval if that is shorter. Must not be greater than interval, or the resource is rejected. |
honorLabels |
boolean |
Optional. If true, keeps the metric's own labels when they collide with target labels. |
honorTimestamps |
boolean |
Optional. If true, keeps timestamps exposed by the target. |
trackTimestampsStaleness |
boolean |
Optional. Tracks staleness of metrics that carry an explicit timestamp. Has no effect if honorTimestamps is false. |
filterRunning |
boolean |
Optional. Drops pods that are not running (for example Failed or Succeeded) during discovery. Enabled when unset. |
followRedirects |
boolean |
Optional. Follows HTTP 3xx redirects. |
enableHttp2 |
boolean |
Optional. Set to false to disable HTTP/2. |
authorization |
object |
Optional. Credentials for the Authorization header. See spec.endpoints[].authorization. Mutually exclusive with basicAuth and oauth2. |
basicAuth |
object |
Optional. Basic authentication credentials. See spec.endpoints[].basicAuth. Mutually exclusive with authorization and oauth2. |
oauth2 |
object |
Optional. OAuth2 client credentials. See spec.endpoints[].oauth2. Mutually exclusive with authorization and basicAuth. |
tlsConfig |
object |
Optional. TLS settings for the scrape. See spec.endpoints[].tlsConfig. |
relabelings |
[]object |
Optional. Relabeling rules applied to target metadata before the scrape. See relabeling. The original job name is available in the __tmp_prometheus_job_name label. |
metricRelabelings |
[]object |
Optional. Relabeling rules applied to samples before ingestion. Use these to drop or rename series. See relabeling. |
proxyUrl |
string |
Optional. HTTP proxy to use. Must match ^(http\|https\|socks5)://.+$. |
noProxy |
string |
Optional. Comma-separated IPs, CIDRs and domain names that bypass the proxy. |
proxyFromEnvironment |
boolean |
Optional. Uses the HTTP_PROXY, HTTPS_PROXY and NO_PROXY environment variables. |
proxyConnectHeader |
map[string][]SecretKeySelector |
Optional. Headers to send to the proxy during CONNECT requests. Each value references keys in Secrets. |
spec.endpoints[].authorization¶
| Field | Type | Description |
|---|---|---|
type |
string |
Optional. The authentication type, case-insensitive. Defaults to Bearer. Basic is not supported, use basicAuth instead. |
credentials |
SecretKeySelector |
Optional. A key in a Secret in the same namespace that holds the credentials. See Secret and ConfigMap references. |
spec.endpoints[].basicAuth¶
| Field | Type | Description |
|---|---|---|
username |
SecretKeySelector |
Optional. A key in a Secret that holds the username. |
password |
SecretKeySelector |
Optional. A key in a Secret that holds the password. |
spec.endpoints[].oauth2¶
| Field | Type | Description |
|---|---|---|
tokenUrl |
string |
Required. The URL to fetch the token from. Must start with http:// or https://. |
clientId |
SecretOrConfigMap |
Required. A key in a Secret or ConfigMap that holds the client ID. |
clientSecret |
SecretKeySelector |
Required. A key in a Secret that holds the client secret. |
scopes |
[]string |
Optional. OAuth2 scopes for the token request. |
endpointParams |
map[string]string |
Optional. HTTP parameters to append to the token URL. |
tlsConfig |
object |
Optional. TLS settings for the connection to the token server. Same fields as spec.endpoints[].tlsConfig, except caFile, certFile and keyFile. |
proxyUrl |
string |
Optional. HTTP proxy for the token request. |
noProxy |
string |
Optional. Comma-separated IPs, CIDRs and domain names that bypass the proxy. |
proxyFromEnvironment |
boolean |
Optional. Uses the proxy environment variables. |
proxyConnectHeader |
map[string][]SecretKeySelector |
Optional. Headers to send to the proxy during CONNECT requests. |
spec.endpoints[].tlsConfig¶
| Field | Type | Description |
|---|---|---|
ca |
SecretOrConfigMap |
Optional. The certificate authority used to verify the target's certificate. |
cert |
SecretOrConfigMap |
Optional. The client certificate to present for client authentication. |
keySecret |
SecretKeySelector |
Optional. A key in a Secret that holds the client private key. |
serverName |
string |
Optional. The hostname used to verify the target's certificate. |
insecureSkipVerify |
boolean |
Optional. Disables verification of the target's certificate. |
minVersion |
string |
Optional. Minimum TLS version. Allowed values: TLS10, TLS11, TLS12, TLS13. |
maxVersion |
string |
Optional. Maximum TLS version. Allowed values: TLS10, TLS11, TLS12, TLS13. |
caFile |
string |
Optional. Path to a CA certificate file inside the scraper container. |
certFile |
string |
Optional. Path to a client certificate file inside the scraper container. |
keyFile |
string |
Optional. Path to a client key file inside the scraper container. |
The file fields refer to files inside the scraper's container, which you do not
control. Use ca, cert and keySecret to reference Secrets or ConfigMaps in
your namespace instead.
spec.endpoints[].relabelings[] and spec.endpoints[].metricRelabelings[]¶
Both lists use the Prometheus relabel_config format.
relabelings run on target metadata before the scrape, and metricRelabelings
run on each sample before ingestion.
| Field | Type | Description |
|---|---|---|
sourceLabels |
[]string |
Optional. Labels whose values are concatenated with separator and matched against regex. |
separator |
string |
Optional. The string placed between concatenated source label values. Prometheus defaults to ;. |
regex |
string |
Optional. The regular expression matched against the concatenated value. Prometheus defaults to (.*). |
targetLabel |
string |
Optional. The label to write the result to. Required for the replace, hashmod, lowercase, uppercase, keepequal and dropequal actions. Capture groups are available. |
replacement |
string |
Optional. The value written by a replace action if regex matches. Capture groups are available. Prometheus defaults to $1. |
modulus |
integer |
Optional. The modulus for the hashmod action. Minimum 0. |
action |
string |
Optional. The action to perform. Allowed values: replace, keep, drop, hashmod, labelmap, labeldrop, labelkeep, lowercase, uppercase, keepequal, dropequal. Capitalized forms such as Replace are also accepted. Defaults to replace. |
Secret and ConfigMap References¶
Credential fields use two reference types. Both must point to objects in the
same namespace as the ServiceMonitor.
SecretKeySelectorhas the fieldsname(the Secret name),key(required, the key in the Secret) andoptional(whether the Secret or key may be missing). It is the same type assecretKeyRefin the Kubernetes Pod API.SecretOrConfigMaphas two optional fields,secret(aSecretKeySelector) andconfigMap(aConfigMapKeySelectorwith the samename,keyandoptionalfields). Set one of them.
Deprecated Fields¶
The following fields are deprecated upstream and not documented above:
spec.endpoints[].bearerTokenFile. Useauthorizationinstead.spec.endpoints[].bearerTokenSecret. Useauthorizationinstead.
Status (status)¶
The Prometheus Operator writes status only when it runs with the
StatusForConfigurationResources feature gate enabled, which is off by
default upstream. The field can be empty.
| Field | Type | Description |
|---|---|---|
bindings[].resource |
string |
The type of workload that selected this resource, for example prometheuses or prometheusagents. |
bindings[].name |
string |
The name of the workload that selected this resource. |
bindings[].namespace |
string |
The namespace of that workload. |
bindings[].conditions[].type |
string |
The condition type. Only Accepted is supported. |
bindings[].conditions[].status |
string |
The condition status, for example True or False. |
bindings[].conditions[].reason |
string |
A short reason for the last transition. |
bindings[].conditions[].message |
string |
A human-readable explanation, for example why the operator rejected the resource. |
bindings[].conditions[].observedGeneration |
integer |
The metadata.generation the condition is based on. If it is lower than the current generation, the condition is out of date. |
Platform Notes¶
- You can create
ServiceMonitorresources in your own namespaces. - As an alternative to a
ServiceMonitor, you can annotate your pods withprometheus.io/scrape,prometheus.io/pathandprometheus.io/port, as described in the getting started guide.