{
  "tags": [],
  "title": "K8s / Storage / Volumes / Cluster",
  "widgets": [
    {
      "layout": {
        "x": 0,
        "y": 0,
        "width": 12,
        "height": 2
      },
      "definition": {
        "type": "group",
        "title": "Current Alerts",
        "widgets": [
          {
            "layout": {
              "x": 0,
              "y": 0,
              "width": 3,
              "height": 1
            },
            "definition": {
              "type": "note",
              "content": "**Infrastructure Namespace Volumes Full in Week Based on Daily Use Rate**\n\nUnsupported panel (singlestat)\n\n- predict_linear() has no Datadog equivalent; the `and` set operator and the `< 0` comparison filter are also not expressible in Datadog formulas\n\nOriginal PromQL:\n```\nA: (\n  count (\n    (kubelet_volume_stats_available_bytes {namespace=~\"(openshift-.*|kube-.*|default|logging)\"})\n    and\n    (predict_linear(kubelet_volume_stats_available_bytes {namespace=~\"(openshift-.*|kube-.*|default|logging)\"}[1d], 7 * 24 * 60 * 60) < 0)\n  )\n)\nor\nvector(0)\n```",
              "tick_pos": "50%",
              "font_size": "14",
              "show_tick": false,
              "tick_edge": "left",
              "text_align": "left",
              "has_padding": true,
              "vertical_align": "top",
              "background_color": "white"
            }
          },
          {
            "layout": {
              "x": 3,
              "y": 0,
              "width": 4,
              "height": 1
            },
            "definition": {
              "type": "note",
              "content": "**User Namespace Volumes Full in Week Based on Daily Use Rate**\n\nUnsupported panel (singlestat)\n\n- predict_linear() has no Datadog equivalent; the `and` set operator and the `< 0` comparison filter are also not expressible in Datadog formulas\n\nOriginal PromQL:\n```\nA: (\n  count (\n    (\n      (kubelet_volume_stats_available_bytes {namespace!~\"(openshift-.*|kube-.*|default|logging)\"})\n    )\n    and\n    (predict_linear(kubelet_volume_stats_available_bytes[1d], 7 * 24 * 60 * 60) < 0)\n  )\n)\nor\nvector(0)\n```",
              "tick_pos": "50%",
              "font_size": "14",
              "show_tick": false,
              "tick_edge": "left",
              "text_align": "left",
              "has_padding": true,
              "vertical_align": "top",
              "background_color": "white"
            }
          },
          {
            "layout": {
              "x": 7,
              "y": 0,
              "width": 5,
              "height": 2
            },
            "definition": {
              "type": "note",
              "content": "**Dashboard Alerts**\n\nUnsupported panel (alertlist)\n\n- no Datadog equivalent for plugin alertlist\n\nOriginal PromQL:\n```\n(none)\n```",
              "tick_pos": "50%",
              "font_size": "14",
              "show_tick": false,
              "tick_edge": "left",
              "text_align": "left",
              "has_padding": true,
              "vertical_align": "top",
              "background_color": "white"
            }
          },
          {
            "layout": {
              "x": 0,
              "y": 1,
              "width": 3,
              "height": 1
            },
            "definition": {
              "type": "note",
              "content": "**Running PVCs Above % Used Warning Threshold**\n\nUnsupported panel (singlestat)\n\n- `and` set operator with a threshold comparison cannot be expressed in Datadog; counting series matching a threshold is not supported in a metric query\n\nOriginal PromQL:\n```\nA: count (max by (persistentvolumeclaim,namespace) (kubelet_volume_stats_used_bytes ) and (max by (persistentvolumeclaim,namespace) (kubelet_volume_stats_used_bytes )) / (max by (persistentvolumeclaim,namespace) (kubelet_volume_stats_capacity_bytes )) >= ($pvc_percent_used_warning_threshold / 100)) or vector (0)\n```",
              "tick_pos": "50%",
              "font_size": "14",
              "show_tick": false,
              "tick_edge": "left",
              "text_align": "left",
              "has_padding": true,
              "vertical_align": "top",
              "background_color": "white"
            }
          },
          {
            "layout": {
              "x": 3,
              "y": 1,
              "width": 4,
              "height": 1
            },
            "definition": {
              "type": "query_value",
              "title": "Unbound PVCs",
              "requests": [
                {
                  "queries": [
                    {
                      "name": "query1",
                      "query": "sum:pv_collector_unbound_pvc_count{*} by {exported_namespace}",
                      "aggregator": "last",
                      "data_source": "metrics"
                    }
                  ],
                  "formulas": [
                    {
                      "formula": "default_zero(query1)"
                    }
                  ],
                  "response_format": "scalar",
                  "conditional_formats": [
                    {
                      "value": 1,
                      "palette": "white_on_yellow",
                      "comparator": ">="
                    },
                    {
                      "value": 1,
                      "palette": "white_on_yellow",
                      "comparator": ">="
                    }
                  ]
                }
              ],
              "autoscale": true,
              "precision": 2,
              "text_align": "center"
            }
          }
        ],
        "show_title": true,
        "layout_type": "ordered"
      }
    },
    {
      "layout": {
        "x": 0,
        "y": 2,
        "width": 12,
        "height": 10
      },
      "definition": {
        "type": "group",
        "title": "Alert Details",
        "widgets": [
          {
            "layout": {
              "x": 0,
              "y": 0,
              "width": 3,
              "height": 4
            },
            "definition": {
              "type": "note",
              "content": "**Infrastructure Namespace Volumes Full in Week Based on Daily Use Rate - Current**\n\nUnsupported panel (table)\n\n- predict_linear() and the `and`/`< 0` filtering have no Datadog equivalent\n\nOriginal PromQL:\n```\nA: (\n  (\n    (kubelet_volume_stats_available_bytes {namespace=~\"(openshift-.*|kube-.*|default|logging)\"} ) \n    and\n    (predict_linear(kubelet_volume_stats_available_bytes {namespace=~\"(openshift-.*|kube-.*|default|logging)\"}[1w], 7 * 24 * 60 * 60) < 0)\n  )\n)\n```",
              "tick_pos": "50%",
              "font_size": "14",
              "show_tick": false,
              "tick_edge": "left",
              "text_align": "left",
              "has_padding": true,
              "vertical_align": "top",
              "background_color": "white"
            }
          },
          {
            "layout": {
              "x": 3,
              "y": 0,
              "width": 9,
              "height": 2
            },
            "definition": {
              "type": "note",
              "content": "**Infrastructure Namespace Volumes Full in Week Based on Daily Use Rate - Alert History**\n\nUnsupported panel (graph)\n\n- predict_linear() and the `and`/`< 0` alert-history pattern have no Datadog equivalent\n\nOriginal PromQL:\n```\nA: (\n  (\n    ((kubelet_volume_stats_used_bytes {namespace=~\"(openshift-.*|kube-.*|default|logging)\"} ) * 0) + 1\n    and\n    (predict_linear(kubelet_volume_stats_available_bytes {namespace=~\"(openshift-.*|kube-.*|default|logging)\"}[1d], 7 * 24 * 60 * 60) < 0)\n  )\n)\n```",
              "tick_pos": "50%",
              "font_size": "14",
              "show_tick": false,
              "tick_edge": "left",
              "text_align": "left",
              "has_padding": true,
              "vertical_align": "top",
              "background_color": "white"
            }
          },
          {
            "layout": {
              "x": 3,
              "y": 2,
              "width": 9,
              "height": 2
            },
            "definition": {
              "type": "note",
              "content": "**Infrastructure Namespace Volumes Full in Week Based on Daily Use Rate - Predicted Available Space History**\n\nUnsupported panel (graph)\n\n- predict_linear() has no Datadog equivalent\n\nOriginal PromQL:\n```\nA: (\n  (predict_linear(kubelet_volume_stats_available_bytes {namespace=~\"(openshift-.*|kube-.*|default|logging)\"}[1d], 7 * 24 * 60 * 60) < 0)\n)\n```",
              "tick_pos": "50%",
              "font_size": "14",
              "show_tick": false,
              "tick_edge": "left",
              "text_align": "left",
              "has_padding": true,
              "vertical_align": "top",
              "background_color": "white"
            }
          },
          {
            "layout": {
              "x": 0,
              "y": 4,
              "width": 3,
              "height": 3
            },
            "definition": {
              "type": "note",
              "content": "**User Namespace Volumes Full in Week Based on Daily Use Rate - Current**\n\nUnsupported panel (table)\n\n- predict_linear() and the `and`/`< 0` filtering have no Datadog equivalent\n\nOriginal PromQL:\n```\nA: (\n  (\n    (\n      (kubelet_volume_stats_used_bytes {namespace!~\"(openshift-.*|kube-.*|default|logging)\"})\n    )\n    and\n    (predict_linear(kubelet_volume_stats_available_bytes[1d], 7 * 24 * 60 * 60) < 0)\n  )\n)\n```",
              "tick_pos": "50%",
              "font_size": "14",
              "show_tick": false,
              "tick_edge": "left",
              "text_align": "left",
              "has_padding": true,
              "vertical_align": "top",
              "background_color": "white"
            }
          },
          {
            "layout": {
              "x": 3,
              "y": 4,
              "width": 9,
              "height": 2
            },
            "definition": {
              "type": "note",
              "content": "**User Namespace Volumes Full in Week Based on Daily Use Rate - Alert History**\n\nUnsupported panel (graph)\n\n- predict_linear() and the `and`/`< 0` alert-history pattern have no Datadog equivalent\n\nOriginal PromQL:\n```\nA: (\n  (\n    (\n      (\n        (kubelet_volume_stats_used_bytes {namespace!~\"(openshift-.*|kube-.*|default|logging)\"})\n      )\n      * 0 + 1\n    )\n    and\n    (predict_linear(kubelet_volume_stats_available_bytes[1d], 7 * 24 * 60 * 60) < 0)\n  )\n)\n```",
              "tick_pos": "50%",
              "font_size": "14",
              "show_tick": false,
              "tick_edge": "left",
              "text_align": "left",
              "has_padding": true,
              "vertical_align": "top",
              "background_color": "white"
            }
          },
          {
            "layout": {
              "x": 3,
              "y": 6,
              "width": 9,
              "height": 2
            },
            "definition": {
              "type": "note",
              "content": "**User Namespace Volumes Full in Week Based on Daily Use Rate - Predicted Available Space History**\n\nUnsupported panel (graph)\n\n- predict_linear() has no Datadog equivalent\n\nOriginal PromQL:\n```\nA: (\n  (predict_linear(kubelet_volume_stats_available_bytes {namespace!~\"(openshift-.*|kube-.*|default|logging)\"}[1d], 7 * 24 * 60 * 60) < 0)\n)\n```",
              "tick_pos": "50%",
              "font_size": "14",
              "show_tick": false,
              "tick_edge": "left",
              "text_align": "left",
              "has_padding": true,
              "vertical_align": "top",
              "background_color": "white"
            }
          },
          {
            "layout": {
              "x": 0,
              "y": 7,
              "width": 3,
              "height": 2
            },
            "definition": {
              "type": "query_table",
              "title": "Running PVCs Above % Used Warning Threshold Stats - Current",
              "requests": [
                {
                  "queries": [
                    {
                      "name": "query1",
                      "query": "max:kubernetes.kubelet.volume.stats.used_bytes{*} by {persistentvolumeclaim,kube_namespace}",
                      "aggregator": "last",
                      "data_source": "metrics"
                    },
                    {
                      "name": "query2",
                      "query": "max:kubernetes.kubelet.volume.stats.capacity_bytes{*} by {persistentvolumeclaim,kube_namespace}",
                      "aggregator": "last",
                      "data_source": "metrics"
                    }
                  ],
                  "formulas": [
                    {
                      "limit": {
                        "count": 50,
                        "order": "desc"
                      },
                      "formula": "query1 / query2"
                    }
                  ],
                  "response_format": "scalar"
                }
              ],
              "has_search_bar": "auto"
            }
          },
          {
            "layout": {
              "x": 3,
              "y": 8,
              "width": 9,
              "height": 2
            },
            "definition": {
              "type": "note",
              "content": "**Running PVCs Above % Used Warning Threshold - History**\n\nUnsupported panel (graph)\n\n- `and` set operator with a threshold comparison cannot be expressed in Datadog formulas\n\nOriginal PromQL:\n```\nA: (\n  ((max by (persistentvolumeclaim,namespace) (kubelet_volume_stats_used_bytes )) * 0 +1)\n  and\n  (\n    (max by (persistentvolumeclaim,namespace) (kubelet_volume_stats_used_bytes )) / (max by (persistentvolumeclaim,namespace) (kubelet_volume_stats_capacity_bytes ))\n  ) >= ($pvc_percent_used_warning_threshold / 100)\n)\n```",
              "tick_pos": "50%",
              "font_size": "14",
              "show_tick": false,
              "tick_edge": "left",
              "text_align": "left",
              "has_padding": true,
              "vertical_align": "top",
              "background_color": "white"
            }
          }
        ],
        "show_title": true,
        "layout_type": "ordered"
      }
    },
    {
      "layout": {
        "x": 0,
        "y": 12,
        "width": 12,
        "height": 7
      },
      "definition": {
        "type": "group",
        "title": "Stats",
        "widgets": [
          {
            "layout": {
              "x": 0,
              "y": 0,
              "width": 1,
              "height": 1
            },
            "definition": {
              "type": "query_value",
              "title": "Bound PVCs",
              "requests": [
                {
                  "queries": [
                    {
                      "name": "query1",
                      "query": "sum:pv_collector_bound_pvc_count{*}",
                      "aggregator": "last",
                      "data_source": "metrics"
                    }
                  ],
                  "formulas": [
                    {
                      "formula": "default_zero(query1)"
                    }
                  ],
                  "response_format": "scalar"
                }
              ],
              "autoscale": true,
              "precision": 2,
              "text_align": "center"
            }
          },
          {
            "layout": {
              "x": 0,
              "y": 1,
              "width": 12,
              "height": 6
            },
            "definition": {
              "type": "query_table",
              "title": "PVC Stats",
              "requests": [
                {
                  "queries": [
                    {
                      "name": "query1",
                      "query": "avg:kube_persistentvolumeclaim_info{*}",
                      "aggregator": "last",
                      "data_source": "metrics"
                    },
                    {
                      "name": "query2",
                      "query": "max:kubernetes.kubelet.volume.stats.used_bytes{*} by {persistentvolumeclaim,kube_namespace}",
                      "aggregator": "last",
                      "data_source": "metrics"
                    },
                    {
                      "name": "query3",
                      "query": "max:kubernetes.kubelet.volume.stats.capacity_bytes{*} by {persistentvolumeclaim,kube_namespace}",
                      "aggregator": "last",
                      "data_source": "metrics"
                    },
                    {
                      "name": "query4",
                      "query": "max:kubernetes.kubelet.volume.stats.available_bytes{*} by {persistentvolumeclaim,kube_namespace}",
                      "aggregator": "last",
                      "data_source": "metrics"
                    },
                    {
                      "name": "query5",
                      "query": "max:kubernetes.kubelet.volume.stats.used_bytes{*} by {persistentvolumeclaim,kube_namespace}",
                      "aggregator": "last",
                      "data_source": "metrics"
                    },
                    {
                      "name": "query6",
                      "query": "max:kubernetes.kubelet.volume.stats.capacity_bytes{*} by {persistentvolumeclaim,kube_namespace}",
                      "aggregator": "last",
                      "data_source": "metrics"
                    }
                  ],
                  "formulas": [
                    {
                      "limit": {
                        "count": 50,
                        "order": "desc"
                      },
                      "formula": "query1"
                    },
                    {
                      "formula": "query2"
                    },
                    {
                      "formula": "query3"
                    },
                    {
                      "formula": "query4"
                    },
                    {
                      "formula": "query5 / query6"
                    }
                  ],
                  "response_format": "scalar"
                }
              ],
              "has_search_bar": "auto"
            }
          }
        ],
        "show_title": true,
        "layout_type": "ordered"
      }
    },
    {
      "layout": {
        "x": 0,
        "y": 19,
        "width": 12,
        "height": 4
      },
      "definition": {
        "type": "group",
        "title": "Use",
        "widgets": [
          {
            "layout": {
              "x": 0,
              "y": 0,
              "width": 12,
              "height": 2
            },
            "definition": {
              "type": "timeseries",
              "title": "All Running PVCs Used Bytes",
              "yaxis": {
                "max": "auto",
                "min": "auto",
                "label": "",
                "scale": "linear",
                "include_zero": false
              },
              "requests": [
                {
                  "style": {
                    "palette": "dog_classic",
                    "line_type": "solid",
                    "line_width": "normal"
                  },
                  "queries": [
                    {
                      "name": "query1",
                      "query": "max:kubernetes.kubelet.volume.stats.used_bytes{*} by {persistentvolumeclaim,kube_namespace}",
                      "data_source": "metrics"
                    }
                  ],
                  "formulas": [
                    {
                      "formula": "query1"
                    }
                  ],
                  "display_type": "line",
                  "response_format": "timeseries"
                }
              ],
              "show_legend": true,
              "legend_layout": "auto",
              "legend_columns": [
                "avg",
                "min",
                "max",
                "value",
                "sum"
              ]
            }
          },
          {
            "layout": {
              "x": 0,
              "y": 2,
              "width": 12,
              "height": 2
            },
            "definition": {
              "type": "timeseries",
              "title": "Running PVCs % Used",
              "yaxis": {
                "max": "auto",
                "min": "auto",
                "label": "",
                "scale": "linear",
                "include_zero": false
              },
              "requests": [
                {
                  "style": {
                    "palette": "dog_classic",
                    "line_type": "solid",
                    "line_width": "normal"
                  },
                  "queries": [
                    {
                      "name": "query1",
                      "query": "max:kubernetes.kubelet.volume.stats.used_bytes{*} by {persistentvolumeclaim,kube_namespace}",
                      "data_source": "metrics"
                    },
                    {
                      "name": "query2",
                      "query": "max:kubernetes.kubelet.volume.stats.capacity_bytes{*} by {persistentvolumeclaim,kube_namespace}",
                      "data_source": "metrics"
                    }
                  ],
                  "formulas": [
                    {
                      "formula": "(query1 / query2) * 100"
                    }
                  ],
                  "display_type": "line",
                  "response_format": "timeseries"
                }
              ],
              "show_legend": true,
              "legend_layout": "auto",
              "legend_columns": [
                "avg",
                "min",
                "max",
                "value",
                "sum"
              ]
            }
          }
        ],
        "show_title": true,
        "layout_type": "ordered"
      }
    },
    {
      "layout": {
        "x": 0,
        "y": 23,
        "width": 12,
        "height": 6
      },
      "definition": {
        "type": "group",
        "title": "Use Rate",
        "widgets": [
          {
            "layout": {
              "x": 0,
              "y": 0,
              "width": 12,
              "height": 2
            },
            "definition": {
              "type": "timeseries",
              "title": "Hourly Volume Use Rate",
              "yaxis": {
                "max": "auto",
                "min": "auto",
                "label": "",
                "scale": "linear",
                "include_zero": false
              },
              "requests": [
                {
                  "style": {
                    "palette": "dog_classic",
                    "line_type": "solid",
                    "line_width": "normal"
                  },
                  "queries": [
                    {
                      "name": "query1",
                      "query": "avg:kubernetes.kubelet.volume.stats.used_bytes{*} by {persistentvolumeclaim,kube_namespace}",
                      "data_source": "metrics"
                    }
                  ],
                  "formulas": [
                    {
                      "formula": "per_second(query1)"
                    }
                  ],
                  "display_type": "line",
                  "response_format": "timeseries"
                }
              ],
              "show_legend": true,
              "legend_layout": "auto",
              "legend_columns": [
                "avg",
                "min",
                "max",
                "value",
                "sum"
              ]
            }
          },
          {
            "layout": {
              "x": 0,
              "y": 2,
              "width": 12,
              "height": 2
            },
            "definition": {
              "type": "timeseries",
              "title": "Daily Volume Use Rate",
              "yaxis": {
                "max": "auto",
                "min": "auto",
                "label": "",
                "scale": "linear",
                "include_zero": false
              },
              "requests": [
                {
                  "style": {
                    "palette": "dog_classic",
                    "line_type": "solid",
                    "line_width": "normal"
                  },
                  "queries": [
                    {
                      "name": "query1",
                      "query": "avg:kubernetes.kubelet.volume.stats.used_bytes{*} by {persistentvolumeclaim,kube_namespace}",
                      "data_source": "metrics"
                    }
                  ],
                  "formulas": [
                    {
                      "formula": "per_second(query1)"
                    }
                  ],
                  "display_type": "line",
                  "response_format": "timeseries"
                }
              ],
              "show_legend": true,
              "legend_layout": "auto",
              "legend_columns": [
                "avg",
                "min",
                "max",
                "value",
                "sum"
              ]
            }
          },
          {
            "layout": {
              "x": 0,
              "y": 4,
              "width": 12,
              "height": 2
            },
            "definition": {
              "type": "timeseries",
              "title": "Weekly Volume Use Rate",
              "yaxis": {
                "max": "auto",
                "min": "auto",
                "label": "",
                "scale": "linear",
                "include_zero": false
              },
              "requests": [
                {
                  "style": {
                    "palette": "dog_classic",
                    "line_type": "solid",
                    "line_width": "normal"
                  },
                  "queries": [
                    {
                      "name": "query1",
                      "query": "avg:kubernetes.kubelet.volume.stats.used_bytes{*} by {persistentvolumeclaim,kube_namespace}",
                      "data_source": "metrics"
                    }
                  ],
                  "formulas": [
                    {
                      "formula": "per_second(query1)"
                    }
                  ],
                  "display_type": "line",
                  "response_format": "timeseries"
                }
              ],
              "show_legend": true,
              "legend_layout": "auto",
              "legend_columns": [
                "avg",
                "min",
                "max",
                "value",
                "sum"
              ]
            }
          }
        ],
        "show_title": true,
        "layout_type": "ordered"
      }
    }
  ],
  "description": "Converted from Grafana dashboard https://grafana.com/grafana/dashboards/11454 (by Ian Tewksbury). Dashboard of Kubernetes / OpenShift volume information at cluster level as exported by Prometheus connected to Kubernetes / OpenShift.",
  "layout_type": "ordered",
  "reflow_type": "fixed",
  "template_variables": [
    {
      "name": "pvc_percent_used_warning_threshold",
      "prefix": "pvc_percent_used_warning_threshold",
      "default": "80",
      "available_values": []
    }
  ]
}