{
  "tags": [],
  "title": "Erlang-Distribution",
  "widgets": [
    {
      "layout": {
        "x": 0,
        "y": 0,
        "width": 6,
        "height": 2
      },
      "definition": {
        "type": "query_value",
        "title": "All distribution links",
        "requests": [
          {
            "queries": [
              {
                "name": "query1",
                "query": "sum:erlang_vm_dist_node_state{$rabbitmq_cluster,$namespace,$endpoint} by {host,peer}",
                "aggregator": "last",
                "data_source": "metrics"
              }
            ],
            "formulas": [
              {
                "formula": "default_zero(count_not_null(query1))"
              }
            ],
            "response_format": "scalar",
            "conditional_formats": [
              {
                "value": 2,
                "palette": "white_on_gray",
                "comparator": ">="
              },
              {
                "value": 6,
                "palette": "white_on_green",
                "comparator": ">="
              }
            ]
          }
        ],
        "autoscale": true,
        "precision": 2,
        "text_align": "center",
        "timeseries_background": {
          "type": "area"
        }
      }
    },
    {
      "layout": {
        "x": 3,
        "y": 2,
        "width": 3,
        "height": 1
      },
      "definition": {
        "type": "query_value",
        "title": "Established distribution links",
        "requests": [
          {
            "queries": [
              {
                "name": "query1",
                "query": "sum:erlang_vm_dist_node_state{$rabbitmq_cluster,$namespace,$endpoint} by {host,peer}",
                "aggregator": "last",
                "data_source": "metrics"
              }
            ],
            "formulas": [
              {
                "formula": "default_zero(count_not_null(query1))"
              }
            ],
            "response_format": "scalar",
            "conditional_formats": [
              {
                "value": 2,
                "palette": "white_on_gray",
                "comparator": ">="
              },
              {
                "value": 6,
                "palette": "white_on_green",
                "comparator": ">="
              }
            ]
          }
        ],
        "autoscale": true,
        "precision": 2,
        "text_align": "center",
        "timeseries_background": {
          "type": "area"
        }
      }
    },
    {
      "layout": {
        "x": 6,
        "y": 0,
        "width": 3,
        "height": 1
      },
      "definition": {
        "type": "query_value",
        "title": "Connecting distribution links",
        "requests": [
          {
            "queries": [
              {
                "name": "query1",
                "query": "sum:erlang_vm_dist_node_state{$rabbitmq_cluster,$namespace,$endpoint} by {host,peer}",
                "aggregator": "last",
                "data_source": "metrics"
              }
            ],
            "formulas": [
              {
                "formula": "default_zero(count_not_null(query1))"
              }
            ],
            "response_format": "scalar",
            "conditional_formats": [
              {
                "value": 1,
                "palette": "white_on_gray",
                "comparator": ">="
              },
              {
                "value": 6,
                "palette": "white_on_gray",
                "comparator": ">="
              }
            ]
          }
        ],
        "autoscale": true,
        "precision": 2,
        "text_align": "center",
        "timeseries_background": {
          "type": "area"
        }
      }
    },
    {
      "layout": {
        "x": 9,
        "y": 0,
        "width": 3,
        "height": 1
      },
      "definition": {
        "type": "query_value",
        "title": "Waiting distribution links",
        "requests": [
          {
            "queries": [
              {
                "name": "query1",
                "query": "sum:erlang_vm_dist_node_state{$rabbitmq_cluster,$namespace,$endpoint} by {host,peer}",
                "aggregator": "last",
                "data_source": "metrics"
              }
            ],
            "formulas": [
              {
                "formula": "default_zero(count_not_null(query1))"
              }
            ],
            "response_format": "scalar",
            "conditional_formats": [
              {
                "value": 1,
                "palette": "white_on_gray",
                "comparator": ">="
              },
              {
                "value": 6,
                "palette": "white_on_gray",
                "comparator": ">="
              }
            ]
          }
        ],
        "autoscale": true,
        "precision": 2,
        "text_align": "center",
        "timeseries_background": {
          "type": "area"
        }
      }
    },
    {
      "layout": {
        "x": 0,
        "y": 3,
        "width": 12,
        "height": 1
      },
      "definition": {
        "type": "group",
        "title": "distribution links",
        "widgets": [
          {
            "layout": {
              "x": 0,
              "y": 0,
              "width": 6,
              "height": 1
            },
            "definition": {
              "type": "note",
              "content": "**State of distribution links**\n\nUnsupported panel (status-history)\n\n- no Datadog equivalent for plugin status-history\n\nOriginal PromQL:\n```\nA: erlang_vm_dist_node_state{peer!~\"rabbitmqcli.*\"} * on(instance, job) group_left(rabbitmq_cluster, rabbitmq_node) rabbitmq_identity_info{rabbitmq_cluster=\"$rabbitmq_cluster\", namespace=\"$namespace\", rabbitmq_endpoint=\"$endpoint\"}\n```",
              "tick_pos": "50%",
              "font_size": "14",
              "show_tick": false,
              "tick_edge": "left",
              "text_align": "left",
              "has_padding": true,
              "vertical_align": "top",
              "background_color": "white"
            }
          },
          {
            "layout": {
              "x": 6,
              "y": 0,
              "width": 6,
              "height": 1
            },
            "definition": {
              "type": "timeseries",
              "title": "Data buffered in the distribution links queue",
              "yaxis": {
                "max": "auto",
                "min": "auto",
                "label": "",
                "scale": "linear",
                "include_zero": false
              },
              "markers": [
                {
                  "value": "y = 65536000",
                  "display_type": "error dashed"
                }
              ],
              "requests": [
                {
                  "style": {
                    "palette": "dog_classic",
                    "line_type": "solid",
                    "line_width": "normal"
                  },
                  "queries": [
                    {
                      "name": "query1",
                      "query": "avg:erlang_vm_dist_node_queue_size_bytes{$rabbitmq_cluster,$namespace,$endpoint} by {rabbitmq_node,peer}",
                      "data_source": "metrics"
                    }
                  ],
                  "formulas": [
                    {
                      "formula": "query1"
                    }
                  ],
                  "display_type": "line",
                  "response_format": "timeseries"
                }
              ],
              "show_legend": true,
              "legend_layout": "auto",
              "legend_columns": [
                "avg",
                "min",
                "max",
                "value",
                "sum"
              ]
            }
          }
        ],
        "show_title": true,
        "layout_type": "ordered"
      }
    },
    {
      "layout": {
        "x": 0,
        "y": 4,
        "width": 12,
        "height": 3
      },
      "definition": {
        "type": "group",
        "title": "inet socket",
        "widgets": [
          {
            "layout": {
              "x": 0,
              "y": 0,
              "width": 6,
              "height": 1
            },
            "definition": {
              "type": "timeseries",
              "title": "Data sent to peer node / s",
              "yaxis": {
                "max": "auto",
                "min": "auto",
                "label": "",
                "scale": "linear",
                "include_zero": false
              },
              "markers": [
                {
                  "value": "y = 80",
                  "display_type": "error dashed"
                }
              ],
              "requests": [
                {
                  "style": {
                    "palette": "dog_classic",
                    "line_type": "solid",
                    "line_width": "normal"
                  },
                  "queries": [
                    {
                      "name": "query1",
                      "query": "avg:erlang_vm_dist_send_bytes.count{$rabbitmq_cluster,$namespace,$endpoint} by {rabbitmq_node,peer}.as_rate()",
                      "data_source": "metrics"
                    }
                  ],
                  "formulas": [
                    {
                      "formula": "query1"
                    }
                  ],
                  "display_type": "line",
                  "response_format": "timeseries"
                }
              ],
              "show_legend": true,
              "legend_layout": "auto",
              "legend_columns": [
                "avg",
                "min",
                "max",
                "value",
                "sum"
              ]
            }
          },
          {
            "layout": {
              "x": 6,
              "y": 0,
              "width": 6,
              "height": 1
            },
            "definition": {
              "type": "timeseries",
              "title": "Data received from peer node / s",
              "yaxis": {
                "max": "auto",
                "min": "auto",
                "label": "",
                "scale": "linear",
                "include_zero": false
              },
              "markers": [
                {
                  "value": "y = 80",
                  "display_type": "error dashed"
                }
              ],
              "requests": [
                {
                  "style": {
                    "palette": "dog_classic",
                    "line_type": "solid",
                    "line_width": "normal"
                  },
                  "queries": [
                    {
                      "name": "query1",
                      "query": "avg:erlang_vm_dist_recv_bytes.count{$rabbitmq_cluster,$namespace,$endpoint} by {rabbitmq_node,peer}.as_rate()",
                      "data_source": "metrics"
                    }
                  ],
                  "formulas": [
                    {
                      "formula": "query1"
                    }
                  ],
                  "display_type": "line",
                  "response_format": "timeseries"
                }
              ],
              "show_legend": true,
              "legend_layout": "auto",
              "legend_columns": [
                "avg",
                "min",
                "max",
                "value",
                "sum"
              ]
            }
          },
          {
            "layout": {
              "x": 0,
              "y": 1,
              "width": 6,
              "height": 1
            },
            "definition": {
              "type": "timeseries",
              "title": "Distribution messages sent to peer node / s",
              "yaxis": {
                "max": "auto",
                "min": "auto",
                "label": "",
                "scale": "linear",
                "include_zero": false
              },
              "markers": [
                {
                  "value": "y = 80",
                  "display_type": "error dashed"
                }
              ],
              "requests": [
                {
                  "style": {
                    "palette": "dog_classic",
                    "line_type": "solid",
                    "line_width": "normal"
                  },
                  "queries": [
                    {
                      "name": "query1",
                      "query": "avg:erlang_vm_dist_send_cnt.count{$rabbitmq_cluster,$namespace,$endpoint} by {rabbitmq_node,peer}.as_rate()",
                      "data_source": "metrics"
                    }
                  ],
                  "formulas": [
                    {
                      "formula": "query1"
                    }
                  ],
                  "display_type": "line",
                  "response_format": "timeseries"
                }
              ],
              "show_legend": true,
              "legend_layout": "auto",
              "legend_columns": [
                "avg",
                "min",
                "max",
                "value",
                "sum"
              ]
            }
          },
          {
            "layout": {
              "x": 6,
              "y": 1,
              "width": 6,
              "height": 1
            },
            "definition": {
              "type": "timeseries",
              "title": "Distribution messages received from peer node / s",
              "yaxis": {
                "max": "auto",
                "min": "auto",
                "label": "",
                "scale": "linear",
                "include_zero": false
              },
              "markers": [
                {
                  "value": "y = 80",
                  "display_type": "error dashed"
                }
              ],
              "requests": [
                {
                  "style": {
                    "palette": "dog_classic",
                    "line_type": "solid",
                    "line_width": "normal"
                  },
                  "queries": [
                    {
                      "name": "query1",
                      "query": "avg:erlang_vm_dist_recv_cnt.count{$rabbitmq_cluster,$namespace,$endpoint} by {rabbitmq_node,peer}.as_rate()",
                      "data_source": "metrics"
                    }
                  ],
                  "formulas": [
                    {
                      "formula": "query1"
                    }
                  ],
                  "display_type": "line",
                  "response_format": "timeseries"
                }
              ],
              "show_legend": true,
              "legend_layout": "auto",
              "legend_columns": [
                "avg",
                "min",
                "max",
                "value",
                "sum"
              ]
            }
          },
          {
            "layout": {
              "x": 0,
              "y": 2,
              "width": 6,
              "height": 1
            },
            "definition": {
              "type": "timeseries",
              "title": "Average inet packet size sent to peer node",
              "yaxis": {
                "max": "auto",
                "min": "auto",
                "label": "",
                "scale": "linear",
                "include_zero": false
              },
              "markers": [
                {
                  "value": "y = 80",
                  "display_type": "error dashed"
                }
              ],
              "requests": [
                {
                  "style": {
                    "palette": "dog_classic",
                    "line_type": "solid",
                    "line_width": "normal"
                  },
                  "queries": [
                    {
                      "name": "query1",
                      "query": "avg:erlang_vm_dist_send_bytes.count{$rabbitmq_cluster,$namespace,$endpoint} by {rabbitmq_node,peer}.as_rate()",
                      "data_source": "metrics"
                    },
                    {
                      "name": "query2",
                      "query": "avg:erlang_vm_dist_send_cnt.count{$rabbitmq_cluster,$namespace,$endpoint} by {rabbitmq_node,peer}.as_rate()",
                      "data_source": "metrics"
                    }
                  ],
                  "formulas": [
                    {
                      "formula": "query1 / query2"
                    }
                  ],
                  "display_type": "line",
                  "response_format": "timeseries"
                }
              ],
              "show_legend": true,
              "legend_layout": "auto",
              "legend_columns": [
                "avg",
                "min",
                "max",
                "value",
                "sum"
              ]
            }
          },
          {
            "layout": {
              "x": 6,
              "y": 2,
              "width": 6,
              "height": 1
            },
            "definition": {
              "type": "timeseries",
              "title": "Average inet packet size received from peer node",
              "yaxis": {
                "max": "auto",
                "min": "auto",
                "label": "",
                "scale": "linear",
                "include_zero": false
              },
              "markers": [
                {
                  "value": "y = 80",
                  "display_type": "error dashed"
                }
              ],
              "requests": [
                {
                  "style": {
                    "palette": "dog_classic",
                    "line_type": "solid",
                    "line_width": "normal"
                  },
                  "queries": [
                    {
                      "name": "query1",
                      "query": "avg:erlang_vm_dist_recv_bytes.count{$rabbitmq_cluster,$namespace,$endpoint} by {rabbitmq_node,peer}.as_rate()",
                      "data_source": "metrics"
                    },
                    {
                      "name": "query2",
                      "query": "avg:erlang_vm_dist_recv_cnt.count{$rabbitmq_cluster,$namespace,$endpoint} by {rabbitmq_node,peer}.as_rate()",
                      "data_source": "metrics"
                    }
                  ],
                  "formulas": [
                    {
                      "formula": "query1 / query2"
                    }
                  ],
                  "display_type": "line",
                  "response_format": "timeseries"
                }
              ],
              "show_legend": true,
              "legend_layout": "auto",
              "legend_columns": [
                "avg",
                "min",
                "max",
                "value",
                "sum"
              ]
            }
          }
        ],
        "show_title": true,
        "layout_type": "ordered"
      }
    },
    {
      "layout": {
        "x": 0,
        "y": 7,
        "width": 12,
        "height": 1
      },
      "definition": {
        "type": "group",
        "title": "port driver",
        "widgets": [
          {
            "layout": {
              "x": 0,
              "y": 0,
              "width": 6,
              "height": 1
            },
            "definition": {
              "type": "timeseries",
              "title": "Memory used by the port driver",
              "yaxis": {
                "max": "auto",
                "min": "auto",
                "label": "",
                "scale": "linear",
                "include_zero": false
              },
              "markers": [
                {
                  "value": "y = 80",
                  "display_type": "error dashed"
                }
              ],
              "requests": [
                {
                  "style": {
                    "palette": "dog_classic",
                    "line_type": "solid",
                    "line_width": "normal"
                  },
                  "queries": [
                    {
                      "name": "query1",
                      "query": "avg:erlang_vm_dist_port_memory_bytes{$rabbitmq_cluster,$namespace,$endpoint} by {rabbitmq_node,peer}",
                      "data_source": "metrics"
                    }
                  ],
                  "formulas": [
                    {
                      "formula": "query1"
                    }
                  ],
                  "display_type": "line",
                  "response_format": "timeseries"
                }
              ],
              "show_legend": true,
              "legend_layout": "auto",
              "legend_columns": [
                "avg",
                "min",
                "max",
                "value",
                "sum"
              ]
            }
          },
          {
            "layout": {
              "x": 6,
              "y": 0,
              "width": 6,
              "height": 1
            },
            "definition": {
              "type": "timeseries",
              "title": "Data in the port driver buffer",
              "yaxis": {
                "max": "auto",
                "min": "auto",
                "label": "",
                "scale": "linear",
                "include_zero": true
              },
              "markers": [
                {
                  "value": "y = 0",
                  "display_type": "error dashed"
                }
              ],
              "requests": [
                {
                  "style": {
                    "palette": "dog_classic",
                    "line_type": "solid",
                    "line_width": "normal"
                  },
                  "queries": [
                    {
                      "name": "query1",
                      "query": "avg:erlang_vm_dist_port_queue_size_bytes{$rabbitmq_cluster,$namespace,$endpoint} by {rabbitmq_node,peer}",
                      "data_source": "metrics"
                    }
                  ],
                  "formulas": [
                    {
                      "formula": "query1"
                    }
                  ],
                  "display_type": "area",
                  "response_format": "timeseries"
                }
              ],
              "show_legend": true,
              "legend_layout": "auto",
              "legend_columns": [
                "avg",
                "min",
                "max",
                "value",
                "sum"
              ]
            }
          }
        ],
        "show_title": true,
        "layout_type": "ordered"
      }
    },
    {
      "layout": {
        "x": 0,
        "y": 8,
        "width": 12,
        "height": 2
      },
      "definition": {
        "type": "group",
        "title": "$erlang_vm_dist_proc_type process",
        "widgets": [
          {
            "layout": {
              "x": 0,
              "y": 0,
              "width": 6,
              "height": 1
            },
            "definition": {
              "type": "note",
              "content": "**Process state**\n\nUnsupported panel (status-history)\n\n- no Datadog equivalent for plugin status-history\n\nOriginal PromQL:\n```\nA: erlang_vm_dist_proc_status{type=\"$erlang_vm_dist_proc_type\"} * on(instance, job) group_left(rabbitmq_node, rabbitmq_cluster) rabbitmq_identity_info{rabbitmq_cluster=\"$rabbitmq_cluster\", namespace=\"$namespace\", rabbitmq_endpoint=\"$endpoint\"}\n```",
              "tick_pos": "50%",
              "font_size": "14",
              "show_tick": false,
              "tick_edge": "left",
              "text_align": "left",
              "has_padding": true,
              "vertical_align": "top",
              "background_color": "white"
            }
          },
          {
            "layout": {
              "x": 6,
              "y": 0,
              "width": 6,
              "height": 1
            },
            "definition": {
              "type": "timeseries",
              "title": "Messages in the process queue",
              "yaxis": {
                "max": "auto",
                "min": "auto",
                "label": "",
                "scale": "linear",
                "include_zero": false
              },
              "markers": [
                {
                  "value": "y = 0",
                  "display_type": "error dashed"
                }
              ],
              "requests": [
                {
                  "style": {
                    "palette": "dog_classic",
                    "line_type": "solid",
                    "line_width": "normal"
                  },
                  "queries": [
                    {
                      "name": "query1",
                      "query": "avg:erlang_vm_dist_proc_message_queue_len{$erlang_vm_dist_proc_type,$rabbitmq_cluster,$namespace,$endpoint} by {rabbitmq_node,peer}",
                      "data_source": "metrics"
                    }
                  ],
                  "formulas": [
                    {
                      "formula": "query1"
                    }
                  ],
                  "display_type": "line",
                  "response_format": "timeseries"
                }
              ],
              "show_legend": true,
              "legend_layout": "auto",
              "legend_columns": [
                "avg",
                "min",
                "max",
                "value",
                "sum"
              ]
            }
          },
          {
            "layout": {
              "x": 0,
              "y": 1,
              "width": 6,
              "height": 1
            },
            "definition": {
              "type": "timeseries",
              "title": "Memory used by the process",
              "yaxis": {
                "max": "auto",
                "min": "auto",
                "label": "",
                "scale": "linear",
                "include_zero": false
              },
              "markers": [
                {
                  "value": "y = 80",
                  "display_type": "error dashed"
                }
              ],
              "requests": [
                {
                  "style": {
                    "palette": "dog_classic",
                    "line_type": "solid",
                    "line_width": "normal"
                  },
                  "queries": [
                    {
                      "name": "query1",
                      "query": "avg:erlang_vm_dist_proc_memory_bytes{$erlang_vm_dist_proc_type,$rabbitmq_cluster,$namespace,$endpoint} by {rabbitmq_node,peer}",
                      "data_source": "metrics"
                    }
                  ],
                  "formulas": [
                    {
                      "formula": "query1"
                    }
                  ],
                  "display_type": "line",
                  "response_format": "timeseries"
                }
              ],
              "show_legend": true,
              "legend_layout": "auto",
              "legend_columns": [
                "avg",
                "min",
                "max",
                "value",
                "sum"
              ]
            }
          },
          {
            "layout": {
              "x": 6,
              "y": 1,
              "width": 6,
              "height": 1
            },
            "definition": {
              "type": "timeseries",
              "title": "Process reductions / s",
              "yaxis": {
                "max": "auto",
                "min": "auto",
                "label": "",
                "scale": "linear",
                "include_zero": false
              },
              "markers": [
                {
                  "value": "y = 80",
                  "display_type": "error dashed"
                }
              ],
              "requests": [
                {
                  "style": {
                    "palette": "dog_classic",
                    "line_type": "solid",
                    "line_width": "normal"
                  },
                  "queries": [
                    {
                      "name": "query1",
                      "query": "avg:erlang_vm_dist_proc_reductions.count{$erlang_vm_dist_proc_type,$rabbitmq_cluster,$namespace,$endpoint} by {rabbitmq_node,peer}.as_rate()",
                      "data_source": "metrics"
                    }
                  ],
                  "formulas": [
                    {
                      "formula": "query1"
                    }
                  ],
                  "display_type": "line",
                  "response_format": "timeseries"
                }
              ],
              "show_legend": true,
              "legend_layout": "auto",
              "legend_columns": [
                "avg",
                "min",
                "max",
                "value",
                "sum"
              ]
            }
          }
        ],
        "show_title": true,
        "layout_type": "ordered"
      }
    }
  ],
  "description": "Converted from Grafana dashboard https://grafana.com/grafana/dashboards/11352 (by RabbitMQ). Erlang Distribution links, inet socket, port driver, dist process + tls_connection & tls_sender",
  "layout_type": "ordered",
  "reflow_type": "fixed",
  "template_variables": [
    {
      "name": "namespace",
      "prefix": "namespace",
      "default": "*",
      "available_values": []
    },
    {
      "name": "rabbitmq_cluster",
      "prefix": "rabbitmq_cluster",
      "default": "*",
      "available_values": []
    },
    {
      "name": "endpoint",
      "prefix": "rabbitmq_endpoint",
      "default": "*",
      "available_values": []
    },
    {
      "name": "erlang_vm_dist_proc_type",
      "prefix": "type",
      "default": "*",
      "available_values": []
    }
  ]
}