From: Leon Romanovsky <leon@kernel•org>
To: Steve Wise <swise@opengridcomputing•com>
Cc: dsahern@gmail•com, stephen@networkplumber•org,
netdev@vger•kernel.org, linux-rdma@vger•kernel.org
Subject: Re: [PATCH RFC iproute-next 3/5] rdma: Add CQ resource tracking information
Date: Tue, 20 Feb 2018 15:09:16 +0200 [thread overview]
Message-ID: <20180220130916.GG7709@mtr-leonro.local> (raw)
In-Reply-To: <a61b42e445d3936b32fe6ea7e4d101a31a7c5ca4.1519071002.git.swise@opengridcomputing.com>
[-- Attachment #1: Type: text/plain, Size: 6477 bytes --]
On Wed, Feb 14, 2018 at 01:07:01PM -0800, Steve Wise wrote:
> Sample output:
>
> # rdma resource show cq
> link cxgb4_0/- cqe 46 usecnt 2 pid 30503 comm rping
> link cxgb4_0/- cqe 46 usecnt 2 pid 30498 comm rping
> link mlx4_0/- cqe 63 usecnt 2 pid 30494 comm rping
> link mlx4_0/- cqe 63 usecnt 2 pid 30489 comm rping
> link mlx4_0/- cqe 1023 usecnt 2 poll_ctx WORKQUEUE pid 0 comm [ib_core]
>
> # rdma resource show cq pid 30489
> link mlx4_0/- cqe 63 usecnt 2 pid 30489 comm rping
>
> Signed-off-by: Steve Wise <swise@opengridcomputing•com>
> ---
> rdma/res.c | 123 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> rdma/utils.c | 5 +++
> 2 files changed, 128 insertions(+)
>
> diff --git a/rdma/res.c b/rdma/res.c
> index beae7dc..27c1efd 100644
> --- a/rdma/res.c
> +++ b/rdma/res.c
> @@ -21,6 +21,8 @@ static int res_help(struct rd *rd)
> pr_out(" resource show qp link [DEV/PORT] [FILTER-NAME FILTER-VALUE]\n");
> pr_out(" resource show cm_id link [DEV/PORT]\n");
> pr_out(" resource show cm_id link [DEV/PORT] [FILTER-NAME FILTER-VALUE]\n");
> + pr_out(" resource show cq link [DEV/PORT]\n");
> + pr_out(" resource show cq link [DEV/PORT] [FILTER-NAME FILTER-VALUE]\n");
> return 0;
> }
>
> @@ -705,6 +707,118 @@ static int res_cm_id_parse_cb(const struct nlmsghdr *nlh, void *data)
> return MNL_CB_OK;
> }
>
> +static void print_cqe(struct rd *rd, uint32_t val)
> +{
> + if (rd->json_output)
> + jsonw_uint_field(rd->jw, "cqe", val);
> + else
> + pr_out("cqe %u ", val);
> +}
> +
> +static void print_usecnt(struct rd *rd, uint64_t val)
> +{
> + if (rd->json_output)
> + jsonw_uint_field(rd->jw, "usecnt", val);
> + else
> + pr_out("usecnt %" PRIu64 " ", val);
Interesting, how many users are actually know what the "usecnt" actually means?
Will it be more clear to call it "users" instead of "usecnt"?
> +}
> +
> +static const char *poll_ctx_to_str(uint8_t idx)
> +{
> + static const char * const cm_id_states_str[] = { "DIRECT", "SOFTIRQ",
> + "WORKQUEUE"};
> +
> + if (idx < ARRAY_SIZE(cm_id_states_str))
> + return cm_id_states_str[idx];
> + return "UNKNOWN";
> +}
> +
> +static void print_poll_ctx(struct rd *rd, uint8_t poll_ctx)
> +{
> + if (rd->json_output) {
> + jsonw_string_field(rd->jw, "poll_ctx", poll_ctx_to_str(poll_ctx));
> + return;
> + }
> + pr_out("poll_ctx %s ", poll_ctx_to_str(poll_ctx));
> +}
> +
> +static int res_cq_parse_cb(const struct nlmsghdr *nlh, void *data)
> +{
> + struct nlattr *tb[RDMA_NLDEV_ATTR_MAX] = {};
> + struct nlattr *nla_table, *nla_entry;
> + struct rd *rd = data;
> + const char *name;
> + uint32_t idx;
> +
> + mnl_attr_parse(nlh, 0, rd_attr_cb, tb);
> + if (!tb[RDMA_NLDEV_ATTR_DEV_INDEX] ||
> + !tb[RDMA_NLDEV_ATTR_DEV_NAME] ||
> + !tb[RDMA_NLDEV_ATTR_RES_CQ])
> + return MNL_CB_ERROR;
> +
> + name = mnl_attr_get_str(tb[RDMA_NLDEV_ATTR_DEV_NAME]);
> + idx = mnl_attr_get_u32(tb[RDMA_NLDEV_ATTR_DEV_INDEX]);
> + nla_table = tb[RDMA_NLDEV_ATTR_RES_CQ];
> +
> + mnl_attr_for_each_nested(nla_entry, nla_table) {
> + struct nlattr *nla_line[RDMA_NLDEV_ATTR_MAX] = {};
> + char *comm = NULL;
> + uint32_t pid = 0;
> + uint8_t poll_ctx = 0;
> + uint64_t usecnt;
> + uint32_t cqe;
> + int err;
> +
> + err = mnl_attr_parse_nested(nla_entry, rd_attr_cb, nla_line);
> + if (err != MNL_CB_OK)
> + return MNL_CB_ERROR;
> +
> + if (!nla_line[RDMA_NLDEV_ATTR_RES_CQE] ||
> + !nla_line[RDMA_NLDEV_ATTR_RES_USECNT] ||
I'm not sure that we will have USECNT in the future, let's not put
requirement for RDMA_NLDEV_ATTR_RES_USECNT here.
> + (!nla_line[RDMA_NLDEV_ATTR_RES_PID] &&
> + !nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])) {
> + return MNL_CB_ERROR;
> + }
> +
> + cqe = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_CQE]);
> + usecnt = mnl_attr_get_u64(nla_line[RDMA_NLDEV_ATTR_RES_USECNT]);
> + if (nla_line[RDMA_NLDEV_ATTR_RES_POLL_CTX])
> + poll_ctx = mnl_attr_get_u8(nla_line[RDMA_NLDEV_ATTR_RES_POLL_CTX]);
> +
> + if (nla_line[RDMA_NLDEV_ATTR_RES_PID]) {
> + pid = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_PID]);
> + comm = get_task_name(pid);
> + }
> +
> + if (rd_check_is_filtered(rd, "pid", pid))
free(comm);
> + continue;
> +
> + if (nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])
> + /* discard const from mnl_attr_get_str */
> + comm = (char *)mnl_attr_get_str(nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]);
> +
> + if (rd->json_output)
> + jsonw_start_array(rd->jw);
> +
> + print_link(rd, idx, name, 0, nla_line);
> + print_cqe(rd, cqe);
> + print_usecnt(rd, usecnt);
> + if (nla_line[RDMA_NLDEV_ATTR_RES_POLL_CTX])
> + print_poll_ctx(rd, poll_ctx);
> + print_pid(rd, pid);
> + print_comm(rd, comm, nla_line);
> +
> + if (nla_line[RDMA_NLDEV_ATTR_RES_PID])
> + free(comm);
> +
> + if (rd->json_output)
> + jsonw_end_array(rd->jw);
> + else
> + pr_out("\n");
> + }
> + return MNL_CB_OK;
> +}
> +
> RES_FUNC(res_no_args, RDMA_NLDEV_CMD_RES_GET, NULL, true);
>
> static const struct
> @@ -758,12 +872,21 @@ filters cm_id_valid_filters[MAX_NUMBER_OF_FILTERS] = {{ .name = "link",
> RES_FUNC(res_cm_id, RDMA_NLDEV_CMD_RES_CM_ID_GET, cm_id_valid_filters,
> false);
>
> +static const struct
> +filters cq_valid_filters[MAX_NUMBER_OF_FILTERS] = {{ .name = "link",
> + .is_number = false },
> + { .name = "pid",
> + .is_number = true }};
Can you please add filter of usecnt too? It will give us easy view on
"over crowded" CQs.
> +
> +RES_FUNC(res_cq, RDMA_NLDEV_CMD_RES_CQ_GET, cq_valid_filters, true);
> +
> static int res_show(struct rd *rd)
> {
> const struct rd_cmd cmds[] = {
> { NULL, res_no_args },
> { "qp", res_qp },
> { "cm_id", res_cm_id },
> + { "cq", res_cq },
> { 0 }
> };
>
> diff --git a/rdma/utils.c b/rdma/utils.c
> index 906ca73..11b34fe 100644
> --- a/rdma/utils.c
> +++ b/rdma/utils.c
> @@ -387,6 +387,11 @@ static const enum mnl_attr_data_type nldev_policy[RDMA_NLDEV_ATTR_MAX] = {
> [RDMA_NLDEV_ATTR_RES_DEV_TYPE] = MNL_TYPE_U8,
> [RDMA_NLDEV_ATTR_RES_TRANSPORT_TYPE] = MNL_TYPE_U8,
> [RDMA_NLDEV_ATTR_RES_NETWORK_TYPE] = MNL_TYPE_U8,
> + [RDMA_NLDEV_ATTR_RES_CQ] = MNL_TYPE_NESTED,
> + [RDMA_NLDEV_ATTR_RES_CQ_ENTRY] = MNL_TYPE_NESTED,
> + [RDMA_NLDEV_ATTR_RES_CQE] = MNL_TYPE_U32,
> + [RDMA_NLDEV_ATTR_RES_USECNT] = MNL_TYPE_U64,
> + [RDMA_NLDEV_ATTR_RES_POLL_CTX] = MNL_TYPE_U8,
> };
>
> int rd_attr_cb(const struct nlattr *attr, void *data)
> --
> 1.8.3.1
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
next prev parent reply other threads:[~2018-02-20 13:09 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-02-19 20:10 [PATCH RFC iproute-next 0/5] cm_id, cq, mr, and pd resource tracking Steve Wise
2018-02-14 21:05 ` [PATCH RFC iproute-next 1/5] rdma: update rdma_netlink.h Steve Wise
2018-02-14 21:07 ` [PATCH RFC iproute-next 5/5] rdma: Add PD resource tracking information Steve Wise
2018-02-23 14:22 ` Leon Romanovsky
2018-02-26 15:09 ` Steve Wise
2018-02-27 0:47 ` Steve Wise
2018-02-14 21:07 ` [PATCH RFC iproute-next 3/5] rdma: Add CQ " Steve Wise
2018-02-20 13:09 ` Leon Romanovsky [this message]
2018-02-26 15:06 ` Steve Wise
2018-02-14 21:07 ` [PATCH RFC iproute-next 2/5] rdma: Add CM_ID " Steve Wise
2018-02-20 12:57 ` Leon Romanovsky
2018-02-20 15:15 ` Parav Pandit
2018-02-26 15:05 ` Steve Wise
2018-02-14 21:07 ` [PATCH RFC iproute-next 4/5] rdma: Add MR " Steve Wise
2018-02-20 14:12 ` Leon Romanovsky
2018-02-26 15:08 ` Steve Wise
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180220130916.GG7709@mtr-leonro.local \
--to=leon@kernel$(echo .)org \
--cc=dsahern@gmail$(echo .)com \
--cc=linux-rdma@vger$(echo .)kernel.org \
--cc=netdev@vger$(echo .)kernel.org \
--cc=stephen@networkplumber$(echo .)org \
--cc=swise@opengridcomputing$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox