* [PATCH] media: rkvdec: hevc: cap EXT SPS RPS control counts before descriptor assembly
@ 2026-05-13 18:19 Michael Bommarito
2026-05-19 13:04 ` Detlev Casanova
2026-05-27 19:47 ` [PATCH v2 0/3] media: rkvdec: hevc: bound EXT SPS RPS control counts Michael Bommarito
0 siblings, 2 replies; 8+ messages in thread
From: Michael Bommarito @ 2026-05-13 18:19 UTC (permalink / raw)
To: Detlev Casanova, Ezequiel Garcia, Mauro Carvalho Chehab,
Heiko Stuebner, linux-media
Cc: linux-rockchip, linux-arm-kernel, linux-kernel
V4L2_CID_STATELESS_HEVC_EXT_SPS_ST_RPS and
V4L2_CID_STATELESS_HEVC_EXT_SPS_LT_RPS are registered as dynamic-size
controls with a per-control element cap of 65. The V4L2 control core
enforces only the payload-element cap. It does not bound the spec-derived
count fields that the rkvdec HEVC helper later uses to walk fixed hardware
descriptor tables and temporary helper arrays:
- struct rkvdec_rps::refs[32]
- struct rkvdec_rps::short_term_ref_sets[64]
- struct calculated_rps_st_set::delta_poc_s0[16] / delta_poc_s1[16]
A userspace V4L2 client that can open the Rockchip RKVDEC m2m decoder node
may submit SPS/RPS controls whose counts exceed those capacities or whose
prediction reference index underflows. rkvdec_hevc_assemble_hw_rps() then
walks past the descriptor table or temporary-array bounds.
KASAN under a small KUnit harness wrapping the real helper reports
slab-out-of-bounds in all of:
- num_short_term_ref_pic_sets = 65 (write past short_term_ref_sets[64])
- num_long_term_ref_pics_sps = 33 (write past refs[32], intra-struct)
- ext_sps_st_rps[i].num_negative_pics or num_positive_pics > 16
(write past delta_poc_s0[16] inside calculated_rps_st_set)
- INTER_REF_PIC_SET_PRED with delta_idx_minus1 + 1 > i
(u8 ref_rps_idx underflow then OOB read on calculated_rps_st_sets)
Validate the SPS/RPS counts before calling the assembly helpers. The cap
values match both the HEVC spec ranges (num_short_term_ref_pic_sets <= 64,
num_long_term_ref_pics_sps <= 32) and the fixed driver descriptor and
helper-array capacities. Reject controls whose counts exceed those, and
reject prediction entries whose reference index would underflow.
Fixes: c9a59dc2acc7 ("media: rkvdec: Add HEVC support for the VDPU381 variant")
Signed-off-by: Michael Bommarito <michael.bommarito@gmail•com>
Assisted-by: Claude:claude-opus-4-7
---
I don't have a RK3588 / RK3576 board to confirm this through a real
/dev/videoN request path yet, but was convinced enough by:
1. Static reach: registration of EXT_SPS_ST_RPS / EXT_SPS_LT_RPS with
.dims = { 65 } at drivers/media/platform/rockchip/rkvdec/rkvdec.c
:239-287, the SPS-count-driven loops in rkvdec-hevc-common.c
:213-225 and :228-251, and v4l2-ctrls-core.c :1213-1277, which
validates only EXT RPS flags and not the spec-derived count fields.
2. A KUnit harness (separate, not in this patch) that allocates one
struct rkvdec_rps + a single calculated_rps_st_sets element via
kunit_kzalloc / kzalloc and calls the real
rkvdec_hevc_assemble_hw_rps() helper. Under UML + KASAN_GENERIC with
the kasan_multi_shot boot param, on a stock tree it produces these
reports:
BUG: KASAN: slab-out-of-bounds in
rkvdec_hevc_assemble_hw_rps+0xb0c/0x1080
(num_short_term_ref_pic_sets = 65, write of size 36 0 bytes past
the rps allocation)
BUG: KASAN: slab-out-of-bounds in
rkvdec_hevc_assemble_hw_rps (num_negative_pics = 64, write past
the single-element kzalloc'd calculated_rps_st_sets buffer)
BUG: KASAN: slab-use-after-free / slab-out-of-bounds reads via
u8 ref_rps_idx underflow at calculated_rps_st_sets[255]
(INTER_REF_PIC_SET_PRED with delta_idx_minus1 = 0, idx = 0)
The num_long_term_ref_pics_sps = 33 case is invisible to KASAN
(the OOB write lands inside struct rkvdec_rps) but corrupts
short_term_ref_sets[0]; the harness asserts that case explicitly.
3. Same harness on the patched tree: all five cases (four adversarial
plus a legitimate-limit regression with ST=64, LT=32, num_neg=1)
pass clean, no KASAN reports.
If hardware-side validation actually does reject these counts before
rkvdec_hevc_assemble_hw_rps() runs and this patch is unnecessary, please
say so and I will withdraw it. If it is reachable, I will follow up with
a runtime hardware splat once the Orange Pi board I bought arrives.
Let me know if you want a patch set with the KUnit harnesses too.
checkpatch.pl: 0 errors / 0 warnings.
.../rockchip/rkvdec/rkvdec-hevc-common.c | 49 +++++++++++++++++++
1 file changed, 49 insertions(+)
diff --git a/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c b/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c
index 3119f3bc9f98..895fb16bc572 100644
--- a/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c
+++ b/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c
@@ -408,9 +408,58 @@ static void rkvdec_hevc_prepare_hw_st_rps(struct rkvdec_hevc_run *run, struct rk
memcpy(cache, run->ext_sps_st_rps, sizeof(struct v4l2_ctrl_hevc_ext_sps_st_rps));
}
+/*
+ * V4L2 caps the EXT_SPS RPS payload length but not the SPS-derived counts
+ * that the helpers walk. Caps match the HEVC spec ranges.
+ */
+#define RKVDEC_HEVC_MAX_SHORT_TERM_REF_PIC_SETS 64
+#define RKVDEC_HEVC_MAX_LONG_TERM_REF_PICS_SPS 32
+#define RKVDEC_HEVC_MAX_RPS_NEG_POS_PICS 16
+
+static int rkvdec_hevc_validate_rps_ctrls(struct rkvdec_hevc_run *run)
+{
+ const struct v4l2_ctrl_hevc_sps *sps = run->sps;
+
+ if (run->ext_sps_lt_rps &&
+ sps->num_long_term_ref_pics_sps >
+ RKVDEC_HEVC_MAX_LONG_TERM_REF_PICS_SPS)
+ return -EINVAL;
+
+ if (run->ext_sps_st_rps) {
+ unsigned int i;
+
+ if (sps->num_short_term_ref_pic_sets >
+ RKVDEC_HEVC_MAX_SHORT_TERM_REF_PIC_SETS)
+ return -EINVAL;
+
+ for (i = 0; i < sps->num_short_term_ref_pic_sets; i++) {
+ const struct v4l2_ctrl_hevc_ext_sps_st_rps *r =
+ &run->ext_sps_st_rps[i];
+
+ if (r->num_negative_pics >
+ RKVDEC_HEVC_MAX_RPS_NEG_POS_PICS ||
+ r->num_positive_pics >
+ RKVDEC_HEVC_MAX_RPS_NEG_POS_PICS)
+ return -EINVAL;
+
+ if ((r->flags &
+ V4L2_HEVC_EXT_SPS_ST_RPS_FLAG_INTER_REF_PIC_SET_PRED) &&
+ (unsigned int)r->delta_idx_minus1 + 1 > i)
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
void rkvdec_hevc_assemble_hw_rps(struct rkvdec_hevc_run *run, struct rkvdec_rps *rps,
struct v4l2_ctrl_hevc_ext_sps_st_rps *st_cache)
{
+ if (rkvdec_hevc_validate_rps_ctrls(run)) {
+ pr_err_ratelimited("rkvdec: rejecting HEVC SPS/RPS controls with out-of-range counts\n");
+ return;
+ }
+
rkvdec_hevc_prepare_hw_st_rps(run, rps, st_cache);
rkvdec_hevc_assemble_hw_lt_rps(run, rps);
}
--
2.53.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH] media: rkvdec: hevc: cap EXT SPS RPS control counts before descriptor assembly
2026-05-13 18:19 [PATCH] media: rkvdec: hevc: cap EXT SPS RPS control counts before descriptor assembly Michael Bommarito
@ 2026-05-19 13:04 ` Detlev Casanova
2026-05-21 0:57 ` Michael Bommarito
2026-05-27 19:47 ` [PATCH v2 0/3] media: rkvdec: hevc: bound EXT SPS RPS control counts Michael Bommarito
1 sibling, 1 reply; 8+ messages in thread
From: Detlev Casanova @ 2026-05-19 13:04 UTC (permalink / raw)
To: Michael Bommarito, Ezequiel Garcia, Mauro Carvalho Chehab,
Heiko Stuebner, linux-media
Cc: linux-rockchip, linux-arm-kernel, linux-kernel
Hi Michael,
On 5/13/26 14:19, Michael Bommarito wrote:
> V4L2_CID_STATELESS_HEVC_EXT_SPS_ST_RPS and
> V4L2_CID_STATELESS_HEVC_EXT_SPS_LT_RPS are registered as dynamic-size
> controls with a per-control element cap of 65. The V4L2 control core
> enforces only the payload-element cap. It does not bound the spec-derived
> count fields that the rkvdec HEVC helper later uses to walk fixed hardware
> descriptor tables and temporary helper arrays:
>
> - struct rkvdec_rps::refs[32]
> - struct rkvdec_rps::short_term_ref_sets[64]
> - struct calculated_rps_st_set::delta_poc_s0[16] / delta_poc_s1[16]
Still, did you try just changing the cap to 64 (.cfg.dims = { 64 },) ?
You'd need a test that sets the control from userspace though.
It should refuse setting the control if there are more than 64 elements,
therefore the hevc decoder will not run any function using the count
values from the SPS (See rkvdec-vdpu381-hevc.c:601)
> A userspace V4L2 client that can open the Rockchip RKVDEC m2m decoder node
> may submit SPS/RPS controls whose counts exceed those capacities or whose
> prediction reference index underflows. rkvdec_hevc_assemble_hw_rps() then
> walks past the descriptor table or temporary-array bounds.
>
> KASAN under a small KUnit harness wrapping the real helper reports
> slab-out-of-bounds in all of:
>
> - num_short_term_ref_pic_sets = 65 (write past short_term_ref_sets[64])
> - num_long_term_ref_pics_sps = 33 (write past refs[32], intra-struct)
> - ext_sps_st_rps[i].num_negative_pics or num_positive_pics > 16
> (write past delta_poc_s0[16] inside calculated_rps_st_set)
> - INTER_REF_PIC_SET_PRED with delta_idx_minus1 + 1 > i
> (u8 ref_rps_idx underflow then OOB read on calculated_rps_st_sets)
>
> Validate the SPS/RPS counts before calling the assembly helpers. The cap
> values match both the HEVC spec ranges (num_short_term_ref_pic_sets <= 64,
> num_long_term_ref_pics_sps <= 32) and the fixed driver descriptor and
> helper-array capacities. Reject controls whose counts exceed those, and
> reject prediction entries whose reference index would underflow.
>
> Fixes: c9a59dc2acc7 ("media: rkvdec: Add HEVC support for the VDPU381 variant")
> Signed-off-by: Michael Bommarito <michael.bommarito@gmail•com>
> Assisted-by: Claude:claude-opus-4-7
> ---
>
> I don't have a RK3588 / RK3576 board to confirm this through a real
> /dev/videoN request path yet, but was convinced enough by:
>
> 1. Static reach: registration of EXT_SPS_ST_RPS / EXT_SPS_LT_RPS with
> .dims = { 65 } at drivers/media/platform/rockchip/rkvdec/rkvdec.c
> :239-287, the SPS-count-driven loops in rkvdec-hevc-common.c
> :213-225 and :228-251, and v4l2-ctrls-core.c :1213-1277, which
> validates only EXT RPS flags and not the spec-derived count fields.
>
> 2. A KUnit harness (separate, not in this patch) that allocates one
> struct rkvdec_rps + a single calculated_rps_st_sets element via
> kunit_kzalloc / kzalloc and calls the real
> rkvdec_hevc_assemble_hw_rps() helper. Under UML + KASAN_GENERIC with
> the kasan_multi_shot boot param, on a stock tree it produces these
> reports:
>
> BUG: KASAN: slab-out-of-bounds in
> rkvdec_hevc_assemble_hw_rps+0xb0c/0x1080
> (num_short_term_ref_pic_sets = 65, write of size 36 0 bytes past
> the rps allocation)
>
> BUG: KASAN: slab-out-of-bounds in
> rkvdec_hevc_assemble_hw_rps (num_negative_pics = 64, write past
> the single-element kzalloc'd calculated_rps_st_sets buffer)
>
> BUG: KASAN: slab-use-after-free / slab-out-of-bounds reads via
> u8 ref_rps_idx underflow at calculated_rps_st_sets[255]
> (INTER_REF_PIC_SET_PRED with delta_idx_minus1 = 0, idx = 0)
>
> The num_long_term_ref_pics_sps = 33 case is invisible to KASAN
> (the OOB write lands inside struct rkvdec_rps) but corrupts
> short_term_ref_sets[0]; the harness asserts that case explicitly.
>
> 3. Same harness on the patched tree: all five cases (four adversarial
> plus a legitimate-limit regression with ST=64, LT=32, num_neg=1)
> pass clean, no KASAN reports.
>
> If hardware-side validation actually does reject these counts before
> rkvdec_hevc_assemble_hw_rps() runs and this patch is unnecessary, please
> say so and I will withdraw it. If it is reachable, I will follow up with
> a runtime hardware splat once the Orange Pi board I bought arrives.
>
> Let me know if you want a patch set with the KUnit harnesses too.
>
> checkpatch.pl: 0 errors / 0 warnings.
>
> .../rockchip/rkvdec/rkvdec-hevc-common.c | 49 +++++++++++++++++++
> 1 file changed, 49 insertions(+)
>
> diff --git a/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c b/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c
> index 3119f3bc9f98..895fb16bc572 100644
> --- a/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c
> +++ b/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c
> @@ -408,9 +408,58 @@ static void rkvdec_hevc_prepare_hw_st_rps(struct rkvdec_hevc_run *run, struct rk
> memcpy(cache, run->ext_sps_st_rps, sizeof(struct v4l2_ctrl_hevc_ext_sps_st_rps));
> }
>
> +/*
> + * V4L2 caps the EXT_SPS RPS payload length but not the SPS-derived counts
> + * that the helpers walk. Caps match the HEVC spec ranges.
> + */
> +#define RKVDEC_HEVC_MAX_SHORT_TERM_REF_PIC_SETS 64
> +#define RKVDEC_HEVC_MAX_LONG_TERM_REF_PICS_SPS 32
> +#define RKVDEC_HEVC_MAX_RPS_NEG_POS_PICS 16
> +
> +static int rkvdec_hevc_validate_rps_ctrls(struct rkvdec_hevc_run *run)
> +{
> + const struct v4l2_ctrl_hevc_sps *sps = run->sps;
> +
> + if (run->ext_sps_lt_rps &&
> + sps->num_long_term_ref_pics_sps >
> + RKVDEC_HEVC_MAX_LONG_TERM_REF_PICS_SPS)
> + return -EINVAL;
> +
> + if (run->ext_sps_st_rps) {
> + unsigned int i;
> +
> + if (sps->num_short_term_ref_pic_sets >
> + RKVDEC_HEVC_MAX_SHORT_TERM_REF_PIC_SETS)
> + return -EINVAL;
> +
> + for (i = 0; i < sps->num_short_term_ref_pic_sets; i++) {
> + const struct v4l2_ctrl_hevc_ext_sps_st_rps *r =
> + &run->ext_sps_st_rps[i];
> +
> + if (r->num_negative_pics >
> + RKVDEC_HEVC_MAX_RPS_NEG_POS_PICS ||
> + r->num_positive_pics >
> + RKVDEC_HEVC_MAX_RPS_NEG_POS_PICS)
> + return -EINVAL;
> +
> + if ((r->flags &
> + V4L2_HEVC_EXT_SPS_ST_RPS_FLAG_INTER_REF_PIC_SET_PRED) &&
> + (unsigned int)r->delta_idx_minus1 + 1 > i)
> + return -EINVAL;
> + }
> + }
> +
> + return 0;
> +}
> +
> void rkvdec_hevc_assemble_hw_rps(struct rkvdec_hevc_run *run, struct rkvdec_rps *rps,
> struct v4l2_ctrl_hevc_ext_sps_st_rps *st_cache)
> {
> + if (rkvdec_hevc_validate_rps_ctrls(run)) {
> + pr_err_ratelimited("rkvdec: rejecting HEVC SPS/RPS controls with out-of-range counts\n");
> + return;
> + }
> +
> rkvdec_hevc_prepare_hw_st_rps(run, rps, st_cache);
> rkvdec_hevc_assemble_hw_lt_rps(run, rps);
> }
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] media: rkvdec: hevc: cap EXT SPS RPS control counts before descriptor assembly
2026-05-19 13:04 ` Detlev Casanova
@ 2026-05-21 0:57 ` Michael Bommarito
2026-05-22 13:06 ` Detlev Casanova
0 siblings, 1 reply; 8+ messages in thread
From: Michael Bommarito @ 2026-05-21 0:57 UTC (permalink / raw)
To: Detlev Casanova
Cc: Ezequiel Garcia, Mauro Carvalho Chehab, Heiko Stuebner,
linux-media, linux-rockchip, linux-arm-kernel, linux-kernel
On Tue, May 19, 2026 at 9:04 AM Detlev Casanova
<detlev.casanova@collabora•com> wrote:
> Still, did you try just changing the cap to 64 (.cfg.dims = { 64 },) ?
> You'd need a test that sets the control from userspace though.
>
> It should refuse setting the control if there are more than 64 elements,
> therefore the hevc decoder will not run any function using the count
> values from the SPS (See rkvdec-vdpu381-hevc.c:601)
Sure, I can test that and send a v2 for ST. My understanding is that
we have four spots we need to check across the flow though:
1. ST count > 64
2. LT count > 32
3. num_negative_pics / num_positive_pics > 16
4. delta_idx_minus1 + 1 > i
So would you also want the same .cfg approach for the LT cap?
Thanks,
Mike
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] media: rkvdec: hevc: cap EXT SPS RPS control counts before descriptor assembly
2026-05-21 0:57 ` Michael Bommarito
@ 2026-05-22 13:06 ` Detlev Casanova
0 siblings, 0 replies; 8+ messages in thread
From: Detlev Casanova @ 2026-05-22 13:06 UTC (permalink / raw)
To: Michael Bommarito
Cc: Ezequiel Garcia, Mauro Carvalho Chehab, Heiko Stuebner,
linux-media, linux-rockchip, linux-arm-kernel, linux-kernel
Hi Mike,
On 5/20/26 20:57, Michael Bommarito wrote:
> On Tue, May 19, 2026 at 9:04 AM Detlev Casanova
> <detlev.casanova@collabora•com> wrote:
>> Still, did you try just changing the cap to 64 (.cfg.dims = { 64 },) ?
>> You'd need a test that sets the control from userspace though.
>>
>> It should refuse setting the control if there are more than 64 elements,
>> therefore the hevc decoder will not run any function using the count
>> values from the SPS (See rkvdec-vdpu381-hevc.c:601)
> Sure, I can test that and send a v2 for ST. My understanding is that
> we have four spots we need to check across the flow though:
>
> 1. ST count > 64
> 2. LT count > 32
ST and LT can be handled through cfg.dims and when num_negative_pics /
num_positive_pics are non-zero, the code check that ST and LT have
actually been set. If not, they can be skipped.
> 3. num_negative_pics / num_positive_pics > 16
This could be checked in the validation step of the control in v4l2-core.
> 4. delta_idx_minus1 + 1 > i
This one would need to be checked in the construction code indeed.
>
> So would you also want the same .cfg approach for the LT cap?
So yes for LT dimensions.
Detlev.
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v2 0/3] media: rkvdec: hevc: bound EXT SPS RPS control counts
2026-05-13 18:19 [PATCH] media: rkvdec: hevc: cap EXT SPS RPS control counts before descriptor assembly Michael Bommarito
2026-05-19 13:04 ` Detlev Casanova
@ 2026-05-27 19:47 ` Michael Bommarito
2026-05-27 19:47 ` [PATCH v2 1/3] media: rkvdec: hevc: tighten EXT SPS RPS control dimensions Michael Bommarito
` (2 more replies)
1 sibling, 3 replies; 8+ messages in thread
From: Michael Bommarito @ 2026-05-27 19:47 UTC (permalink / raw)
To: Detlev Casanova, Ezequiel Garcia, Mauro Carvalho Chehab
Cc: Hans Verkuil, Nicolas Dufresne, Heiko Stuebner, linux-media,
linux-rockchip, linux-arm-kernel, linux-kernel, stable
A userspace V4L2 client that can open the Rockchip RKVDEC m2m decoder
node can submit HEVC EXT SPS RPS controls whose spec-derived count
fields exceed the driver's fixed hardware descriptor tables and
temporary helper arrays. KASAN under a KUnit harness wrapping the real
rkvdec_hevc_assemble_hw_rps() helper confirms slab-out-of-bounds writes
on num_short_term_ref_pic_sets > 64, num_long_term_ref_pics_sps > 32,
num_negative/positive_pics > 16, and an OOB read via u8 ref_rps_idx
underflow when delta_idx_minus1 + 1 > idx.
v2 splits the single validation function from v1 into three layers at
the appropriate level, per Detlev's review:
1/3 Tighten .cfg.dims on EXT_SPS_ST_RPS (65 -> 64) and
EXT_SPS_LT_RPS (65 -> 32) to match the HEVC spec limits and
let the V4L2 control framework reject oversized payloads.
2/3 Add SPS ST/LT count validation plus
num_negative_pics / num_positive_pics validation in
v4l2-ctrls-core.c so every consumer driver is protected.
3/3 Guard the delta_idx_minus1 underflow in
st_ref_pic_set_prediction() in the rkvdec construction code.
Cc: stable@vger•kernel.org
Changes in v2:
- Split the monolithic rkvdec-hevc-common.c validation function into
dims, v4l2-core, and construction-code layers as Detlev suggested.
- Drop the rkvdec-local #defines and pr_err_ratelimited; the V4L2
framework now handles the bulk of the rejection, including the SPS
count fields that drive the rkvdec loops.
Michael Bommarito (3):
media: rkvdec: hevc: tighten EXT SPS RPS control dimensions
media: v4l2-ctrls: validate HEVC EXT SPS RPS counts
media: rkvdec: hevc: guard INTER_REF_PIC_SET_PRED index underflow
.../platform/rockchip/rkvdec/rkvdec-hevc-common.c | 3 +++
drivers/media/platform/rockchip/rkvdec/rkvdec.c | 4 ++--
drivers/media/v4l2-core/v4l2-ctrls-core.c | 15 +++++++++++++++
3 files changed, 20 insertions(+), 2 deletions(-)
base-commit: 7fd2df204f342fc17d1a0bfcd474b24232fb0f32
--
2.53.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v2 1/3] media: rkvdec: hevc: tighten EXT SPS RPS control dimensions
2026-05-27 19:47 ` [PATCH v2 0/3] media: rkvdec: hevc: bound EXT SPS RPS control counts Michael Bommarito
@ 2026-05-27 19:47 ` Michael Bommarito
2026-05-27 19:47 ` [PATCH v2 2/3] media: v4l2-ctrls: validate HEVC EXT SPS RPS counts Michael Bommarito
2026-05-27 19:47 ` [PATCH v2 3/3] media: rkvdec: hevc: guard INTER_REF_PIC_SET_PRED index underflow Michael Bommarito
2 siblings, 0 replies; 8+ messages in thread
From: Michael Bommarito @ 2026-05-27 19:47 UTC (permalink / raw)
To: Detlev Casanova, Ezequiel Garcia, Mauro Carvalho Chehab
Cc: Hans Verkuil, Nicolas Dufresne, Heiko Stuebner, linux-media,
linux-rockchip, linux-arm-kernel, linux-kernel, stable
The VDPU381 HEVC driver registers V4L2_CID_STATELESS_HEVC_EXT_SPS_ST_RPS
and V4L2_CID_STATELESS_HEVC_EXT_SPS_LT_RPS with .cfg.dims = { 65 }, but
the HEVC spec caps num_short_term_ref_pic_sets at 64 (ITU-T H.265 7.4.8)
and num_long_term_ref_pics_sps at 32 (7.4.3.2.1). The hardware descriptor
table (struct rkvdec_rps) sizes match those spec limits: refs[32] and
short_term_ref_sets[64].
Reduce the dims to { 64 } and { 32 } respectively so the V4L2 control
framework rejects oversized payloads before any driver code runs.
Fixes: c9a59dc2acc7 ("media: rkvdec: Add HEVC support for the VDPU381 variant")
Cc: stable@vger•kernel.org
Suggested-by: Detlev Casanova <detlev.casanova@collabora•com>
Assisted-by: Claude:claude-opus-4-7
Signed-off-by: Michael Bommarito <michael.bommarito@gmail•com>
---
drivers/media/platform/rockchip/rkvdec/rkvdec.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/media/platform/rockchip/rkvdec/rkvdec.c b/drivers/media/platform/rockchip/rkvdec/rkvdec.c
index 1d1e9bfef8e96..4fd06f4c04db0 100644
--- a/drivers/media/platform/rockchip/rkvdec/rkvdec.c
+++ b/drivers/media/platform/rockchip/rkvdec/rkvdec.c
@@ -278,12 +278,12 @@ static const struct rkvdec_ctrl_desc vdpu38x_hevc_ctrl_descs[] = {
{
.cfg.id = V4L2_CID_STATELESS_HEVC_EXT_SPS_ST_RPS,
.cfg.ops = &rkvdec_ctrl_ops,
- .cfg.dims = { 65 },
+ .cfg.dims = { 64 },
},
{
.cfg.id = V4L2_CID_STATELESS_HEVC_EXT_SPS_LT_RPS,
.cfg.ops = &rkvdec_ctrl_ops,
- .cfg.dims = { 65 },
+ .cfg.dims = { 32 },
},
};
--
2.53.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH v2 2/3] media: v4l2-ctrls: validate HEVC EXT SPS RPS counts
2026-05-27 19:47 ` [PATCH v2 0/3] media: rkvdec: hevc: bound EXT SPS RPS control counts Michael Bommarito
2026-05-27 19:47 ` [PATCH v2 1/3] media: rkvdec: hevc: tighten EXT SPS RPS control dimensions Michael Bommarito
@ 2026-05-27 19:47 ` Michael Bommarito
2026-05-27 19:47 ` [PATCH v2 3/3] media: rkvdec: hevc: guard INTER_REF_PIC_SET_PRED index underflow Michael Bommarito
2 siblings, 0 replies; 8+ messages in thread
From: Michael Bommarito @ 2026-05-27 19:47 UTC (permalink / raw)
To: Detlev Casanova, Ezequiel Garcia, Mauro Carvalho Chehab
Cc: Hans Verkuil, Nicolas Dufresne, Heiko Stuebner, linux-media,
linux-rockchip, linux-arm-kernel, linux-kernel, stable
The HEVC SPS control carries the short-term and long-term RPS counts
that decoder drivers use to walk the matching EXT SPS dynamic arrays.
Reject SPS values that exceed the HEVC limits of 64 short-term sets and
32 long-term references so drivers cannot later index beyond those
controls.
Also reject EXT SPS ST RPS entries whose negative or positive picture
counts exceed the 16-entry arrays, or whose combined delta-POC count
exceeds the HEVC DPB maximum.
Fixes: c9a59dc2acc7 ("media: rkvdec: Add HEVC support for the VDPU381 variant")
Cc: stable@vger•kernel.org
Suggested-by: Detlev Casanova <detlev.casanova@collabora•com>
Assisted-by: Claude:claude-opus-4-7
Signed-off-by: Michael Bommarito <michael.bommarito@gmail•com>
---
drivers/media/v4l2-core/v4l2-ctrls-core.c | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c
index 6b375720e395c..a1d773e5de20c 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-core.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c
@@ -16,6 +16,9 @@
static const union v4l2_ctrl_ptr ptr_null;
+#define V4L2_HEVC_MAX_SHORT_TERM_REF_PIC_SETS 64
+#define V4L2_HEVC_MAX_LONG_TERM_REF_PICS_SPS 32
+
static void fill_event(struct v4l2_event *ev, struct v4l2_ctrl *ctrl,
u32 changes)
{
@@ -1213,6 +1216,10 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx,
case V4L2_CTRL_TYPE_HEVC_SPS:
p_hevc_sps = p;
+ if (p_hevc_sps->num_short_term_ref_pic_sets >
+ V4L2_HEVC_MAX_SHORT_TERM_REF_PIC_SETS)
+ return -EINVAL;
+
if (!(p_hevc_sps->flags & V4L2_HEVC_SPS_FLAG_PCM_ENABLED)) {
p_hevc_sps->pcm_sample_bit_depth_luma_minus1 = 0;
p_hevc_sps->pcm_sample_bit_depth_chroma_minus1 = 0;
@@ -1223,6 +1230,9 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx,
if (!(p_hevc_sps->flags &
V4L2_HEVC_SPS_FLAG_LONG_TERM_REF_PICS_PRESENT))
p_hevc_sps->num_long_term_ref_pics_sps = 0;
+ else if (p_hevc_sps->num_long_term_ref_pics_sps >
+ V4L2_HEVC_MAX_LONG_TERM_REF_PICS_SPS)
+ return -EINVAL;
break;
case V4L2_CTRL_TYPE_HEVC_PPS:
@@ -1267,6 +1277,11 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx,
if (p_hevc_st_rps->flags & ~V4L2_HEVC_EXT_SPS_ST_RPS_FLAG_INTER_REF_PIC_SET_PRED)
return -EINVAL;
+ if (p_hevc_st_rps->num_negative_pics > 16 ||
+ p_hevc_st_rps->num_positive_pics > 16 ||
+ p_hevc_st_rps->num_negative_pics +
+ p_hevc_st_rps->num_positive_pics > 16)
+ return -EINVAL;
break;
case V4L2_CTRL_TYPE_HEVC_EXT_SPS_LT_RPS:
--
2.53.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH v2 3/3] media: rkvdec: hevc: guard INTER_REF_PIC_SET_PRED index underflow
2026-05-27 19:47 ` [PATCH v2 0/3] media: rkvdec: hevc: bound EXT SPS RPS control counts Michael Bommarito
2026-05-27 19:47 ` [PATCH v2 1/3] media: rkvdec: hevc: tighten EXT SPS RPS control dimensions Michael Bommarito
2026-05-27 19:47 ` [PATCH v2 2/3] media: v4l2-ctrls: validate HEVC EXT SPS RPS counts Michael Bommarito
@ 2026-05-27 19:47 ` Michael Bommarito
2 siblings, 0 replies; 8+ messages in thread
From: Michael Bommarito @ 2026-05-27 19:47 UTC (permalink / raw)
To: Detlev Casanova, Ezequiel Garcia, Mauro Carvalho Chehab
Cc: Hans Verkuil, Nicolas Dufresne, Heiko Stuebner, linux-media,
linux-rockchip, linux-arm-kernel, linux-kernel, stable
st_ref_pic_set_prediction() computes the reference RPS index as
st_rps_idx - (delta_idx_minus1 + 1) per HEVC spec equation 7-59.
Both operands are u8, so when delta_idx_minus1 + 1 exceeds the
current index the subtraction wraps and the subsequent array access
at calculated_rps_st_sets[ref_rps_idx] reads far out of bounds.
A userspace V4L2 client that can open the RKVDEC m2m decoder can
submit an EXT_SPS_ST_RPS control with INTER_REF_PIC_SET_PRED set
and delta_idx_minus1 crafted to trigger the underflow.
Reject the entry early when the reference index would underflow.
Fixes: c9a59dc2acc7 ("media: rkvdec: Add HEVC support for the VDPU381 variant")
Cc: stable@vger•kernel.org
Suggested-by: Detlev Casanova <detlev.casanova@collabora•com>
Assisted-by: Claude:claude-opus-4-7
Signed-off-by: Michael Bommarito <michael.bommarito@gmail•com>
---
drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c b/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c
index 3119f3bc9f98b..898d1ce74f38a 100644
--- a/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c
+++ b/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c
@@ -268,6 +268,9 @@ static void st_ref_pic_set_prediction(struct rkvdec_hevc_run *run, int idx,
int i, j;
int dPoc;
+ if ((unsigned int)rps_data->delta_idx_minus1 + 1 > idx)
+ return;
+
ref_rps_idx = st_rps_idx - (rps_data->delta_idx_minus1 + 1); /* 7-59 */
delta_rps = (1 - 2 * rps_data->delta_rps_sign) *
(rps_data->abs_delta_rps_minus1 + 1); /* 7-60 */
--
2.53.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
end of thread, other threads:[~2026-05-27 19:48 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-05-13 18:19 [PATCH] media: rkvdec: hevc: cap EXT SPS RPS control counts before descriptor assembly Michael Bommarito
2026-05-19 13:04 ` Detlev Casanova
2026-05-21 0:57 ` Michael Bommarito
2026-05-22 13:06 ` Detlev Casanova
2026-05-27 19:47 ` [PATCH v2 0/3] media: rkvdec: hevc: bound EXT SPS RPS control counts Michael Bommarito
2026-05-27 19:47 ` [PATCH v2 1/3] media: rkvdec: hevc: tighten EXT SPS RPS control dimensions Michael Bommarito
2026-05-27 19:47 ` [PATCH v2 2/3] media: v4l2-ctrls: validate HEVC EXT SPS RPS counts Michael Bommarito
2026-05-27 19:47 ` [PATCH v2 3/3] media: rkvdec: hevc: guard INTER_REF_PIC_SET_PRED index underflow Michael Bommarito
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox