Subject: Re: [BUG] mtk-t7xx driver on aarch64/cortex-a53

From: Robert <[email-masked]>
Date: Fri, 22 Mar 2024 13:18:38 +0100

decode_stacktarce.sh output:

[  236.043132] kmodloader: loading kernel modules from /etc/modules.d/*
[  236.055665] mtk_t7xx 0003:01:00.0: assign IRQ: got 113
[  236.060820] mtk_t7xx 0003:01:00.0: enabling device (0000 -> 0002)
[  236.067013] mtk_t7xx 0003:01:00.0: enabling bus mastering
[  236.073529] (unnamed net_device) (dummy): netif_napi_add_weight() called with weight 128
[  236.082757] mtk-pcie-gen3 11280000.pcie: msi#0x1 address_hi 0x0 address_lo 0x11280c00 data 1
[  236.091292] mtk-pcie-gen3 11280000.pcie: msi#0x2 address_hi 0x0 address_lo 0x11280c00 data 2
[  236.094906] Unable to handle kernel paging request at virtual address ffffffc08521f004
[  236.099754] mtk-pcie-gen3 11280000.pcie: msi#0x3 address_hi 0x0 address_lo 0x11280c00 data 3
[  236.101619] Unable to handle kernel paging request at virtual address ffffffc08521d004
[  236.101624] Mem abort info:
[  236.101625]   ESR = 0x0000000096000061
[  236.101627]   EC = 0x25: DABT (current EL), IL = 32 bits
[  236.101630]   SET = 0, FnV = 0
[  236.101632]   EA = 0, S1PTW = 0
[  236.101634]   FSC = 0x21: alignment fault
[  236.101636] Data abort info:
[  236.101637]   ISV = 0, ISS = 0x00000061, ISS2 = 0x00000000
[  236.101639]   CM = 0, WnR = 1, TnD = 0, TagAccess = 0
[  236.101641]   GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
[  236.101644] swapper pgtable: 4k pages, 39-bit VAs, pgdp=0000000046ad6000
[  236.101646] [ffffffc08521d004] pgd=100000013ffff003, p4d=100000013ffff003, pud=100000013ffff003, pmd=0068000020a00711
[  236.101656] Internal error: Oops: 0000000096000061 [#1] SMP
[  236.101659] Modules linked in: mtk_t7xx(+) qcserial pppoe ppp_async option nft_fib_inet nf_flow_table_inet mt7921u(O) mt7921s(O) mt7921e(O) mt7921_common(O) iwlmvm(O) iwldvm(O) usb_wwan rndis_host qmi_wwan pppox ppp_generic nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject nft_redir nft_quota nft_numgen nft_nat nft_masq nft_log nft_limit nft_hash nft_flow_offload nft_fib_ipv6 nft_fib_ipv4 nft_fib nft_ct nft_chain_nat nf_tables nf_nat nf_flow_table nf_conntrack mt7996e(O) mt792x_usb(O) mt792x_lib(O) mt7915e(O) mt76_usb(O) mt76_sdio(O) mt76_connac_lib(O) mt76(O) mac80211(O) iwlwifi(O) huawei_cdc_ncm cfg80211(O) cdc_ncm cdc_ether wwan usbserial usbnet slhc sfp rtc_pcf8563 nfnetlink nf_reject_ipv6 nf_reject_ipv4 nf_log_syslog nf_defrag_ipv6 nf_defrag_ipv4 mt6577_auxadc mdio_i2c libcrc32c compat(O) cdc_wdm cdc_acm at24 crypto_safexcel pwm_fan i2c_gpio i2c_smbus industrialio i2c_algo_bit i2c_mux_reg i2c_mux_pca954x i2c_mux_pca9541 i2c_mux_gpio i2c_mux dummy oid_registry tun sha512_arm64 sha1_ce sha1_generic seqiv
[  236.101784]  md5 geniv des_generic libdes cbc authencesn authenc leds_gpio xhci_plat_hcd xhci_pci xhci_mtk_hcd xhci_hcd nvme nvme_core gpio_button_hotplug(O) dm_mirror dm_region_hash dm_log dm_crypt dm_mod dax usbcore usb_common ptp aquantia pps_core mii tpm encrypted_keys trusted
[  236.101825] CPU: 2 PID: 4331 Comm: kworker/u9:1 Tainted: G           O       6.6.22 #0
[  236.101830] Hardware name: Bananapi BPI-R4 (DT)
[  236.101833] Workqueue: md_hk_wq t7xx_fsm_uninit [mtk_t7xx]
[  236.101850] pstate: 804000c5 (Nzcv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[  236.101854] pc : t7xx_cldma_hw_set_start_addr+0x1c/0x3c mtk_t7xx
[  236.101866] lr : t7xx_cldma_start+0xac/0x13c mtk_t7xx
[  236.101876] sp : ffffffc084ae3d30
[  236.101877] x29: ffffffc084ae3d30 x28: 0000000000000000 x27: 0000000000000000
[  236.101883] x26: 0000000000000000 x25: ffffff80c2786380 x24: ffffff80c7bd8805
[  236.101888] x23: 0000000000000000 x22: ffffff80c820c9b8 x21: ffffff80c820c128
[  236.101893] x20: 0000000000000001 x19: ffffff80c820c080 x18: 0000000000000014
[  236.101898] x17: 0000000092cfe117 x16: 0000000024f52690 x15: 00000000966b8fca
[  236.101903] x14: 0000000003e1b73b x13: 00000000b4eeebcb x12: 0000000000000001
[  236.101908] x11: 0000000000000000 x10: 0000000000000000 x9 : 0000000000000000
[  236.101913] x8 : ffffff80cdcaa8b4 x7 : ffffff80c820c818 x6 : 0000000000000018
[  236.101918] x5 : 0000000000000870 x4 : 0000000000000000 x3 : 0000000000000000
[  236.101922] x2 : 000000010dc9d000 x1 : ffffffc08521d004 x0 : ffffffc08521d004
[  236.101928] Call trace:
[  236.101930] t7xx_cldma_hw_set_start_addr+0x1c/0x3c mtk_t7xx
[  236.101941] t7xx_fsm_uninit+0x578/0x5ec mtk_t7xx
[  236.101951] process_one_work+0x154/0x2a0 
[  236.101960] worker_thread+0x2ac/0x488 
[  236.101964] kthread+0xe0/0xec 
[  236.101969] ret_from_fork+0x10/0x20 
[ 236.101976] Code: f9400800 91001000 8b214001 d50332bf (f9000022)
All code
========
   0:	f9400800 	ldr	x0, [x0, #16]
   4:	91001000 	add	x0, x0, #0x4
   8:	8b214001 	add	x1, x0, w1, uxtw
   c:	d50332bf 	dmb	oshst
  10:*	f9000022 	str	x2, [x1]		<-- trapping instruction

Code starting with the faulting instruction
===========================================
   0:	f9000022 	str	x2, [x1]
[  236.101978] ---[ end trace 0000000000000000 ]---
[  236.107624] Mem abort info:
[  236.107625]   ESR = 0x0000000096000061
[  236.107626]   EC = 0x25: DABT (current EL), IL = 32 bits
[  236.107629]   SET = 0, FnV = 0
[  236.107630]   EA = 0, S1PTW = 0
[  236.107632]   FSC = 0x21: alignment fault
[  236.107633] Data abort info:
[  236.107634]   ISV = 0, ISS = 0x00000061, ISS2 = 0x00000000
[  236.107636]   CM = 0, WnR = 1, TnD = 0, TagAccess = 0
[  236.107638]   GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
[  236.107640] swapper pgtable: 4k pages, 39-bit VAs, pgdp=0000000046ad6000
[  236.107643] [ffffffc08521f004] pgd=100000013ffff003, p4d=100000013ffff003, pud=100000013ffff003, pmd=0068000020a00711
[  236.109126] pstore: backend (ramoops) writing error (-28)
[  236.116167] mtk-pcie-gen3 11280000.pcie: msi#0x4 address_hi 0x0 address_lo 0x11280c00 data 4
[  236.124017] Kernel panic - not syncing: Oops: Fatal exception
[  236.124020] SMP: stopping secondary CPUs
[  237.136907] SMP: failed to stop secondary CPUs 0,2
[  237.136910] Kernel Offset: disabled
[  237.136911] CPU features: 0x0,00000000,20000000,1000400b
[  237.136914] Memory Limit: none
PANIC at PC : 0x000000004300490c
Temat: [BUG] mtk-t7xx driver on aarch64/cortex-a53
Data: 2024-03-22 9:06
Nadawca: "Robert" &lt;[email-masked]>
Adresat: [email-masked]; 
DW: [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; 

> 
>> Hi
> 
> We are facing possible bug in the driver mtk-t7xx under OpenWrt linux
6.1/6.6.
> 
> From first glance it looks like the driver is accessing an address in
the PCIe MMIO range in 32-bit alignment (ffffffc084a1d004) 
> but likely the SoC only supports 64-bit aligned (so only addresses
ending on 0 or 8 will work) access there, 
> hence the [ 294.051349] FSC = 0x21: alignment fault.
> 
> 
> 
> [CUT]
> ...
> [   12.285356] mtk_t7xx 0003:01:00.0: assign IRQ: got 113
> [   12.290512] mtk_t7xx 0003:01:00.0: enabling device (0000 -> 0002)
> [   12.296612] mtk_t7xx 0003:01:00.0: enabling bus mastering
> [   12.303087] (unnamed net_device) (dummy): netif_napi_add_weight()
called with weight 128
> [   12.312160] mtk-pcie-gen3 11280000.pcie: msi#0x1 address_hi 0x0
address_lo 0x11280c00 data 1
> [   12.320666] mtk-pcie-gen3 11280000.pcie: msi#0x2 address_hi 0x0
address_lo 0x11280c00 data 2
> [   12.329153] mtk-pcie-gen3 11280000.pcie: msi#0x3 address_hi 0x0
address_lo 0x11280c00 data 3
> [   12.331706] Unable to handle kernel paging request at virtual
address ffffffc083a1d004
> [   12.345488] Mem abort info:
> [   12.345518] mtk-pcie-gen3 11280000.pcie: msi#0x4 address_hi 0x0
address_lo 0x11280c00 data 4
> [   12.348269]   ESR = 0x0000000096000061
> [   12.356716] mtk-pcie-gen3 11280000.pcie: msi#0x5 address_hi 0x0
address_lo 0x11280c00 data 5
> [   12.360421]   EC = 0x25: DABT (current EL), IL = 32 bits
> [   12.368862] mtk-pcie-gen3 11280000.pcie: msi#0x6 address_hi 0x0
address_lo 0x11280c00 data 6
> [   12.374133]   SET = 0, FnV = 0
> [   12.374135]   EA = 0, S1PTW = 0
> [   12.382574] mtk-pcie-gen3 11280000.pcie: msi#0x7 address_hi 0x0
address_lo 0x11280c00 data 7
> [   12.385593]   FSC = 0x21: alignment fault
> [   12.388751] mtk-pcie-gen3 11280000.pcie: msi#0x8 address_hi 0x0
address_lo 0x11280c00 data 8
> [   12.397137] Data abort info:
> [   12.397138]   ISV = 0, ISS = 0x00000061, ISS2 = 0x00000000
> [   12.397140]   CM = 0, WnR = 1, TnD = 0, TagAccess = 0
> [   12.422958]   GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
> [   12.428261] swapper pgtable: 4k pages, 39-bit VAs,
pgdp=0000000046ad6000
> [   12.434950] [ffffffc083a1d004] pgd=100000013ffff003,
p4d=100000013ffff003, pud=100000013ffff003, pmd=0068000020a00711
> [   12.445552] Internal error: Oops: 0000000096000061 [#1] SMP
> [   12.451113] Modules linked in: mtk_t7xx mt7996e(O) mt792x_usb(O)
mt792x_lib(O) mt7915e(O) mt76_usb(O) mt76_sdio(O) mt76_connac_lib(O)
mt76(O) mac80211(O) iwlwifi(O) huawei_cdc_ncm cfg80211(O) cdc_ncm cdc_ether
wwan usbserial usbnet slhc sfp rtc_pcf8563 nfnetlink nf_reject_ipv6
nf_reject_ipv4 nf_log_syslog nf_defrag_ipv6 nf_defrag_ipv4 mt6577_auxadc
mdio_i2c libcrc32c compat(O) cdc_wdm cdc_acm at24 crypto_safexcel pwm_fan
i2c_gpio i2c_smbus industrialio i2c_algo_bit i2c_mux_reg i2c_mux_pca954x
i2c_mux_pca9541 i2c_mux_gpio i2c_mux dummy oid_registry tun sha512_arm64
sha1_ce sha1_generic seqiv md5 geniv des_generic libdes cbc authencesn
authenc leds_gpio xhci_plat_hcd xhci_pci xhci_mtk_hcd xhci_hcd nvme
nvme_core gpio_button_hotplug(O) dm_mirror dm_region_hash dm_log dm_crypt
dm_mod dax usbcore usb_common ptp aquantia pps_core mii tpm encrypted_keys
trusted
> [   12.526834] CPU: 2 PID: 1526 Comm: kworker/u9:0 Tainted: G         
 O       6.6.22 #0
> [   12.534740] Hardware name: Bananapi BPI-R4 (DT)
> [   12.539259] Workqueue: md_hk_wq t7xx_fsm_uninit [mtk_t7xx]
> [   12.542217] sfp sfp2: module XICOM            XC-SFP+-SR       rev
A    sn C202307141626    dc 230714
> [   12.544746] pstate: 804000c5 (Nzcv daIF +PAN -UAO -TCO -DIT -SSBS
BTYPE=--)
> [   12.554144] mtk_soc_eth 15100000.ethernet eth1: switched to
inband/10gbase-r link mode
> [   12.561064] pc : t7xx_cldma_hw_set_start_addr+0x1c/0x3c [mtk_t7xx]
> [   12.575139] lr : t7xx_cldma_start+0xac/0x13c [mtk_t7xx]
> [   12.580359] sp : ffffffc0813dbd30
> [   12.583661] x29: ffffffc0813dbd30 x28: 0000000000000000 x27:
0000000000000000
> [   12.590786] x26: 0000000000000000 x25: ffffff80c6888140 x24:
ffffff80c11f7e05
> [   12.591893] sfp sfp1: module XICOM            XC-SFP+-LR       rev
A    sn C202307141707    dc 230714
> [   12.593855] hwmon hwmon2: temp1_input not attached to any thermal
zone
> [   12.597909] x23: 0000000000000000 x22: ffffff80c0fdb9b8
> [   12.607297] mtk_soc_eth 15100000.ethernet eth2: switched to
inband/10gbase-r link mode
> [   12.613792]  x21: ffffff80c0fdb128
> [   12.613794] x20: 0000000000000001 x19: ffffff80c0fdb080 x18:
0000000000000014
> [   12.631986] hwmon hwmon3: temp1_input not attached to any thermal
zone
> [   12.637419] x17: 00000000752a0f20 x16: 00000000468ff952 x15:
00000000246d1885
> [   12.651056] x14: 00000000b48c7dff x13: 000000001b6aa29e x12:
0000000000000001
> [   12.658180] x11: 0000000000000000 x10: 0000000000000000 x9 :
0000000000000000
> [   12.665304] x8 : ffffff80c90fdfb4 x7 : ffffff80c0fdb818 x6 :
0000000000000018
> [   12.672428] x5 : 0000000000000870 x4 : 0000000000000000 x3 :
0000000000000000
> [   12.679553] x2 : 00000001090f0000 x1 : ffffffc083a1d004 x0 :
ffffffc083a1d004
> [   12.686678] Call trace:
> [   12.689114]  t7xx_cldma_hw_set_start_addr+0x1c/0x3c [mtk_t7xx]
> [   12.694942]  t7xx_fsm_uninit+0x578/0x5ec [mtk_t7xx]
> [   12.699814]  process_one_work+0x154/0x2a0
> [   12.703818]  worker_thread+0x2ac/0x488
> [   12.707558]  kthread+0xe0/0xec
> [   12.710603]  ret_from_fork+0x10/0x20
> [   12.714172] Code: f9400800 91001000 8b214001 d50332bf (f9000022)
> [   12.720253] ---[ end trace 0000000000000000 ]---
> [   12.731558] pstore: backend (ramoops) writing error (-28)
> [   12.736948] Kernel panic - not syncing: Oops: Fatal exception
> [   12.742680] SMP: stopping secondary CPUs
> [   12.746593] Kernel Offset: disabled
> [   12.750069] CPU features: 0x0,00000000,20000000,1000400b
> [   12.755370] Memory Limit: none
> [   12.765071] Rebooting in 1 seconds..
> PANIC at PC : 0x000000004300490c
> 
> Kindly
> Robert
> 
>