Subject: Re: HTB classify perfomance

From: Badalian Vyacheslav <[email-masked]>
Date: Fri, 11 Jan 2008 11:53:19 +0300

New info. Wait some time and reset oprifile statistic (i think info 
abount ipt_unregister_table its get what run some script... ).
That clear info after add FILTER:

First PC
CPU: P4 / Xeon, speed 3409.96 MHz (estimated)
Counted GLOBAL_POWER_EVENTS events (time during which processor is not 
stopped) with a unit mask of 0x01 (mandatory) count 100000
samples  %        app name                 symbol name
1158171  19.1292  vmlinux                  ipt_do_table
722416   11.9319  vmlinux                  e1000_intr
627406   10.3627  vmlinux                  u32_classify
565286    9.3367  vmlinux                  e1000_irq_enable
269309    4.4481  vmlinux                  htb_dequeue
191016    3.1550  vmlinux                  ip_route_input
187127    3.0907  vmlinux                  sfq_dequeue
172775    2.8537  vmlinux                  e1000_clean_tx_irq
154654    2.5544  vmlinux                  e1000_clean_rx_irq
146926    2.4267  vmlinux                  sfq_enqueue
116782    1.9289  vmlinux                  htb_add_to_wait_tree
79398     1.3114  vmlinux                  rb_erase
74411     1.2290  vmlinux                  e1000_xmit_frame
65451     1.0810  vmlinux                  kfree
59966     0.9904  vmlinux                  irq_entries_start
59893     0.9892  vmlinux                  eth_type_trans
55510     0.9168  vmlinux                  dev_queue_xmit
52688     0.8702  vmlinux                  e1000_alloc_rx_buffers

> Hello all.
> I N days try to tune system for best performance and see strange thing.
>
> Have N htb classes
> root class is HTB. param: default 7 (if not classify - go to 1:7)
>
> filters classify only mached ip. others go to HTB DEFAULT rule.
>
> run oprofile:
> First pc (htb and iptables compile in kernel):
> CPU: P4 / Xeon, speed 3409.94 MHz (estimated)
> Counted GLOBAL_POWER_EVENTS events (time during which processor is not 
> stopped) with a unit mask of 0x01 (mandatory) count 100000
> samples  %        app name                 symbol name
> 743501   47.6081  vmlinux                  htb_classify
> 208718   13.3647  vmlinux                  ipt_do_table
> 94473     6.0493  vmlinux                  u32_classify
> 43088     2.7590  vmlinux                  e1000_intr
> 35086     2.2466  vmlinux                  e1000_clean_tx_irq
> 34925     2.2363  vmlinux                  ip_route_input
> 33972     2.1753  vmlinux                  e1000_irq_enable
> 33788     2.1635  vmlinux                  htb_dequeue
> 29197     1.8696  vmlinux                  e1000_clean_rx_irq
> 20177     1.2920  vmlinux                  sfq_dequeue
> 17825     1.1414  vmlinux                  sfq_enqueue
> 15135     0.9691  vmlinux                  e1000_xmit_frame
> 15123     0.9684  vmlinux                  eth_type_trans
> 13081     0.8376  vmlinux                  kfree
> 12153     0.7782  vmlinux                  dev_queue_xmit
>
> Second PC (htb and iptables is modules)
> CPU: P4 / Xeon with 2 hyper-threads, speed 3192.35 MHz (estimated)
> Counted GLOBAL_POWER_EVENTS events (time during which processor is not 
> stopped) with a unit mask of 0x01 (mandatory) count 100000
> samples  %        app name                 symbol name
> 102108   30.7351  sch_htb                  (no symbols)
> 21559     6.4894  vmlinux                  e1000_intr
> 17428     5.2459  cls_u32                  (no symbols)
> 13887     4.1801  ip_tables                (no symbols)
> 11984     3.6072  sch_sfq                  (no symbols)
> 11785     3.5473  vmlinux                  e1000_irq_enable
> 9684      2.9149  vmlinux                  mwait_idle_with_hints
> 9227      2.7774  vmlinux                  e1000_clean_rx_irq
> 8686      2.6145  vmlinux                  e1000_clean_tx_irq
> 6747      2.0309  vmlinux                  ip_route_input
> 6533      1.9665  vmlinux                  irq_entries_start
> 6419      1.9322  vmlinux                  e1000_xmit_frame
> 5605      1.6871  vmlinux                  dev_queue_xmit
> 4030      1.2131  vmlinux                  __kfree_skb
> 3997      1.2031  vmlinux                  __qdisc_run
> 3931      1.1833  vmlinux                  e1000_clean
> 3565      1.0731  vmlinux                  net_rx_action
> 3518      1.0589  vmlinux                  ip_rcv
> 3377      1.0165  vmlinux                  getnstimeofday
> 3215      0.9677  vmlinux                  rb_erase
> 2973      0.8949  vmlinux                  eth_type_trans
> 2707      0.8148  vmlinux                  ip_output
> 2586      0.7784  vmlinux                  handle_fasteoi_irq
>
> Hmm.. strange... look to code htb_classify i see only one place where 
> it may get many CPU.
>
> ok... try to add to the end of tc batch file..
> filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 800:: match 
> ip protocol 1 0x00 flowid 1:7
> filter add dev eth0 protocol ip parent 1:0 prio 5 u32 ht 800:: match 
> ip protocol 1 0x00 flowid 1:7
> (offtopic... strange... i not found that i can add filter without any 
> match)
>
> Wow!
> CPU: P4 / Xeon, speed 3409.94 MHz (estimated)
> Counted GLOBAL_POWER_EVENTS events (time during which processor is not 
> stopped) with a unit mask of 0x01 (mandatory) count 100000
> samples  %        app name                 symbol name
> 153128   20.9497  vmlinux                  ipt_unregister_table
> 121569   16.6321  vmlinux                  e1000_request_irq
> 60727     8.3082  vmlinux                  e1000_update_itr
> 47241     6.4631  vmlinux                  u32_delete
> 25836     3.5347  vmlinux                  htb_dequeue
> 18304     2.5042  vmlinux                  ipt_do_table
> 15980     2.1862  vmlinux                  mwait_idle_with_hints
> 15977     2.1858  vmlinux                  irq_entries_start
> 13337     1.8247  vmlinux                  htb_classify
> 12512     1.7118  vmlinux                  __ip_route_output_key
> 8821      1.2068  vmlinux                  sfq_init
> 8495      1.1622  vmlinux                  e1000_clean_rx_irq
> 8408      1.1503  vmlinux                  htb_enqueue
> 8018      1.0970  vmlinux                  e1000_xmit_frame
> 7867      1.0763  vmlinux                  e1000_clean_tx_ring
> 6336      0.8668  vmlinux                  htb_delete
> 5828      0.7973  vmlinux                  ___pskb_trim
> 5781      0.7909  vmlinux                  s_start
> 5234      0.7161  vmlinux                  e1000_clean_rx_irq_ps
> 4504      0.6162  vmlinux                  cache_alloc_refill
> 4133      0.5654  vmlinux                  radix_tree_delete
>
> Second PC
> CPU: P4 / Xeon with 2 hyper-threads, speed 3192.35 MHz (estimated)
> Counted GLOBAL_POWER_EVENTS events (time during which processor is not 
> stopped) with a unit mask of 0x01 (mandatory) count 100000
> samples  %        app name                 symbol name
> 37747    13.3616  sch_htb                  (no symbols)
> 23606     8.3560  vmlinux                  e1000_intr
> 18158     6.4275  cls_u32                  (no symbols)
> 14726     5.2127  ip_tables                (no symbols)
> 13137     4.6502  vmlinux                  e1000_irq_enable
> 12307     4.3564  sch_sfq                  (no symbols)
> 9974      3.5306  vmlinux                  mwait_idle_with_hints
> 9855      3.4884  vmlinux                  e1000_clean_rx_irq
> 9077      3.2131  vmlinux                  e1000_clean_tx_irq
> 7293      2.5816  vmlinux                  irq_entries_start
> 6956      2.4623  vmlinux                  ip_route_input
> 6652      2.3547  vmlinux                  e1000_xmit_frame
> 6202      2.1954  vmlinux                  dev_queue_xmit
> 4403      1.5586  vmlinux                  __kfree_skb
> 4230      1.4973  vmlinux                  net_rx_action
> 4224      1.4952  vmlinux                  e1000_clean
> 4042      1.4308  vmlinux                  __qdisc_run
> 3513      1.2435  vmlinux                  ip_rcv
> 3509      1.2421  vmlinux                  getnstimeofday
> 3377      1.1954  vmlinux                  rb_erase
> 3191      1.1295  vmlinux                  eth_type_trans
> 2953      1.0453  vmlinux                  handle_fasteoi_irq
> 2830      1.0018  vmlinux                  ip_output
>
>
> Looks great!
> I hope i found interesting place for optimization.
>
> Thanks.
> Slavon.
> -- 
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to [email-masked]
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>