Left: | ||
Right: |
LEFT | RIGHT |
---|---|
1 Netmap NetDevice | 1 Netmap NetDevice |
2 ------------------------- | 2 ------------------------- |
3 .. include:: replace.txt | 3 .. include:: replace.txt |
4 .. highlight:: cpp | 4 .. highlight:: cpp |
5 | 5 |
6 .. heading hierarchy: | 6 .. heading hierarchy: |
7 ------------- Chapter | 7 ------------- Chapter |
8 ************* Section (#.#) | 8 ************* Section (#.#) |
9 ============= Subsection (#.#.#) | 9 ============= Subsection (#.#.#) |
10 ############# Paragraph (no number) | 10 ############# Paragraph (no number) |
11 | 11 |
Tom Henderson
2018/04/14 20:22:33
Suggest to start by defining netmap to the uniniti
| |
12 The ``fd-net-device`` module provides the ``NetmapNetDevice`` class, a class der ived | 12 The ``fd-net-device`` module provides the ``NetmapNetDevice`` class, a class der ived |
13 from the ``FdNetDevice`` which is able to read and write traffic using a netmap file descriptor | 13 from the ``FdNetDevice`` which is able to read and write traffic using a netmap file descriptor |
14 provided by the user. This file descriptor must be associated to a real ethernet device | 14 provided by the user. This file descriptor must be associated to a real ethernet device |
15 in the host machine. | 15 in the host machine. |
16 netmap is a fast packets processing that bypasses the host networking stack and gains | 16 netmap is a fast packets processing that bypasses the host networking stack and gains |
17 direct access to network device. | 17 direct access to network device. |
18 | 18 |
19 The ``EmuFdNetDeviceHelper`` provides the method ``SetNetmapMode`` to enable | 19 The ``EmuFdNetDeviceHelper`` provides the method ``SetNetmapMode`` to enable |
20 the emulation of a real device in netmap mode. | 20 the emulation of a real device in netmap mode. |
21 | 21 |
(...skipping 18 matching lines...) Expand all Loading... | |
40 This device provides support for flow-control by means a separate thread. The de vice use | 40 This device provides support for flow-control by means a separate thread. The de vice use |
41 a NetDeviceQueue interface to represent the netmap transmission ring. In the ope ration of write, after | 41 a NetDeviceQueue interface to represent the netmap transmission ring. In the ope ration of write, after |
42 the packet copy in the transmission ring, if there is no room for other packets the device stops the | 42 the packet copy in the transmission ring, if there is no room for other packets the device stops the |
43 queue and notify a separate thread about this event. The separate thread waits f or the next slot available | 43 queue and notify a separate thread about this event. The separate thread waits f or the next slot available |
44 for transmission and when there is room for other packets wake the queue. In mea nwhile, the main process | 44 for transmission and when there is room for other packets wake the queue. In mea nwhile, the main process |
45 run separately. | 45 run separately. |
46 | 46 |
47 The netmap device provides support for queue limits in the netmap ring by period ic notification to queue | 47 The netmap device provides support for queue limits in the netmap ring by period ic notification to queue |
48 limits about the transmitted bytes. | 48 limits about the transmitted bytes. |
49 | 49 |
50 Requirments | 50 Requirments |
Tom Henderson
2018/04/14 20:22:33
Requirements
| |
51 ************ | 51 ************ |
52 | 52 |
53 This device requires a netmap installation in the host machine. The user can fol low the | 53 This device requires a netmap installation in the host machine. The user can fol low the |
54 tutorial provided by netmap or the wiki page related to this device to download, to configure and | 54 tutorial provided by netmap or the wiki page related to this device to download, to configure and |
55 to install netmap in the host machine. | 55 to install netmap in the host machine. |
Tom Henderson
2018/04/14 20:22:34
Can you provide a URL to readers?
| |
56 | 56 |
57 After the installation of netmap, supports for this device can be found in the o utput of the ``waf configure``: | 57 After the installation of netmap, supports for this device can be found in the o utput of the ``waf configure``: |
Tom Henderson
2018/04/14 20:22:32
support
| |
58 | 58 |
59 .. sourcecode:: text | 59 .. sourcecode:: text |
60 | 60 |
61 Netmap Emulated FdNetDevice : enabled | 61 Netmap Emulated FdNetDevice : enabled |
62 | 62 |
63 The user must set the right privileges for the creator program by enabling the ` `--enable-sudo`` | 63 The user must set the right privileges for the creator program by enabling the ` `--enable-sudo`` |
64 flag when performing ``waf configure``. | 64 flag when performing ``waf configure``. |
Tom Henderson
2018/04/14 20:22:33
Will 'enabled' show if the user has not provided -
| |
65 | 65 |
66 When the user use netmap in emulated mode, it may disable the generic_txqdisc by setting to 0 this | 66 When the user use netmap in emulated mode, it may disable the generic_txqdisc by setting to 0 this |
Tom Henderson
2018/04/14 20:22:32
Please define better the user-perceived difference
| |
67 value of netmap (By default netmap in emulated mode introduce a netmap aware que ue disc on the netmap switched device). | 67 value of netmap (By default netmap in emulated mode introduce a netmap aware que ue disc on the netmap switched device). |
Tom Henderson
2018/04/14 20:22:33
'introduces a netmap aware'
| |
68 Also, the user can reduce the generic_ringsize value of netmap (netmap uses a de fault value of 1024 when it is unable | 68 Also, the user can reduce the generic_ringsize value of netmap (netmap uses a de fault value of 1024 when it is unable |
69 to read the real tx ring from the device, i.e. when the device do not supports e thtool). | 69 to read the real tx ring from the device, i.e. when the device do not supports e thtool). |
Tom Henderson
2018/04/14 20:22:34
Please clarify, when would a user want to do this
| |
70 | 70 |
71 Finally, when the user use netmap in native mode, it must compile and load the n etmap aware device driver. The user can follow | 71 Finally, when the user use netmap in native mode, it must compile and load the n etmap aware device driver. The user can follow |
72 the netmap doc page or the wiki page related to this device to compile and load the netmap aware device driver. | 72 the netmap doc page or the wiki page related to this device to compile and load the netmap aware device driver. |
Tom Henderson
2018/04/14 20:22:32
Can you provide an example set of commands to the
| |
73 | 73 |
74 Usage | 74 Usage |
75 ***** | 75 ***** |
76 | 76 |
77 After the configuration, the user must load the netmap module in the host machin e before | 77 After the configuration, the user must load the netmap module in the host machin e before |
78 to use this device, e.g. by the insmod of the netmap kernel module generated wit h the netmap compilation. | 78 to use this device, e.g. by the insmod of the netmap kernel module generated wit h the netmap compilation. |
Tom Henderson
2018/04/14 20:22:34
this seems to duplicate the last paragraph of the
| |
79 | 79 |
80 The NetmapNetDevice will be used to interact with the host system and the user c an follow the typical | 80 The NetmapNetDevice will be used to interact with the host system and the user c an follow the typical |
81 usage pattern for the FdNetDevice in such cases. For instance, the user sets the netmap emulation mode on the | 81 usage pattern for the FdNetDevice in such cases. For instance, the user sets the netmap emulation mode on the |
82 FdNetDeviceEmuHelper, sets the real time simulation and enables the checksum cal culaction: | 82 FdNetDeviceEmuHelper, sets the real time simulation and enables the checksum cal culaction: |
83 | 83 |
84 :: | 84 :: |
85 | 85 |
86 EmuFdNetDeviceHelper emu; | 86 EmuFdNetDeviceHelper emu; |
87 emu.SetDeviceName (deviceName); | 87 emu.SetDeviceName (deviceName); |
88 emu.SetNetmapMode (); | 88 emu.SetNetmapMode (); |
89 ... | 89 ... |
90 GlobalValue::Bind ("SimulatorImplementationType", StringValue ("ns3::RealtimeS imulatorImpl")); | 90 GlobalValue::Bind ("SimulatorImplementationType", StringValue ("ns3::RealtimeS imulatorImpl")); |
91 GlobalValue::Bind ("ChecksumEnabled", BooleanValue (true)); | 91 GlobalValue::Bind ("ChecksumEnabled", BooleanValue (true)); |
92 | 92 |
93 The real device interface must be in promiscuous mode and the IP of the emulated device must be different from | 93 The real device interface must be in promiscuous mode and the IP of the emulated device must be different from |
94 the real OS IP for that device but on the same subnet. | 94 the real OS IP for that device but on the same subnet. |
95 | 95 |
96 Examples | 96 Examples |
97 ******** | 97 ******** |
98 | 98 |
99 Three examples are provided about the NetmapNetDevice: | 99 Four examples are provided about the NetmapNetDevice: |
100 | 100 |
101 * ``fd-netmap-emu-ping.cc``: This example is aimed at providing a test of this device on a real ethernet device. | 101 * ``fd-netmap-emu-ping.cc``: This example is aimed at providing a test of this device on a real ethernet device. |
102 This example sends ICMP traffic over a real device emulated in netmap mode wit h UDP background traffic. | 102 This example sends ICMP traffic over a real device emulated in netmap mode wit h UDP background traffic. |
Tom Henderson
2018/04/14 20:22:33
What does the test show? That pings can be receiv
| |
103 | 103 |
104 * ``fd-netmap-emu-onoff.cc``: This example is aimed at measuring the throughput of the | 104 * ``fd-netmap-emu-onoff.cc``: This example is aimed at measuring the throughput of the |
105 NetmapNetDevice connected back-to-back to a simulated (with this example) UDP or TCP server or real application | 105 NetmapNetDevice connected back-to-back to a simulated (with this example) UDP or TCP server or real application |
106 UDP or TCP server (e.g. iperf). | 106 UDP or TCP server (e.g. iperf). |
107 | 107 |
108 * ``fd-netmap-emu-send.cc``: This example is aimed at measuring the maximum tra nsmission rate in pps achievable | |
109 with the NetmapNetDevice. | |
110 | |
108 * ``fd-netmap-emu-tc.cc``: This example build a node with two interface in netm ap mode. | 111 * ``fd-netmap-emu-tc.cc``: This example build a node with two interface in netm ap mode. |
109 The user can explore different qdiscs behaviours on the backlog of a device em ulated with netmap. | 112 The user can explore different qdiscs behaviours on the backlog of a device em ulated with netmap. |
110 | 113 |
111 Performance evaluation | 114 Performance evaluation |
112 ********************** | 115 ********************** |
113 | 116 |
114 This device was evaluated by comparing the behavior of this device with the fd-n et-device in emulated mode on a back-to-back through | 117 Throughput and rtt |
115 ethernet. We used the ``netmap-emu-onoff.cc`` example to perform this evaluation . | 118 ================== |
116 We compiled the |ns3| code in optimized mode to perfrom our evaluation. | |
117 | 119 |
118 The throughput performance of this device is very similar to the socket one (i.e ., the fd-net-device configured in emulation mode). | 120 This device was evaluated by comparing the behavior of the NetmapNetDevice with the FdNetDevice in emulated mode on a back-to-back through |
119 We evaluated a UDP throughput up to 700 Mbps on a 1 Gbps link with netmap in emu lated and in native mode with a packet size of 1400 bytes. | 121 ethernet. We used the ``fd-netmap-emu-onoff.cc`` example to perform this evaluat ion. and the ns-3 code compiled in optimized mode. |
120 In case of TCP the throughput performance is lower than the UDP case and it is o f about 400 Mbps (we used a segment size of 1400 bytes). | 122 |
123 We used two hosts equipped with Intel i7 at 3.8 GHz cpu and Intel e1000e 1 Gbps ethernet card. The two host was connected by a cross ethernet cable. | |
Tom Henderson
2018/04/14 20:22:33
'two hosts were'
| |
124 Clint side, the OS IP for the emulated device was 10.0.1.1 while the ns-3 IP was 10.0.1.11, the interface was in promisc mode and we used netmap in emulated mod e | |
Tom Henderson
2018/04/14 20:22:32
Client
| |
125 (i.e., without driver changes) with its generic_txqdisc=0 (to keep out the queue disc netmap aware introduced by netmap in emulated mode). | |
126 Server side the OS IP was 10.0.1.2. | |
127 The server at specified address had an instance of an UDP server (e.g., an insta nce of iperf in UDP server mode with "iperf -s -p 8000 -u") and an estimation | |
128 of the received throughput could be performed by Wireshark in capture mode on th e server interface. | |
129 | |
130 The user can run the example client side by: | |
131 | |
132 ./waf --run "src/fd-net-device/examples/fd-netmap-emu-onoff --netmapMode=true -- ping=true --transportProt='Udp' --server='10.0.1.2' --bql=false" | |
133 | |
134 to evaluate the NetmapNetDevice. | |
Tom Henderson
2018/04/14 20:22:33
Can you add a comment about why you set bql to fal
| |
135 | |
136 The user can perform the evaluation in the same scenario with --netmapMode=false to evaluate the FdNetDevice in emulation mode. | |
137 The example output will be the ping reponse, the backlog in traffic-control qdis c and the bytes inflight in the netmap transmission ring (in case of emulation w ith netmap). | |
138 Finally, in case of emulation in netmap mode, the user can explore the use of BQ L with --bql=true and observes a reduction of the bytes in flight in the | |
139 netmap transmission ring. | |
Tom Henderson
2018/04/14 20:22:32
Can you better clarify, how does the user observe
| |
140 | |
141 The throughput performance of this device is very similar to the socket one (i.e ., the FdNetDevice configured in emulation mode). | |
142 We evaluated a received UDP throughput up to 700 Mbps on a 1 Gbps link with netm ap in emulated (and in native) mode with a packet size of 1400 bytes. | |
143 In case of TCP the throughput performance is lower than the UDP case and it is o f about 500 Mbps. | |
121 We evaluated with oprofile a performance bottleneck in the modules IP and TCP of ns-3. | 144 We evaluated with oprofile a performance bottleneck in the modules IP and TCP of ns-3. |
Tom Henderson
2018/04/14 20:22:34
Restate? "We evaluated performance with oprofile,
| |
122 | 145 |
123 Conversely, the delay performance with netmap is different from the socket one. Indeed, the delay evaluated with netmap is smaller than | 146 Conversely, the delay performance with netmap is different from the socket one. Indeed, the delay evaluated by means the ping response with netmap |
Tom Henderson
2018/04/14 20:22:33
'Indeed, the delay evaluated by means...' Can you
| |
124 the socket one. This is due to a more realistic device queue and queue disc occu pancy with netmap compared with the higher | 147 is smaller than the socket one. This is due to a more realistic queues occupancy with netmap compared with the higher |
125 socket buffer occupancy (where the delay grows in an uncontrolled manner). | 148 socket buffer (where the delay grows in an uncontrolled manner). Further optimiz ations can leads to more stable delay with netmap. |
126 | 149 |
127 Finally, in case of link saturation, e.g. in case of 100 Mbps ethernet link, the support for flow control and queue limits for this device | 150 Finally, in case of link saturation, e.g. in case of 100 Mbps ethernet link, the support for flow control and queue limits for this device |
128 allows keeping a backlog in traffic-control where the packets can be managed by advanced queue discs. The user can explore different qdiscs | 151 allows keeping a backlog in traffic-control where the packets can be managed by advanced queue discs. The user can explore different qdiscs |
129 behaviours with netmap emulated device. | 152 behaviors with netmap emulated devices. |
153 | |
154 Pps | |
155 === | |
156 | |
157 A further evaluation regards the maximum transmission rate achievable with the N etmapNetDevice in pps. | |
Tom Henderson
2018/04/14 20:22:32
s/regards/is available regarding/
| |
158 We used the ``fd-netmap-emu-send.cc`` example to perform this evaluation and the ns-3 code compiled in optimized mode. | |
159 | |
160 We used the same hosts configuration of the previous evaluation without iperf se rver side. | |
161 | |
162 The user can run the example with | |
163 | |
164 ./waf --run "src/fd-net-device/examples/fd-netmap-emu-send --netmapMode=true --m ode=0" | |
165 | |
166 to evaluate the maximum pps achievable with netmap with the lowest primitive (i. e., the Write method). | |
167 | |
168 The user can perform the evaluation in the same scenario with --netmapMode=false to evaluate the FdNetDevice in emulation mode with the lowest | |
169 primitive (i.e., the socket write). | |
170 The example output will be the number of packets sent in 1 s, the actual period of measure in ms, the sent failed, and the estimated throughput. | |
Tom Henderson
2018/04/14 20:22:34
s/sent failed/number of sent failed/ ?
| |
171 | |
172 The pps achievable with the NetmapNetDevice was up to 1.38 Mpps on the e1000e ad apter on the i7 cpu at full frequency of 3.8 GHZ. | |
173 This value is according to netmap results on the Intel e1000e 1 Gbps adapter ava ilable in the netmap documentation. | |
174 The pps provide by socket (i.e., FdNetDevice in emulated mode) is in line to the netmap one on this testbed at full cpu frequency. | |
175 | |
176 We explored the reduction of the cpu frequency to better present the gain provid ed by netmap on 1 Gbps link. | |
177 With cpu frequency of 2.5 GHz (set with cpufrequtil) we observed a gain in the p ps with netmap of about 10% (netmap achieves 1.38 Mpps while | |
178 the socket achieves 1.25 Mpps). A further reduction to 2 GHz leads to a gain of about 30 % with netmap (netmap achieves 1.38 Mpps while | |
179 the socket achieves 1.01 Mpps). | |
LEFT | RIGHT |