Minor comments included (mainly documentation). What do you think about making netmap just a mode of operation on the existing 'fd-emu-*.cc' examples? I'm wondering about reducing the number of similar examples to compile. https://codereview.appspot.com/330220043/diff/40001/src/fd-net-device/doc/net... File src/fd-net-device/doc/netmap-net-device.rst (right): https://codereview.appspot.com/330220043/diff/40001/src/fd-net-device/doc/net... src/fd-net-device/doc/netmap-net-device.rst:11: Suggest to start by defining netmap to the uninitiated reader, and providing a reference. See how references are added by consulting another documentation file, such as in the LTE module. https://codereview.appspot.com/330220043/diff/40001/src/fd-net-device/doc/net... src/fd-net-device/doc/netmap-net-device.rst:50: Requirments Requirements https://codereview.appspot.com/330220043/diff/40001/src/fd-net-device/doc/net... src/fd-net-device/doc/netmap-net-device.rst:55: to install netmap in the host machine. Can you provide a URL to readers? https://codereview.appspot.com/330220043/diff/40001/src/fd-net-device/doc/net... src/fd-net-device/doc/netmap-net-device.rst:57: After the installation of netmap, supports for this device can be found in the output of the ``waf configure``: support https://codereview.appspot.com/330220043/diff/40001/src/fd-net-device/doc/net... src/fd-net-device/doc/netmap-net-device.rst:64: flag when performing ``waf configure``. Will 'enabled' show if the user has not provided --enable-sudo? Please clarify. https://codereview.appspot.com/330220043/diff/40001/src/fd-net-device/doc/net... src/fd-net-device/doc/netmap-net-device.rst:66: When the user use netmap in emulated mode, it may disable the generic_txqdisc by setting to 0 this Please define better the user-perceived difference between running in emulated mode and native mode. https://codereview.appspot.com/330220043/diff/40001/src/fd-net-device/doc/net... src/fd-net-device/doc/netmap-net-device.rst:67: value of netmap (By default netmap in emulated mode introduce a netmap aware queue disc on the netmap switched device). 'introduces a netmap aware' https://codereview.appspot.com/330220043/diff/40001/src/fd-net-device/doc/net... src/fd-net-device/doc/netmap-net-device.rst:69: to read the real tx ring from the device, i.e. when the device do not supports ethtool). Please clarify, when would a user want to do this (how would he or she know to do this)? https://codereview.appspot.com/330220043/diff/40001/src/fd-net-device/doc/net... src/fd-net-device/doc/netmap-net-device.rst:72: the netmap doc page or the wiki page related to this device to compile and load the netmap aware device driver. Can you provide an example set of commands to the reader, such as 'sudo insmod ./netmap.ko sudo insmod ./ixgbe/ixgbe.ko sudo insmod ./e1000/e1000.ko'? https://codereview.appspot.com/330220043/diff/40001/src/fd-net-device/doc/net... src/fd-net-device/doc/netmap-net-device.rst:78: to use this device, e.g. by the insmod of the netmap kernel module generated with the netmap compilation. this seems to duplicate the last paragraph of the preceding section. Can you clarify? https://codereview.appspot.com/330220043/diff/40001/src/fd-net-device/doc/net... src/fd-net-device/doc/netmap-net-device.rst:102: This example sends ICMP traffic over a real device emulated in netmap mode with UDP background traffic. What does the test show? That pings can be received (i.e. it at least is minimally functional)? https://codereview.appspot.com/330220043/diff/40001/src/fd-net-device/doc/net... src/fd-net-device/doc/netmap-net-device.rst:123: We used two hosts equipped with Intel i7 at 3.8 GHz cpu and Intel e1000e 1 Gbps ethernet card. The two host was connected by a cross ethernet cable. 'two hosts were' https://codereview.appspot.com/330220043/diff/40001/src/fd-net-device/doc/net... src/fd-net-device/doc/netmap-net-device.rst:124: Clint side, the OS IP for the emulated device was 10.0.1.1 while the ns-3 IP was 10.0.1.11, the interface was in promisc mode and we used netmap in emulated mode Client https://codereview.appspot.com/330220043/diff/40001/src/fd-net-device/doc/net... src/fd-net-device/doc/netmap-net-device.rst:134: to evaluate the NetmapNetDevice. Can you add a comment about why you set bql to false here? https://codereview.appspot.com/330220043/diff/40001/src/fd-net-device/doc/net... src/fd-net-device/doc/netmap-net-device.rst:139: netmap transmission ring. Can you better clarify, how does the user observe the bytes in flight in the netmap transmission ring, and roughly what numbers do you see on your system when bql is true vs. bql is false? https://codereview.appspot.com/330220043/diff/40001/src/fd-net-device/doc/net... src/fd-net-device/doc/netmap-net-device.rst:144: We evaluated with oprofile a performance bottleneck in the modules IP and TCP of ns-3. Restate? "We evaluated performance with oprofile, and found that the performance bottleneck in the process was in the TCP/IP modules of ns-3, and not related to the netmap device." https://codereview.appspot.com/330220043/diff/40001/src/fd-net-device/doc/net... src/fd-net-device/doc/netmap-net-device.rst:146: Conversely, the delay performance with netmap is different from the socket one. Indeed, the delay evaluated by means the ping response with netmap 'Indeed, the delay evaluated by means...' Can you please reword this sentence, it is unclear? https://codereview.appspot.com/330220043/diff/40001/src/fd-net-device/doc/net... src/fd-net-device/doc/netmap-net-device.rst:157: A further evaluation regards the maximum transmission rate achievable with the NetmapNetDevice in pps. s/regards/is available regarding/ https://codereview.appspot.com/330220043/diff/40001/src/fd-net-device/doc/net... src/fd-net-device/doc/netmap-net-device.rst:170: The example output will be the number of packets sent in 1 s, the actual period of measure in ms, the sent failed, and the estimated throughput. s/sent failed/number of sent failed/ ? https://codereview.appspot.com/330220043/diff/40001/src/fd-net-device/example... File src/fd-net-device/examples/fd-netmap-emu-onoff.cc (right): https://codereview.appspot.com/330220043/diff/40001/src/fd-net-device/example... src/fd-net-device/examples/fd-netmap-emu-onoff.cc:49: * destinated to the OS IP for that interface will be placed in the sw rings of the netmap. I think you are saying that: + std::string client ("10.0.1.11"); + std::string server ("10.0.1.22"); + std::string netmask ("255.255.255.0"); should be changed to (unused) addresses on the same network as the OS IP for that interface (or the OS interface should not carry an IP address?) https://codereview.appspot.com/330220043/diff/40001/src/fd-net-device/example... File src/fd-net-device/examples/fd-netmap-emu-ping.cc (right): https://codereview.appspot.com/330220043/diff/40001/src/fd-net-device/example... src/fd-net-device/examples/fd-netmap-emu-ping.cc:68: * destinated to the OS IP for that interface will be placed in the sw rings of the netmap. destined https://codereview.appspot.com/330220043/diff/40001/src/fd-net-device/helper/... File src/fd-net-device/helper/emu-fd-net-device-helper.h (right): https://codereview.appspot.com/330220043/diff/40001/src/fd-net-device/helper/... src/fd-net-device/helper/emu-fd-net-device-helper.h:67: void SetNetmapMode (); should this take a boolean argument allowing user to set to either true or false? https://codereview.appspot.com/330220043/diff/40001/src/network/utils/net-dev... File src/network/utils/net-device-queue-interface.h (right): https://codereview.appspot.com/330220043/diff/40001/src/network/utils/net-dev... src/network/utils/net-device-queue-interface.h:95: bool IsStopped (void); rather than take away the constness of this method, should m_mutex be declared mutable?