TCP measurement relative scripts

add  tcp measurement scripts for single flow and multi flow
add plot scripts for output files from measurement scripts

[Xiaodong Zhang, 2016-09-28 08:21:11]

asd

https://codereview.appspot.com/307370043/diff/1/measure.sh
File measure.sh (right):

https://codereview.appspot.com/307370043/diff/1/measure.sh#newcode9
measure.sh:9: TIME=50
what is this!

https://codereview.appspot.com/307370043/diff/1/measure.sh#newcode15
measure.sh:15: modprobe tcp_probe port=5201 full=1
what is this?

[pandan0315, 2016-09-28 11:33:14]

https://codereview.appspot.com/307370043/diff/1/measure.sh
File measure.sh (right):

https://codereview.appspot.com/307370043/diff/1/measure.sh#newcode9
measure.sh:9: TIME=50
On 2016/09/28 08:21:11, zhaxd123 wrote:
> what is this!

I forgot to write  comments for every code. I will add later.

https://codereview.appspot.com/307370043/diff/1/measure.sh#newcode15
measure.sh:15: modprobe tcp_probe port=5201 full=1
On 2016/09/28 08:21:11, zhaxd123 wrote:
> what is this?

I forgot to write  comments for every code. I will add later.

[njwade1233, 2016-09-28 11:42:36]

https://codereview.appspot.com/307370043/diff/1/measure.sh
File measure.sh (right):

https://codereview.appspot.com/307370043/diff/1/measure.sh#newcode9
measure.sh:9: TIME=50
On 2016/09/28 08:21:11, zhaxd123 wrote:
> what is this!

I don't know!

[khalidomer88, 2016-09-28 17:55:43]

I think we should push the first draft

[pandan0315, 2016-09-29 11:14:46]

https://codereview.appspot.com/307370043/diff/1/measure.sh
File measure.sh (right):

https://codereview.appspot.com/307370043/diff/1/measure.sh#newcode9
measure.sh:9: TIME=50
On 2016/09/28 11:42:36, njwade1233 wrote:
> On 2016/09/28 08:21:11, zhaxd123 wrote:
> > what is this!
> 
> I don't know!

Acknowledged.

https://codereview.appspot.com/307370043/diff/1/measure.sh#newcode9
measure.sh:9: TIME=50
On 2016/09/28 11:33:14, pandan0315 wrote:
> On 2016/09/28 08:21:11, zhaxd123 wrote:
> > what is this!
> 
> I forgot to write  comments for every code. I will add later.

Done.

[pandan0315, 2016-10-09 18:53:40]

modify script for loading new tcp_probe module

[khalidomer88, 2016-10-09 19:03:09]

Good work, I like it. I have two minor comments to be considered. Keep up the
good work

https://codereview.appspot.com/307370043/diff/40001/measure.sh
File measure.sh (right):

https://codereview.appspot.com/307370043/diff/40001/measure.sh#newcode1
measure.sh:1: #!/bin/bash
It is nice that the script has comments, but it would be nicer to have some top
description of the script function on top. two lines shows what is does this
script do and how to use it (by using simple comments on top only).

https://codereview.appspot.com/307370043/diff/40001/measure.sh#newcode58
measure.sh:58: gnuplot plot.gp
I think it is better to have different names for the plot files, such as
rtt_rto_cwnd.gp and bandwidth.gp . we keep the shifted tag for the shifted
(zoomed) graphs

[SunnyGu0928, 2016-10-09 19:06:14]

The comments seem nice.

https://codereview.appspot.com/307370043/diff/40001/measure.sh
File measure.sh (right):

https://codereview.appspot.com/307370043/diff/40001/measure.sh#newcode61
measure.sh:61: gnuplot plot1_shifted.gp
GLTM

[pandan0315, 2016-10-09 19:17:49]

modified according to group members's suggestions

[SunnyGu0928, 2016-10-09 19:21:30]

Then it is clearer now. Good job Dan!

https://codereview.appspot.com/307370043/diff/60001/measure.sh
File measure.sh (right):

https://codereview.appspot.com/307370043/diff/60001/measure.sh#newcode65
measure.sh:65: gnuplot throughput_zoomed.gp
GLTM

[khalidomer88, 2016-10-09 19:22:22]

LGTM

[njwade1233, 2016-10-09 19:23:59]

On 2016/10/09 19:17:49, pandan0315 wrote:
> modified according to group members's suggestions

GLTM

[Xiaodong Zhang, 2016-10-09 19:32:19]

https://codereview.appspot.com/307370043/diff/60001/measure.sh
File measure.sh (right):

https://codereview.appspot.com/307370043/diff/60001/measure.sh#newcode65
measure.sh:65: gnuplot throughput_zoomed.gp
GLTM

[yiangmuyu, 2016-10-09 20:11:19]

GLTM!

[pandan0315, 2016-10-09 20:20:06]

https://codereview.appspot.com/307370043/diff/40001/measure.sh
File measure.sh (right):

https://codereview.appspot.com/307370043/diff/40001/measure.sh#newcode1
measure.sh:1: #!/bin/bash
On 2016/10/09 19:03:09, khalidomer88 wrote:
> It is nice that the script has comments, but it would be nicer to have some
top
> description of the script function on top. two lines shows what is does this
> script do and how to use it (by using simple comments on top only).

Done.

https://codereview.appspot.com/307370043/diff/40001/measure.sh#newcode58
measure.sh:58: gnuplot plot.gp
On 2016/10/09 19:03:09, khalidomer88 wrote:
> I think it is better to have different names for the plot files, such as
> rtt_rto_cwnd.gp and bandwidth.gp . we keep the shifted tag for the shifted
> (zoomed) graphs

Done.

[SunnyGu0928, 2016-10-13 20:29:28]

Looks great to me.  Let's talk tomorrow about phase 2 plotting. XD

https://codereview.appspot.com/307370043/diff/80001/rtt_rto_cwnd.gp
File rtt_rto_cwnd.gp (right):

https://codereview.appspot.com/307370043/diff/80001/rtt_rto_cwnd.gp#newcode52
rtt_rto_cwnd.gp:52: plot "rtt.csv" using 1:4  title "RTT"
GLTM

https://codereview.appspot.com/307370043/diff/80001/rtt_rto_cwnd_zoomed.gp
File rtt_rto_cwnd_zoomed.gp (right):

https://codereview.appspot.com/307370043/diff/80001/rtt_rto_cwnd_zoomed.gp#ne...
rtt_rto_cwnd_zoomed.gp:54: #exit multiplot layout
GLTM

https://codereview.appspot.com/307370043/diff/80001/throughput_zoomed.gp
File throughput_zoomed.gp (right):

https://codereview.appspot.com/307370043/diff/80001/throughput_zoomed.gp#newc...
throughput_zoomed.gp:21: plot "iperf.csv"
GLTM

https://codereview.appspot.com/307370043/diff/80001/througput.gp
File througput.gp (right):

https://codereview.appspot.com/307370043/diff/80001/througput.gp#newcode20
througput.gp:20: plot "iperf.csv"
GLTM

[Xiaodong Zhang, 2016-10-14 11:38:51]

https://codereview.appspot.com/307370043/diff/100001/measure_three_flows.sh
File measure_three_flows.sh (right):

https://codereview.appspot.com/307370043/diff/100001/measure_three_flows.sh#n...
measure_three_flows.sh:80: 
LGTM

[khalidomer88, 2016-10-14 19:38:29]

LGTM

[khalidomer88, 2016-10-14 19:40:54]

LGTM
great work Dan

[khalidomer88, 2016-10-14 19:41:08]

LGTM

[khalidomer88, 2016-10-22 14:14:37]

please consider this. This will not affect our previous measurements anyway.

https://codereview.appspot.com/307370043/diff/80001/rtt_rto_cwnd_zoomed.gp
File rtt_rto_cwnd_zoomed.gp (right):

https://codereview.appspot.com/307370043/diff/80001/rtt_rto_cwnd_zoomed.gp#ne...
rtt_rto_cwnd_zoomed.gp:40: "" using 1:9  linetype 5 title "RTO value" axis x1y2
I think it would be good to separate the graphs in different files. instead of
three graphs written in the same file. This could be done by separating the
files into three plot files or reset some parameters after each plot command

[khalidomer88, 2016-10-22 15:50:01]

Hi Dan,

Can you please add the modified tcp_probe to git.

BR,
Khalid

https://codereview.appspot.com/307370043/diff/100001/measure_three_flows.sh
File measure_three_flows.sh (right):

https://codereview.appspot.com/307370043/diff/100001/measure_three_flows.sh#n...
measure_three_flows.sh:30: #load new tcp_probe module(including rto value)
Hi Dan,

Could you please add this modified tcp_probe to the list of the files in git
(also send it for review).

BR,
Khalid

[khalidomer88, 2016-10-23 10:29:33]

Hej, just a question. Have we compared the values of RTO we calculate with the
value shown by the command (ss)?

https://codereview.appspot.com/307370043/diff/120001/tcp_probe_add/tcp_probe_...
File tcp_probe_add/tcp_probe_add.c (right):

https://codereview.appspot.com/307370043/diff/120001/tcp_probe_add/tcp_probe_...
tcp_probe_add/tcp_probe_add.c:164: p->rto = (tp->srtt_us >> 3) + tp->rttvar_us;
How sure are we of this calculations. Have we tried to compare with ss command
output?

[kirill.sc, 2016-10-23 11:44:18]

FYI, you are using this code review tool in a slightly awkward way. 

You suppose to have different issues (e.g., 307370043) for different features of
the project. In order to achieve this, you need to run "git cl..." command when
you located in a specific feature branch in your local git repository. When you
run git-cl from the same branch all the time (e.g., master) all your commits
treated as part of one bug and thus being added here infinitely extending Patch
numbers. 

Finally, in the tcp probe code, it is not clear what is new and what is old, it
appears that the entire file has been rewritten.

[pandan0315, 2016-10-23 12:07:12]

On 2016/10/23 11:44:18, kirill.sc wrote:
> FYI, you are using this code review tool in a slightly awkward way. 
> 
> You suppose to have different issues (e.g., 307370043) for different features
of
> the project. In order to achieve this, you need to run "git cl..." command
when
> you located in a specific feature branch in your local git repository. When
you
> run git-cl from the same branch all the time (e.g., master) all your commits
> treated as part of one bug and thus being added here infinitely extending
Patch
> numbers. 
> 
> Finally, in the tcp probe code, it is not clear what is new and what is old,
it
> appears that the entire file has been rewritten.

sorry, I don't know how to do according to you way suggested, I will look up for
that.
and regarding to the tcp probe code, I said only check the commented lines,
which are added by me

[pandan0315, 2016-10-23 12:07:13]

On 2016/10/23 11:44:18, kirill.sc wrote:
> FYI, you are using this code review tool in a slightly awkward way. 
> 
> You suppose to have different issues (e.g., 307370043) for different features
of
> the project. In order to achieve this, you need to run "git cl..." command
when
> you located in a specific feature branch in your local git repository. When
you
> run git-cl from the same branch all the time (e.g., master) all your commits
> treated as part of one bug and thus being added here infinitely extending
Patch
> numbers. 
> 
> Finally, in the tcp probe code, it is not clear what is new and what is old,
it
> appears that the entire file has been rewritten.

sorry, I don't know how to do according to you way suggested, I will look up for
that.
and regarding to the tcp probe code, I said only check the commented lines,
which are added by me

[khalidomer88, 2016-10-23 14:34:08]

tc.sh LGTM , nice work

[khalidomer88, 2016-10-29 17:37:57]

Hi Everyone,

reviewing the scripts above means that you know what the scripts do and know how
you can use it. make sure to thoroughly understand the scripts before typing  (L
G T M).

Best Regards,
Khalid

[khalidomer88, 2016-10-31 18:30:10]

On 2016/10/30 19:42:05, pandan0315 wrote:

Hi Dan, good work as usual. It would have been nice if you could have created
another thread for this code review. Any way, I have written some comments.
Actually I have addressed those comments and some other issues on updated script
that is based on yours with few modifications. My modified scripts are in
source1 under /home/csd/khalid/random/ik2200/

Changes are:
1. addresssing the comments shown above (typoes and single flow plot files).
2. Changing the directory structure and input for the script. Currently it takes
4 parameters and create four directories in the home directory with one for each
flow and one for the sum, I have instead set the input of the script to only one
file taking the name of an outer directory and the other direcotries take fixed
names of (vm1,vm2,vm3 and total) inside that directory.
3. I have created small script set on sourceVM1,sourceVM2 and sourceVM3 that
accept times in the future and then blocks until this time is reched. Purpose is
to make sure the three flows start at the same time (now flows start time is for
sure very close to each others).

Please check these scripts and merge them as you see appropriate.

Best Regards,
Khalid

[khalidomer88, 2016-10-31 18:31:18]

Hi Dan, good work as usual. It would have been nice if you could have created
another thread for this code review. Any way, I have written some comments.
Actually I have addressed those comments and some other issues on updated script
that is based on yours with few modifications. My modified scripts are in
source1 under /home/csd/khalid/random/ik2200/ Changes are: 1. addresssing the
comments shown above (typoes and single flow plot files). 2. Changing the
directory structure and input for the script. Currently it takes 4 parameters
and create four directories in the home directory with one for each flow and one
for the sum, I have instead set the input of the script to only one file taking
the name of an outer directory and the other direcotries take fixed names of
(vm1,vm2,vm3 and total) inside that directory. 3. I have created small script
set on sourceVM1,sourceVM2 and sourceVM3 that accept times in the future and
then blocks until this time is reched. Purpose is to make sure the three flows
start at the same time (now flows start time is for sure very close to each
others). Please check these scripts and merge them as you see appropriate. 
Best Regards, Khalid

https://codereview.appspot.com/307370043/diff/160001/xen_three_flows/throughp...
File xen_three_flows/throughput_zoomed.gp (right):

https://codereview.appspot.com/307370043/diff/160001/xen_three_flows/throughp...
xen_three_flows/throughput_zoomed.gp:10: set xlabe "time stamp(senconds)"
typo (seconds)

https://codereview.appspot.com/307370043/diff/160001/xen_three_flows/throughp...
xen_three_flows/throughput_zoomed.gp:16: "iperf4.csv"  linetype 8 title "total"
again doesn't work for one flow case

https://codereview.appspot.com/307370043/diff/160001/xen_three_flows/througpu...
File xen_three_flows/througput.gp (left):

https://codereview.appspot.com/307370043/diff/160001/xen_three_flows/througpu...
xen_three_flows/througput.gp:18: set xlabe "time stamp(senconds)"
type senconds (seconds)

https://codereview.appspot.com/307370043/diff/160001/xen_three_flows/througpu...
File xen_three_flows/througput.gp (right):

https://codereview.appspot.com/307370043/diff/160001/xen_three_flows/througpu...
xen_three_flows/througput.gp:18: "iperf4.csv"  linetype 8 title "total"
there should be separate files for 3 flows and one flow. in One flow case the
plot will fail since there are no flow2 and flow3

[pandan0315, 2016-11-01 08:21:16]

On 2016/10/31 18:31:18, khalidomer88 wrote:
> Hi Dan, good work as usual. It would have been nice if you could have created
> another thread for this code review. Any way, I have written some comments.
> Actually I have addressed those comments and some other issues on updated
script
> that is based on yours with few modifications. My modified scripts are in
> source1 under /home/csd/khalid/random/ik2200/ Changes are: 1. addresssing the
> comments shown above (typoes and single flow plot files). 2. Changing the
> directory structure and input for the script. Currently it takes 4 parameters
> and create four directories in the home directory with one for each flow and
one
> for the sum, I have instead set the input of the script to only one file
taking
> the name of an outer directory and the other direcotries take fixed names of
> (vm1,vm2,vm3 and total) inside that directory. 3. I have created small script
> set on sourceVM1,sourceVM2 and sourceVM3 that accept times in the future and
> then blocks until this time is reched. Purpose is to make sure the three flows
> start at the same time (now flows start time is for sure very close to each
> others). Please check these scripts and merge them as you see appropriate. 
> Best Regards, Khalid
> 
>
https://codereview.appspot.com/307370043/diff/160001/xen_three_flows/throughp...
> File xen_three_flows/throughput_zoomed.gp (right):
> 
>
https://codereview.appspot.com/307370043/diff/160001/xen_three_flows/throughp...
> xen_three_flows/throughput_zoomed.gp:10: set xlabe "time stamp(senconds)"
> typo (seconds)
> 
>
https://codereview.appspot.com/307370043/diff/160001/xen_three_flows/throughp...
> xen_three_flows/throughput_zoomed.gp:16: "iperf4.csv"  linetype 8 title
"total"
> again doesn't work for one flow case
> 
>
https://codereview.appspot.com/307370043/diff/160001/xen_three_flows/througpu...
> File xen_three_flows/througput.gp (left):
> 
>
https://codereview.appspot.com/307370043/diff/160001/xen_three_flows/througpu...
> xen_three_flows/througput.gp:18: set xlabe "time stamp(senconds)"
> type senconds (seconds)
> 
>
https://codereview.appspot.com/307370043/diff/160001/xen_three_flows/througpu...
> File xen_three_flows/througput.gp (right):
> 
>
https://codereview.appspot.com/307370043/diff/160001/xen_three_flows/througpu...
> xen_three_flows/througput.gp:18: "iperf4.csv"  linetype 8 title "total"
> there should be separate files for 3 flows and one flow. in One flow case the
> plot will fail since there are no flow2 and flow3

Sorry, it is my fault, I forgot to create another issue.
And I will fix the codes later.