<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
On 05/14/2012 03:59 PM, Alexander Lezhoev wrote:
<blockquote cite="mid:4FB17224.1050105@xyratex.com" type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
<span
style="font-size:16px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"
id="internal-source-marker_0.5377666371228946">Hi there,</span><br>
<span
style="font-size:16px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"></span><br>
</blockquote>
<br>
..snipped...<br>
<blockquote cite="mid:4FB17224.1050105@xyratex.com" type="cite"> <span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"></span><br>
<span
style="font-size:16px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">They
cannot be run separately, because the next index uses the code
of previous one. This means all tests should be run in groups of
letter indexes, or they should be refactored to run
independently. </span><br>
</blockquote>
<br>
Is there really a hard 'style' that has the letter indices meaning
they depend on previous tests ? It seem to vary by test, including
where a test 6d is just the 'Dth' style of running test6.<br>
<blockquote cite="mid:4FB17224.1050105@xyratex.com" type="cite"> <span
style="font-size:16px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"></span><br>
<span
style="font-size:16px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">Some
of tests have been already refactored to run “letters”
separately, but we have to make a rule which we should follow
and use for further refactoring.</span><br>
<span
style="font-size:16px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">There
are three decisions we can take about this situation</span>
<ul style="margin-top: 0pt; margin-bottom: 0pt;">
<li
style="list-style-type:disc;font-size:16px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"><span
style="font-size:16px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">Join
the code of all test steps into single test with
corresponding number. So we will have one test_1 instead of
test_1a .. test_1d in the described case.</span></li>
</ul>
</blockquote>
<br>
It would certainly be much cleaner to have a single test contain all
of the steps it needs. It is then repeatable and includes all of the
setup and cleanup required to complete the test.<br>
<blockquote cite="mid:4FB17224.1050105@xyratex.com" type="cite">
<ul style="margin-top:0pt;margin-bottom:0pt;">
</ul>
<span
style="font-size:16px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"></span><br>
<span
style="font-size:16px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">The
first variant could be implemented as follows.</span><br>
<span
style="font-size:16px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"></span><br>
<span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">test_1()
{</span><br>
<p dir="ltr" style="text-indent: 36pt; margin-top: 0pt;
margin-bottom: 0pt;"><span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">touch
$DIR1/f1</span></p>
<p dir="ltr" style="text-indent: 36pt; margin-top: 0pt;
margin-bottom: 0pt;"><span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">[
-f $DIR2/f1 ] || error "check create on 2 mtpt's failed"</span></p>
<p dir="ltr" style="text-indent: 36pt; margin-top: 0pt;
margin-bottom: 0pt;"><span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">chmod
777 $DIR2/f1</span></p>
<p dir="ltr" style="text-indent: 36pt; margin-top: 0pt;
margin-bottom: 0pt;"><span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">$CHECKSTAT
-t file -p 0777 $DIR1/f1 || error "check attribute updates on
2 mtpt's failed"</span></p>
<p dir="ltr" style="text-indent: 36pt; margin-top: 0pt;
margin-bottom: 0pt;"><span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">chmod
a-x $DIR2/f1</span></p>
<p dir="ltr" style="text-indent: 36pt; margin-top: 0pt;
margin-bottom: 0pt;"><span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">$CHECKSTAT
-t file -p 0666 $DIR1/f1 || error "check after remount
attribute updates on 2 mtpt's failed"</span></p>
<p dir="ltr" style="text-indent: 36pt; margin-top: 0pt;
margin-bottom: 0pt;"><span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">rm
$DIR2/f1</span></p>
<p dir="ltr" style="text-indent: 36pt; margin-top: 0pt;
margin-bottom: 0pt;"><span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">$CHECKSTAT
-a $DIR1/f1 || error "unlink on one mountpoint removes file on
other failed"</span></p>
<span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">}</span><br>
<span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">run_test
1 "check attributes updates on 2 mtpt's"</span><br>
<span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"></span><br>
<span
style="font-size:16px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">This
approach has disadvantage that such kind of refactoring will
lead to reduction of test numbering and it will hard to work
with regression history of the refactored tests.</span><br>
<span
style="font-size:16px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">The
second case of refactoring can look like this:</span><br>
<span
style="font-size:16px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"></span><br>
</blockquote>
I'm not sure this is a huge negative for me. The test is very clear
and it gives us precise errors on each step -- something we should
make sure is necessary.<br>
<br>
As we rework these tests, we'll be changing them enough that we'll
effectively wipe some of the regression history. Perhaps it is
enough to change the regression history to just look at test
numbering, ignoring letters ?<br>
<br>
<br>
...snipped...<br>
<blockquote cite="mid:4FB17224.1050105@xyratex.com" type="cite"> <span
style="font-size:16px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">We
can separate these tests:</span><br>
<span
style="font-size:16px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"></span><br>
<span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">test_1()
{</span><br>
<span
style="font-size:15px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
</span><span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">test_1_mcreate</span><br>
<span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">}</span><br>
<span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">run_test
1 "mcreate: drop req, drop rep"</span><br>
<span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"></span><br>
<span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">test_2()
{</span><br>
<span
style="font-size:15px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
</span><span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">test_1_mcreate</span><br>
<span
style="font-size:15px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
</span><span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">test_2_chmod</span><br>
<span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">}</span><br>
<span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">run_test
2 "chmod: drop req, drop rep"</span><br>
<span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"></span><br>
<span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">test_3()
{</span><br>
<span
style="font-size:15px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
</span><span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">test_1_mcreate</span><br>
<span
style="font-size:15px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
</span><span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">test_3_stat</span><br>
<span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">}</span><br>
<span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">run_test
3 "stat: drop req, drop rep"</span><br>
</blockquote>
<br>
I think this makes it much more complex and harder to code, not to
mention adds overhead. Each instance of 'run_test' invokes a host of
setup & cleanup code that can add significantly to the test
runtime. <br>
<br>
<blockquote cite="mid:4FB17224.1050105@xyratex.com" type="cite"> <span
style="font-size:15px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"></span><br>
<span
style="font-size:16px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">or
join them into one and remove test_2 and test_3.</span><br>
<span
style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"></span><br>
<span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">test_1()
{</span><br>
<span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
drop_request "mcreate $DIR/f1" || return 1</span><br>
<span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
drop_reint_reply "mcreate $DIR/f2" || return 2</span><br>
<span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
drop_request "tchmod 111 $DIR/f2" || return 3</span><br>
<span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
drop_reint_reply "tchmod 666 $DIR/f2" || return 4</span><br>
<span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
drop_request "statone $DIR/f2" || return 5</span><br>
<span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
drop_reply "statone $DIR/f2" || return 6</span><br>
<span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">}</span><br>
<span
style="font-size:13px;font-family:Arial;color:#000080;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">run_test
1 "mcreate, chmod,stat: drop req, drop,req"</span><br>
<span
style="font-size:16px;font-family:Arial;color:#000000;background-color:#ffffff;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"></span><br>
</blockquote>
<br>
The return codes are not typically enough to distinguish errors from
a test log. The style above with 'error "MSG with values $x $y"'
seems more rich and helpful.<br>
<br>
Cheers,<br>
Nic<br>
<br>
</body>
</html>