<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi Terri<br>
<br>
The chart you are referring to is an aggregated summary of all the
testing results for master in our test results database (maloo)
broken down by test suite and git tag (which we create ~ every two
weeks).<br>
<br>
The numbers (x/y) show the number of test runs that passed with no
failures (x) and the total number of test runs. Each test suite
executes many tests. The number of tests run should increase over
time as new tests are added in conjunction with bugfixes.<br>
<br>
Some test suites have been removed because they were deemed to be
obsolete (e.g. liblustre) and some have been added (e.g. mds-survey)<br>
<br>
The best way to analyze this report is dynamically online at
<a class="moz-txt-link-freetext" href="https://maloo.whamcloud.com/reports">https://maloo.whamcloud.com/reports</a>. There you are able to drill
into the individual test runs and see what configuration was run and
the details of any failures (with links to JIRA)<br>
<br>
Our goal is work through all the test failures (which are mostly due
to either tests that were written with certain assumptions that are
not reliably valid in our present-day testing environment or to
certain configurations which the test environment does not fully
support yet) so that all tests will show as green unless there is an
actual bug introduced.<br>
<br>
As to how well this correlates to the quality of the work, that is
open to debate. Personally, I think that, while I would not look at
this data in isolation, I think that it has the advantage of being
largely comparable to earlier Lustre releases and less affected by
unrelated factors. For example, if we looked at the number of bugs
reported against a given release, say, this could be affected by the
the total number of sites running Lustre, or the introduction of new
hardware that breaks former coding assumptions and so overall
quality improvements could be offset by "noise" (or vice versa if
people are slow to adopt newer releases and so do not discover and
report bugs, say)<br>
<br>
I'll be happy to answer any further questions that you have either
via email or on the upcoming CDWG call on Thursday.<br>
<br>
Regards<br>
<br>
Peter<br>
<br>
<div class="moz-cite-prefix">On 12-07-24 9:55 AM, Quinn, Terri
wrote:<br>
</div>
<blockquote
cite="mid:7F337A3E11411E4BB74DE2B8942F2913019DBC188F44@NSPEXMBX-D.the-lab.llnl.gov"
type="cite">
<pre wrap="">CDWG,
Perhaps someone could help me better understand this report. I have a few questions but I'll start with this one. What should I glean from the Quality Metrics included in the report on pages 4 and 5? What I see are the results of a number of tests. Some passed some did not. This is all understandable, but what does this say about the quality of the work? I might conclude that since I see less red as time goes on that things are getting better. However I also note that less tests are being performed.
Regards, Terri
Terri Quinn
Principal Deputy Department Head
Integrated Computing and Communications
Lawrence Livermore National Laboratory
<a class="moz-txt-link-abbreviated" href="mailto:quinn1@llnl.gov">quinn1@llnl.gov</a>
cell 925 321-2879
office 925 423-2385
-----Original Message-----
From: <a class="moz-txt-link-abbreviated" href="mailto:cdwg-bounces@lists.opensfs.org">cdwg-bounces@lists.opensfs.org</a> [<a class="moz-txt-link-freetext" href="mailto:cdwg-bounces@lists.opensfs.org">mailto:cdwg-bounces@lists.opensfs.org</a>] On Behalf Of Hamilton, Pam
Sent: Thursday, July 12, 2012 10:11 PM
To: '<a class="moz-txt-link-abbreviated" href="mailto:cdwg@lists.opensfs.org">cdwg@lists.opensfs.org</a>'; OpenSFS Execs (<a class="moz-txt-link-abbreviated" href="mailto:Execs@lists.opensfs.org">Execs@lists.opensfs.org</a>)
Subject: [cdwg] Lustre Development Community Tree Maintenance Q2 2012 report
Hi all,
Attached is the Q2 2012 quarterly report from Whamcloud for the Lustre Development Community Tree Maintenance contract. Please send any questions/comments to <a class="moz-txt-link-abbreviated" href="mailto:cdwg@lists.opensfs.org">cdwg@lists.opensfs.org</a>.
Regards,
Pam
___________________________________
Pam Hamilton
Lawrence Livermore National Lab
P.O. Box 808, L-556
Livermore, CA 94551-9900
E-Mail: <a class="moz-txt-link-abbreviated" href="mailto:pgh@llnl.gov">pgh@llnl.gov</a>
Phone: 925-423-1332 Fax: 925-423-8719
</pre>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
cdwg mailing list
<a class="moz-txt-link-abbreviated" href="mailto:cdwg@lists.opensfs.org">cdwg@lists.opensfs.org</a>
<a class="moz-txt-link-freetext" href="http://lists.opensfs.org/listinfo.cgi/cdwg-opensfs.org">http://lists.opensfs.org/listinfo.cgi/cdwg-opensfs.org</a>
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Peter Jones
Whamcloud, Inc.
<a class="moz-txt-link-abbreviated" href="http://www.whamcloud.com">www.whamcloud.com</a>
</pre>
</body>
</html>