[cdwg] Monitoring Ticket Progress

Jones, Peter A peter.a.jones at intel.com
Fri Aug 9 05:04:50 PDT 2013 

Kevin

Re-reading this email sent at the end of a long day, I wonder if it does
not come across in the spirit that it was intended. Sorry if that is the
case and I will try and make amends now.

I appreciate that this is not really your area, so you must feel pretty
concerned to step in like this, so I really hope that I can set your mind
at ease here.

I think that you have picked out a very useful part of the thread. I would
absolutely love to have some better tools that allowed me to quickly spot
patterns and anomalies amongst all the hundreds of patches flowing through
the system. The current mechanisms are a little crude and definitely not
intuitive. There is also a lack of metrics for us to track our
effectiveness over time.

Ultimately, though, such a system is simply a tool to help us improve the
patch landing process itself. It will enable us to identify patches where
things are not running smoothly for whatever reason, review them, and take
corrective action. It is by this iterative process that will help us
improve over time.

My concern is that I do not want us to wait until we have an improved
tracking system in place before we start other improvements to our
processes. While not needing to dwell on the specifics of the tickets, my
takeaway was that there was low hanging fruit in this area across the
board, so I think that it illustrates the value of using the options at
our disposal today to improve things in parallel to any longer-term
initiative to have a more automated mechanism in place.

John Forgan of Xyratex sent out some filters which could generate some
possible candidates for discussion and, to my mind, each individual
engineer has a more manageable number of patches that they authored and
know intimately and so should be able to flag situations where closer
attention seems warranted.

Discussing such matters has been a long-standing - but rarely used -
function of the CDWG and I really think that we would all benefit from
taking better advantage of this.

I hope that this clarifies things for you. If not, then please feel free
to call me and let's talk this over or, of course, we can talk about this
on next week's CDWG call.

Regards

Peter

On 8/8/13 10:08 PM, "Jones, Peter A" <peter.a.jones at intel.com> wrote:

>Hi Kevin
>
>I did understand what Denis was proposing and my only objection to it was
>that it would take time to set this up and I personally feel that this
>time could be better spent elsewhere.
>
>I think that it is perfectly healthy to discuss this and I do not think
>that you should feel threatened by such dialog.
>
>In my view, this thread has been quite productive in that Denis has had a
>flawed assumption clarified.
>
>From what I have seen of Denis's work, I would expect him to respond to
>that in a positive manner.
>
>Regards
>
>Peter
>
>On 8/8/13 7:44 PM, "Kevin Canady" <kevin_canady at xyratex.com> wrote:
>
>>Chris,
>>Peter and you continue to miss the main point Denis was making.  There
>>never was any implication of animosity, so no need to be defensive or
>>stir such topic.  There is no conflict.  Somehow, and quite
>>unfortunately, this thread continues to be stuck on the defense of
>>specific patch activity when the main point that was being made was…
>>
>>>> Those tickets are probably all have good explanation why.
>>>> My suggestion was to build process, that will resolve such issues
>>>>automatically and will provide us stat and info about Sustaining review
>>>>activities.
>>>> What I don't want is to do it manually.
>>>> 
>>>> What we propose is to define all stages of Sustaining process to
>>>>gather all needed info to understand where we have most problems:
>>>> * review resources
>>>> * incorrect test failures
>>>> * rebases
>>>> * code freeze
>>>> * some other…
>>>> 
>>>> Process for all, not just Xyratex, but for all reviews.
>>>> 
>>>> That will give us more visibility and possibility to improve.
>>>> We are not saint and we need to improve too. We have issues with
>>>>upstream submission too and there are no fault here in this discussion.
>>>> 
>>>> Right now we know that only based on our experience in our heads,
>>>>intuitively.
>>>> 
>>>> Of course we could use any suggestions. If we all think that just
>>>>e-mail with list will resolve this situation - we will use that.
>>>> 
>>>> I just started this topic to share our process that could be useful,
>>>>and gather other feedbacks - what could be done.
>>
>>It would be extremely helpful if we can remain centered on these
>>discussions to make this a productive dialog.
>>
>>Kevin
>>
>>On Aug 8, 2013, at 6:14 PM, Christopher J. Morrone <morrone2 at llnl.gov>
>>wrote:
>>
>>> On 08/08/2013 09:06 AM, Denis Kondratenko wrote:
>>>> That how this hole thread was started - we do want to do this under
>>>>CDWG and just searching right way to do it.
>>> 
>>> Great!
>>> 
>>> I believe the right way to do it is for the patch submitter to keep
>>>track of his/her submissions.  If a submitted patch needs attention,
>>>they should raise the issue in the associated Jira ticket.  I find that
>>>often just a friendly reminder is all it takes to get attention on a
>>>patch, or an explanation of why the patch needs to wait.
>>> 
>>> If that does not solve the problem, the next step is to raise the issue
>>>on the CDWG mailing list.  List the specific patch(es) that you think
>>>need attention.  We will discuss and come to a decision together.
>>> 
>>> Lets try it out with the list that you supplied:
>>> 
>>> >> http://review.whamcloud.com/#/c/3553/
>>> 
>>> Landed on master, August 6, 2013.
>>> No action needed.
>>> 
>>> >> http://review.whamcloud.com/#/c/3277/
>>> 
>>> Landed on master, June 21, 2013.
>>> No action needed.
>>> 
>>> >> http://review.whamcloud.com/#/c/5213/
>>> 
>>> -1 code review from Andreas, 30 inline comments.
>>> Waiting on Xyratex to fix and push next version for review.
>>> 
>>> >> http://review.whamcloud.com/#/c/4372/
>>> 
>>> Landed on master, June 13, 2013.
>>> No action needed.
>>> 
>>> >> http://review.whamcloud.com/#/c/4641/
>>> 
>>> Landed on b2_1, May 20, 2013.
>>> Out of scope for OpenSFS "tree contract", but nevertheless, no action
>>>needed.
>>> 
>>> >> http://review.whamcloud.com/#/c/4664/
>>> 
>>> This is probably a good example of one that has sat a while.  We could
>>>chat about what happened here.  If you go to the associated jira ticket:
>>> 
>>>  https://jira.hpdd.intel.com/browse/LU-2377
>>> 
>>> it seems clear to me that Intel's opinion was that upstream ext4 was a
>>>more appropriate place for this patch than carrying it in the ldiskfs
>>>patch stack.
>>> 
>>> Conversation between Xyratex and Intel looks like it went nicely.
>>> 
>>> Did Xyratex follow through with sending this patch to the upstream ext4
>>>developers?
>>> 
>>> >> http://review.whamcloud.com/#/c/3725/
>>> 
>>> Patch is for b2_2.  There are no community endorsed b2_2 maintenance
>>>releases.  No action should be expected on this patch.
>>> 
>>> If this is a problem that still exists on master, Xyratex should submit
>>>a patch to master.
>>> 
>>> >> http://review.whamcloud.com/#/c/2406/
>>> 
>>> Patch is for b1_8.  There are no longer any community endorsed b1_8
>>>maintenance releases (some Lustre vendors offer their customers
>>>maintenance releases as they see fit).  No action should be expected on
>>>this patch.
>>> 
>>> -----
>>> 
>>> That is how I would envision the process going in the future.  When
>>>developer feels that a patch isn't making enough progress, and didn't
>>>get an adequate explanation in Jira, they raise here with the CDWG.  I
>>>will look it over as Technical Representative on the tree contract,
>>>along with anyone else in the CDWG that wants to contribute.
>>> 
>>> Together with the interested party and Intel, we will decide on the
>>>needed course of action.
>>> 
>>> > Again, I don't want to start any war or discussion about those
>>>tickets, or about some other.
>>> > I don't want to blame anyone about something. But for me obviously
>>>some problems exit.
>>> 
>>> Ok, maybe those patches are not the right ones to discuss.  But you
>>>really should start discussions about patches that are currently a
>>>problem.  Bringing a problem to our attention should not be the start of
>>>a war.  It should be the start of a discussion, which will result in a
>>>plan of action, which will guide us to a solution.
>>> 
>>> Chris
>>> 
>>> 
>>>>> http://review.whamcloud.com/#/c/3553/
>>>>> http://review.whamcloud.com/#/c/3277/<http://whamcloud.com/#/c/3277>
>>>>> http://review.whamcloud.com/#/c/5213/
>>>>> http://review.whamcloud.com/#/c/4372/
>>>>> http://review.whamcloud.com/#/c/4641/
>>>>> http://review.whamcloud.com/#/c/4664/
>>>>> http://review.whamcloud.com/#/c/3725/
>>>>> http://review.whamcloud.com/#/c/2406/
>>>>> 
>>>>> Again, I don't want to start any war or discussion about those
>>>>>tickets, or about some other.
>>>>> I don't want to blame anyone about something. But for me obviously
>>>>>some problems exit.
>>>>> 
>>>>> Those tickets are probably all have good explanation why.
>>>>> My suggestion was to build process, that will resolve such issues
>>>>>automatically and will provide us stat and info about Sustaining
>>>>>review activities.
>>>>> What I don't want is to do it manually.
>>>>> 
>>>>> What we propose is to define all stages of Sustaining process to
>>>>>gather all needed info to understand where we have most problems:
>>>>> * review resources
>>>>> * incorrect test failures
>>>>> * rebases
>>>>> * code freeze
>>>>> * some other…
>>>>> 
>>>>> Process for all, not just Xyratex, but for all reviews.
>>>>> 
>>>>> That will give us more visibility and possibility to improve.
>>>>> We are not saint and we need to improve too. We have issues with
>>>>>upstream submission too and there are no fault here in this
>>>>>discussion.
>>>>> 
>>>>> Right now we know that only based on our experience in our heads,
>>>>>intuitively.
>>>>> 
>>>>> Of course we could use any suggestions. If we all think that just
>>>>>e-mail with list will resolve this situation - we will use that.
>>>>> 
>>>>> I just started this topic to share our process that could be useful,
>>>>>and gather other feedbacks - what could be done.
>>>>> 
>>>>> Thanks,
>>>>> Denis
>>>>> 
>>>>> On 7 авг. 2013, at 19:07, "Jones, Peter A"
>>>>><peter.a.jones at intel.com<mailto:peter.a.jones at intel.com>> wrote:
>>>>> 
>>>>> Thanks Denis I would appreciate it. I absolutely agree that this
>>>>>needs to
>>>>> be an iterative process and that we need data to analyze and improve.
>>>>> 
>>>>> The last time this concern was raised (just before ISC) I did
>>>>>complete an
>>>>> analysis of every single Xyratex patch submitted against master and
>>>>>(other
>>>>> than a couple of feature patches that had been delayed by the 2.4
>>>>>feature
>>>>> freeze) the longest wait that I saw at that time was only a few
>>>>>weeks.
>>>>> 
>>>>> Xyratex committed at that time to provide a consolidated list of
>>>>>issues
>>>>> that are a concern. I suggest that we focus on that as a first step
>>>>>before
>>>>> getting into possible process changes so that we can be sure that our
>>>>> process reflects the actual situation.
>>>>> 
>>>>> 
>>>>> On 8/7/13 8:11 AM, "Denis Kondratenko"
>>>>><denis_kondratenko at xyratex.com<mailto:denis_kondratenko at xyratex.com>>
>>>>> wrote:
>>>>> 
>>>>> OK, I will go and try to search.
>>>>> 
>>>>> Maybe I have overestimated that, my apologize for that - will provide
>>>>>you
>>>>> numbers and examples.
>>>>> Point wasn't in that.
>>>>> 
>>>>> Point was - we have no numbers - so nothing to analyze and improve.
>>>>> 
>>>>> Thanks,
>>>>> Denis
>>>>> 
>>>>> On 7 авг. 2013, at 18:03, "Jones, Peter A"
>>>>><peter.a.jones at intel.com<mailto:peter.a.jones at intel.com>>
>>>>> wrote:
>>>>> 
>>>>> <snip>
>>>>> 
>>>>> 
>>>>> It is not a rare occurrence - patches that were submitted 6-12 months
>>>>> after it was first pushed for review.
>>>>> 
>>>>> Denis, please give me three examples where patches pushed to master
>>>>>have
>>>>> waited 6-12 months after being pushed for review?
>>>>> 
>>>>> I take this very seriously.
>>>>> 
>>>>> There are certainly busy times when lower priority patches might need
>>>>>to
>>>>> wait but I have not seen anything that long.
>>>>> 
>>>>> Feature patches submitted during the feature freeze, but that is
>>>>>usually
>>>>> more like 3-4 months in the worst case scenario.
>>>>> 
>>>>> Sometimes people push patches against maintenance branches, which can
>>>>>be
>>>>> useful for the community but if these land it will only be at the
>>>>>time
>>>>> we
>>>>> create a maintenance release against that branch.
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> 
>>>>> 
>>>>> ------------------------------
>>>>> For additional information including the registered office and the
>>>>> treatment of Xyratex confidential information please visit
>>>>>www.xyratex.com<http://www.xyratex.com>
>>>>> 
>>>>> ------------------------------
>>>>> _______________________________________________
>>>>> cdwg mailing list
>>>>> cdwg at lists.opensfs.org<mailto:cdwg at lists.opensfs.org>
>>>>> http://lists.opensfs.org/listinfo.cgi/cdwg-opensfs.org
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> ________________________________
>>>>> For additional information including the registered office and the
>>>>>treatment of Xyratex confidential information please visit
>>>>>www.xyratex.com<http://www.xyratex.com/>
>>>>> 
>>>>> ________________________________
>>>> 
>>>> 
>>> 
>>> _______________________________________________
>>> cdwg mailing list
>>> cdwg at lists.opensfs.org
>>> http://lists.opensfs.org/listinfo.cgi/cdwg-opensfs.org
>>
>>_______________________________________________
>>cdwg mailing list
>>cdwg at lists.opensfs.org
>>http://lists.opensfs.org/listinfo.cgi/cdwg-opensfs.org
>

More information about the cdwg mailing list