[cdwg] CDWG Meeting minutes

[Christopher J. Morrone]

On 04/04/2013 10:53 AM, Cory Spitz wrote:
> Hi,
>
> Sorry I couldn't make the CDWG meeting this week.  Thanks for posting the
> minutes.
>
> I noted that you plan to "Quell fears about CLIO performance".  How do we
> plan to do that?  Do you have some performance charts that you plan to
> share?

No, I'll avoid numbers, because it definitely slower in some workloads. 
  Multiple tasks writing to the same file on the same client, for 
instance, is quite bad.  For other workloads, it is perfectly adequate.

Among the development community, we need to press hard to address 
further CLIO issues.  Publicly I think we need to do damage control and 
reduce concerns about CLIO, while being careful not to mislead.

My prediction is that it will take 12-18 months before the majority of 
the CLIO issues are addressed.  But that is _if_ we begin a concerted 
effort to address the problems.

Nevertheless, I think that it will be problematic to still have people 
running 1.8 for very much longer.  At least it will be if those folks 
ever plan to upgrade or inter-operate with anything newer.  For 
instance, there is no planned interoperability with 1.8 in Lustre 2.5. 
LLNL experience is that 1.8 to 2.1 interoperability was shaky, so folks 
should really understand how difficult it is going to be to move forward 
if they don't keep up.

So the message is something along the lines of:

Yes, in some ways the 2.X client code is not as good as 1.8.  But for 
many workloads the performance in the 2.X client is perfectly adequate, 
and there are many other benefits of moving to the 2.X code (MDS 
performance, new features, etc.).  2.X is where the community expending 
the vast majority of its efforts, and the code that will see the most 
improvements and bugs fixes.

> In any case, I think that we should gather some performance numbers as
> part of the release process.   We don't necessarily have to put a stake in
> the ground, but I would like us to publicly track some performance
> metrics.  I know that Intel and WC in the past have monitored this kind of
> information.  So, I think that we can pull this off for the upcoming 2.4.0
> release.  How do we go about this?  What progress has the BWG made?  Maybe
> they have identified a concise set of useful tests.

I would personally think that we're better off deciding on some 
performance gathering requirements early in the 2.5 process.  At least 
that would fit my schedule better. :)

But if other folks have the time to invest in this for 2.4, that would 
be great.

Chris