[Twg] Xyratex roadmap items

[Andreas Dilger]

On 2012-01-04, at 1:51 PM, Nathan Rutman wrote:
> On Jan 4, 2012, at 12:40 PM, Andreas Dilger wrote:
>> On 2012-01-04, at 11:12 AM, Nathan Rutman wrote:
>>> at SC11 John had a nice slide of a community roadmap, which was sadly lacking any Xyratex items.  Since we're officially part of OpenSFS now, I'd like to add our current plans, with the usual caveats about how we might change our minds.
>> 
>> Nathan,
>> thanks for submitting these.  Some comments inline.
>> 
>>> 3/12 CRC-32c SW (crypto-api version of CRC-32c so that it is no longer cpu-specific.)
>> 
>> This was implemented inside Lustre because the CRC32c crypto API was not
>> available in all of the kernel versions that were supported at the time.
>> 
>> The other reason it was done this way is that we don't want to emulate
>> CRC32c in software on platforms that don't support it in hardware, since
>> it is slower than the Adler32 software support.  With the current code
>> CRC32c is only used if both the client and server support it in hardware.
> 
> In most cases it's only the server HW support that matters, since client calculations are "free" in large-client systems.  Anyway, using the crypto-api version allows the flexibility.

That is true when there are many clients doing IO, but if there the client
IO is less than the aggregate OST bandwidth the client CPU usage is quite
important for maximizing the IO bandwidth.

>> This might be improved with newer software implementations in crypto API.
>> 
>>> 3/12 CRC-32 HW (add HW acceleration to crypto-api version of CRC-32, and use it.)
>> 
>> Same comments as above, but doubly so because CRC32 is even slower than
>> CRC32c when implemented in software.
> 
> It's all in SW right now, and people are using it; this just provides the HW accel for platforms that support it.
> 
>>> 4/12 4MB IO
>>> 4/12 NRS noop (null policy)
>>> 5/12 NRS obrr (object-based round-robin policy)
>>> 7/12 End-to-End Data Integrity with T10
>> 
>> It would be great to see some presentations about these at LUG'12, in
>> particular the E2E T10 feature since this a new feature that hasn't been
>> in the pipeline previously, and hasn't been widely discussed.
> 
> Yes, I'm planning on talking about it.


Cheers, Andreas
--
Andreas Dilger                       Whamcloud, Inc.
Principal Engineer                   http://www.whamcloud.com/