[Twg] T10 End-to-End Data Integrity HLD

[Nathan Rutman]

On Apr 11, 2012, at 1:57 PM, Christopher J. Morrone wrote:

> On 04/11/2012 08:09 AM, Nathan Rutman wrote:
>> 
>> On Apr 11, 2012, at 7:03 AM, John Carrier wrote:
>> 
>>> I thought Sun's Merkle tree solution was tied to ZFS.
>> That's right.  The ptlrpc change will be flexible enough to describe multiple schemes.
> 
> The merkle tree solution was certainly motivated by ZFS, but that doesn't mean that it can't be a generic solution, does it?
The major motivation was that ZFS already does this, so there's no cost to the server to do it at the Lustre level instead.  Without ZFS, it's certainly still usable, but not a clear win like it is for ZFS.
Incidentally, that same motivation (no cost to the server) is the one of the same reasons we're following the T10 path.
There should be room for multiple options here.

>  Perhaps we we make the checksum algorithm negotiable, and the base leaf size as well.
Yes, our intent was to allow a variety of data integrity possibilities in the ptlrpc layer. 
> 
> But the T10 solution did not sound very general on first read.  It also seems to not mention dealing with things like synchronous partial block writes, which the merkle tree design seems to address.
Version mirroring component addresses current RAID limitations -- I'll speak to this at the LUG.

> 
> Do we really want to expose the lustre clients to the details of on-disk storage?
Not just the lustre clients, but potentially the application level as well.  This is the philosophy behind T10-DIX, anyhow: "the data I wrote and checksummed myself is the exact same data that got written to the disk sector".  That said, I look at our project more as facilitating T10-DIX use, rather than tightly integrating it in the Lustre levels.  Lustre more or less is just passing the info along.

> 
> I'm not fully up to date on T10, and quick web search didn't answer it for me.  Does this HLD allow 4K sectors?  The HLD only seems to talk about 512-byte sectors.  It sounds like the padding for 4K sectors is enough that maybe there are just 8 512-byte checksums concatenated, but I don't know if that is the way it is done, or whether the whole 4K sector is checksummed as a single block.
The latter, and it results in a lower coverage of the checksum for 4k sectors, but it doesn't change the design.

> 
> Chris