[Twg] [Discuss] T10 End-to-End Data Integrity HLD

[Andrew Perepechko]

Hello Alex!

This is my understanding of the end-to-end integrity framework:

When a device is configured to work in DIX mode, Linux calculates
tags when processing an I/O request (see bio_integrity_prep())
unless integrity data is already attached to the bio.

We seem to have the following options about metadata vs DIX here:

1) do not change anything, let Linux do all of DIX work, this would
detect errors, but Lustre is not prepared to handle them; this
would also take additional CPU time to generate tags at Linux
block layer; we need to check if there are any races similar to
mmap-write races (buffer is changed after guard tags calculated).
2) use special tags 0xffff for metadata, errors will go undetected,
little additional CPU time, mostly like without DIX support;
3) process DIX errors everywhere, restart metadata operations,
refilling buffers from potentially safe sources. This would require
some ost code rewrite. Even if we can simply restart most metadata
operations, journal code needs to be changed to handle this; also,
complex ldiskfs operations may fail after having updated some blocks
on disk - there does not seem to be any code that attempts to handle
that...

Thank you,
Andrew


On 04/25/2012 01:19 AM, Alexey Lyashkov wrote:
> some results of online discussion.
>
> Johann worry about journal checksumming (to avoid overhead in that area)
> or other metadata checks on
> client store uncommitted data in own queue for a replay (if need), but
> in case journal checksum errors on write client will able to resend
> failed request via special protocol extension (don't exist now).
> same situation if we need a T10 support for these data we will be need
> trigger a failover or change a wire protocol to resend a failed request.
> failover is easily.
>
>
> On Apr 24, 2012, at 14:53, Johann Lombardi wrote:
>
>> On Tue, Apr 24, 2012 at 9:44 PM, Alexey Lyashkov
>> <alexey_lyashkov at xyratex.com <mailto:alexey_lyashkov at xyratex.com>> wrote:
>>
>>     what you mean as metadata blocks? extent's on OST or MDT disk io
>>     traffic ?
>>
>>
>> Yes, anything that goes to the journal (inode, block bitmap, ...)
>> which isn't user data.
>>
>> Johann
>