[Twg] T10 End-to-End Data Integrity HLD

[Alexey Lyashkov]

On Apr 19, 2012, at 01:55, Andreas Dilger wrote:
> 
> 
>>>> From a technical point of view, the tradeoff here is between transmitting a small amount of data (4k per 1MB) and recalculating that data on the servers.  Depending on where/how often we re-verify the checksums, we could remove the recalculation from the server entirely.
>>> 
>>> That is potentially true, but if the checksums are not verified until you get down to the disk, if there ever is a real checksum error you won't have any idea where the error is being introduced.
>> 
>> That's a question about debugging really.  Just as today many sites run without checksums at all due to server CPU loading, it makes sense to me to make intermediate verifications optional.
> 
> Based on the data in LU-1201, I think with Nehalem CPUs and multi-threaded ptlrpcd the checksum overhead will become a non-issue, and people will happily leave it enabled all the time.  The unfortunate thing is that the CRC16 used by T10-DIF has such poor software performance that I agree there is no option but to leave the verification to the hardware.
> 
Based on Xyratex test results - crc32c HW implementation isn't best choose, and multi thread ptlrpcd isn't help to improve a  OST side loading.
yes, main problem is OST side, not a client side checksumming.

Typical speed is less 1Gb/s for crc32c hw, and it's less speed then adler may provide.
Current top speed list - adler, crc32 hw based on (not crc32c), crc32c hw, both crc32c table implementation.




--------------------------------------------
Alexey Lyashkov
alexey_lyashkov at xyratex.com