[Twg] T10 End-to-End Data Integrity HLD

Sat Apr 21 06:06:28 UTC 2012

No, I'm saying that if the PI generation in SW is too expensive, then we can use a different checksum between the client and server.  I recognize that this is not compatible with Nathan's HLD and that it will mean the server has to validate the checksum before the HCA touches the data.  The point is that there is no requirement for the client to do the PI validation.  I agree that doing the PI validation/generation on the client is a clean end-to-end solution.  But if the PI calculation so significantly reduces throughput, then there is no point using it.  

-----Original Message-----
From: Alexey Lyashkov [mailto:alexey_lyashkov at xyratex.com] 
Sent: Friday, April 20, 2012 10:54 PM
To: John Carrier
Cc: Andreas Dilger; AndrewPerepechko; twg at lists.opensfs.org; discuss at lists.opensfs.org
Subject: Re: [Twg] T10 End-to-End Data Integrity HLD

John,

don't understand question. Per Andrew HLD - client generate a CRC and attach information to the data transfer (bulk transfer) - CRC verified via HCA on client side. 
so i think it's same as you suggest ?

On Apr 21, 2012, at 09:50, John Carrier wrote:

> Alexey, 
> 
> If you are using the HCA to offload PI calculation on the server, why not use a CRC between the client and server that is optimal for software calculations.  The DIX architecture from SNIA does not require the PI calculation to extend to the app layer and describes translation between segments of the path.
> 
> --jc
> 
> -----Original Message-----
> From: twg-bounces at lists.opensfs.org [mailto:twg-bounces at lists.opensfs.org] On Behalf Of Alexey Lyashkov
> Sent: Friday, April 20, 2012 10:39 PM
> To: Andreas Dilger
> Cc: AndrewPerepechko; twg at lists.opensfs.org
> Subject: Re: [Twg] T10 End-to-End Data Integrity HLD
> 
> 
> On Apr 20, 2012, at 08:41, Andreas Dilger wrote:
> 
>> On 2012-04-19, at 11:57, Alexey Lyashkov <alexey_lyashkov at xyratex.com> wrote:
>>> On Apr 19, 2012, at 01:55, Andreas Dilger wrote:
>>>> 
>>>> Based on the data in LU-1201, I think with Nehalem CPUs and multi-threaded ptlrpcd the checksum overhead will become a non-issue, and people will happily leave it enabled all the time.  The unfortunate thing is that the CRC16 used by T10-DIF has such poor software performance that I agree there is no option but to leave the verification to the hardware.
>>>> 
>>> Based on Xyratex test results - crc32c HW implementation isn't best choose, and multi thread ptlrpcd isn't help to improve a  OST side loading.
>>> yes, main problem is OST side, not a client side checksumming.
>>> 
>>> Typical speed is less 1Gb/s for crc32c hw, and it's less speed then adler may provide.
>>> Current top speed list - adler, crc32 hw based on (not crc32c), crc32c hw, both crc32c table implementation.
>> 
>> I think we both agree in any case that the current crc16-t10dif is not nearly as fast as any of these 32-bit checksums. This may prove to be a major discouragement to using T10-PI even if it is available. 
>> 
> i forget one note, in case T10 DIF OSS CPU isn't included into CRC verification - it's done by raid/sas adapter transparency so we will dramatically reduce load on OSS node with that type checksumming enabled.
> from OSC view - not have differences in handling EIO error - if it's real disk IO error or it's error during network transmitting, in both cases we need resend a data to OSS from a client.
> so i think we don't need any separation for both these cases.
> 
> 
> --------------------------------------------
> Alexey Lyashkov
> alexey_lyashkov at xyratex.com
> 
> 
> 
> 
> _______________________________________________
> twg mailing list
> twg at lists.opensfs.org
> http://lists.opensfs.org/listinfo.cgi/twg-opensfs.org

______________________________________________________________________
This email may contain privileged or confidential information, which should only be used for the purpose for which it was sent by Xyratex. No further rights or licenses are granted to use such information. If you are not the intended recipient of this message, please notify the sender by return and delete it. You may not use, copy, disclose or rely on the information contained in it.

Internet email is susceptible to data corruption, interception and unauthorised amendment for which Xyratex does not accept liability. While we have taken reasonable precautions to ensure that this email is free of viruses, Xyratex does not accept liability for the presence of any computer viruses in this email, nor for any losses caused as a result of viruses.

Xyratex Technology Limited (03134912), Registered in England & Wales, Registered Office, Langstone Road, Havant, Hampshire, PO9 1SA.

The Xyratex group of companies also includes, Xyratex Ltd, registered in Bermuda, Xyratex International Inc, registered in California, Xyratex (Malaysia) Sdn Bhd registered in Malaysia, Xyratex Technology (Wuxi) Co Ltd registered in The People's Republic of China and Xyratex Japan Limited registered in Japan.
______________________________________________________________________