[Twg] T10 End-to-End Data Integrity HLD

[Andreas Dilger]

On 2012-04-18, at 3:55 PM, Andreas Dilger wrote:
> On 2012-04-17, at 11:02 AM, Nathan Rutman wrote:
>> That's a question about debugging really.  Just as today many sites run without checksums at all due to server CPU loading, it makes sense to me to make intermediate verifications optional.
> 
> Based on the data in LU-1201, I think with Nehalem CPUs and multi-threaded ptlrpcd the checksum overhead will become a non-issue, and people will happily leave it enabled all the time.  The unfortunate thing is that the CRC16 used by T10-DIF has such poor software performance that I agree there is no option but to leave the verification to the hardware.
> 
> See the table in https://ext4.wiki.kernel.org/articles/e/x/t/Ext4_Metadata_Checksums_4d24.html
> 
> for a useful recent comparison of checksum algorithm performance.  It would be interesting to see if there was some way to improve the kernel crc16-t10 performance through loop unrolling or table lookups, given that significant performance gains were seen for crc32c-kern to the updated crc32c-by8 implementation.

Just reposting as requested by discussion on the call today.  The summary is that crc32c is fast in both hardware (4.4GB/s) and software (1.6GB/s), while crc16-t10dif is very slow in software (300MB/s).

There is a generic CRC instruction in newer Intel CPUs that might be usable to improve the crc16-t10dif performance.  Nathan mentioned that someone at Xyratex can take a look at whether this could be implemented efficiently in hardware (via an asm wrapper in the short term, via cryptoapi in the long term).

Cheers, Andreas
--
Andreas Dilger                       Whamcloud, Inc.
Principal Lustre Engineer            http://www.whamcloud.com/