From Nathan_Rutman at xyratex.com Wed Jan 4 18:12:43 2012 From: Nathan_Rutman at xyratex.com (Nathan Rutman) Date: Wed, 4 Jan 2012 10:12:43 -0800 Subject: [Twg] Xyratex roadmap items Message-ID: <73AED5C780AE05478241DB067651A92102378A72@XYUS-EX22.xyus.xyratex.com> Hi all - at SC11 John had a nice slide of a community roadmap, which was sadly lacking any Xyratex items. Since we're officially part of OpenSFS now, I'd like to add our current plans, with the usual caveats about how we might change our minds. 3/12 CRC-32c SW (crypto-api version of CRC-32c so that it is no longer cpu-specific.) 3/12 CRC-32 HW (add HW acceleration to crypto-api version of CRC-32, and use it.) 4/12 4MB IO 4/12 NRS noop (null policy) 5/12 NRS obrr (object-based round-robin policy) 7/12 End-to-End Data Integrity with T10 = From adilger at whamcloud.com Wed Jan 4 20:40:11 2012 From: adilger at whamcloud.com (Andreas Dilger) Date: Wed, 4 Jan 2012 13:40:11 -0700 Subject: [Twg] Xyratex roadmap items In-Reply-To: <73AED5C780AE05478241DB067651A92102378A72@XYUS-EX22.xyus.xyratex.com> References: <73AED5C780AE05478241DB067651A92102378A72@XYUS-EX22.xyus.xyratex.com> Message-ID: On 2012-01-04, at 11:12 AM, Nathan Rutman wrote: > at SC11 John had a nice slide of a community roadmap, which was sadly lacking any Xyratex items. Since we're officially part of OpenSFS now, I'd like to add our current plans, with the usual caveats about how we might change our minds. Nathan, thanks for submitting these. Some comments inline. > 3/12 CRC-32c SW (crypto-api version of CRC-32c so that it is no longer cpu-specific.) This was implemented inside Lustre because the CRC32c crypto API was not available in all of the kernel versions that were supported at the time. The other reason it was done this way is that we don't want to emulate CRC32c in software on platforms that don't support it in hardware, since it is slower than the Adler32 software support. With the current code CRC32c is only used if both the client and server support it in hardware. This might be improved with newer software implementations in crypto API. > 3/12 CRC-32 HW (add HW acceleration to crypto-api version of CRC-32, and use it.) Same comments as above, but doubly so because CRC32 is even slower than CRC32c when implemented in software. > 4/12 4MB IO > 4/12 NRS noop (null policy) > 5/12 NRS obrr (object-based round-robin policy) > 7/12 End-to-End Data Integrity with T10 It would be great to see some presentations about these at LUG'12, in particular the E2E T10 feature since this a new feature that hasn't been in the pipeline previously, and hasn't been widely discussed. Cheers, Andreas -- Andreas Dilger Whamcloud, Inc. Principal Engineer http://www.whamcloud.com/ From Nathan_Rutman at xyratex.com Wed Jan 4 20:51:07 2012 From: Nathan_Rutman at xyratex.com (Nathan Rutman) Date: Wed, 4 Jan 2012 12:51:07 -0800 Subject: [Twg] Xyratex roadmap items In-Reply-To: References: <73AED5C780AE05478241DB067651A92102378A72@XYUS-EX22.xyus.xyratex.com> Message-ID: <73AED5C780AE05478241DB067651A92102378A76@XYUS-EX22.xyus.xyratex.com> On Jan 4, 2012, at 12:40 PM, Andreas Dilger wrote: > On 2012-01-04, at 11:12 AM, Nathan Rutman wrote: >> at SC11 John had a nice slide of a community roadmap, which was sadly lacking any Xyratex items. Since we're officially part of OpenSFS now, I'd like to add our current plans, with the usual caveats about how we might change our minds. > > Nathan, > thanks for submitting these. Some comments inline. > >> 3/12 CRC-32c SW (crypto-api version of CRC-32c so that it is no longer cpu-specific.) > > This was implemented inside Lustre because the CRC32c crypto API was not > available in all of the kernel versions that were supported at the time. > > The other reason it was done this way is that we don't want to emulate > CRC32c in software on platforms that don't support it in hardware, since > it is slower than the Adler32 software support. With the current code > CRC32c is only used if both the client and server support it in hardware. In most cases it's only the server HW support that matters, since client calculations are "free" in large-client systems. Anyway, using the crypto-api version allows the flexibility. > > This might be improved with newer software implementations in crypto API. > >> 3/12 CRC-32 HW (add HW acceleration to crypto-api version of CRC-32, and use it.) > > Same comments as above, but doubly so because CRC32 is even slower than > CRC32c when implemented in software. It's all in SW right now, and people are using it; this just provides the HW accel for platforms that support it. > >> 4/12 4MB IO >> 4/12 NRS noop (null policy) >> 5/12 NRS obrr (object-based round-robin policy) >> 7/12 End-to-End Data Integrity with T10 > > It would be great to see some presentations about these at LUG'12, in > particular the E2E T10 feature since this a new feature that hasn't been > in the pipeline previously, and hasn't been widely discussed. Yes, I'm planning on talking about it.= From adilger at whamcloud.com Wed Jan 4 21:45:02 2012 From: adilger at whamcloud.com (Andreas Dilger) Date: Wed, 4 Jan 2012 14:45:02 -0700 Subject: [Twg] Xyratex roadmap items In-Reply-To: <73AED5C780AE05478241DB067651A92102378A76@XYUS-EX22.xyus.xyratex.com> References: <73AED5C780AE05478241DB067651A92102378A72@XYUS-EX22.xyus.xyratex.com> <73AED5C780AE05478241DB067651A92102378A76@XYUS-EX22.xyus.xyratex.com> Message-ID: <4395C25D-54E1-440D-A4D6-9068C3002192@whamcloud.com> On 2012-01-04, at 1:51 PM, Nathan Rutman wrote: > On Jan 4, 2012, at 12:40 PM, Andreas Dilger wrote: >> On 2012-01-04, at 11:12 AM, Nathan Rutman wrote: >>> at SC11 John had a nice slide of a community roadmap, which was sadly lacking any Xyratex items. Since we're officially part of OpenSFS now, I'd like to add our current plans, with the usual caveats about how we might change our minds. >> >> Nathan, >> thanks for submitting these. Some comments inline. >> >>> 3/12 CRC-32c SW (crypto-api version of CRC-32c so that it is no longer cpu-specific.) >> >> This was implemented inside Lustre because the CRC32c crypto API was not >> available in all of the kernel versions that were supported at the time. >> >> The other reason it was done this way is that we don't want to emulate >> CRC32c in software on platforms that don't support it in hardware, since >> it is slower than the Adler32 software support. With the current code >> CRC32c is only used if both the client and server support it in hardware. > > In most cases it's only the server HW support that matters, since client calculations are "free" in large-client systems. Anyway, using the crypto-api version allows the flexibility. That is true when there are many clients doing IO, but if there the client IO is less than the aggregate OST bandwidth the client CPU usage is quite important for maximizing the IO bandwidth. >> This might be improved with newer software implementations in crypto API. >> >>> 3/12 CRC-32 HW (add HW acceleration to crypto-api version of CRC-32, and use it.) >> >> Same comments as above, but doubly so because CRC32 is even slower than >> CRC32c when implemented in software. > > It's all in SW right now, and people are using it; this just provides the HW accel for platforms that support it. > >>> 4/12 4MB IO >>> 4/12 NRS noop (null policy) >>> 5/12 NRS obrr (object-based round-robin policy) >>> 7/12 End-to-End Data Integrity with T10 >> >> It would be great to see some presentations about these at LUG'12, in >> particular the E2E T10 feature since this a new feature that hasn't been >> in the pipeline previously, and hasn't been widely discussed. > > Yes, I'm planning on talking about it. Cheers, Andreas -- Andreas Dilger Whamcloud, Inc. Principal Engineer http://www.whamcloud.com/ From shay at opensfs.org Mon Jan 9 23:48:18 2012 From: shay at opensfs.org (Shay Seager) Date: Mon, 9 Jan 2012 15:48:18 -0800 Subject: [Twg] LUG 2012 Registration is open Message-ID: Hello! Happy New Year! We here at OpenSFS hope that you have had a wonderful start to your 2011. OpenSFS would like to invite you to enjoy spring in beautiful Austin Tx - LUG 2012 will be held at the Hyatt Regency Austin, Downtown on the Lake, Monday, April 23rd 2012 through Wednesday, April 25th 2012. This 2 1/2 day event is the primary venue for discussion and seminars on open source parallel file system technologies with a unique focus on the Lustre™ parallel file system. To Register please follow this link http://www.opensfs.org/lug/registration LUG is a user-led event whose organizing committee is made up of representatives from Indiana University (IU), Naval Research Laboratory (NRL), Oak Ridge National Laboratory (ORNL), Sandia National Laboratories (SNL), Lawrence Livermore National Laboratory (LLNL), and Texas Advanced Computing Center (TACC). Feel free to contact me with any questions. Best, Shay - - - Shay Seager Open SFS Secretary www.opensfs.org (925) 290-7641 shay at opensfs.org -------------- next part -------------- An HTML attachment was scrubbed... URL: