From pjones at whamcloud.com Fri Feb 3 21:17:53 2012 From: pjones at whamcloud.com (Peter Jones) Date: Fri, 03 Feb 2012 13:17:53 -0800 Subject: [cdwg] Lustre 2.2 update - February 3rd 2012 Message-ID: <4F2C4F01.7020508@whamcloud.com> Hi there Here is an update on the Lustre 2.2 release. Minor Enhancements ================== -Work delayed on remaining patch to add FC15 client support but now work is underway again (LU506) -RHEL 6.2 support landed (LU781) Testing ======= Testing has switched to the 2.1.55 tag, focusing on interop and upgrading from this tag with the 2.1 and 1.8.7-wc1 releases Significant Bugfixes ==================== -LU797 and LU1048 have been very disruptive to testing resulting in many autotest failures to ost-pools and conf-sanity test, respectively; fixes for both have now landed Tests for Features Landed ========================= In response to interest from the CDWG list, here are the details of the tests landed for the 2.2 features - Asynchronous glimpse (LU925) Two new tests created to test this feature http://git.whamcloud.com/?p=fs/lustre-release.git;a=commit;h=6f5813d36102a19f314c9aab409972e8a9f1112b - Client parallel checksums (LU884)Performance re-engineering so covered by performance testing and existing tests - Imperative Recovery (LU580) 6 new tests added http://git.whamcloud.com/?p=fs/lustre-release.git;a=commit;h=a37738fcbd79e283c687a630d9ccb7d7a9fedb3e - Large xattrs (LU80) 4 tests were adapted to test this functionality http://git.whamcloud.com/?p=fs/lustre-release.git;a=commit;h=8a1606a7cbbcaf418fefa5d97a2335df0c0edb09 - Mds-survey (LU593/LU633) Test will land once script has finalised - Multi threaded ptlrpcd Performance re-engineering so covered by performance testing and existing tests - OSD API (LU909) No additional tests required; this code path is exercised extensively by existing tests - Parallel directory operations (pDirOps) (LU50) No additional tests required; this code path is exercised extensively by existing tests - Quota protocol/code improvements (LU847) three new tests in sanity-quota http://git.whamcloud.com/?p=fs/lustre-release.git;a=commit;h=bd51ebb7a130088b85708875f3ef473620d71c2a - Statahead (LU389) These are covered by existing sanity tests (test_123a is for performance, test_123b is for correctness) We will continue to provide periodic updates on our progress on this release. In the meantime, you can always see the landings as they happen at http://git.whamcloud.com/?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master and follow the patch reviews and testing at http://review.whamcloud.com/#q,status:open+project:fs/lustre-release+branch:master,n,z Regards Peter -- Peter Jones Whamcloud, Inc. www.whamcloud.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From pjones at whamcloud.com Mon Feb 6 22:03:26 2012 From: pjones at whamcloud.com (Peter Jones) Date: Mon, 06 Feb 2012 14:03:26 -0800 Subject: [cdwg] Lustre 2.3 release Message-ID: <4F304E2E.9080100@whamcloud.com> Hi there Although we are still very busy with Lustre 2.2, we have started planning for the content of Lustre 2.3 release. Landings for 2.3 will start as soon as Lustre 2.2 has been branched from master. Lustre 2.3 is targeted for release in September 2012 and, as such, any features to be included will need to be landed to master by the end of June at the latest in order to allow adequate time for stabilization ahead of the release. If features only become ready for landing at the end of June there may not be sufficient time to land them all, so features that are available sooner are more likely to be tested, inspected, and landed before the feature freeze. If anyone is working on features that could be candidates to include in this release then please let me know so that we can consider them in our planning. Features that were initially targeted for 2.2 but were not ready in time are already on our list - http://wiki.whamcloud.com/display/PUB/Lustre+Community+Development+in+Progress Thanks Peter -- Peter Jones Whamcloud, Inc. www.whamcloud.com From hamilton5 at llnl.gov Wed Feb 8 19:01:02 2012 From: hamilton5 at llnl.gov (Hamilton, Pam) Date: Wed, 8 Feb 2012 11:01:02 -0800 Subject: [cdwg] Agenda items for next OpenSFS CDWG call Message-ID: Hi all, I am planning to hold an OpenSFS Community Development Working Group call next Wednesday, 2/15, at 9am Pacific. Please send me any agenda items you have. Here is what I'm thinking about so far: * Lustre 1.8 to 2.1 migration experience (LLNL, others?) * Lustre 2.2 release status (Whamcloud) * Lustre 2.3 release planning (Community) * Open discussion Regards, Pam ___________________________________ Pam Hamilton Lawrence Livermore National Lab P.O. Box 808, L-556 Livermore, CA  94551-9900 E-Mail:  pgh at llnl.gov Phone:  925-423-1332          Fax:  925-423-8719 From shay at opensfs.org Wed Feb 8 22:39:25 2012 From: shay at opensfs.org (Shay Seager) Date: Wed, 8 Feb 2012 14:39:25 -0800 Subject: [cdwg] LUG 2012 Message-ID: It’s that time of year again. That’s right, LUG 2012, the Lustre parallel file system’s premier conference that joins individuals both internationally and across a wide variety of disciplines, will take place in Austin, Texas from April 23-25. "LUG is the preeminent venue for users of the Lustre parallel file system, drawing individuals both internationally and across a variety of disciplines,” said Oak Ridge National Laboratory’s (ORNL’s) Technology Integration Group Leader Galen Shipman. And because 2012 is a leap year, participants get an extra day to receive the $140 early registration discount. "The LUG offers participants opportunities to share knowledge, ideas, and achievements with a diverse audience. We're anticipating an interesting program and would encourage you to be a part of it,” said Stephen Simms, Manager of High Performance File Systems at Indiana University. To register for the event http://www.opensfs.org/lug/registration. The LUG program committee would like to invite members of the Lustre community to submit presentation abstracts for inclusion in this year's meeting. It is not necessary to submit a technical paper, just an abstract of your proposed talk no more than a page in length. Talks should target a half an hour in length and reflect the work that you or your site are doing with Lustre. The deadline to submit presentation abstracts is February 14, 2012. For questions or to submit a presentation abstract, send mail to submissions at lists.opensfs.org or contact the program committee chair, Stephen Simms (ssimms at indiana.edu). Anyone who has ever heard the word Lustre is encouraged to attend. Your participation is key to ensuring a stable, viable, and ever-improving Lustre file system. Simply put, all users represent the future of Lustre. By attending LUG, you and your employees will gain depth, perspective, and valuable knowledge to give you a step up in the competitive HPC world. Besides two-and-a-half days of amazing speakers, technical sessions, and keystone networking experience, attendees will enjoy spring in Austin, Texas and will receive attendee gifts. Attendees are recommended to fly in on the morning of Sunday, April 22nd, due to two great events planned for that evening. From 2-5 p.m. the Open Benchmark Working Group meeting will be open to the entire HPC community, while the Whamcloud and DDN event will take place from 6-10 p.m. (details will be posted soon). Attendees are encouraged to fly out the evening of Wednesday, April 25th or the morning of April 26th, so that they can attend the OpenSFS general meeting, OpenSFS Technical Working Group Meeting, and the Community Development Working Group Meeting on Wednesday, April 25th from 1pm-5pm. All events are open to all LUG attendees. Join us in Austin, TX this April 23-25th 2012! Hurry and register today at http://www.opensfs.org/lug/registration. And don't forget to book your hotel room with the Austin Hyatt on the Lake. Rooms are going fast and we want all attendees to get the government and room block discounted rates while they last. To reserve a room at the Hyatt Regency Austin visit http://www.opensfs.org/lug/registration or https://resweb.passkey.com/Resweb.do?mode=welcome_gi_new&groupID=6750202. LUG 2012 would like to thank the following sponsors for making this event possible: Bull, Cray, DDN, Dell, EMC, Intel, NetAPP, Raid, Inc., SGI, Terascala, Whamcloud, and Xyratex. We look forward to see you in Austin! - - - Shay Seager Open SFS Secretary www.opensfs.org (925) 290-7641 shay at opensfs.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From hamilton5 at llnl.gov Mon Feb 13 16:42:08 2012 From: hamilton5 at llnl.gov (Hamilton, Pam) Date: Mon, 13 Feb 2012 08:42:08 -0800 Subject: [cdwg] OpenSFS Community Development WG call Wednesday, 2/15, at 9am Pacific Message-ID: When: Wednesday, February 15, 2012 9:00 AM-10:00 AM (UTC-08:00) Pacific Time (US & Canada). Where: Call-in: 866-914-3976 (925-424-8105) Passcode: 534986# Note: The GMT offset above does not reflect daylight saving time adjustments. *~*~*~*~*~*~*~*~*~* Hi all, We will have an OpenSFS Community Development Working Group conference call on 2/15 at 9am Pacific. Call-in: 866-914-3976 (925-424-8105) Passcode: 534986# Agenda: * Lustre 1.8 to 2.1 migration experience (LLNL, others?) * Lustre 2.2 release status (Whamcloud) * Lustre 2.3 release planning (Community) * Open discussion Regards, Pam -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 2176 bytes Desc: not available URL: From spitzcor at cray.com Wed Feb 15 17:54:52 2012 From: spitzcor at cray.com (Cory Spitz) Date: Wed, 15 Feb 2012 11:54:52 -0600 Subject: [cdwg] Lustre 2.3 release In-Reply-To: <4F304E2E.9080100@whamcloud.com> References: <4F304E2E.9080100@whamcloud.com> Message-ID: <4F3BF16C.1080904@cray.com> Hi, Peter. Cray is working on Linux 3.0 (and SLES 11 SP2) support in LU-812. We'd like to add this to the 2.3 feature set. [I'd also like to see FC15 (LU-506) added to the list too] Thanks, -Cory On 02/06/2012 04:03 PM, Peter Jones wrote: > Hi there > > Although we are still very busy with Lustre 2.2, we have started > planning for the content of Lustre 2.3 release. Landings for 2.3 will > start as soon as Lustre 2.2 has been branched from master. > > Lustre 2.3 is targeted for release in September 2012 and, as such, any > features to be included will need to be landed to master by the end of > June at the latest in order to allow adequate time for stabilization > ahead of the release. If features only become ready for landing at the > end of June there may not be sufficient time to land them all, so > features that are available sooner are more likely to be tested, > inspected, and landed before the feature freeze. > > If anyone is working on features that could be candidates to include in > this release then please let me know so that we can consider them in our > planning. > > Features that were initially targeted for 2.2 but were not ready in time > are already on our list - > http://wiki.whamcloud.com/display/PUB/Lustre+Community+Development+in+Progress > > Thanks > > Peter > From pjones at whamcloud.com Wed Feb 15 18:05:13 2012 From: pjones at whamcloud.com (Peter Jones) Date: Wed, 15 Feb 2012 10:05:13 -0800 Subject: [cdwg] OpenSFS Community Development WG call Wednesday, 2/15, at 9am Pacific In-Reply-To: References: Message-ID: <4F3BF3D9.1090608@whamcloud.com> Here's the wiki page we discussed in the call http://wiki.whamcloud.com/display/PUB/Lustre+Community+Development+in+Progress On 12-02-13 8:42 AM, Hamilton, Pam wrote: > When: Wednesday, February 15, 2012 9:00 AM-10:00 AM (UTC-08:00) > Pacific Time (US & Canada). > Where: Call-in: 866-914-3976 (925-424-8105) Passcode: 534986# > Note: The GMT offset above does not reflect daylight saving time > adjustments. > *~*~*~*~*~*~*~*~*~* > Hi all, > We will have an OpenSFS Community Development Working Group conference > call on 2/15 at 9am Pacific. Call-in: 866-914-3976 (925-424-8105) > Passcode: 534986# > Agenda: > * Lustre 1.8 to 2.1 migration experience (LLNL, others?) > * Lustre 2.2 release status (Whamcloud) > * Lustre 2.3 release planning (Community) > * Open discussion > Regards, > Pam > > > _______________________________________________ > cdwg mailing list > cdwg at lists.opensfs.org > http://lists.opensfs.org/listinfo.cgi/cdwg-opensfs.org -- Peter Jones Whamcloud, Inc. www.whamcloud.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From pjones at whamcloud.com Wed Feb 15 18:15:30 2012 From: pjones at whamcloud.com (Peter Jones) Date: Wed, 15 Feb 2012 10:15:30 -0800 Subject: [cdwg] Lustre 2.3 release In-Reply-To: <4F3BF16C.1080904@cray.com> References: <4F304E2E.9080100@whamcloud.com> <4F3BF16C.1080904@cray.com> Message-ID: <4F3BF642.2090205@whamcloud.com> Thanks Cory. I have added this to the list On 12-02-15 9:54 AM, Cory Spitz wrote: > Hi, Peter. > > Cray is working on Linux 3.0 (and SLES 11 SP2) support in LU-812. We'd > like to add this to the 2.3 feature set. > > [I'd also like to see FC15 (LU-506) added to the list too] > > Thanks, > -Cory > > On 02/06/2012 04:03 PM, Peter Jones wrote: >> Hi there >> >> Although we are still very busy with Lustre 2.2, we have started >> planning for the content of Lustre 2.3 release. Landings for 2.3 will >> start as soon as Lustre 2.2 has been branched from master. >> >> Lustre 2.3 is targeted for release in September 2012 and, as such, any >> features to be included will need to be landed to master by the end of >> June at the latest in order to allow adequate time for stabilization >> ahead of the release. If features only become ready for landing at the >> end of June there may not be sufficient time to land them all, so >> features that are available sooner are more likely to be tested, >> inspected, and landed before the feature freeze. >> >> If anyone is working on features that could be candidates to include in >> this release then please let me know so that we can consider them in our >> planning. >> >> Features that were initially targeted for 2.2 but were not ready in time >> are already on our list - >> http://wiki.whamcloud.com/display/PUB/Lustre+Community+Development+in+Progress >> >> Thanks >> >> Peter >> -- Peter Jones Whamcloud, Inc. www.whamcloud.com From pjones at whamcloud.com Wed Feb 15 18:55:45 2012 From: pjones at whamcloud.com (Peter Jones) Date: Wed, 15 Feb 2012 10:55:45 -0800 Subject: [cdwg] [wc-discuss] Lustre 2.3 release In-Reply-To: <73AED5C780AE05478241DB067651A92102F107AC@XYUS-EX22.xyus.xyratex.com> References: <4F304E2E.9080100@whamcloud.com> <73AED5C780AE05478241DB067651A92102F107AC@XYUS-EX22.xyus.xyratex.com> Message-ID: <4F3BFFB1.8080600@whamcloud.com> Nathan I have created placeholders for these items on the wiki. It would be great if you could elaborate as to the nature of these features - preferably creating a JIRA ticket with design docs etc attached. I am assuming for now that the NRS items will be covered by the existing NRS ticket (LU-398) but please let me know if that is not the case. Thanks Peter On 12-02-15 10:10 AM, Nathan Rutman wrote: > Re: cdwg concall this morning, > Peter sent out a link to the community's ongoing development wiki page last week. > > I'd like to add some Xyratex plans that I posted to TWG in early Jan: > > 3/12 CRC-32c SW (crypto-api version of CRC-32c so that it is no longer cpu-specific.) > 3/12 CRC-32 HW (add HW acceleration to crypto-api version of CRC-32, and use it.) > 4/12 4MB IO > 4/12 NRS noop (null policy) > 5/12 NRS obrr (object-based round-robin policy) > 7/12 End-to-End Data Integrity with T10 > > > On Feb 6, 2012, at 2:03 PM, Peter Jones wrote: > >> Hi there >> >> Although we are still very busy with Lustre 2.2, we have started planning for the content of Lustre 2.3 release. Landings for 2.3 will start as soon as Lustre 2.2 has been branched from master. >> >> Lustre 2.3 is targeted for release in September 2012 and, as such, any features to be included will need to be landed to master by the end of June at the latest in order to allow adequate time for stabilization ahead of the release. If features only become ready for landing at the end of June there may not be sufficient time to land them all, so features that are available sooner are more likely to be tested, inspected, and landed before the feature freeze. >> >> If anyone is working on features that could be candidates to include in this release then please let me know so that we can consider them in our planning. >> >> Features that were initially targeted for 2.2 but were not ready in time are already on our list - http://wiki.whamcloud.com/display/PUB/Lustre+Community+Development+in+Progress >> >> Thanks >> >> Peter >> >> -- >> Peter Jones >> Whamcloud, Inc. >> www.whamcloud.com >> -- Peter Jones Whamcloud, Inc. www.whamcloud.com From kevin_canady at xyratex.com Wed Feb 15 20:53:46 2012 From: kevin_canady at xyratex.com (Kevin Canady) Date: Wed, 15 Feb 2012 12:53:46 -0800 Subject: [cdwg] [wc-discuss] Lustre 2.3 release In-Reply-To: <4F3BFFB1.8080600@whamcloud.com> Message-ID: Peter This is great, many thanks, it's a good demonstration how much is being done to move Lustre forward. If it would not further complicate, it would be helpful to have some reference to the targeted release of the features.Can we add a column to the list? The obvious "right to change" would be in place. The reference is only to help set expectations within the community. Sound reasonable? Thanks again, Kevin On 2/15/12 10:55 AM, "Peter Jones" wrote: >Nathan > >I have created placeholders for these items on the wiki. It would be >great if you could elaborate as to the nature of these features - >preferably creating a JIRA ticket with design docs etc attached. I am >assuming for now that the NRS items will be covered by the existing NRS >ticket (LU-398) but please let me know if that is not the case. > >Thanks > >Peter > >On 12-02-15 10:10 AM, Nathan Rutman wrote: >> Re: cdwg concall this morning, >> Peter sent out a link to the community's ongoing development wiki page >>last week. >> >> I'd like to add some Xyratex plans that I posted to TWG in early Jan: >> >> 3/12 CRC-32c SW (crypto-api version of CRC-32c so that it is no longer >>cpu-specific.) >> 3/12 CRC-32 HW (add HW acceleration to crypto-api version of CRC-32, >>and use it.) >> 4/12 4MB IO >> 4/12 NRS noop (null policy) >> 5/12 NRS obrr (object-based round-robin policy) >> 7/12 End-to-End Data Integrity with T10 >> >> >> On Feb 6, 2012, at 2:03 PM, Peter Jones wrote: >> >>> Hi there >>> >>> Although we are still very busy with Lustre 2.2, we have started >>>planning for the content of Lustre 2.3 release. Landings for 2.3 will >>>start as soon as Lustre 2.2 has been branched from master. >>> >>> Lustre 2.3 is targeted for release in September 2012 and, as such, any >>>features to be included will need to be landed to master by the end of >>>June at the latest in order to allow adequate time for stabilization >>>ahead of the release. If features only become ready for landing at the >>>end of June there may not be sufficient time to land them all, so >>>features that are available sooner are more likely to be tested, >>>inspected, and landed before the feature freeze. >>> >>> If anyone is working on features that could be candidates to include >>>in this release then please let me know so that we can consider them in >>>our planning. >>> >>> Features that were initially targeted for 2.2 but were not ready in >>>time are already on our list - >>>http://wiki.whamcloud.com/display/PUB/Lustre+Community+Development+in+Pr >>>ogress >>> >>> Thanks >>> >>> Peter >>> >>> -- >>> Peter Jones >>> Whamcloud, Inc. >>> www.whamcloud.com >>> > >-- >Peter Jones >Whamcloud, Inc. >www.whamcloud.com > >_______________________________________________ >cdwg mailing list >cdwg at lists.opensfs.org >http://lists.opensfs.org/listinfo.cgi/cdwg-opensfs.org From pjones at whamcloud.com Wed Feb 15 21:27:08 2012 From: pjones at whamcloud.com (Peter Jones) Date: Wed, 15 Feb 2012 13:27:08 -0800 Subject: [cdwg] [wc-discuss] Lustre 2.3 release In-Reply-To: References: Message-ID: <4F3C232C.4010208@whamcloud.com> Kevin I have added this column to the table. As I explained on the call, my concern with having such a column was that it might cause confusion in the community if the targeted release supplied does not prove to be realistic. In the past we ran into issues where people had relied on timelines in aggressive roadmaps and planned around them, and then were left in trouble when the features were not delivered according to the schedule. That is why we have consciously published conservative roadmaps with the intention of over-delivering. However, I can see this option being of interest to the community so I am ok with giving this a try. Hopefully having all the design documents, test plan being in the open in JIRA and the code uploaded into gerrit will enable interested parties to see for themselves how things are progressing and perhaps it will prove useful to spark a dialogue if there appears to be a divergence in expectations. Regards Peter On 12-02-15 12:53 PM, Kevin Canady wrote: > Peter > This is great, many thanks, it's a good demonstration how much is being > done to move Lustre forward. If it would not further complicate, it would > be helpful to have some reference to the targeted release of the > features.Can we add a column to the list? The obvious "right to change" > would be in place. The reference is only to help set expectations within > the community. > > Sound reasonable? > > Thanks again, > Kevin > > On 2/15/12 10:55 AM, "Peter Jones" wrote: > >> Nathan >> >> I have created placeholders for these items on the wiki. It would be >> great if you could elaborate as to the nature of these features - >> preferably creating a JIRA ticket with design docs etc attached. I am >> assuming for now that the NRS items will be covered by the existing NRS >> ticket (LU-398) but please let me know if that is not the case. >> >> Thanks >> >> Peter >> >> On 12-02-15 10:10 AM, Nathan Rutman wrote: >>> Re: cdwg concall this morning, >>> Peter sent out a link to the community's ongoing development wiki page >>> last week. >>> >>> I'd like to add some Xyratex plans that I posted to TWG in early Jan: >>> >>> 3/12 CRC-32c SW (crypto-api version of CRC-32c so that it is no longer >>> cpu-specific.) >>> 3/12 CRC-32 HW (add HW acceleration to crypto-api version of CRC-32, >>> and use it.) >>> 4/12 4MB IO >>> 4/12 NRS noop (null policy) >>> 5/12 NRS obrr (object-based round-robin policy) >>> 7/12 End-to-End Data Integrity with T10 >>> >>> >>> On Feb 6, 2012, at 2:03 PM, Peter Jones wrote: >>> >>>> Hi there >>>> >>>> Although we are still very busy with Lustre 2.2, we have started >>>> planning for the content of Lustre 2.3 release. Landings for 2.3 will >>>> start as soon as Lustre 2.2 has been branched from master. >>>> >>>> Lustre 2.3 is targeted for release in September 2012 and, as such, any >>>> features to be included will need to be landed to master by the end of >>>> June at the latest in order to allow adequate time for stabilization >>>> ahead of the release. If features only become ready for landing at the >>>> end of June there may not be sufficient time to land them all, so >>>> features that are available sooner are more likely to be tested, >>>> inspected, and landed before the feature freeze. >>>> >>>> If anyone is working on features that could be candidates to include >>>> in this release then please let me know so that we can consider them in >>>> our planning. >>>> >>>> Features that were initially targeted for 2.2 but were not ready in >>>> time are already on our list - >>>> http://wiki.whamcloud.com/display/PUB/Lustre+Community+Development+in+Pr >>>> ogress >>>> >>>> Thanks >>>> >>>> Peter >>>> >>>> -- >>>> Peter Jones >>>> Whamcloud, Inc. >>>> www.whamcloud.com >>>> >> -- >> Peter Jones >> Whamcloud, Inc. >> www.whamcloud.com >> >> _______________________________________________ >> cdwg mailing list >> cdwg at lists.opensfs.org >> http://lists.opensfs.org/listinfo.cgi/cdwg-opensfs.org > > -- Peter Jones Whamcloud, Inc. www.whamcloud.com From Nathan_Rutman at xyratex.com Wed Feb 15 18:10:04 2012 From: Nathan_Rutman at xyratex.com (Nathan Rutman) Date: Wed, 15 Feb 2012 10:10:04 -0800 Subject: [cdwg] [wc-discuss] Lustre 2.3 release In-Reply-To: <4F304E2E.9080100@whamcloud.com> References: <4F304E2E.9080100@whamcloud.com> Message-ID: <73AED5C780AE05478241DB067651A92102F107AC@XYUS-EX22.xyus.xyratex.com> Re: cdwg concall this morning, Peter sent out a link to the community's ongoing development wiki page last week. I'd like to add some Xyratex plans that I posted to TWG in early Jan: 3/12 CRC-32c SW (crypto-api version of CRC-32c so that it is no longer cpu-specific.) 3/12 CRC-32 HW (add HW acceleration to crypto-api version of CRC-32, and use it.) 4/12 4MB IO 4/12 NRS noop (null policy) 5/12 NRS obrr (object-based round-robin policy) 7/12 End-to-End Data Integrity with T10 On Feb 6, 2012, at 2:03 PM, Peter Jones wrote: > Hi there > > Although we are still very busy with Lustre 2.2, we have started planning for the content of Lustre 2.3 release. Landings for 2.3 will start as soon as Lustre 2.2 has been branched from master. > > Lustre 2.3 is targeted for release in September 2012 and, as such, any features to be included will need to be landed to master by the end of June at the latest in order to allow adequate time for stabilization ahead of the release. If features only become ready for landing at the end of June there may not be sufficient time to land them all, so features that are available sooner are more likely to be tested, inspected, and landed before the feature freeze. > > If anyone is working on features that could be candidates to include in this release then please let me know so that we can consider them in our planning. > > Features that were initially targeted for 2.2 but were not ready in time are already on our list - http://wiki.whamcloud.com/display/PUB/Lustre+Community+Development+in+Progress > > Thanks > > Peter > > -- > Peter Jones > Whamcloud, Inc. > www.whamcloud.com > From tao.peng at emc.com Thu Feb 16 02:15:41 2012 From: tao.peng at emc.com (tao.peng at emc.com) Date: Wed, 15 Feb 2012 21:15:41 -0500 Subject: [cdwg] [wc-discuss] Lustre 2.3 release In-Reply-To: <4F304E2E.9080100@whamcloud.com> References: <4F304E2E.9080100@whamcloud.com> Message-ID: Hi Peter, Thanks for bringing it up. As the target date for 2.3 is September 2012, I'd like to add some EMC tasks (not necessary features though) that we hope to land in 2.3 release cycle. Please consider including them in your planning. 1. remove obsolete configure macros that are there to support old kernels (older than 2.6.18-latest) 2. Lustre client dcache clean up 3. separate server connection handling from ptlrpc layer 4. client kernel API adjustment to latest upstream kernel 5. client module build process change to adapt to vanilla kernel Cheers, Tao > -----Original Message----- > From: Peter Jones [mailto:pjones at whamcloud.com] > Sent: Tuesday, February 07, 2012 6:03 AM > To: cdwg at lists.opensfs.org; wc-discuss at whamcloud.com > Subject: [wc-discuss] Lustre 2.3 release > > Hi there > > Although we are still very busy with Lustre 2.2, we have started > planning for the content of Lustre 2.3 release. Landings for 2.3 will > start as soon as Lustre 2.2 has been branched from master. > > Lustre 2.3 is targeted for release in September 2012 and, as such, any > features to be included will need to be landed to master by the end of > June at the latest in order to allow adequate time for stabilization > ahead of the release. If features only become ready for landing at the > end of June there may not be sufficient time to land them all, so > features that are available sooner are more likely to be tested, > inspected, and landed before the feature freeze. > > If anyone is working on features that could be candidates to include in > this release then please let me know so that we can consider them in our > planning. > > Features that were initially targeted for 2.2 but were not ready in time > are already on our list - > http://wiki.whamcloud.com/display/PUB/Lustre+Community+Development+in+Progress > > Thanks > > Peter > > -- > Peter Jones > Whamcloud, Inc. > www.whamcloud.com > From simmonsja at ornl.gov Thu Feb 16 17:21:28 2012 From: simmonsja at ornl.gov (Simmons, James A.) Date: Thu, 16 Feb 2012 12:21:28 -0500 Subject: [cdwg] [wc-discuss] Lustre 2.3 release In-Reply-To: <73AED5C780AE05478241DB067651A92102F107AC@XYUS-EX22.xyus.xyratex.com> References: <4F304E2E.9080100@whamcloud.com>, <73AED5C780AE05478241DB067651A92102F107AC@XYUS-EX22.xyus.xyratex.com> Message-ID: <6005A018DA581B4A80C06850774590950AAE7CAA04@EXCHMB.ornl.gov> >I'd like to add some Xyratex plans that I posted to TWG in early Jan: >... >4/12 4MB IO I'm also interested in this work. Let me know when you start on this. I have some early code bug haven't had the time to complete it. From pjones at whamcloud.com Thu Feb 16 18:30:49 2012 From: pjones at whamcloud.com (Peter Jones) Date: Thu, 16 Feb 2012 10:30:49 -0800 Subject: [cdwg] [wc-discuss] Lustre 2.3 release In-Reply-To: References: <4F304E2E.9080100@whamcloud.com> Message-ID: <4F3D4B59.2060004@whamcloud.com> Thank you Tao. This is most helpful. Except where noted below I have created wiki table entries with placeholders in anticipation of a JIRA ticket being opened with fuller details about the rationale and designs for the intended development. On 12-02-15 6:15 PM, tao.peng at emc.com wrote: > Hi Peter, > > Thanks for bringing it up. As the target date for 2.3 is September 2012, I'd like to add some EMC tasks (not necessary features though) that we hope to land in 2.3 release cycle. Please consider including them in your planning. > > 1. remove obsolete configure macros that are there to support old kernels (older than 2.6.18-latest) I believe that we are already tracking this task under LU-709. Your assistance in accelerating this work would be most appreciated. > 2. Lustre client dcache clean up > 3. separate server connection handling from ptlrpc layer > 4. client kernel API adjustment to latest upstream kernel Cray are already working on supporting Linux 3.0 under LU-812 and there is a ticket already opened for Linux 3.1 support (LU-858). Perhaps it makes sense to collaborate on these earlier versions as there will be some overlap between the work required. > 5. client module build process change to adapt to vanilla kernel > > Cheers, > Tao > >> -----Original Message----- >> From: Peter Jones [mailto:pjones at whamcloud.com] >> Sent: Tuesday, February 07, 2012 6:03 AM >> To: cdwg at lists.opensfs.org; wc-discuss at whamcloud.com >> Subject: [wc-discuss] Lustre 2.3 release >> >> Hi there >> >> Although we are still very busy with Lustre 2.2, we have started >> planning for the content of Lustre 2.3 release. Landings for 2.3 will >> start as soon as Lustre 2.2 has been branched from master. >> >> Lustre 2.3 is targeted for release in September 2012 and, as such, any >> features to be included will need to be landed to master by the end of >> June at the latest in order to allow adequate time for stabilization >> ahead of the release. If features only become ready for landing at the >> end of June there may not be sufficient time to land them all, so >> features that are available sooner are more likely to be tested, >> inspected, and landed before the feature freeze. >> >> If anyone is working on features that could be candidates to include in >> this release then please let me know so that we can consider them in our >> planning. >> >> Features that were initially targeted for 2.2 but were not ready in time >> are already on our list - >> http://wiki.whamcloud.com/display/PUB/Lustre+Community+Development+in+Progress >> >> Thanks >> >> Peter >> >> -- >> Peter Jones >> Whamcloud, Inc. >> www.whamcloud.com >> -- Peter Jones Whamcloud, Inc. www.whamcloud.com From surya1 at llnl.gov Thu Feb 16 23:36:05 2012 From: surya1 at llnl.gov (Prakash Surya) Date: Thu, 16 Feb 2012 15:36:05 -0800 Subject: [cdwg] [wc-discuss] Lustre 2.3 release In-Reply-To: <4F3D4B59.2060004@whamcloud.com> References: <4F304E2E.9080100@whamcloud.com> <4F3D4B59.2060004@whamcloud.com> Message-ID: <20120216233605.GC6772@llnl.gov> On Thu, Feb 16, 2012 at 10:30:49AM -0800, Peter Jones wrote: > Thank you Tao. This is most helpful. Except where noted below I have > created wiki table entries with placeholders in anticipation of a JIRA > ticket being opened with fuller details about the rationale and designs > for the intended development. > > On 12-02-15 6:15 PM, tao.peng at emc.com wrote: > > Hi Peter, > > > > Thanks for bringing it up. As the target date for 2.3 is September 2012, I'd like to add some EMC tasks (not necessary features though) that we hope to land in 2.3 release cycle. Please consider including them in your planning. > > > > 1. remove obsolete configure macros that are there to support old kernels (older than 2.6.18-latest) > I believe that we are already tracking this task under LU-709. Your > assistance in accelerating this work would be most appreciated. I think any cleanup and/or improvements to the configure and build infrastructure is much appreciated. Patches are welcomed. :) > > 2. Lustre client dcache clean up > > 3. separate server connection handling from ptlrpc layer > > 4. client kernel API adjustment to latest upstream kernel > Cray are already working on supporting Linux 3.0 under LU-812 and there > is a ticket already opened for Linux 3.1 support (LU-858). Perhaps it > makes sense to collaborate on these earlier versions as there will be > some overlap between the work required. > > 5. client module build process change to adapt to vanilla kernel > > > > Cheers, > > Tao > > > >> -----Original Message----- > >> From: Peter Jones [mailto:pjones at whamcloud.com] > >> Sent: Tuesday, February 07, 2012 6:03 AM > >> To: cdwg at lists.opensfs.org; wc-discuss at whamcloud.com > >> Subject: [wc-discuss] Lustre 2.3 release > >> > >> Hi there > >> > >> Although we are still very busy with Lustre 2.2, we have started > >> planning for the content of Lustre 2.3 release. Landings for 2.3 will > >> start as soon as Lustre 2.2 has been branched from master. > >> > >> Lustre 2.3 is targeted for release in September 2012 and, as such, any > >> features to be included will need to be landed to master by the end of > >> June at the latest in order to allow adequate time for stabilization > >> ahead of the release. If features only become ready for landing at the > >> end of June there may not be sufficient time to land them all, so > >> features that are available sooner are more likely to be tested, > >> inspected, and landed before the feature freeze. > >> > >> If anyone is working on features that could be candidates to include in > >> this release then please let me know so that we can consider them in our > >> planning. > >> > >> Features that were initially targeted for 2.2 but were not ready in time > >> are already on our list - > >> http://wiki.whamcloud.com/display/PUB/Lustre+Community+Development+in+Progress > >> > >> Thanks > >> > >> Peter > >> > >> -- > >> Peter Jones > >> Whamcloud, Inc. > >> www.whamcloud.com > >> > > -- > Peter Jones > Whamcloud, Inc. > www.whamcloud.com > -- Cheers, Prakash From morrone2 at llnl.gov Fri Feb 17 01:35:05 2012 From: morrone2 at llnl.gov (Christopher J. Morrone) Date: Thu, 16 Feb 2012 17:35:05 -0800 Subject: [cdwg] Fwd: Re: [wc-discuss] Lustre 2.3 release In-Reply-To: <4F3D254A.3000107@whamcloud.com> References: <4F3D254A.3000107@whamcloud.com> Message-ID: <4F3DAEC9.9010007@llnl.gov> Whoops, I only replied to one list by accident, so here it is again. Sorry for the double post for those that did see it. -------- Original Message -------- Subject: Re: [cdwg] [wc-discuss] Lustre 2.3 release Date: Thu, 16 Feb 2012 07:48:26 -0800 From: Peter Jones To: wc-discuss at whamcloud.com Chris I certainly endorse your view that JIRA should be the primary source of information relating to Lustre development. Having a dynamic list in JIRA is certainly more convenient than editing tables wiki markup :-) I have created a filter for features - http://jira.whamcloud.com/secure/IssueNavigator.jspa?mode=hide&requestId=10194 - which can be accessed directly through this link or else through Manage filters - Popular. I still think that the wiki table is useful though because (at least at the moment) it contains a superset of the information in JIRA. Regards Peter On 12-02-15 6:07 PM, Christopher J. Morrone wrote: > I think I like the idea of keeping the list on the wiki fairly generic > for the reason that Peter gives. > > Perhaps we should use jira more for this kind of release planning. > jira already has a "New Feature" issue type. Like Peter suggested, > there should be jira tickets for each of these items with design > documents attached, and where we can have conversations about the > features and give status updates. > > We can also mark which OpenSFS Lustre release the feature is planned > to have, but among us developer folks it would be accepted that the > target is by no means a guarantee of delivery. A feature goes in when > it is reviewed and accepted. If it misses its planned target version, > the version target can easily be changed in jira. > > By using jira, it is easy for all of the developer folks to get lists > of features being worked on, and planned completion dates. > > We might even formalize the freeze dates in jira by having additional > version targets like "2.2.0-freeze". Which allows us to have separate > bugs and such that are allowed in after the freeze to target "2.2.0". > Things that miss "2.2.0-freeze" get moved to "2.3.0-freeze". Or > something along those lines... > > Then maybe the wiki that will summarize things for users and managers > will only get a target version listed when the feature has > demonstrated through jira ticket traffic and gerrit patch submissions > that it has sufficient development momentum that it can be reasonably > expected to meet its target date. > > In other words, we use jira as the detailed release planning tool, > with the understanding that some features can, and will, miss their > target dates. Stuff happens. The wiki will be a summary of features > in jira that are showing good progress and that we are confident will > finish on schedule. > > That way we can manage expectations for the broader Lustre user base > while still having the detailed discussions and planning amongst the > various development teams. > > Chris > > On 02/15/2012 01:27 PM, Peter Jones wrote: >> Kevin >> >> I have added this column to the table. >> >> As I explained on the call, my concern with having such a column was >> that it might cause confusion in the community if the targeted release >> supplied does not prove to be realistic. >> >> In the past we ran into issues where people had relied on timelines in >> aggressive roadmaps and planned around them, and then were left in >> trouble when the features were not delivered according to the schedule. >> That is why we have consciously published conservative roadmaps with the >> intention of over-delivering. >> >> However, I can see this option being of interest to the community so I >> am ok with giving this a try. >> >> Hopefully having all the design documents, test plan being in the open >> in JIRA and the code uploaded into gerrit will enable interested parties >> to see for themselves how things are progressing and perhaps it will >> prove useful to spark a dialogue if there appears to be a divergence in >> expectations. >> >> Regards >> >> Peter >> >> On 12-02-15 12:53 PM, Kevin Canady wrote: >>> Peter >>> This is great, many thanks, it's a good demonstration how much is being >>> done to move Lustre forward. If it would not further complicate, it >>> would >>> be helpful to have some reference to the targeted release of the >>> features.Can we add a column to the list? The obvious "right to >>> change" >>> would be in place. The reference is only to help set expectations >>> within >>> the community. >>> >>> Sound reasonable? > -- Peter Jones Whamcloud, Inc. www.whamcloud.com . From tao.peng at emc.com Fri Feb 17 08:58:18 2012 From: tao.peng at emc.com (tao.peng at emc.com) Date: Fri, 17 Feb 2012 03:58:18 -0500 Subject: [cdwg] [wc-discuss] Lustre 2.3 release In-Reply-To: <4F3D4B59.2060004@whamcloud.com> References: <4F304E2E.9080100@whamcloud.com> <4F3D4B59.2060004@whamcloud.com> Message-ID: > -----Original Message----- > From: Peter Jones [mailto:pjones at whamcloud.com] > Sent: Friday, February 17, 2012 2:31 AM > To: Peng, Tao > Cc: cdwg at lists.opensfs.org; wc-discuss at whamcloud.com; Tang, Haiying; faibish, sorin > Subject: Re: [wc-discuss] Lustre 2.3 release > > Thank you Tao. This is most helpful. Except where noted below I have > created wiki table entries with placeholders in anticipation of a JIRA > ticket being opened with fuller details about the rationale and designs > for the intended development. > It is great that we have the discussion so that our work don't get superfluous. > On 12-02-15 6:15 PM, tao.peng at emc.com wrote: > > Hi Peter, > > > > Thanks for bringing it up. As the target date for 2.3 is September 2012, I'd like to add some EMC > tasks (not necessary features though) that we hope to land in 2.3 release cycle. Please consider > including them in your planning. > > > > 1. remove obsolete configure macros that are there to support old kernels (older than 2.6.18-latest) > I believe that we are already tracking this task under LU-709. Your > assistance in accelerating this work would be most appreciated. Thank you for the information. I have done some work as well. I will look at LU-709 more closely and find a way to collaborate with Yang Shen. > > 2. Lustre client dcache clean up I created LU-1113 (http://jira.whamcloud.com/browse/LU-1113) for this one and attached related design document there. I also updated the wiki page to add a pointer to it. > > 3. separate server connection handling from ptlrpc layer > > 4. client kernel API adjustment to latest upstream kernel > Cray are already working on supporting Linux 3.0 under LU-812 and there > is a ticket already opened for Linux 3.1 support (LU-858). Perhaps it > makes sense to collaborate on these earlier versions as there will be > some overlap between the work required. Agreed. We will try to collaborate with community on LU-812 and LU-858, and base our work of supporting latest Linux kernel on top of them. Thanks, Tao > > 5. client module build process change to adapt to vanilla kernel > > > > Cheers, > > Tao > > > >> -----Original Message----- > >> From: Peter Jones [mailto:pjones at whamcloud.com] > >> Sent: Tuesday, February 07, 2012 6:03 AM > >> To: cdwg at lists.opensfs.org; wc-discuss at whamcloud.com > >> Subject: [wc-discuss] Lustre 2.3 release > >> > >> Hi there > >> > >> Although we are still very busy with Lustre 2.2, we have started > >> planning for the content of Lustre 2.3 release. Landings for 2.3 will > >> start as soon as Lustre 2.2 has been branched from master. > >> > >> Lustre 2.3 is targeted for release in September 2012 and, as such, any > >> features to be included will need to be landed to master by the end of > >> June at the latest in order to allow adequate time for stabilization > >> ahead of the release. If features only become ready for landing at the > >> end of June there may not be sufficient time to land them all, so > >> features that are available sooner are more likely to be tested, > >> inspected, and landed before the feature freeze. > >> > >> If anyone is working on features that could be candidates to include in > >> this release then please let me know so that we can consider them in our > >> planning. > >> > >> Features that were initially targeted for 2.2 but were not ready in time > >> are already on our list - > >> http://wiki.whamcloud.com/display/PUB/Lustre+Community+Development+in+Progress > >> > >> Thanks > >> > >> Peter > >> > >> -- > >> Peter Jones > >> Whamcloud, Inc. > >> www.whamcloud.com > >> > > -- > Peter Jones > Whamcloud, Inc. > www.whamcloud.com > From pjones at whamcloud.com Fri Feb 17 14:57:25 2012 From: pjones at whamcloud.com (Peter Jones) Date: Fri, 17 Feb 2012 06:57:25 -0800 Subject: [cdwg] Lustre 2.2 update - February 17th 2012 Message-ID: <4F3E6AD5.9060902@whamcloud.com> Hi there Here is an update on the Lustre 2.2 release. Minor Enhancements ================== -The changes required for the remaining patch for fc15 client support are considered too risky to accept this close to GA so fc15 support has been descoped from 2.2; we expect this patch to be amongst the first landings when master reopens for 2.3 landings (LU506) Testing ======= -Testing has completed on the 2.1.55 tag; a 2.1.56 tag will be created shortly Blockers ======== -At the time of writing we are down to 11 blockers - see http://jira.whamcloud.com/secure/IssueNavigator.jspa?mode=hide&requestId=10184 for details Other ===== -An e2fsprogs release that will support the new features in 2.2 was released this week and is available at http://downloads.whamcloud.com/public/e2fsprogs/1.41.90.wc4/ We will continue to provide periodic updates on our progress on this release. In the meantime, you can always see the landings as they happen at http://git.whamcloud.com/?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master and follow the patch reviews and testing at http://review.whamcloud.com/#q,status:open+project:fs/lustre-release+branch:master,n,z Regards Peter -- Peter Jones Whamcloud, Inc. www.whamcloud.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From simmonsja at ornl.gov Fri Feb 17 15:08:40 2012 From: simmonsja at ornl.gov (Simmons, James A.) Date: Fri, 17 Feb 2012 10:08:40 -0500 Subject: [cdwg] Fwd: Re: [wc-discuss] Lustre 2.3 release In-Reply-To: <4F3DAEC9.9010007@llnl.gov> References: <4F3D254A.3000107@whamcloud.com>,<4F3DAEC9.9010007@llnl.gov> Message-ID: <6005A018DA581B4A80C06850774590950AAE7CAA06@EXCHMB.ornl.gov> >I certainly endorse your view that JIRA should be the primary source of >information relating to Lustre development. Having a dynamic list in >JIRA is certainly more convenient than editing tables wiki markup :-) I >have created a filter for features - >http://jira.whamcloud.com/secure/IssueNavigator.jspa?mode=hide&requestId=10194 >- which can be accessed directly through this link or else through >Manage filters - Popular. I still think that the wiki table is useful >though because (at least at the moment) it contains a superset of the >information in JIRA. Thanks Chris for setting that up. From hamilton5 at llnl.gov Tue Feb 21 20:21:34 2012 From: hamilton5 at llnl.gov (Hamilton, Pam) Date: Tue, 21 Feb 2012 12:21:34 -0800 Subject: [cdwg] Hyperion system restrictions Message-ID: Hi all, On the CDWG call last week, I clarified the Hyperion root account restrictions. Specifically, that Sensitive Country Foreign Nationals (SCFNs) are NOT allowed to have root accounts. Here is a link to a site at LBL which LLNL uses as a reference for which countries are considered sensitive: http://www.lbl.gov/ehs/ops/ufva/issm_sensitive.shtml Foreign Nationals of countries not on this list are allowed to have root accounts. Regards, Pam ___________________________________ Pam Hamilton Lawrence Livermore National Lab P.O. Box 808, L-556 Livermore, CA  94551-9900 E-Mail:  pgh at llnl.gov Phone:  925-423-1332          Fax:  925-423-8719 From hamilton5 at llnl.gov Fri Feb 24 04:16:24 2012 From: hamilton5 at llnl.gov (Hamilton, Pam) Date: Thu, 23 Feb 2012 20:16:24 -0800 Subject: [cdwg] Canceled: OpenSFS Community Development WG call Message-ID: When: Wednesday, February 29, 2012 9:00 AM-10:00 AM (UTC-08:00) Pacific Time (US & Canada). Where: Call-in: 866-914-3976 (925-424-8105) Passcode: 534986# Note: The GMT offset above does not reflect daylight saving time adjustments. *~*~*~*~*~*~*~*~*~* Hi all, This week’s call is cancelled. I anticipate having a call on March 14th. Regards, Pam -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 2620 bytes Desc: not available URL: From shay at opensfs.org Wed Feb 29 18:52:27 2012 From: shay at opensfs.org (Shay Seager) Date: Wed, 29 Feb 2012 10:52:27 -0800 Subject: [cdwg] LUG Early Registration Extended! Message-ID: <3E47F968-BD08-4297-9E71-A2C796C4C20E@opensfs.org> Happy Leap Day! To celebrate getting an extra day this year we are giving you 9 extra days to register! We are extending early bird registration to MARCH 9th, 2012, but don't wait until the last minute, our room block is filling up fast! Make sure you get good hotel rates by registering today! The LUG Program will be published March 1st! Make sure you check http://www.opensfs.org/lug/program to get the full event schedule. We have also extended government rated rooms to April 25th, so you can stay Wednesday for the OpenSFS General meeting, Technical Working Group Meeting, and Community Development Working Group Meeting. These OpenSFS meetings are the perfect place to get your creative juices flowing and contribute to the future of Lustre and OpenSFS; with all the inspiration from 2.5 days of amazing speakers and innovative designs, you are going to be bursting with great ideas, so make sure you stick around with us from 1pm-5pm on Wednesday April 25th. To ensure that you get to enjoy every ounce of LUG we recommend flying in sunday April 22 in the morning and flying out Thursday April 26th in the morning. Register today! http://www.opensfs.org/lug/registration We look forward to seeing you in Austin! - - - Shay Seager Open SFS Secretary www.opensfs.org (925) 290-7641 shay at opensfs.org -------------- next part -------------- An HTML attachment was scrubbed... URL: