From carrier at cray.com Mon May 2 17:46:18 2011 From: carrier at cray.com (John Carrier) Date: Mon, 2 May 2011 12:46:18 -0500 Subject: [Twg] TWG meeting minutes for 2011-04-28 Message-ID: Attached are the meeting minutes from last week's TWG call where we continued the discussion of the release collateral that OpenSFS will require for delivery of features that result from our RFPs. Please post questions/comments to discuss at lists.opensfs.org Next meeting is this Thursday (5/5) @ 9:30a PDT/12:30p EDT, (715) 726-4994, ID/password 72090. Thanks, --jc -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- OpenSFS Technical Working Group Meeting minutes : 04/28/2011 Concall: start : 9:30a PT, adjourn : 10:35a PT Next meeting: Thursday, 5/05/2011 @ 9:30a PDT/12:30p EDT Dial-in numbers are 715-726-4994 or 866-304-8294. Meeting ID and password are 72090 (note the new code) Attending : Name Organization email ----------------- -------------- ---------------------------- Diego Moreno Bull Diego.Moreno-Lazaro at bull.net John Carrier Cray carrier at cray.com Alex Kulyavtsev FNAL aik at fnal.gov Chris Morrone LLNL morrone2 at llnl.gov Shay Seager OpenSFS shay at opensfs.org Dave Dillow ORNL dillowda at ornl.gov Galen Shipman ORNL gshipman at ornl.gov Andreas Dilger Whamcloud adilger at whamcloud.com Eric Barton Whamcloud eeb at whamcloud.com Nathan Rutman Xyratex nathan_rutman at xyratex.com agenda continue discussion of release collateral discussion The current proposal for the release collateral is - high level design (HLD) - test plan - test results - code inspections for the features (release checklist) - standard lustre coding guidelines (coding checklist) Check lists : Last week's meeting ended with descriptions of coding and release checklists. Whamcloud has a version of the coding checklist on their wiki. We asked for sources of the release checklist be sent to the reflector. Emails to the reflector indicated that CFS derived the release checklist from the Software Engineering Institute (SEI) at CMU. As a result, we need to create our own. Release checklist items should include well-chosen short-list rather than a comprehensive long-list. Things to consider: consistent naming, thread model verification, locking, etc. Agreed to take creation of the list to the reflector. AI: chairs to start thread on discuss at lists.opensfs.org Design docs : As part of the release collateral, we need vendors to provide design documents for the features they have implemented. We started this discussion by reviewing the docs the Lustre team has used in the past: ALD = architecture-level design HLD = high-level design DLD = detail-level design The DLD is of more use to the vendor writing the code since it includes detailed work flows, procedure skeletons, data structures, and anything else needed to guide implementation of the code. For the TWG and the rest of the community, the code itself will be the documentation for what the design does. We agreed that the TWG needed to see well-written English documents describing the ideas behind the design, similar to the HLDs that the Lustre team had generated before. This high-level design document will describe the ideas behind the design and provide background to anyone reading the code. It should include a description of the implementation (but no procedure or structure definitions) and diagrams needed to clarify the design. Vendors will be expected to update the document after the implementation is complete as part of the release collateral. The HLD represents considerable work. Vendors need to engage the community with the architectural overview of the design to ensure that they have addressed the requirements and provided the necessary scope. This solution architecture is recorded in the ALD. We agreed that statements of work (SOWs) for feature development resulting from the OpenSFS RFPs will require two reviews prior to feature implementation. The first will review the proposed architecture and its scope (ALD) and the second will review the high-level design (HLD). The documents will be posted to the OpenSFS website and the TWG will hold concalls for each review. We will investigate whether we can record the meetings for those unable to attend the concalls. Summary: * The TWG will require an ALD and HLD for OpenSFS SOWs - The ALD provides a review of the requirements and an outline and scope of the architecture before proceeding to the HLD. - The HLD provides a description of the architecture and an overview of the implementation. - The vendor will review both documents during a TWG concall with the Lustre community. * The HLD will be required as release collateral - We expect anyone contributing new features to the Lustre canonical tree to provide this document before a code review. We also discussed the need for more than one deliverable during the implementation phase. The OpenSFS RFPs provide this structure and we should expect the SOW to make those deliverables explicit. Test Plan We then spent time discussing the test plan. We recognized that not all features, because of their scope, can be tested with well-defined unit-tests. We need to allow for these features to be tested with system tests. The TWG should review the test plan as part of the SOW for OpenSFS RFPs. The test plan should be an english description of the tests of new functionality that verifies the new code does not negatively affect existing functionality. The plan needs to include a minumum scale required to test the feature. Reviewers may also require a minumum scale for the tests before allowing the review to complete. AI: Whamcloud agreed to provide text describing the test plan Inspectors It was suggested on the reflector that we pre-register inspectors. Gerrit, which Whamcloud is using, allows anyone to sign up as an inspector. While we want to encourage broad review, OpenSFS needs inspection by at least one recognized expert. We agreed to send out a call to the community for people who feel knowledgable enough to review code. Inspectors must also indicate the subsystems they feel competent to review. AI: chairs to send out inspector request adjourned 10:35 From spitzcor at cray.com Mon May 2 18:59:26 2011 From: spitzcor at cray.com (Cory Spitz) Date: Mon, 2 May 2011 13:59:26 -0500 Subject: [Twg] TWG meeting minutes for 2011-04-28 In-Reply-To: References: Message-ID: <4DBEFF0E.6060203@cray.com> Hi. I missed the call last week, but I would like to propose an additional item for release collateral, but I don't know if we can make it work. I'd very much prefer that the designs update the Lustre Internals Documentation (LID at http://wiki.lustre.org/lid), including both the self documenting code and the long-form writings at http://wiki.lustre.org/lid/agi/agi.html. However, I don't know if the community can continue to leverage this resource. Any comments on that issue? Thanks, -Cory On 05/02/2011 12:46 PM, John Carrier wrote: > Attached are the meeting minutes from last week’s TWG call where we > continued the discussion of the release collateral that OpenSFS will > require for delivery of features that result from our RFPs. > > > > Please post questions/comments to discuss at lists.opensfs.org > > > > > Next meeting is this Thursday (5/5) @ 9:30a PDT/12:30p EDT, (715) > 726-4994, ID/password 72090. > > > > > > Thanks, > > > > --jc > > > From adilger at whamcloud.com Mon May 2 20:03:46 2011 From: adilger at whamcloud.com (Andreas Dilger) Date: Mon, 2 May 2011 14:03:46 -0600 Subject: [Twg] [Discuss] TWG meeting minutes for 2011-04-28 In-Reply-To: <4DBEFF0E.6060203@cray.com> References: <4DBEFF0E.6060203@cray.com> Message-ID: <1DBFA14D-791E-4E52-A04C-8D322652E189@whamcloud.com> Currently I don't think anyone outside Oracle has write access on the wiki, so it may be problematic. Cheers, Andreas On 2011-05-02, at 12:59 PM, Cory Spitz wrote: > Hi. > > I missed the call last week, but I would like to propose an additional > item for release collateral, but I don't know if we can make it work. > > I'd very much prefer that the designs update the Lustre Internals > Documentation (LID at http://wiki.lustre.org/lid), including both the > self documenting code and the long-form writings at > http://wiki.lustre.org/lid/agi/agi.html. > > However, I don't know if the community can continue to leverage this > resource. Any comments on that issue? > > Thanks, > -Cory > > On 05/02/2011 12:46 PM, John Carrier wrote: >> Attached are the meeting minutes from last week’s TWG call where we >> continued the discussion of the release collateral that OpenSFS will >> require for delivery of features that result from our RFPs. >> >> >> >> Please post questions/comments to discuss at lists.opensfs.org >> >> >> >> >> Next meeting is this Thursday (5/5) @ 9:30a PDT/12:30p EDT, (715) >> 726-4994, ID/password 72090. >> >> >> >> >> >> Thanks, >> >> >> >> --jc >> >> >> > _______________________________________________ > discuss mailing list > discuss at lists.opensfs.org > http://lists.opensfs.org/listinfo.cgi/discuss-opensfs.org From ssseager at gmail.com Mon May 2 23:58:56 2011 From: ssseager at gmail.com (Shay Seager) Date: Mon, 2 May 2011 16:58:56 -0700 Subject: [Twg] OpenSFS LUG Participation Meeting Notes Message-ID: <0B396AA9-CDE9-4FBD-AD80-F7186F72EF07@gmail.com> Hello Working Groups, We have posted the LUG Participation Meeting notes under the Resource Tab on the website, http://www.opensfs.org/?page_id=80. Best, Shay - - - Shay Seager Open SFS Secretary www.opensfs.org (925) 290-7641 shay at opensfs.org From adilger at whamcloud.com Thu May 5 16:54:19 2011 From: adilger at whamcloud.com (Andreas Dilger) Date: Thu, 5 May 2011 10:54:19 -0600 Subject: [Twg] Managing code history and test collateral References: Message-ID: > From: Chris Gearing > > The greatest problem facing the code acceptance process for Lustre is establishing the complete history of the code being submitted. To fully establish the history the code development process must be documented from conception thru landing. This documentation which will form part of the landing collateral will include at least the requirements specification, the design document, test specification/plan, results of testing, the code itself and for copyright purposes the identity of the owner of the submitted code. A key piece of information that must be present is evidence in the form of failing tests that indicate the tests built are fit for purpose. Results showing the failing tests before code implementation are vital to demonstrate that passing results have value. > > As a timeline this collateral can be defined as; > 1/ What is the requirement > 2/ How will the requirement be met > 3/ How will success be measured > 4/ Validation that the code does not already meet the requirement > 5/ The Code > 6/ Validation the code now meets the requirement including general regression testing. > 7/ Code ownership information. > > The documents can be extensive and detailed for a major project like the health network or updates of a bugtracker for a bug or small improvement, however in all cases each of the 7 collateral points must be covered. Another key requirement of all documents is that they must be free standing in their own right, this means that whilst references to other specifications or documents are obviously acceptable references to 'lustre-devel', 'email-threads' or 'Skype discussions' etc are not. In all cases a lustre capable gatekeeper should have at hand everything to answer the 7 points on the timeline without the need for any historical knowledge of the code change. > Cheers, Andreas -- Andreas Dilger Principal Engineer Whamcloud, Inc. -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter_bojanic at xyratex.com Thu May 5 17:10:23 2011 From: peter_bojanic at xyratex.com (Peter Bojanic) Date: Thu, 5 May 2011 14:10:23 -0300 Subject: [Twg] Managing code history and test collateral In-Reply-To: References: Message-ID: Andreas, Chris, I acknowledge the challenge associated with widely distributed "snippets" of technical documentation. A solid body of knowledge associated with code changes (features and bug fixes) will be increasingly important with diverse parties contributing to Lustre. At Xyratex we're weaving a lot of process around Google Docs, which enables us to have shared tables and forms (leveraging spreadsheets extensively) that can be updated and maintained as workflow progresses. Nothing (Jira, Bugzilla, etc.) is quite as flexible as Google docs and there's robust change tracking as well as (new) issue discussion/resolution features that have just shown up in the Enterprise edition. We track development items as they progress through stages of our process (based on TSP) so everyone knows exactly what steps we have completed and what is left to do. Our control tables accommodate linking to other documents (requirements, test plans, high level designs, etc.) so you've got a complete inventory of the associated artefacts. I know that many of us (OpenSFS, probably WhamCloud) are using Google Docs already and you're probably doing something similar. I would highly recommend that the TWG craft process that enables our respective organizations to use our own internal quality development processes yet ties it all together into a common framework for advancing contributions to canonical code base. I'd be happy to contribute to such an effort. Cheers, Bojanic On May 5, 2011, at 13:54 , Andreas Dilger wrote: >> From: Chris Gearing >> >> The greatest problem facing the code acceptance process for Lustre is establishing the complete history of the code being submitted. To fully establish the history the code development process must be documented from conception thru landing. This documentation which will form part of the landing collateral will include at least the requirements specification, the design document, test specification/plan, results of testing, the code itself and for copyright purposes the identity of the owner of the submitted code. A key piece of information that must be present is evidence in the form of failing tests that indicate the tests built are fit for purpose. Results showing the failing tests before code implementation are vital to demonstrate that passing results have value. >> >> As a timeline this collateral can be defined as; >> 1/ What is the requirement >> 2/ How will the requirement be met >> 3/ How will success be measured >> 4/ Validation that the code does not already meet the requirement >> 5/ The Code >> 6/ Validation the code now meets the requirement including general regression testing. >> 7/ Code ownership information. >> >> The documents can be extensive and detailed for a major project like the health network or updates of a bugtracker for a bug or small improvement, however in all cases each of the 7 collateral points must be covered. Another key requirement of all documents is that they must be free standing in their own right, this means that whilst references to other specifications or documents are obviously acceptable references to 'lustre-devel', 'email-threads' or 'Skype discussions' etc are not. In all cases a lustre capable gatekeeper should have at hand everything to answer the 7 points on the timeline without the need for any historical knowledge of the code change. >> > > > Cheers, Andreas > -- > Andreas Dilger > Principal Engineer > Whamcloud, Inc. > > > > _______________________________________________ > twg mailing list > twg at lists.opensfs.org > http://lists.opensfs.org/listinfo.cgi/twg-opensfs.org ______________________________________________________________________ This email may contain privileged or confidential information, which should only be used for the purpose for which it was sent by Xyratex. No further rights or licenses are granted to use such information. If you are not the intended recipient of this message, please notify the sender by return and delete it. You may not use, copy, disclose or rely on the information contained in it. Internet email is susceptible to data corruption, interception and unauthorised amendment for which Xyratex does not accept liability. While we have taken reasonable precautions to ensure that this email is free of viruses, Xyratex does not accept liability for the presence of any computer viruses in this email, nor for any losses caused as a result of viruses. Xyratex Technology Limited (03134912), Registered in England & Wales, Registered Office, Langstone Road, Havant, Hampshire, PO9 1SA. The Xyratex group of companies also includes, Xyratex Ltd, registered in Bermuda, Xyratex International Inc, registered in California, Xyratex (Malaysia) Sdn Bhd registered in Malaysia, Xyratex Technology (Wuxi) Co Ltd registered in The People's Republic of China and Xyratex Japan Limited registered in Japan. ______________________________________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: From carrier at cray.com Thu May 5 17:25:43 2011 From: carrier at cray.com (John Carrier) Date: Thu, 5 May 2011 12:25:43 -0500 Subject: [Twg] Managing code history and test collateral In-Reply-To: References: Message-ID: Peter, I don't see requirements for the landing collateral impacting the internal processes any organization uses. We are trying to define the common framework for what needs to be delivered for code to be accepted and integrated into the main tree. This definition will then be a basis for what OpenSFS will require in the statements of work resulting from the RFPs. I very much appreciate your contributions to this discussion. --jc From: twg-bounces at lists.opensfs.org [mailto:twg-bounces at lists.opensfs.org] On Behalf Of Peter Bojanic Sent: Thursday, May 05, 2011 10:10 AM To: Andreas Dilger; Chris Gearing Cc: twg at lists.opensfs.org Subject: Re: [Twg] Managing code history and test collateral Andreas, Chris, I acknowledge the challenge associated with widely distributed "snippets" of technical documentation. A solid body of knowledge associated with code changes (features and bug fixes) will be increasingly important with diverse parties contributing to Lustre. At Xyratex we're weaving a lot of process around Google Docs, which enables us to have shared tables and forms (leveraging spreadsheets extensively) that can be updated and maintained as workflow progresses. Nothing (Jira, Bugzilla, etc.) is quite as flexible as Google docs and there's robust change tracking as well as (new) issue discussion/resolution features that have just shown up in the Enterprise edition. We track development items as they progress through stages of our process (based on TSP) so everyone knows exactly what steps we have completed and what is left to do. Our control tables accommodate linking to other documents (requirements, test plans, high level designs, etc.) so you've got a complete inventory of the associated artefacts. I know that many of us (OpenSFS, probably WhamCloud) are using Google Docs already and you're probably doing something similar. I would highly recommend that the TWG craft process that enables our respective organizations to use our own internal quality development processes yet ties it all together into a common framework for advancing contributions to canonical code base. I'd be happy to contribute to such an effort. Cheers, Bojanic On May 5, 2011, at 13:54 , Andreas Dilger wrote: From: Chris Gearing > The greatest problem facing the code acceptance process for Lustre is establishing the complete history of the code being submitted. To fully establish the history the code development process must be documented from conception thru landing. This documentation which will form part of the landing collateral will include at least the requirements specification, the design document, test specification/plan, results of testing, the code itself and for copyright purposes the identity of the owner of the submitted code. A key piece of information that must be present is evidence in the form of failing tests that indicate the tests built are fit for purpose. Results showing the failing tests before code implementation are vital to demonstrate that passing results have value. As a timeline this collateral can be defined as; 1/ What is the requirement 2/ How will the requirement be met 3/ How will success be measured 4/ Validation that the code does not already meet the requirement 5/ The Code 6/ Validation the code now meets the requirement including general regression testing. 7/ Code ownership information. The documents can be extensive and detailed for a major project like the health network or updates of a bugtracker for a bug or small improvement, however in all cases each of the 7 collateral points must be covered. Another key requirement of all documents is that they must be free standing in their own right, this means that whilst references to other specifications or documents are obviously acceptable references to 'lustre-devel', 'email-threads' or 'Skype discussions' etc are not. In all cases a lustre capable gatekeeper should have at hand everything to answer the 7 points on the timeline without the need for any historical knowledge of the code change. Cheers, Andreas -- Andreas Dilger Principal Engineer Whamcloud, Inc. _______________________________________________ twg mailing list twg at lists.opensfs.org http://lists.opensfs.org/listinfo.cgi/twg-opensfs.org ______________________________________________________________________ This email may contain privileged or confidential information, which should only be used for the purpose for which it was sent by Xyratex. No further rights or licenses are granted to use such information. If you are not the intended recipient of this message, please notify the sender by return and delete it. You may not use, copy, disclose or rely on the information contained in it. Internet email is susceptible to data corruption, interception and unauthorised amendment for which Xyratex does not accept liability. While we have taken reasonable precautions to ensure that this email is free of viruses, Xyratex does not accept liability for the presence of any computer viruses in this email, nor for any losses caused as a result of viruses. Xyratex Technology Limited (03134912), Registered in England & Wales, Registered Office, Langstone Road, Havant, Hampshire, PO9 1SA. The Xyratex group of companies also includes, Xyratex Ltd, registered in Bermuda, Xyratex International Inc, registered in California, Xyratex (Malaysia) Sdn Bhd registered in Malaysia, Xyratex Technology (Wuxi) Co Ltd registered in The People's Republic of China and Xyratex Japan Limited registered in Japan. ______________________________________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: From carrier at cray.com Mon May 9 04:45:56 2011 From: carrier at cray.com (John Carrier) Date: Sun, 8 May 2011 23:45:56 -0500 Subject: [Twg] TWG meeting minutes for 2011-05-05 Message-ID: Attached are the meeting minutes from last week's TWG call where we discussed AIs for the release collateral discussion and introduced the instructions for the RFP process. Please post questions/comments to discuss at lists.opensfs.org. Next meeting is this Thursday (5/12) @ 9:30a PDT/12:30p EDT, (715) 726-4994, ID/password 72090. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- OpenSFS Technical Working Group Meeting minutes : 05/05/2011 Concall: start : 9:30a PT, adjourn : Next meeting: Thursday, 05/12/2011 @ 9:30a PDT/12:30p EDT Dial-in numbers are 715-726-4994 or 866-304-8294. Meeting ID and password are 72090 Attending Name Organization email ----------------- -------------- ------------------------------- Diego Moreno Bull Diego.Moreno-Lazaro at bull.net Cory Spitz Cray spitzcor at cray.com John Carrier Cray carrier at cray.com Alex Kulyavtsev FNAL aik at fnal.gov Steve Simms IU ssimms at indiana.edu Chris Morrone LLNL morrone2 at llnl.gov Shay Seager OpenSFS shay at opensfs.org Dave Dillow ORNL dillowda at ornl.gov Galen Shipman ORNL gshipman at ornl.gov Andreas Dilger Whamcloud adilger at whamcloud.com Agenda release collateral update (JohnC) RFP process (Galen) Discussion Release Collateral There were a couple action items that need to be started on the reflector: - release check list - code inspector sign-up - test plan definition JohnC has text for the first two and will send to the reflector soon. Andreas has text for the test plan, which he sent during the meeting. The TWG will make use of this material as a requirement in our SOWs for development of OpenSFS Lustre features. We plan to make this material available on the OpenSFS website. RFP Process The following are the proposed instructions for processing the RFP responses: Overview: * RFP responses are confidential. * Because the information is business sensitive, review of RFP responses is only open to OpenSFS participants (ie, those organizations that have officially signed up with OpenSFS). * Current organizations eligible for the review are (Cray, IU, LLNL, ORNL, RAID Inc.). DDN and the RFP participants will not participate in the review. To participate in the review, HPCFS members need to contact Norm Morse to initiate due diligence for participation in OpenSFS. * Reviewers may not disclose any materials related to the RFP outside of the TWG-RFP team. * The review team should be relatively small to complete the review expediently. The target is to present the final review to the board within two weeks of starting the process. * Independent reviews should be collected within one week of the start of the review. There will be a twg-rfp group discussion to be held at the end of this week. * After the initial independent review, the review team may ask questions of clarification from those that have submitted proposals. Replies are expected from vendors promptly. * The review team will prepare a final review within one week of the independent review or receipt of any requested proposal clarifications. This document will be submitted to the board for review. * Final reviews will be available to vendors that have submitted proposals. Reviewers are reminded to be critical but fair. Schedule 5/9 - release proposals and evaluation criteria to twg-rfp 5/10 - twg-rfp meets to start review and get commitment from reviewers to participate 5/16 - initial independent reviews due 5/17 - twg-rfp meets to discuss reviews and decide if follow-up questions are needed. 5/23 - draft final review (assuming no questions sent to vendors) 5/24 - twg-rfp meets to approve final review before sending to board adjourn 10:01 From carrier at cray.com Mon May 9 04:55:14 2011 From: carrier at cray.com (John Carrier) Date: Sun, 8 May 2011 23:55:14 -0500 Subject: [Twg] RFP review Message-ID: OpenSFS plans to kick off the RFP review this week on the twg-rfp reflector. If you are interested in participating in the review, please subscribe to the twg-rfp reflector. Because the information is business sensitive, review of RFP responses is only open to OpenSFS participants (ie, those organizations that have officially signed up with OpenSFS). Current organizations eligible for the review are (Cray, IU, LLNL, ORNL, RAID Inc.). DDN and the RFP participants will not participate in the review. Proposed Schedule: 5/9 - release proposals and evaluation criteria to twg-rfp 5/10 - twg-rfp meets to start review and get commitment from reviewers to participate 5/16 - initial independent reviews due 5/17 - twg-rfp meets to discuss reviews and decide if follow-up questions are needed. 5/23 - draft final review (assuming no questions sent to vendors) 5/24 - twg-rfp meets to approve final review before sending to board -------------- next part -------------- An HTML attachment was scrubbed... URL: From carrier at cray.com Mon May 9 05:01:36 2011 From: carrier at cray.com (John Carrier) Date: Mon, 9 May 2011 00:01:36 -0500 Subject: [Twg] release check list Message-ID: We want to create a checklist of items that need to be verified during code reviews of features to be developed in response to OpenSFS RFPs. We hope the list is general enough that it can be generally used as the release collateral for other contributions to the Lustre canonical tree. We are trying to generate a short well-chosen list of key factors rather than a long comprehensive list of items. Examples suggested so far are consistent function and variable naming, valid threading model, consistent locking, impact on wire protocol and disk format compatibility. Please provide descriptions of suggested check list items to discuss at lists.opensfs.org. The completed document will be shared with the community. Thanks, --jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From carrier at cray.com Mon May 9 05:09:05 2011 From: carrier at cray.com (John Carrier) Date: Mon, 9 May 2011 00:09:05 -0500 Subject: [Twg] request for code reviewers Message-ID: OpenSFS has started an RFP process to generate new Lustre features. The new code will be available on an open repository so that it can be inspected by the community. However, we seek recognized experts in each area of Lustre who can commit to reviewing new features when they are available. If you are interested, please reply to discuss at lists.opensfs.org and describe the Lustre subsystem you are willing to review. Below is a list of Lustre development areas as defined in bugzilla to provide a starting point for discussing your area of expertise. Thanks, --jc Lustre Subsystems: Client - LOV Client - LMV Client - MDC Client - obdecho Client - OSC Client - CLIO Client - VFS liblustre Client - VFS Linux Disk Storage - llog Disk Storage - ldiskfs OSD Disk Storage - DMU OSD Documentation Infrastructure - configuration Infrastructure - debugging Infrastructure - libcfs Infrastructure - liblustreapi Infrastructure - obdclass Infrastructure - lprocfs Infrastructure - user tools Infrastructure - Build & Packaging LNET - core LNET - IB(o2iblnd) LNET - ptllnd LNET - routing LNET - self test LNET - TCP(socklnd) Protocol - grant Protocol - Interoperability Protocol - LDLM Protocol - ptlrpc Protocol - quota Protocol - recovery Protocol - security (GSS/Kerberos) Servers - FID/SEQ Servers - Linux kernel Servers - CMD Servers - LOD/OSP Servers - MDT/MDD Servers - obdecho Servers - OST/obdfilter/OFD Testing -------------- next part -------------- An HTML attachment was scrubbed... URL: From carrier at cray.com Thu May 12 18:40:08 2011 From: carrier at cray.com (John Carrier) Date: Thu, 12 May 2011 13:40:08 -0500 Subject: [Twg] TWG meeting minutes for 2011-05-12 In-Reply-To: References: Message-ID: Attached are the meeting minutes from today's TWG call. We discussed my summary of the release collateral discussion we've had the past few meetings: Proposed Release Collateral: - high level design (HLD) document - test plan and results - code following standard Lustre guidelines - complete release checklist - patches sized for easy inspection - review by two independent inspectors See today's minutes for more information. Discussion of these points were mostly covered in minutes from our 4/21 and 4/28 meetings. Please post questions/comments to discuss at lists.opensfs.org. Next meeting is next Thursday (5/19) @ 9:30a PDT/12:30p EDT, (715) 726-4994, ID/password 72090. -------------- next part -------------- OpenSFS Technical Working Group Meeting minutes : 05/12/2011 Concall: start : 9:30a PT, adjourn : 9:55a PT Next meeting: Thursday, 05/17/2011 @ 9:30a PDT/12:30p EDT Dial-in numbers are 715-726-4994 Meeting ID and password are 72090 Attending Name Organization email ----------------- -------------- ---------------------------- Cory Spitz Cray spitzcor at cray.com John Carrier Cray carrier at cray.com Alex Kulyavtsev FNAL aik at fnal.gov Joshua Walgenbach IU jjw at indiana.edu Chris Morrone LLNL morrone2 at llnl.gov Dave Dillow ORNL dillowda at ornl.gov Agenda review collateral proposals Discussion John described an email conversation he had with Jim Ryan and Evan Felix about an industry group trying to identify a neutral repository for Lustre. They are interested in our efforts to define release collateral. John invited them to join our dicussion on discuss at lists.opensfs.org. The following lists summarize the discussions of release collateral from this and our three preceding meetings. Please send suggestions for improving the proposals to discuss at lists.opensfs.org. Proposed Release Collateral --------------------------- - high level design (HLD) document - test plan and results - code following standard Lustre guidelines - complete release checklist - patches sized for easy inspection - review by two independent inspectors Notes: We have asked the community for 1) help defining the release 2) volunteers to review patches We agreed not to ask for a detailed design document (DLD). This is largely of use only to the organization writing the code. After implementation is complete, the code itself is the design document. The HLD, however, must provide sufficient detail to understand how the code is implemented. Tests defined in the test plan must be publically available. John is to send this summary of release collateral to lustre-devel to get feedback from a broader audience. Proposed SOW Requirements ------------------------- - design documents - architecture level design (ALD) - high level design (HLD) - implementation checkpoints - publically available development branch - intermediate milestones to verify functionality - publically available test plans - code to land upstream in a public, neutral repository (TBD) - include all release collateral defined above Notes: The ALD summarizes the requirements and provides an overview of the proposed solution. It is the first opportunity for the Lustre community, outside the RFP review team, to see the feature proposal. The HLD is done after presentation and discussion of the ALD. It will explain the ideas behind the design, an overview of the proposed implementation, and any diagrams needed to clarify the design. There should be no procedure prototypes or data structure definitions. Since OpenSFS is requiring this document before coding starts, we expect the HLD to be updated after the implementation is complete and before inspections begin for release. The SOW needs to define where the testing for the intermediate steps will be done and by whom. If we expect OpenSFS members to run the tests or use their clusters, we need commitments for these resources. adjourn 9:55PT From Nathan_Rutman at xyratex.com Fri May 13 17:18:43 2011 From: Nathan_Rutman at xyratex.com (Nathan Rutman) Date: Fri, 13 May 2011 10:18:43 -0700 Subject: [Twg] [Discuss] request for code reviewers In-Reply-To: References: Message-ID: <73AED5C780AE05478241DB067651A9210237850B@XYUS-EX22.xyus.xyratex.com> On May 8, 2011, at 10:09 PM, John Carrier wrote: OpenSFS has started an RFP process to generate new Lustre features. The new code will be available on an open repository so that it can be inspected by the community. However, we seek recognized experts in each area of Lustre who can commit to reviewing new features when they are available. If you are interested, please reply to discuss at lists.opensfs.org and describe the Lustre subsystem you are willing to review. Below is a list of Lustre development areas as defined in bugzilla to provide a starting point for discussing your area of expertise. Areas I'm interested in: Documentation Infrastructure - configuration Infrastructure - debugging Infrastructure - libcfs Infrastructure - liblustreapi Infrastructure - obdclass Infrastructure - lprocfs Infrastructure - user tools Protocol - Interoperability Protocol - ptlrpc Protocol - recovery Protocol - security (GSS/Kerberos) Servers - FID/SEQ Servers - MDT/MDD Servers - obdecho Servers - OST/obdfilter/OFD Testing ______________________________________________________________________ This email may contain privileged or confidential information, which should only be used for the purpose for which it was sent by Xyratex. No further rights or licenses are granted to use such information. If you are not the intended recipient of this message, please notify the sender by return and delete it. You may not use, copy, disclose or rely on the information contained in it. Internet email is susceptible to data corruption, interception and unauthorised amendment for which Xyratex does not accept liability. While we have taken reasonable precautions to ensure that this email is free of viruses, Xyratex does not accept liability for the presence of any computer viruses in this email, nor for any losses caused as a result of viruses. Xyratex Technology Limited (03134912), Registered in England & Wales, Registered Office, Langstone Road, Havant, Hampshire, PO9 1SA. The Xyratex group of companies also includes, Xyratex Ltd, registered in Bermuda, Xyratex International Inc, registered in California, Xyratex (Malaysia) Sdn Bhd registered in Malaysia, Xyratex Technology (Wuxi) Co Ltd registered in The People's Republic of China and Xyratex Japan Limited registered in Japan. ______________________________________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: From Nathan_Rutman at xyratex.com Mon May 16 18:32:12 2011 From: Nathan_Rutman at xyratex.com (Nathan Rutman) Date: Mon, 16 May 2011 11:32:12 -0700 Subject: [Twg] [Discuss] request for code reviewers In-Reply-To: <73AED5C780AE05478241DB067651A9210237850B@XYUS-EX22.xyus.xyratex.com> References: <73AED5C780AE05478241DB067651A9210237850B@XYUS-EX22.xyus.xyratex.com> Message-ID: <73AED5C780AE05478241DB067651A92102378515@XYUS-EX22.xyus.xyratex.com> A couple more Xyratex inspectors (not on the opensfs mailling list I think): Andrew Perepechko I'd be willing to participate in inspection of Infrastructure - lprocfs Infrastructure - user tools Infrastructure - debugging Infrastructure - libcfs Protocol - quota Servers - Linux kernel Servers - OST/obdfilter/OFD Testing Thank you, Andrew Isaac Huang My list: Infrastructure - libcfs LNET - * Isaac On May 13, 2011, at 10:18 AM, Nathan Rutman wrote: On May 8, 2011, at 10:09 PM, John Carrier wrote: OpenSFS has started an RFP process to generate new Lustre features. The new code will be available on an open repository so that it can be inspected by the community. However, we seek recognized experts in each area of Lustre who can commit to reviewing new features when they are available. If you are interested, please reply to discuss at lists.opensfs.org and describe the Lustre subsystem you are willing to review. Below is a list of Lustre development areas as defined in bugzilla to provide a starting point for discussing your area of expertise. Areas I'm interested in: Documentation Infrastructure - configuration Infrastructure - debugging Infrastructure - libcfs Infrastructure - liblustreapi Infrastructure - obdclass Infrastructure - lprocfs Infrastructure - user tools Protocol - Interoperability Protocol - ptlrpc Protocol - recovery Protocol - security (GSS/Kerberos) Servers - FID/SEQ Servers - MDT/MDD Servers - obdecho Servers - OST/obdfilter/OFD Testing ______________________________________________________________________ This email may contain privileged or confidential information, which should only be used for the purpose for which it was sent by Xyratex. No further rights or licenses are granted to use such information. If you are not the intended recipient of this message, please notify the sender by return and delete it. You may not use, copy, disclose or rely on the information contained in it. Internet email is susceptible to data corruption, interception and unauthorised amendment for which Xyratex does not accept liability. While we have taken reasonable precautions to ensure that this email is free of viruses, Xyratex does not accept liability for the presence of any computer viruses in this email, nor for any losses caused as a result of viruses. Xyratex Technology Limited (03134912), Registered in England & Wales, Registered Office, Langstone Road, Havant, Hampshire, PO9 1SA. The Xyratex group of companies also includes, Xyratex Ltd, registered in Bermuda, Xyratex International Inc, registered in California, Xyratex (Malaysia) Sdn Bhd registered in Malaysia, Xyratex Technology (Wuxi) Co Ltd registered in The People's Republic of China and Xyratex Japan Limited registered in Japan. ______________________________________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: From hamilton5 at llnl.gov Tue May 24 16:57:33 2011 From: hamilton5 at llnl.gov (Hamilton, Pam) Date: Tue, 24 May 2011 09:57:33 -0700 Subject: [Twg] (no subject) Message-ID: Hi all, As I mentioned on the OpenSFS Lustre Community call last week, we are in the process of reforming the former Release Planning and Support Working Groups within OpenSFS into one working group, the Community Development Working Group. This will be an open working group with anyone from the greater Lustre community invited to participate. One purpose of this working group is to start a discussion about a community development model, taking into consideration the needs/concerns of the Lustre vendors. For this to be successful, it's important to have vendor participation. Other topics suggested include how to better marshal test resources for the 2.1 release and facilitating deep dive technical Lustre talks by experienced developers with interested community developers. I'm proposing a bi-weekly conference call on Wednesdays at 9am Pacific starting tomorrow, May 25th. I know this is very late notice for this week's call. We'll see who is able to join this week but really kick things off with more advertisement for the call on June 8th. Tomorrow we can also discuss what to do about a mail list. Maybe we should just use discuss at lists.opensfs.org? Here is the call-in info: 866-914-3976 (925-424-8105) Passcode: 534986# Regards, Pam ___________________________________ Pam Hamilton Lawrence Livermore National Lab P.O. Box 808, L-556 Livermore, CA  94551-9900 E-Mail:  pgh at llnl.gov Phone:  925-423-1332          Fax:  925-423-8719 From hamilton5 at llnl.gov Tue May 24 16:59:44 2011 From: hamilton5 at llnl.gov (Hamilton, Pam) Date: Tue, 24 May 2011 09:59:44 -0700 Subject: [Twg] Announcing the new OpenSFS Community Development Working Group In-Reply-To: References: Message-ID: Hi all, As I mentioned on the OpenSFS Lustre Community call last week, we are in the process of reforming the former Release Planning and Support Working Groups within OpenSFS into one working group, the Community Development Working Group. This will be an open working group with anyone from the greater Lustre community invited to participate. One purpose of this working group is to start a discussion about a community development model, taking into consideration the needs/concerns of the Lustre vendors. For this to be successful, it's important to have vendor participation. Other topics suggested include how to better marshal test resources for the 2.1 release and facilitating deep dive technical Lustre talks by experienced developers with interested community developers. I'm proposing a bi-weekly conference call on Wednesdays at 9am Pacific starting tomorrow, May 25th. I know this is very late notice for this week's call. We'll see who is able to join this week but really kick things off with more advertisement for the call on June 8th. Tomorrow we can also discuss what to do about a mail list. Maybe we should just use discuss at lists.opensfs.org? Here is the call-in info: 866-914-3976 (925-424-8105) Passcode: 534986# Regards, Pam ___________________________________ Pam Hamilton Lawrence Livermore National Lab P.O. Box 808, L-556 Livermore, CA  94551-9900 E-Mail:  pgh at llnl.gov Phone:  925-423-1332          Fax:  925-423-8719 From carrier at cray.com Thu May 26 06:41:19 2011 From: carrier at cray.com (John Carrier) Date: Thu, 26 May 2011 01:41:19 -0500 Subject: [Twg] no TWG meeting 5/26 Message-ID: The TWG meeting for 5/26/2011 is cancelled. The TWG-RFP team needs the time slot to discuss the RFP response. Thanks, --jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From seager1 at llnl.gov Tue May 31 21:42:18 2011 From: seager1 at llnl.gov (Seager, Mark K.) Date: Tue, 31 May 2011 14:42:18 -0700 Subject: [Twg] Canceled: OpenSFS Working Group Cross Fertalization Telecon Message-ID: <6DAF24F9BB747E47A4BAE7A6F5B80C7D0107EA004FA9@NSPEXMBX-A.the-lab.llnl.gov> When: Occurs every 2 weeks on Tuesday effective 11/30/2010 from 9:00 AM to 10:00 AM (UTC-08:00) Pacific Time (US & Canada). Where: 517-308-1709x5282795 Note: The GMT offset above does not reflect daylight saving time adjustments. *~*~*~*~*~*~*~*~*~* Dear OpenSFS working group members. The OpenSFS board suggests that as a reward for your diligent service in the OpenSFS working groups, you should be invited to participate in a OpenSFS working group cross-fertilization telecon. This biweekly telecon will interleave with the OpenSFS byweakly (SP!) community telecom. The purpose of this telecom is to sych up the working groups with the board and vice versa and to discuss coordination, boundary, communications, tools (e.g., mailing lists, wiki’s, web sites, git repositories, document management systems, etc) requirements and other x-wg issues. We will also discuss the white papers and processes to move the working group activities forward. Please plan to attend. Regards, ++Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 7976 bytes Desc: not available URL: