From carrier at cray.com Thu Nov 1 15:40:32 2012 From: carrier at cray.com (John Carrier) Date: Thu, 1 Nov 2012 15:40:32 +0000 Subject: [Twg] no TWG call today Message-ID: No TWG call today. Apologies for the late notice. --jc -------------- next part -------------- An HTML attachment was scrubbed... URL: From morrone2 at llnl.gov Wed Nov 7 22:04:47 2012 From: morrone2 at llnl.gov (Christopher J. Morrone) Date: Wed, 07 Nov 2012 14:04:47 -0800 Subject: [Twg] RFP status? In-Reply-To: References: Message-ID: <509ADAFF.8000204@llnl.gov> Could we please get an update on what is going on with the RFP? Is it going to be released soon? If it is not going to be released soon, can we set a target date for ourselves? Here's my high-level summary of the process thus far: Nov 2011 (maybe earlier) - Begin planning RFP June 2012 - Completed list of items to include in RFP Nov 2012 - ??? Nov 2013 - Profit :) Chris From gshipman at ornl.gov Wed Nov 7 22:24:06 2012 From: gshipman at ornl.gov (Shipman, Galen M.) Date: Wed, 7 Nov 2012 17:24:06 -0500 Subject: [Twg] RFP status? Message-ID: Thanks for checking on this Chris. I would also like to understand if we are moving forward with the design/scoping RFP as a first stage. Thanks Galen -----Original Message----- From: Christopher J. Morrone [morrone2 at llnl.gov] Sent: Wednesday, November 07, 2012 05:11 PM Eastern Standard Time To: twg at lists.opensfs.org Subject: [Twg] RFP status? Could we please get an update on what is going on with the RFP? Is it going to be released soon? If it is not going to be released soon, can we set a target date for ourselves? Here's my high-level summary of the process thus far: Nov 2011 (maybe earlier) - Begin planning RFP June 2012 - Completed list of items to include in RFP Nov 2012 - ??? Nov 2013 - Profit :) Chris _______________________________________________ twg mailing list twg at lists.opensfs.org http://lists.opensfs.org/listinfo.cgi/twg-opensfs.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From dillowda at ornl.gov Thu Nov 8 16:47:50 2012 From: dillowda at ornl.gov (David Dillow) Date: Thu, 08 Nov 2012 11:47:50 -0500 Subject: [Twg] No call today (11/8) Message-ID: <1352393270.2108.1.camel@frustration.ornl.gov> Sorry for the late notice, thought I sent this out last night. I'll be sending a separate email shortly with the details for the TWG meeting at SC. -- Dave Dillow National Center for Computational Science Oak Ridge National Laboratory (865) 241-6602 office From dillowda at ornl.gov Fri Nov 9 04:18:14 2012 From: dillowda at ornl.gov (David Dillow) Date: Thu, 08 Nov 2012 23:18:14 -0500 Subject: [Twg] TWG Meeting at SC12 Message-ID: <1352434694.28730.6.camel@obelisk.thedillows.org> When: Tuesday November 13th 10am-11am MST Where: Intel/Whamcloud whisper suite (Deer Valley 1) Salt Lake Marriott Downtown @ City Creek Conference bridge: 888-875-9370 or 916-356-2663 Bridge #3 Passcode 5149071 Agenda: Intel Contract update IU Contract update RFP update and planning Other items as suggested (send to list) Much thanks to Intel for providing us the room and the conference bridge! -- Dave Dillow National Center for Computational Science Oak Ridge National Laboratory (865) 241-6602 office From hamilton5 at llnl.gov Fri Nov 9 17:43:42 2012 From: hamilton5 at llnl.gov (Hamilton, Pam) Date: Fri, 9 Nov 2012 17:43:42 +0000 Subject: [Twg] [Discuss] TWG Meeting at SC12 In-Reply-To: <1352434694.28730.6.camel@obelisk.thedillows.org> References: <1352434694.28730.6.camel@obelisk.thedillows.org> Message-ID: Hi Dave and TWG, A suggestion was discussed on a recent CDWG call that I wanted to share with the TWG. It seems there are ideas for improvements to Lustre which can be characterized as small discrete items/tasks. Instead of, or in addition to, putting out an RFP which is large in scope, how about RFPs for these small discrete items/tasks? Presumably, these could be executed much more quickly than trying to prioritize a longer list of items and drafting the subsequent more complicated Statement of Work. The SOW for a single item would certainly be easier to write. I'll plan to join in on your TWG meeting in Salt Lake. Regards, Pam ___________________________________ Pam Hamilton Lawrence Livermore National Lab P.O. Box 808, L-556 Livermore, CA  94551-9900 E-Mail:  pgh at llnl.gov Phone:  925-423-1332          Fax:  925-423-8719 > -----Original Message----- > From: discuss-bounces at lists.opensfs.org [mailto:discuss- > bounces at lists.opensfs.org] On Behalf Of David Dillow > Sent: Thursday, November 08, 2012 8:18 PM > To: twg at lists.opensfs.org > Cc: discuss at lists.opensfs.org > Subject: [Discuss] TWG Meeting at SC12 > > When: > Tuesday November 13th > 10am-11am MST > > Where: > Intel/Whamcloud whisper suite (Deer Valley 1) > Salt Lake Marriott Downtown @ City Creek > > Conference bridge: > 888-875-9370 or 916-356-2663 > Bridge #3 Passcode 5149071 > > Agenda: > Intel Contract update > IU Contract update > RFP update and planning > Other items as suggested (send to list) > > > Much thanks to Intel for providing us the room and the conference > bridge! > -- > Dave Dillow > National Center for Computational Science > Oak Ridge National Laboratory > (865) 241-6602 office > > _______________________________________________ > discuss mailing list > discuss at lists.opensfs.org > http://lists.opensfs.org/listinfo.cgi/discuss-opensfs.org From spitzcor at cray.com Tue Nov 13 17:34:39 2012 From: spitzcor at cray.com (Cory Spitz) Date: Tue, 13 Nov 2012 11:34:39 -0600 Subject: [Twg] TWG Meeting at SC12 In-Reply-To: <1352434694.28730.6.camel@obelisk.thedillows.org> References: <1352434694.28730.6.camel@obelisk.thedillows.org> Message-ID: <50A284AF.6060409@cray.com> TWG folks at SC'12: The phone just went out, can you dial back in please? Thanks, -Cory On 11/08/2012 10:18 PM, David Dillow wrote: > When: > Tuesday November 13th > 10am-11am MST > > Where: > Intel/Whamcloud whisper suite (Deer Valley 1) > Salt Lake Marriott Downtown @ City Creek > > Conference bridge: > 888-875-9370 or 916-356-2663 > Bridge #3 Passcode 5149071 > > Agenda: > Intel Contract update > IU Contract update > RFP update and planning > Other items as suggested (send to list) > > > Much thanks to Intel for providing us the room and the conference > bridge! From nathan_rutman at xyratex.com Tue Nov 13 18:31:53 2012 From: nathan_rutman at xyratex.com (Nathan Rutman) Date: Tue, 13 Nov 2012 10:31:53 -0800 Subject: [Twg] [Discuss] TWG Meeting at SC12 In-Reply-To: <50A284AF.6060409@cray.com> References: <1352434694.28730.6.camel@obelisk.thedillows.org> <50A284AF.6060409@cray.com> Message-ID: Apologies - I was on another call. Is there anything I can contribute post-facto? On Nov 13, 2012, at 9:34 AM, Cory Spitz wrote: > TWG folks at SC'12: > > The phone just went out, can you dial back in please? > > Thanks, > -Cory > > On 11/08/2012 10:18 PM, David Dillow wrote: >> When: >> Tuesday November 13th >> 10am-11am MST >> >> Where: >> Intel/Whamcloud whisper suite (Deer Valley 1) >> Salt Lake Marriott Downtown @ City Creek >> >> Conference bridge: >> 888-875-9370 or 916-356-2663 >> Bridge #3 Passcode 5149071 >> >> Agenda: >> Intel Contract update >> IU Contract update >> RFP update and planning >> Other items as suggested (send to list) >> >> >> Much thanks to Intel for providing us the room and the conference >> bridge! > _______________________________________________ > discuss mailing list > discuss at lists.opensfs.org > http://lists.opensfs.org/listinfo.cgi/discuss-opensfs.org From nathan_rutman at xyratex.com Tue Nov 13 19:38:21 2012 From: nathan_rutman at xyratex.com (Nathan Rutman) Date: Tue, 13 Nov 2012 11:38:21 -0800 Subject: [Twg] [Discuss] TWG Meeting at SC12 In-Reply-To: References: <1352434694.28730.6.camel@obelisk.thedillows.org> <50A284AF.6060409@cray.com> Message-ID: <2235DC0E-B26E-4B67-8A78-0008DABD9A5A@xyratex.com> Hi all - I was told there was some discussion about replacing CLIO. Nikita recently put together a presentation for us not only on what CLIO is but why it was done - please see the beginning of this presentation and page 25. Basically, the design was chosen to support planned features that were never implemented (eg. SNS, p2p caching, etc.). While I understand there are problems with CLIO performance, beware that replacing it may preclude some of these options. -------------- next part -------------- A non-text attachment was scrubbed... Name: clio.pdf Type: application/pdf Size: 478067 bytes Desc: not available URL: -------------- next part -------------- On Nov 13, 2012, at 10:31 AM, Nathan Rutman wrote: > Apologies - I was on another call. > Is there anything I can contribute post-facto? > > On Nov 13, 2012, at 9:34 AM, Cory Spitz wrote: > >> TWG folks at SC'12: >> >> The phone just went out, can you dial back in please? >> >> Thanks, >> -Cory >> >> On 11/08/2012 10:18 PM, David Dillow wrote: >>> When: >>> Tuesday November 13th >>> 10am-11am MST >>> >>> Where: >>> Intel/Whamcloud whisper suite (Deer Valley 1) >>> Salt Lake Marriott Downtown @ City Creek >>> >>> Conference bridge: >>> 888-875-9370 or 916-356-2663 >>> Bridge #3 Passcode 5149071 >>> >>> Agenda: >>> Intel Contract update >>> IU Contract update >>> RFP update and planning >>> Other items as suggested (send to list) >>> >>> >>> Much thanks to Intel for providing us the room and the conference >>> bridge! >> _______________________________________________ >> discuss mailing list >> discuss at lists.opensfs.org >> http://lists.opensfs.org/listinfo.cgi/discuss-opensfs.org > From spitzcor at cray.com Tue Nov 13 20:04:15 2012 From: spitzcor at cray.com (Cory Spitz) Date: Tue, 13 Nov 2012 14:04:15 -0600 Subject: [Twg] [Lustre-discuss] The latest updates around Lustre and open source files systems from OpenSFS and EOFS In-Reply-To: References: Message-ID: <50A2A7BF.3060309@cray.com> Hi, Norman. Thanks for sending out the link to http://www.opensfs.org/events/super-computing-2012. If the presenters have slides, I'd suggest we link them in there. Maybe it would be best if presenters added PDFs to wiki.opensfs.org. Then who can update the www.opensfs.org page to link them in? Thanks, -Cory On 11/05/2012 11:35 PM, Norman Morse wrote: > You headed to Salt Lake City? SC is always the main HPC conference of > the year. OpenSFS and EOFS will be at SC12 Nov 12 - 15th talking open > source and file systems at booth 2101. We've had a really busy year with > some great progress around Lustre development in particular! Also some > new important participants have joined. Lots of great momentum, come by > and find out the latest! > > At SC12, we've got great talks at our booth Monday evening at 7:30 for > starters and our popular popcorn and beer reception wil be Tuesday > evening from 4 to 6. And we've got talks at the booth on Tuesday and > Wednesday covering the Lustre roadmap, Lustre WAN issues, Sequoia topics > and tons more. OpenSFS/EOFS Participants DataDirect Networks, Indiana > University, Lawrence Livermore National Laboratory (LNLL), NetApp, > University of Florida, Whamcloud/Intel, and Xyratex will all be talking > open source file systems. > > We'll have an 'unofficial' Birds of a Feather (BOF) session Wednesday > evening the 14th. Please stop by our booth for time, location and agenda. > > The updates from the OpenSFS Community Development Working Group (CDWG) > and the OpenSFS Technical Working Group (TWG) in particular will be of > interest to open source file system technologists and users. > > Mon, 7:30pm, Pam Hamilton, OpenSFS - "OpenSFS Community Development > Working Group – Bringing the Lustre Community Together" > > Mon, 8:00pm, Dave Dillow, OpenSFS - "OpenSFS Technical Working Group > (TWG) 2012 Goals and Accomplishments" > > Full OpenSFS/EOFS Participant talks schedule: > http://www.opensfs.org/events-2/supercomputing2012 > > Come by and talk open source file systems! > > With best regards, > > Norm (OpenSFS), Hugo (EOFS) > > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss From andreas.dilger at intel.com Wed Nov 14 04:01:25 2012 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Wed, 14 Nov 2012 04:01:25 +0000 Subject: [Twg] [Discuss] TWG Meeting at SC12 In-Reply-To: <2235DC0E-B26E-4B67-8A78-0008DABD9A5A@xyratex.com> References: <1352434694.28730.6.camel@obelisk.thedillows.org> <50A284AF.6060409@cray.com> , <2235DC0E-B26E-4B67-8A78-0008DABD9A5A@xyratex.com> Message-ID: Ironically, while one of the original reasons for CLIO was to reduce bugs and improve performance of the "current" IO code (i.e. 1.8). However, the complexity of CLIO has shown it to be just the opposite. Even your recent postings of "defect escapes" shows that CLIO is the largest source of bugs in the code, and the performance is still not matching that of the 1.8 client (though LU-744 is working to address this). Another consideration is the abstraction is very heavyweight if we are trying to get the client into the kernel. That said, this was just a discussion about potential technical debt that needs to be addressed. There is not enough information to know whether the right solution is to fix CLIO, reduce the complexity of CLIO to be more understandable and efficient, or to revert to the 1.8 IO stack. Cheers, Andreas On 2012-11-13, at 12:42, "Nathan Rutman" wrote: > Hi all - > I was told there was some discussion about replacing CLIO. Nikita recently put together a presentation for us not only on what CLIO is but why it was done - please see the beginning of this presentation and page 25. Basically, the design was chosen to support planned features that were never implemented (eg. SNS, p2p caching, etc.). > While I understand there are problems with CLIO performance, beware that replacing it may preclude some of these options. > > > > On Nov 13, 2012, at 10:31 AM, Nathan Rutman wrote: > >> Apologies - I was on another call. >> Is there anything I can contribute post-facto? >> >> On Nov 13, 2012, at 9:34 AM, Cory Spitz wrote: >> >>> TWG folks at SC'12: >>> >>> The phone just went out, can you dial back in please? >>> >>> Thanks, >>> -Cory >>> >>> On 11/08/2012 10:18 PM, David Dillow wrote: >>>> When: >>>> Tuesday November 13th >>>> 10am-11am MST >>>> >>>> Where: >>>> Intel/Whamcloud whisper suite (Deer Valley 1) >>>> Salt Lake Marriott Downtown @ City Creek >>>> >>>> Conference bridge: >>>> 888-875-9370 or 916-356-2663 >>>> Bridge #3 Passcode 5149071 >>>> >>>> Agenda: >>>> Intel Contract update >>>> IU Contract update >>>> RFP update and planning >>>> Other items as suggested (send to list) >>>> >>>> >>>> Much thanks to Intel for providing us the room and the conference >>>> bridge! >>> _______________________________________________ >>> discuss mailing list >>> discuss at lists.opensfs.org >>> http://lists.opensfs.org/listinfo.cgi/discuss-opensfs.org > > _______________________________________________ > twg mailing list > twg at lists.opensfs.org > http://lists.opensfs.org/listinfo.cgi/twg-opensfs.org From gshipman at ornl.gov Sun Nov 25 16:46:01 2012 From: gshipman at ornl.gov (Shipman, Galen M.) Date: Sun, 25 Nov 2012 11:46:01 -0500 Subject: [Twg] Next OpenSFS Development RFP - call for RFP team members Message-ID: <71087BB8-359D-4AE3-B43F-A77F91793C3E@ornl.gov> TWG, As discussed at our TWG meeting at SC I took the action item of creating a draft RFP to get things started for our next round of development, I have completed a draft and am ready to send this out to the RFP team. Our next step is to iterate on this RFP with the "RFP team members" but to do this I need a list of individuals who would like to participate in this process. To avoid any conflict of interest we should keep this part of the process limited to those who do not plan on responding to the RFP. To help organizations consider if they may respond to the RFP, the areas we expect to be covered are: Lustre feature development Parallel file system tools Lustre documentation Open source parallel file system incubators Send me a quick note if you wish to be on the RFP team and I will get the twg-rfp list updated with the new team members. Thanks, Galen From carrier at cray.com Thu Nov 29 15:51:10 2012 From: carrier at cray.com (John Carrier) Date: Thu, 29 Nov 2012 15:51:10 +0000 Subject: [Twg] no TWG meeting today (11/29/12) Message-ID: Galen is leading the TWG-RFP team to create the RFP document. There is no other TWG business to discuss, so our meeting today is canceled. If you are interested in helping prepare and evaluate the RFP and your company is an OpenSFS member that will not be responding to the RFP, then please contact Galen to get involved. --jc -------------- next part -------------- An HTML attachment was scrubbed... URL: