[Twg] [Discuss] FW: OpenSFS TWG Requi... - Have we achieved the single-server MD...

Tue May 1 20:50:46 UTC 2012

Thanks Andreas for that link.
These look like great improvements certainly - but on an absolute scale: 
is one minute for 100k 0-stripe files sufficient?
4.5 min for 100k 4-stripe files? 
If not, is there a particular number that we think will be acceptable?  Anyone have some GPFS numbers?

For the graph below, are these 0-stripe files?  Liang's original pdirops report didn't show nearly these gains.  Are there further changes is 2.3?

On Apr 30, 2012, at 4:01 PM, Andreas Dilger wrote:

	On 2012-04-30, at 2:48 PM, Nathan Rutman wrote:

		Right, these are different things.  But both were intended to be metadata improvements.

		The pdirops results have been posted, but for actual files (e.g. with OST objects) the create/unlink rates didn't show much improvement.

	I see this as a case of Amdahl's law - the MDS-only performance improvements from pdirops were reasonably good, but this is only part of the system being tested and improving performance of just the MDS will show diminishing returns if the OST performance isn't similarly improved.

	I've long suspected that the MDS-OSS precreate operations could be improved through some in-depth analysis and profiling, and Ben @ Terascala showed some of this at LUG 2011 as well.  Some of the MDS-side performance improvements (pdirops) will be available in the OSS stack once it moves over to using the OFD module (i.e. obdfilter on top of OSD), and some of the current SMP scaling work being done for LNet, libcfs, ptlrpc will also apply to the OSS.

		I have not seen any measurements of 'ls -la' for any delivered features (although that doesn't mean they don't exist).

	I posted some of these results in a recent presentation:
	http://storageconference.com/2012/Presentations/T01.Dilger.pdf

	The graphs on slide 13 were generated based on data from Fan Yong's testing, and I'd think this was delivered to OpenSFS, but I don't know if there was a separate presentation for these results elsewhere.

		The question now is, do we as OpenSFS members want to continue to prioritize single-server MDS performance improvements? 

	I think it would be reasonable to have follow-on project for OSS SMP tuning (including OST precreate, among other things) once the current SMP work for LNET/ptlrpc is finished, along with a similar investment on the client side to ensure that it is not hitting SMP scaling bottlenecks as the number of cores on client nodes goes through the roof.

	Cheers, Andreas

		On Apr 30, 2012, at 12:47 PM, Alex Tomas wrote:

			hmm? pdirops is not supposed to improve directory listing, AFAIK.

			thanks, Alex

			On Mon, Apr 30, 2012 at 11:23 PM, Nathan Rutman <Nathan_Rutman at xyratex.com> wrote:

			pdirops was delivered, but the improvement results were somewhat disappointing.  I've seen no metrics for various directory listing improvements, although they may exist.

			Should we set a target rate? E.g. "ls -al" for a dir with 100K single-stripe files?

			On Apr 30, 2012, at 11:53 AM, John Carrier wrote:

				Nathan adds a good question:  metadata performance was the focus of our last RFP.  Are there any performance targets not covered by our current development contract?

				--jc

				Nathan Rutman added a comment to OpenSFS TWG Requirements 2012

				Nathan Rutman

				Metadata server performance

				Have we achieved the single-server MDS performance goals? Do we need to move some more solid requirements to 2012?

				_______________________________________________

				discuss mailing list

				discuss at lists.opensfs.org

				http://lists.opensfs.org/listinfo.cgi/discuss-opensfs.org

			_______________________________________________

			twg mailing list

			twg at lists.opensfs.org

			http://lists.opensfs.org/listinfo.cgi/twg-opensfs.org

		_______________________________________________

		twg mailing list

		twg at lists.opensfs.org

		http://lists.opensfs.org/listinfo.cgi/twg-opensfs.org

	Cheers, Andreas
	--
	Andreas Dilger                       Whamcloud, Inc.
	Principal Lustre Engineer            http://www.whamcloud.com/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensfs.org/pipermail/twg_lists.opensfs.org/attachments/20120501/9b1901ab/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PastedGraphic-2.tiff
Type: image/tiff
Size: 1076054 bytes
Desc: PastedGraphic-2.tiff
URL: <http://lists.opensfs.org/pipermail/twg_lists.opensfs.org/attachments/20120501/9b1901ab/attachment.tiff>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PastedGraphic-1.tiff
Type: image/tiff
Size: 1009486 bytes
Desc: PastedGraphic-1.tiff
URL: <http://lists.opensfs.org/pipermail/twg_lists.opensfs.org/attachments/20120501/9b1901ab/attachment-0001.tiff>