[Twg] [JIRA] Updated: (LU-50) pdirops patch for ldiskfs

[Liang Zhen]

Nathan, I've posted a simple patch on the Jira ticket which is just for tests. Yes, 0-stripe is using O_LOV_DELAY_CREATE flag of Lustre and creating file without OST object.

Regards
Liang

On Feb 2, 2012, at 8:16 AM, Nathan Rutman wrote:

> Also, I didn't understand what you meant by the "0-stripe" tests.  Were these tests with no ost objects somehow?
> 
> 
> On Feb 1, 2012, at 10:13 AM, Nathan Rutman wrote:
> 
>> 
>> Hi Liang -- mdtest doesn't have an option for mknod that I'm aware of; did you patch it?
>> Would you mind publishing your patch?
>> 
>> On Feb 1, 2012, at 4:53 AM, Liang Zhen (JIRA) wrote:
>> 
>>> 
>>>   [ http://jira.whamcloud.com/browse/LU-50?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
>>> 
>>> Liang Zhen updated LU-50:
>>> -------------------------
>>> 
>>>  Attachment: Opensfs_PDO_performance.pdf
>>> 
>>> First round testing results
>>> 
>>>> pdirops patch for ldiskfs
>>>> -------------------------
>>>> 
>>>>              Key: LU-50
>>>>              URL: http://jira.whamcloud.com/browse/LU-50
>>>>          Project: Lustre
>>>>       Issue Type: Improvement
>>>> Affects Versions: Lustre 2.1.0
>>>>         Reporter: Liang Zhen
>>>>         Assignee: Liang Zhen
>>>>         Priority: Major
>>>>          Fix For: Lustre 2.2.0
>>>> 
>>>>      Attachments: Opensfs_PDO_performance.pdf
>>>> 
>>>> 
>>>> Changing operations of shared directory are not good even with all SMP improvements, I made some efforts to localize changing operations on a few cores to decrease contention and they can somehow help to improvement performance of pdirops (see my comment on LU-55).
>>>> At the same time, it's worth to try to increase pdirops performance in ldiskfs level. We had a dynlock based pdirops patch for ldiskfs several years ago, but we didn't maintain it. Actually, I found it's hard to maintain it because it's a complex patch and made a lot of changes to htree code of ext3/ext4. So I'm thinking about to have a more simpler implementation of pdirops patch. 
>>>> The idea is quite simple (just like how we implement pdirop in ldlm level): 
>>>> * design & implement a advanced lock with multiple modes (CR, PR, CW, PW, EX)
>>>> * it's just a resource lock resides in osd-ldiskfs & ldiskfs, which will replace current osd_object::oo_ext_idx_sem
>>>> * we can partially protect resource by locking N hash keys after holding CR & CW
>>>> ** for CR, sub locking is PR
>>>> ** for CW, sub locking is PW
>>>> * for !is_dx(dir), change operations are always holding EX lock on dir, lookup & readdir hold PR lock on dir
>>>> * for is_dx(dir)
>>>> ** change operations take CW lock, then take two sub-locks: name-hash as the first key, block number as the second key
>>>> ** lookup take CR lock, then take two sub-locks: name-hash as the fist key, block number as the second key
>>>> ** readdir take PR lock (it's not necessary to take PR lock, but we already have PR lock in ldlm level, so it doesn't matter)
>>>> * if we need to split any blocks (name entries block or indexing block) while have CW lock, we release CW lock and take EX lock and retry.
>>>> * we can disable pdirops ldiskfs by turning osd-ldiskfs pdir_ldiskfs=0/1, if we disabled pdirops for osd-ldiskfs then osd_ldiskfs always EX lock dir on changing and PR lock dir on lookup/readdir.
>>>> Although implementation of the lock is a little complex(, but we just need very few changes to ext4/ldiskfs and osd-ldiskfs with this lock: 
>>>> * osd-ldiskfs: replace oo_ext_idx_sem with the new lock
>>>> * ldiskfs: switch lock mode if we need to split/append (in ldiskfs).
>>>> PS: I'm thinking about rename parl_lock to htree_lock, because it's really designed for htree operations, and Eric hates confusing naming like "parl_lock"
>>> 
>>> --
>>> This message is automatically generated by JIRA.
>>> For more information on JIRA, see: http://www.atlassian.com/software/jira
>>