[Twg] [JIRA] Updated: (LU-50) pdirops patch for ldiskfs

[Nathan Rutman]

Hi Liang -- mdtest doesn't have an option for mknod that I'm aware of; did you patch it?
Would you mind publishing your patch?

On Feb 1, 2012, at 4:53 AM, Liang Zhen (JIRA) wrote:

> 
>     [ http://jira.whamcloud.com/browse/LU-50?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
> 
> Liang Zhen updated LU-50:
> -------------------------
> 
>    Attachment: Opensfs_PDO_performance.pdf
> 
> First round testing results
> 
>> pdirops patch for ldiskfs
>> -------------------------
>> 
>>                Key: LU-50
>>                URL: http://jira.whamcloud.com/browse/LU-50
>>            Project: Lustre
>>         Issue Type: Improvement
>>   Affects Versions: Lustre 2.1.0
>>           Reporter: Liang Zhen
>>           Assignee: Liang Zhen
>>           Priority: Major
>>            Fix For: Lustre 2.2.0
>> 
>>        Attachments: Opensfs_PDO_performance.pdf
>> 
>> 
>> Changing operations of shared directory are not good even with all SMP improvements, I made some efforts to localize changing operations on a few cores to decrease contention and they can somehow help to improvement performance of pdirops (see my comment on LU-55).
>> At the same time, it's worth to try to increase pdirops performance in ldiskfs level. We had a dynlock based pdirops patch for ldiskfs several years ago, but we didn't maintain it. Actually, I found it's hard to maintain it because it's a complex patch and made a lot of changes to htree code of ext3/ext4. So I'm thinking about to have a more simpler implementation of pdirops patch. 
>> The idea is quite simple (just like how we implement pdirop in ldlm level): 
>> * design & implement a advanced lock with multiple modes (CR, PR, CW, PW, EX)
>> * it's just a resource lock resides in osd-ldiskfs & ldiskfs, which will replace current osd_object::oo_ext_idx_sem
>> * we can partially protect resource by locking N hash keys after holding CR & CW
>> ** for CR, sub locking is PR
>> ** for CW, sub locking is PW
>> * for !is_dx(dir), change operations are always holding EX lock on dir, lookup & readdir hold PR lock on dir
>> * for is_dx(dir)
>> ** change operations take CW lock, then take two sub-locks: name-hash as the first key, block number as the second key
>> ** lookup take CR lock, then take two sub-locks: name-hash as the fist key, block number as the second key
>> ** readdir take PR lock (it's not necessary to take PR lock, but we already have PR lock in ldlm level, so it doesn't matter)
>> * if we need to split any blocks (name entries block or indexing block) while have CW lock, we release CW lock and take EX lock and retry.
>> * we can disable pdirops ldiskfs by turning osd-ldiskfs pdir_ldiskfs=0/1, if we disabled pdirops for osd-ldiskfs then osd_ldiskfs always EX lock dir on changing and PR lock dir on lookup/readdir.
>> Although implementation of the lock is a little complex(, but we just need very few changes to ext4/ldiskfs and osd-ldiskfs with this lock: 
>> * osd-ldiskfs: replace oo_ext_idx_sem with the new lock
>> * ldiskfs: switch lock mode if we need to split/append (in ldiskfs).
>> PS: I'm thinking about rename parl_lock to htree_lock, because it's really designed for htree operations, and Eric hates confusing naming like "parl_lock"
> 
> --
> This message is automatically generated by JIRA.
> For more information on JIRA, see: http://www.atlassian.com/software/jira