mm: thp: optimize compound_trans_huge
Currently we don't clobber page_tail->first_page during split_huge_page, so compound_trans_head can be set to compound_head without adverse effects, and this mostly optimizes away a smp_rmb. It looks worthwhile to keep around the implementation that doesn't relay on page_tail->first_page not to be clobbered, because it would be necessary if we'll decide to enforce page->private to zero at all times whenever PG_private is not set, also for anonymous pages. For anonymous pages enforcing such an invariant doesn't matter as anonymous pages don't use page->private so we can get away with this microoptimization. Signed-off-by: Andrea Arcangeli <aarcange@redhat.com> Cc: Khalid Aziz <khalid.aziz@oracle.com> Cc: Pravin Shelar <pshelar@nicira.com> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Ben Hutchings <bhutchings@solarflare.com> Cc: Christoph Lameter <cl@linux.com> Cc: Johannes Weiner <jweiner@redhat.com> Cc: Mel Gorman <mgorman@suse.de> Cc: Rik van Riel <riel@redhat.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: Minchan Kim <minchan@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This commit is contained in:
Родитель
ebf360f9bb
Коммит
ca641514f4
|
@ -157,6 +157,26 @@ static inline int hpage_nr_pages(struct page *page)
|
|||
return HPAGE_PMD_NR;
|
||||
return 1;
|
||||
}
|
||||
/*
|
||||
* compound_trans_head() should be used instead of compound_head(),
|
||||
* whenever the "page" passed as parameter could be the tail of a
|
||||
* transparent hugepage that could be undergoing a
|
||||
* __split_huge_page_refcount(). The page structure layout often
|
||||
* changes across releases and it makes extensive use of unions. So if
|
||||
* the page structure layout will change in a way that
|
||||
* page->first_page gets clobbered by __split_huge_page_refcount, the
|
||||
* implementation making use of smp_rmb() will be required.
|
||||
*
|
||||
* Currently we define compound_trans_head as compound_head, because
|
||||
* page->private is in the same union with page->first_page, and
|
||||
* page->private isn't clobbered. However this also means we're
|
||||
* currently leaving dirt into the page->private field of anonymous
|
||||
* pages resulting from a THP split, instead of setting page->private
|
||||
* to zero like for every other page that has PG_private not set. But
|
||||
* anonymous pages don't use page->private so this is not a problem.
|
||||
*/
|
||||
#if 0
|
||||
/* This will be needed if page->private will be clobbered in split_huge_page */
|
||||
static inline struct page *compound_trans_head(struct page *page)
|
||||
{
|
||||
if (PageTail(page)) {
|
||||
|
@ -174,6 +194,9 @@ static inline struct page *compound_trans_head(struct page *page)
|
|||
}
|
||||
return page;
|
||||
}
|
||||
#else
|
||||
#define compound_trans_head(page) compound_head(page)
|
||||
#endif
|
||||
|
||||
extern int do_huge_pmd_numa_page(struct mm_struct *mm, struct vm_area_struct *vma,
|
||||
unsigned long addr, pmd_t pmd, pmd_t *pmdp);
|
||||
|
|
Загрузка…
Ссылка в новой задаче