path: root/arch/arm64/include/asm/pgalloc.h
AgeCommit message (Collapse)AuthorFilesLines
2016-06-24arm64: get rid of superfluous __GFP_REPEATMichal Hocko1-1/+1
__GFP_REPEAT has a rather weak semantic but since it has been introduced around 2.6.12 it has been ignored for low order allocations. {pte,pmd,pud}_alloc_one{_kernel}, late_pgtable_alloc use PGALLOC_GFP for __get_free_page (aka order-0). pgd_alloc is slightly more complex because it allocates from pgd_cache if PGD_SIZE != PAGE_SIZE and PGD_SIZE depends on the configuration (CONFIG_ARM64_VA_BITS, PAGE_SHIFT and CONFIG_PGTABLE_LEVELS). As per config PGTABLE_LEVELS int default 2 if ARM64_16K_PAGES && ARM64_VA_BITS_36 default 2 if ARM64_64K_PAGES && ARM64_VA_BITS_42 default 3 if ARM64_64K_PAGES && ARM64_VA_BITS_48 default 3 if ARM64_4K_PAGES && ARM64_VA_BITS_39 default 3 if ARM64_16K_PAGES && ARM64_VA_BITS_47 default 4 if !ARM64_64K_PAGES && ARM64_VA_BITS_48 we should have the following options CONFIG_ARM64_VA_BITS:48 CONFIG_PGTABLE_LEVELS:4 PAGE_SIZE:4k size:4096 pages:1 CONFIG_ARM64_VA_BITS:48 CONFIG_PGTABLE_LEVELS:4 PAGE_SIZE:16k size:16 pages:1 CONFIG_ARM64_VA_BITS:48 CONFIG_PGTABLE_LEVELS:3 PAGE_SIZE:64k size:512 pages:1 CONFIG_ARM64_VA_BITS:47 CONFIG_PGTABLE_LEVELS:3 PAGE_SIZE:16k size:16384 pages:1 CONFIG_ARM64_VA_BITS:42 CONFIG_PGTABLE_LEVELS:2 PAGE_SIZE:64k size:65536 pages:1 CONFIG_ARM64_VA_BITS:39 CONFIG_PGTABLE_LEVELS:3 PAGE_SIZE:4k size:4096 pages:1 CONFIG_ARM64_VA_BITS:36 CONFIG_PGTABLE_LEVELS:2 PAGE_SIZE:16k size:16384 pages:1 All of them fit into a single page (aka order-0). This means that this flag has never been actually useful here because it has always been used only for PAGE_ALLOC_COSTLY requests. Link: Signed-off-by: Michal Hocko <> Acked-by: Will Deacon <> Cc: Catalin Marinas <> Signed-off-by: Andrew Morton <> Signed-off-by: Linus Torvalds <>
2016-02-16arm64: mm: add __{pud,pgd}_populateMark Rutland1-4/+22
We currently have __pmd_populate for creating a pmd table entry given the physical address of a pte, but don't have equivalents for the pud or pgd levels of table. To enable us to manipulate tables which are mapped outside of the linear mapping (where we have a PA, but not a linear map VA), it is useful to have these functions. This patch adds __{pud,pgd}_populate. As these should not be called when the kernel uses folded {pmd,pud}s, in these cases they expand to BUILD_BUG(). So long as the appropriate checks are made on the {pud,pgd} entry prior to attempting population, these should be optimized out at compile time. Signed-off-by: Mark Rutland <> Reviewed-by: Catalin Marinas <> Tested-by: Ard Biesheuvel <> Reviewed-by: Ard Biesheuvel <> Tested-by: Jeremy Linton <> Cc: Laura Abbott <> Cc: Will Deacon <> Signed-off-by: Catalin Marinas <>
2015-10-12arm64: move PGD_SIZE definition to pgalloc.hAndrey Ryabinin1-0/+1
This will be used by KASAN latter. Signed-off-by: Andrey Ryabinin <> Acked-by: Catalin Marinas <> Signed-off-by: Catalin Marinas <>
2015-04-14arm64: expose number of page table levels on Kconfig levelKirill A. Shutemov1-4/+4
We would want to use number of page table level to define mm_struct. Let's expose it as CONFIG_PGTABLE_LEVELS. ARM64_PGTABLE_LEVELS is renamed to PGTABLE_LEVELS and defined before sourcing init/Kconfig: arch/Kconfig will define default value and it's sourced from init/Kconfig. Signed-off-by: Kirill A. Shutemov <> Acked-by: Catalin Marinas <> Cc: Will Deacon <> Tested-by: Guenter Roeck <> Signed-off-by: Andrew Morton <> Signed-off-by: Linus Torvalds <>
2014-11-20arm64: pgalloc: consistently use PGALLOC_GFPMark Rutland1-4/+4
We currently allocate different levels of page tables with a variety of differing flags, and the PGALLOC_GFP flags, intended for use when allocating any level of page table, are only used for ptes in pte_alloc_one. On x86, PGALLOC_GFP is used for all page table allocations. Currently the major differences are: * __GFP_NOTRACK -- Needed to ensure page tables are always accessible in the presence of kmemcheck to prevent recursive faults. Currently kmemcheck cannot be selected for arm64. * __GFP_REPEAT -- Causes the allocator to try to reclaim pages and retry upon a failure to allocate. * __GFP_ZERO -- Sometimes passed explicitly, sometimes zalloc variants are used. While we've no encountered issues so far, it would be preferable to be consistent. This patch ensures all levels of table are allocated in the same manner, with PGALLOC_GFP. Cc: Steve Capper <> Acked-by: Catalin Marinas <> Signed-off-by: Mark Rutland <> Signed-off-by: Will Deacon <>
2014-07-23arm64: Convert bool ARM64_x_LEVELS to int ARM64_PGTABLE_LEVELSCatalin Marinas1-4/+4
Rather than having several Kconfig options, define int ARM64_PGTABLE_LEVELS which will be also useful in converting some of the pgtable macros. Signed-off-by: Catalin Marinas <> Tested-by: Jungseok Lee <>
2014-07-23arm64: mm: Implement 4 levels of translation tablesJungseok Lee1-0/+20
This patch implements 4 levels of translation tables since 3 levels of page tables with 4KB pages cannot support 40-bit physical address space described in [1] due to the following issue. It is a restriction that kernel logical memory map with 4KB + 3 levels (0xffffffc000000000-0xffffffffffffffff) cannot cover RAM region from 544GB to 1024GB in [1]. Specifically, ARM64 kernel fails to create mapping for this region in map_mem function since __phys_to_virt for this region reaches to address overflow. If SoC design follows the document, [1], over 32GB RAM would be placed from 544GB. Even 64GB system is supposed to use the region from 544GB to 576GB for only 32GB RAM. Naturally, it would reach to enable 4 levels of page tables to avoid hacking __virt_to_phys and __phys_to_virt. However, it is recommended 4 levels of page table should be only enabled if memory map is too sparse or there is about 512GB RAM. References ---------- [1]: Principles of ARM Memory Maps, White Paper, Issue C Signed-off-by: Jungseok Lee <> Reviewed-by: Sungjinn Chung <> Acked-by: Kukjin Kim <> Reviewed-by: Christoffer Dall <> Reviewed-by: Steve Capper <> [ MEMBLOCK_INITIAL_LIMIT removed, same as PUD_SIZE] [ early_ioremap_init() updated for 4 levels] [ 48-bit VA depends on BROKEN until KVM is fixed] Signed-off-by: Catalin Marinas <> Tested-by: Jungseok Lee <>
2014-07-23arm64: Introduce VA_BITS and translation level optionsJungseok Lee1-2/+2
This patch adds virtual address space size and a level of translation tables to kernel configuration. It facilicates introduction of different MMU options, such as 4KB + 4 levels, 16KB + 4 levels and 64KB + 3 levels, easily. The idea is based on the discussion with Catalin Marinas: Signed-off-by: Jungseok Lee <> Reviewed-by: Sungjinn Chung <> Acked-by: Kukjin Kim <> Reviewed-by: Christoffer Dall <> Signed-off-by: Catalin Marinas <> Tested-by: Jungseok Lee <>
2013-11-15arm64: handle pgtable_page_ctor() failKirill A. Shutemov1-3/+6
Signed-off-by: Kirill A. Shutemov <> Cc: Catalin Marinas <> Cc: Will Deacon <> Signed-off-by: Andrew Morton <> Signed-off-by: Linus Torvalds <>
2012-09-17arm64: MMU fault handling and page table managementCatalin Marinas1-0/+113
This patch adds support for the handling of the MMU faults (exception entry code introduced by a previous patch) and page table management. The user translation table is pointed to by TTBR0 and the kernel one (swapper_pg_dir) by TTBR1. There is no translation information shared or address space overlapping between user and kernel page tables. Signed-off-by: Will Deacon <> Signed-off-by: Catalin Marinas <> Acked-by: Tony Lindgren <> Acked-by: Nicolas Pitre <> Acked-by: Olof Johansson <> Acked-by: Santosh Shilimkar <> Acked-by: Arnd Bergmann <>