davidlt/kernel-ark
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
os-build
kernel-ark/arch/riscv/kernel/head.S

443 lines
8.6 KiB
ArmAsm
Raw Permalink Normal View History

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 286 Based on 1 normalized pattern(s): this program is free software you can redistribute it and or modify it under the terms of the gnu general public license as published by the free software foundation version 2 this program is distributed in the hope that it will be useful but without any warranty without even the implied warranty of merchantability or fitness for a particular purpose see the gnu general public license for more details extracted by the scancode license scanner the SPDX license identifier GPL-2.0-only has been chosen to replace the boilerplate/reference in 97 file(s). Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Allison Randal <allison@lohutok.net> Reviewed-by: Alexios Zavras <alexios.zavras@intel.com> Cc: linux-spdx@vger.kernel.org Link: https://lkml.kernel.org/r/20190529141901.025053186@linutronix.de Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-05-29 14:18:00 +00:00
/* SPDX-License-Identifier: GPL-2.0-only */
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 01:00:26 +00:00
/*
* Copyright (C) 2012 Regents of the University of California
*/
#include <asm/asm-offsets.h>
#include <asm/asm.h>
#include <linux/init.h>
#include <linux/linkage.h>
#include <asm/thread_info.h>
#include <asm/page.h>
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 06:35:14 +00:00
#include <asm/pgtable.h>
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 01:00:26 +00:00
#include <asm/csr.h>
riscv: clear the instruction cache and all registers when booting When we get booted we want a clear slate without any leaks from previous supervisors or the firmware. Flush the instruction cache and then clear all registers to known good values. This is really important for the upcoming nommu support that runs on M-mode, but can't really harm when running in S-mode either. Vaguely based on the concepts from opensbi. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 12:10:40 +00:00
#include <asm/hwcap.h>
RISC-V: Add an Image header that boot loader can parse. Currently, the last stage boot loaders such as U-Boot can accept only uImage which is an unnecessary additional step in automating boot process. Add an image header that boot loader understands and boot Linux from flat Image directly. This header is based on ARM64 boot image header and provides an opportunity to combine both ARM64 & RISC-V image headers in future. Also make sure that PE/COFF header can co-exist in the same image so that EFI stub can be supported for RISC-V in future. EFI specification needs PE/COFF image header in the beginning of the kernel image in order to load it as an EFI application. In order to support EFI stub, code0 should be replaced with "MZ" magic string and res4(at offset 0x3c) should point to the rest of the PE/COFF header (which will be added during EFI support). Tested on both QEMU and HiFive Unleashed using OpenSBI + U-Boot + Linux. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Karsten Merker <merker@debian.org> Tested-by: Karsten Merker <merker@debian.org> (QEMU+OpenSBI+U-Boot) Tested-by: Kevin Hilman <khilman@baylibre.com> (OpenSBI + U-Boot + Linux) [paul.walmsley@sifive.com: fixed whitespace in boot-image-header.txt; converted structure comment to kernel-doc format and added some detail] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-06-06 23:08:00 +00:00
#include <asm/image.h>
RISC-V: Add PE/COFF header for EFI stub Linux kernel Image can appear as an EFI application With appropriate PE/COFF header fields in the beginning of the Image header. An EFI application loader can directly load a Linux kernel Image and an EFI stub residing in kernel can boot Linux kernel directly. Add the necessary PE/COFF header. Signed-off-by: Atish Patra <atish.patra@wdc.com> Link: https://lore.kernel.org/r/20200421033336.9663-3-atish.patra@wdc.com [ardb: - use C prefix for c.li to ensure the expected opcode is emitted - align all image sections according to PE/COFF section alignment ] Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-09-17 22:37:13 +00:00
#include "efi-header.S"
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 01:00:26 +00:00
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 06:35:14 +00:00
#ifdef CONFIG_XIP_KERNEL
.macro XIP_FIXUP_OFFSET reg
REG_L t0, _xip_fixup
add \reg, \reg, t0
.endm
_xip_fixup: .dword CONFIG_PHYS_RAM_BASE - CONFIG_XIP_PHYS_ADDR - XIP_OFFSET
#else
.macro XIP_FIXUP_OFFSET reg
.endm
#endif /* CONFIG_XIP_KERNEL */
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-18 01:11:39 +00:00
__HEAD
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 01:00:26 +00:00
ENTRY(_start)
RISC-V: Add an Image header that boot loader can parse. Currently, the last stage boot loaders such as U-Boot can accept only uImage which is an unnecessary additional step in automating boot process. Add an image header that boot loader understands and boot Linux from flat Image directly. This header is based on ARM64 boot image header and provides an opportunity to combine both ARM64 & RISC-V image headers in future. Also make sure that PE/COFF header can co-exist in the same image so that EFI stub can be supported for RISC-V in future. EFI specification needs PE/COFF image header in the beginning of the kernel image in order to load it as an EFI application. In order to support EFI stub, code0 should be replaced with "MZ" magic string and res4(at offset 0x3c) should point to the rest of the PE/COFF header (which will be added during EFI support). Tested on both QEMU and HiFive Unleashed using OpenSBI + U-Boot + Linux. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Karsten Merker <merker@debian.org> Tested-by: Karsten Merker <merker@debian.org> (QEMU+OpenSBI+U-Boot) Tested-by: Kevin Hilman <khilman@baylibre.com> (OpenSBI + U-Boot + Linux) [paul.walmsley@sifive.com: fixed whitespace in boot-image-header.txt; converted structure comment to kernel-doc format and added some detail] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-06-06 23:08:00 +00:00
/*
* Image header expected by Linux boot-loaders. The image header data
* structure is described in asm/image.h.
* Do not modify it without modifying the structure and all bootloaders
* that expects this header format!!
*/
RISC-V: Add PE/COFF header for EFI stub Linux kernel Image can appear as an EFI application With appropriate PE/COFF header fields in the beginning of the Image header. An EFI application loader can directly load a Linux kernel Image and an EFI stub residing in kernel can boot Linux kernel directly. Add the necessary PE/COFF header. Signed-off-by: Atish Patra <atish.patra@wdc.com> Link: https://lore.kernel.org/r/20200421033336.9663-3-atish.patra@wdc.com [ardb: - use C prefix for c.li to ensure the expected opcode is emitted - align all image sections according to PE/COFF section alignment ] Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-09-17 22:37:13 +00:00
#ifdef CONFIG_EFI
/*
* This instruction decodes to "MZ" ASCII required by UEFI.
*/
c.li s4,-13
j _start_kernel
#else
RISC-V: Add an Image header that boot loader can parse. Currently, the last stage boot loaders such as U-Boot can accept only uImage which is an unnecessary additional step in automating boot process. Add an image header that boot loader understands and boot Linux from flat Image directly. This header is based on ARM64 boot image header and provides an opportunity to combine both ARM64 & RISC-V image headers in future. Also make sure that PE/COFF header can co-exist in the same image so that EFI stub can be supported for RISC-V in future. EFI specification needs PE/COFF image header in the beginning of the kernel image in order to load it as an EFI application. In order to support EFI stub, code0 should be replaced with "MZ" magic string and res4(at offset 0x3c) should point to the rest of the PE/COFF header (which will be added during EFI support). Tested on both QEMU and HiFive Unleashed using OpenSBI + U-Boot + Linux. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Karsten Merker <merker@debian.org> Tested-by: Karsten Merker <merker@debian.org> (QEMU+OpenSBI+U-Boot) Tested-by: Kevin Hilman <khilman@baylibre.com> (OpenSBI + U-Boot + Linux) [paul.walmsley@sifive.com: fixed whitespace in boot-image-header.txt; converted structure comment to kernel-doc format and added some detail] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-06-06 23:08:00 +00:00
/* jump to start kernel */
j _start_kernel
/* reserved */
.word 0
RISC-V: Add PE/COFF header for EFI stub Linux kernel Image can appear as an EFI application With appropriate PE/COFF header fields in the beginning of the Image header. An EFI application loader can directly load a Linux kernel Image and an EFI stub residing in kernel can boot Linux kernel directly. Add the necessary PE/COFF header. Signed-off-by: Atish Patra <atish.patra@wdc.com> Link: https://lore.kernel.org/r/20200421033336.9663-3-atish.patra@wdc.com [ardb: - use C prefix for c.li to ensure the expected opcode is emitted - align all image sections according to PE/COFF section alignment ] Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-09-17 22:37:13 +00:00
#endif
RISC-V: Add an Image header that boot loader can parse. Currently, the last stage boot loaders such as U-Boot can accept only uImage which is an unnecessary additional step in automating boot process. Add an image header that boot loader understands and boot Linux from flat Image directly. This header is based on ARM64 boot image header and provides an opportunity to combine both ARM64 & RISC-V image headers in future. Also make sure that PE/COFF header can co-exist in the same image so that EFI stub can be supported for RISC-V in future. EFI specification needs PE/COFF image header in the beginning of the kernel image in order to load it as an EFI application. In order to support EFI stub, code0 should be replaced with "MZ" magic string and res4(at offset 0x3c) should point to the rest of the PE/COFF header (which will be added during EFI support). Tested on both QEMU and HiFive Unleashed using OpenSBI + U-Boot + Linux. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Karsten Merker <merker@debian.org> Tested-by: Karsten Merker <merker@debian.org> (QEMU+OpenSBI+U-Boot) Tested-by: Kevin Hilman <khilman@baylibre.com> (OpenSBI + U-Boot + Linux) [paul.walmsley@sifive.com: fixed whitespace in boot-image-header.txt; converted structure comment to kernel-doc format and added some detail] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-06-06 23:08:00 +00:00
.balign 8
riscv: Set text_offset correctly for M-Mode M-Mode Linux is loaded at the start of RAM, not 2MB later. Perhaps this should be calculated based on PAGE_OFFSET somehow? Even better would be to deprecate text_offset and instead introduce something absolute. Signed-off-by: Sean Anderson <seanga2@gmail.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-10-22 20:30:12 +00:00
#ifdef CONFIG_RISCV_M_MODE
/* Image load offset (0MB) from start of RAM for M-mode */
.dword 0
#else
RISC-V: Add an Image header that boot loader can parse. Currently, the last stage boot loaders such as U-Boot can accept only uImage which is an unnecessary additional step in automating boot process. Add an image header that boot loader understands and boot Linux from flat Image directly. This header is based on ARM64 boot image header and provides an opportunity to combine both ARM64 & RISC-V image headers in future. Also make sure that PE/COFF header can co-exist in the same image so that EFI stub can be supported for RISC-V in future. EFI specification needs PE/COFF image header in the beginning of the kernel image in order to load it as an EFI application. In order to support EFI stub, code0 should be replaced with "MZ" magic string and res4(at offset 0x3c) should point to the rest of the PE/COFF header (which will be added during EFI support). Tested on both QEMU and HiFive Unleashed using OpenSBI + U-Boot + Linux. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Karsten Merker <merker@debian.org> Tested-by: Karsten Merker <merker@debian.org> (QEMU+OpenSBI+U-Boot) Tested-by: Kevin Hilman <khilman@baylibre.com> (OpenSBI + U-Boot + Linux) [paul.walmsley@sifive.com: fixed whitespace in boot-image-header.txt; converted structure comment to kernel-doc format and added some detail] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-06-06 23:08:00 +00:00
#if __riscv_xlen == 64
/* Image load offset(2MB) from start of RAM */
.dword 0x200000
#else
/* Image load offset(4MB) from start of RAM */
.dword 0x400000
riscv: Set text_offset correctly for M-Mode M-Mode Linux is loaded at the start of RAM, not 2MB later. Perhaps this should be calculated based on PAGE_OFFSET somehow? Even better would be to deprecate text_offset and instead introduce something absolute. Signed-off-by: Sean Anderson <seanga2@gmail.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-10-22 20:30:12 +00:00
#endif
RISC-V: Add an Image header that boot loader can parse. Currently, the last stage boot loaders such as U-Boot can accept only uImage which is an unnecessary additional step in automating boot process. Add an image header that boot loader understands and boot Linux from flat Image directly. This header is based on ARM64 boot image header and provides an opportunity to combine both ARM64 & RISC-V image headers in future. Also make sure that PE/COFF header can co-exist in the same image so that EFI stub can be supported for RISC-V in future. EFI specification needs PE/COFF image header in the beginning of the kernel image in order to load it as an EFI application. In order to support EFI stub, code0 should be replaced with "MZ" magic string and res4(at offset 0x3c) should point to the rest of the PE/COFF header (which will be added during EFI support). Tested on both QEMU and HiFive Unleashed using OpenSBI + U-Boot + Linux. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Karsten Merker <merker@debian.org> Tested-by: Karsten Merker <merker@debian.org> (QEMU+OpenSBI+U-Boot) Tested-by: Kevin Hilman <khilman@baylibre.com> (OpenSBI + U-Boot + Linux) [paul.walmsley@sifive.com: fixed whitespace in boot-image-header.txt; converted structure comment to kernel-doc format and added some detail] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-06-06 23:08:00 +00:00
#endif
/* Effective size of kernel image */
.dword _end - _start
.dword __HEAD_FLAGS
.word RISCV_HEADER_VERSION
.word 0
.dword 0
riscv: modify the Image header to improve compatibility with the ARM64 header Part of the intention during the definition of the RISC-V kernel image header was to lay the groundwork for a future merge with the ARM64 image header. One error during my original review was not noticing that the RISC-V header's "magic" field was at a different size and position than the ARM64's "magic" field. If the existing ARM64 Image header parsing code were to attempt to parse an existing RISC-V kernel image header format, it would see a magic number 0. This is undesirable, since it's our intention to align as closely as possible with the ARM64 header format. Another problem was that the original "res3" field was not being initialized correctly to zero. Address these issues by creating a 32-bit "magic2" field in the RISC-V header which matches the ARM64 "magic" field. RISC-V binaries will store "RSC\x05" in this field. The intention is that the use of the existing 64-bit "magic" field in the RISC-V header will be deprecated over time. Increment the minor version number of the file format to indicate this change, and update the documentation accordingly. Fix the assembler directives in head.S to ensure that reserved fields are properly zero-initialized. Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com> Reported-by: Palmer Dabbelt <palmer@sifive.com> Reviewed-by: Palmer Dabbelt <palmer@sifive.com> Cc: Atish Patra <atish.patra@wdc.com> Cc: Karsten Merker <merker@debian.org> Link: https://lore.kernel.org/linux-riscv/194c2f10c9806720623430dbf0cc59a965e50448.camel@wdc.com/T/#u Link: https://lore.kernel.org/linux-riscv/mhng-755b14c4-8f35-4079-a7ff-e421fd1b02bc@palmer-si-x1e/T/#t
2019-09-14 01:35:50 +00:00
.ascii RISCV_IMAGE_MAGIC
RISC-V: Add an Image header that boot loader can parse. Currently, the last stage boot loaders such as U-Boot can accept only uImage which is an unnecessary additional step in automating boot process. Add an image header that boot loader understands and boot Linux from flat Image directly. This header is based on ARM64 boot image header and provides an opportunity to combine both ARM64 & RISC-V image headers in future. Also make sure that PE/COFF header can co-exist in the same image so that EFI stub can be supported for RISC-V in future. EFI specification needs PE/COFF image header in the beginning of the kernel image in order to load it as an EFI application. In order to support EFI stub, code0 should be replaced with "MZ" magic string and res4(at offset 0x3c) should point to the rest of the PE/COFF header (which will be added during EFI support). Tested on both QEMU and HiFive Unleashed using OpenSBI + U-Boot + Linux. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Karsten Merker <merker@debian.org> Tested-by: Karsten Merker <merker@debian.org> (QEMU+OpenSBI+U-Boot) Tested-by: Kevin Hilman <khilman@baylibre.com> (OpenSBI + U-Boot + Linux) [paul.walmsley@sifive.com: fixed whitespace in boot-image-header.txt; converted structure comment to kernel-doc format and added some detail] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-06-06 23:08:00 +00:00
.balign 4
riscv: modify the Image header to improve compatibility with the ARM64 header Part of the intention during the definition of the RISC-V kernel image header was to lay the groundwork for a future merge with the ARM64 image header. One error during my original review was not noticing that the RISC-V header's "magic" field was at a different size and position than the ARM64's "magic" field. If the existing ARM64 Image header parsing code were to attempt to parse an existing RISC-V kernel image header format, it would see a magic number 0. This is undesirable, since it's our intention to align as closely as possible with the ARM64 header format. Another problem was that the original "res3" field was not being initialized correctly to zero. Address these issues by creating a 32-bit "magic2" field in the RISC-V header which matches the ARM64 "magic" field. RISC-V binaries will store "RSC\x05" in this field. The intention is that the use of the existing 64-bit "magic" field in the RISC-V header will be deprecated over time. Increment the minor version number of the file format to indicate this change, and update the documentation accordingly. Fix the assembler directives in head.S to ensure that reserved fields are properly zero-initialized. Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com> Reported-by: Palmer Dabbelt <palmer@sifive.com> Reviewed-by: Palmer Dabbelt <palmer@sifive.com> Cc: Atish Patra <atish.patra@wdc.com> Cc: Karsten Merker <merker@debian.org> Link: https://lore.kernel.org/linux-riscv/194c2f10c9806720623430dbf0cc59a965e50448.camel@wdc.com/T/#u Link: https://lore.kernel.org/linux-riscv/mhng-755b14c4-8f35-4079-a7ff-e421fd1b02bc@palmer-si-x1e/T/#t
2019-09-14 01:35:50 +00:00
.ascii RISCV_IMAGE_MAGIC2
RISC-V: Add PE/COFF header for EFI stub Linux kernel Image can appear as an EFI application With appropriate PE/COFF header fields in the beginning of the Image header. An EFI application loader can directly load a Linux kernel Image and an EFI stub residing in kernel can boot Linux kernel directly. Add the necessary PE/COFF header. Signed-off-by: Atish Patra <atish.patra@wdc.com> Link: https://lore.kernel.org/r/20200421033336.9663-3-atish.patra@wdc.com [ardb: - use C prefix for c.li to ensure the expected opcode is emitted - align all image sections according to PE/COFF section alignment ] Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-09-17 22:37:13 +00:00
#ifdef CONFIG_EFI
.word pe_head_start - _start
pe_head_start:
__EFI_PE_HEADER
#else
RISC-V: Add an Image header that boot loader can parse. Currently, the last stage boot loaders such as U-Boot can accept only uImage which is an unnecessary additional step in automating boot process. Add an image header that boot loader understands and boot Linux from flat Image directly. This header is based on ARM64 boot image header and provides an opportunity to combine both ARM64 & RISC-V image headers in future. Also make sure that PE/COFF header can co-exist in the same image so that EFI stub can be supported for RISC-V in future. EFI specification needs PE/COFF image header in the beginning of the kernel image in order to load it as an EFI application. In order to support EFI stub, code0 should be replaced with "MZ" magic string and res4(at offset 0x3c) should point to the rest of the PE/COFF header (which will be added during EFI support). Tested on both QEMU and HiFive Unleashed using OpenSBI + U-Boot + Linux. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Karsten Merker <merker@debian.org> Tested-by: Karsten Merker <merker@debian.org> (QEMU+OpenSBI+U-Boot) Tested-by: Kevin Hilman <khilman@baylibre.com> (OpenSBI + U-Boot + Linux) [paul.walmsley@sifive.com: fixed whitespace in boot-image-header.txt; converted structure comment to kernel-doc format and added some detail] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-06-06 23:08:00 +00:00
.word 0
RISC-V: Add PE/COFF header for EFI stub Linux kernel Image can appear as an EFI application With appropriate PE/COFF header fields in the beginning of the Image header. An EFI application loader can directly load a Linux kernel Image and an EFI stub residing in kernel can boot Linux kernel directly. Add the necessary PE/COFF header. Signed-off-by: Atish Patra <atish.patra@wdc.com> Link: https://lore.kernel.org/r/20200421033336.9663-3-atish.patra@wdc.com [ardb: - use C prefix for c.li to ensure the expected opcode is emitted - align all image sections according to PE/COFF section alignment ] Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-09-17 22:37:13 +00:00
#endif
RISC-V: Add an Image header that boot loader can parse. Currently, the last stage boot loaders such as U-Boot can accept only uImage which is an unnecessary additional step in automating boot process. Add an image header that boot loader understands and boot Linux from flat Image directly. This header is based on ARM64 boot image header and provides an opportunity to combine both ARM64 & RISC-V image headers in future. Also make sure that PE/COFF header can co-exist in the same image so that EFI stub can be supported for RISC-V in future. EFI specification needs PE/COFF image header in the beginning of the kernel image in order to load it as an EFI application. In order to support EFI stub, code0 should be replaced with "MZ" magic string and res4(at offset 0x3c) should point to the rest of the PE/COFF header (which will be added during EFI support). Tested on both QEMU and HiFive Unleashed using OpenSBI + U-Boot + Linux. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Karsten Merker <merker@debian.org> Tested-by: Karsten Merker <merker@debian.org> (QEMU+OpenSBI+U-Boot) Tested-by: Kevin Hilman <khilman@baylibre.com> (OpenSBI + U-Boot + Linux) [paul.walmsley@sifive.com: fixed whitespace in boot-image-header.txt; converted structure comment to kernel-doc format and added some detail] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-06-06 23:08:00 +00:00
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-18 01:11:39 +00:00
.align 2
#ifdef CONFIG_MMU
relocate:
/* Relocate return address */
riscv: Introduce structure that group all variables regarding kernel mapping We have a lot of variables that are used to hold kernel mapping addresses, offsets between physical and virtual mappings and some others used for XIP kernels: they are all defined at different places in mm/init.c, so group them into a single structure with, for some of them, more explicit and concise names. Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-06-17 13:53:07 +00:00
la a1, kernel_map
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 06:35:14 +00:00
XIP_FIXUP_OFFSET a1
riscv: Introduce structure that group all variables regarding kernel mapping We have a lot of variables that are used to hold kernel mapping addresses, offsets between physical and virtual mappings and some others used for XIP kernels: they are all defined at different places in mm/init.c, so group them into a single structure with, for some of them, more explicit and concise names. Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-06-17 13:53:07 +00:00
REG_L a1, KERNEL_MAP_VIRT_ADDR(a1)
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-18 01:11:39 +00:00
la a2, _start
sub a1, a1, a2
add ra, ra, a1
/* Point stvec to virtual address of intruction after satp write */
la a2, 1f
add a2, a2, a1
csrw CSR_TVEC, a2
/* Compute satp for kernel page tables, but don't load it yet */
srl a2, a0, PAGE_SHIFT
li a1, SATP_MODE
or a2, a2, a1
/*
* Load trampoline page directory, which will cause us to trap to
* stvec if VA != PA, or simply fall through if VA == PA. We need a
* full fence here because setup_vm() just wrote these PTEs and we need
* to ensure the new translations are in use.
*/
la a0, trampoline_pg_dir
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 06:35:14 +00:00
XIP_FIXUP_OFFSET a0
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-18 01:11:39 +00:00
srl a0, a0, PAGE_SHIFT
or a0, a0, a1
sfence.vma
csrw CSR_SATP, a0
.align 2
1:
riscv: Setup exception vector for nommu platform Exception vector is missing on nommu platform and that is an issue. This patch is tested in Sipeed Maix Bit Dev Board. Fixes: 79b1feba5455 ("RISC-V: Setup exception vector early") Suggested-by: Anup Patel <anup@brainfault.org> Suggested-by: Atish Patra <atishp@atishpatra.org> Signed-off-by: Qiu Wenbo <qiuwenbo@phytium.com.cn> Reviewed-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-08-13 03:38:04 +00:00
/* Set trap vector to spin forever to help debug */
la a0, .Lsecondary_park
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-18 01:11:39 +00:00
csrw CSR_TVEC, a0
/* Reload the global pointer */
.option push
.option norelax
la gp, __global_pointer$
.option pop
/*
* Switch to kernel page tables. A full fence is necessary in order to
* avoid using the trampoline translations, which are only correct for
* the first superpage. Fetching the fence is guarnteed to work
* because that first superpage is translated the same way.
*/
csrw CSR_SATP, a2
sfence.vma
ret
#endif /* CONFIG_MMU */
#ifdef CONFIG_SMP
RISC-V: Add supported for ordered booting method using HSM Currently, all harts have to jump Linux in RISC-V. This complicates the multi-stage boot process as every transient stage also has to ensure all harts enter to that stage and jump to Linux afterwards. It also obstructs a clean Kexec implementation. SBI HSM extension provides alternate solutions where only a single hart need to boot and enter Linux. The booting hart can bring up secondary harts one by one afterwards. Add SBI HSM based cpu_ops that implements an ordered booting method in RISC-V. This change is also backward compatible with older firmware not implementing HSM extension. If a latest kernel is used with older firmware, it will continue to use the default spinning booting method. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-18 01:11:43 +00:00
.global secondary_start_sbi
secondary_start_sbi:
/* Mask all interrupts */
csrw CSR_IE, zero
csrw CSR_IP, zero
/* Load the global pointer */
.option push
.option norelax
la gp, __global_pointer$
.option pop
/*
* Disable FPU to detect illegal usage of
* floating point in kernel space
*/
li t0, SR_FS
csrc CSR_STATUS, t0
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-18 01:11:39 +00:00
/* Set trap vector to spin forever to help debug */
la a3, .Lsecondary_park
csrw CSR_TVEC, a3
slli a3, a0, LGREG
RISC-V: Add supported for ordered booting method using HSM Currently, all harts have to jump Linux in RISC-V. This complicates the multi-stage boot process as every transient stage also has to ensure all harts enter to that stage and jump to Linux afterwards. It also obstructs a clean Kexec implementation. SBI HSM extension provides alternate solutions where only a single hart need to boot and enter Linux. The booting hart can bring up secondary harts one by one afterwards. Add SBI HSM based cpu_ops that implements an ordered booting method in RISC-V. This change is also backward compatible with older firmware not implementing HSM extension. If a latest kernel is used with older firmware, it will continue to use the default spinning booting method. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-18 01:11:43 +00:00
la a4, __cpu_up_stack_pointer
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 06:35:14 +00:00
XIP_FIXUP_OFFSET a4
RISC-V: Add supported for ordered booting method using HSM Currently, all harts have to jump Linux in RISC-V. This complicates the multi-stage boot process as every transient stage also has to ensure all harts enter to that stage and jump to Linux afterwards. It also obstructs a clean Kexec implementation. SBI HSM extension provides alternate solutions where only a single hart need to boot and enter Linux. The booting hart can bring up secondary harts one by one afterwards. Add SBI HSM based cpu_ops that implements an ordered booting method in RISC-V. This change is also backward compatible with older firmware not implementing HSM extension. If a latest kernel is used with older firmware, it will continue to use the default spinning booting method. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-18 01:11:43 +00:00
la a5, __cpu_up_task_pointer
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 06:35:14 +00:00
XIP_FIXUP_OFFSET a5
RISC-V: Add supported for ordered booting method using HSM Currently, all harts have to jump Linux in RISC-V. This complicates the multi-stage boot process as every transient stage also has to ensure all harts enter to that stage and jump to Linux afterwards. It also obstructs a clean Kexec implementation. SBI HSM extension provides alternate solutions where only a single hart need to boot and enter Linux. The booting hart can bring up secondary harts one by one afterwards. Add SBI HSM based cpu_ops that implements an ordered booting method in RISC-V. This change is also backward compatible with older firmware not implementing HSM extension. If a latest kernel is used with older firmware, it will continue to use the default spinning booting method. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-18 01:11:43 +00:00
add a4, a3, a4
add a5, a3, a5
REG_L sp, (a4)
REG_L tp, (a5)
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-18 01:11:39 +00:00
.global secondary_start_common
secondary_start_common:
#ifdef CONFIG_MMU
/* Enable virtual memory and relocate to virtual address */
la a0, swapper_pg_dir
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 06:35:14 +00:00
XIP_FIXUP_OFFSET a0
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-18 01:11:39 +00:00
call relocate
#endif
riscv: Setup exception vector for nommu platform Exception vector is missing on nommu platform and that is an issue. This patch is tested in Sipeed Maix Bit Dev Board. Fixes: 79b1feba5455 ("RISC-V: Setup exception vector early") Suggested-by: Anup Patel <anup@brainfault.org> Suggested-by: Atish Patra <atishp@atishpatra.org> Signed-off-by: Qiu Wenbo <qiuwenbo@phytium.com.cn> Reviewed-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-08-13 03:38:04 +00:00
call setup_trap_vector
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-18 01:11:39 +00:00
tail smp_callin
#endif /* CONFIG_SMP */
riscv: Setup exception vector for nommu platform Exception vector is missing on nommu platform and that is an issue. This patch is tested in Sipeed Maix Bit Dev Board. Fixes: 79b1feba5455 ("RISC-V: Setup exception vector early") Suggested-by: Anup Patel <anup@brainfault.org> Suggested-by: Atish Patra <atishp@atishpatra.org> Signed-off-by: Qiu Wenbo <qiuwenbo@phytium.com.cn> Reviewed-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-08-13 03:38:04 +00:00
.align 2
setup_trap_vector:
/* Set trap vector to exception handler */
la a0, handle_exception
csrw CSR_TVEC, a0
/*
* Set sup0 scratch register to 0, indicating to exception vector that
* we are presently executing in kernel.
*/
csrw CSR_SCRATCH, zero
ret
riscv: fix misalgned trap vector base address The trap vector marked by label .Lsecondary_park must align on a 4-byte boundary, as the {m,s}tvec is defined to require 4-byte alignment. Signed-off-by: Chen Lu <181250012@smail.nju.edu.cn> Reviewed-by: Anup Patel <anup.patel@wdc.com> Fixes: e011995e826f ("RISC-V: Move relocate and few other functions out of __init") Cc: stable@vger.kernel.org Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-10-18 05:22:38 +00:00
.align 2
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-18 01:11:39 +00:00
.Lsecondary_park:
/* We lack SMP support or have too many harts, so park this hart */
wfi
j .Lsecondary_park
END(_start)
ENTRY(_start_kernel)
RISC-V: Access CSRs using CSR numbers We should prefer accessing CSRs using their CSR numbers because: 1. It compiles fine with older toolchains. 2. We can use latest CSR names in #define macro names of CSR numbers as-per RISC-V spec. 3. We can access newly added CSRs even if toolchain does not recognize newly addes CSRs by name. Signed-off-by: Anup Patel <anup.patel@wdc.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2019-04-25 08:38:41 +00:00
/* Mask all interrupts */
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 12:10:32 +00:00
csrw CSR_IE, zero
csrw CSR_IP, zero
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 01:00:26 +00:00
riscv: read the hart ID from mhartid on boot When in M-Mode, we can use the mhartid CSR to get the ID of the running HART. Doing so, direct M-Mode boot without firmware is possible. Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Reviewed-by: Atish Patra <atish.patra@wdc.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 12:10:39 +00:00
#ifdef CONFIG_RISCV_M_MODE
riscv: clear the instruction cache and all registers when booting When we get booted we want a clear slate without any leaks from previous supervisors or the firmware. Flush the instruction cache and then clear all registers to known good values. This is really important for the upcoming nommu support that runs on M-mode, but can't really harm when running in S-mode either. Vaguely based on the concepts from opensbi. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 12:10:40 +00:00
/* flush the instruction cache */
fence.i
/* Reset all registers except ra, a0, a1 */
call reset_regs
RISC-V: Skip setting up PMPs on traps The RISC-V ISA manual says that PMPs are WARL, but it appears the K210 doesn't implement them and instead traps on the unsupported accesses. This patch handles those traps by just skipping the PMP initialization entirely, under the theory that machines that trap on PMP accesses must allow memory accesses as otherwise they're pretty useless. Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-07 18:33:40 +00:00
/*
* Setup a PMP to permit access to all of memory. Some machines may
* not implement PMPs, so we set up a quick trap handler to just skip
* touching the PMPs on any trap.
*/
la a0, pmp_done
csrw CSR_TVEC, a0
riscv: set pmp configuration if kernel is running in M-mode When the kernel is running in S-mode, the expectation is that the bootloader or SBI layer will configure the PMP to allow the kernel to access physical memory. But, when the kernel is running in M-mode and is started with the ELF "loader", there's probably no bootloader or SBI layer involved to configure the PMP. Thus, we need to configure the PMP ourselves to enable the kernel to access all regions. Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-01-09 03:17:40 +00:00
li a0, -1
csrw CSR_PMPADDR0, a0
li a0, (PMP_A_NAPOT | PMP_R | PMP_W | PMP_X)
csrw CSR_PMPCFG0, a0
RISC-V: Skip setting up PMPs on traps The RISC-V ISA manual says that PMPs are WARL, but it appears the K210 doesn't implement them and instead traps on the unsupported accesses. This patch handles those traps by just skipping the PMP initialization entirely, under the theory that machines that trap on PMP accesses must allow memory accesses as otherwise they're pretty useless. Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-07 18:33:40 +00:00
.align 2
pmp_done:
riscv: set pmp configuration if kernel is running in M-mode When the kernel is running in S-mode, the expectation is that the bootloader or SBI layer will configure the PMP to allow the kernel to access physical memory. But, when the kernel is running in M-mode and is started with the ELF "loader", there's probably no bootloader or SBI layer involved to configure the PMP. Thus, we need to configure the PMP ourselves to enable the kernel to access all regions. Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-01-09 03:17:40 +00:00
riscv: read the hart ID from mhartid on boot When in M-Mode, we can use the mhartid CSR to get the ID of the running HART. Doing so, direct M-Mode boot without firmware is possible. Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Reviewed-by: Atish Patra <atish.patra@wdc.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 12:10:39 +00:00
/*
* The hartid in a0 is expected later on, and we have no firmware
* to hand it to us.
*/
csrr a0, CSR_MHARTID
riscv: clear the instruction cache and all registers when booting When we get booted we want a clear slate without any leaks from previous supervisors or the firmware. Flush the instruction cache and then clear all registers to known good values. This is really important for the upcoming nommu support that runs on M-mode, but can't really harm when running in S-mode either. Vaguely based on the concepts from opensbi. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 12:10:40 +00:00
#endif /* CONFIG_RISCV_M_MODE */
riscv: read the hart ID from mhartid on boot When in M-Mode, we can use the mhartid CSR to get the ID of the running HART. Doing so, direct M-Mode boot without firmware is possible. Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Reviewed-by: Atish Patra <atish.patra@wdc.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 12:10:39 +00:00
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 01:00:26 +00:00
/* Load the global pointer */
.option push
.option norelax
la gp, __global_pointer$
.option pop
/*
* Disable FPU to detect illegal usage of
* floating point in kernel space
*/
li t0, SR_FS
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 12:10:32 +00:00
csrc CSR_STATUS, t0
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 01:00:26 +00:00
arch/riscv: disable excess harts before picking main boot hart Harts with id greater than or equal to CONFIG_NR_CPUS need to be disabled. But the kernel can pick any hart as the main hart. So, before picking the main hart, the kernel must disable harts with ids greater than or equal to CONFIG_NR_CPUS. Signed-off-by: Xiang Wang <merle@hardenedlinux.org> Reviewed-by: Palmer Dabbelt <palmer@sifive.com> Reviewed-by: Anup Patel <anup.patel@wdc.com> [paul.walmsley@sifive.com: updated to apply; cleaned up patch description] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-09-06 03:56:09 +00:00
#ifdef CONFIG_SMP
li t0, CONFIG_NR_CPUS
riscv: make sure the cores stay looping in .Lsecondary_park The code in secondary_park is currently placed in the .init section. The kernel reclaims and clears this code when it finishes booting. That causes the cores parked in it to go to somewhere unpredictable, so we move this function out of init to make sure the cores stay looping there. The instruction bgeu a0, t0, .Lsecondary_park may have "a relocation truncated to fit" issue during linking time. It is because that sections are too far to jump. Let's use tail to jump to the .Lsecondary_park. Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Reviewed-by: Anup Patel <anup.patel@sifive.com> Cc: Andreas Schwab <schwab@suse.de> Cc: stable@vger.kernel.org Fixes: 76d2a0493a17d ("RISC-V: Init and Halt Code") Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2020-01-15 06:54:36 +00:00
blt a0, t0, .Lgood_cores
tail .Lsecondary_park
.Lgood_cores:
arch/riscv: disable excess harts before picking main boot hart Harts with id greater than or equal to CONFIG_NR_CPUS need to be disabled. But the kernel can pick any hart as the main hart. So, before picking the main hart, the kernel must disable harts with ids greater than or equal to CONFIG_NR_CPUS. Signed-off-by: Xiang Wang <merle@hardenedlinux.org> Reviewed-by: Palmer Dabbelt <palmer@sifive.com> Reviewed-by: Anup Patel <anup.patel@wdc.com> [paul.walmsley@sifive.com: updated to apply; cleaned up patch description] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-09-06 03:56:09 +00:00
#endif
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 06:35:14 +00:00
#ifndef CONFIG_XIP_KERNEL
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 01:00:26 +00:00
/* Pick one hart to run the main boot sequence */
la a3, hart_lottery
li a2, 1
amoadd.w a3, a2, (a3)
bnez a3, .Lsecondary_start
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 06:35:14 +00:00
#else
/* hart_lottery in flash contains a magic number */
la a3, hart_lottery
mv a2, a3
XIP_FIXUP_OFFSET a2
lw t1, (a3)
amoswap.w t0, t1, (a2)
/* first time here if hart_lottery in RAM is not set */
beq t0, t1, .Lsecondary_start
la sp, _end + THREAD_SIZE
XIP_FIXUP_OFFSET sp
mv s0, a0
call __copy_data
/* Restore a0 copy */
mv a0, s0
#endif
#ifndef CONFIG_XIP_KERNEL
RISC-V: Build flat and compressed kernel images This patch extends Linux RISC-V build system to build and install: Image - Flat uncompressed kernel image Image.gz - Flat and GZip compressed kernel image Quiet a few bootloaders (such as Uboot, UEFI, etc) are capable of booting flat and compressed kernel images. In case of Uboot, booting Image or Image.gz is achieved using bootm command. The flat and uncompressed kernel image (i.e. Image) is very useful in pre-silicon developent and testing because we can create back-door HEX files for RAM on FPGAs from Image. Signed-off-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2018-11-12 05:55:15 +00:00
/* Clear BSS for flat non-ELF images */
la a3, __bss_start
la a4, __bss_stop
ble a4, a3, clear_bss_done
clear_bss:
REG_S zero, (a3)
add a3, a3, RISCV_SZPTR
blt a3, a4, clear_bss
clear_bss_done:
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 06:35:14 +00:00
#endif
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 01:00:26 +00:00
/* Save hart ID and DTB physical address */
mv s0, a0
mv s1, a1
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 06:35:14 +00:00
RISC-V: Use Linux logical CPU number instead of hartid Setup the cpu_logical_map during boot. Moreover, every SBI call and PLIC context are based on the physical hartid. Use the logical CPU to hartid mapping to pass correct hartid to respective functions. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2018-10-02 19:15:05 +00:00
la a2, boot_cpu_hartid
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 06:35:14 +00:00
XIP_FIXUP_OFFSET a2
RISC-V: Use Linux logical CPU number instead of hartid Setup the cpu_logical_map during boot. Moreover, every SBI call and PLIC context are based on the physical hartid. Use the logical CPU to hartid mapping to pass correct hartid to respective functions. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2018-10-02 19:15:05 +00:00
REG_S a0, (a2)
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 01:00:26 +00:00
/* Initialize page tables and relocate to virtual addresses */
la sp, init_thread_union + THREAD_SIZE
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 06:35:14 +00:00
XIP_FIXUP_OFFSET sp
riscv: add BUILTIN_DTB support for MMU-enabled targets Sometimes, especially in a production system we may not want to use a "smart bootloader" like u-boot to load kernel, ramdisk and device tree from a filesystem on eMMC, but rather load the kernel from a NAND partition and just run it as soon as we can, and in this case it is convenient to have device tree compiled into the kernel binary. Since this case is not limited to MMU-less systems, let's support it for these which have MMU enabled too. While at it, provide __dtb_start as a parameter to setup_vm() in BUILTIN_DTB case, so we don't have to duplicate BUILTIN_DTB specific processing in MMU-enabled and MMU-disabled versions of setup_vm(). Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-01-15 23:49:48 +00:00
#ifdef CONFIG_BUILTIN_DTB
la a0, __dtb_start
#else
RISC-V: Setup initial page tables in two stages Currently, the setup_vm() does initial page table setup in one-shot very early before enabling MMU. Due to this, the setup_vm() has to map all possible kernel virtual addresses since it does not know size and location of RAM. This means we have kernel mappings for non-existent RAM and any buggy driver (or kernel) code doing out-of-bound access to RAM will not fault and cause underterministic behaviour. Further, the setup_vm() creates PMD mappings (i.e. 2M mappings) for RV64 systems. This means for PAGE_OFFSET=0xffffffe000000000 (i.e. MAXPHYSMEM_128GB=y), the setup_vm() will require 129 pages (i.e. 516 KB) of memory for initial page tables which is never freed. The memory required for initial page tables will further increase if we chose a lower value of PAGE_OFFSET (e.g. 0xffffff0000000000) This patch implements two-staged initial page table setup, as follows: 1. Early (i.e. setup_vm()): This stage maps kernel image and DTB in a early page table (i.e. early_pg_dir). The early_pg_dir will be used only by boot HART so it can be freed as-part of init memory free-up. 2. Final (i.e. setup_vm_final()): This stage maps all possible RAM banks in the final page table (i.e. swapper_pg_dir). The boot HART will start using swapper_pg_dir at the end of setup_vm_final(). All non-boot HARTs directly use the swapper_pg_dir created by boot HART. We have following advantages with this new approach: 1. Kernel mappings for non-existent RAM don't exists anymore. 2. Memory consumed by initial page tables is now indpendent of the chosen PAGE_OFFSET. 3. Memory consumed by initial page tables on RV64 system is 2 pages (i.e. 8 KB) which has significantly reduced and these pages will be freed as-part of the init memory free-up. The patch also provides a foundation for implementing strict kernel mappings where we protect kernel text and rodata using PTE permissions. Suggested-by: Mike Rapoport <rppt@linux.ibm.com> Signed-off-by: Anup Patel <anup.patel@wdc.com> [paul.walmsley@sifive.com: updated to apply; fixed a checkpatch warning] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-06-28 20:36:21 +00:00
mv a0, s1
riscv: add BUILTIN_DTB support for MMU-enabled targets Sometimes, especially in a production system we may not want to use a "smart bootloader" like u-boot to load kernel, ramdisk and device tree from a filesystem on eMMC, but rather load the kernel from a NAND partition and just run it as soon as we can, and in this case it is convenient to have device tree compiled into the kernel binary. Since this case is not limited to MMU-less systems, let's support it for these which have MMU enabled too. While at it, provide __dtb_start as a parameter to setup_vm() in BUILTIN_DTB case, so we don't have to duplicate BUILTIN_DTB specific processing in MMU-enabled and MMU-disabled versions of setup_vm(). Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-01-15 23:49:48 +00:00
#endif /* CONFIG_BUILTIN_DTB */
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 01:00:26 +00:00
call setup_vm
riscv: add nommu support The kernel runs in M-mode without using page tables, and thus can't run bare metal without help from additional firmware. Most of the patch is just stubbing out code not needed without page tables, but there is an interesting detail in the signals implementation: - The normal RISC-V syscall ABI only implements rt_sigreturn as VDSO entry point, but the ELF VDSO is not supported for nommu Linux. We instead copy the code to call the syscall onto the stack. In addition to enabling the nommu code a new defconfig for a small kernel image that can run in nommu mode on qemu is also provided, to run a kernel in qemu you can use the following command line: qemu-system-riscv64 -smp 2 -m 64 -machine virt -nographic \ -kernel arch/riscv/boot/loader \ -drive file=rootfs.ext2,format=raw,id=hd0 \ -device virtio-blk-device,drive=hd0 Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> [paul.walmsley@sifive.com: updated to apply; add CONFIG_MMU guards around PCI_IOBASE definition to fix build issues; fixed checkpatch issues; move the PCI_IO_* and VMEMMAP address space macros along with the others; resolve sparse warning] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 12:10:41 +00:00
#ifdef CONFIG_MMU
RISC-V: Setup initial page tables in two stages Currently, the setup_vm() does initial page table setup in one-shot very early before enabling MMU. Due to this, the setup_vm() has to map all possible kernel virtual addresses since it does not know size and location of RAM. This means we have kernel mappings for non-existent RAM and any buggy driver (or kernel) code doing out-of-bound access to RAM will not fault and cause underterministic behaviour. Further, the setup_vm() creates PMD mappings (i.e. 2M mappings) for RV64 systems. This means for PAGE_OFFSET=0xffffffe000000000 (i.e. MAXPHYSMEM_128GB=y), the setup_vm() will require 129 pages (i.e. 516 KB) of memory for initial page tables which is never freed. The memory required for initial page tables will further increase if we chose a lower value of PAGE_OFFSET (e.g. 0xffffff0000000000) This patch implements two-staged initial page table setup, as follows: 1. Early (i.e. setup_vm()): This stage maps kernel image and DTB in a early page table (i.e. early_pg_dir). The early_pg_dir will be used only by boot HART so it can be freed as-part of init memory free-up. 2. Final (i.e. setup_vm_final()): This stage maps all possible RAM banks in the final page table (i.e. swapper_pg_dir). The boot HART will start using swapper_pg_dir at the end of setup_vm_final(). All non-boot HARTs directly use the swapper_pg_dir created by boot HART. We have following advantages with this new approach: 1. Kernel mappings for non-existent RAM don't exists anymore. 2. Memory consumed by initial page tables is now indpendent of the chosen PAGE_OFFSET. 3. Memory consumed by initial page tables on RV64 system is 2 pages (i.e. 8 KB) which has significantly reduced and these pages will be freed as-part of the init memory free-up. The patch also provides a foundation for implementing strict kernel mappings where we protect kernel text and rodata using PTE permissions. Suggested-by: Mike Rapoport <rppt@linux.ibm.com> Signed-off-by: Anup Patel <anup.patel@wdc.com> [paul.walmsley@sifive.com: updated to apply; fixed a checkpatch warning] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-06-28 20:36:21 +00:00
la a0, early_pg_dir
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 06:35:14 +00:00
XIP_FIXUP_OFFSET a0
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 01:00:26 +00:00
call relocate
riscv: add nommu support The kernel runs in M-mode without using page tables, and thus can't run bare metal without help from additional firmware. Most of the patch is just stubbing out code not needed without page tables, but there is an interesting detail in the signals implementation: - The normal RISC-V syscall ABI only implements rt_sigreturn as VDSO entry point, but the ELF VDSO is not supported for nommu Linux. We instead copy the code to call the syscall onto the stack. In addition to enabling the nommu code a new defconfig for a small kernel image that can run in nommu mode on qemu is also provided, to run a kernel in qemu you can use the following command line: qemu-system-riscv64 -smp 2 -m 64 -machine virt -nographic \ -kernel arch/riscv/boot/loader \ -drive file=rootfs.ext2,format=raw,id=hd0 \ -device virtio-blk-device,drive=hd0 Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> [paul.walmsley@sifive.com: updated to apply; add CONFIG_MMU guards around PCI_IOBASE definition to fix build issues; fixed checkpatch issues; move the PCI_IO_* and VMEMMAP address space macros along with the others; resolve sparse warning] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 12:10:41 +00:00
#endif /* CONFIG_MMU */
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 01:00:26 +00:00
riscv: Setup exception vector for nommu platform Exception vector is missing on nommu platform and that is an issue. This patch is tested in Sipeed Maix Bit Dev Board. Fixes: 79b1feba5455 ("RISC-V: Setup exception vector early") Suggested-by: Anup Patel <anup@brainfault.org> Suggested-by: Atish Patra <atishp@atishpatra.org> Signed-off-by: Qiu Wenbo <qiuwenbo@phytium.com.cn> Reviewed-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-08-13 03:38:04 +00:00
call setup_trap_vector
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 01:00:26 +00:00
/* Restore C environment */
la tp, init_task
riscv: simplify the stack pointer setup in head.S We don't need THREAD_SIZE in asm-offsets.c as we can just calculate the value of init_thread_union + THREAD_SIZE using cpp, just like we do a few lines above. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Atish Patra <atish.patra@wdc.com> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2019-04-15 09:14:37 +00:00
la sp, init_thread_union + THREAD_SIZE
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 01:00:26 +00:00
riscv: Add KASAN support This patch ports the feature Kernel Address SANitizer (KASAN). Note: The start address of shadow memory is at the beginning of kernel space, which is 2^64 - (2^39 / 2) in SV39. The size of the kernel space is 2^38 bytes so the size of shadow memory should be 2^38 / 8. Thus, the shadow memory would not overlap with the fixmap area. There are currently two limitations in this port, 1. RV64 only: KASAN need large address space for extra shadow memory region. 2. KASAN can't debug the modules since the modules are allocated in VMALLOC area. We mapped the shadow memory, which corresponding to VMALLOC area, to the kasan_early_shadow_page because we don't have enough physical space for all the shadow memory corresponding to VMALLOC area. Signed-off-by: Nick Hu <nickhu@andestech.com> Reported-by: Greentime Hu <green.hu@gmail.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-01-06 18:38:32 +00:00
#ifdef CONFIG_KASAN
call kasan_early_init
#endif
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 01:00:26 +00:00
/* Start the kernel */
riscv: Add SOC early init support Add a mechanism for early SoC initialization for platforms that need additional hardware initialization not possible through the regular device tree and drivers mechanism. With this, a SoC specific initialization function can be called very early, before DTB parsing is done by parse_dtb() in Linux RISC-V kernel setup code. This can be very useful for early hardware initialization for No-MMU kernels booted directly in M-mode because it is quite likely that no other booting stage exist prior to the No-MMU kernel. Example use of a SoC early initialization is as follows: static void vendor_abc_early_init(const void *fdt) { /* * some early init code here that can use simple matches * against the flat device tree file. */ } SOC_EARLY_INIT_DECLARE("vendor,abc", abc_early_init); This early initialization function is executed only if the flat device tree for the board has a 'compatible = "vendor,abc"' entry; Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Signed-off-by: Anup Patel <anup.patel@wdc.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-16 00:47:38 +00:00
call soc_early_init
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 01:00:26 +00:00
tail start_kernel
.Lsecondary_start:
#ifdef CONFIG_SMP
/* Set trap vector to spin forever to help debug */
la a3, .Lsecondary_park
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 12:10:32 +00:00
csrw CSR_TVEC, a3
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 01:00:26 +00:00
slli a3, a0, LGREG
la a1, __cpu_up_stack_pointer
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 06:35:14 +00:00
XIP_FIXUP_OFFSET a1
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 01:00:26 +00:00
la a2, __cpu_up_task_pointer
RISC-V: enable XIP Introduce XIP (eXecute In Place) support for RISC-V platforms. It allows code to be executed directly from non-volatile storage directly addressable by the CPU, such as QSPI NOR flash which can be found on many RISC-V platforms. This makes way for significant optimization of RAM footprint. The XIP kernel is not compressed since it has to run directly from flash, so it will occupy more space on the non-volatile storage. The physical flash address used to link the kernel object files and for storing it has to be known at compile time and is represented by a Kconfig option. XIP on RISC-V will for the time being only work on MMU-enabled kernels. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> [Alex: Rebase on top of "Move kernel mapping outside the linear mapping" ] Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> [Palmer: disable XIP for allyesconfig] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-13 06:35:14 +00:00
XIP_FIXUP_OFFSET a2
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 01:00:26 +00:00
add a1, a3, a1
add a2, a3, a2
/*
* This hart didn't win the lottery, so we wait for the winning hart to
* get far enough along the boot process that it should continue.
*/
.Lwait_for_cpu_up:
/* FIXME: We should WFI to save some energy here. */
REG_L sp, (a1)
REG_L tp, (a2)
beqz sp, .Lwait_for_cpu_up
beqz tp, .Lwait_for_cpu_up
fence
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-18 01:11:39 +00:00
tail secondary_start_common
riscv: add nommu support The kernel runs in M-mode without using page tables, and thus can't run bare metal without help from additional firmware. Most of the patch is just stubbing out code not needed without page tables, but there is an interesting detail in the signals implementation: - The normal RISC-V syscall ABI only implements rt_sigreturn as VDSO entry point, but the ELF VDSO is not supported for nommu Linux. We instead copy the code to call the syscall onto the stack. In addition to enabling the nommu code a new defconfig for a small kernel image that can run in nommu mode on qemu is also provided, to run a kernel in qemu you can use the following command line: qemu-system-riscv64 -smp 2 -m 64 -machine virt -nographic \ -kernel arch/riscv/boot/loader \ -drive file=rootfs.ext2,format=raw,id=hd0 \ -device virtio-blk-device,drive=hd0 Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> [paul.walmsley@sifive.com: updated to apply; add CONFIG_MMU guards around PCI_IOBASE definition to fix build issues; fixed checkpatch issues; move the PCI_IO_* and VMEMMAP address space macros along with the others; resolve sparse warning] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 12:10:41 +00:00
#endif
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 01:00:26 +00:00
RISC-V: Move relocate and few other functions out of __init The secondary hart booting and relocation code are under .init section. As a result, it will be freed once kernel booting is done. However, ordered booting protocol and CPU hotplug always requires these functions to be present to bringup harts after initial kernel boot. Move the required functions to a different section and make sure that they are in memory within first 2MB offset as trampoline page directory only maps first 2MB. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-18 01:11:39 +00:00
END(_start_kernel)
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 01:00:26 +00:00
riscv: clear the instruction cache and all registers when booting When we get booted we want a clear slate without any leaks from previous supervisors or the firmware. Flush the instruction cache and then clear all registers to known good values. This is really important for the upcoming nommu support that runs on M-mode, but can't really harm when running in S-mode either. Vaguely based on the concepts from opensbi. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 12:10:40 +00:00
#ifdef CONFIG_RISCV_M_MODE
ENTRY(reset_regs)
li sp, 0
li gp, 0
li tp, 0
li t0, 0
li t1, 0
li t2, 0
li s0, 0
li s1, 0
li a2, 0
li a3, 0
li a4, 0
li a5, 0
li a6, 0
li a7, 0
li s2, 0
li s3, 0
li s4, 0
li s5, 0
li s6, 0
li s7, 0
li s8, 0
li s9, 0
li s10, 0
li s11, 0
li t3, 0
li t4, 0
li t5, 0
li t6, 0
riscv: fix scratch register clearing in M-mode. This patch fixes that the sscratch register clearing in M-mode. It cleared sscratch register in M-mode, but it should clear mscratch register. That will cause kernel trap if the CPU core doesn't support S-mode when trying to access sscratch. Fixes: 9e80635619b5 ("riscv: clear the instruction cache and all registers when booting") Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-12-19 06:44:59 +00:00
csrw CSR_SCRATCH, 0
riscv: clear the instruction cache and all registers when booting When we get booted we want a clear slate without any leaks from previous supervisors or the firmware. Flush the instruction cache and then clear all registers to known good values. This is really important for the upcoming nommu support that runs on M-mode, but can't really harm when running in S-mode either. Vaguely based on the concepts from opensbi. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 12:10:40 +00:00
#ifdef CONFIG_FPU
csrr t0, CSR_MISA
andi t0, t0, (COMPAT_HWCAP_ISA_F | COMPAT_HWCAP_ISA_D)
riscv: Fixup obvious bug for fp-regs reset CSR_MISA is defined in Privileged Architectures' spec: 3.1.1 Machine ISA Register misa. Every bit:1 indicate a feature, so we should beqz reset_done when there is no F/D bit in csr_misa register. Signed-off-by: Guo Ren <ren_guo@c-sky.com> [paul.walmsley@sifive.com: fix typo in commit message] Fixes: 9e80635619b51 ("riscv: clear the instruction cache and all registers when booting") Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2020-01-05 02:52:14 +00:00
beqz t0, .Lreset_regs_done
riscv: clear the instruction cache and all registers when booting When we get booted we want a clear slate without any leaks from previous supervisors or the firmware. Flush the instruction cache and then clear all registers to known good values. This is really important for the upcoming nommu support that runs on M-mode, but can't really harm when running in S-mode either. Vaguely based on the concepts from opensbi. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 12:10:40 +00:00
li t1, SR_FS
csrs CSR_STATUS, t1
fmv.s.x f0, zero
fmv.s.x f1, zero
fmv.s.x f2, zero
fmv.s.x f3, zero
fmv.s.x f4, zero
fmv.s.x f5, zero
fmv.s.x f6, zero
fmv.s.x f7, zero
fmv.s.x f8, zero
fmv.s.x f9, zero
fmv.s.x f10, zero
fmv.s.x f11, zero
fmv.s.x f12, zero
fmv.s.x f13, zero
fmv.s.x f14, zero
fmv.s.x f15, zero
fmv.s.x f16, zero
fmv.s.x f17, zero
fmv.s.x f18, zero
fmv.s.x f19, zero
fmv.s.x f20, zero
fmv.s.x f21, zero
fmv.s.x f22, zero
fmv.s.x f23, zero
fmv.s.x f24, zero
fmv.s.x f25, zero
fmv.s.x f26, zero
fmv.s.x f27, zero
fmv.s.x f28, zero
fmv.s.x f29, zero
fmv.s.x f30, zero
fmv.s.x f31, zero
csrw fcsr, 0
/* note that the caller must clear SR_FS */
#endif /* CONFIG_FPU */
.Lreset_regs_done:
ret
END(reset_regs)
#endif /* CONFIG_RISCV_M_MODE */
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 01:00:26 +00:00
__PAGE_ALIGNED_BSS
/* Empty zero page */
.balign PAGE_SIZE
Reference in New Issue Copy Permalink