This patch removes unnecessary `isb` from the enable DCO sequence as
there is no need to synchronize this operation.
Change-Id: I0191e684bbc7fdba635c3afbc4e4ecd793b6f06f
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>
This patch moves the code to disable DCO operations out from common
CPU files. This allows the platform code to call thsi API as and
when required. There are certain CPU power down states which require
the DCO to be kept ON and platforms can decide selectively now.
Change-Id: Icb946fe2545a7d8c5903c420d1ee169c4921a2d1
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>
The errata is enabled by default on r0p4, which is confusing given that
we state we do not enable errata by default.
This patch clarifies this sentence by saying it is enabled in hardware
by default.
Change-Id: I70a062d93e1da2416d5f6d5776a77a659da737aa
Signed-off-by: Douglas Raillard <douglas.raillard@arm.com>
This patch adds support for all variants of the Denver CPUs. The
variants export their cpu_ops to allow all Denver platforms to run
the Trusted Firmware stack.
Change-Id: I1488813ddfd506ffe363d8a32cda1b575e437035
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>
The ARMv8v.1 architecture extension has introduced support for far
atomics, which includes compare-and-swap. Compare and Swap instruction
is only available for AArch64.
Introduce build options to choose the architecture versions to target
ARM Trusted Firmware:
- ARM_ARCH_MAJOR: selects the major version of target ARM
Architecture. Default value is 8.
- ARM_ARCH_MINOR: selects the minor version of target ARM
Architecture. Default value is 0.
When:
(ARM_ARCH_MAJOR > 8) || ((ARM_ARCH_MAJOR == 8) && (ARM_ARCH_MINOR >= 1)),
for AArch64, Compare and Swap instruction is used to implement spin
locks. Otherwise, the implementation falls back to using
load-/store-exclusive instructions.
Update user guide, and introduce a section in Firmware Design guide to
summarize support for features introduced in ARMv8 Architecture
Extensions.
Change-Id: I73096a0039502f7aef9ec6ab3ae36680da033f16
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>
Perform stat accounting for retention/standby states also when
requested at multiple power levels.
Change-Id: I2c495ea7cdff8619bde323fb641cd84408eb5762
Signed-off-by: dp-arm <dimitris.papastamos@arm.com>
This patch introduces the following three platform interfaces:
* void plat_psci_stat_accounting_start(const psci_power_state_t *state_info)
This is an optional hook that platforms can implement in order
to perform accounting before entering a low power state. This
typically involves capturing a timestamp.
* void plat_psci_stat_accounting_stop(const psci_power_state_t *state_info)
This is an optional hook that platforms can implement in order
to perform accounting after exiting from a low power state. This
typically involves capturing a timestamp.
* u_register_t plat_psci_stat_get_residency(unsigned int lvl,
const psci_power_state_t *state_info,
unsigned int last_cpu_index)
This is an optional hook that platforms can implement in order
to calculate the PSCI stat residency.
If any of these interfaces are overridden by the platform, it is
recommended that all of them are.
By default `ENABLE_PSCI_STAT` is disabled. If `ENABLE_PSCI_STAT`
is set but `ENABLE_PMF` is not set then an alternative PSCI stat
collection backend must be provided. If both are set, then default
weak definitions of these functions are provided, using PMF to
calculate the residency.
NOTE: Previously, platforms did not have to explicitly set
`ENABLE_PMF` since this was automatically done by the top-level
Makefile.
Change-Id: I17b47804dea68c77bc284df15ee1ccd66bc4b79b
Signed-off-by: dp-arm <dimitris.papastamos@arm.com>
Replace all use of memset by zeromem when zeroing moderately-sized
structure by applying the following transformation:
memset(x, 0, sizeof(x)) => zeromem(x, sizeof(x))
As the Trusted Firmware is compiled with -ffreestanding, it forbids the
compiler from using __builtin_memset and forces it to generate calls to
the slow memset implementation. Zeromem is a near drop in replacement
for this use case, with a more efficient implementation on both AArch32
and AArch64.
Change-Id: Ia7f3a90e888b96d056881be09f0b4d65b41aa79e
Signed-off-by: Douglas Raillard <douglas.raillard@arm.com>
Introduce zeromem_dczva function on AArch64 that can handle unaligned
addresses and make use of DC ZVA instruction to zero a whole block at a
time. This zeroing takes place directly in the cache to speed it up
without doing external memory access.
Remove the zeromem16 function on AArch64 and replace it with an alias to
zeromem. This zeromem16 function is now deprecated.
Remove the 16-bytes alignment constraint on __BSS_START__ in
firmware-design.md as it is now not mandatory anymore (it used to comply
with zeromem16 requirements).
Change the 16-bytes alignment constraints in SP min's linker script to a
8-bytes alignment constraint as the AArch32 zeromem implementation is now
more efficient on 8-bytes aligned addresses.
Introduce zero_normalmem and zeromem helpers in platform agnostic header
that are implemented this way:
* AArch32:
* zero_normalmem: zero using usual data access
* zeromem: alias for zero_normalmem
* AArch64:
* zero_normalmem: zero normal memory using DC ZVA instruction
(needs MMU enabled)
* zeromem: zero using usual data access
Usage guidelines: in most cases, zero_normalmem should be preferred.
There are 2 scenarios where zeromem (or memset) must be used instead:
* Code that must run with MMU disabled (which means all memory is
considered device memory for data accesses).
* Code that fills device memory with null bytes.
Optionally, the following rule can be applied if performance is
important:
* Code zeroing small areas (few bytes) that are not secrets should use
memset to take advantage of compiler optimizations.
Note: Code zeroing security-related critical information should use
zero_normalmem/zeromem instead of memset to avoid removal by
compilers' optimizations in some cases or misbehaving versions of GCC.
FixesARM-software/tf-issues#408
Change-Id: Iafd9663fc1070413c3e1904e54091cf60effaa82
Signed-off-by: Douglas Raillard <douglas.raillard@arm.com>
The errata reporting policy is as follows:
- If an errata workaround is enabled:
- If it applies (i.e. the CPU is affected by the errata), an INFO
message is printed, confirming that the errata workaround has been
applied.
- If it does not apply, a VERBOSE message is printed, confirming
that the errata workaround has been skipped.
- If an errata workaround is not enabled, but would have applied had
it been, a WARN message is printed, alerting that errata workaround
is missing.
The CPU errata messages are printed by both BL1 (primary CPU only) and
runtime firmware on debug builds, once for each CPU/errata combination.
Relevant output from Juno r1 console when ARM Trusted Firmware is built
with PLAT=juno LOG_LEVEL=50 DEBUG=1:
VERBOSE: BL1: cortex_a57: errata workaround for 806969 was not applied
VERBOSE: BL1: cortex_a57: errata workaround for 813420 was not applied
INFO: BL1: cortex_a57: errata workaround for disable_ldnp_overread was applied
WARNING: BL1: cortex_a57: errata workaround for 826974 was missing!
WARNING: BL1: cortex_a57: errata workaround for 826977 was missing!
WARNING: BL1: cortex_a57: errata workaround for 828024 was missing!
WARNING: BL1: cortex_a57: errata workaround for 829520 was missing!
WARNING: BL1: cortex_a57: errata workaround for 833471 was missing!
...
VERBOSE: BL31: cortex_a57: errata workaround for 806969 was not applied
VERBOSE: BL31: cortex_a57: errata workaround for 813420 was not applied
INFO: BL31: cortex_a57: errata workaround for disable_ldnp_overread was applied
WARNING: BL31: cortex_a57: errata workaround for 826974 was missing!
WARNING: BL31: cortex_a57: errata workaround for 826977 was missing!
WARNING: BL31: cortex_a57: errata workaround for 828024 was missing!
WARNING: BL31: cortex_a57: errata workaround for 829520 was missing!
WARNING: BL31: cortex_a57: errata workaround for 833471 was missing!
...
VERBOSE: BL31: cortex_a53: errata workaround for 826319 was not applied
INFO: BL31: cortex_a53: errata workaround for disable_non_temporal_hint was applied
Also update documentation.
Change-Id: Iccf059d3348adb876ca121cdf5207bdbbacf2aba
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>
Some side-channel attacks involve an attacker inferring something from
the time taken for a memory compare operation to complete, for example
when comparing hashes during image authentication. To mitigate this,
timingsafe_bcmp() must be used for such operations instead of the
standard memcmp().
This function executes in constant time and so doesn't leak any timing
information to the caller.
Change-Id: I470a723dc3626a0ee6d5e3f7fd48d0a57b8aa5fd
Signed-off-by: dp-arm <dimitris.papastamos@arm.com>
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
This code has been imported and slightly adapted from FreeBSD:
6253393ad8/lib/libc/string/strnlen.c
Change-Id: Ie5ef5f92e6e904adb88f8628077fdf1d27470eb3
Signed-off-by: Sandrine Bailleux <sandrine.bailleux@arm.com>
One nasty part of ATF is some of boolean macros are always defined
as 1 or 0, and the rest of them are only defined under certain
conditions.
For the former group, "#if FOO" or "#if !FOO" must be used because
"#ifdef FOO" is always true. (Options passed by $(call add_define,)
are the cases.)
For the latter, "#ifdef FOO" or "#ifndef FOO" should be used because
checking the value of an undefined macro is strange.
Here, IMAGE_BL* is handled by make_helpers/build_macro.mk like
follows:
$(eval IMAGE := IMAGE_BL$(call uppercase,$(3)))
$(OBJ): $(2)
@echo " CC $$<"
$$(Q)$$(CC) $$(TF_CFLAGS) $$(CFLAGS) -D$(IMAGE) -c $$< -o $$@
This means, IMAGE_BL* is defined when building the corresponding
image, but *undefined* for the other images.
So, IMAGE_BL* belongs to the latter group where we should use #ifdef
or #ifndef.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
NOTE - this is patch does not address all occurrences of system
includes not being in alphabetical order, just this one case.
Change-Id: I3cd23702d69b1f60a4a9dd7fd4ae27418f15b7a3
Delete old version of libfdt at lib/libfdt. Move new libfdt API
headers to include/lib/libfdt and all other files to lib/libfdt.
Change-Id: I32b7888f1f20d62205310e363accbef169ad7b1b
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
* Add libfdt.mk helper makefile
* Remove unused libfdt files
* Minor changes to fdt.h and libfdt.h to make them C99 compliant
Adapted from 754d78b1b3.
Change-Id: I0847f1c2e6e11f0c899b0b7ecc522c0ad7de210c
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
Import libfdt code from https://git.kernel.org/cgit/utils/dtc/dtc.git
tag "v1.4.2" commit ec02b34c05be04f249ffaaca4b666f5246877dea.
This version includes commit d0b3ab0a0f46ac929b4713da46f7fdcd893dd3bd,
which fixes a buffer overflow in fdt_offset_ptr().
Change-Id: I05a30511ea68417ee7ff26477da3f99e0bd4e06b
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
Various CPU drivers in ARM Trusted Firmware register functions to handle
power-down operations. At present, separate functions are registered to
power down individual cores and clusters.
This scheme operates on the basis of core and cluster, and doesn't cater
for extending the hierarchy for power-down operations. For example,
future CPUs might support multiple threads which might need powering
down individually.
This patch therefore reworks the CPU operations framework to allow for
registering power down handlers on specific level basis. Henceforth:
- Generic code invokes CPU power down operations by the level
required.
- CPU drivers explicitly mention CPU_NO_RESET_FUNC when the CPU has no
reset function.
- CPU drivers register power down handlers as a list: a mandatory
handler for level 0, and optional handlers for higher levels.
All existing CPU drivers are adapted to the new CPU operations framework
without needing any functional changes within.
Also update firmware design guide.
Change-Id: I1826842d37a9e60a9e85fdcee7b4b8f6bc1ad043
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>
Unsigned conditions should be used instead of signed ones when comparing
addresses or sizes in assembly.
Signed-off-by: Douglas Raillard <douglas.raillard@arm.com>
Change-Id: Id3bd9ccaf58c37037761af35ac600907c4bb0580
Testing showed that the time spent in a cluster power down
operation is dominated by cache flushes. Add two more timestamps
in runtime instrumentation to keep track of the time spent
flushing the L1/L2 caches.
Change-Id: I4c5a04e7663543225a85d3c6b271d7b706deffc4
Signed-off-by: dp-arm <dimitris.papastamos@arm.com>
In AArch64, depending on the granularity of the translation tables,
level 0 and/or level 1 of the translation tables may not support block
descriptors, only table descriptors.
This patch introduces a check to make sure that, even if theoretically
it could be possible to create a block descriptor to map a big memory
region, a new subtable will be created to describe its mapping.
Change-Id: Ieb9c302206bfa33fbaf0cdc6a5a82516d32ae2a7
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
Added the definitions `PLAT_PHY_ADDR_SPACE_SIZE` and
`PLAT_VIRT_ADDR_SPACE_SIZE` which specify respectively the physical
and virtual address space size a platform can use.
`ADDR_SPACE_SIZE` is now deprecated. To maintain compatibility, if any
of the previous defines aren't present, the value of `ADDR_SPACE_SIZE`
will be used instead.
For AArch64, register ID_AA64MMFR0_EL1 is checked to calculate the
max PA supported by the hardware and to verify that the previously
mentioned definition is valid. For AArch32, a 40 bit physical
address space is considered.
Added asserts to check for overflows.
Porting guide updated.
Change-Id: Ie8ce1da5967993f0c94dbd4eb9841fc03d5ef8d6
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
Each translation table level entry can only map a given virtual
address onto physical addresses of the same granularity. For example,
with the current configuration, a level 2 entry maps blocks of 2 MB,
so the physical address must be aligned to 2 MB. If the address is not
aligned, the MMU will just ignore the lower bits.
This patch adds an assertion to make sure that physical addresses are
always aligned to the correct boundary.
Change-Id: I0ab43df71829d45cdbe323301b3053e08ca99c2c
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
There is no guarantee on the signedness of char. It can be either
signed or unsigned. On ARM it is unsigned and hence this memcmp()
implementation works as intended.
On other machines, char can be signed (x86 for example). In that case
(and assuming a 2's complement implementation), interpreting a
bit-pattern of 0xFF as signed char can yield -1. If *s1 is 0 and *s2
is 255 then the difference *s1 - *s2 should be negative. The C
integer promotion rules guarantee that the unsigned chars will be
converted to int before the operation takes place. The current
implementation will return a positive value (0 - (-1)) instead, which
is wrong.
Fix it by changing the signedness to unsigned to avoid surprises for
anyone using this code on non-ARM systems.
Change-Id: Ie222fcaa7c0c4272d7a521a6f2f51995fd5130cc
Signed-off-by: dp-arm <dimitris.papastamos@arm.com>
The AArch32 Procedure call Standard mandates that the stack must be aligned
to 8 byte boundary at external interfaces. This patch does the required
changes.
This problem was detected when a crash was encountered in
`psci_print_power_domain_map()` while printing 64 bit values. Aligning
the stack to 8 byte boundary resolved the problem.
FixesARM-Software/tf-issues#437
Change-Id: I517bd8203601bb88e9311bd36d477fb7b3efb292
Signed-off-by: Soby Mathew <soby.mathew@arm.com>
There are many instances in ARM Trusted Firmware where control is
transferred to functions from which return isn't expected. Such jumps
are made using 'bl' instruction to provide the callee with the location
from which it was jumped to. Additionally, debuggers infer the caller by
examining where 'lr' register points to. If a 'bl' of the nature
described above falls at the end of an assembly function, 'lr' will be
left pointing to a location outside of the function range. This misleads
the debugger back trace.
This patch defines a 'no_ret' macro to be used when jumping to functions
from which return isn't expected. The macro ensures to use 'bl'
instruction for the jump, and also, for debug builds, places a 'nop'
instruction immediately thereafter (unless instructed otherwise) so as
to leave 'lr' pointing within the function range.
Change-Id: Ib34c69fc09197cfd57bc06e147cc8252910e01b0
Co-authored-by: Douglas Raillard <douglas.raillard@arm.com>
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>
This patch resets EL2 and EL3 registers that have architecturally
UNKNOWN values on reset and that also provide EL2/EL3 configuration
and trap controls.
Specifically, the EL2 physical timer is disabled to prevent timer
interrups into EL2 - CNTHP_CTL_EL2 and CNTHP_CTL for AArch64 and AArch32,
respectively.
Additionally, for AArch64, HSTR_EL2 is reset to avoid unexpected traps of
non-secure access to certain system registers at EL1 or lower.
For AArch32, the patch also reverts the reset to SDCR which was
incorrectly added in a previous change.
Change-Id: If00eaa23afa7dd36a922265194ccd6223187414f
Signed-off-by: David Cunado <david.cunado@arm.com>
This patch fixes an issue in the normal memory bakery lock
implementation. During assertion of lock status, there is a possibility
that the assertion could fail. This is because the previous update done
to the lock status by the owning CPU when not participating in cache
coherency could result in stale data in the cache due to cache maintenance
operations not propagating to all the caches. This patch fixes this issue
by doing an extra read cache maintenance operation prior to the assertion.
FixesARM-software/tf-issues#402
Change-Id: I0f38a7c52476a4f58e17ebe0141d256d198be88d
Signed-off-by: Soby Mathew <soby.mathew@arm.com>
In order to avoid unexpected traps into EL3/MON mode, this patch
resets the debug registers, MDCR_EL3 and MDCR_EL2 for AArch64,
and SDCR and HDCR for AArch32.
MDCR_EL3/SDCR is zero'ed when EL3/MON mode is entered, at the
start of BL1 and BL31/SMP_MIN.
For MDCR_EL2/HDCR, this patch zero's the bits that are
architecturally UNKNOWN values on reset. This is done when
exiting from EL3/MON mode but only on platforms that support
EL2/HYP mode but choose to exit to EL1/SVC mode.
FixesARM-software/tf-issues#430
Change-Id: Idb992232163c072faa08892251b5626ae4c3a5b6
Signed-off-by: David Cunado <david.cunado@arm.com>
The values of CP15BEN, nTWI & nTWE bits in SCTLR_EL1 are architecturally
unknown if EL3 is AARCH64 whereas they reset to 1 if EL3 is AArch32. This
might be a compatibility break for legacy AArch32 normal world software if
these bits are not set to 1 when EL3 is AArch64. This patch enables the
CP15BEN, nTWI and nTWE bits in the SCTLR_EL1 if the lower non-secure EL is
AArch32. This unifies the SCTLR settings for lower non-secure EL in AArch32
mode for both AArch64 and AArch32 builds of Trusted Firmware.
FixesARM-software/tf-issues#428
Change-Id: I3152d1580e4869c0ea745c5bd9da765f9c254947
Signed-off-by: Soby Mathew <soby.mathew@arm.com>
In order to quantify the overall time spent in the PSCI software
implementation, an initial collection of PMF instrumentation points
has been added.
Instrumentation has been added to the following code paths:
- Entry to PSCI SMC handler. The timestamp is captured as early
as possible during the runtime exception and stored in memory
before entering the PSCI SMC handler.
- Exit from PSCI SMC handler. The timestamp is captured after
normal return from the PSCI SMC handler or if a low power state
was requested it is captured in the bl31 warm boot path before
return to normal world.
- Entry to low power state. The timestamp is captured before entry
to a low power state which implies either standby or power down.
As these power states are mutually exclusive, only one timestamp
is defined to describe both. It is possible to differentiate between
the two power states using the PSCI STAT interface.
- Exit from low power state. The timestamp is captured after a standby
or power up operation has completed.
To calculate the number of cycles spent running code in Trusted Firmware
one can perform the following calculation:
(exit_psci - enter_psci) - (exit_low_pwr - enter_low_pwr).
The resulting number of cycles can be converted to time given the
frequency of the counter.
Change-Id: Ie3b8f3d16409b6703747093b3a2d5c7429ad0166
Signed-off-by: dp-arm <dimitris.papastamos@arm.com>
At present the `el3_entrypoint_common` macro uses `memcpy`
function defined in lib/stdlib/mem.c file, to copy data
from ROM to RAM for BL1. Depending on the compiler being
used the stack could potentially be used, in `memcpy`,
for storing the local variables. Since the stack is
initialized much later in `el3_entrypoint_common` it
may result in unknown behaviour.
This patch adds `memcpy4` function definition in assembly so
that it can be used before the stack is initialized and it
also replaces `memcpy` by `memcpy4` in `el3_entrypoint_common`
macro, to copy data from ROM to RAM for BL1.
Change-Id: I3357a0e8095f05f71bbbf0b185585d9499bfd5e0
This patch introduces a `psci_lib_args_t` structure which must be
passed into `psci_setup()` which is then used to initialize the PSCI
library. The `psci_lib_args_t` is a versioned structure so as to enable
compatibility checks during library initialization. Both BL31 and SP_MIN
are modified to use the new structure.
SP_MIN is also modified to add version string and build message as part
of its cold boot log just like the other BLs in Trusted Firmware.
NOTE: Please be aware that this patch modifies the prototype of
`psci_setup()`, which breaks compatibility with EL3 Runtime Firmware
(excluding BL31 and SP_MIN) integrated with the PSCI Library.
Change-Id: Ic3761db0b790760a7ad664d8a437c72ea5edbcd6
This patch adds ARM Cortex-A32 MPCore Processor support
in the CPU specific operations framework. It also includes
this support for the Base FVP port.
Change-Id: If3697b88678df737c29f79cf3fa1ea2cb6fa565d
This patch adds common changes to support AArch32 state in
BL1 and BL2. Following are the changes:
* Added functions for disabling MMU from Secure state.
* Added AArch32 specific SMC function.
* Added semihosting support.
* Added reporting of unhandled exceptions.
* Added uniprocessor stack support.
* Added `el3_entrypoint_common` macro that can be
shared by BL1 and BL32 (SP_MIN) BL stages. The
`el3_entrypoint_common` is similar to the AArch64
counterpart with the main difference in the assembly
instructions and the registers that are relevant to
AArch32 execution state.
* Enabled `LOAD_IMAGE_V2` flag in Makefile for
`ARCH=aarch32` and added check to make sure that
platform has not overridden to disable it.
Change-Id: I33c6d8dfefb2e5d142fdfd06a0f4a7332962e1a3
This patch adds support for NODE_HW_STATE PSCI API by introducing a new
PSCI platform hook (get_node_hw_state). The implementation validates
supplied arguments, and then invokes this platform-defined hook and
returns its result to the caller. PSCI capabilities are updated
accordingly.
Also updates porting and firmware design guides.
Change-Id: I808e55bdf0c157002a7c104b875779fe50a68a30
The `psci_plat_pm_ops` global pointer is initialized during cold boot by the
primary CPU and will be accessed by the secondary CPUs before enabling data
cache during warm boot. This patch adds a missing data cache flush of
`psci_plat_psci_ops` after initialization during psci_setup() so that
secondaries can see the updated `psci_plat_psci_ops` pointer.
FixesARM-software/tf-issues#424
Change-Id: Id4554800b5646302b944115a33be69507d53cedb
This patch fixes a bug in context management library when writing
SCTLR register during context initialization. The write happened
prior to initialization of the register context pointer. This
resulted in the compiler optimizing the write sequence from the
final binary and hence SCTLR remains uninitialized when
entering normal world. The bug is fixed by doing the
initialization of the register context pointer earlier in the
sequence.
Change-Id: Ic7465593a74534046b79f40446ffa1165c52ed76
Instead of hardcoding a level 1 table as the base translation level
table, let the code decide which level is the most appropriate given
the virtual address space size.
As the table granularity is 4 KB, this allows the code to select
level 0, 1 or 2 as base level for AArch64. This way, instead of
limiting the virtual address space width to 39-31 bits, widths of
48-25 bit can be used.
For AArch32, this change allows the code to select level 1 or 2
as the base translation level table and use virtual address space
width of 32-25 bits.
Also removed some unused definitions related to translation tables.
FixesARM-software/tf-issues#362
Change-Id: Ie3bb5d6d1a4730a26700b09827c79f37ca3cdb65
This patch adds AArch32 support to PSCI library, as follows :
* The `psci_helpers.S` is implemented for AArch32.
* AArch32 version of internal helper function `psci_get_ns_ep_info()` is
defined.
* The PSCI Library is responsible for the Non Secure context initialization.
Hence a library interface `psci_prepare_next_non_secure_ctx()` is introduced
to enable EL3 runtime firmware to initialize the non secure context without
invoking context management library APIs.
Change-Id: I25595b0cc2dbfdf39dbf7c589b875cba33317b9d
This patch adds AArch32 support to cpu ops, context management,
per-cpu data and spinlock libraries. The `entrypoint_info`
structure is modified to add support for AArch32 register
arguments. The CPU operations for AEM generic cpu in AArch32
mode is also added.
Change-Id: I1e52e79f498661d8f31f1e7b3a29e222bc7a4483
This patch adds translation library supports for AArch32 platforms.
The library only supports long descriptor formats for AArch32.
The `enable_mmu_secure()` enables the MMU for secure world with
`TTBR0` pointing to the populated translation tables.
Change-Id: I061345b1779391d098e35e7fe0c76e3ebf850e08
This patch adds various assembly helpers for AArch32 like :
* cache management : Functions to flush, invalidate and clean
cache by MVA. Also helpers to do cache operations by set-way
are also added.
* stack management: Macros to declare stack and get the current
stack corresponding to current CPU.
* Misc: Macros to access co processor registers in AArch32,
macros to define functions in assembly, assert macros, generic
`do_panic()` implementation and function to zero block of memory.
Change-Id: I7b78ca3f922c0eda39beb9786b7150e9193425be
This patch moves the assembly exclusive lock library code
`spinlock.S` into architecture specific folder `aarch64`.
A stub file which includes the file from new location is
retained at the original location for compatibility. The BL
makefiles are also modified to include the file from the new
location.
Change-Id: Ide0b601b79c439e390c3a017d93220a66be73543
This patch fixes the translation table library for wraparound cases. These
cases are not expected to occur on AArch64 platforms because only the
48 bits of the 64 bit address space are used. But it is a possibility for
AArch32 platforms.
Change-Id: Ie7735f7ba2977019381e1c124800381471381499
A PSCI CPU_SUSPEND request to place a CPU in retention states at power levels
higher than the CPU power level is subject to the same state coordination as a
power down state. A CPU could implement multiple retention states at a
particular power level. When exiting WFI, the non-CPU power levels may be in a
different retention state to what was initially requested, therefore each CPU
should refresh its view of the states of all power levels.
Previously, a CPU re-used the state of the power levels when it entered the
retention state. This patch fixes this issue by ensuring that a CPU upon exit
from retention reads the state of each power level afresh.
Change-Id: I93b5f5065c63400c6fd2598dbaafac385748f989
This patch adds a runtime check that psci_find_target_suspend_lvl()
returns a valid value back to psci_cpu_suspend() and psci_get_stat().
If it is invalid, BL31 will now panic.
Note that on the PSCI CPU suspend path there is already a debug
assertion checking the validity of the target composite power state,
which effectively also checks the validity of the target suspend level.
Therefore, the error condition would already be caught in debug builds,
but in a release build this assertion would be compiled out.
On the PSCI stat path, there is currently no debug assertion checking
the validity of the power state before using it as an index into
the power domain state array.
Although BL31 platforms ports are responsible for validating the
power state parameter, the security impact (i.e. an out-of-bounds
array access) of a potential platform port bug in this code would
be quite high, given that this parameter comes from an untrusted
source. The cost of checking this in runtime generic code is low.
Change-Id: Icea85b8020e39928ac03ec0cd49805b5857b3906
This patch introduces the PSCI Library interface. The major changes
introduced are as follows:
* Earlier BL31 was responsible for Architectural initialization during cold
boot via bl31_arch_setup() whereas PSCI was responsible for the same during
warm boot. This functionality is now consolidated by the PSCI library
and it does Architectural initialization via psci_arch_setup() during both
cold and warm boots.
* Earlier the warm boot entry point was always `psci_entrypoint()`. This was
not flexible enough as a library interface. Now PSCI expects the runtime
firmware to provide the entry point via `psci_setup()`. A new function
`bl31_warm_entrypoint` is introduced in BL31 and the previous
`psci_entrypoint()` is deprecated.
* The `smc_helpers.h` is reorganized to separate the SMC Calling Convention
defines from the Trusted Firmware SMC helpers. The former is now in a new
header file `smcc.h` and the SMC helpers are moved to Architecture specific
header.
* The CPU context is used by PSCI for context initialization and
restoration after power down (PSCI Context). It is also used by BL31 for SMC
handling and context management during Normal-Secure world switch (SMC
Context). The `psci_smc_handler()` interface is redefined to not use SMC
helper macros thus enabling to decouple the PSCI context from EL3 runtime
firmware SMC context. This enables PSCI to be integrated with other runtime
firmware using a different SMC context.
NOTE: With this patch the architectural setup done in `bl31_arch_setup()`
is done as part of `psci_setup()` and hence `bl31_platform_setup()` will be
invoked prior to architectural setup. It is highly unlikely that the platform
setup will depend on architectural setup and cause any failure. Please be
be aware of this change in sequence.
Change-Id: I7f497a08d33be234bbb822c28146250cb20dab73
This patch moves the PSCI services and BL31 frameworks like context
management and per-cpu data into new library components `PSCI` and
`el3_runtime` respectively. This enables PSCI to be built independently from
BL31. A new `psci_lib.mk` makefile is introduced which adds the relevant
PSCI library sources and gets included by `bl31.mk`. Other changes which
are done as part of this patch are:
* The runtime services framework is now moved to the `common/` folder to
enable reuse.
* The `asm_macros.S` and `assert_macros.S` helpers are moved to architecture
specific folder.
* The `plat_psci_common.c` is moved from the `plat/common/aarch64/` folder
to `plat/common` folder. The original file location now has a stub which
just includes the file from new location to maintain platform compatibility.
Most of the changes wouldn't affect platform builds as they just involve
changes to the generic bl1.mk and bl31.mk makefiles.
NOTE: THE `plat_psci_common.c` FILE HAS MOVED LOCATION AND THE STUB FILE AT
THE ORIGINAL LOCATION IS NOW DEPRECATED. PLATFORMS SHOULD MODIFY THEIR
MAKEFILES TO INCLUDE THE FILE FROM THE NEW LOCATION.
Change-Id: I6bd87d5b59424995c6a65ef8076d4fda91ad5e86
This patch reworks type usage in generic code, drivers and ARM platform files
to make it more portable. The major changes done with respect to
type usage are as listed below:
* Use uintptr_t for storing address instead of uint64_t or unsigned long.
* Review usage of unsigned long as it can no longer be assumed to be 64 bit.
* Use u_register_t for register values whose width varies depending on
whether AArch64 or AArch32.
* Use generic C types where-ever possible.
In addition to the above changes, this patch also modifies format specifiers
in print invocations so that they are AArch64/AArch32 agnostic. Only files
related to upcoming feature development have been reworked.
Change-Id: I9f8c78347c5a52ba7027ff389791f1dad63ee5f8
This patch adds cpumerrsr_el1 and l2merrsr_el1 to the register dump on
error for applicable CPUs.
These registers hold the ECC errors on L1 and L2 caches.
This patch updates the A53, A57, A72, A73 (l2merrsr_el1 only) CPU libraries.
Signed-off-by: Naga Sureshkumar Relli <nagasure@xilinx.com>
This patch introduces a new header file: include/lib/utils.h.
Its purpose is to provide generic macros and helper functions that
are independent of any BL image, architecture, platform and even
not specific to Trusted Firmware.
For now, it contains only 2 macros: ARRAY_SIZE() and
IS_POWER_OF_TWO(). These were previously defined in bl_common.h and
xlat_tables.c respectively.
bl_common.h includes utils.h to retain compatibility for platforms
that relied on bl_common.h for the ARRAY_SIZE() macro. Upstream
platform ports that use this macro have been updated to include
utils.h.
Change-Id: I960450f54134f25d1710bfbdc4184f12c049a9a9
This patch introduces the MT_EXECUTE/MT_EXECUTE_NEVER memory mapping
attributes in the translation table library to specify the
access permissions for instruction execution of a memory region.
These new attributes should be used only for normal, read-only
memory regions. For other types of memory, the translation table
library still enforces the following rules, regardless of the
MT_EXECUTE/MT_EXECUTE_NEVER attribute:
- Device memory is always marked as execute-never.
- Read-write normal memory is always marked as execute-never.
Change-Id: I8bd27800a8c1d8ac1559910caf4a4840cf25b8b0
This patch clarifies the mmap_desc() function by adding some comments
and reorganising its code. No functional change has been introduced.
Change-Id: I873493be17b4e60a89c1dc087dd908b425065401
This patch adds Performance Measurement Framework(PMF) in the
ARM Trusted Firmware. PMF is implemented as a library and the
SMC interface is provided through ARM SiP service.
The PMF provides capturing, storing, dumping and retrieving the
time-stamps, by enabling the development of services by different
providers, that can be easily integrated into ARM Trusted Firmware.
The PMF capture and retrieval APIs can also do appropriate cache
maintenance operations to the timestamp memory when the caller
indicates so.
`pmf_main.c` consists of core functions that implement service
registration, initialization, storing, dumping and retrieving
the time-stamp.
`pmf_smc.c` consists SMC handling for registered PMF services.
`pmf.h` consists of the macros that can be used by the PMF service
providers to register service and declare time-stamp functions.
`pmf_helpers.h` consists of internal macros that are used by `pmf.h`
By default this feature is disabled in the ARM trusted firmware.
To enable it set the boolean flag `ENABLE_PMF` to 1.
NOTE: The caller is responsible for specifying the appropriate cache
maintenance flags and for acquiring/releasing appropriate locks
before/after capturing/retrieving the time-stamps.
Change-Id: Ib45219ac07c2a81b9726ef6bd9c190cc55e81854
* Move libfdt API headers to include/lib/libfdt
* Add libfdt.mk helper makefile
* Remove unused libfdt files
* Minor changes to fdt.h and libfdt.h to make them C99 compliant
Co-Authored-By: Jens Wiklander <jens.wiklander@linaro.org>
Change-Id: I425842c2b111dcd5fb6908cc698064de4f77220e
* Move stdlib header files from include/stdlib to include/lib/stdlib for
consistency with other library headers.
* Fix checkpatch paths to continue excluding stdlib files.
* Create stdlib.mk to define the stdlib source files and include directories.
* Include stdlib.mk from the top level Makefile.
* Update stdlib header path in the fip_create Makefile.
* Update porting-guide.md with the new paths.
Change-Id: Ia92c2dc572e9efb54a783e306b5ceb2ce24d27fa
This patch adds ARM Cortex-A73 MPCore Processor support
in the CPU specific operations framework. It also includes
this support for the Base FVP port.
Change-Id: I0e26b594f2ec1d28eb815db9810c682e3885716d
This patch fixes the computation of the bitmask used to isolate
the level 1 field of a virtual address. The whole computation needs
to work on 64-bit values to produce the correct bitmask value.
XLAT_TABLE_ENTRIES_MASK being a C constant, it is a 32-bit value
so it needs to be extended to a 64-bit value before it takes part
in any other computation.
This patch fixes this bug by casting XLAT_TABLE_ENTRIES_MASK as
an unsigned long long.
Note that this bug doesn't manifest itself in practice because
address spaces larger than 39 bits are not yet supported in the
Trusted Firmware.
Change-Id: I955fd263ecb691ca94b29b9c9f576008ce1d87ee
The only case in which regions can now overlap is if they are
identity mapped or they have the same virtual to physical address
offset (identity mapping is just a particular case of the latter).
They must overlap completely (i.e. one of them must be completely
inside the other one) and not cover the same area.
This allow future enhancements to the xlat_tables library without
having to support unnecessarily complex edge cases.
Outer regions are now sorted by mmap_add_region() before inner
regions with the same base virtual address for consistency: all
regions contained inside another one must be placed after the outer
one in the list.
If an inner region has the same attributes as the outer ones it will
be merged when creating the tables with init_xlation_table(). This
cannot be done as regions are added because there may be cases where
adding a region makes previously mergeable regions no longer
mergeable.
If the attributes of an inner region are different than the outer
region, new pages will be generated regardless of how "restrictive"
they are. For example, RO memory is more restrictive than RW. The
old implementation would give priority to RO if there is an overlap,
the new one doesn't.
NOTE: THIS IS THEORETICALLY A COMPATABILITY BREAK FOR PLATFORMS THAT
USE THE XLAT_TABLES LIBRARY IN AN UNEXPECTED WAY. PLEASE RAISE A
TF-ISSUE IF YOUR PLATFORM IS AFFECTED.
Change-Id: I75fba5cf6db627c2ead70da3feb3cc648c4fe2af
The AArch32 long descriptor format and the AArch64 descriptor format
correspond to each other which allows possible sharing of xlat_tables
library code between AArch64 and AArch32. This patch refactors the
xlat_tables library code to seperate the common functionality from
architecture specific code. Prior to this patch, all of the xlat_tables
library code were in `lib/aarch64/xlat_tables.c` file. The refactored code
is now in `lib/xlat_tables/` directory. The AArch64 specific programming
for xlat_tables is in `lib/xlat_tables/aarch64/xlat_tables.c` and the rest
of the code common to AArch64 and AArch32 is in
`lib/xlat_tables/xlat_tables_common.c`. Also the data types used in
xlat_tables library APIs are reworked to make it compatible between AArch64
and AArch32.
The `lib/aarch64/xlat_tables.c` file now includes the new xlat_tables
library files to retain compatibility for existing platform ports.
The macros related to xlat_tables library are also moved from
`include/lib/aarch64/arch.h` to the header `include/lib/xlat_tables.h`.
NOTE: THE `lib/aarch64/xlat_tables.c` FILE IS DEPRECATED AND PLATFORM PORTS
ARE EXPECTED TO INCLUDE THE NEW XLAT_TABLES LIBRARY FILES IN THEIR MAKEFILES.
Change-Id: I3d17217d24aaf3a05a4685d642a31d4d56255a0f
lib/aarch64/xlat_helpers.c defines helper functions to build
translation descriptors, but no common code or upstream platform
port uses them. As the rest of the xlat_tables code evolves, there
may be conflicts with these helpers, therefore this code should be
removed.
Change-Id: I9f5be99720f929264818af33db8dada785368711
The assembler helper function `print_revision_warning` is used when a
CPU specific operation is enabled in the debug build (e.g. an errata
workaround) but doesn't apply to the executing CPU's revision/part number.
However, in some cases the system integrator may want a single binary to
support multiple platforms with different IP versions, only some of which
contain a specific erratum. In this case, the warning can be emitted very
frequently when CPUs are being powered on/off.
This patch modifies this warning print behaviour so that it is emitted only
when LOG_LEVEL >= LOG_LEVEL_VERBOSE. The `debug.h` header file now contains
guard macros so that it can be included in assembly code.
Change-Id: Ic6e7a07f128dcdb8498a5bfdae920a8feeea1345
The current translation table code maps in a series of regions, zeroing
the unmapped table entries before and in between the mapped regions. It
doesn't, however, zero the unmapped entries after the last mapped
region, leaving those entries at whatever value that memory has
initially.
This is bad because those values can look like valid translation table
entries, pointing to valid physical addresses. The CPU is allowed to do
speculative reads from any such addresses. If the addresses point to
device memory, the results can be unpredictable.
This patch zeroes the translation table entries following the last
mapped region, ensuring all table entries are either valid or zero
(invalid).
In addition, it limits the value of ADDR_SPACE_SIZE to those allowed by
the architecture and supported by the current code (see D4.2.5 in the
Architecture Reference Manual). This simplifies this patch a lot and
ensures existing code doesn't do unexpected things.
Change-Id: Ic28b6c3f89d73ef58fa80319a9466bb2c7131c21
At the moment, the memory translation library allows to create memory
mappings of 2 types:
- Device nGnRE memory (named MT_DEVICE in the library);
- Normal, Inner Write-back non-transient, Outer Write-back
non-transient memory (named MT_MEMORY in the library).
As a consequence, the library code treats the memory type field as a
boolean: everything that is not device memory is normal memory and
vice-versa.
In reality, the ARMv8 architecture allows up to 8 types of memory to
be used at a single time for a given exception level. This patch
reworks the memory attributes such that the memory type is now defined
as an integer ranging from 0 to 7 instead of a boolean. This makes it
possible to extend the list of memory types supported by the memory
translation library.
The priority system dictating memory attributes for overlapping
memory regions has been extended to cope with these changes but the
algorithm at its core has been preserved. When a memory region is
re-mapped with different memory attributes, the memory translation
library examines the former attributes and updates them only if
the new attributes create a more restrictive mapping. This behaviour
is unchanged, only the manipulation of the value has been modified
to cope with the new format.
This patch also introduces a new type of memory mapping in the memory
translation library: MT_NON_CACHEABLE, meaning Normal, Inner
Non-cacheable, Outer Non-cacheable memory. This can be useful to map
a non-cacheable memory region, such as a DMA buffer for example.
The rules around the Execute-Never (XN) bit in a translation table
for an MT_NON_CACHEABLE memory mapping have been aligned on the rules
used for MT_MEMORY mappings:
- If the memory is read-only then it is also executable (XN = 0);
- If the memory is read-write then it is not executable (XN = 1).
The shareability field for MT_NON_CACHEABLE mappings is always set as
'Outer-Shareable'. Note that this is not strictly needed since
shareability is only relevant if the memory is a Normal Cacheable
memory type, but this is to align with the existing device memory
mappings setup. All Device and Normal Non-cacheable memory regions
are always treated as Outer Shareable, regardless of the translation
table shareability attributes.
This patch also removes the 'ATTR_SO' and 'ATTR_SO_INDEX' #defines.
They were introduced to map memory as Device nGnRnE (formerly called
"Strongly-Ordered" memory in the ARMv7 architecture) but were not
used anywhere in the code base. Removing them avoids any confusion
about the memory types supported by the library.
Upstream platforms do not currently use the MT_NON_CACHEABLE memory
type.
NOTE: THIS CHANGE IS SOURCE COMPATIBLE BUT PLATFORMS THAT RELY ON THE
BINARY VALUES OF `mmap_attr_t` or the `attr` argument of
`mmap_add_region()` MAY BE BROKEN.
Change-Id: I717d6ed79b4c845a04e34132432f98b93d661d79
All C files of stdlib were included into std.c, which was the file
that the Makefile actually compiled. This is a poor way of compiling
all the files and, while it may work fine most times, it's
discouraged.
In this particular case, each C file included its own headers, which
were later included into std.c. For example, this caused problems
because a duplicated typedef of u_short in both subr_prf.c and
types.h. While that may require an issue on its own, this kind of
problems are avoided if all C files are as independent as possible.
Change-Id: I9a7833fd2933003f19a5d7db921ed8542ea2d04a
In the Cortex-A35/A53/A57 CPUs library code, some of the CPU specific
reset operations are skipped if they have already been applied in a
previous invocation of the reset handler. This precaution is not
required, as all these operations can be reapplied safely.
This patch removes the unneeded test-before-set instructions in
the reset handler for these CPUs.
Change-Id: Ib175952c814dc51f1b5125f76ed6c06a22b95167
The LDNP/STNP instructions as implemented on Cortex-A53 and
Cortex-A57 do not behave in a way most programmers expect, and will
most probably result in a significant speed degradation to any code
that employs them. The ARMv8-A architecture (see Document ARM DDI
0487A.h, section D3.4.3) allows cores to ignore the non-temporal hint
and treat LDNP/STNP as LDP/STP instead.
This patch introduces 2 new build flags:
A53_DISABLE_NON_TEMPORAL_HINT and A57_DISABLE_NON_TEMPORAL_HINT
to enforce this behaviour on Cortex-A53 and Cortex-A57. They are
enabled by default.
The string printed in debug builds when a specific CPU errata
workaround is compiled in but skipped at runtime has been
generalised, so that it can be reused for the non-temporal hint use
case as well.
Change-Id: I3e354f4797fd5d3959872a678e160322b13867a1
The debug prints used to debug translation table setup in xlat_tables.c
used the `printf()` standard library function instead of the stack
optimized `tf_printf()` API. DEBUG_XLAT_TABLE option was used to enable
debug logs within xlat_tables.c and it configured a much larger stack
size for the platform in case it was enabled. This patch modifies these
debug prints within xlat_tables.c to use tf_printf() and modifies the format
specifiers to be compatible with tf_printf(). The debug prints are now enabled
if the VERBOSE prints are enabled in Trusted Firmware via LOG_LEVEL build
option.
The much larger stack size definition when DEBUG_XLAT_TABLE is defined
is no longer required and the platform ports are modified to remove this
stack size definition.
Change-Id: I2f7d77ea12a04b827fa15e2adc3125b1175e4c23
Migrate all direct usage of __attribute__ to usage of their
corresponding macros from cdefs.h.
e.g.:
- __attribute__((unused)) -> __unused
Signed-off-by: Soren Brinkmann <soren.brinkmann@xilinx.com>
This patch adds support for ARM Cortex-A35 processor in the CPU
specific framework, as described in the Cortex-A35 TRM (r0p0).
Change-Id: Ief930a0bdf6cd82f6cb1c3b106f591a71c883464
On the ARMv8 architecture, cache maintenance operations by set/way on the last
level of integrated cache do not affect the system cache. This means that such a
flush or clean operation could result in the data being pushed out to the system
cache rather than main memory. Another CPU could access this data before it
enables its data cache or MMU. Such accesses could be serviced from the main
memory instead of the system cache. If the data in the sysem cache has not yet
been flushed or evicted to main memory then there could be a loss of
coherency. The only mechanism to guarantee that the main memory will be updated
is to use cache maintenance operations to the PoC by MVA(See section D3.4.11
(System level caches) of ARMv8-A Reference Manual (Issue A.g/ARM DDI0487A.G).
This patch removes the reliance of Trusted Firmware on the flush by set/way
operation to ensure visibility of data in the main memory. Cache maintenance
operations by MVA are now used instead. The following are the broad category of
changes:
1. The RW areas of BL2/BL31/BL32 are invalidated by MVA before the C runtime is
initialised. This ensures that any stale cache lines at any level of cache
are removed.
2. Updates to global data in runtime firmware (BL31) by the primary CPU are made
visible to secondary CPUs using a cache clean operation by MVA.
3. Cache maintenance by set/way operations are only used prior to power down.
NOTE: NON-UPSTREAM TRUSTED FIRMWARE CODE SHOULD MAKE EQUIVALENT CHANGES IN
ORDER TO FUNCTION CORRECTLY ON PLATFORMS WITH SUPPORT FOR SYSTEM CACHES.
FixesARM-software/tf-issues#205
Change-Id: I64f1b398de0432813a0e0881d70f8337681f6e9a
Varun Wadekar
danh-arm
Douglas Raillard
davidcunado-arm
Jeenu Viswambharan
dp-arm
Antonio Nino Diaz
Sandrine Bailleux
Masahiro Yamada
Soby Mathew
Yatharth Kochar
Achin Gupta
Naga Sureshkumar Relli
Kristina Martsenko
Soren Brinkmann