When acquiring or releasing the power domain locks for a given CPU the
parent nodes are looked up by walking the up the PD tree list on both the
acquire and release path, only one set of lookups is needed. Fetch the
parent nodes first and pass this list into both the acquire and release
functions to avoid the double lookup.
This also allows us to not have to do this lookup after coherency has
been exited during the core power down sequence. The shared struct
psci_cpu_pd_nodes is not placed in coherent memory like is done
for psci_non_cpu_pd_nodes and doing so would negatively affect
performance. With this patch we remove the need to have it in coherent
memory by moving the access out of psci_release_pwr_domain_locks().
Signed-off-by: Andrew F. Davis <afd@ti.com>
Change-Id: I7b9cfa9d31148dea0f5e21091c8b45ef7fe4c4ab
Enforce full include path for includes. Deprecate old paths.
The following folders inside include/lib have been left unchanged:
- include/lib/cpus/${ARCH}
- include/lib/el3_runtime/${ARCH}
The reason for this change is that having a global namespace for
includes isn't a good idea. It defeats one of the advantages of having
folders and it introduces problems that are sometimes subtle (because
you may not know the header you are actually including if there are two
of them).
For example, this patch had to be created because two headers were
called the same way: e0ea0928d5 ("Fix gpio includes of mt8173 platform
to avoid collision."). More recently, this patch has had similar
problems: 46f9b2c3a2 ("drivers: add tzc380 support").
This problem was introduced in commit 4ecca33988 ("Move include and
source files to logical locations"). At that time, there weren't too
many headers so it wasn't a real issue. However, time has shown that
this creates problems.
Platforms that want to preserve the way they include headers may add the
removed paths to PLAT_INCLUDES, but this is discouraged.
Change-Id: I39dc53ed98f9e297a5966e723d1936d6ccf2fc8f
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
psci_set_power_off_state only initializes a local variable, so there
isn't any reason why it should be done while the lock is held.
Change-Id: I1c62f4cd5d860d102532e5a5350152180d41d127
Signed-off-by: Roberto Vargas <roberto.vargas@arm.com>
This patch enables the 'sign-compare' flag, to enable warning/errors
for comparisons between signed/unsigned variables. The warning has
been enabled for all the Tegra platforms, to start with.
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>
To make software license auditing simpler, use SPDX[0] license
identifiers instead of duplicating the license text in every file.
NOTE: Files that have been imported by FreeBSD have not been modified.
[0]: https://spdx.org/
Change-Id: I80a00e1f641b8cc075ca5a95b10607ed9ed8761a
Signed-off-by: dp-arm <dimitris.papastamos@arm.com>
The current PSCI implementation can apply certain optimizations upon the
assumption that all PSCI participants are cache-coherent.
- Skip performing cache maintenance during power-up.
- Skip performing cache maintenance during power-down:
At present, on the power-down path, CPU driver disables caches and
MMU, and performs cache maintenance in preparation for powering down
the CPU. This means that PSCI must perform additional cache
maintenance on the extant stack for correct functioning.
If all participating CPUs are cache-coherent, CPU driver would
neither disable MMU nor perform cache maintenance. The CPU being
powered down, therefore, remain cache-coherent throughout all PSCI
call paths. This in turn means that PSCI cache maintenance
operations are not required during power down.
- Choose spin locks instead of bakery locks:
The current PSCI implementation must synchronize both cache-coherent
and non-cache-coherent participants. Mutual exclusion primitives are
not guaranteed to function on non-coherent memory. For this reason,
the current PSCI implementation had to resort to bakery locks.
If all participants are cache-coherent, the implementation can
enable MMU and data caches early, and substitute bakery locks for
spin locks. Spin locks make use of architectural mutual exclusion
primitives, and are lighter and faster.
The optimizations are applied when HW_ASSISTED_COHERENCY build option is
enabled, as it's expected that all PSCI participants are cache-coherent
in those systems.
Change-Id: Iac51c3ed318ea7e2120f6b6a46fd2db2eae46ede
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>
The PSCI implementation performs cache maintenance operations on its
data structures to ensure their visibility to both cache-coherent and
non-cache-coherent participants. These cache maintenance operations
can be skipped if all PSCI participants are cache-coherent. When
HW_ASSISTED_COHERENCY build option is enabled, we assume PSCI
participants are cache-coherent.
For usage abstraction, this patch introduces wrappers for PSCI cache
maintenance and barrier operations used for state coordination: they are
effectively NOPs when HW_ASSISTED_COHERENCY is enabled, but are
applied otherwise.
Also refactor local state usage and associated cache operations to make
it clearer.
Change-Id: I77f17a90cba41085b7188c1345fe5731c99fad87
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>
This patch introduces the following three platform interfaces:
* void plat_psci_stat_accounting_start(const psci_power_state_t *state_info)
This is an optional hook that platforms can implement in order
to perform accounting before entering a low power state. This
typically involves capturing a timestamp.
* void plat_psci_stat_accounting_stop(const psci_power_state_t *state_info)
This is an optional hook that platforms can implement in order
to perform accounting after exiting from a low power state. This
typically involves capturing a timestamp.
* u_register_t plat_psci_stat_get_residency(unsigned int lvl,
const psci_power_state_t *state_info,
unsigned int last_cpu_index)
This is an optional hook that platforms can implement in order
to calculate the PSCI stat residency.
If any of these interfaces are overridden by the platform, it is
recommended that all of them are.
By default `ENABLE_PSCI_STAT` is disabled. If `ENABLE_PSCI_STAT`
is set but `ENABLE_PMF` is not set then an alternative PSCI stat
collection backend must be provided. If both are set, then default
weak definitions of these functions are provided, using PMF to
calculate the residency.
NOTE: Previously, platforms did not have to explicitly set
`ENABLE_PMF` since this was automatically done by the top-level
Makefile.
Change-Id: I17b47804dea68c77bc284df15ee1ccd66bc4b79b
Signed-off-by: dp-arm <dimitris.papastamos@arm.com>
Testing showed that the time spent in a cluster power down
operation is dominated by cache flushes. Add two more timestamps
in runtime instrumentation to keep track of the time spent
flushing the L1/L2 caches.
Change-Id: I4c5a04e7663543225a85d3c6b271d7b706deffc4
Signed-off-by: dp-arm <dimitris.papastamos@arm.com>
In order to quantify the overall time spent in the PSCI software
implementation, an initial collection of PMF instrumentation points
has been added.
Instrumentation has been added to the following code paths:
- Entry to PSCI SMC handler. The timestamp is captured as early
as possible during the runtime exception and stored in memory
before entering the PSCI SMC handler.
- Exit from PSCI SMC handler. The timestamp is captured after
normal return from the PSCI SMC handler or if a low power state
was requested it is captured in the bl31 warm boot path before
return to normal world.
- Entry to low power state. The timestamp is captured before entry
to a low power state which implies either standby or power down.
As these power states are mutually exclusive, only one timestamp
is defined to describe both. It is possible to differentiate between
the two power states using the PSCI STAT interface.
- Exit from low power state. The timestamp is captured after a standby
or power up operation has completed.
To calculate the number of cycles spent running code in Trusted Firmware
one can perform the following calculation:
(exit_psci - enter_psci) - (exit_low_pwr - enter_low_pwr).
The resulting number of cycles can be converted to time given the
frequency of the counter.
Change-Id: Ie3b8f3d16409b6703747093b3a2d5c7429ad0166
Signed-off-by: dp-arm <dimitris.papastamos@arm.com>
This patch moves the PSCI services and BL31 frameworks like context
management and per-cpu data into new library components `PSCI` and
`el3_runtime` respectively. This enables PSCI to be built independently from
BL31. A new `psci_lib.mk` makefile is introduced which adds the relevant
PSCI library sources and gets included by `bl31.mk`. Other changes which
are done as part of this patch are:
* The runtime services framework is now moved to the `common/` folder to
enable reuse.
* The `asm_macros.S` and `assert_macros.S` helpers are moved to architecture
specific folder.
* The `plat_psci_common.c` is moved from the `plat/common/aarch64/` folder
to `plat/common` folder. The original file location now has a stub which
just includes the file from new location to maintain platform compatibility.
Most of the changes wouldn't affect platform builds as they just involve
changes to the generic bl1.mk and bl31.mk makefiles.
NOTE: THE `plat_psci_common.c` FILE HAS MOVED LOCATION AND THE STUB FILE AT
THE ORIGINAL LOCATION IS NOW DEPRECATED. PLATFORMS SHOULD MODIFY THEIR
MAKEFILES TO INCLUDE THE FILE FROM THE NEW LOCATION.
Change-Id: I6bd87d5b59424995c6a65ef8076d4fda91ad5e86
Andrew F. Davis
Antonio Nino Diaz
Roberto Vargas
Varun Wadekar
dp-arm
Jeenu Viswambharan
Soby Mathew