After a CPU comes online, clk_set_rate() silently refuses to change said CPU's
frequency to the frequency it was running at before going offline (since it
thinks that we are setting the same frequency redundantly). By default, each CPU
runs at its minimum frequency when coming online, so if the governor decides to
keep a CPU running at the frequency it used before going offline, then
clk_set_rate() will ignore the frequency change request and the CPU will stay
stuck running at its minimum frequency for a potentially long period of time
(i.e. until the governor decides to change the frequency to something
different). This can cause severe lag when a device is woken up from deep sleep.
In order to prevent a CPU from being stuck at its default frequency for a
potentially long period of time, set the CLKFLAG_NO_RATE_CACHE flag so that the
governor's frequency change requests will always be honored right after a CPU
comes online.
Note that this requires a CPU's component clocks to be using CLKFLAG_NO_RATE_CACHE
as well in order to fix the issue.
Signed-off-by: Sultanxda <sultanxda@gmail.com>
__cpufreq_driver_target() checks if policy->cur is same as target_freq
without holding any lock. This function is used by governor to
directly set CPU frequency. Governor calling this function can't hold
any CPUfreq framework locks due to deadlock possibility.
However, this results in a race condition where one thread could see
a stale policy->cur while another thread is changing CPU frequency.
Thread A: Governor calls __cpufreq_driver_target(), starts increasing
frequency but hasn't sent out CPUFREQ_POSTCHANGE notification yet.
Thread B: Some other driver (could be thermal mitigation) starts
limiting frequency using cpufreq_update_policy(). Every limits are
applied to policy->min/max and final policy->max happens to be same as
policy->cur. __cpufreq_driver_target() simply returns 0.
Thread A: Governor finish scaling and now policy->cur violates
policy->max and could last forever until next CPU frequency scaling
happens.
Shifting the responsibility of checking policy->cur and target_freq
to CPUfreq device driver would resolve the race as long as the device
driver holds a common mutex.
Change-Id: I6f943228e793a4a4300c58b3ae0143e09ed01d7d
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Different structures might need to be saved and restored based on
different scheduling policies of current thread. Saving and restoring
priority using scheduler APIs is very fragile due to potential changes
in scheduler code. In addition, the priority change doesn't
provide any starvation guarantee because threads can be preempted
before the priority change.
Therefore remove save and restore of priority to avoid potential bugs
when scheduler API changes. Caller will now be responsible for setting
the right priority for their CPU frequency scaling workqueue/thread.
Change-Id: I2a5d8599e75c0c4aa902df3214c17ab2b13dc9a9
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
qcom-cpufreq blocks CPU frequency change request during suspend, because
its dependencies might be suspended. Thus a freq change request would
fail silently, and CPU clock won't change until first frequency update
is requested after system comes out of suspend. This creates a period
when thermal driver cannot perform frequency mitigation, even though
policy->min/max have been correctly updated.
Check each online CPU's policy during resume to correct any frequency
violation as soon as possible.
Change-Id: I3be79cf91e7d5e361314020c9806b770823c0b72
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
It's no longer a requirement to pin frequency change on the CPU that
is being scaled. Therefore, there is no longer a need for per-cpu
workqueue in qcom-cpufreq. Remove the workqueue.
Change-Id: Ic6fd7f898fa8b1b1226a178b04530c24f0398daa
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
MSM_CPU_FREQ_SET_MIN_MAX and related Kconfigs are deprecated. Purge
them from Kconfig and qcom-cpufreq.
Change-Id: I8ac786c155c7e235154b60c79f97d76ea15dace2
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
qcom-cpufreq use a per-cpu work to scale CPU frequency. For CPUs
in sync, only the first CPU that is plugged in has its work
initialized. When that CPU goes offline, it hands over policy
to another CPU, which doesn't have its work initialized. If CPU
scaling happens then, an uninitialized work will be queued onto
workqueue, causing a crash.
Initialize workqueue for all CPUs in sync in msm_cpufreq_init().
Change-Id: I4c3bc08182c4088de4a3675c47a8e0e10c8e4f47
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Some SoCs contain CPU clock trees the elements of which are
gated off when the CPUs enter power collapse. This includes
sources of glitch free muxes; therefore when the CPUs enter
power collapse, those muxes cannot be switched.
Now in the cpufreq driver, the CPU clocks are disabled in
the CPU_DEAD notifier, which implies that the CPU muxes
are switched to a safe source *after* the CPUs are power
collapsed. However, the source of the GFMUX may already be
turned off, causing the mux to get stuck.
Ideally, the mux should allow a static switch, since the
clock to the CPU is gated. Some implementations do not
allow this.
Change-Id: I37d3426f20250c59756a0b55d1284efad5359a23
Signed-off-by: Vikram Mulukutla <markivx@codeaurora.org>
Even if all CPUs share same frequency table, there might still be use
case where governor-per-policy is useful. Move it before returning
from parsing common CPU freq table.
Change-Id: I0254dd4d09b6ea6595a183207da036b224c90f04
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
If CPUs have different performance characteristics, it makes sense to have
different CPU frequency tables for each unique CPU clock. Add support for
parsing different frequency tables for each unique CPU clock.
Change-Id: Ia9b064dfd1f84320d26dd41070339cec548abe7c
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
CPUfreq device is the provider of CPU frequency table, and
it should always keep track of freq tables for every CPU itself.
cpufreq_frequency_get_table() is an API for frequency table user.
qcom-cpufreq should not use this API to obtain CPU freq table in
the init function. Otherwise, the following sequence will lead to init
failure.
Assume CPU 0 and 1 are under control of same policy and online. Say
policy->cpu is CPU 0.
1) Hot unplug CPU0
update_policy_cpu() will transfer ownership of policy to CPU1
cpufreq_frequency_table_update_policy_cpu() is called during
the process to set CPU1's freq table to be the same as CPU0's freq
table. CPU0's freq table is now NULL.
2) Hot unplug CPU1
All CPUs in policy are now offline.
3) Hot plug in CPU0
cpufreq_driver->init() is called for initialization. If this init
function calls cpufreq_frequency_get_table() on CPU0, the result
will be NULL.
Current qcom-cpufreq checks return value of cpufreq_frequency_get_table()
and returns an error in init.
Remove the usage of cpufreq_frequency_get_table() and fill in frequency
table for policy->cpu in msm_cpufreq_init().
Change-Id: I2d41af776938c31b4444c9f280a341dd1bed1548
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
CPUs under different policies might need different governor tunables
on certain targets. Enable support for this feature by parsing
"qcom,governor-per-policy" DT property.
Change-Id: I670135822d67d392030b54f40feab9a3aef11542
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
The devfreq cpufreq governor and the devfreq devices for CPUBW and cache
now handle CPUBW and cache frequency scaling based on CPU frequency. So,
delete all the unused code in the cpufreq driver that relates to scaling
CPUBW and cache.
Also update the msm-cpufreq DT format/data accordingly.
Change-Id: I30bf7d8478c964ffa300dd78039cd8c280ec63fc
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
The CPU_DOWN_PREPARE notifier in cpufreq.c already takes care
of performing a CPUFREQ_GOV_STOP, which waits for in-progress
frequency switches to complete and prevents any additional
frequency switches from being performed. Remove the redundant
code for accomplishing the same thing from qcom-cpufreq.
Besides being good cleanup, this change should also resolve
a potential deadlock in out-of-memory conditions. The scenario
arises if CPU1, executing in msm_cpufreq_target(), holds the
qcom-cpufreq suspend_mutex while waiting for the completion
at the end of that same function. In an out-of-memory condition,
the work performing that completion may not be able to
immediately run.
At the same time, CPU2 may be executing CPU_DOWN_PREPARE hotplug
notifiers. It will be holding the global hotplug mutex at this
time, and will eventually become blocked in
msm_cpufreq_cpu_callback() while waiting for the CPU1 to release
the device_suspended lock.
If invoked, the out-of-memory killer will attempt to acquire
the hotplug mutex via get_online_cpus() as part of
out_of_memory(). This lock is already held by CPU2. CPU2 will
not release the lock until the device_suspended lock is released
by CPU1. The device_suspended lock will not be release until the
completion in msm_cpufreq_target() is marked as complete, which
won't happen until the out-of-memory killer is able to free
some memory. Nasty deadlock.
Clearly, the out-of-memory condition itself needs to be resolved,
but there is no harm in resolving the deadlock also.
Change-Id: I5f59864aeb12a8c2aa81de16446af09a6d514da4
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
For targets with synchronous CPU clocks, only CPU0 has a clock pointer in
cpu_clks[]. This forces CPU0 to be the last CPU to be hotplugged out, or go
to certain low power mode that needs to disable CPU clocks.
Remove this requirement by adding clock pointer for all CPUs. Drivers
that ask for CPU clocks will get same clock for CPUs in sync.
Change-Id: Ib97af2c3b0fdbc8dbb78364178225cc64ed02036
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Change 2b63b10 qcom-cpufreq: Block hotplug until cpufreq is ready
intended to block hotplug until qcom-cpufreq driver has got all
clocks. However, its implementation still leaves a small window when
CPU 0 has got its clock but others haven't. If a hotplug happens on
another CPU that doesn't have its clock pointer, refcount of that CPU
clock will again be wrong.
Fix the issue by introducing a ready flag.
Change-Id: I1e05b063b0584088f717b82709a3f1d55bc6561b
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
It is a requirement of PLLs in MSM SoCs that the PLL power
supplies be enabled while the PLL is running. If the PLL is
locked and outputs enabled, turning off the regulator(s)
supplying the PLL will cause the PLL to enter an unpredictable
state.
Now in the CPU_UP_PREPARE notifier, the CPU clocks are prepared
and enabled, causing the source HFPLLs to also turn on. Note
that the CPU isn't clocked yet. It is possible that execution is
pre-empted and the CPU running the notifier enters power
collapse. If all other CPUs also enter power collapse, then it
is possible for an RPM notification to go out, allowing the RPM
to transition the Apps processor to its sleep set. This can
result in the HFPLL supplies being turned off while the HFPLL is
running, violating the requirement mentioned above. Once the CPU
is unclamped, the CPU is effectively unclocked, due to the HFPLL
being in an unknown state.
There is a check that is enabled in the PM code's CPU_UP_PREPARE
notifier callback. This check ensures that the problematic RPM
notification cannot occur until the core that is being brought
online also enters power collapse. However, there is no ordering
guarantee between that PM's hotplug notifier callback's execution
and the cpufreq hotplug notifier callback's execution. This
ordering depends on program link order, which is unreliable.
It is necessary to ensure that once the HFPLL is enabled, the
RPM cannot transition apps to its sleep set. Move the enabling
of the CPU clocks to the CPU_STARTING notifier, which runs on
the CPU bring brought online. The CPU_STARTING notifier is
guaranteed to run after the CPU_UP_PREPARE notifier, which
implies that the aforementioned do-not-notify-rpm check is
executed *before* the HFPLL is enabled. Therefore even if all
cores enter power collapse after the HFPLL is enabled, the
HFPLL supplies are guaranteed to stay on, or the CPU clock
is switched to the safe source and the HFPLL is turned off.
CRs-Fixed: 622738
Change-Id: I136841405806c07e755919859e649ded0c719abb
Signed-off-by: Vikram Mulukutla <markivx@codeaurora.org>
Hotplug before qcom-cpufreq is ready could lead to inconsistent CPU
clock state. Block hotplug by returning NOTIFY_BAD in hotplug callback
until qcom-cpufreq is probed.
Change-Id: I72a2f98c083c9b21b95ecafdb5a5be7a7682e842
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Architecutural changes in the ARM Linux kernel tree mandate the
eventual removal of the mach-* directories. Move mach/cpufreq to
driver/cpufreq/. Also move related header to include/linux.
Change-Id: I6dcf69e275b7ca7ba913e945353a42f0d6321731
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Sultanxda