When ufshcd_host_reset_and_restore() is called, it is possible
that ufshcd_probe_hba() fails and return an error value.
However, the later call to crypto_engine_reset() might succeed
and override the previous error value.
This will result in ufshcd_host_reset_and_restore() returning
success while in fact a serious error may have occured which
should prevent us from continuing in the normal sequence.
Instead, call crypto_engine_reset() only if ufshcd_probe_hba()
was successfull.
Change-Id: I0462f674b8750bfb9e0657cb37833aa2bd2aa4f0
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
A race condition exists between request requeueing and scsi layer
error handling:
When UFS driver queuecommand returns a busy status for a request,
it will be requeued and its tag will be freed and set to -1.
At the same time it is possible that the request will timeout and
scsi layer will start error handling for it. The scsi layer reuses
the request and its tag to send error related commands to the device,
however its tag is no longer valid.
As this request was never really sent to the device, there is no
point to start error handling with the device.
Implement the scsi error handling timeout callback and bypass SCSI
error handling for request that were not actually sent to the device.
For such requests simply reset the block layer timer. Otherwise, let
SCSI layer perform the usual error handling.
Change-Id: Iebed05c95a61f7dcf0a177e5c371b6575ead54d8
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
When running benchmarks on a system and inspecting the effect of
performance related changes it can be useful to know in a
simple way how long it takes for UFS requests to complete.
Add statistics for UFS requests completion times and measure
minimum, maximum and average completion times for all requests
and also according to request type: read, write and flush.
Change-Id: I85639d629939e3ae3bba6de065ceadae0da3dc99
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
rq_data_dir(rq) will always be either READ or WRITE so conditional
logic could never classify a request as a flush request causing
flush statistics to always be zero.
In addition it is neccesary to check the request type is FS in
order not to count other request type as read/write.
Change-Id: I678203ba47f8296caefa3f197566f2c65abdf059
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
Whenever some UFS failure occurs the driver prints the UFS
registers in order to help with analysis of the failure.
However this may not be sufficient in some cases, so having
the host controller state as it is represented and managed in
the driver will contribute to analysis efforts.
Added prints of various fields in the hba struct which may be
of interest.
Change-Id: I8385a6b40f6937f07d8f0f0c51f5cf36babf7515
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
hba->vops is allowed to be NULL, depending on the specific UFS
host driver. Add checks to verify hba->vops is not null to protect
against null pointer access.
Change-Id: I7507c0842f44cbc3600645249a4163277710dccf
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
Use the user-defined sector_range instead of the previously hard-coded
TEST_MAX_SECTOR_RANGE. If the user does not supply a sector_range, this
variable will default to 512MiB.
For the long sequential tests, sector_range will define the size of the
sequential I/O to submit. For the long random tests, sector_range will
define the range in which to submit random I/O as well as the total size
of the random I/O which is defined by the combination of sector_range and
LONG_RAND_TEST_REQ_RATIO.
Change-Id: Ifc7332e6def75c49448aadbebd35b7b9b3903447
Signed-off-by: Lee Susman <lsusman@codeaurora.org>
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
Unit tests submit large requests of 512KB made of 128 bios.
Current allocation was done via kmalloc which may not be able
to allocate such a large buffer which is also physically contiguous.
Using kmalloc to allocate each bio separately is also problematic as
it might not be page aligned. Some bio may end up using more than a
single memory segment which will fail the mapping of the bios to
the request which supports up to 128 physical segments.
To avoid such failure, allocate a separate page for each bio
(bio size is single page size).
Change-Id: Id0da394d458942e093d924bc7aa23aa3231cdca7
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
Current test-iosched design enables running only a single test
for a single block device.
This change modifies the test-iosched framework to allow running
several tests on several block devices.
Change-Id: I051d842733873488b64e89053d9c4e30e1249870
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
Multi-query test fails on write descriptor requests becuase it
tries to write to the unit descriptor which is read-only according
to the spec.
The only writeable descriptors are the configuration and OEM ID
string descriptors which will be non-trivial to use.
Avoid sending write descriptor query for the time being, and replace
it with a read descriptor operation.
Change-Id: Ic121d9ff20f144e724df5436f9dbd44cde72e97d
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
Add support for exposing debug registers via debugfs. This
can be used for runtime debugging of failures/errors without
the need to add more prints to the console.
In order to avoid unnecessary code duplication, the function
for dumping registers to a file (in debugfs.c) has been
slightly modified and exposed as an external function.
Signed-off-by: Dov Levenglick <dovl@codeaurora.org>
Change-Id: I77f9f3a6d041f805a93d095446f3e8077977036f
Adds support for configuring and reading the test bus and debug
registers. The configuration is controlled by debugfs. Reading can
be triggered either by debugfs or by the kernel code.
Change-Id: I943e3c1b1e383a91a8abbb8dab9714434c56b6f5
Signed-off-by: Dov Levenglick <dovl@codeaurora.org>
Add ICE debug prints for debugging purposes. These prints
will be shown as part of dumping registers during error handling.
Enabling can be controlled by debugfs.
Change-Id: I7d070cc8df099e6c526997d39973a3e1c161091b
Signed-off-by: Dov Levenglick <dovl@codeaurora.org>
Provide a mechanism for the userspace to suppress specific
debug prints via debugfs. This is useful in order to avoid
cases where too much printing would cause watchdog timers
to expire.
Change-Id: I2500b7621b631e260d98595ed8cfe7d5a496dc10
Signed-off-by: Dov Levenglick <dovl@codeaurora.org>
On some device, in some cases, the initialization time is
uncharacteristically long. This is within the standard which states
in section 13.4.6.1.2 (Dynamic Capacity Procedural Flow):
"The device initialization may take a time longer than normal
initialization..."
Therefore, making the loop pending on fDeviceInit to clear
longer.
CRs-Fixed: 776748
Change-Id: Ida491467da63ac9630d3ecc9b8f5c3e9ac5187c2
Signed-off-by: Dov Levenglick <dovl@codeaurora.org>
When sending a query to the device returns with a timeout error,
we clear the corresponding bit in the DOORBELL register but
we don't clear the outstanding_request field as we should.
This patch fixes this bug.
Change-Id: Ia7e0aa29b0d822742654ba206a5837b9091fdb4a
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
As of HW major version 2, a new bit 'UFS_DEV_REF_CLK_EN' was added
to UFS_CFG1 register which needs to be set as part of hibernate
enter/exit sequences during suspend/resume.
Change-Id: I66a6a75dc5a1cf130b1cee90ae20f9f950edfb3a
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
Currently vendor specific callback clk_scale_notify() is called even
if clock scaling state is not changed, this change optimizes it.
Change-Id: I5887effa43c957cc4fe434a239419353204de6d3
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
If we write REG_UFS_SYS1CLK_1US & REG_UFS_TX_SYMBOL_CLK_NS_US registers
multiple times (especially when UFS symbol clocks are not running) then
it might make the HW to go back to default value of these registers.
This change updates above registers only if there is a change in these
registers value.
Change-Id: I7c94c477e52b36b26a1624fa3ddfa75a49916d58
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Provides a mechanism for the userspace to suppress specific
debug prints via the debugfs. This is useful in order to avoid
cases where too much printing would cause watchdog timers to expire.
Change-Id: I1ab068cd616ddac767ec5f30ab130e3b1ac34e15
Signed-off-by: Dov Levenglick <dovl@codeaurora.org>
Before calling ufshcd_send_command() the host spinlock is taken.
In case error is returned ufshcd_release_all() is called, which
in turn attempts to lock the host spinlock again.
Unlocked the host spinlock in case error was returned to
prevent the spinlock recursion.
Change-Id: I630ac8b901f03285a37547bfbcd49ec575e7545a
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
In case the power configuration fails, skip further processing
of the probing function and return immediately. This has 2 reasons:
1. Don't allow the UFS to continue running in PWM
2. Avoid multiple calls to pm_runtime_put_sync() when not
in error handling or power management contexts
Change-Id: I8d6faefadf42385aace903f90ab967c6ddcf7460
Signed-off-by: Dov Levenglick <dovl@codeaurora.org>
ice_init may need UFS clocks to be enabled in order to allow
enabling ICE as part of its initialization. To allow that,
ice_init should be done as part of UFS initialization sequence.
In such a case ice_reset is no longer needed to be called as part of
UFS initialization and should be moved to host reset scenarios.
Change-Id: Iaa851083a9f73045c7ca02d69dd65b3a00b85bff
Signed-off-by: Maya Erez <merez@codeaurora.org>
If device raises the exception event in the response to the commands
sent during the runtime/system PM callbacks, exception event handler
might run in parallel with PM callbacks and may see unclocked register
accesses. This change fixes this issue by not scheduling the exception
event handler while PM callbacks are running.
Change-Id: I8644f50366d5c87ba6a5048e48703b3723ab241b
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Consider following sequence of events:
1. UFS is runtime suspended, link_state = Hibern8, device_state = sleep
2. System goes into system suspend, ufshcd_system_suspend() brings both
link and device to active state and then puts the device in Power_Down
state and link in OFF state.
3. System resumes at some later point in time, ufshcd_system_resume()
doesn't do anything as UFS state is runtime suspended. Note that link
is still on OFF state and device is in Power_Down state.
4. Now system again goes into suspend without any UFS accesses before it.
ufshcd_system_suspend() again brings both link and device to active
state and then puts the device in Power_Down state and link if OFF
state. But it's unnecessary to bring the link & device in active state
as both link and device are already in desired low power states. This
change fixes this issue by adding proper state checks in
ufshcd_system_suspend().
Change-Id: I6ada697f5f844724e0da77fa960401d1d6a1288e
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
The function is called with locked spinlock and disabled interrupts
which is done in a wrapper function ufshcd_pm_qos_hold().
Before flushing the PM QoS voting work spinlock must be released
and interrupts must be restored. As the interrupts flags state is
saved in the wrapper function, we cannot restore the state.
Moving the locking to the internal function fixes this, which
makes the wrapper function redundant so it is deleted.
Change-Id: I167292305a25d2b8c3524dcddb51f5a7109444aa
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
PM QoS request type PM_QOS_REQ_AFFINE_CORES specifies for which CPU
cores the voting is applied to by the cpu affinity mask.
This change defines the cpu mask to be used for the voting in the
device tree node so it can be customized for each target.
Change-Id: I004dea47b42eaf3cdf0489427b2bb894c9982f22
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
DME command delay is used to ensure a sufficient interval between
subsequent commands. The hardware programming guide specified 1ms.
Value of 700us was used as optimization, but it seems to be too short
so DME commands would sometimes fail.
Update the command delay to 1ms as recommended.
Change-Id: I8960f8488c261013ec95ef8c49b58ceda9d065f6
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
ufshcd_queuecommand() vote for the resources in this order: clocks,
pm_qos latency, hibern8 exit. If any of these votes are not already
applied, each one has to be applied asynchronously and in that case we are
releasing all the previously applied resource votes (for example, if
hibern8 exit has to be completed asynchronously, we release the votes for
pm_qos and clocks as well). This is not a optimal solution instead we
should skip scheduling the unvoting work for already voted resources.
Change-Id: Ie700d9b3bf64370a5885787f7313d41adb5b3566
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
ufshcd_queuecommand() function calls ufshcd_hibern8_hold() to bring the
link out of hibern8 asynchronously. If the link was in hibern8 state then
it schedules a work to execute the hibern8 without any timeout and returns
error back to ufshcd_queuecommand(). ufshcd_queuecommand() calls
ufshcd_release() which schedules the clock gating work to be executed
after clock gating timeout (50ms at this moment). Ideally hibern8 exit
work should have finished much before clock gating work starts execution
but it may not be true on heavily loaded system. To protect this race,
hibern8 exit work should make sure that clocks are running (by calling
ufshcd_hold()) before executing hibern8 exit.
Change-Id: I572b26881277886f5ec17b0eed96ee9e59ca1fbf
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Gilad Broner