Patch f598282f51 exposed a problem in
powerpc MSI-X functionality, making network interfaces such as ixgbe
and cxgb3 stop to work when MSI-X is enabled. RX interrupts were not
being generated.
The problem was caused because MSI irq was not being effectively
unmasked after device initialization.
Signed-off-by: Andre Detsch <adetsch@br.ibm.com>
Signed-off-by: Michael Ellerman <michael@ellerman.id.au>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
There's no way for us to express to firmware that we want a
discontiguous, or non-zero based, range of MSI-X entries. So we
must reject such requests.
Signed-off-by: Michael Ellerman <michael@ellerman.id.au>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
There are hardware limitations on the number of available MSIs,
which firmware expresses using a property named "ibm,pe-total-#msi".
This property tells us how many MSIs are available for devices below
the point in the PCI tree where we find the property.
For old firmwares which don't have the property, we assume there are
8 MSIs available per "partitionable endpoint" (PE). The PE can be
found using existing EEH code, which uses the methods described in
PAPR. For our purposes we want the parent of the node that's
identified using this method.
When a driver requests n MSIs for a device, we first establish where
the "ibm,pe-total-#msi" property above that device is, or we find the
PE if the property is not found. In both cases we call this node
the "pe_dn".
We then count all non-bridge devices below the pe_dn, to establish
how many devices in total may need MSIs. The quota is then simply the
total available divided by the number of devices, if the request is
less than or equal to the quota, the request is fine and we're done.
If the request is greater than the quota, we try to determine if there
are any "spare" MSIs which we can give to this device. Spare MSIs are
found by looking for other devices which can never use their full
quota, because their "req#msi(-x)" property is less than the quota.
If we find any spare, we divide the spares by the number of devices
that could request more than their quota. This ensures the spare
MSIs are spread evenly amongst all over-quota requestors.
Signed-off-by: Michael Ellerman <michael@ellerman.id.au>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
If a driver asks for more MSIs than the devices "req#msi(-x)" property,
we currently return -ENOSPC. This doesn't give the driver any chance to
make a new request with a number that might work.
So if "req#msi(-x)" is less than the request, return its value. To be
100% safe, make sure we return an error if req_msi == 0.
Signed-off-by: Michael Ellerman <michael@ellerman.id.au>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
If we can't allocate the requested number of MSIs, we can still tell the
generic code how many we were able to allocate. That can then be passed
onto the driver, allowing it to request that many in future, and
probably succeeed.
Signed-off-by: Michael Ellerman <michael@ellerman.id.au>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
We also need to check that the device isn't using MSI-X in the irq fixup
routine, otherwise we might leave MSI-Xs configured at boot.
Signed-off-by: Michael Ellerman <michael@ellerman.id.au>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Firmware encodes the number of MSI-X requested by a device in a
different property than for MSI. Pull the property name out as a
parameter and share the logic for both cases.
Signed-off-by: Michael Ellerman <michael@ellerman.id.au>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
We need to increment i in the loop that queries what interrupts firmware
gave us, otherwise we'll incorrectly use the first value over and over.
Signed-off-by: Michael Ellerman <michael@ellerman.id.au>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
There are plans afoot to use pci_restore_msi_state() to restore MSI
state after a device reset. In order for this to work for the RTAS MSI
backend, we need to read back the MSI message from config space after
it has been setup by firmware.
This should be sufficient for restoring the MSI state after a device
reset, however we will need to revisit this for suspend to disk if that
is ever implemented on pseries.
Signed-off-by: Michael Ellerman <michael@ellerman.id.au>
Acked-by: Linas Vepstas <linas@austin.ibm.com>
Signed-off-by: Paul Mackerras <paulus@samba.org>
Currently rtas_change_msi() returns either the error code from RTAS, or if
the RTAS call succeeded the number of irqs that were configured by RTAS.
This makes checking the return value more complicated than it needs to be.
Instead, have rtas_change_msi() check that the number of irqs configured by
RTAS is equal to what we requested - and return an error otherwise. This makes
the return semantics match the usual 0 for success, something else for error.
Signed-off-by: Michael Ellerman <michael@ellerman.id.au>
Signed-off-by: Paul Mackerras <paulus@samba.org>
rtas_setup_msi_irqs() doesn't need to call teardown() itself, the
generic code will do this for us as long as we return a non-zero
value.
Signed-off-by: Michael Ellerman <michael@ellerman.id.au>
Acked-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: Paul Mackerras <paulus@samba.org>
Implement MSI support via RTAS (RTAS = run-time firmware on pSeries
machines). For now we assumes that if the required RTAS tokens for
MSI are present, then we want to use the RTAS MSI routines.
When RTAS is managing MSIs for us, it will/may enable MSI on devices that
support it by default. This is contrary to the Linux model where a device
is in LSI mode until the driver requests MSIs.
To remedy this we add a pci_irq_fixup call, which disables MSI if they've
been assigned by firmware and the device also supports LSI. Devices that
don't support LSI at all will be left as is, drivers are still expected
to call pci_enable_msi() before using the device.
At the moment there is no pci_irq_fixup on pSeries, so we can just set it
unconditionally. If other platforms use the RTAS MSI backend they'll need
to check that still holds.
Signed-off-by: Michael Ellerman <michael@ellerman.id.au>
Signed-off-by: Paul Mackerras <paulus@samba.org>
