X-Git-Url: http://git.rot13.org/?a=blobdiff_plain;f=Documentation%2FMSI-HOWTO.txt;h=a51f693c15419e9b2da87df025b69d898f648b51;hb=6bb8383bebc02dae08a17f561401f58005f75c03;hp=3ec6c720b0166b4a76c576391414c795898d47b5;hpb=1ef43204f4bd24dcd3156185b19b31b6b4151ae9;p=powerpc.git diff --git a/Documentation/MSI-HOWTO.txt b/Documentation/MSI-HOWTO.txt index 3ec6c720b0..a51f693c15 100644 --- a/Documentation/MSI-HOWTO.txt +++ b/Documentation/MSI-HOWTO.txt @@ -219,7 +219,7 @@ into the field vector of each element contained in a second argument. Note that the pre-assigned IOAPIC dev->irq is valid only if the device operates in PIN-IRQ assertion mode. In MSI-X mode, any attempt at using dev->irq by the device driver to request for interrupt service -may result unpredictabe behavior. +may result in unpredictable behavior. For each MSI-X vector granted, a device driver is responsible for calling other functions like request_irq(), enable_irq(), etc. to enable @@ -241,68 +241,7 @@ address space of the MSI-X table/MSI-X PBA. Otherwise, the PCI subsystem will fail enabling MSI-X on its hardware device when it calls the function pci_enable_msix(). -5.3.2 Handling MSI-X allocation - -Determining the number of MSI-X vectors allocated to a function is -dependent on the number of MSI capable devices and MSI-X capable -devices populated in the system. The policy of allocating MSI-X -vectors to a function is defined as the following: - -#of MSI-X vectors allocated to a function = (x - y)/z where - -x = The number of available PCI vector resources by the time - the device driver calls pci_enable_msix(). The PCI vector - resources is the sum of the number of unassigned vectors - (new) and the number of released vectors when any MSI/MSI-X - device driver switches its hardware device back to a legacy - mode or is hot-removed. The number of unassigned vectors - may exclude some vectors reserved, as defined in parameter - NR_HP_RESERVED_VECTORS, for the case where the system is - capable of supporting hot-add/hot-remove operations. Users - may change the value defined in NR_HR_RESERVED_VECTORS to - meet their specific needs. - -y = The number of MSI capable devices populated in the system. - This policy ensures that each MSI capable device has its - vector reserved to avoid the case where some MSI-X capable - drivers may attempt to claim all available vector resources. - -z = The number of MSI-X capable devices pupulated in the system. - This policy ensures that maximum (x - y) is distributed - evenly among MSI-X capable devices. - -Note that the PCI subsystem scans y and z during a bus enumeration. -When the PCI subsystem completes configuring MSI/MSI-X capability -structure of a device as requested by its device driver, y/z is -decremented accordingly. - -5.3.3 Handling MSI-X shortages - -For the case where fewer MSI-X vectors are allocated to a function -than requested, the function pci_enable_msix() will return the -maximum number of MSI-X vectors available to the caller. A device -driver may re-send its request with fewer or equal vectors indicated -in the return. For example, if a device driver requests 5 vectors, but -the number of available vectors is 3 vectors, a value of 3 will be -returned as a result of pci_enable_msix() call. A function could be -designed for its driver to use only 3 MSI-X table entries as -different combinations as ABC--, A-B-C, A--CB, etc. Note that this -patch does not support multiple entries with the same vector. Such -attempt by a device driver to use 5 MSI-X table entries with 3 vectors -as ABBCC, AABCC, BCCBA, etc will result as a failure by the function -pci_enable_msix(). Below are the reasons why supporting multiple -entries with the same vector is an undesirable solution. - - - The PCI subsystem cannot determine the entry that - generated the message to mask/unmask MSI while handling - software driver ISR. Attempting to walk through all MSI-X - table entries (2048 max) to mask/unmask any match vector - is an undesirable solution. - - - Walking through all MSI-X table entries (2048 max) to handle - SMP affinity of any match vector is an undesirable solution. - -5.3.4 API pci_enable_msix +5.3.2 API pci_enable_msix int pci_enable_msix(struct pci_dev *dev, struct msix_entry *entries, int nvec) @@ -339,7 +278,7 @@ a failure. This failure may be a result of duplicate entries specified in second argument, or a result of no available vector, or a result of failing to initialize MSI-X table entries. -5.3.5 API pci_disable_msix +5.3.3 API pci_disable_msix void pci_disable_msix(struct pci_dev *dev) @@ -349,7 +288,7 @@ always call free_irq() on all MSI-X vectors it has done request_irq() on before calling this API. Failure to do so results in a BUG_ON() and a device will be left with MSI-X enabled and leaks its vectors. -5.3.6 MSI-X mode vs. legacy mode diagram +5.3.4 MSI-X mode vs. legacy mode diagram The below diagram shows the events which switch the interrupt mode on the MSI-X capable device function between MSI-X mode and @@ -407,7 +346,7 @@ between MSI mod MSI-X mode during a run-time. MSI/MSI-X support requires support from both system hardware and individual hardware device functions. -5.5.1 System hardware support +5.5.1 Required x86 hardware support Since the target of MSI address is the local APIC CPU, enabling MSI/MSI-X support in the Linux kernel is dependent on whether existing @@ -470,7 +409,68 @@ LOC: 324553 325068 ERR: 0 MIS: 0 -6. FAQ +6. MSI quirks + +Several PCI chipsets or devices are known to not support MSI. +The PCI stack provides 3 possible levels of MSI disabling: +* on a single device +* on all devices behind a specific bridge +* globally + +6.1. Disabling MSI on a single device + +Under some circumstances it might be required to disable MSI on a +single device. This may be achieved by either not calling pci_enable_msi() +or all, or setting the pci_dev->no_msi flag before (most of the time +in a quirk). + +6.2. Disabling MSI below a bridge + +The vast majority of MSI quirks are required by PCI bridges not +being able to route MSI between busses. In this case, MSI have to be +disabled on all devices behind this bridge. It is achieves by setting +the PCI_BUS_FLAGS_NO_MSI flag in the pci_bus->bus_flags of the bridge +subordinate bus. There is no need to set the same flag on bridges that +are below the broken bridge. When pci_enable_msi() is called to enable +MSI on a device, pci_msi_supported() takes care of checking the NO_MSI +flag in all parent busses of the device. + +Some bridges actually support dynamic MSI support enabling/disabling +by changing some bits in their PCI configuration space (especially +the Hypertransport chipsets such as the nVidia nForce and Serverworks +HT2000). It may then be required to update the NO_MSI flag on the +corresponding devices in the sysfs hierarchy. To enable MSI support +on device "0000:00:0e", do: + + echo 1 > /sys/bus/pci/devices/0000:00:0e/msi_bus + +To disable MSI support, echo 0 instead of 1. Note that it should be +used with caution since changing this value might break interrupts. + +6.3. Disabling MSI globally + +Some extreme cases may require to disable MSI globally on the system. +For now, the only known case is a Serverworks PCI-X chipsets (MSI are +not supported on several busses that are not all connected to the +chipset in the Linux PCI hierarchy). In the vast majority of other +cases, disabling only behind a specific bridge is enough. + +For debugging purpose, the user may also pass pci=nomsi on the kernel +command-line to explicitly disable MSI globally. But, once the appro- +priate quirks are added to the kernel, this option should not be +required anymore. + +6.4. Finding why MSI cannot be enabled on a device + +Assuming that MSI are not enabled on a device, you should look at +dmesg to find messages that quirks may output when disabling MSI +on some devices, some bridges or even globally. +Then, lspci -t gives the list of bridges above a device. Reading +/sys/bus/pci/devices/0000:00:0e/msi_bus will tell you whether MSI +are enabled (1) or disabled (0). In 0 is found in a single bridge +msi_bus file above the device, MSI cannot be enabled. + +7. FAQ Q1. Are there any limitations on using the MSI?