The files in this directory provide Xilinx PCIe DMA drivers, example software,
and example test scripts that can be used to exercise the Xilinx PCIe DMA IP.

This software can be used directly or referenced to create drivers and software
for your Xilinx FPGA hardware design.

Operating System Support:
=========================
Operating System Architecture: x86_64
Linux kernel:	3.10, 4.18.0, 5.14.0, 5.15, 6.8
RHEL/CentOS:	7.6, 7.7, 7.8, 7.9,
		8.1, 8.2
ALMA OS:	8.10, 9.4
Ubuntu:		16.04.5 LTS, 16.04.6 LTS
		18.04.1 LTS, 18.04.2 LTS, 18.04.4 LTS3, 18.04.5 LTS
		20.04 LTS, 20.04.1 LTS, 22.04 LTS, 24.04 LTS

HW Requirement:
===============
The PCIe DMA supports UltraScale+, UltraScale, Virtex-7 XT and 7 Series Gen2
devices.

Please refer to the Xilinx documentation "PG195 DMA/Bridge Subsystem for PCI
Express" for details of the IP.


Directory and file description:
===============================
 - xdma/: This directory contains the Xilinx PCIe DMA kernel module
       driver files.

 - include/: This directory contains all include files that are needed for
	compiling driver.

 - tests/: This directory contains example application software to exercise the
	provided kernel module driver and Xilinx PCIe DMA IP. This directory
	also contains the following scripts and directories.

	- load_driver.sh:
		This script loads the kernel module and creates the necissary
		kernel nodes used by the provided software.
		The The kernel device nodes will be created under /dev/xdma*.
		Additional device nodes are created under /dev/xdma/card* to
		more easily differentiate between multiple PCIe DMA enabled
		cards. Root permissions will be required to run this script.

	- run_test.sh: 
	- dma_memory_mapped_test.sh, dma_streaming_test.sh:
	- data/:
		run_test.sh runs sample tests on a Xilinx PCIe DMA target and
		returns a pass (0) or fail (1) result.
		This script calls 2 other scripts in the same directory:
		- dma_memory_mapped_test.sh for memory-mapped dma mode
		- dma_streaming_test.sh for streaming dma mode
		The data/ directory contains binary data files that are used
		for DMA data transfers to the Xilinx FPGA PCIe endpoint device
		with the run_test.sh.
		!NOTE!:
		=======
		These scripts are intended for use with the PCIe DMA EXAMPLE
		DESIGN ONLY.

	 - perform_hwcount.sh:
		This script runs hardware performance for XDMA for both Host to
		Card (H2C) and Card to Host (C2H). The result are copied to
		'hw_log_h2c.txt' and hw_log_c2h.txt' text files. 
		For each direction the performance script loops from 64 bytes
		to 4MBytes and generate performance numbers (byte size doubles
		for each loop count).
		You can grep for 'data rate' on those two files to see data
		rate values.
		Data rate values are in percentage of maximum throughput.
		Maximum data rate for x8 Gen3 is 8Gbytes/s, so for a x8Gen3
		design value of 0.81 data rate is 0.81*8 = 6.48Gbytes/s.
		Maximum data rate for x16 Gen3 is 16Gbytes/s, so for a x16Gen3
		design value of 0.78 data rate is 0.78*16 = 12.48Gbytes/s.
		This program can be run on AXI-MM example design.
		AXI-ST example design is a loopback design, both H2C and C2H
		are connected. Running on AXI-ST example design will not
		generate proper numbers.
		If a AXI-ST design is independent of H2C and C2H, performance
		number can be generated. 

	- scripts_mm/
		This directory contains a set of scripts to check basic driver
		loading/unloading and perform dma operations in memory-mapped
		mode.
		Compare with dma_memory_mapped_test.sh, the test is more
		extensive with more dma size and it also utilizes fio tool in
		addition to dma_from/to_device tools.

		- xdma_mm.sh
			top level script.

		- io_sweep.sh, io.sh, unaligned
			dma test via dma_from/to_device

		- fio_test.sh fio_parse_result.sh
			dma test via fio tool

	- scripts_mm/ dependency
		Some test in script_mm/ requires fio tool and python extension

		Install fio:
		- Centos/RHEL: yum install fio
		- Ubuntu: apt install fio

		Install python extension openpyxl, xlrd(version 1.2.0)
			python --version
			pip2 install openpyxl
			pip2 install xlrd=1.2.0

Usage:
  - Change directory to the driver directory.
        cd xdma
  - Compile and install the kernel module driver.
        make install
  - Change directory to the tools directory.
        cd tools
  - Compile the provided example test tools.
        make
  - Load the kernel module driver:
	a. modprobe xdma
	b. using the provided script.
		cd tests
        	./load_driver.sh
  - Run the provided test script to generate basic DMA traffic.
        ./run_test.sh
 	
	For more extensive memory mapped test:
	assume the XDMA FGPA is at pci slot 0000:01:00.0
		cd scripts_mm
		./xdma_mm.sh 0000:01:00.0 | tee /tmp/xdma_mm.log

  - Check driver Version number
        modinfo xdma (or)
        modinfo ../xdma/xdma.ko    

Updates and Backward Compaitiblity:
  - The following features were added to the PCIe DMA IP and driver in Vivado
    2016.1. These features cannot be used with PCIe DMA IP if the IP was
    generated using a Vivado build earlier than 2016.1.
      - Poll Mode: Earlier versions of Vivado only support interrupt mode which
	is the default behavior of the driver.
      - Source/Destination Address: Earlier versions of Vivado PCIe DMA IP
	required the low-order bits of the Source and Destination address to be
	the same.
	As of 2016.1 this restriction has been removed and the Source and
	Destination addresses can be any arbitrary address that is valid for
        your system.

Frequently asked questions:
  Q: How do I uninstall the kernel module driver?
  A: Use the following commands to uninstall the driver.
       - Uninstall the kernel module.
             rmmod -s xdma

  Q: How do I modify the PCIe Device IDs recognized by the kernel module driver?
  A: The xdma/xdma_mod.c file constains the pci_device_id struct that identifies
     the PCIe Device IDs that are recognized by the driver in the following
     format:
         { PCI_DEVICE(0x10ee, 0x8038), },
     Add, remove, or modify the PCIe Device IDs in this struct as desired. Then
     uninstall the existing xdma kernel module, compile the driver again, and
     re-install the driver using the load_driver.sh script.

  Q: By default the driver uses interupts to signal when DMA transfers are
     completed. How do I modify the driver to use polling rather than
     interrupts to determine when DMA transactions are completed?
  A: The driver can be changed from being interrupt driven (default) to being
     polling driven (poll mode) when the kernel module is inserted. To do this
     modify the load_driver.sh file as follows:
        Change: insmod xdma/xdma.ko
        To:     insmod xdma/xdma.ko poll_mode=1
     Note: Interrupt vs Poll mode will apply to all DMA channels. If desired the
     driver can be modified such that some channels are interrupt driven while
     others are polling driven. Refer to the poll mode section of PG195 for
     additional information on using the PCIe DMA IP in poll mode. 
