From 14be935a4f8231233487d510c8db0b544bcf0f69 Mon Sep 17 00:00:00 2001 From: wangzhengquan <wangzhengquan85@126.com> Date: 星期一, 25 一月 2021 17:40:29 +0800 Subject: [PATCH] fix conflict --- test_rdma/driver-install.md | 118 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 118 insertions(+), 0 deletions(-) diff --git a/test_rdma/driver-install.md b/test_rdma/driver-install.md new file mode 100644 index 0000000..268b06f --- /dev/null +++ b/test_rdma/driver-install.md @@ -0,0 +1,118 @@ + +## RoCE Configuration for Ubuntu + +To configure RoCE on the adapter for an Ubuntu host, RDMA must be installed and configured on the Ubuntu host. To configure and set up RoCE for Ubuntu 14.04.5/16.04.1 Linux: + +1. When you begin installing the Ubuntu server, verify if the basic packages, modules, and tools are available for Ethernet and RDMA. Log in as ROOT,and install all required packages. + + a. Install the basic packages required for Ubuntu: + + ```bash + apt-get install build-essential cmake gcc \ + pkg-config vlan automake autoconf dkms git ninja-build \ + libudev-dev libnl-3-dev libnl-route-3-dev valgrind python3-dev cython3 python3-docutils pandoc + + ``` + b. Install the following RDMA packages required for Ubuntu: + ```bash + apt-get install 鈥揻 libibverbs* librdma* libibcm.* libibmad.* libibumad* + ``` + + Or, Install from source code + + ```bash + git clone https://github.com/linux-rdma/rdma-core.git + ``` + + ```bash + [ -d build ] || mkdir build + cd build + cmake -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_BUILD_TYPE=Release .. + cmake --build . + sudo cmake --build . --target install + ``` + + c.Install RDMA user space tools and libraries required for Ubuntu: + + ```bash + apt-get install -f libtool ibutils ibverbs-utils \ + rdmacm-utils infiniband-diags perftest librdmacm-dev \ + libibverbs-dev numactl libnuma-dev libnl-3-200 \ + libnl-route-3-200 libnl-route-3-dev libnl-utils + ``` + +2. If the file /etc/udev/rules.d/40-rdma.rules does not exist, create it with the following content: + + ```bash +cat > /etc/udev/rules.d/40-rdma.rules << "EOF" +KERNEL=="umad*", NAME="infiniband/%k" +KERNEL=="issm*", NAME="infiniband/%k" +KERNEL=="ucm*", NAME="infiniband/%k", MODE="0666" +KERNEL=="uverbs*", NAME="infiniband/%k", MODE="0666" +KERNEL=="ucma", NAME="infiniband/%k", MODE="0666" +KERNEL=="rdma_cm", NAME="infiniband/%k", MODE="0666" +EOF +``` + +3. Edit the /etc/security/limits.conf file to increase the size of memory, which can be locked by a non-root user. Add the following lines, and then log out: + + ```bash + cat >> /etc/security/limits.conf << "EOF" + * soft memlock unlimited + * hard memlock unlimited + root soft memlock unlimited + root hard memlock unlimited + EOF + ``` +4. Log into the system again, or verify after reboot. Then issue the following command: + ```bash +ulimit -l + ``` +You should get the output as `unlimited` . + +5. Reboot the system. + +6. To allow the device to be recognized as an InfiniBand device that can be used by OFED, install the FastLinQ package by issuing the following commands: + ```bash +cd fastlinq-X.X.X.X +make clean +make install +``` +7. Install the libqedr libraries to work with RDMA user space applications using one of the following command options: + Option 1: + + ```bash + cd fastlinq-X.X.X.X + make libqedr_install + ``` + + Option 2: + + ```bash + cd fastlinq-X.X.X.X/libqedr-X.X.X.X/ + ./configure --prefix=/usr --libdir=${exec_prefix}/lib --sysconfdir=/etc make install + ``` + +8. Before loading the QLogic Ethernet and RDMA drivers, uninstall the existing out-of-box or inbox drivers by issuing the following commands: + + ```bash + modprobe 鈥搑 qede + depmod 鈥揳 + modprobe -v qedr + ``` + +9. Load the RDMA modules by issuing the following commands. You must perform this step whenever you reboot the system. + + ```bash + modprobe rdma_cm + modprobe ib_uverbs + modprobe rdma_ucm + modprobe ib_ucm + modprobe ib_umad + ``` + +10. To list RoCE devices, issue the following command: + + ```bash + ibv_devinfo + ``` \ No newline at end of file -- Gitblit v1.8.0