New file |
| | |
| | | ## RoCE Configuration for Ubuntu |
| | | |
| | | To configure RoCE on the adapter for an Ubuntu host, RDMA must be installed |
| | | and configured on the Ubuntu host. |
| | | To configure and set up RoCE for Ubuntu 14.04.5/16.04.1 Linux: |
| | | |
| | | 1. When you begin installing the Ubuntu server, verify if the basic packages, |
| | | modules, and tools are available for Ethernet and RDMA. Log in as ROOT,and install all required packages. |
| | | |
| | | a. Install the basic packages required for Ubuntu: |
| | | |
| | | ```bash |
| | | apt-get install build-essential cmake gcc \ |
| | | pkg-config vlan automake autoconf dkms git ninja-build \ |
| | | libudev-dev libnl-3-dev libnl-route-3-dev valgrind python3-dev cython3 python3-docutils pandoc |
| | | |
| | | ``` |
| | | b. Install the following RDMA packages required for Ubuntu: |
| | | ```bash |
| | | apt-get install –f libibverbs* librdma* libibcm.* libibmad.* libibumad* |
| | | ``` |
| | | |
| | | Or, Install from source code |
| | | |
| | | ```bash |
| | | git clone https://github.com/linux-rdma/rdma-core.git |
| | | ``` |
| | | |
| | | ```bash |
| | | [ -d build ] || mkdir build |
| | | cd build |
| | | cmake -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_BUILD_TYPE=Release .. |
| | | cmake --build . |
| | | sudo cmake --build . --target install |
| | | ``` |
| | | |
| | | c.Install RDMA user space tools and libraries required for Ubuntu: |
| | | |
| | | ```bash |
| | | apt-get install -f libtool ibutils ibverbs-utils \ |
| | | rdmacm-utils infiniband-diags perftest librdmacm-dev \ |
| | | libibverbs-dev numactl libnuma-dev libnl-3-200 \ |
| | | libnl-route-3-200 libnl-route-3-dev libnl-utils |
| | | ``` |
| | | |
| | | 2. If the file /etc/udev/rules.d/40-rdma.rules does not exist, create it with the following content: |
| | | |
| | | ```bash |
| | | cat > /etc/udev/rules.d/40-rdma.rules << "EOF" |
| | | KERNEL=="umad*", NAME="infiniband/%k" |
| | | KERNEL=="issm*", NAME="infiniband/%k" |
| | | KERNEL=="ucm*", NAME="infiniband/%k", MODE="0666" |
| | | KERNEL=="uverbs*", NAME="infiniband/%k", MODE="0666" |
| | | KERNEL=="ucma", NAME="infiniband/%k", MODE="0666" |
| | | KERNEL=="rdma_cm", NAME="infiniband/%k", MODE="0666" |
| | | EOF |
| | | ``` |
| | | |
| | | 3. Edit the /etc/security/limits.conf file to increase the size of memory, which can be locked by a non-root user. Add the following lines, and then log out: |
| | | |
| | | ```bash |
| | | cat >> /etc/security/limits.conf << "EOF" |
| | | * soft memlock unlimited |
| | | * hard memlock unlimited |
| | | root soft memlock unlimited |
| | | root hard memlock unlimited |
| | | EOF |
| | | ``` |
| | | 4. Log into the system again, or verify after reboot. Then issue the following command: |
| | | ```bash |
| | | #ulimit -l |
| | | ``` |
| | | You should get the output as `unlimited` . |
| | | |
| | | 5. Reboot the system. |