intel qat 加速卡安装配置

发表于 | 分类于 | | 游览

intel qat 加速卡安装配置

英特尔 Quick Assist Technology （以下简称 QAT ）是 英特尔 针对网络安全和数据存储 推出 的一个硬件 加速 技术。 QAT 支持对称数据加密算法（如AES）中的密码操作和验证操作运算和公钥非对称数据加密算法。
Intel QAT加速卡结合Intel其QAT_Engine测试性能的提升，其支持的异步模式对性能的提升很大

相关URL

QAT 官网 ： https://www.intel.cn/content/www/cn/zh/architecture-and-technology/intel-quick-assist-technology-overview.html
QAT卡说明：https://01.org/sites/default/files/downloads/intelr-quickassist-technology/intelquickassisttechnologyopensslperformance.pdf

QAT卡相关驱动、资料：https://01.org/intel-quickassist-technology
QAT驱动安装文档：https://01.org/sites/default/files/downloads//336212-intelrquickassisttechnology-gsg-revision008.pdf
QAT安装视频：https://software.intel.com/content/www/us/en/develop/videos/intel-quickassist-technology-openssl-1-1-x-qat-engine.html
性能对比：https://software.intel.com/content/www/cn/zh/develop/articles/improving-openssl-performance.html

centos7.5 系统环境

yum package
1
2
3
4
5
6
7
8
9
10
11
12
yum install epel-release.noarch
yum -y install  wget gcc gcc-c++ gcc-g77 autoconf automake zlib* fiex* libxml* ncurses-devel libmcrypt* libtool-ltdl-devel* make cmake bind-utils ntp ntpdate lrzsz rsync gzip unzip vim telnet openssl-devel nscd g++ sysstat ncurses-libs bzip2-devel git lsof sqlite-devel ftp net-tools

yum install -y openssl-devel pciutils zlib-devel gcc libudev-devel boost-devel

yum -y groupinstall "Development Tools"
yum -y install pciutils
yum -y install openssl-devel  zlib-devel  gcc   libudev-devel boost-devel  pciutils

yum install perl perl-devel
yum groupinstall perl*
yum install -y kernel kernel-devel kernel-devel-$(uname -r)
关闭服务
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
systemctl stop firewalld
systemctl disable firewalld
systemctl stop NetworkManager
systemctl disable NetworkManager

sed -i "/^SELINUX=/s/enforcing/disabled/g" /etc/selinux/config

vim  /etc/security/limits.conf
*       soft nofile 655350
*       hard nofile 655350
*       soft nproc 655350
*       hard nproc 655350
*       soft core 655350
*       hard core 655350
*       soft memlock 655350
*       hard memlock 655350

###安装驱动：
驱动下载地址： https://01.org/packet-processing/intel%C2%AE-quickassist-technology-drivers-and-patches

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
cd /home
wget https://downloadmirror.intel.com/30178/eng/QAT1.7.L.4.13.0-00009.tar.gz
mkdir  /home/QAT
tar -zxvf QAT1.7.L.4.13.0-00009.tar.gz -C /home/QAT
cd /home/QAT
./configure    开启api  ./configure --enable-kapi
我的
./configure --enable-qat-lkcf --enable-icp-dc-sym-only --enable-kapi
make
make install
make samples-install  

在build下， 会出 cpa_sample_code.ko 模块， 可以用  
insmod ./build/cpa_sample_code.ko 加载
cpa_sample_code   执行测试

查看服务

1
2
3
/etc/init.d/qat_service status
/etc/init.d/qat_service restart
systemctl restart qat_service

升级 openssl

qat 用到的 openssl 的版本需要 1.1.0 之上的。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
wget https://www.openssl.org/source/openssl-1.1.1g.tar.gz
tar -zxvf openssl-1.1.1g.tar.gz
ln -s openssl-1.1.1g openssl
cd openssl
./config --prefix=/usr/local/ssl
make depend
make
make install

添加环境变量
export OPENSSL_ENGINES=/usr/local/ssl/lib/engines-1.1   并添加到   vim /etc/profile   source /etc/profile

添加动态库
echo /usr/local/ssl/lib/ > /etc/ld.so.conf.d/qat.conf
ldconfig

验证
/usr/local/ssl/bin/openssl version

设置环境变量

统一在这里整理一份

1
2
3
4
5
6
7
8
9
10
11
12
13
14
vim /etc/profile

export OPENSSL_LIB=/usr/local/ssl
export OPENSSL_ENGINES=/usr/local/ssl/lib/engines-1.1
export PERL5LIB=$PERL5LIB:/home/openssl
export PATH=$PATH:/usr/local/sbin
export SSL_INC=/usr/local/ssl/include
export SSL_LIB=/usr/local/ssl/lib
export QZ_ROOT=/home/QATzip
export OPENSSL_LIB=/usr/local/ssl
export ICP_ROOT=/home/QAT
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$QZ_ROOT/utils:/usr/lib64:/usr/local/lib64:/usr/local/ssl/lib:/usr/local/ssl/lib/engines-1.1:/usr/lib64/openssl/engines:/$ICP_ROOT/build

source /etc/profile

QAT引擎编译

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
cd /home
git clone https://github.com/01org/QAT_Engine.git
cd QAT_Engine
./autogen.sh
./configure --with-qat_dir=/root/qat --with-openssl_dir=/root/openssl-1.1.1g --with-openssl_install_dir=/usr/local/ssl --with-qat_hw_dir=/root/qat --enable-upstream_driver --enable-usdm
或
./configure --with-qat_dir=$ICP_ROOT --with-openssl_dir=/home/openssl --with-openssl_install_dir=/usr/local/ssl --with-qat_hw_dir=$ICP_ROOT --enable-qat_sw --enable-upstream_driver  --enable-usdm
export PERL5LIB=$PERL5LIB:/home/openssl   不配置这个 make 会报错
make
make install

cd /home/QAT_Engine/qat_contig_mem
make

cp /etc/dh895xcc_dev0.conf /etc/dh895xcc_dev0.conf_backup1
cp qat/config/dh895xcc/multi_process_optimized/dh895xcc_dev0.conf /etc/   后续需要整合配置文件
/etc/init.d/qat_service restart
systemctl restart qat_service

验证

测试验证：
1
2
3
4
/usr/local/ssl/bin/openssl  engine -t -c -vvvv qatengine
或  (看生成的是名字， 是qat 还是 qatengine)
/usr/local/ssl/bin/openssl  engine -t -c -vvvv qat
`

系统正常跑

1
/usr/local/ssl/bin/openssl speed -elapsed rsa2048

加速卡跑

1
/usr/local/ssl/bin/openssl speed -engine qat_dev0 -elapsed rsa2048
openssl speed测试命令
  • RSA2048
1
2
3
4
5
6
7
8
  # Software 纯软
/usr/local/ssl/bin/opensslspeed -elapsed rsa2048
  # Synchronous 同步
/usr/local/ssl/bin/opensslspeed -engine qatengine -elapsed rsa2048
  # Asynchronous1 异步
/usr/local/ssl/bin/openssl speed -engine qatengine -elapsed -async_jobs 36 rsa2048
    # Asynchronous2 异步
/usr/local/ssl/bin/openssl speed -engine qatengine -elapsed -async_jobs 72 rsa2048
  • ECDSA-P256
1
2
3
4
5
6
7
8
  # Software 纯软
/usr/local/ssl/bin/openssl speed -elapsed ecdsap256
  # Synchronous 同步
/usr/local/ssl/bin/openssl speed -engine qatengine -elapsed ecdsap256
  # Asynchronous1 异步1
/usr/local/ssl/bin/openssl speed -engine qatengine -elapsed -async_jobs 36 ecdsap256
    # Asynchronous2 异步2
/usr/local/ssl/bin/opensslspeed -engine qatengine -elapsed -async_jobs 72 ecdsap256

查看 qat 卡 接受处理数据

QAT卡在工作的时候，计数会一直变化

1
cat /sys/kernel/debug/qat_dh895xcc_0000\:07\:00.0/fw_counters

配置文件：

QAT 默认配置文件 /etc/dh895xcc_dev0.conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
#########################################################################
#
# @par
# This file is provided under a dual BSD/GPLv2 license.  When using or
#   redistributing this file, you may do so under either license.
#
#   GPL LICENSE SUMMARY
#
#   Copyright(c) 2007-2021 Intel Corporation. All rights reserved.
#
#   This program is free software; you can redistribute it and/or modify
#   it under the terms of version 2 of the GNU General Public License as
#   published by the Free Software Foundation.
#
#   This program is distributed in the hope that it will be useful, but
#   WITHOUT ANY WARRANTY; without even the implied warranty of
#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
#   General Public License for more details.
#
#   You should have received a copy of the GNU General Public License
#   along with this program; if not, write to the Free Software
#   Foundation, Inc., 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA.
#   The full GNU General Public License is included in this distribution
#   in the file called LICENSE.GPL.
#
#   Contact Information:
#   Intel Corporation
#
#   BSD LICENSE
#
#   Copyright(c) 2007-2021 Intel Corporation. All rights reserved.
#
#   Redistribution and use in source and binary forms, with or without
#   modification, are permitted provided that the following conditions
#   are met:
#
#     * Redistributions of source code must retain the above copyright
#       notice, this list of conditions and the following disclaimer.
#     * Redistributions in binary form must reproduce the above copyright
#       notice, this list of conditions and the following disclaimer in
#       the documentation and/or other materials provided with the
#       distribution.
#     * Neither the name of Intel Corporation nor the names of its
#       contributors may be used to endorse or promote products derived
#       from this software without specific prior written permission.
#
#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
#
#########################################################################
[GENERAL]
ServicesEnabled = dc;cy

# Set the service profile to determine available features
# =====================================================================
#                               DEFAULT    CRYPTO  COMPRESSION  CUSTOM1
# Asymmetric Crypto                *         *                     *
# Symmetric Crypto                 *         *                     *
# MGF KeyGen                       *         *
# SSL/TLS KeyGen                   *         *                     *
# HKDF                                       *                     *
# Compression                      *                    *          *
# Decompression (stateless)        *                    *          *
# Decompression (stateful)         *                    *
# Service Chaining                                      *
# Device Utilization                         *                     *
# Rate Limiting                              *                     *
# =====================================================================
ServicesProfile = DEFAULT

ConfigVersion = 2

#Default values for number of concurrent requests*/
CyNumConcurrentSymRequests = 512
CyNumConcurrentAsymRequests = 64

#Statistics, valid values: 1,0
statsGeneral = 1
statsDh = 1
statsDrbg = 1
statsDsa = 1
statsEcc = 1
statsKeyGen = 1
statsDc = 1
statsLn = 1
statsPrime = 1
statsRsa = 1
statsSym = 1

# Debug feature, if set to 1 it enables additional entries in /proc filesystem
ProcDebug = 1

# This flag is to enable device auto reset on heartbeat error
AutoResetOnError = 0

##############################################
# Kernel Instances Section
##############################################
[KERNEL]
NumberCyInstances = 0
NumberDcInstances = 0

##############################################
# User Process Instance Section
##############################################
[SHIM]
NumberCyInstances = 1
NumberDcInstances = 0
NumProcesses = 32
LimitDevAccess = 1

# Crypto - User space
Cy0Name = "UserCY0"
Cy0IsPolled = 1
Cy0CoreAffinity = 0-31

##############################################
# User Process Instance Section
##############################################
[SSL]
NumberCyInstances = 2
NumberDcInstances = 2
NumProcesses = 1
LimitDevAccess = 0

# Crypto - User instance #0
Cy0Name = "SSL0"
Cy0IsPolled = 1
# List of core affinities
Cy0CoreAffinity = 0

# Crypto - User instance #1
Cy1Name = "SSL1"
Cy1IsPolled = 1
# List of core affinities
Cy1CoreAffinity = 1

# Data Compression - User instance #0
Dc0Name = "Dc0"
Dc0IsPolled = 1
# List of core affinities
Dc0CoreAffinity = 0

# Data Compression - User instance #1
Dc1Name = "Dc1"
Dc1IsPolled = 1
# List of core affinities
Dc1CoreAffinity = 1

[KERNEL_QAT]
NumberCyInstances = 2
NumberDcInstances = 2

# Crypto - Kernel instance #0
Cy0Name = "IPSec0"
Cy0IsPolled = 0
Cy0CoreAffinity = 1

# Crypto - Kernel instance #1
Cy1Name = "IPSec1"
Cy1IsPolled = 0
Cy1CoreAffinity = 2

# Data Compression - Kernel instance #0
Dc0Name = "IPComp0"
Dc0IsPolled = 0
Dc0CoreAffinity = 3

# Data Compression - Kernel instance #1
Dc1Name = "IPComp1"
Dc1IsPolled = 0
Dc1CoreAffinity = 4
重启服务
1
2
3
4
5
6
7
8
/etc/init.d/qat_service restart
systemctl restart qat_service

验证
/usr/local/ssl/bin/openssl  engine -t -c -vvvv qatengine
和
cpa_sample_code
都可以跑了
感谢您的支持！