My goal is to export ZFS zvol volumes through iSCSI to other machines. The platform I'm using is Debian Wheezy.
There are three iSCSI target solutions available for Linux:
I've briefly played with LIO but the targetcli tool is interactive only. If you want to automate and use scripts, you need to learn the Python API. I wonder what's wrong with a plain old text-based configuration file.
iscsitarget or IET is broken on Debian Wheezy. If you just 'apt-get install iscsitarget', the iSCSI service will just crash as soon as you connect to it. This has been the case for years. I wonder why they don't just drop this package. It is true that you can manually download the "latest" version of IET, but don't bother, it seems abandoned. The latest release stems from 2010.
It seems that SCST is at least maintained and uses plain old text-based configuration files. So it has that going for it, which is nice. SCST does not require kernel patches to run. But particularly a patch regarding "CONFIG_TCP_ZERO_COPY_TRANSFER_COMPLETION_NOTIFICATION" is said to improve performance.
To use full power of TCP zero-copy transmit functions, especially
dealing with user space supplied via scst_user module memory, iSCSI-SCST
needs to be notified when Linux networking finished data transmission.
For that you should enable CONFIG_TCP_ZERO_COPY_TRANSFER_COMPLETION_NOTIFICATION
kernel config option. This is highly recommended, but not required.
Basically, iSCSI-SCST works fine with an unpatched Linux kernel with the
same or better speed as other open source iSCSI targets, including IET,
but if you want even better performance you have to patch and rebuild
the kernel.
So in general, patching your kernel is not always required, but an example will be given anyway.
Getting the source
cd /usr/src
We need the following files:
wget http://heanet.dl.sourceforge.net/project/scst/scst/scst-3.0.0.tar.bz2
wget http://heanet.dl.sourceforge.net/project/scst/iscsi-scst/iscsi-scst-3.0.0.tar.bz2
wget http://heanet.dl.sourceforge.net/project/scst/scstadmin/scstadmin-3.0.0.tar.bz2
We extract them with:
tar xjf scst-3.0.0.tar.bz2
tar xjf iscsi-scst-3.0.0.tar.bz2
tar xjf scstadmin-3.0.0.tar.bz2
Patching the kernel
You can skip this part if you don't feel like you need or want to patch your kernel.
apt-get install linux-source kernel-package
We need to extract the kernel source:
cd /usr/src
tar xjf linux-source-3.2.tar.bz2
cd linux-source-3.2
Now we first copy the kernel configuration from the current system:
cp /boot/config-3.2.0-4-amd64 .config
We patch the kernel with two patches:
patch -p1 < /usr/src/scst-3.0.0/kernel/scst_exec_req_fifo-3.2.patch
patch -p1 < /usr/src/iscsi-scst-3.0.0/kernel/patches/put_page_callback-3.2.57.patch
It seems that for many different kernel versions, separate patches can be found in the above paths. If you follow these steps at a later date, please check the version numbers.
The patches are based on stock kernels from kernel.org. I've applied the patches against the Debian-patched kernel and faced no problems, but your milage may vary.
Let's build the kernel (will take a while):
yes | make-kpkg -j $(nproc) --initrd --revision=1.0.custom.scst kernel_image
The 'yes' is piped into the make-kpkg command to answer some questions with 'yes' during compilation. You could also add the appropriate value in the .config file.
The end-result of this command is a kernel package in .deb format in /usr/src. Install it like this:
dpkg -i /usr/src/<custom kernel image>.deb
Now reboot into the new kernel:
reboot
Compiling SCST, ISCS-SCST and SCSTADMIN
cd /usr/src/scst-3.0.0
make install
cd /usr/src/iscsi-scst-3.0.0
make install
cd /usr/src/scstadmin-3.0.0
make install
Make SCST start at boot
On Debian Jessie:
systemctl enable scst.service
Configure SCST
Copy the example configuration file to /etc:
cp /usr/src/iscsi-scst-3.0.0/etc/scst.conf /etc
Edit /etc/scst.conf to your liking. This is an example:
HANDLER vdisk_fileio {
DEVICE disk01 {
filename /dev/sdb
nv_cache 1
}
}
TARGET_DRIVER iscsi {
enabled 1
TARGET iqn.2015-10.net.vlnb:tgt {
IncomingUser "someuser somepasswordof12+chars"
HeaderDigest "CRC32C,None"
DataDigest "CRC32C,None"
LUN 0 disk01
enabled 1
}
}
Please note that the password must be at least 12 characters.
After this, you can start the SCST module and connect your initiator to the appropriate LUN.
/etc/init.d/scst start
Closing words
It turned out that setting up SCST and compiling a kernel wasn't that much of a hassle. The main issue with patching kernels is that you have to repeat the procedure every time a new kernel version is released. And there is always a risk that a new kernel version breaks the SCST patches.
However, the whole process can be easily automated and thus run as a test in a virtual environment.
