NeuroAgent

How to Speed Up USB 3.0 for SMB in Proxmox with TrueNAS: Complete Guide

Learn how to fix low USB 3.0 performance for SMB transfers in Proxmox with TrueNAS. Optimize USB passthrough, network, and SMB settings for maximum transfer speeds.

#nas-performance#network-optimization#samba-tuning#smb-proxmox#truenas-configuration#usb-3-0-optimization#usb-passthrough
Question

How can I improve USB 3.0 performance for SMB transfers in Proxmox with a TrueNAS virtual machine?

I’ve set up a NAS system based on Proxmox with a TrueNAS virtual machine, but I’m experiencing very low SMB transfer speeds (10-30 MB/s), despite the USB 3.0 hardware being capable of much higher speeds (150-180 MB/s).

Current configuration:

  • External HDD enclosure with two 8 TB and 6 TB drives (RAID disabled, normal/clear mode)
  • Mini PC with:
    • 16 GB RAM (dual-channel mode)
    • 256 GB SSD for the system
    • USB 3.0 ports (using front ports as I’m unsure about the rear ones)
  • Proxmox installed on bare metal
  • TrueNAS VM with:
    • 16 GB SSD for the system
    • Two HDDs connected via USB using the command: qm set 100 -scsi1 /dev/disk/by-id/ata-[serial_number]
    • A storage pool created from both HDDs

Requirements met:

  1. Online server for home use with a shared disk pool
  2. Streaming movies to devices in the home network (works great with Jellyfin)
  3. File access via SMB with upload/download capabilities (this is where the problem lies)

Network configuration:

  • Mini PC connected to a Wi-Fi repeater (2.5 Gbit bandwidth)
  • Ethernet cable: 1 Gbit
  • Router: ~1.2 Gbit
  • Testing was conducted via Wi-Fi with 1 Gbit bandwidth on Windows and macOS

Problem:

Despite the hardware being capable of 150-180 MB/s speeds (verified on Windows), SMB transfers through the TrueNAS VM are limited to 10-30 MB/s. This is significantly below the expected USB 3.0 performance and even below the theoretical minimum of ~125 MB/s for a 1 Gbit network connection.

Question:

How can I troubleshoot and improve USB 3.0 performance for SMB transfers in my Proxmox configuration with a TrueNAS virtual machine? What settings or configurations might be causing this bottleneck?

NeuroAgent

Low USB 3.0 Performance for SMB Transfers in Your Proxmox Configuration with TrueNAS VM

Low USB 3.0 performance for SMB transfers in your Proxmox configuration with a TrueNAS virtual machine is typically related to USB device passthrough issues, network settings, or SMB parameters. The main causes may include suboptimal SCSI controller configuration, USB interrupt problems, network stack limitations, or incorrect SMB settings in TrueNAS.

Table of Contents

Analysis of Current Configuration

Your current configuration has several potential bottlenecks:

USB Passthrough Issues:

  • Using front USB ports instead of rear ones can cause power and interrupt issues
  • Configuration using qm set 100 -scsi1 /dev/disk/by-id/ata-[serial_number] may not provide optimal performance
  • Lack of direct USB 3.0 access without a virtual SCSI controller

Network Limitations:

  • Using Wi-Fi for testing even with the stated 1 Gbit capacity
  • An intermediate Wi-Fi repeater can add latency

Storage Settings:

  • Disk mode set to “normal/clear” instead of more optimized modes
  • Lack of caching at the TrueNAS level

For diagnostics, start by checking the current USB device configuration in Proxmox:

bash
lsusb
lspci | grep -i usb

USB Passthrough Optimization

Using Direct USB Passthrough Instead of SCSI

Instead of passing through disks via SCSI controller, try using direct USB passthrough:

  1. Identify the USB device:
bash
lsusb -t
  1. Configure the VM for direct USB passthrough:
bash
qm set 100 -usb1 1-1
  1. Or add a USB controller to the VM:
bash
qm set 100 -usbadd
qm set 100 -usbattach 0

USB Controller Configuration

For optimal performance, use a USB 3.0 controller instead of USB 2.0:

bash
qm set 100 -args "-device usb-ehci,id=ehci"
qm set 100 -args "-device usb-xhci,id=xhci"

USB Interrupt Optimization

Interrupt issues can significantly reduce performance:

  1. Check the current interrupt status:
bash
cat /proc/interrupts | grep usb
  1. Disable power saving for USB ports:
bash
echo 'auto' > /sys/bus/usb/devices/usb*/power/control
echo 'on' > /sys/bus/usb/devices/usb*/power/autosuspend

Network Performance Configuration

Proxmox Network Configuration Optimization

To improve network performance, make the following changes:

  1. Set acceptable TCP buffer values:
bash
echo 'net.core.rmem_max = 134217728' >> /etc/sysctl.conf
echo 'net.core.wmem_max = 134217728' >> /etc/sysctl.conf
echo 'net.ipv4.tcp_rmem = 4096 87380 134217728' >> /etc/sysctl.conf
echo 'net.ipv4.tcp_wmem = 4096 65536 134217728' >> /etc/sysctl.conf
  1. Optimize TCP parameters:
bash
echo 'net.core.netdev_max_backlog = 3000' >> /etc/sysctl.conf
echo 'net.ipv4.tcp_congestion_control = bbr' >> /etc/sysctl.conf
echo 'net.ipv4.tcp_no_metrics_save = 1' >> /etc/sysctl.conf
  1. Apply changes:
bash
sysctl -p

TrueNAS VM Network Configuration

Inside the TrueNAS virtual machine, optimize network settings:

  1. Increase TCP buffer size:
bash
sysctl -w net.core.rmem_max=134217728
sysctl -w net.core.wmem_max=134217728
  1. Configure SMB parameters:
bash
sysctl -w net.ipv4.tcp_rmem='4096 87380 134217728'
sysctl -w net.ipv4.tcp_wmem='4096 65536 134217728'

Using Ethernet Instead of Wi-Fi

For maximum performance, connect the client device directly to Ethernet:

  • Use Cat 5e or better cable
  • Connect to the same switch as the Proxmox host
  • Disable Wi-Fi for performance testing

SMB Optimization in TrueNAS

SMB Version Configuration

Use SMB 3.1.1 for maximum performance:

  1. Edit the SMB configuration:
bash
nano /etc/samba/smb.conf
  1. Add the following parameters:
ini
[global]
server min protocol = SMB3
server max protocol = SMB3_11
smb2 leases = yes
smb2 max credits = 8192
server multi channel support = yes
server encrypt passwords = yes
  1. Restart the SMB service:
bash
service samba restart

Disk Performance Optimization

  1. Enable caching in TrueNAS:
bash
sysctl -w vfs.read_max_bufspace=104857600
sysctl -w vfs.write_max_bufspace=104857600
  1. Optimize ZFS parameters (if using ZFS):
bash
sysctl -w kstat.zfs.arc.max=8589934592  # 8GB
sysctl -w kstat.zfs.prefetch_disable=0

SMB Transfer Size Configuration

Increase the maximum SMB packet size:

bash
sysctl -w net.ipv4.tcp_mtu_discover=2
sysctl -w net.core.optmem_max=65536

Additional Performance Settings

Disk Caching Configuration

  1. Enable disk caching in Proxmox:
bash
qm set 100 -args "-drive cache=writeback"
  1. Configure disk parameters in VM:
bash
qm set 100 -args "-device virtio-blk-pci,scsi=off,drive=drive0"
qm set 100 -args "-device virtio-blk-pci,scsi=off,drive=drive1"

CPU and Memory Optimization

  1. Ensure the VM has sufficient vCPU:
bash
qm set 100 -cores 4
  1. Configure disk I/O priority:
bash
qm set 100 -args "-device virtio-blk-pci,scsi=off,drive=drive0,iothread=1"
qm set 100 -args "-device virtio-blk-pci,scsi=off,drive=drive1,iothread=1"

Alternative Storage Configurations

Consider alternative approaches to storage organization:

  1. Using iSCSI instead of USB passthrough:
bash
qm set 100 -args "-iscsi initiator-name=iqn.2024-01.proxmox:100"
  1. NFS export instead of SMB:
bash
qm set 100 -args "-nfs share=/mnt/data"
  1. Direct disk access (if supported):
bash
qm set 100 -args "-drive file=/dev/sdb,if=none,id=drive0"
qm set 100 -args "-device virtio-blk-pci,drive=drive0"

Monitoring and Diagnosis

Performance Monitoring Tools

Use the following commands for performance monitoring:

  1. I/O monitoring in Proxmox:
bash
iotop -oP
iostat -x 1
  1. Network monitoring:
bash
nethogs
iftop -nNP
  1. Monitoring in VM:
bash
vmstat 1
iostat -x 1
sar -n DEV 1

USB Configuration Verification

Check the current USB device configuration:

bash
qm showconfig 100 | grep -i usb
ls -la /dev/disk/by-id/ata*

Performance Testing

Use the following tools for testing:

  1. Read/write speed in VM:
bash
dd if=/dev/zero of=/mnt/data/testfile bs=1M count=1024 oflag=direct
dd if=/mnt/data/testfile of=/dev/null bs=1M iflag=direct
  1. SMB performance testing:
bash
smbclient //<server_ip>/share -U <username> -c 'put /dev/zero testfile'
smbclient //<server_ip>/share -U <username> -c 'get testfile'
  1. Network packet analysis:
bash
tcpdump -i any -s 0 -w capture.pcap 'port 445'

Conclusion

To improve USB 3.0 performance for SMB transfers in Proxmox with a TrueNAS virtual machine, perform the following steps:

  1. Switch to direct USB passthrough instead of SCSI passthrough for lower latency
  2. Optimize network settings for both Proxmox and TrueNAS VM with increased TCP buffers
  3. Use SMB 3.1.1 protocol with multi-channel support enabled
  4. Check physical USB device connection and use rear ports instead of front ones
  5. Enable I/O caching at all system levels
  6. Use Ethernet instead of Wi-Fi for testing maximum performance

The main suspected bottlenecks: issues with USB passthrough via SCSI, network stack limitations, and suboptimal SMB settings. Start by checking the physical connection of USB devices and switching to direct USB passthrough, which should provide the greatest performance boost.

Sources

  1. Proxmox USB Passthrough Documentation
  2. TrueNAS SMB Performance Optimization Guide
  3. SMB Protocol Performance Tuning
  4. Linux TCP Tuning for High Performance
  5. ZFS Performance Best Practices
How to configure direct USB passthrough in Proxmox for maximum performance?What are the optimal SMB 3.1.1 settings for high-speed transfers in TrueNAS?How to compare USB 3.0 performance with other storage interfaces in Proxmox?What performance monitoring tools to use for diagnosing USB/SMB issues?How to configure Linux network parameters for maximum SMB transfer performance?What alternative disk connection methods to Proxmox provide better performance than USB passthrough?
Ask NeuroAgent