dd / nc netcat disk clone over SSH
ssh root@sourcehost 'dd if=/dev/sda'| dd of=/dev/target status=progress Source: https://www.maketecheasier.com/ssh-pipes-linux/ Combined with the rsync tip, to copy 32 GB: dd if=/dev/sda status=progress bs=4M count=8192 | ssh -T -c arcfour -o Compression=no -x user@targethost 'dd of=/folder/on/target/file.img' Got 42,5 MB/s with this from SSD to HDD over 1 Gbit.
Verify the file
dd if=/dev/sda status=progress bs=4M count=8192|md5sum md5sum /folder/on/target/file.img
Netcat - higher speed, unencrypted
On the source: cat /path/to/source | pv | nc -l -p 5555 To only use one IP on the source and not all: cat /path/to/source | pv | nc -l -s 192.168.1.2 -p 5555 On the target: nc <source-host> 5555 | pv > /path/to/target Speeds were ~10-90 MB/s with this method.
Regarding VirtualBox VM:s
To increase speed between a host and a virtual machine switch network adapter on the host from Intel PRO 1000/MT Desktop to PCnet-FAST III, also replace NAT connection with a bridged connection if possible. This increased speed by around +10 MB/s.
Synchronize only changes on block devices
If you already have an outdated copy of the block device you want to copy, then you can save by just updating it with the changes. However, rsync does not work with block devices, but there are alternatives that do.
rsync alternative - bscp
Installation: cd /opt git clone https://github.com/dop251/diskrsync ln -s /opt/bscp/bscp /usr/bin/bscp Usage: bscp <source> <host>:<path> <block size> <hash, defaults to sha1> bscp /dev/sdXY host:/path/to/device-or-file Notes: bscp does NOT need to be installed on the remote side. It is NOT possible to specify a user with user@ before the hostname, it tries to log in with the username of the current local user. Specifying a directory that does not exist results in cryptic errors. What hashes that are available is not sure. Speed test 6 GB file over a wireless network: 07:15.
rsync alternative - diskrsync
Installation - note, the binary MUST be installed on both the client AND the host: apt install golang cd /opt mkdir diskrsync cd diskrsync export GOPATH=$(pwd) go get github.com/dop251/diskrsync/diskrsync ln -s /opt/diskrsync/bin/diskrsync /usr/local/bin/diskrsync Usage: diskrsync --no-compress --verbose <source> <host>:<target> diskrsync --no-compress --verbose /dev/sdXY host:/path/to-device-or-file Notes: diskrsync binary must exist on both client and server the client it runs it on the server. Login must be done using a private/public SSH key. By default it stores files compressed in a spgz format, to avoid this use --no-compress. Speed test 6 GB file over a wireless network: 10:17.
This is a personal note. Last updated: 2023-12-25 00:50:48.