LUKS - encrypted installation

This will setup the following

/dev/sda1 -> /boot - 256 MB?
/dev/sda5 -> / - 20-30 GB?
/dev/sda6 -> /home - the rest

Run the installer as usual, but in the partition use the Manual mode.
Create three partitions.

256 MB, use as /boot, Ext4
5-30 GB, use as Physical volume for encryption
? GB, use as Physical volume

When done, use the Create encrypted volumes, check both the encryptable volumes and continue. Enter a passprhase for both.

Choose to use the created sdXX_crypt and sdXY_crypt partitions as / and /home.

https://xo.tc/setting-up-full-disk-encryption-on-debian-jessie.html

Unmount partitions with keys located in the root partition

It is possible to avoid entering password for every partition by adding a key inside the root partition.

You will then only be prompted for the root partition password on boot.

To do so:  

# go root
sudo bash
# make a directory for the keys
mkdir /etc/keys
# generate a key
dd if=/dev/random of=/etc/keys/sdXZ.key bs=1 count=32
# ensure only root can read it
chmod 400 /etc/keys/home.key
# add it as a key to the home partition
cryptsetup luksAddKey /dev/sdXZ /etc/keys/home.key

# edit 
vi /etc/crypttab
# replace the word none for the home partition with /etc/keys/sdXY.key and save it

Go to /boot and delete the initrd-<version>.img file.

update-initramfs -c -k all

Warning! All warnings about sdXY_crypt being ignored should be taken seriously and results in drops to initramfs on the next boot if not fixed.

Warning! Do NOT set a key file for the root partition. Doing this makes update-initramfs not to include cryptsetup which results in that initramfs will not be able to find the boot partition at boot and then drop to a shell.

Bypass login for root partition

If you want to bypass the login for the root partition, then use a keyscript. 

Option 1: use your plain text password below

Option 2: 
  generate a key with:
	dd if=/dev/urandom of=/etc/keys/root.key bs=1 count=32
	or
	</dev/urandom tr -dc 'A-Za-z0-9!"#$%&'\''()*+,-./:;<=>?@[\]^_`{|}~' | head -c <length, like 32, 64 or so>  > /etc/keys/root.key
  add it with:
	cryptsetup luksAddKey /dev/sdX /etc/keys/root.key
  open the file, copy the text into the echo phrase below

Create the file/script/somewhere/on/root/partition and fill it with:
#!/bin/sh
echo -n "password-or-key-here-in-plain-text"

Save the file and run chmod +x on it (otherwise will initramfs-update/cryptsetup refuse)

Edit /etc/crypttab so it looks like this:
sdXX_crypt UUID=XXXXXXXXXXXXX luks,keyscript=/script/somewhere/on/root/partition

Remove /boot/initrd-<img>.img file and run update-initramfs -c -k all

Broken - bypass root login using a keyfile located in /boot

It is also possible to use a keyfile located on /boot partition. This avoids the need to have the password stored in plaintext on the disk. 

Note, this is broken in Debian 9.6 and it seems to be due to internal conflicts over how the keyscript parameter should be further implemented within or not within systemd.

There are actually 2 or more bugs.

Bug 1: Systemd cannot make use of keyscript= and other parameters for each line in /etc/crypttab that the previous initialization system could. More info at https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=618862 .

Bug 2: Systemd gets confused by lines like this in /etc/crypttab that are used by initramfs:
sda1_crypt UUID=<uuid1> <uuid-to-boot-partition>:/path/to/key luks,keyscript=passdev
Instead of ignoring them it tries to mount the volumes a second time, which results in 1,5 minute long timeouts and boot stops. More info at https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=786559 .

Source for other distros: https://dradisframework.com/support/guides/customization/auto-unlock-luks-encrypted-drive.html

Solution with a script:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=786393

Switch between locked and unlocked using scripts

It is possible to lock and unlock the partitions from the (root) terminal. This is convenient if you want to be able to switch between locked and unlocked state, for instance when in unsafer environments and when in safer environments.


Script to lock - put it in /usr/bin/cryptlock

#!/bin/bash
if [ "$(readlink -- "/etc/crypttab")" = "crypttab.locked" ]; then
        echo "Already locked";
fi
mv /etc/crypttab /etc/crypttab.backup
echo 'Using locked crypttab';
ln -s /etc/crypttab.locked /etc/crypttab
cd /boot/
# echo 'Removing initrd';
# rm initrd.img*
echo 'Regenerating initrd';
update-initramfs -c -k all -v


Script to unlock - put it in /usr/bin/cryptunlock:

#!/bin/bash
if [ "$(readlink -- "/etc/crypttab")" = "crypttab.unlocked" ]; then
        echo "Already unlocked";
fi
mv /etc/crypttab /etc/crypttab.backup
echo 'Using unlocked crypttab';
ln -s /etc/crypttab.unlocked /etc/crypttab
cd /boot/
# echo 'Removing initrd';
# rm initrd.img*
echo 'Regenerating initrd';
update-initramfs -c -k all -v

Make both files executable.

Note, the removal of initrd files has been commented out since sometimes update-initramfs fails to generate an initrd, then the system halts on boot since it is left without any initrd, please check the comment about this about boot failure in another section in this note below.

Then copy /etc/crypttab to two files:
/etc/crypttab.unlocked - edit this copy so it has the keyscript option on relevant partitions
/etc/crypttab.locked - edit this copy so it does not have the keyscript option on relevant partition, just luks

Then when you run cryptlock or cryptunlock it will make crypttab symbolic link to one of these files and regenerate initramfs with the linked file.

Note, if update-initrd says Available versions: without any version and then Nothing to do, then it failed to generate the locked/unlocked initrd version, check the comment about boot failure below.

Dropping to a shell recovery

If the boot fails you will be dropped to a shell. To recover from this you may need a live installation iso/USB stick. Boot from that and do the following.

sudo bash
apt-get update
apt install cryptsetup lvm2
# open the root partition
cryptsetup luksOpen /dev/sdXY sdXY_crypt 
vgchange -ay
mount /dev/mapper/sdXY_crypt /mnt
mount /dev/mapper/sdXZ_crypt /mnt/home/  
mount /dev/sdXX /mnt/boot

if UEFI:
mkdir -p /mnt/boot/efi
mount /dev/sdNN /mnt/boot/efi

mkdir -p /mnt/run/udev

for i in /dev /dev/pts /proc /sys /sys/firmware/efi/efivars /run/udev; do mount -B $i /mnt/$i ; done

OR:

mount -t proc proc /mnt/proc
mount -o bind /dev /mnt/dev
mount -o bind /dev/pts /mnt/dev/pts
mount -o bind /sys /mnt/sys
mount -o bind /sys/firmware/efi/efivars /mnt/sys/firmware/efi/efivars
mount --bind /run/udev /mnt/run/udev

The mounting of /run/udev might be EFI related, but if not done it might output the following warning if GRUB is installed from within the chroot:
WARNING: Device /dev/sdXY not initialized in udev database even after waiting 10000000 microseconds.

The mounting of /sys/firmware/efi/efivars is because otherwise the error "EFI variables are not supported on this system" will be outputted by grub-install and efibootmgr. Reference: https://unix.stackexchange.com/questions/91620/efi-variables-are-not-supported-on-this-system.

chroot /mnt
# check for the network interface name
ip link show
# check for connection
ping google.com
# if it responds, do Ctrl+C, if not - this responds File exists, but works anyway:
dhclient en0ps3
mount -t sysfs sys /sys
apt-get install lvm2
# go into boot, otherwise initramfs cannot be regenerated, returns Cannot create version <version> already exists
cd /boot
# remove the initd.img-<version> file here
update-initramfs -c -k all
exit
reboot

Resize luks partitions with gparted

Gparted 0.28+ is required. Gparted 0.25 is included in Debian 9.9, so download the live ISO from Gparted instead.
Open a terminal, do the usual cryptsetup luksOpen /dev/sdXX sdXX_crypt for each partition
Then start or refresh gparted

https://gist.github.com/dragon788/e777ba64d373210e4f6306ad40ee0e80

Convert unencrypted Debian 9+ system to encrypted using luksipc

This will convert an unencrypted Debian system to encrypted, using luksipc.

Note, if you are missing a /boot partition and want that located before the root partition, then it may be wise to deal with that first. See separate note about changing order of partitions.

Warning! Read the details in luksipc read me:
https://github.com/johndoe31415/luksipc/blob/master/README.md

- Ensure that there is 10+ MB free to use in the source partition(s) to be converted.
- Make a backup of the partition contents.

Starting off with a small Debian 9.6 system (in a VirtualBox), consisting of this:
/dev/sda1 5 GB, unencrypted, containing ext4, /, with 10+ MB free

This will be converted to:
/dev/sda1
-- /dev/sda1_crypt - encrypted, containing ext4, /
/dev/sda2 - unencrypted, containing /boot

The boot process in short: boot sector -> grub -> /dev/sda2 -> initramfs mounts root -> /dev/sda1 -> systemd mounts the rest.

Requirements are: Debian Live ISO

Note: Debian Live ISO 9.6 contains Gparted 0.25 while version 0.28+ is needed to work with encrypted partitions, this is circumveited below by first editing partitions, then encrypting them.

Insert Debian Live ISO and boot, open a terminal.

# install
sudo bash
apt update
apt install cryptsetup git gparted lvm2 -y

# run gparted
gparted

Resize /dev/sda1 and make space for a 256 MB partition either to the left or to the right of it.
Create a new partition of 256 MB, format it with Ext4, this will be /boot.

Important: switch the boot flag on this partition. This is needed, otherwise grub will not find the file system. This may break the boot though, so you may need to move it back to where it was (partition one?) afterwards.

# get luksipc:
apt install gcc make
git clone https://github.com/johndoe31415/luksipc.git
cd luksipc
make

# maybe not needed, but resize2fs complained:
e2fsck -f /dev/sda1

# get partition details:
tune2fs -l /dev/sda1|grep Block

# check Block size and Block count, Block size * Block count = bytes used.
# to get size in MB take bytes used / 1024 / 1024
# then decrease with at least 10MB, but to be safe 100MB
# then recalculate the new value * 1024 * 1024 / block size = value to give to resize2fs
# to resize a file system to 4500 MB, calculate:
# 4500 MB * 1024 * 1024 / 4096 blocks => 1152000
# resize2fs /dev/sda1 1152000
# if you get "New size too large to be expressed in 32 bits"
# then you may not have divided by block size

resize2fs /dev/sda1 <sectors>

# read the warnings, then type YES here
luksipc -d /dev/sda1

# very important, add a password to gain access, since the live key will be dropped
cryptsetup luksAddKey /dev/sda1 --key-file=/root/initial_keyfile.bin

# you now are safe to gain access if the system will be rebooted, verify, check that Key Slot 0-1 are enabled:
cryptsetup luksDump /dev/sda1

# remove the keyfile, now slot 0 gets freed:
cryptsetup luksKillSlot /dev/sda1 0

# add key again to get it on slot 0, enter password again:
cryptsetup luksAddKey /dev/sda1

# drop the previous password on slot 0
cryptsetup luksKillSlot /dev/sda1 1

# check that slot 0 is the only enabled, the password:
cryptsetup luksDump /dev/sda1

# open sda1, this maps /dev/sda1 to /dev/mapper/sda1_crypt
cryptsetup luksOpen /dev/sda1 sda1_crypt

# resize fs back to normal:
resize2fs /dev/mapper/sda1_crypt

# maybe not needed
vgchange -aay

# -- note, rebooted here to add missing boot partition

# mount root
mount /dev/mapper/sda1_crypt /mnt

# work with boot files, move them out of the way and back into the partition
mkdir -p /mnt/oldboot
mv /mnt/boot/* /mnt/oldboot

# mount boot - do not forget this even if you already has separate boot partition:
mount /dev/sda2 /mnt/boot

mv /mnt/oldboot/* /mnt/boot
rmdir /mnt/oldboot

# continue mounting
mount -t proc proc /mnt/proc
mount -o bind /dev /mnt/dev
mount -o bind /dev/pts /mnt/dev/pts
chroot /mnt

# you are now inside the root partition
mount -t sysfs sys /sys

# install to root partition
apt update
apt install cryptsetup lvm2

# take note of outer root partition (sda1) and boot partition (sda2) UUID
blkid

# edit fstab
vi /etc/fstab

# in this file add the following line after <dump> 
# but before the root mount without a bracket in the beginning,
# note, do NOT put any options (like 0 0) after the file system type, it interrupts the boot process
# UUID=<boot-partition-UUID> /boot ext4

# optional, test
mount -a

# edit crypttab
vi /etc/crypttab

# in this file add this line without bracket in the beginning:
# sda1_crypt UUID=<UUID-of-sda1> none luks

# optional, test the file:
sudo cryptdisks_start sda1_crypt

# update grub
update-grub

# optional, ensure grub is in sda
grub-install /dev/sda

# optional, remove unwanted kernels, check what's used and remove the rest
uname -r
dpkg --list|grep linux-image
dpkg --list|grep linux-headers
apt --purge remove linux-image-<any to remove>

# update initramfs - remove old initrd and regenerate
cd /boot
rm initrd*
update-initramfs -c -k all

# run gparted and put the boot flag back to partition 1?

# get out and reboot
exit
reboot

You may now end up in:

A) grub rescue mode, this was solved by booting live ISO and setting the boot flag on /dev/sda2

B) initramfs, not being able to find partition, re-gain access by following the section for when dropping to a shell and ensure /etc/crypttab is correct

C) boot scripts, halted with a prompt press Ctrl+D or password, login, do journalctl -xb and read what may have failed and attempt to fix it

D) the system back online, encrypted.

WARNING: Setting CRYPTSETUP in /etc/initramfs-tools/initramfs.conf is deprecated

If you run update-initramfs -c -k all you may get this warning:
WARNING: Setting CRYPTSETUP in /etc/initramfs-tools/initramfs.conf is deprecated and will stop working in the future. Use /etc/cryptsetup-initramfs/conf-hook instead.

But if you check /etc/initramfs-tools/initramfs.conf then no CRYPTSETUP is there. This is because it is done by a hook and it is an internal bug and nothing to worry about.

Source: https://askubuntu.com/questions/982583/warnings-about-cryptsetup-on-new-ubuntu-17-10-installation

Source UEFI: https://www.linuxtopic.com/2017/08/convert-mbr-to-gpt.html

Boot failure after lock/unlock due to update-initramfs failing to generate initrd

When using the lock script on kernel 4.19.0-16 I was left in a broken boot without any /boot/initrd-image.

The reason? 

The lock script removes all initrd* files, but update-initramfs -c -k all did not generate a new /boot/initrd nor did it output any error.

Running update-initramfs -c -k all -v displayed something like:
Available versions:
Nothing to do, exiting.

Note the Available versions: line, after that the kernel version should have been displayed. Shame on this tool in that silently exits without any warnings - this is a huge show stopper that is just ignored.

The solution was to take note of what kernel version that was requested, chroot into the encrypted partition from another updated installation, then do:

apt update

apt upgrade

Then use mkinitramfs to do what update-initramfs fails to do:
cd /boot; mkdir temp; export TMPDIR=/boot/temp; mkinitramfs -o initrd.img-<requested kernel version>

Replace <requested kernel version> with for example 4.19.0-16-amd64.

Then try update-initramfs again:
update-initramfs -c -k all

Reference: https://askubuntu.com/questions/456903/ubuntu-14-04-not-booting-update-initramfs-nothing-to-do-exiting