Dienstag, 31. März 2015

HOWTO: Moving Raspberry Pi 2 filesystem to a USB.stick or USB.harddisk


by paulv » Sat May 18, 2013 1:26 pm
I use one of my Pi's (Pies?) in a more or less Mission Critical Application on another continent, and was getting concerned with the reports about SD cards getting corrupted. Apparently, these cards are not really made for heavy read/write applications, although nice and cheap in educational and hobby environments.

If you want to offload the main filesystem to another medium, there are several examples if you Google around enough. Unfortunately, some are not really applicable to the Raspberry Pi, others are for super-super users, who know what they are doing.

I created this description to make it also usable for power-users, although I am by no means a super-power-user myself. Beware! :roll:

To make it simple and easier, I created two parts to the procedure.
1. For the simple use of the Pi with only 1 extra USB drive or stick
2. An extended version if you need more than one drive attached to your Pi. (Raspbmc users?) Note that this version is a little more complicated.

We will start off with a brand new SD card, and a brand new USB stick.
The size and speed of the SD card is no longer important and any card over 2G will do. (it has to be 2G because we're going to load complete Debian on it)
The size of the USB drive/stick depends on your application, anything over 2G will do. Speed matters a bit more here, but I'm not going for a speed improvement, that's another topic.

In this description I will use a micro SDHC memory card with 4GM, class 10, because I use the cards for other applications as well and I don't like the SD card sticking outside of the Pi cover (already broken one SD card adapter). And I also use a micro USB stick with a size of 8GB.

First of all, you need to load the Wheezy Raspbian image (I used 2013-02-09) on the SD card. There are enough procedures to tell you how to do that. I use win32diskimager on my W7 laptop.
Once that is done, you can put it in your Pi and boot it up.
If you use a LAN cable, you're good to go. If you run a headless Pi (like I do) over WIFI, you need to configure that first, again, there are many procedures on how to do that for your particular dongle. I use an EDIMAX in this particular setup. Obviously, I'm using ssh, but that does not matter.


First of all, you should run raspi-config and setup ONLY the following:
Code: Select all
sudo raspi-config

configure_keyboard (not needed for a headless Pi)
change_pass (change the raspberry password if you wish at this moment, can also be done later)
change_locale
change_timezone
memory_split (a headless can be set to 16MB)
Do not update/upgrade at this moment! That will cause a lot of thrashing on the SD card, which is what we want to avoid.

After a reboot to load the new parameters, we're good to go.

Check the filesystem:
Code: Select all
pi@raspberrypi ~ $ df -h
Filesystem      Size  Used Avail Use% Mounted on
rootfs          1.8G  1.4G  247M  86% /
/dev/root       1.8G  1.4G  247M  86% /
devtmpfs        235M     0  235M   0% /dev
tmpfs            49M  248K   49M   1% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs            98M     0   98M   0% /run/shm
/dev/mmcblk0p1   56M   19M   38M  34% /boot
pi@raspberrypi ~ $
Notice that although I use an 8GB SD card, the default is 1.8G in size, because we did not use expand_rootfs.
The entry /dev/mmcblk0p1 is the boot partition (p1) on device mmcblk0, the SD card.

The boot partition will be the only piece on the SD card used when we're done, and that is because of the particular way the Pi goes through the boot process. It actually starts with the graphics chip loading this data first, and it then releases the reset pin of the CPU which continues and takes over.

Now plug in your USB drive/stick if you have not done that yet (I'll refer to stick from now on)

Check to see if it's recognized:
Code: Select all
pi@raspberrypi ~ $ lsusb
Bus 001 Device 002: ID 0424:9512 Standard Microsystems Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 003: ID 0424:ec00 Standard Microsystems Corp.
Bus 001 Device 004: ID 7392:7811 Edimax Technology Co., Ltd EW-7811Un 802.11n Wireless Adapter [Realtek RTL8188CUS]
Bus 001 Device 006: ID 8644:800b
pi@raspberrypi ~ $
Device 006 is the USB adapter. The ID shows a unique manufacturer identifier.
Also check to see if the OS recognized the device: (you can also use the command dmesg)
Code: Select all
pi@raspberrypi ~ $ tail /var/log/messages
May 18 11:44:31 raspberrypi kernel: [   64.093866] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
May 18 11:44:31 raspberrypi kernel: [   64.093883] usb 1-1.3: Product: Intenso Micro Line
May 18 11:44:31 raspberrypi kernel: [   64.093898] usb 1-1.3: Manufacturer: Intenso
May 18 11:44:31 raspberrypi kernel: [   64.093912] usb 1-1.3: SerialNumber: 12390000000004E2
May 18 11:44:31 raspberrypi kernel: [   64.112228] scsi1 : usb-storage 1-1.3:1.0
May 18 11:44:32 raspberrypi kernel: [   65.113335] scsi 1:0:0:0: Direct-Access     Intenso  Micro Line       1.01 PQ: 0 ANSI: 2
May 18 11:44:32 raspberrypi kernel: [   65.114862] sd 1:0:0:0: [sda] 15669248 512-byte logical blocks: (8.02 GB/7.47 GiB)
May 18 11:44:32 raspberrypi kernel: [   65.115607] sd 1:0:0:0: [sda] Write Protect is off
May 18 11:44:32 raspberrypi kernel: [   65.126062]  sda: sda1
May 18 11:44:32 raspberrypi kernel: [   65.129907] sd 1:0:0:0: [sda] Attached SCSI removable disk
pi@raspberrypi ~ $
Yep, the USB stick can be referenced as /dev/sda

There is no way of knowing what name the OS assigns to additional USB drives, if that's what you want to do, you need to follow the extended version of the procedure down below.

If one USB memory stick is what you're going to use, start here:

Create a filesystem on the USB stick, use "p" to list the partitions
Code: Select all
pi@raspberrypi ~ $ sudo fdisk /dev/sda

Command (m for help): p

Disk /dev/sda: 8022 MB, 8022654976 bytes
94 heads, 14 sectors/track, 11906 cylinders, total 15669248 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1              32    15669247     7834608    b  W95 FAT32

Command (m for help):
There is a partition, formatted for Windows. We need to reformat the drive for Linux.
Enter d to delete partion(s).
Note
If you are going to use the stick only for the Debian filesystem, go on. If you have a large stick or a USB drive, you may want to consider making one partition for Debian, and other ones for whatever you like. This is beyond the scope of this article, but I'm sure you can figure it out.
We will continue with the simple solution, one partition.

Enter n to create a new primary partition ( number 1) and use the full capacity by hitting return a few times until done:

Code: Select all
Command (m for help): d
Selected partition 1

Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-15669247, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-15669247, default 15669247):
Using default value 15669247

Command (m for help):
Now write the new information to the stick by entering "w":

Code: Select all
Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
pi@raspberrypi ~ $
Done, so now we can format the partition:
Code: Select all
pi@raspberrypi ~ $ sudo mke2fs -t ext4 -L rootfs /dev/sda1
mke2fs 1.42.5 (29-Jul-2012)
Filesystem label=rootfs
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
490560 inodes, 1958652 blocks
97932 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2009071616
60 block groups
32768 blocks per group, 32768 fragments per group
8176 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

pi@raspberrypi ~ $
And we need to mount the partition to start using it:
Code: Select all
pi@raspberrypi ~ $ sudo mount /dev/sda1 /mnt
pi@raspberrypi ~ $
Let's see what we've done so far:
Code: Select all
pi@raspberrypi ~ $ df -h
Filesystem      Size  Used Avail Use% Mounted on
rootfs          1.8G  1.4G  247M  86% /
/dev/root       1.8G  1.4G  247M  86% /
devtmpfs        235M     0  235M   0% /dev
tmpfs            49M  248K   49M   1% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs            98M     0   98M   0% /run/shm
/dev/mmcblk0p1   56M   19M   38M  34% /boot
/dev/sda1       7.4G  145M  6.9G   3% /mnt
pi@raspberrypi ~ $
We made the stick /dev/sda1 available as /mnt for Linux, so now we can start to use it.

We can copy (mirror) the data in /dev/root to the new drive.
We can use dd, but that is a bit copy program and because it also copies the empty space on the SD card, it is very, very slow.
Better is to use rsync.
Install that first:
Code: Select all
pi@raspberrypi ~ $ sudo apt-get install rsync
...
pi@raspberrypi ~ $
And run it. Although much faster, it will still take some time to create a mirror image of the file system on the USB drive. We will copy everything from "/" to the just mounted drive /mnt (which is /dev/sda1)

Code: Select all
pi@raspberrypi ~ $ sudo rsync -axv / /mnt
...
On my system it took about 15 minutes. Make sure this process does not get interrupted!
If you want, you can open another ssh session and continue with the process, so you don't have to wait for the rsync process to complete.

We're going to tell the Pi where to find the new data when it boots Debian.
The file we need to change is /boot/cmdline.txt
Before we're going to do that, we're going to create a backup copy so we can change things back if there is a problem. Be careful, if you mess up this file, you will almost certainly stop the Pi from booting, and you may have to start all over again by reloading the image.

Code: Select all
sudo cp /boot/cmdline.txt /boot/cmdline.orig

The contends of this file is one very long command string, and we need to replace the reference to the partition on the SD card (root=/dev/mmcblk0p2) to the new USB stick (root=/dev/sda1)
Here is a copy of the original file:
Code: Select all
pi@raspberrypi ~ $ cat /boot/cmdline.txt
dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline rootwait
pi@raspberrypi ~ $
To be on the safe side, there is one more command we should add. During the power-up/boot sequence, it may take the USB stick a little longer to be ready, than the OS expects. If we add a delay of 5 seconds (rootdelay=5), the USB stick will be ready when it's needed. You may not need it, or a smaller amount, so go figure it out. When my remote system has to reboot, I want it to be safe, a few seconds more is not an issue.

Use your favorite editor to add these two changes like so:
Code: Select all
sudo nano /boot/cmdline.txt

It should look like this:
Code: Select all
pi@raspberrypi ~ $ cat /boot/cmdline.txt
dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 root=/dev/sda1 rootfstype=ext4 elevator=deadline rootwait rootdelay=5
pi@raspberrypi ~ $

Next we need to enter a few things into the filesystem table, /etc/fstab. (on the stick, not the SD card)
Code: Select all
sudo nano /mnt/etc/fstab
Add the following line :
Code: Select all
/dev/sda1    /   ext4    defaults,noatime  0       1
And put a comment in front of the reference to the SD card:
Code: Select all
#/dev/mmcblk0p2  /               ext4    defaults,noatime       0       1

It should now look like this:
Code: Select all
proc            /proc           proc    defaults          0       0
/dev/sda1       /               ext4    defaults,noatime  0       1
/dev/mmcblk0p1  /boot           vfat    defaults          0       2
#/dev/mmcblk0p2  /               ext4    defaults,noatime  0       1
# a swapfile is not a swap partition, so no using swapon|off from here on, use  dphys-swapfile swap[on|off]  for that

This is it for the simple version. you can now reboot the system:
Code: Select all
sudo reboot
Note that you will not see much activity on the ACT LED anymore. This LED only shows R/W activity on the SD card.
After the Pi has booted, check the filesystem:
Code: Select all
pi@raspberrypi ~ $ df -h
Filesystem      Size  Used Avail Use% Mounted on
rootfs          7.4G  1.6G  5.5G  22% /
/dev/root       7.4G  1.6G  5.5G  22% /
devtmpfs        235M     0  235M   0% /dev
tmpfs            49M  248K   49M   1% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs            98M     0   98M   0% /run/shm
/dev/mmcblk0p1   56M   19M   38M  34% /boot
pi@raspberrypi ~ $
The rootfs is now on the stick, notice the size of 7.5G

Changing the swap file size
There is no need to mess with swap partitions on the Pi, this is implemented differently from other Linux systems. Raspbian uses dphys-swapfile, which is a swap-file based solution instead of the "standard" swap-partition solution. It is also much easier to change the size of the swap.

If you need to change the swap file size, proceed as follows:

Look at the swap data first by using the top command:
Code: Select all
pi@raspberrypi ~ $ top
top - 13:28:20 up 3 min,  1 user,  load average: 0.18, 0.37, 0.18
Tasks:  66 total,   1 running,  65 sleeping,   0 stopped,   0 zombie
%Cpu(s):  1.0 us,  0.0 sy,  0.0 ni, 98.7 id,  0.0 wa,  0.0 hi,  0.3 si,  0.0 st
KiB Mem:    497544 total,    55700 used,   441844 free,     5432 buffers
KiB Swap:   102396 total,        0 used,   102396 free,    27592 cached

102396 is 100MB of swap space

The configuration file for the swap space is: /etc/dphys-swapfile

The content is very simple. By default Raspbian has 100MB of swap space:
Code: Select all
pi@raspberrypi ~ $ cat /etc/dphys-swapfile
CONF_SWAPSIZE=100
pi@raspberrypi ~ $
If you want to change the size, you need to modify the number and then resize and restart the dphys-swapfile:

Code: Select all
pi@raspberrypi ~ $ sudo nano /etc/dphys-swapfile
pi@raspberrypi ~ $
I changed the size from 100 MB to 512MB, the full amount of RAM.
After the parameter in the file has been changed run:
Code: Select all
pi@raspberrypi ~ $ sudo dphys-swapfile setup
want /var/swap=512MByte, checking existing: deleting wrong size file (104857600), generating  swapfile ... of 512MBytes
pi@raspberrypi ~ $
To load the new size, we need to stop the swapfile process:
Code: Select all
pi@raspberrypi ~ $ sudo /etc/init.d/dphys-swapfile stop
Stopping dphys-swapfile swapfile setup ..., done.
pi@raspberrypi ~ $
And restart it:
Code: Select all
pi@raspberrypi ~ $ sudo /etc/init.d/dphys-swapfile start
Starting dphys-swapfile swapfile setup ...
want /var/swap=512MByte, checking existing: keeping it
done.
pi@raspberrypi ~ $
And now we can check it again with top:
Code: Select all
top - 13:37:22 up 12 min,  1 user,  load average: 0.01, 0.08, 0.12
Tasks:  64 total,   1 running,  63 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.7 us,  0.3 sy,  0.0 ni, 99.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem:    497544 total,    57016 used,   440528 free,     5556 buffers
KiB Swap:   524284 total,        0 used,   524284 free,    28604 cached
Done!
Now that you have everything running on the USB stick, it's time to update and upgrade Debian:
Code: Select all
sudo apt-get update
sudo apt-get upgrade -y
sudo apt-get autoremove
And off you go...

Changing the process back to the SD card
If you ever run into problems booting from the USB stick, you can try to move the original /boot/cmdline.txt back by putting the SD card back into your PC, and use the Filemanager to copy the file /boot/cmdline.txt to something else (/boot/cmdline.usb) and copy the name of /boot/cmdline.orig to /boot/cmdline.txt. (watch out: do not use rename, because Windows may rename the file called cmdline.txt to cmdline.txt.txt !)
If your system still works with the USB stick, you can also mount the windows partition on the SD card and make the changes to the cmdline files from within Debian.

Before you do that however, remember that we modified the /etc/fstab file on the SD card and then moved it to the USB stick. We need to change the contends back on the SD card to the original settings before we can easily change the filesystem back to the SD card.
If you followed the procedure above, do the following:
Code: Select all
pi@raspberrypi ~ $ df -h
Filesystem      Size  Used Avail Use% Mounted on
rootfs          7.4G  2.5G  4.6G  36% /
/dev/root       7.4G  2.5G  4.6G  36% /
devtmpfs        235M     0  235M   0% /dev
tmpfs            49M  240K   49M   1% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs            98M     0   98M   0% /run/shm
/dev/mmcblk0p1   56M   19M   38M  33% /boot
Use the device name without the partition (p1)
Code: Select all
pi@raspberrypi ~ $ sudo fdisk /dev/mmcblk0

Command (m for help): p

Disk /dev/mmcblk0: 4035 MB, 4035969024 bytes
4 heads, 16 sectors/track, 123168 cylinders, total 7882752 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00014d34

        Device Boot      Start         End      Blocks   Id  System
/dev/mmcblk0p1            8192      122879       57344    c  W95 FAT32 (LBA)
/dev/mmcblk0p2          122880     7882751     3879936   83  Linux

Command (m for help): q
So now that we know what the Linux file system partition is (p2), we can mount it.
Code: Select all
pi@raspberrypi ~ $ sudo mount /dev/mmcblk0p2 /mnt
And check it
Code: Select all
pi@raspberrypi ~ $ df -h
Filesystem      Size  Used Avail Use% Mounted on
rootfs          7.4G  2.5G  4.6G  36% /
/dev/root       7.4G  2.5G  4.6G  36% /
devtmpfs        235M     0  235M   0% /dev
tmpfs            49M  240K   49M   1% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs            98M     0   98M   0% /run/shm
/dev/mmcblk0p1   56M   19M   38M  33% /boot
/dev/mmcblk0p2  3.7G  1.7G  1.9G  47% /mnt
OK, now we can edit the fstab file on the SD card so it looks like the original version again.
Code: Select all
pi@raspberrypi ~ $ sudo nano /mnt/etc/fstab
And check it.
Code: Select all
pi@raspberrypi ~ $ sudo cat /mnt/etc/fstab
proc            /proc           proc    defaults          0       0
/dev/mmcblk0p1  /boot           vfat    defaults          0       2
/dev/mmcblk0p2  /               ext4    defaults,noatime  0       1
# a swapfile is not a swap partition, so no using swapon|off from here on, use  dphys-swapfile swap[on|off]  for that
#filesystem on USB stick

pi@raspberrypi ~ $

With a bit of luck, by just doing a copy of the right /boot/cmdline version on the Windows partition of the SD card, you will be able to change the booting process from the SD card. If not, there is enough information in this procedure to help you out I hope. ;)


Extended procedure:

Because we must force Debian to boot from the right USB device, we need to assign unique identifiers for the partition and the filesystem, rather than the names the system gives to the USB devices, like /dev/sda.

I'm assuming that after reading the above, we can be a little more terse.
Instead of using the standard fdisk command, we will use the more modern version gdisk.
We can install gdisk with:
Code: Select all
sudo apt-get install gdisk

We start the partitioning with:
Code: Select all
sudo gdisk /dev/sda

And we actually use the same sequence as with fdisk. First "d" for delete, then "n" for new and "w" to write it all out. Again use the defaults by hitting return. If you want more partitions, now is the time.

When you use the "i" command (information), you can see the following details:
Code: Select all
pi@raspberrypi ~ $ sudo gdisk /dev/sda
GPT fdisk (gdisk) version 0.8.5

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help): i
Using 1
Partition GUID code: 0FC63DAF-8483-4772-8E79-3D69D8477DE4 (Linux filesystem)
Partition unique GUID: 2C7FD5E5-5CC4-434C-BFF6-F45336F7140B
First sector: 2048 (at 1024.0 KiB)
Last sector: 15669214 (at 7.5 GiB)
Partition size: 15667167 sectors (7.5 GiB)
Attribute flags: 0000000000000000
Partition name: 'Linux filesystem'

Command (? for help):
And this is what we need: Partition unique GUID: 2C7FD5E5-5CC4-434C-BFF6-F45336F7140B
This information (PARTUUID) goes into the /boot/cmdline.txt like so:
Code: Select all
pi@raspberrypi ~ $ cat /boot/cmdline.txt
dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 root=PARTUUID=2C7FD5E5-5CC4-434C-BFF6-F45336F7140B rootfstype=ext4 elevator=deadline rootwait rootdelay=5
pi@raspberrypi ~ $
Don't forget to add the rootdelay!

We now continue with formatting, mounting, installing rsync and the mirroring:
Code: Select all
sudo mke2fs -t ext4 -L rootfs /dev/sda1
sudo mount /dev/sda1 /mnt
sudo apt-get install rsync
sudo rsync -axv / /mnt
Now we need to get a unique identifier for the fstab drive information:

Code: Select all
pi@raspberrypi ~ $ sudo tune2fs -l /dev/sda1
tune2fs 1.42.5 (29-Jul-2012)
Filesystem volume name:   rootfs
Last mounted on:          /
Filesystem UUID:          3d2cb3c9-7e2b-4255-ac63-863bd3ad7993
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
...
pi@raspberrypi ~ $
And this is what we need: Filesystem UUID: 3d2cb3c9-7e2b-4255-ac63-863bd3ad7993

Next we can enter the information into the filesystem table, /etc/fstab (on the stick, not the SD card). Add the following line:
Code: Select all
/dev/disk/by-uuid/3d2cb3c9-7e2b-4255-ac63-863bd3ad7993    /   ext4    defaults,noatime  0       1

And put a comment in front of the reference on the SD card:
Code: Select all
#/dev/mmcblk0p2  /               ext4    defaults,noatime       0       1

And that's it. If you are done checking everything (don't mix up the partition GUID and the filesystem UUID!!!) , you can reboot now and check if all is well.

Have fun! :)

[edit] I have made the changes based on the input from violagirl23 on sept-26-2013. Thank you girl!
Last edited by paulv on Fri Sep 27, 2013 11:29 am, edited 12 times in total.

Keine Kommentare:

Kommentar veröffentlichen

comments are moderated, to prevent spam and trolls: