Bootable USB Key


From Knoppix Documentation Wiki
Revision as of 19:08, 20 June 2007 by Neva (Talk | contribs)

Jump to: navigation, search

Knoppix 5.0.1 boots rather nicely from a USB key. It's getting easier to get it working, as the USB bugs from the previous versions have been fixed.

This is a "recipe" for creating a Knoppix bootable USB key. It was originally posted to the forums: http://knoppix.net/forum/viewtopic.php?t=25402

Ingredients

  • Knoppix 5.0.1 CD (or any later version)

It must be the CD. The DVD is much too large.

However, if your USB key is 5GB or larger, you can experiment with trying to use the DVD instead of the CD.

  • 1GB or larger USB key, U3-free

I recommend a USB key of exactly 1GB in size.

Smaller sizes won't hold the entire CD.

Larger sizes may have trouble booting on older BIOS's (see below about USB-ZIP).

Back up all data on your USB key first, because we will be erasing and reformatting it!

Unfortunately, many newer USB keys come with "U3" Windows software (badware?) preinstalled. Removing U3 is highly recommended, and will free up additional space on your USB key, as well as eliminate a source of potential conflicts.

Find a Windows machine, and try these programs to uninstall U3. Unfortunately, they are a bit flaky. If one does not work, try the other:

http://www.sandisk.com/Assets/u3/launchpadremoval.exe

http://www.u3.com/uninstall/final.aspx

To my knowledge, U3 has not yet been reverse-engineered, and the U3 vendors are secretive, so there's no way to manipulate it from Linux.

The U3 removal software searches for a valid U3 installation before removing it! There seems to be no way to just wipe U3 cleanly away, unfortunately. You have to infect your computer with U3, in order to use their software to clean your USB key. About the only workaround for this Catch-22 situation is to avoid purchasing a USB key with U3 in the first place.

Instructions

  • Boot Knoppix 5.0.1 (or any later version) from CD, and make sure that it works.

As soon as the title screen of Knoppix appears (the cheatcode entry screen), put in the USB key.

Do this before Knoppix starts to load, so that your USB key is sure to be found by Knoppix's autodetection.

Do not use the "toram" cheatcode when booting Knoppix. The RAM copy of the CD omits the /boot directory, but we require this directory, so we can't use the RAM copy.

  • Once Knoppix is up and running, open a root shell (from the penguin icon).

Now, we need to find the drive letter of your USB key. It will be something like /dev/sda, /dev/sdb, /dev/sdc, and so on. Be very careful and make absolutely sure you have the drive letter correct, because you will be erasing the USB key, and you don't want to erase your hard drive by mistake!

Examples of possible commands to try:

dmesg | grep -i scsi
dmesg | grep -i usb
cat /proc/partitions
fdisk -l

Look at the sizes of the various drives you see here. Find your USB key, and do not get it confused with your hard drive! For the rest of this example, I'm going to be using /dev/sdz, a fictional drive letter that nobody has, just to prevent bad mistakes from typing in commands blindly.

Note 
If your USB stick is FAT-formatted and has enough free capacity, you can skip the following repartitioning phase and jump to installing SYSLINUX section. Just make sure your stick's partition has it's boot flag activated so the boot loader works properly. This can be done with the command fdisk /dev/sdz and subsequent commands (example: a 1 p w )
  • Erase your entire USB key.
dd if=/dev/zero of=/dev/sdz

Did you use the right drive letter? Did you? If not, you'll be crying, right about now....

This command will take a while to finish. By writing to every block on the USB key, we are also testing it for bad blocks.

You should get the "no space left on device" error message when it finishes erasing and reaches the end of the drive. Any other error messages might mean bad blocks: if you notice any, your key might be worn out and you'll need to buy another one (cheap keys are notorious for wearing out very quickly).

  • Run the mkdiskimage script, to automatically partition and format your USB key.

Choose only one of these sections to try, based on the size of your USB key.

If your USB key is exactly 1GB in size:

mkdiskimage -4 /dev/sdz 0 64 32

Here are some alternatives you can try, for larger USB keys. If the above command worked for your USB key, then you do not need to try these! Simply proceed to the SYSLINUX step below.

If your USB key is 2GB or less in size, but larger than 1GB:

mkdiskimage -4 /dev/sdz 0 128 32

If your USB key is 8GB or less in size, but larger than 2GB:

mkdiskimage -F -4 /dev/sdz 0 255 63

If your USB key is larger than 8GB in size (see cautions below):

mkdiskimage -F -4 /dev/sdz 1 255 63
dd if=/dev/zero of=/dev/sdz bs=1 seek=446 count=64
echo -e ',0\n,0\n,0\n,,C,*' | sfdisk /dev/sdz
mkdosfs /dev/sdz4

The final section above, for keys larger than 8GB, is rather tricky. The mkdiskimage script can't handle partitions larger than 8GB, so I have to hack around it a bit. What we are doing here is making a dummy partition of only 1MB(!) in size, so that mkdiskimage can prepare the MBR properly. Then, we manually delete that partition, by doing some surgery in the partition table. We then use sfdisk to re-create the partition properly, using the full size of your USB key as FAT32, and finally format it with mkdosfs.

Please be very careful typing in the command(s) in your section above. If using the final section for keys larger than 8GB, the 0's in the echo command above are zeroes, not the letter O, and please be sure to use single quotes instead of double quotes. This should work for all USB storage devices, up to 2TB in size. I tested this using a 320GB USB hard drive, and it worked. Beyond 2TB, you're on your own....

This lays down a new MBR and partition table on the drive, and partitions and formats the entire drive as one big partition, either FAT16 or FAT32. Even though I prefer FAT32, I'm still recommending FAT16 for smaller keys, for increased compatibility with some strange BIOS's.

The mkdiskimage script is USB-ZIP compliant, which may help it boot on more systems than USB-HDD. It's nice that Knoppix now includes this script by default. Here's the webpage for mkdiskimage, for additional reference:

http://syslinux.zytor.com/usbkey.php

  • Close KDE's popup window.

KDE may throw a stupid popup window in your face, once it recognizes your new partition. If this happens, close the window.

  • Run the SYSLINUX installer on your new partition.
syslinux -s /dev/sdz4

This installs the SYSLINUX bootloader onto the newly-created partition, and includes a special file, ldlinux.sys. The -s option is for safety and additional compatibility. Notice the partition is the fourth partition on the drive, not the first. This is for USB-ZIP compliance.

  • Mount your new partition.
mount /media/sdz4

Knoppix should have autodetected the new partition and set up a fstab entry for it. The /media/sdz4 directory should now correspond to the /dev/sdz4 partition.

cd /media/sdz4
ls

You should notice one, and only one, file in this directory now: ldlinux.sys

If you don't see this, then something probably went wrong earlier. Go back and try again before continuing.

  • Copy the contents of the Knoppix CD's boot directory, onto your USB key.

You should already be in the /media/sdz4 directory, from the cd command above.

cp -av /cdrom/boot/isolinux/* .
mv isolinux.cfg syslinux.cfg
rm -f isolinux.bin

Don't forget the dot on the end. It tells the cp command to copy everything into your current directory, which is what we want. Notice that there is a space between the asterisk and the dot!

Ignore any errors about "failed to preserve ownership". This is normal, because we're using the FAT filesystem, which does not support file ownership. This is not a problem for us.

What we're doing here is copying the Knoppix CD's /boot directory first. This increases reliability, by getting it as close to the beginning of the drive as possible. This should help on systems with buggy CHS handling of USB keys, which may become an issue as new USB keys get larger.

We are renaming Knoppix's isolinux.cfg file to syslinux.cfg, and then removing the unnecessary isolinux.bin file. ISOLINUX (used to boot the CD) and SYSLINUX (what we just installed to your USB key) are really two different variations of the same bootloader program. They accept the same config file format, so we don't need to change anything at all, just rename.

BTW, since ISOLINUX has isolinux.bin as its main program file, can you guess what is the equivalent file for SYSLINUX? If you guessed ldlinux.sys, the file you saw earlier, good job!

  • Finish copying the rest of the Knoppix CD.

Again, you should still be in the /media/sdz4 directory. Again, don't forget the dot on the end of this command.

cp -av /cdrom/* .
rm -rf ./boot

This should take a while. We are filling up the USB key with the contents of the Knoppix CD.

We already copied Knoppix's /boot directory earlier, in the previous step. We don't need two copies of it just wasting space, so we remove the extra copy.

  • You are now ready to boot!

Cleanly unmount the USB key.

sync
cd /
umount /media/sdz4
sync

Shut down Knoppix, and remove the CD when it tells you to.

  • Reset your computer and fiddle with the BIOS until it sees the USB key.

After unmounting above, the USB key can be removed. Some BIOS's require a removal and re-insertion before they will recognize it.

Other things to try, if you're having trouble getting the BIOS to see your USB key and let you boot from it:

  1. Try a different USB port. Some BIOS's only search a few ports, not all of them. Avoid using a USB hub, if possible.
  2. Hit a special key to bring up a boot menu. Some BIOS's can only boot from USB when manually selected.
  3. Go into your BIOS's setup menu, and walk through the boot settings. Look under all menus, especially "hard drives" and "removable devices" and other things like that. Some BIOS's will not search for newly attached devices until you manually go through the menus again.
  4. Turn your computer completely off and on again. Some BIOS's require you to physically turn the computer off and on again, before they will see newly attached "hard drives".
  5. If all else fails, try a different PC. It's still fairly rare these days to find a computer that has a good ability to boot from USB devices, and it's not something that motherboard makers advertise, so it takes a bit of luck to find a computer that can do this.
  • Boot into Knoppix from your USB key.

The USB key will be slow to load at first, especially during the cheatcode screen, because most BIOS's will default to running USB devices in a very slow mode. Once Knoppix detects and autoconfigures the USB port during booting, though, it will return to normal speed.

If it works, that's great. Notice how much faster it is. There's zero seek time on a USB key, so even though it isn't very fast, it feels a lot faster than the CD. Programs are surprisingly quick to load.

Find the drive letter for your USB key again. It should not change, but it might have.

  • Make the USB key writeable.

Knoppix assumes that the "CD-ROM" that it booted from is impossible to write to, but we have a USB key now, not a CD-ROM. So, we must fool Knoppix into letting you write to it.

mount -o remount,rw /cdrom
mount --bind /cdrom /media/sdz4
  • Optional step: Modify the Knoppix boot menu

This is an optional step, for customization. We want Knoppix to autofind the persistent home directory (once we make it), and fix a few other things.

vi /media/sdz4/syslinux.cfg

On line 3, the default timeout is 300. Lower this, to around 50 or so (5 seconds, since this counter is in tenths of seconds).

It's nice to have it boot faster, without waiting for you to hit Enter, so you can put in the USB key and walk away, coming back in a few minutes to a fully booted system.

On line 2, add a few more options to the end of the line.

noswap noeject noprompt dma home=scan

Here's a little explanation of what each option does.

noswap 
Security fix: assuming you're going to be using the USB key on a bunch of different computers you don't own, you don't want to be swapping your memory onto their hard drives.
noeject 
You can't physically eject a USB key.
noprompt 
Gets rid of the "remove CD and close drive door" prompt when shutting down, as this prompt is rather misleading when it's really a USB key instead.
dma 
Gets you a significant speed increase when dealing with many IDE drives, a good thing to have.
home=scan 
This will cause Knoppix to pick up your persistent home directory automatically the next time you boot up, asking you for the password as needed. You will get an extra menu during bootup, asking you if you want to mount your home directory or not.

Save the file and quit the editor.

  • Optional step: Create a Knoppix persistent home directory

Now, we can run the Knoppix persistent utility, to create something that will save your settings across a reboot.

(It's under the penguin menu, Configure, Create Persistent.)

Choose your drive from the menu that appears, and follow the instructions.

When making your encrypted persistent home directory, choose a size that is a good balance. I recommend not using the entire remaining space on the USB key. It's handy to have some extra free space, for quick file copying and such. Also, if you have a large USB key with roughly 260MB of free space or more, and need to also work under Windows, you can install PortableApps (see below).

The program should complete without errors. If the program suddenly disappears midway through, check the kernel logs with "dmesg" again. If you see pagefuls of SCSI errors and such, beware, you might have a worn-out USB key!

You're done!

Now, you should be all set. Reboot again, to make sure it all works.

Enjoy your Knoppix-on-a-keychain!

Josh

--Krellan 00:43, 10 Sep 2006 (GMT)

Large USB keys

The mkdiskimage command is very sensitive to the size of your USB key. I have included several variations to try, in the above article. Here are some more alternatives, for completeness.

These will break USB-ZIP compatibility, as it is a USB-ZIP requirement to have 64 heads and 32 sectors. Unfortunately, there is no other way, as a valid cylinder count (less than 1024) is also a requirement. So, try USB-HDD instead of USB-ZIP. The tradeoff is that it won't be as compatible with as many BIOS's as USB-ZIP.

mkdiskimage -4 /dev/sdz 0 255 63

This will allow you to partition a USB key up to 8GB in size. You probably could also use 256 heads instead of 255, for a little more space, but 255 heads is recommended, as it's slightly more compatible.

You may then run into the 2GB limit of the FAT16 partition. Beyond 2GB, it is probably best to use FAT32 instead. Supply the -F option to mkdiskimage.

mkdiskimage -F -4 /dev/sdz 0 255 63

At first, I did recommend FAT32 for all users: http://www.knoppix.net/forum/viewtopic.php?t=25402

However, it did not work on as many BIOS's, so I now recommend FAT16, unless you have a USB key that is larger than the 2GB limit of FAT16.

The other instructions on the page above should still be valid. Only the mkdiskimage command line needs to be changed, for use with large keys.

--Krellan 20:30, 19 Oct 2006 (GMT)

PortableApps

Here's also something great I found. If you have roughly 260MB or more of free space remaining on your USB key, you might want to drop PortableApps on there.

http://portableapps.com/suite

http://portableapps.com/

This is a suite of freeware Windows utilities, and will help you if you need to do work within Windows without being able to reboot into Knoppix. Much better and cleaner than U3!

When installing PortableApps to a USB key that already has Knoppix on it, PortableApps will complain about overwriting the autorun.inf file. This is fine. Say Yes to the overwrite. This will allow the USB key to bring up the PortableApps menu, instead of the Knoppix homepage, when clicking on it from within Windows.

With a 1GB key formatted as FAT16, as per the above instructions, there is, unfortunately, just barely not enough room for both Knoppix and the full PortableApps suite to coexist. The workaround is to "install" PortableApps to a spare empty folder on your hard drive first. Make edits to that folder as necessary, deleting some apps you can live without. Then, copy the entire folder manually to your USB key from there.

Formatting as FAT32 will work, as FAT32 is more efficient at cramming in the thousands of small files PortableApps creates. A larger USB key will provide much more room for both Knoppix and PortableApps, and enough extra left over for a decently-sized Knoppix home directory as well. See above for alternate mkdiskimage command lines you can try, when formatting larger USB keys or using FAT32.

--Krellan 19:23, 27 Nov 2006 (GMT)


There is an another project based on PortableApps, with the addition of TOR-networking enabled by default. This makes it possible to anonymize web surfing at the cost of speed.

More info at http://www.democrakey.com/

--Neva 18:54, 20 Jun 2007 (GMT)


mkdiskimage won't set up a 2 gb USB device with the USB-ZIP format using the example given. According to other FAQ's this is an issue with too many cylinders. A 1 gb drive is supposed to work fine. You may be able to manually create two drives on the device and manually format it in order to set it up as USB-ZIP.

Depending on the USB device, you can use fdisk to figure out the cylinders, heads and sectors and get mkdiskimage to work with these values. The one I used had 1015 cylinders, 64 heads and 62 sectors per track -- these values worked fine and I was able to create a bootable Knoppix USB key.


I was able to use this guide to setup and boot from Knoppix on a Creative Zen Vision:M MP3 player. Details here: http://www.francisshanahan.com/detail.aspx?cid=509