Bootable USB Key


From Knoppix Documentation Wiki
Revision as of 08:25, 10 September 2008 by 195.165.32.178 (Talk)

Jump to: navigation, search

Knoppix 5.0.1 boots rather nicely from a USB key. It's getting easier to get it working, as the USB bugs from the previous versions have been fixed.

This is a "recipe" for creating a Knoppix bootable USB key. It was originally posted to the forums: http://knoppix.net/forum/viewtopic.php?t=25402

Preparation

  • Download a Knoppix image and burn it to CD or DVD.
  • Back up all data on your USB key, because we will be erasing and reformatting it!
  • Make sure U3 is not installed on the USB key

Unfortunately, many newer USB keys come with "U3" Windows software (badware?) preinstalled. Removing U3 is highly recommended, and will free up additional space on your USB key, as well as eliminate a source of potential conflicts.

Find a Windows machine, and try these programs to uninstall U3. Unfortunately, they are a bit flaky. If one does not work, try the other:

http://www.sandisk.com/Assets/u3/launchpadremoval.exe

http://www.u3.com/uninstall/

To my knowledge, U3 has not yet been reverse-engineered, and the U3 vendors are secretive, so there's no way to manipulate it from Linux.

The U3 removal software searches for a valid U3 installation before removing it! There seems to be no way to just wipe U3 cleanly away, unfortunately. You have to infect your computer with U3, in order to use their software to clean your USB key. About the only workaround for this Catch-22 situation is to avoid purchasing a USB key with U3 in the first place.

Newer SanDisk USB keys come with a built in uninstall application. Try that first, before trying the above solutions.

Instructions

  • Boot Knoppix 5.0.1 (or any later version) from CD, and make sure that it works.

As soon as the title screen of Knoppix appears (the cheatcode entry screen), put in the USB key.

Do this before Knoppix starts to load, so that your USB key is sure to be found by Knoppix's autodetection.

Do not use the "toram" cheatcode when booting Knoppix. The RAM copy of the CD omits the /boot directory, but we require this directory, so we can't use the RAM copy.

  • Once Knoppix is up and running, open a root shell (from the penguin icon).

Now, we need to find the drive letter of your USB key. It will be something like /dev/sda, /dev/sdb, /dev/sdc, and so on. Be very careful and make absolutely sure you have the drive letter correct, because you will be erasing the USB key, and you don't want to erase your hard drive by mistake!

Examples of possible commands to try:

dmesg | grep -i scsi
dmesg | grep -i usb
cat /proc/partitions
fdisk -l
lsscsi -k

Look at the sizes of the various drives you see here. Find your USB key, and do not get it confused with your hard drive! For the rest of this example, I'm going to be using /dev/sdz, a fictional drive letter that nobody has, just to prevent bad mistakes from typing in commands blindly.

Note 
If your USB stick is FAT-formatted and has enough free capacity, you can skip the following repartitioning phase and jump to installing SYSLINUX section. Just make sure your stick's partition has it's boot flag activated so the boot loader works properly. This can be done with the command fdisk /dev/sdz and subsequent commands (example: a 1 p w )
Note
If you're repeating your installation and have already zeroed your USB key at least once, then you can add count=1k to the line below. It becomes dd if=/dev/zero of=/dev/sdz count=1k which will just zero out the first few sectors of your USB key. This avoids wear and tear on your USB key, but still writes to enough sectors to clean out any existing formatting on your USB key.
  • Erase your entire USB key.
dd if=/dev/zero of=/dev/sdz

Did you use the right drive letter? Did you? If not, you'll be crying, right about now....

This command will take a while to finish. By writing to every block on the USB key, we are also testing it for bad blocks.

You should get the "no space left on device" error message when it finishes erasing and reaches the end of the drive. Any other error messages might mean bad blocks: if you notice any, your key might be worn out and you'll need to buy another one (cheap keys are notorious for wearing out very quickly).

  • Run the mkdiskimage script, to automatically partition and format your USB key.

Choose only one of these sections to try, based on the size of your USB key.

If your USB key is exactly 1GB in size:

mkdiskimage -4 /dev/sdz 0 64 32

Here are some alternatives you can try, for larger USB keys. If the above command worked for your USB key, then you do not need to try these! Simply proceed to the SYSLINUX step below.

If your USB key is 2GB or less in size, but larger than 1GB:

mkdiskimage -4 /dev/sdz 0 128 32

If your USB key is 8GB or less in size, but larger than 2GB:

mkdiskimage -F -4 /dev/sdz 0 255 63

If your USB key is larger than 8GB in size (see cautions below):

mkdiskimage -F -4 /dev/sdz 1 255 63
dd if=/dev/zero of=/dev/sdz bs=1 seek=446 count=64
echo -e ',0\n,0\n,0\n,,C,*' | sfdisk /dev/sdz
mkdosfs /dev/sdz4

The final section above, for keys larger than 8GB, is rather tricky. The mkdiskimage script can't handle partitions larger than 8GB, so I have to hack around it a bit. What we are doing here is making a dummy partition of only 1MB(!) in size, so that mkdiskimage can prepare the MBR properly. Then, we manually delete that partition, by doing some surgery in the partition table. We then use sfdisk to re-create the partition properly, using the full size of your USB key as FAT32, and finally format it with mkdosfs.

Please be very careful typing in the command(s) in your section above. If using the final section for keys larger than 8GB, the 0's in the echo command above are zeroes, not the letter O, and please be sure to use single quotes instead of double quotes. This should work for all USB storage devices, up to 2TB in size. I tested this using a 320GB USB hard drive, and it worked. Beyond 2TB, you're on your own....

This lays down a new MBR and partition table on the drive, and partitions and formats the entire drive as one big partition, either FAT16 or FAT32. Even though I prefer FAT32, I'm still recommending FAT16 for smaller keys, for increased compatibility with some strange BIOS's.

The mkdiskimage script is USB-ZIP compliant, which may help it boot on more systems than USB-HDD. It's nice that Knoppix now includes this script by default. Here's the webpage for mkdiskimage, for additional reference:

http://syslinux.zytor.com/usbkey.php

  • Close KDE's popup window.

KDE may throw a stupid popup window in your face, once it recognizes your new partition. If this happens, close the window.

  • Run the SYSLINUX installer on your new partition.
syslinux -s /dev/sdz4

This installs the SYSLINUX bootloader onto the newly-created partition, and includes a special file, ldlinux.sys. The -s option is for safety and additional compatibility, although if you want your USB key to boot a little faster, you can omit the -s option. Notice the partition is the fourth partition on the drive, not the first. This is for USB-ZIP compliance.

  • Mount your new partition.
mount /dev/sdz4 /media/sdz4

Although Knoppix should have autodetected the new partition and set up a fstab entry for it, Knoppix's fstab options are incorrect. So, by giving both the device and the mountpoint directory on the command line, we mount it manually, with no options. The /media/sdz4 directory should now correspond to the /dev/sdz4 partition.

cd /media/sdz4
ls

You should notice one, and only one, file in this directory now: ldlinux.sys

If you don't see this, then something probably went wrong earlier. Go back and try again before continuing.

  • Copy the contents of the Knoppix CD's boot directory, onto your USB key.

You should already be in the /media/sdz4 directory, from the cd command above.

rsync -av /cdrom/boot/isolinux/ ./
mv isolinux.cfg syslinux.cfg
rm -f isolinux.bin

This copies all files from /cdrom/boot/isolinux/ to the current directory (the rsync command is an improvement over the cp command for handling large amounts of files, as it is better at handling errors). Ignore any warnings about "failed to preserve ownership". This is normal, because we're using the FAT filesystem, which does not support file ownership. This is not a problem for us.

What we're doing here is copying the Knoppix CD's /boot directory first. This increases reliability, by getting it as close to the beginning of the drive as possible. This should help on systems with buggy CHS handling of USB keys, which may become an issue as new USB keys get larger.

We are renaming Knoppix's isolinux.cfg file to syslinux.cfg, and then removing the unnecessary isolinux.bin file. ISOLINUX (used to boot the CD) and SYSLINUX (what we just installed to your USB key) are really two different variations of the same bootloader program. They accept the same config file format, so we don't need to change anything at all, just rename.

BTW, since ISOLINUX has isolinux.bin as its main program file, can you guess what is the equivalent file for SYSLINUX? If you guessed ldlinux.sys, the file you saw earlier, good job!

  • Finish copying the rest of the Knoppix CD.

Again, you should still be in the /media/sdz4 directory. Again, don't forget the dot on the end of the rsync command.

rsync -av /cdrom/ ./ --exclude boot

This should take a while -- we are filling up the USB key with the contents of the Knoppix CD. The excluded "boot" directory was already copied earlier.

If, during the copy, it asks if you want to overwrite the files cdrom.ico and index.html, say Yes. This message appears because in later Knoppix versions, these files were doubled-up on the Knoppix CD (possibly a bug).

  • You are now ready to boot!

Cleanly unmount the USB key.

sync
cd /
umount /media/sdz4
sync

Shut down Knoppix, and remove the CD when it tells you to.

  • Reset your computer and fiddle with the BIOS until it sees the USB key.

After unmounting above, the USB key can be removed. Some BIOS's require a removal and re-insertion before they will recognize it.

Other things to try, if you're having trouble getting the BIOS to see your USB key and let you boot from it:

  1. Try a different USB port. Some BIOS's only search a few ports, not all of them. Avoid using a USB hub, if possible.
  2. Hit a special key to bring up a boot menu. Some BIOS's can only boot from USB when manually selected.
  3. Go into your BIOS's setup menu, and walk through the boot settings. Look under all menus, especially "hard drives" and "removable devices" and other things like that. Some BIOS's will not search for newly attached devices until you manually go through the menus again.
  4. Enable "USB Keyboard and Legacy Support" if it is disabled.
  5. Turn your computer completely off and on again. Some BIOS's require you to physically turn the computer off and on again, before they will see newly attached "hard drives".
  6. If all else fails, try a different PC. It's still fairly rare these days to find a computer that has a good ability to boot from USB devices, and it's not something that motherboard makers advertise, so it takes a bit of luck to find a computer that can do this.
  • Boot into Knoppix from your USB key.

The USB key will be slow to load at first, especially during the cheatcode screen, because most BIOS's will default to running USB devices in a very slow mode. Once Knoppix detects and autoconfigures the USB port during booting, though, it will return to normal speed.

If it works, that's great. Notice how much faster it is. There's zero seek time on a USB key, so even though it isn't very fast, it feels a lot faster than the CD. Programs are surprisingly quick to load.

Find the drive letter for your USB key again. It should not change, but it might have.

  • Make the USB key writeable.

Knoppix assumes that the "CD-ROM" that it booted from is impossible to write to, but we have a USB key now, not a CD-ROM. So, we must fool Knoppix into letting you write to it.

mount -o remount,rw /cdrom
mount --bind /cdrom /media/sdz4
  • Optional step: Modify the Knoppix boot menu

This is an optional step, for customization. We want Knoppix to autofind the persistent home directory (once we make it), and fix a few other things.

vi /media/sdz4/syslinux.cfg

On line 3, the default timeout is 300. Lower this, to around 50 or so (5 seconds, since this counter is in tenths of seconds).

It's nice to have it boot faster, without waiting for you to hit Enter, so you can put in the USB key and walk away, coming back in a few minutes to a fully booted system.

On line 2, add a few more options to the end of the line.

noswap noeject noprompt dma home=scan

Here's a little explanation of what each option does.

noswap 
Security fix: assuming you're going to be using the USB key on a bunch of different computers you don't own, you don't want to be swapping your memory onto their hard drives.
noeject 
You can't physically eject a USB key.
noprompt 
Gets rid of the "remove CD and close drive door" prompt when shutting down, as this prompt is rather misleading when it's really a USB key instead.
dma 
Gets you a significant speed increase when dealing with many IDE drives, a good thing to have.
home=scan 
This will cause Knoppix to pick up your persistent home directory automatically the next time you boot up, asking you for the password as needed. You will get an extra menu during bootup, asking you if you want to mount your home directory or not.

Save the file and quit the editor.

  • Optional step: Create a Knoppix persistent home directory

Now, we can run the Knoppix persistent utility, to create something that will save your settings across a reboot.

(It's under the penguin menu, Configure, Create Persistent.)

Choose your drive from the menu that appears, and follow the instructions.

When making your encrypted persistent home directory, choose a size that is a good balance. I recommend not using the entire remaining space on the USB key. It's handy to have some extra free space, for quick file copying and such. Also, if you have a large USB key with roughly 260MB of free space or more, and need to also work under Windows, you can install PortableApps (see below).

The program should complete without errors. If the program suddenly disappears midway through, check the kernel logs with "dmesg" again. If you see pagefuls of SCSI errors and such, beware, you might have a worn-out USB key!

You're done!

Now, you should be all set. Reboot again, to make sure it all works.

Enjoy your Knoppix-on-a-keychain!

Josh

--Krellan 00:43, 10 Sep 2006 (GMT)

Large USB keys

The mkdiskimage command is very sensitive to the size of your USB key. I have included several variations to try, in the above article. Here are some more alternatives, for completeness.

These will break USB-ZIP compatibility, as it is a USB-ZIP requirement to have 64 heads and 32 sectors. Unfortunately, there is no other way, as a valid cylinder count (less than 1024) is also a requirement. So, try USB-HDD instead of USB-ZIP. The tradeoff is that it won't be as compatible with as many BIOS's as USB-ZIP.

mkdiskimage -4 /dev/sdz 0 255 63

This will allow you to partition a USB key up to 8GB in size. You probably could also use 256 heads instead of 255, for a little more space, but 255 heads is recommended, as it's slightly more compatible.

You may then run into the 2GB limit of the FAT16 partition. Beyond 2GB, it is probably best to use FAT32 instead. Supply the -F option to mkdiskimage.

mkdiskimage -F -4 /dev/sdz 0 255 63

At first, I did recommend FAT32 for all users: http://www.knoppix.net/forum/viewtopic.php?t=25402

However, it did not work on as many BIOS's, so I now recommend FAT16, unless you have a USB key that is larger than the 2GB limit of FAT16.

The other instructions on the page above should still be valid. Only the mkdiskimage command line needs to be changed, for use with large keys.

--Krellan 20:30, 19 Oct 2006 (GMT)

PortableApps

Here's also something great I found. If you have roughly 260MB or more of free space remaining on your USB key, you might want to drop PortableApps on there.

http://portableapps.com/suite

http://portableapps.com/

This is a suite of freeware Windows utilities, and will help you if you need to do work within Windows without being able to reboot into Knoppix. Much better and cleaner than U3!

When installing PortableApps to a USB key that already has Knoppix on it, PortableApps will complain about overwriting the autorun.inf file. This is fine. Say Yes to the overwrite. This will allow the USB key to bring up the PortableApps menu, instead of the Knoppix homepage, when clicking on it from within Windows.

With a 1GB key formatted as FAT16, as per the above instructions, there is, unfortunately, just barely not enough room for both Knoppix and the full PortableApps suite to coexist. The workaround is to "install" PortableApps to a spare empty folder on your hard drive first. Make edits to that folder as necessary, deleting some apps you can live without. Then, copy the entire folder manually to your USB key from there.

Formatting as FAT32 will work, as FAT32 is more efficient at cramming in the thousands of small files PortableApps creates. A larger USB key will provide much more room for both Knoppix and PortableApps, and enough extra left over for a decently-sized Knoppix home directory as well. See above for alternate mkdiskimage command lines you can try, when formatting larger USB keys or using FAT32.

--Krellan 19:23, 27 Nov 2006 (GMT)


There is an another project based on PortableApps, with the addition of TOR-networking enabled by default. This makes it possible to anonymize web surfing at the cost of speed.

More info at http://www.democrakey.com/

--Neva 18:54, 20 Jun 2007 (GMT)


mkdiskimage won't set up a 2 gb USB device with the USB-ZIP format using the example given. According to other FAQ's this is an issue with too many cylinders. A 1 gb drive is supposed to work fine. You may be able to manually create two drives on the device and manually format it in order to set it up as USB-ZIP.

Depending on the USB device, you can use fdisk to figure out the cylinders, heads and sectors and get mkdiskimage to work with these values. The one I used had 1015 cylinders, 64 heads and 62 sectors per track -- these values worked fine and I was able to create a bootable Knoppix USB key.


I was able to use this guide to setup and boot from Knoppix on a Creative Zen Vision:M MP3 player. Details here: http://www.francisshanahan.com/detail.aspx?cid=509


For the Cylinders/Head/Sectors issue, an easy way to find the information is simply type: hdparm /dev/sd? Where '?' is the drive letter. It will output the "Geometry: x/y/z" where X=Cylinders, Y=Heads, Z=Sectors. Enter those in for the last three variables on the mkdiskimage command and you are all set.

--Geekner 18:53, 7 Oct 2007 (UTC)

When running Knoppix 5.1.1, and booting from your USB key, sometimes udev hardware detection will fail, or lock up entirely. This is strange, and I don't know why this happens. This doesn't happen when booting from the original CD.

The workaround is to use the "noudev" cheatcode. However, when using this cheatcode, fewer of your peripherals will be detected correctly. It's better than having the system lock up, though. If this cheatcode is necessary for you, consider editing it into your syslinux.cfg file as described above, so that you won't need to enter it every time.

--Krellan 10:55, 18 Oct 2007 (UTC)