My Custom Computer Imaging guide

What is computer imaging? Well, when you install an operating system, you typically install from a cdrom. When you have to install a massive amount of machines, you look for an imaging solution. I've been working on a project for about 7 years now, and a large bulk of that is mass imaging of laptops, various makes and models. Over the course of the years we stopped using Norton Ghost as our imaging solution as they changed the licenses and made it far to expensive for our needs and budget.

So I found F.O.G.

Fog Project

FOG is a free, open source, replacement for ghost. It's quite powerful and with some customizations, it can be a pretty useful system to have in your network. I'm going to outline the steps I took to customize my sever for my own needs, from start to finish. i will have to edit some sensitive information out of course, but by following this guide, anyone should be able to set this system up.

This guide assumes you are using Ubuntu 12.04 as your OS for the server, but it could be one of many linux distros, with very slight differences. It can also be other versions of ubuntu, but this guide is 12.04 specific. This guide also assumes you are using fog.32 but most if not all of the steps still apply to fog.29

Lets begin!

A few things to remember. Linux is case sensitive for commands most of the time assume it is always. Second, you will use “sudo” before most commands to let your user run them as root, think run as administrator in windows. The first time you use it in a terminal session you have to provide the password, as long as you keep doing commands in that terminal, you won’t have to enter it every time. For commands that have you editing files, in most cases you have to edit it the way listed, a simple copy and paste will not always work. Linux names your wired lan Eth0 by default.

When logged in as root graphically, you do not need to use “sudo” before commands, but you shouldn’t be doing much else other than copying files as root.

Linux has an administrator account called "root". Much like windows 7, you aren't supposed to login and use that account. Because of some of the steps that require a graphical root login, I have enabled the root account with a hack. When logged in as a normal user in linux, you will be using command lines. Most of them will require the "sudo" command. You can think of that as "run as administrator" in windows.

Getting Ubuntu installer on a Flashkey

Not all machines will have an optical drive, so here’s the steps for making a bootable flashkey with the Ubuntu installer. Download and install unetbootin-windows-583.exe. Run it. For distribution select Ubuntu 12.04. Check ‘diskimage’ and select ‘iso’ then point to your downloaded ubuntu-12.04.1-desktop-i386.iso. Once it’s finished you should have a bootable LiveUSB ubuntu installer.

Installing Ubuntu


Use ‘unetbootin-windows-583.exe’ to create a bootable flash key using ‘ubuntu-12.04.1-desktop-i386.iso’. Boot the flashkey and select "Install Ubuntu to Hard Disk". Enable 'Download updates while installing' and 'Install this third-party software'. Replace the OS, if present, with Ubuntu, do not do a dualboot.

Select login automatically and do not encrypt home folder for ease of use, don’t encrypt the home folder either way.

Anytime you need to replace a system file, you will have to switch to the root user to do it.
To enable you to login as root you have to manually enable the manual login screen.

Press Control+ALT+T to bring up a terminal. Enter the following commands:

sudo passwd root

This will ask you for your “sudo” password, which you can think of as run as administrator from windows. It will be whatever password you set for your user account. Once you enter your sudo password, it will ask you for a new unix password, this is the root account password. It will make you confirm it.

sudo sh -c 'echo "greeter-show-manual-login=true" >> /etc/lightdm/lightdm.conf'

This actually sets the manual login screen to appear. Make sure to get it exact, including the ‘ ‘. Then shutdown the computer. From this point on, you will have to select ‘shut down’ from the power menu as a logged in user, any other option will take you to the manual login screen. If you have more than 1 user logged in, you will need to log out from that user then shutdown from the other user. Yeah, it's weird.

Once you reboot, you will be able to select “switch user account” from the login menu and when the classic login screen appears, you can select ‘Login’ then for the user enter ‘root’ and the password you set in the steps above. Anytime you need to replace a system file, you will have to switch to the root user to do it.

Installing FOG

Download fog_0.32.tar.gz or copy it from the flash key to ‘Downloads’ and open a terminal window, navigate to it’s download location and run this command(you could also right click and select "extract here"):

sudo tar -xvzf fog_0.32.tar.gz

Enter your password if it asks. This extracts the FOG installer with the appropriate switches. Once done run this command:

cd fog_0.32/bin/

This will move you into the folder that FOG extracted to. Run this command next:

sudo ./installfog.sh

This will begin the FOG installation wizard. When prompted for which linux to create it for select “2” as this setup is using Ubuntu. When prompted for what type of installation, select “N” for normal. “S” is for storage nodes. You must create the master server before any storage nodes can be setup. Leave the IP at it’s defaults. Do not specify a router, do not specify DNS. Do not enable FOG DHCP. Do Not enable international support.
Do NOT specify user names or passwords for MYsql, it might prompt you a few times, leave them blank.

Once the installer is complete, carefully copy down the username and password it provides on screen, then click the link it provides to update the schema. You can now open a webbrowser and load the FOG management page. If you receive any errors, choose work in offline mode.

hxxp://localhost/fog/management/index.php.

(localhost would be the server ip, but if for some reason it's not correct, localhost will always work.)

Overwrite Fog Settings

In the event you made a mistake setting up the fog server or node, use the following command to replace the settings with the firstinstall switch, to let you run through the installer again. This will NOT recover a lost fog service password. Open a terminal and use the following command:

sudo mv /opt/fog/.fogsettings /opt/fog/fogsettings-firstInstall

In the event that your server or nodes changed IPs, open a terminal and

Cd /opt/fog_0.32/bin/
sudo ./installfog.sh --no-upgrade

This will run the installer again, without the need to have internet access, and will show you the IP again, it will NOT show you the fog username and password.

>>>>>>>>>continued

Find or Replace lost FOG Storage Password

If you lost your fog storage password, log onto that machine, open a terminal and use the following command:

Vi /var/www/fog/commons/config.php

You are looking for the following lines:

Define( “MYSQL_DATABASE”, “fog” );
Define( “MYSQL_USERNAME”, “fogstorage” );
Define( “MYSQL_PASSWORD”, “b83cd3” );

You can change the password but keep it at the same 6 character length and make sure you change it on the fog storage node setup screen on the master server as well.

Manual Fog uninstall
In the event you really borked up your fog service, use the following steps and commands to remove enough to allow you to re-run the installer fully. Open a terminal and use the following commands:

Sudo rm /etc/init.d/FOGImageReplicator
Sudo rm /etc/init.d/FOGMulticastmanager
Sudo rm /etc/init.d/FOGScheduler
sudo rm -rf /var/www/fog
sudo rm -rf /opt/fog
sudo rm -rf /tftpboot
sudo rm -rf /images
sudo userdel fog

This is manually removing the files, including the /images/ folder, so if you don't want to lose any images, don't delete that folder!

Storage Nodes

To install a storage node, follow all of the same steps as a server install but pick “S” for storage when prompted. Once the installer finishes copy down the username and password it provides on the last screen. Once the node itself is complete, move to your server management page and do the following:

Click “Storage” to bring up the storage section, then click “Add Storage Node”. Name the node, provide the IP that it was set to (verify it hasn’t changed if you didn’t set a static IP). Make sure “node is enabled” is checked. Enter /images/ as your image directory if it wasn’t selected by default. Set a queue length, this sets the number of clients it can handle at a time before it starts to make them wait in line.

Provide the username and password that the storage node provided, this allows the fog master server to contact the node, use it, and mirror any new images to it. Once all of that information is filled out click “ADD”.

Go back to the main info page (home button) and at the far right in the section called “disk information” select the node from the dropdown. Give it a second to refresh, if it’s setup properly it’s used disk information should show up, if it remains at N/A then either an ip issue or a password issue should be considered.

FOG replicator service automatically mirrors the entire /images/ folder from the master to any enabled storage node. It runs a check every 10 minutes or so to verify the folders are the same, if there is a difference, it begins mirroring the images to the node, this is a slow process and you should not try to image a session while this is taking place as some nodes won’t have the image and will give errors, and those that do will be running slower than normal. It is possible to manually create the images by copying them from a usb disk, but the permissions will be incorrect and FOG will not be able to delete the images, this will have to be done manually.

In the event you need to copy or backup an image to a usb drive use this command:

sudo cp -r -v nameofimagefolder /media/nameofusbdrive/

Once you are at the node you are trying to copy the image over to use this command:

sudo cp -r -v nameofimagefolder /images/

Creating an Image
FOG is a little weird when it comes to creating an image. Before you actually make an image, you have to create an image definition. From the FOG management page select “Images” the button that looks like a painting.

Click ‘New Image’ to bring up the new image options. Name it and select a storage group, we use ‘default’. When selecting an image type, be sure to pick the best one to suit your needs. Our server only allows for 1 type of image that is resizable. If you don’t pick it, the image you make will only push to a client with a harddrive the same size or bigger than the drive you imaged from, regardless of how much space is actually used. Once you have selected the type, click ‘ADD’.

Now move to your client with the image you wish to create and PXE boot it into FOG. Select quick host registration and let it register the client. Once it’s done, the client will reboot, catch it before it tries to boot into the os and either turn it off, or leave it on the boot selection screen of it’s BIOS. The FOG PXE menu will time out eventually and pick the default option selected in the pxe configuration file.

Now go back to the server and click “Hosts” the icon that looks like an LCD monitor. Click ‘List all Hosts’ from it’s menu. You should see an entry for the machine you just registered, the entry will be the client machines MAC address.Select the entry and click Edit.

>>>>>>>>>>>>>>>>>>>>>>continued

Make sure you select the image definition you created as the ‘Host Image’. Ensure you select the correct ‘Host OS’ for the image you are making or it will not boot correctly. Click ‘Add’ when you have it all filled out.

Click ‘Tasks’ from the main menu, it’s a star icon. Click ‘List All Hosts’ and find the client host you registered. Click ‘Upload’. It will ask you what to do when it’s complete, select shutdown. It should say all hosts queued without errors.

Boot your client machine to PXE (F12) and it should go right into the image session. Once it’s finished and shut down, your image will be created. If you have added storage nodes it will, over the next few minutes, start mirroring the new image to the storage nodes.

You should go back to your ‘Hosts’ and ‘List all Hosts’ and find your client machine entry, select it and delete it.

Imaging a Client
Boot your client to PXE and select quick host registration. Once it reboots, turn it off or keep it on the BIOS boot selection menu.

On the FOG management page select ‘Hosts’ and ‘List All Hosts’. Find the host you registered, it will be entered under it’s MAC address. Select it and edit it. Select the desired ‘Host Image’ and set the proper ‘Host OS’ for your image, then click ‘Add’.

Select Tasks (star) from the FOG Management page and click ‘List All Hosts’. Find your client by it’s MAC address and select ‘Deploy’. Set your preferred action for the client machine when the image session is complete and it should queue with no errors.

PXE Boot the client machine and it will start the image session.

Imaging a Group
If you wish to image more than 1 client, follow the host registration process for all clients, then on the FOG management screen, select all of the hosts, and at the bottom of the screen, enter a group name and select the storage group (default).

Once the group is created, click the ‘Groups’ icon (looks like 2 LCD monitors) and then ‘List Groups’. Find the group you created and edit it. Just like a single client set the correct image and os type.

Select Tasks (star) from the FOG management page and ‘List Groups’, select your group and click Deploy. Set the desired post session action and once it says all machines are queued, boot the clients to PXE and they will image.

After the session is complete, go back to the ‘Groups’ page, ‘List Groups’ and select your group and delete it. When it asks, select delete group and all host objects.

Failed Session
If you have a session fail, there are a few steps to take. On the FOG Management page, select ‘Tasks’ (star) and ‘list tasks’ to find your failed session. Click ‘Kill’ and it will terminate the session.

If your clients are trying to boot into a session that doesn’t exist, and they have no host registration entry, you will have to get the MAC address of the client, and manually ‘Add Host’ on the Fog Management server ‘Hosts’ page. You can then either assign it an image, or use one of the ‘Advanced’ options on the ‘Tasks’ page like a disk wipe.

You should also look in /tftpboot/pxelinux.cfg. You should see a file called "default". That's your default PXE menu, what you'll see when you press F12 to network boot, it will be the list where you picked "quick host registration". If you see another file in there, that appears to be a random hash, that is actually the pxe option that will be handed out to a client, that hash is actually the clients MAC address. Issues with clients stuck in a boot loop can usually be resolved be ensuring any files linked to it's MAC address are deleted. This is done automatically after a job completes, but might not if it fails out.

Input Signal is Out of Range
You might get this error, usually on a desktop. The graphical interface has crapped out. Press control+alt+F1 to open a terminal prompt. Type “login” and hit enter. Enter your username and password. If you are on our fog master server, enter Root and the password. Type “startx -- :1” and hit enter. If you get session already open errors, change the 1 to another number. If you logged in as root, when the desktop first appears you will get a warning that you shouldn’t be logged in as a privileged user, do it anyway. Caution is for the weak.

TFTP Timeout

If you’ve rebooted the server you’ll probably get this error when a client tries to connect to the FOG server. Ubuntu 12 seems to have issues starting the TFTP service at boot. You’ll need to enter the following command in a terminal to restart the service:

Sudo service tftpd-hpa restart

The service will go down then back up, and you should be able to boot back into the fog server from clients.

>>>>>>continued

Unable to Host Register or Image
If your client is hanging while attempting to register a host, or you attempt to image a client and you get a warning that the image file cannot be located, you might have an IP issue. The first step is the fog management home screen, to the right side you will see a piechart showing the space in use of your storage node, if you click the drop down you can pick each node to see the info for it. If you see “unable to connect” instead of a pie chart your storage node IP address is incorrect.

Click on the storage node menu option and select ‘view all storage nodes’. Find the one, or ones, that you can’t connect to in the list and select edit. Verify the IP address listed there is the correct IP for that node.

If this doesn’t resolve the issue, go back to the home screen and look to the left under “System Overview”. You will see the IPs for the webserver and tftp server. Those should be the same, and they should be the current IP of the fog server. If they do not match each other, or the fog server, select “other information” the blue circle with a question mark, and select “fog settings”. There are a bunch of places where IPs for services are set in here, the fastest way is to control+f to search and enter the digits to replace. If the server is 204.82.48.65 but the webserver IP is showing .56 search for .56 and replace it. Keep in mind that in this section, each one has it’s own “save changes” button, so you’ll have to change the IP, click save, then continue down the list.

Customizing PXE menu
We have a customized PXE menu for our fog server. As such, we are able to add a few testing tools to the pxe menu removing the need to actually create FOG images of the tools, we can simply use the .iso files. As well ,because we are using fog .32 we are able to replace a fog file, and create a quick image option from the PXE menu allowing us to image without the need to register the hosts.

The first step is copying the noreg.gz for fog .32 file in:

/tftpboot/fog/images

To examine or modify the PXE menu, edit the following file with a text editor (wordpad on windows). I use gedit, so the command to open this would be:

sudo gedit /tftpboot/pxelinux.cfg/default

The first example will be the Dell Diagnostics tool. First go to :

/tftpboot/fog/

This is where you need to create a folder for your iso. Call it dell. Place the diags.iso file in that folder, and from one of the other premade fog folders, copy memdisk over to your dell folder. For any iso you will need to copy memdisk over to it’s folder. Once the files are in place, edit :

/tftpboot/pxelinux.cfg/default

Adding the following entry in the order you want it to appear on the menu:

LABEL Dell Diagnostics
kernel fog/dell/memdisk
append iso initrd=fog/dell/diags.iso raw
MENU LABEL Dell Diagnostics
TEXT HELP DELL(TM) DIAGNOSTIC
ENDTEXT

This defines the menu options, first with a label, then it tells it which kernel to send to the client, in this case memdisk. Then it points to the iso. in this case we only need the "raw" option but some isos require different options. Once you save it, the menu is active.

PXE Cascading Menus

The menu structure works like this:

MENU BEGIN name
MENU LABEL name
MENU TITLE name
MENU END
PROMPT 0
TIMEOUT 300

You can also create menus within menus, as we have done with our setup, it will look like this:

MENU BEGIN name
MENU LABEL name
MENU TITLE name
MENU BEGIN submenu
MENU LABEL submenu
MENU TITLE submenu
MENU END
MENU END
PROMPT 0
TIMEOUT 300

To add a newly created image to the menu, simply copy and paste one of the ones already in place, or the example above. Then host register a client and assign it that image as if you were going to image normally. Start the session, but don’t boot the client into it. Instead, go into your /tftpboot/pxelinux.cfg/ folder and you should see ‘default’ as well as a random hash file. Open that hash file (it’s actually the clients MAC address) and it will have almost the exact command used in my example. Note that any information present in that file, that is not present in my example file, should be removed from the entry before you create the menu option.

It is possible to put a link right to an image here, using the steps in the above paragraph. Some things of note in the menu option that will be created:

osid= (specifies the type of OS, windows xp, windows 7, linux, etc etc)
imgType= (specifies the type of image, multiple partition, resizable, etc)
append initrd=fog/images/init.gz. You need to replace init.qz with noreg.gz for the server to allow you to image without registering, it disables the check.

>>>>>>>>>>> continued

Adding UbuntuLive to Fog

Download ubuntu.12.04.desktop.iso

Create a folder for the iso ‘/tftpboot/fog/ubuntu/’ Then copy the iso into the ubuntu folder. Create a mount point for the iso with the following command, 12 would be the version number or anything you wanted to call it really. Our iso is named ubuntu12.iso. We have to mount the iso like you would in windows with D-tools.

sudo mkdir -p /tftpboot/fog/ubuntu/12

Then open /etc/fstab with this command:

sudo gedit /etc/fstab

Add the following line to the end of /etc/fstab (it’s all a single line):

/tftpboot/fog/ubuntu12.iso /tftpboot/fog/ubuntu/12 udf,iso9660 user,loop 0 0

Now you've told the system to mount that iso to that folder each time the system boots. Test the mount point with the next command, it might warn you it’s mounted as read only, that’s fine.

sudo mount -a

Then list the contents of the iso to be sure with this command:

ls -lash /tftpboot/fog/ubuntu/12/

Edit /etc/exports with this command:

sudo gedit /etc/exports

Add this line to the end of exports:

/tftpboot/fog/ubuntu/12/ *(ro,async,no_wdelay,insecure_locks,no_root_squash,insecure)

The restart the NFS server:

sudo /etc/init.d/nfs-kernel-server restart

Edit the default pxe menu to add an entry for ubuntu:

sudo gedit /tftpboot/pxelinux.cfg/default

Add the menu option for Ubuntu:

LABEL Ubuntu Livecd 12.04
KERNEL fog/ubuntu/12/casper/vmlinuz
APPEND root=/dev/nfs boot=casper netboot=nfs nfsroot=:/tftpboot/fog/ubuntu/12 initrd=fog/ubuntu/12/casper/initrd.lz quiet splash –

You should now have an Ubuntu menu option on the PXE menu that will load ubuntu live, without installing.

Adding Ubuntu Netboot

Download ubuntu-12.04.2-alternate-i386.iso and mount it with the following command. This assumes you are in the folder where you are going to store the iso.

Sudo Mkdir /var/www/ubuntu
Sudo Mkdir /var/www/ubuntu/alt

This creates a folder accessible via http for ubuntu, then a folder inside that for the alternate installer.

Sudo Mount –o loop ubuntu-12.04.2-alternate-i386.iso /var/www/ubuntu/alt

It will probably warn you that it’s mounted as read only, that’s fine. Now download the correct netboot.tar.gz for your version of ubuntu and extract it. Copy the ‘netboot’ folder into your ‘/tftpboot/fog/’ folder. Add the following entry to your '/tftpboot/pxelinux.cfg/default' menu.

LABEL Ubuntu Netboot
MENU LABEL Ubuntu Netboot
KERNEL fog/netboot/ubuntu-installer/i386/linux
APPEND vga=normal initrd=fog/netboot/ubuntu-installer/i386/initrd.gz en_US country=Canada intereth0 hostname=ubuntutester --
TEXT HELP
Ubuntu Netboot
ENDTEXT

This preseeds some of the installer questions, as I aimed for a completely unattended install.

Adding Preseed and firmware
When you are netbooting a client your server hands it 'initrd.gz' which is located at '/fog/netboot/ubuntu-installer/i386/initrd.gz. You can modify this ramdrive to include a preseed answerfile for automated installs as well as adding firmware you might be missing. Copy initrd.gz to your /home/ folder. Download the firmware files or preseed file to your /home/ folder. The preseed file is named 'preseed.cfg' and the firmware for our netbooks is two files: bcm43xx-0.fw & bcm43xx_hdr-0.fw.

Sudo mkdir /tmp/initrd
sudo cp initrd.gz /tmp/initrd
cd /tmp/initrd
gunzip

I'm willing to post up my example files as well as some of the tweaks I did like removing most of the garbage in the registration process, etc etc. The preseed file for ubuntu unattended actually took longer than the entire server setup.

Why don't you just link their wiki instead of copy pasting it here ..

reply to post by H1ght3chHippie


Because most of the steps on here aren't listed on that guide, that would be a good reason I'd think. Also I did link the wiki right at the start, feel free to point out where they explain how to do half of the stuff I've started listing here.