Ephemerboot/Read Me Second.txt

Ephemerboot 1.0

David "Potatoswatter" Krauss, 5/02
See bottom for credits.

What is Ephemerboot?

Ephemerboot is a RAM disk driver which will boot Classic on NewWorld machines.

Why is it cool?

Ephemerboot solves two tough problems, making it Weird.

1. Bit rot

Many Macs in the past have been able to boot from RAM disks, allowing very fast startup.
However, NewWorld machines (ones since the original iMac, without ROMs) can't. They power down (more accurately, cease to refresh) RAM momentarily during reboot, causing its data to become suddenly ephemeral. It simply fades away... but actually most remains intact.

To allow bootable RAM disks despite this, Ephemerboot implements data checksumming & correction to retrieve data lost from RAM transistors to "bit rot".

2. No access

The bootability feature of Apple's own RAM disk driver (".EDisk") is hard-coded into the Mac boot sequence. Older machines store it in ROM; NewWorld machines have it in the "Mac OS ROM" file. Before the Mac OS reads anything from disk, ROM code reserves memory for the RAM disk, loads the driver, and takes a special look at it to give it special booting privileges.

Only problem is, I don't work at Apple. I was a toddler when that code was written.

To accomplish what that little bit of ROM code does, Ephemerboot uses two off-beat components: a chaining bootloader and a patch partition module. A driver in the patch partition module does the real work.

2a. The bootloader

The bootloader phase ("ramboot") reserves RAM for the disk.

As Open Firmware is running, the Mac represents the arrangement of DIMMs on the motherboard as a set of numbers (starting address/# bytes) in the /memory device-tree node. Ramboot alters these in a way similar to that described in QA1099 [1], with a few differences.

Ramboot supports a variable amount of RAM shrinkage.

Ramboot supports discontiguous physical memory. This is a bit technical to explain here, but it makes things a hassle to do properly.

Unlike the QA1099 method, RAMBoot isn't designed simply to waste RAM. It reclaims 2 MB of memory that would otherwise be wasted because it's used by Open Firmware. To see your reclaimed memory, go to Apple System Profiler and look at the "Memory" section.

Ramboot allows the user to choose where memory gets allocated from. This feature was hacked onto the preceding one. Note that physical addresses don't actually correspond to DIMMs when interleaving is used. DRAM interleaving allows RAM chips to act in parallel, which is normally good. It happens when you have two DIMMs of identical size in adjacent slots.

2b. The patch partition module

The patch partition module contains the actual Classic Mac OS driver.

(Nearly) every partitionable disk has a patch partition, an obscure and little-used invisible 256 kB partition. Code from this partition is loaded and run at boot. Normally this does boring things like patch ROM disk drivers from the pre-4 GB era and allow machines predating popular CD-ROM drives to boot from them with the "c" key. Actually, that's all that patch partitions have ever done, far as I know. You might have noticed, however, that these functions both deal with letting the machine boot from things it normally wouldn't. That's because the patch partition is run before the boot volume is determined.

The patch partition consists of a series of modules in a grow-down stack and a series of module descriptions in a grow-up stack. The Ephemerboot Installer adds its code to both stacks.

----------- patch partition
description 1
description 2
Ephemerboot description

- empty space -

Ephemerboot code
patch 2
patch 1
------------ / 256k

The patch code is generated from a single source file, Ephemerboot.c. It first loads from the Name Registry information from the bootloader stage about where the disk is located. If that succeeds, it loads the master checksum from NVRAM and performs the data correction. Then, it loads the driver and the disk.

2c. The driver

The driver is as simple as Mac disk drivers get. It's synchronous and it does I/O with BlockMoveData(). It's not that simple, tho.

Once some physical memory has been cordoned off in Open Firmware, it's tough to access. It's so well-hidden that not even the Nanokernel knows about it <20> just trying to read those addresses causes a crash (an unmapped memory exception). No public Mac API will bring that memory back.

However, there's a cute little app called PowerMacInfo, written as a kernel monitoring tool by some really cool Apple engineers. It can show you physical memory at any address. (By default, it only shows you memory the kernel knows about, but it can be fooled with MacsBug.)
So I reverse-engineered it. It's the "old-style" 68k VM interface. It centers around opword 0xfe0a (that's an f-trap!). The definitions at the top of EphermerBoot.c should be a good enough explanation. The second parameter to VMUnmap controls whether the page should be remapped upon an unmapped access (true) or if unmapped accesses should cause a propagated fault (false). The parameters are UInt32's because they're page numbers, not addresses.

VMMap/VMUnmap are used to create windows in memory through which the disk and checksums are accessed. Three windows exist for two checksums and the disk.
While the disk is not being used, the window pages are set to propagate exceptions. Since they're low in system memory, this can cause MacsBug to complain (especially on "hc all"). If you don't like it, you have the source.


Checksums are created with every write operation. It's set up so that 32 bytes of checksum check every page (4096 bytes) of memory. Even at this ratio (1:128), the checksums are pretty big. To store a top-level checksum in nonvolatile RAM, which is preserved across restarts but is very small, I had to use 3 levels. The first two can be large and are accessed by floating VM windows; the third is a regular block in the system heap which gets stored by Name Registry calls at every write op.

The end effect of calculating 3 large checksums in 68k code, and then saving to slow NVRAM, is that the RAM disk is slowed to the speed of a regular hard drive (on my 333MHz Lombard anyway). I'm sorry about this. Not like anyone can actually install it ;v) .

How do I install it?

It's unlikely that you can. However, if you'd like to try, follow these instructions.
- Update your firmware from apple.com.
- Copy "ramboot" to the root directory of your internal ATA drive.
- Copy "install.f" to the root directory of same drive.
- Double-click the installer & drag a hard-drive icon onto it.
- Turn startup memory tests off with the Memory control panel (cmd-opt to see the option).
- Reboot and immediately hold down cmd-opt-O-F to get into Open Firmware.
- Type
	boot hd:,\install.f
to automatically set your boot environment variables. (That's colon-comma, not a semicolon.) If you want a disk size other than 64MB, do
	setenv ramboot-disk-size xxx00000
where xxx is the size in MB of the disk.
- Hit the power button (or eject key) to shut down, then again to start up again.
The machine should boot into Mac OS. You should get some status messages and get a disk init box.
- Name & format the disk.
- Install system software to it. If you use the actual Apple installer application, be sure to disable updating disk drivers.
- Copy the "Mac OS ROM" file from the RAM disk system folder to the same root dir as "ramboot".
- Set the type of that ROM file from 'tbxi' to 'tbxj'. It'll lose it's icon. If having it there is too freaky 4 U, rename it too.
- Unbless your selected system folder by removing the System or Finder, if it's on the same volume as the tbxj.
- Reboot. The machine should now boot from the RAM disk!

Note that this has only run on the single test machine it was written for. Although no single reason it wouldn't work has been isolated, there appear to be many reasons it doesn't. The test machine is a Lombard PowerBook. So far (at MacHack) it hasn't worked on 2 Pismos (with mutually exclusive sets of problems), a slot-loading iMac, a Sunflower iMac, and a QuickSilver G4.

How do I uninstall it?

Very luckily, this is easier.
- Use Drive Setup to update the driver on the disk you installed it onto.
- In Open Firmware, do
	setenv boot-command mac-boot
	bye
- If you're really hopping mad at me and you want to wipe all traces of this consarned program from your machine, reset NVRAM by restarting and holding down cmd-opt-P-R through a couple chimes.

How do I recompile it?

The Ephemerboot.mcp project only builds the executable code <20> you still need to get that & a patch description into the installer. Open BuildpDES.mcp and hit "Run". It'll rebuild & run the installer builder. You can then use the installer to write over old versions.

Nota bene disclaimer cuidado: Patch partitions can be extremely dangerous. If you call Debugger(), you won't be able to boot from that disk. If you do something that crashes no matter what, you'll probably have to use OS X's disk utility to update the drivers. Let's just be thankful OS X is there.

The installer app is based on an internal app from my company. You can't have the source to the disk stuff. If you like the shell tho (it actually supports things like actual strings and icons and dropping stuff on the Finder icon), you can get it at http://homepage.mac.com/potswa/source/StandardDropin.sit.

How'd ya do it? Who helped?

First off, I couldn't have done this without the help of these helpful helpers:

Miro (meeroh) Jurisic went out & spent a MacHack on this. Imagine going through that install sequence over and over and over and over and over (except without the install.f file), with great programmers having great fun all around you. Thank you meeroh!

Ren<EFBFBD> Vega helped a bit with VM and suggested bit rot would be a problem before anything ran. Prolly woulda given up otherwise. Also he wrote much of PowerMacInfo in the first place.

Quinn "The Eskimo!" for his wonderful InterruptSafeDebug and TradDriverLoaderLib. MoreDisks is also great, tho I don't use it much in here.
Quinn also introduced me to discontiguous physical memory. Seeing as my machines all have contiguous memory, I don't think I'd have caught that.


I wrote this because I wanted to use my patch partition, driver, and bootloader l337 sk1llz on something fun. After reading [1] and seeing PowerMacInfo in action, I knew it was possible. The toughest part was reverse-engineering PowerMacInfo.

Early on, I tried to make the project more ambitious by doing it without the patch partition. I tried to do a hack System file that would load the driver, then unload itself and change the boot drive. I got pretty far along <20> the ROM would reload the real System file, but the second call to InitResources() would always fail. I gave up on that, but the source is included here as MegaBoot.c. That file also includes a really neat a-trap tracer based on ISDebug.

After that, the checksumming took a couple tries to get right. I hadn't expected to do it at the outset, but in the end I think it's cooler than just getting through the Mac OS tangle.

After I decided to use the patch partition, Ephemerboot took about 10 days to implement.


Reference

[1] Apple Technical Q&A QA1099: Reducing the size of Physical Memory in Open Firmware. http://developer.apple.com/qa/qa2001/qa1099.html

Revisions

6/20/02 - 1.0 - Doc written for MacHack.