| CPSC 331 | Operating Systems | Fall 2005 |
The Nachos Road Map document linked above is a useful guide to the Nachos system. However, it describes an older version of the system than the one we are using and so isn't 100% accurate. The discussion below is intended to be read in conjuction with the Road Map, and expands on and updates what is found in the Road Map. The sections below correspond to those in the Road Map to help you stay organized. It is probably best to read the Road Map section first, then read the corresponding Companion section.
Note: You have been provided with a printed copy of the Nachos Road Map. To save paper, please avoid printing it again. Ask me for another copy if you absolutely must have one.
|
Additional material will be added as it becomes relevant. |
The main project for this course is to build an operating system (or at least large parts of one). To make your life easier, you won't be building your OS to run on bare hardware - instead you'll be working in a simulated environment. There are a number of advantages to this approach:
The software we'll be using this semester is known as Nachos, and was developed at the University of California at Berkeley. It combines a barebones OS (which you will be enhancing) with a simulated MIPS workstation. "MIPS" refers to a particular processor architecture (read more here). The details of the MIPS architecture aren't important for our purposes, but the fact that Nachos simulates a MIPS machine rather than the Intel-based computers we'll be using means that a few hoops will have to be jumped through in order to write user programs for Nachos. (But more about that later.)
Lest you worry that you aren't getting "real" OS-writing experience because you are working in a simulated environment, Nachos is designed so that, while the entire Nachos OS+simulator runs as a single user program on a UNIX machine, the code that you write (and most of the provided Nachos OS code) is the same as if the OS was being run on bare hardware. Porting Nachos to run on actual hardware would require replacing the simulated machine code with some machine-dependent assembly code (and some actual computers). For example, instead of calling C++ methods (which manipulate a C++ object) to enable and disable interrupts, running on real hardware would involve assembly-language instructions. However, the steps the code goes through - e.g. the fact that interrupts must be enabled or disabled - remains the same for real hardware as it does in the simulated environment.
The code described is found in the code directory of the Nachos package.
Interestingly, the MIPS machine that Nachos simulates can execute arbitrary programs - in fact, what you can run as a user program under Nachos is limited only by the system calls implemented.
When running, the Nachos simulated environment is a little different than it would be if running on actual hardware. When running on hardware, all code - whether it be kernel code or user code - is executed by the fetch-execute cycle of the CPU. The switch from kernel code to user code and vice versa is achieved by adjusting the contents of the PC register as needed. In Nachos, however, only user programs run on the simulated MIPS machine. Nachos kernel code, on the other hand, runs like a regular UNIX process - the Nachos source code statements are executed by the host machine, not the simulated MIPS machine.
There are several implications of this. One is that writing Nachos user programs is not as simple as just writing and compiling a program on the host machine, because the program won't run on that machine. Instead, it is necessary to use a cross-compiler to compile the C program written on the host machine into MIPS-compatible code for the simulated machine. (An additional step is needed to then convert the result into Nachos-compatible code, since Nachos doesn't exactly simulate a MIPS machine.) Another implication is that when performing a context switch, it is necessary to save both the simulated MIPS registers (to capture the state of the user program) and the real hardware registers of the host machine (to capture the state of kernel code). Normally there is only one set of registers (belonging to the physical machine the OS is running on), and the important user-process state was saved by the interrupt handler before the kernel got control.
The definition for the Machine class is found in machine/machine.h.
Registers cannot be accessed directly; the registers instance variable is private. The ReadRegister() and WriteRegister() routines must be used insted. The symbolic names should be used instead of numbers whenever special-purpose registers are used.
The Nachos MIPS machine is configured with 128 pages of physical memory, not 31. "Byte-addressable" means that each slot of memory holds a single byte.
machine->pageTable points to the location of the current page table in memory. (There may be multiple page tables, one per address space; this pointer may need to be updated when a context switch occurs.) machine->tlb is the "hardware" TLB; the kernel is responsible for maintaining its contents and handling cache misses. Note: while the contents of the TLB can be changed as needed, the value of the machine->tlb must not be changed (conceptually, it points to the single hardware TLB).
Of the operations described, only ReadRegister(), WriteRegister(), and Run() are public.
The steps carried out Run() are performed in a different order than described:
In a real environment, interrupts are generated by the hardware when something occurs (e.g. when a timer goes off or a disk I/O request completes) and the timing of when one will occur is not known in advance. (Interrupts may also be triggered by software e.g. when a system call is made.) When an interrupt occurs, the hardware stores the contents of the program counter, program status, and any other necessary registers on the current thread's stack and transfers control to the appropriate interrupt service routine. The interrupt vector, stored in a fixed location at the bottom of memory, contains an address for the interrupt service routine for each device.
In the simulated environment of Nachos, however, things are a little different. The definition for the interrupt simulator is in machine/interrupt.h. Future interrupts are scheduled ahead of time (e.g. when a disk I/O request is made, the interrupt which will mark its completion is scheduled) and stored in an event queue. The PendingInterrupt class defines an interrupt which will occur in the future. The Interrupt class maintains the queue of pending interrupts and implements the hardware mode bit which tracks whether code is executing in SystemMode (kernel mode), UserMode, or IdleMode (used when the ready list is empty, and needed because Nachos uses a simulated clock instead of a real clock).
To actually handle interrupts, Nachos examines the pending-interrupt queue at each clock tick and carries out any interrupts that are scheduled for the current time by invoking the appropriate interrupt service routines. (Interrupt::OneTick() advances the simulated time and calls Interrupt::CheckIfDue() to process any necessary interrupts.) Nachos avoids having interrupts interrupt the interrupt handler by disabling interrupts in Interrupt::OneTick() before Interrupt::CheckIfDue() is called. If you look carefully at Interrupt::OneTick() and Interrupt::CheckIfDue(), no registers or other execution state is saved - since the Nachos kernel code is not running on the simulated MIPS machine, there's no need to save the (simulated) registers associated with the running user process. Instead, registers are only saved if a context switch occurs.
Since interrupts are only handled when the clock ticks, and the clock only ticks when interrupts are re-enabled or when a user instruction is executed, a long-running kernel thread should call Thread::Yield() periodically to allow other threads a chance to run - otherwise nothing will preempt the thread.
Another difference between Nachos and a real system: no interrupt vector. The interrupt service routines are defined as methods in the classes defining the various devices (i.e. console and disk) - look for the Callback() routine in those classes. The Nachos interrupt "hardware" (in the Interrupt class) knows what service routine to call because it was specified as a parameter to the PendingInterrupt constructor - i.e. when a device indicates that an interrupt will occur in the future, it also specifies the handler to use when the interrupt occurs.
The clock is advanced to the time of the next scheduled event when the ready list is empty because, with the ready list empty, no process is currently able to run. With nothing to run, it doesn't hurt to skip ahead in time. Furthermore, the only thing that could make a process become ready to run is an interrupt signalling the completion of a task that some process is waiting on.
Nachos interrupt handlers are not allowed to invoke a context switch because it could result in interrupts being handled late or worse problems. (If it is necessary for a context switch to occur, use the Interrupt::YieldOnReturn() method to cause a context switch to occur when the handler completes.) The reason is because the check-for-interrupts code is run by the currently-running thread. If there are several interrupts scheduled simultaneously and the first one processed invokes Thread::Sleep(), the current thread will be put to sleep without handling the remaining interrupts - and won't be re-added to the ready queue. That thread may be never run again, and deadlock can result if this happens to enough threads.
Entries in the page table and the TLB are defined by the TranslationEntry class in machine/translate.h.
The terminal console device is defined in the ConsoleInput and ConsoleOutput classes in machine/console.h. The Console classes are an abstraction of a real console device. This simplifies working with the console device since all of the device-dependent details are hidden in the class implementations. (Actually, since the physical device is only simulated in Nachos, there aren't any device-dependent details in the class implementation. However, if you were writing an OS to run on real hardware, you'd likely still want similar classes so that you could hide the device-dependent details in the implementation and thus simplify the kernel's interaction with the actual device.)
Note that the console devices are not synchronized.
The disk device is defined in the Disk class in machine/disk.h. As with the Console classes, the Disk class is an abstraction of a real disk device; if the OS were running on real hardware, the implementation of the class would contain device-dependent code for interacting with an actual physical disk drive.
NumTracks is 64 (not 32). The total number of sectors (SectorsPerTrack x NumTracks = NumSectors.
This section is discussing kernel threads. Nachos doesn't know or care if user threads are supported - all that would be handled in a user thread library and would be part of the executable being run as a user program.
The scheduler is defined in the class Scheduler in threads/scheduler.h. The Thread class, representing a thread control block, is defined in threads/thread.h.
The Kernel class, defined in threads/kernel.h, is primarily for collecting the various parts of the system in a single, globally-accessible place. It contains public instance variables currentThread (a pointer to the currently-executing thread) and scheduler (a pointer to the CPU scheduler with the ready queue), as well as pointers to the simulated machine and other devices. Since these instance variables are public and a global Kernel * variable kernel is declared and initialized in threads/main.cc, they are essentially global variables accessible from anywhere e.g. kernel->currentThread is a global variable for accessing the currently-running thread.
Creating a new thread simply involves creating a new Thread object:
Thread * t = new Thread("demo thread");
Note the dynamic allocation - this thread will (eventually) join the collection of threads being scheduled and run by the kernel, and the information associated with the thread (which is stored in the Thread object itself) needs to persist beyond the subroutine containing this line of code. You do not need to remember to delete t later on - the scheduler takes care of deallocating the Thread object after the thread finishes executing.
Creating a new thread object does only minimal setup. If you want the thread to actually do something, you need to fork the thread:
t->Fork(somefunction,NULL);
Fork() sets up the stack for the new thread and adds it to the ready list so that it will eventually be scheduled. The first parameter is the function for the thread to run and the second is the parameter to give to that function.
Fork() uses a few features of C++ that you may not have seen before. If you look at the declaration of Thread::Fork(), you'll see the following:
void Fork(VoidFunctionPtr func, void *arg); // Make thread run (*func)(arg)
The first is the ability to pass a function as a parameter. VoidFunctionPtr is defined in lib/utility.h and is a typedef for the more complex syntax needed to specify the type for a function which takes a single void * parameter and returns nothing (void). When you supply a value for Fork's first parameter (somefunction in the example), you use the actual name for a function which takes a single void * parameter and has a return type of void - this isn't a function call, and doesn't have parens. The second parameter, void * arg, is a way to achieve a bit of generic programming. We don't want to limit threads to only be able to execute functions that take integer parameters, for example, since many situations would require a string parameter, or a double, or something else. By declaring arg to be void *, we're saying that arg will be a pointer (because of the *) to something. The implementation of somefunction will cast arg to whatever type is appropriate. Note that arg is void * so it can only be cast to another pointer type e.g. int * or string *. This also means that any value you pass to Fork must be a pointer. Use NULL for cases where somefunction doesn't use a parameter.
StackAllocate() is responsible for initializing the thread's stack (by allocating memory and placing a sentinel value at the top of the stack to help detect if a thread overflows its stack during execution) and setting up thread's initial state. Recall how the new thread will begin running the first time - a context switch will switch to it. As a result, the saved thread context must look exactly like what it would be if the thread had previously been switched out and had the context saved by SWITCH().
There is one detail, which is that the new thread doesn't just run the function passed to Fork(). There's a little bit of setup and cleanup that needs to be done. On the setup end, interrupts are off when a newly-in-control thread emerges from SWITCH() in Scheduler::Run() - and the brand-new thread wasn't switched out in the middle of some operation which will turn interrupts back on again, so it is necessary to turn interrupts back on before executing the thread function. On the cleanup end, interrupts are turned off and the thread is put to sleep to remove it from the ready queue and indicate that it is finished. The three steps - setup, running the desired thread function, and cleanup - are accomplished by ThreadRoot, a machine-dependent assembly-language function defined in thread/switch.s. ThreadRoot expects its four parameters (the addresses of the three functions to run, plus the argument for the thread function) to be in certain registers, so StackAllocate() initializes the new thread's saved context with the appropriate values (and sets the saved PC to the address of ThreadRoot). Now, when SWITCH() restores the saved registers to the host machine's real registers, the new thread will begin executing ThreadRoot.
A thread may yield itself with:
kernel->currentThread->Yield();
(Something to think about: why is it that kernel->currentThread->Yield() is guaranteed to yield the thread itself and not some other thread?) Yielding allows a thread to voluntarily give up the CPU, and invokes the scheduler to pick and dispatch a new thread. The yielding thread is returned to the ready queue so that it can be scheduled again in the future.
Thread::Sleep() causes the current thread to yield the CPU, because it has finished its task or is blocked waiting on something. The main difference between Sleep() and Yield() is that Yield() returns the calling thread to the ready queue for future scheduling, while Sleep() does not. A sleeping thread either does not need to be scheduled again (if it has finished) or will be re-added to the ready queue when the event it is blocked on occurs.
The context-switching routine is actually named SWITCH, and is implemented in switch.s (there are several versions, because this is a machine-dependent assembly-language routine). SWITCH is machine-dependent (and written in assembly language) because it is saving host-machine registers in order to capture the current state of the kernel code executing on the host machine. If you are interested, look at the #ifdef x86 section of switch.s - this is the implementation for Intel x86 processors. You may find a description of Intel register names useful. Apart from a few shufflings of the EAX (accumulator register) value because the register is needed for other things, it is a pretty straightforward save of the current registers into memory locations in the old thread's context and load of new register values from memory locations in the new thread's context.
Saving the PC: When SWITCH is called to carry out a context switch, the address of the instruction to be executed after SWITCH is complete (i.e. the return address) is stored on the old thread's stack as part of the normal activation record associated with a function call. This return address is used as the saved PC instead of the current contents of the actual PC register, because the current PC value at the time of the save would be the address of the instruction which is saving the PC - and we want to resume after the SWITCH when we eventually switch back to this thread.
Restoring the PC: SWITCH doesn't explicitly change the PC. Instead, it restores the saved PC into the place for the return address in the activation record on the current stack. As a result, when SWITCH exits and the return-from-function instruction ret is executed, the desired PC value is read from the activation record and execution "returns" to the new thread, completing the context switch.
The steps that Run() goes through are a little different than described:
One tricky thing is that the thread which executes the first four steps is not the same thread which returns from the SWITCH and executes the last two - at least not right away. Make sure you understand why this is. (On a related note, why is the line
DEBUG(dbgThread, "Now in thread: " << oldThread->getName());
after the return from SWITCH() in Scheduler:Run() correct for displaying the name of the thread now executing, instead of newThread->getName()?)
This not-the-same-thread-exiting-as-entering thing is important for explaining the mechanism for destroying threads when they complete. When the currently-running thread determines that it is done, it can't just deallocate its own stack and other memory. The reason? Being done means that a context switch will occur to start a new thread running. The context switch involves calling SWITCH, which saves the old thread's registers in the old thread's memory...which is a problem is that memory has been deallocated. In general, until we've actually loaded the new thread's context and switched the PC, we're still running on the old thread's stack - and deleting that while we're still using it is bad news. So, a solution is to mark the old thread as "ready to be destroyed" when the scheduler is asked to dispatch a new thread, and then have the newly-dispatched new thread do the actual cleanup before it returns to whatever it was doing before it got switched out at some point in the past. The code for both marking the old thread as "ready to be destroyed" and having the new thread do the actual destroying is in Scheduler::Run() - and it works because the call to SWITCH() in the middle has changed which thread is currently running. The old - and now deallocated - thread won't ever be dispatched again, because Thread::Sleep() resulted in the call to Scheduler::Run() so the old thread is not in the ready queue and, since it isn't waiting on anything, it can't ever re-enter the ready queue.
The semaphore implementation in Nachos is the Semaphore class in threads/synch.h. Nachos uses the traditional P (wait) and V (signal) terminology. The operations P() and V() are made atomic by disabling interrupts. (It is assumed that Nachos is running in a single processor environment.) Each semaphore has a queue associated with it, which contains those threads which are waiting for the semaphore to become available. If a thread tries to acquire a semaphore that is not available, it invokes Sleep() on itself. (This causes a context switch to another thread and doesn't place the current thread back on the ready queue, so it won't be scheduled until something else puts it back into the ready queue.) When the semaphore is signalled (V()), one waiting thread is removed from the waiting queue and is added to the scheduler's ready list for future execution.
"a.out" is the default filename used by many compilers for the executable output (try compiling a program and leaving out the -o <executable>), and is also the name of an older format for compiled object files. In this sense, "a.out" is being used to refer to executables.
The coff2noff program mentioned is invoked by the Makefile that has been provided with the sample user programs, so you don't need to worry about calling it.
Each process has its own virtual address space, so the process will always see memory addresses between 0 and MAX_ADDR regardless of where in memory the process is loaded. This virtual address space is arranged as follows:
| logical address | segment |
|---|---|
| 0 | program code (read-only) |
| read-only data (global constants, if initialized) | |
| initialized data (global variables initialized when they are declared) | |
| uninitialized data (global variables which are not initialized when they are declared) | |
| max | stack (includes local variables as part of each function's activation record) |
Nachos uses page-aligned segments, so that each segment starts at a page boundary. This means there may be some internal fragmentation on the last page of each segment, but it avoids pages which contain some writeable data and some read-only data.
As is typical, the bottom of the stack is at the highest logical memory address and the stack grows into lower memory.
The program code, read-only data, and initialized data segments are read from the executable; the uninitialized data and stack segments aren't backed by the executable and are zeroed-out (i.e. initialized with all 0s) when first "loaded".
Quirks/things to be aware of:
Be careful not to overflow the process stack - e.g. by recursing too deeply or allocating large data structures on the stack. If you do, you may get segmentation faults (but segmentation faults occur for a variety of reasons and may not indicate a stack overflow). Some checking is done in the thread class to try to detect stack overflows, but this may not catch everything. There are two workarounds: (1) dynamically allocate memory instead of using automatic variables, or (2) increase the value of StackSize in threads/thread.h.
Sometimes user programs with global data structures don't work correctly in Nachos. Be careful!
The provided Nachos used in projects 1-3 reads the entire process into memory at once, when the AddrSpace object for the process is constructed. There are three methods used: the AddrSpace constructor itself, which calls AddrSpace::Load to actually load the executable, which in turn calls AddrSpace::LoadSegment to load individual segments of the executable.
AddrSpace::Load does the following:
AddrSpace::LoadSegment reads a segment one page at a time. (This is necessary because the physical frames may not be contiguous.) It is passed (among other things) the starting address in the file of the segment (this was obtained from the NOFF header), the number of bytes in the segment (also from the NOFF header), and virtual address for where that segment will begin in the logical address space. While the segment has not been completely read, it repeatedly:
The AddrSpace::SaveUserState and AddrSpace::RestoreUserState methods are actually called AddrSpace::SaveState and AddrSpace::RestoreState. Their jobs are the same - to save/restore any necessary address-space state when a context switch occurs.
All of the details of creating a noff binary are handled by the Makefile that has been provided with the sample user programs, so you can ignore this section if you want.
RaiseException is in the Machine class, and ExceptionHandler is defined in userprog/exception.cc.
System calls provide an interface between the OS kernel and user-level programs. A mechanism is typically provided for making system calls from C programs. In Nachos, this mechanism takes the form of a header file which defines the C interface for each of the system call functions, plus some assembly-language code which is linked with your user program as part of the compilation process. When the user program makes a system call, this assembly-language code places the parameters that were passed to the system call by the user program into certain registers and then triggers Nachos' exception-handling mechanism to carry out the call.
The FileSystem object is defined in filesys/filesys.h.
A consequence of supporting two filesystems (the "stub" front-end for the Unix file system, and the Nachos file system) is that filesys/filesys.h has two definitions for the FileSystem class. The #ifdef FILESYS_STUB ... #else ... #endif ensures that only one definition is active at a time. This acts like an if statement, but the # prefix indicates that these are preprocessor instructions - the preprocessor selects which version to use and passes just that to the compiler. Which is active can be controlled by specifying a particular flag to the compiler - if you look at the DEFINES near the beginning of build/Makefile, you'll see -DFILESYS_STUB which defines "FILESYS_STUB" as a symbol and thus selects the #ifdef FILESYS_STUB part of filesys/filesys.h.
The OpenFile object is defined in filesys/openfile.h.
It does not have a close method - deallocating the OpenFile object to close the file.
The FileHeader and Directory classes have both in-memory and on-disk representations. The on-disk representation is critical, since that is what preserves the information when the machine is off (or, in the case of Nachos, not running). The in-memory representation supports manipulation of the data structure. For both classes, the typical scenario is the following:
The key things to keep in mind here are: (a) creating a FileHeader or Directory object does not allocate space on the disk (or change the disk in any way), (b) creating a FileHeader or Directory object does not retrieve information from disk, and (c) manipulating the in-memory data structure does not automatically change the on-disk representation (WriteBack is needed for that).
The FileHeader object is defined in filesys/filehdr.h.
The size of the array containing the sector numbers for the file's data blocks is defined as follows in filesys/filehdr.h:
#define NumDirect ((SectorSize - 2 * sizeof(int)) / sizeof(int))
The reason for this definition is to ensure that the file header always fits within a single disk sector, regardless of the size of a disk sector or the size of an integer (which may vary a bit from compiler to compiler). SectorSize, defined in machine/disk.h, is the number of bytes in a disk sector. sizeof is a C/C++ routine which returns the number of bytes occupied by a single instance of the type named. The number of direct pointers, NumDirect, is thus calculated to be the number of bytes in the disk sector minus the number of bytes required by two integers (the numBytes and numSectors instance variables), divided by the number of bytes required by an integer (since each sector number will be an integer).
Note that creating a new FileHeader object only creates a new in-memory data structure - FileHeader::Allocate (for new files) or FileHeader::FetchFrom (for existing files) must be called to initialize that in-memory data structure with actual information. Also note that the FileHeader methods other than FetchFrom and WriteBack only work with the in-memory data structure - it is the responsibility of the user of the file header to WriteBack any changes to disk.
FetchFrom and WriteBack use a clever little trick to read/write the file header from/to disk. The body of FetchFrom:
kernel->synchDisk->ReadSector(sector, (char *)this);
WriteBack is similar.
This takes advantage of how the memory is laid out for objects, as well as the fact that memory is really just memory with types being something imposed by the compiler/runtime system. ReadSector expects a sector number and an array of characters (each character = 1 byte) to read data into - of course, in typical C/C++ style, the array of characters is referenced by a pointer to the first slot. ReadSector then reads one sector worth of characters (128 of them) into the array, which it assumes is large enough for this many characters.
Of course, this is not an array of characters - it's a pointer to the current object, a FileHeader. But, as it happens, the memory for an object is laid out so that the instance variables occupy a contiguous chunk and appear in the order they are declared in the header file (so, in this case, numBytes has the first 4 bytes, numSectors has the next 4 bytes, and dataSectors has the next 120 bytes). Since the instance variables occupy a contiguous chunk of memory like an array, we can pass the address of the beginning of this chunk to ReadSector. All that remains is determining the address of the beginning of the chunk of memory - as it happens, the memory allocated for the instance variables begins at the beginning of the object's memory. As a result, this contains the value we need. (It should also work to use &numBytes instead of this, since numBytes is the first instance variable, but using this means the implementation doesn't depend on the exact ordering of the instance variable.)
You can read this section, but keep in mind that the comments may not be relevant for our version of Nachos or your assignment.
|
|