So today we're going to continue our discussion
of enforcing modularity.
And in particular if you remember last time we spent
a lot of time talking about --
Last time we spent a lot of time talking
about how we can create a virtual memory so that we can
have multiple programs running on top of one piece of hardware
that all appear to have a separate address
space or separate memory.
So, two lectures ago, we talked about having
one module per computer.
And then last time we talked about this notion
of wanting to have a module per virtual computer.
And we saw this notion of virtual memory.
OK, and now this time what we're going to do
is we're going to talk about the other piece
that our virtual computer needs, which is a virtual processor.
OK, so the idea that we want here
when we are designing a virtual processor
is to create, have our programs be
able to run on a single physical processor,
but have different programs be able to run as though they
were the only program that was executing on that processor.
So we want the programs to be written in a style
where the programmer doesn't have
to be aware of the other programs that are running
on the machine, but where the machine creates sort
of a virtual processor for each one of these programs
that's running so that each program has its own sort of set
of instructions that it executes.
And those instructions appear to be
independent of all the other programs that are running.
So in order to create this illusion of a virtual processor
--
-- we are going to introduce a very simple concept.
We're going to have each program run in what's
known as a thread.
And thread is really short for a thread of execution.
And a thread is just a collection of all of the state
that we need in order to keep track
of what one single given program is doing
at a particular point in time.
So a thread is a collection of the instructions
for the program that the module is running,
as well as the current state of the program.
In particular, it's the value of the registers on the processor
including of particular import for this lecture the program
counter and the stack pointer.
OK, so there's a bunch of registers on the processor,
say, 32 registers, and there also
are these special registers, the program counter,
that keeps track of which instruction is currently
being executed, and the stack pointer which
keeps track of where on the stack
values are currently being pushed or popped from.
So these things combined together with the stack
are really the things that define one single thread.
So, and the way to think about this
is if you think about a program that's running,
you can really encapsulate almost everything about what
that program is currently doing by the instructions
that it has to execute, the value of the registers that
are currently set in the program,
and the state that's on the stack.
Now, programs may also have some sort
of data that's stored in the global memory
in the global variables our things that
are stored on the heap that have been allocated via some memory
allocation call.
And so those things are a part of the program
as well, although I don't want to explicitly list them
as being a part of a single thread
because if two programs, as we'll see,
two programs can share the same address space.
And if two programs are sharing the same address space,
then they're going to share those global variables,
the stuff that's stored on the heap of the program.
So think of a thread as these things, plus there
is some additional, we'll call it a heap,
set of memory that may be shared between several different
threads that are all running within the same address space.
And I'll explain more about that as we go.
OK, so as I said, now what we want
is to create this illusion that each one of these threads
has its own virtual processor upon which it's running.
So how are we going to accomplish that?
So if you think about this for a second,
the way to accomplish this is simply
to allow each one of these, each thread
to update these various registers,
push things onto the stack, and then
to have some function which we can call,
which will save all of this current state
off for the current thread that's executing,
and then load the state in for another thread that might be
executing, and stop the current thread from executing,
and start the new one executing.
So the idea is if you were to look at a timeline of what
the processor was doing, and if the processor is running
two threads, thread one and thread two,
you would see that during some period of time,
thread one would be running.
Then the computer would switch over
and thread two would start running,
and then again the computer would switch,
and thread one would start running.
So we are going to multiplex the processor resource in this way.
So we have one processor that can be executing instructions.
And at each point in time, that processor
is going to be executing instructions
from one of these two threads.
And when we switch in these points in between threads,
we are going to have to save off this state for the thread that
was running, and restore the state for the thread that
is currently running.
OK, so that's a very high-level picture
of what we're going to be doing with threads.
And there's going to be a number of different ways,
and that's what we're going to look at through this lecture is
how we can actually accomplish this switching,
how we decide when to switch, and what actually happens
when this switch takes place.
OK, so we're going to look at several different methods.
So the first thing we're going to look at
is an approach called cooperative scheduling,
or cooperative switching.
So in cooperative switching, what happens
is that each thread periodically decides
that it's done processing and is willing to let
some other process run.
OK, so the thread calls a routine that says I'm done;
please schedule another thread.
So, this cooperative approach is simple,
and it's easy to think about.
In the grand scheme of enforcing modularity
of creating this illusion that each program really
has its own computer that's running,
and where each program gets to run no matter
what any other program does, cooperative scheduling
has a bit of a problem because if we
are doing cooperative scheduling,
then one process can just never call this function that
says I give up time to the other processes running
on the system.
One module may just never give up the processor.
And then we are in trouble because no other module ever
gets to run.
So instead what we're going to talk about,
the alternative to this, is something
we call preemptive scheduling.
And in preemptive scheduling, what
happens is some part of the computer, say, the kernel,
periodically decides that it forces the currently
running thread to give up the processor,
and starts a new thread running.
And we'll see how that works, OK?
I also want to draw a distinction
between whether these threads are running in the same address
space or in a different address space,
or in a different address space.
OK, so we could, for example, have these two threads, T1
and T2.
They might be running as a part of a single, say, application.
So, for example, they might be a part of a,
say for example, some computer game that we have running.
So suppose we have a computer game like, but let me just load
up my computer here.
Suppose we have a computer game like
pick your favorite computer game, say, Halo.
And we are running Halo, and Halo
is going to consist of some set of threads
that are responsible for running this game.
So we might have one main thread which,
what it does when it runs is to simply update;
it repeats in a loop over and over again,
wait for a little while, check and see
if there's any user input, update the state of the game,
and then, say for example, redraw the display.
And because Halo is a videogame that somebody
is staring at and looking at, it needs
to update that display at some rate,
say, once every 20 times a second in order
to create the illusion of sort of natural animation
in the game.
So this wait step is going to wait a 20th
of a second between every step.
OK, so this might be one example of a thread.
But of course within Halo, there may be multiple other threads
that are also running.
So there may be a thread that's responsible for looking
for input over the network, right,
to see if there is additional data that's
arrived from the user, and from other users that are remote
and updating the state of the game
as other information comes in.
And there may be a third thread that, say for example,
is responsible for doing some cleanup work in the game
like reclaiming memory after some monster that's
running around the game has died and we no longer need
its state, OK?
So if we look at just this, say, main thread within Halo,
as I said here we can encapsulate
the state of this thread by a set of registers,
say, R1 through Rn, as well as a stack pointer
and a program counter.
And then, there also will be a stack
that represents the currently executing program.
So, for example, the stack might,
if we just have entered into this procedure,
it might just have a return address
which is sort of the address that we
should jump to when we're done executing this Halo
program here.
And then finally, we have a program counter
that points to this sort of current place
where we are executing it.
So we might have started off where we are just
executing the first instructions here, the init instruction.
OK, so what we're going to look at now
is this case where we have a bunch of threads,
say for example, in Halo all running within the same address
space.
So they're all part of the Halo program.
They can all access the global variables of Halo.
But we want to have different threads
to do different operations that this program may
need to take care of.
So we want each of these threads to sort of have the illusion
that it has its own processor.
And then later we'll talk about a set of programs,
say, that are running in different address
And we'll talk about what's different about managing
two threads, each of which is running in a different address
space.
OK, so the very simple idea that we're
going to introduce in order to look at this situation
where we are going to start off by looking
at cooperative scheduling in the same address space.
We are going to introduce the notion of a routine called
yield.
OK, so yield is going to be the thing the currently
executing thread calls when it's ready to give up the processor
and let another thread run.
So the thread is going to run for a while
and then it's going to call yield,
and that's going to allow other threads to sort of do
their execution.
And this is cooperative because if a program doesn't
call yield, then no other program will ever be
allowed to run on the system.
So the basic idea is that when a program calls yield,
what it's going to do is exactly what I
described just a minute ago.
So yield is going to save the state of the current thread,
and then it's going to --
-- schedule the next thread, so pick the next thread to run
from all of the available threads that are on the system.
And then it's going to dispatch the next thread,
so it's going to in particular setup
the state for the next thread so that it's ready to run.
And it's going to jump into the return address for that thread.
OK, so I want to take you guys through a very simple example
of a program that has multiple threads in it.
And we're going to have a simple thread scheduler.
So, the thread scheduler is the thing
that decides which thread to run next.
And this thread scheduler is going
to keep an array of fixed size, call it
num_threads, that are all the threads that are currently
running on the system.
It's going to have an integer that is going to tell us
what the next thread to run is.
And then for every one of our threads,
we're going to have a variable that we
call me which is the ID of the currently executing thread.
So, this is local to the thread.
OK so let's look at an example of how
this might work with this Halo thread
that we're talking about here.
OK, so suppose we have a Halo thread.
And we'll call this the main thread of execution.
And as I said, we want to have a couple of other threads
that are also running on the system at the same time.
So we might have a network thread and a cleanup thread.
Now in this case, what we are going to say
is that the yield procedure is going to do these three steps.
And I made these steps a little bit more concrete so
that we can actually think about what
might be happening on the stack as these things are executing.
So, the yield procedure is going to, when it runs,
the first thing it's going to do is save the state.
So to save the state, it simply stores in this table
the stack pointer of the currently executing program,
the stack pointer of the currently executing thread.
And then what it's going to do is
it's going to pick the next thread to run.
So in this case, picking the next thread to run,
the thread scheduler is doing something extremely simple.
It's just cycling through all the available threads one
after another and scheduling them, right?
So it says next gets next plus one, and then modulo
the total number of threads that are in the system.
So suppose num_threads is equal to five.
As soon as next plus one is equal to five,
and then five modulo five is zero,
and we're going to wrap that back around.
So, once we've executed the fourth thread,
we are going to wrap around and start
executing thread number zero.
And then finally what it's going to do
is it's going to restore the stack pointer.
Here, I'll just change this for you right now.
This should be table sub next.
I didn't want to do that.
So, we have this table.
So, we restore the stack pointer for the next thread
that we want to run from the table of stack pointers
that are available.
And then what's going to happen is
that when we exit out of this yield routine,
we are going to load the return address from the stack pointer
that we just have set up.
So you'll see how that works again in a second.
But the basic process, then, is just going to be as follows.
So this only works on a single processor.
And I'm not going to have time to go into too much detail
about what happens in a multiprocessor,
but that book talks carefully about why this doesn't work
on a single processor machine.
OK, so the basic process, then, it's going to be as follows.
So we've got our yield procedure.
We have our three threads.
Say we've numbered them one, two, and three,
as shown up here.
So, num_threads is equal to three,
and say we start off with next equal to one.
So what's going to happen is that the first thread at some
point it's going to call a yield routine.
And what that's going to do is it's going
to cause this yield to execute.
We're going to increment next.
And we're going to schedule the network thread here.
OK, and the network thread is going
to run for a little while.
And then at some point, it's going to call yield,
and that's going to sort of cause the next thread
to be scheduled.
We're going to call cleanup, and then finally we're
going to call yield.
And the third thread is going to call yield and cause
the first thread to run again.
And this process is just going to repeat
over and over and over again.
OK, so --
OK, so what I want to do now is to look at actually
more carefully at what's going on with, how this scheduling
process is actually working.
So we can understand a little bit better
what's happening to the stack pointer,
and how these various yields are actually executing.
So let's look at, so this is going
to be an example of the stack on, say, two of these threads,
OK?
So, suppose we have thread number one.
I'll call this thread one, and this is our Halo thread.
OK, and this is the stack for Halo.
We also have thread number two, which is our network thread.
OK, and this is also some stack associated with this thing.
So we're just looking at the current state
of the stack on these two things, these two threads.
And if you remember, typically we
represent stacks as going down in memory.
So, these are the different addresses
of the different entries in the stack that
represent what's going on currently in the processor.
So we'll say that thread one, perhaps, the stack
starts at address 108, and maybe thread two
starts at address 208.
OK, so suppose we start up this system,
and when we start it up, the stack pointer
is currently pointing here.
So we take a snapshot of this system
at a given point in time.
Suppose the stack pointer is currently pointing at 104.
The currently executing thread is this one, thread one.
And we have, remember our thread table
that captures the current thread number,
and the saved stack pointer for each one of these threads.
So we've got thread number one, thread number two.
OK, so let's suppose we start looking at this point in time
where the stack pointer is pointing here.
And if we look at this program, so if we start off
with the program that we initially had,
in order to understand what happens to the stack,
we need to actually look at when the stack gets manipulated.
So if we are just looking at C code,
typically the C code isn't going to have,
we're not going to see the changes to the stack occurring
within the C code.
So what I've done here is in yellow,
I've annotated this with the sort of additional operations,
the assembly operations, that are happening on the entry
and exit to these procedures.
So what happens just before we call the yield procedure is
that the Halo thread will push the return
address onto the stack.
OK, so if we start executing the yield procedure,
Halo is going to push the return address onto the stack.
And then it's going to jump into the yield procedure.
So, suppose we come onto here.
Now if we look at what the return address is
after we execute the yield procedure,
it's going to be instruction number five.
So we're going to push that address onto the stack
because instruction number five is labeled here
on the left side is where we're going
to return to after the yield.
And then the yield is going to execute.
And what's going to happen as the yield executes
is it's going to change the stack pointer to be
the stack pointer of the next thread, right?
So, I have this typo here again.
So be careful about that.
We're going to change the stack pointer
to be the stack pointer of the next thread.
And so when it executes this pop RA instruction
that we see here, it's actually going
to be popping the return address from the network thread
that it scheduled next as opposed to the return
address from this Halo thread.
OK, so suppose that the saved stack pointer for the network
thread was pointing here at 204.
So, that would have been an entry, 204, here.
And the saved return address that it's going
to jump to is going to be here.
So suppose the return address that it jumps to is 1,000, OK?
So if we look at the state of these two threads
as they have been running, we saw
thread one ran for a little while,
and then it called yield, and the instruction following yield
was instruction number five.
OK, so that was the address that we pushed on
to the stack pointer.
Now, we switched over here to thread two.
And last time we ran thread two, it called yield,
and its return address was 1,000.
So, there was this 1,000 here, OK?
So now, when we schedule thread two, what's going to happen
is that we're going to read the, so this is instruction 1,000.
We're going to start executing from 1,000
because we're going to execute this pop return
So we're going to pop the return address off the stack,
and we're going to start executing here
at instruction 1,000, OK?
So, you guys kind of see how it is that we switch now
from one thread to the other, by switching the stack pointer
and grabbing our return address from the stack of thread two
instead of from thread one.
OK, so now what's going to happen
is this thread two is going to start
executing from this address.
And it's going to run for a little while.
And at some point later, it's going
to call yield again, right, because it's run for a while
and it's decided that it's time to yield.
So, suppose that instruction 1,024 it calls yield.
So when it does that, the same thing is going to happen.
It's going to have run for a little while.
So its stack is going to have grown.
But eventually it's going to push the return address
onto the stack, say here 1,025, and suppose
this value of its stack pointer now has grown down.
So it's gone 204.
It's run for a while, pushed some things on the stack,
and maybe the stack pointer is now at 148.
So, when it calls yield, it's going
to write into the stack pointer address in the thread table
148.
And we're going to restore the stack pointer addressed
from here, which I forgot to show being written in,
but what I should have written in there was 104.
So, we're going to restore the stack address to 104.
We're going to pop the next instruction to execute off
of that stack five, and then we're going to keep executing.
So you just switch back and forth in this way.
OK, so -- --
what I want to do now is, so this is the basic process now
whereby this yield instruction can
be used to switch between the scheduling of these two
procedures, right?
And this is sort of the core of how it is.
So we have these two threads of execution,
and they just sort of run through.
And they periodically call yield.
And that allows another thread to be written, to execute it.
But otherwise these threads have been
written in a style where they don't actually
have to know anything about the other threads that are running.
There could have been two other threads
or 200 other threads running on the system at the same time.
And this approach that I showed you
would have caused all those threads eventually
to have been scheduled and to execute properly.
Right, but as we said before, requiring
these functions to actually call yield
periodically has sort of defeated
the purpose of our enforcing modularity, one
of our goals of enforcing modularity,
which is to make it so that no one thread can interfere
with the operation of the other thread,
or cause that other thread to crash, right,
because if the procedure never calls a yield,
then a module never calls yield, excuse me,
another thread will never be scheduled.
And so, that module will have the ability, essentially,
to take over the whole computer, which is bad.
So what we're going to look at now
is how we go from this cooperative scheduling
where modules call yield to preemptive scheduling
where modules are forced to yield
the processor periodically.
OK, so this is the case where you have no explicit yield
statements.
All right, so the idea here is that turns out
to be very simple.
So programs aren't going to have an explicit yield
statement in them.
But what we're going to do is we're
going to have a special timer that runs periodically within,
say, for example, the kernel.
So suppose the kernel has a timer that runs periodically
that causes the kernel to be sort of woken up and allowed
to execute some code.
So this timer is going to be connected
to what's called an interrupt.
So, we're going to introduce a timer interrupt,
and almost all processors essentially
have some notion of a timer interrupt.
And an interrupt is something that when it fires,
it causes the processor to run a special piece of code, OK?
So, basically this is going to be some processor.
Think of it, if you like, as some line on the processor.
OK, there's a wire that's coming off the processor.
And when this wire gets pulled high, when the timer goes off
and fires, this line is going to be pulled high.
And when that happens, the microprocessor
is going to notice that and is going
to invoke a special function within the kernel.
So this line is going to be checked by the microprocessor
before it executes each instruction, OK?
And if the line is high, the microprocessor
is going to execute one of these special gate functions.
So, we saw the notion of a gate function
before that can be used for a module
to obtain entry into the kernel.
Essentially what's going to happen
is that when the timer interrupt fires,
it's going to go execute one of these special gate functions
as well.
And that's going to cause the kernel to then
be in control of the processor.
So remember when the gate function runs,
it switches the user to kernel mode bit, to kernel mode.
It switches the page map address register to the kernel's page
map so that the kernel is in control,
and it switches the stack pointer to the kernel's saved
stack pointer so that the kernel is in control of the system,
and can execute whatever code that it wants.
So this is going to accomplish basically everything
that we need, right?
Because if we can get the kernel in control of the system, now,
the kernel can do whatever it needs to do to, for example,
schedule the next thread.
So the way that's going to work is basically very simple.
So when the kernel gets run, it knows
which thread, for example, is currently running.
And basically what it does is it just
calls the appropriate yield function,
it calls the yield function for the thread
that it's currently running, forcing that thread to yield.
So, kernel calls yield on current thread, right?
OK, so in order to do this, of course
the kernel needs to know how to capture
the state for the currently running thread.
But for the most part that's pretty simple
because the state is all encapsulated
in the current values of the registers in the current value
of the stack pointer.
So, the kernel is going to call yield on the currently
executing thread, and that's going
to force that thread to go ahead and schedule another thread.
So the module itself hasn't called yield,
but still the module has been forced to sort of give up
its control the processor.
So, when it calls yield, it's just
going to do it we did before, which
is save our state and schedule and run the next thread.
OK, so the only last little detail
that we have to think about is what if the kernel wants
to schedule a thread that's running in a different address
space?
So if a kernel wants to schedule a thread that's
running a different address space,
it well, it has to do what we saw last time when
we saw about how we switch address spaces.
It has to change the value of the PMAR register
so that the next address space gets swapped
And then it can go ahead and jump
into the appropriate location in the sort of newly
scheduled address space.
So that brings us to the next topic of discussion.
So, what we've seen so far now, we saw cooperative
in the same address space, and I introduced the notion
of a preemptive scheduling.
What we want to look at now is sort
of what it means to run multiple threads in different address
spaces.
Typically when we talk about a program that's
running on a computer or in 6.033
we like to call programs running on computers processes.
When we talk about a process, we mean an address space
plus some collection of threads.
So this is sort of the real definition
of what we mean by process.
And alternately, you will see processes called things
like applications or programs.
But any time you see a word like that, what people typically
mean is some address space, some virtual address space in which
we resolve memory addresses.
And then a collection of threads that are currently
executing within that address space,
and where each of those threads includes
the set of instructions and registers and stack
that correspond to it, OK?
So the kernel has explicit support for these processes.
So in particular, what the kernel provides
are routines to create a process and destroy a process.
OK, and these create and destroy methods
are going to sort of do, the create method
is going to do all the initialization
that we need to do in order to create a new address space
and to create at least one thread that is running
within that address space.
So we've sort of seeing all the pieces of this,
but basically what it's going to do
is it's going to allocate the address
space in its table of all the address spaces
that it knows about.
So we saw that in the last time, and it's
going to allocate a piece of physical memory
that corresponds to that address space.
So, it's going to allocate a piece of physical memory
that corresponds to that address space.
It's going to allocate one of these page maps
that the PMAR register is going to point
to when this thing is running.
OK, and now the other thing that it's going to do
is to go ahead and load the code for this thing
into memory and map it into the address space.
So it's going to add the code for this currently running
module into the address space.
And then it's going to create a thread for this new process.
And it's going to add it to the table, the thread table,
and then it's going to set up the value of the stack pointer
and the program counter for this process
so that when the process starts running it,
you sort of into the process at some well-defined entry point,
say the main routine in that process
so that the thread can start executing at whatever
starting point it has.
OK, and now destroy is just going
to basically do the opposite.
It's going to get rid of all this state that we created.
So it's going to remove the address space,
and it's going to remove the thread from the table.
OK, and it may also reclaim any memory
that's associated exclusively with this process.
OK so if we look at --
So if you look at a computer system at any one point
in time, what you'll see is a collection of processes.
So I've drawn these here as these big boxes.
So you might have a process that corresponds to Halo
and, say, we are also editing our code.
So, we have a process that corresponds to emacs.
OK, and each one of these processes
is going to have an address space associated with it.
And then there are going to be some set of threads
that are running in association with this process.
So in the case of Halo, I'm going
to draw these threads as these little squiggly lines.
So in the case of Halo, we saw that maybe it
has three threads that are running within it.
So these are three threads that all run within the same address
space.
Now, emacs might have just one thread that's running in it,
say, although that's probably not true.
emacs is horribly complicated, and probably has many threads
that are running within it.
And so, we're going to have these sort of two processes
with these two collections of threads.
So if you think about the sort of modularity
or the enforcement of modularity that we
have between these processes, we could
say some interesting things.
So first of all, we've enforced modularity.
We have hard enforced modularity between these two processes,
right, because there's no way that the Halo process can muck
with any of the memory that, say, the emacs process has
executed.
And as long as we're using preemptive scheduling,
there's no way that the Halo process could completely
maintain control of the processor.
So the emacs process is going to be allowed to run,
and its memory is going to be protected from the Halo
process.
OK, now within the Halo process, there
is sort of a different story.
So within the Halo process, there
is sort of a different story.
So within the Halo process, these threads,
because they are all within the same address space,
can muck with each other's memory.
So they are not isolated from each other in that way.
And typically, because these are all
within one process in that if we were
to destroy that process in this step, what the operating
system would do is in addition to destroying the address
space, all of the running threads.
So we say that these threads that are running within Halo
share fate.
If one of them dies, or if one of these threads
fails or crashes or does something bad,
then they are essentially all going
to crash or do something bad.
If the operating system kills the process,
then all of these threads are going to go away.
So this is the basic picture of sort
of what's going on within a computer system.
If you stare at this for a little bit,
you see that there is actually sort of a hierarchy of threads
that are running on any one computer system
at any one point in time.
So we have these larger sorts of processes that are running.
And then within these processes, we
have some set of threads that's also running, right?
And so, there is a diagram that sort of useful
to help us understand this hierarchy
or layering of processes or threads on a computer system.
I want to show that to you.
OK, so on our computer system, if we look at the lowest layer,
we have our microprocessor down here.
OK, and this microprocessor, what we've seen
is that the microprocessor has two things
that it can be doing.
Either the microprocessor can be executing, say, some user
program, or the microprocessor can
be interrupted and go execute one of these interrupt handler
functions.
So these interrupts are going to do things like,
is going to be this timer interrupt,
or when some IO device, say for example the disk,
finishes performing some IO operation like reading
a block from memory, then the processor will receive
an interrupt, and the processor can do whatever it needs
to do to service that piece of hardware
so that the hardware can go and, for example,
read the next block.
So in a sense, you can think of the processor itself
as having two threads that are associated with it.
One of them is an interrupt thread, and one of them
is the main thread now running on top of this.
So, these are threads that are really
running within the kernel.
But on top of these things there is this set of applications
like Halo and emacs.
And, these are the kind of user programs
that are all running on the system.
And there may be a whole bunch of these.
It's not just two, but it's any number of threads
that we can multiplex on top of this main thread.
And then each one of these may, in turn,
have sub-threads that are running as a part of it.
So if you think about what's going on in this system,
you can see that at any one level,
these two threads don't really need to know anything
about the other threads that are running at that same level.
So, for example, within Halo, these individual sub-threads
don't really know anything about what the other sub-threads are
doing.
But it is the case that the threads at a lower level
need to know about the threads that
are running above them because these threads at the lower
level implement this thread scheduling policy, right?
So, the Halo program decides which of its sub-threads
to run next.
So the Halo program in particular is,
so the parent thread implements a scheduling --
-- policy for all of its children threads.
So Halo has some policy for deciding
what thread to run next.
The operating system has some.
The kernel has some policy for deciding which of these user
level threads to run next.
And the parent thread also provides some switching
mechanism.
So we studied two switching mechanisms today.
We looked at this notion of having this sort of cooperative
switching where threads call yield in order to allow
the next thread to run.
And, we looked at this notion of preemptive scheduling
that forces the next thread in the schedule to run.
So, if you look at computer systems, in fact
it's a fairly common organization
to see that there is preemptive scheduling at the kernel
level that causes different user level applications to run.
But within a particular user program,
that program may use cooperative scheduling.
So the individual threads of Halo
may hand off control to the next thread only when they want to.
And this sort of makes sense because if I'm
a developer of a program, I may very well
trust that the other threads that I
have running in the program at the same time,
I know I want them to run.
So, we can talk about sort of different switching mechanisms
at different levels of this hierarchy.
OK, so what we've seen up to this point
is this sort of basic mechanism that we
have for setting up a set of threads that are
running on a computer system.
And what we haven't really talked at all about
is how we can actually share information
between these threads or coordinate access
to some shared resource between these threads.
So what I want to do with the rest of the time today
is to talk about this notion of coordinating
access between threads.
And we're going to talk about this
in the context of a slightly different example.
So -- --
let's suppose that I am building a Web server.
And I decide that I want to structure my Web
server as follows: I want to have some network thread,
and I want to have some thread that services disk requests.
OK, so the network thread is going
to do things like accept incoming connections
from the clients and process those connections
and parse the HTTP requests and generate the HTML results
that we send back to users.
And the disk request thread is going
to be in charge of doing these expensive operations where
it goes out to disk and reads in some data
that it may need to assemble these HTML
pages that we generate.
So, for next recitation, you're going
to read about a system called Flash, which
is a multithreaded Web server.
And so this should be a little taste of what you're
going to see for tomorrow.
So, these two threads are likely to want
to communicate with each other through some data
structure, some queue of requests,
or a queue of outstanding information.
So suppose that what we're looking at in fact
in particular is a queue of disk blocks
that are being sent from a disk request thread
out to the network thread.
So, whenever the disk finishes reading a block,
it enqueues a value into the network thread
so that the network thread can then later pull that value off
and deliver it out to the user.
So in this case, these are two threads
that are both running within the same address space
within the same Web server.
So they both have direct access to this queue data structure.
They can both read and write to it at the same time.
This queue data structure, think of it as a global variable.
It's in the memory that's mapped in the address space
so both threads can access it.
So let's look at what happens when these two threads, let's
look at some pseudocode that shows what might be happening
inside of these two threads.
So, suppose within this first thread
here, this network thread, we have
a loop that says while true, do the following.
De-queue a requestn -- call it M --
from the thread, and then process,
de-queue a disk block that's in the queue
and then go ahead and process that disk queue.
Go ahead and process that.
And now, within the disk request thread,
we also have a while loop that just loops forever.
And what this does is it gets the next disk block to send,
however it does that, goes off and reads the block from disk,
and then enqueues that block onto the queue.
So if you like, you can think of this as simply the disk request
thread is going to stick some blocks into here,
and then the network thread is going
to sort of pull those blocks off the top of the queue.
So if you think about this process running,
there's kind of a problem with the way
that I've written this, right, which
is that suppose that the disk request thread runs much
faster than the network thread.
Suppose that for every one network,
one block that the network thread is able to pull off
and process, the disk thread can enqueue two blocks.
OK, so if you think about this for awhile,
if you run this system for a long time,
eventually you're going to have enqueued
a huge amount of stuff.
And typically the way queues are implemented
is they are sort of some fixed size.
You don't want them to grow to fill the whole memory
of the processor.
So you limit them to some particular fixed size.
And eventually we are going to have the problem that the queue
is going to fill up.
It's going to overflow, and we're
going to have a disk block that we don't have anything
that we could do with.
We don't know what to do with it.
Right, so OK, you say that's easy.
There is an easy way to fix this.
Why don't we just wait until there is some extra space here.
So we'll introduce a while full statement here
that just sort of sits in a spin loop
and waits until this full condition is not true.
OK, so as soon as the full condition is not true,
we can go ahead and enqueue the next thing on the queue.
And similarly we're going to need to do something over here
on the process side because we can't really
de-queue a message if the queue is empty.
So if the processing thread is running faster
than the enqueueing thread, we're going to be in trouble.
So we're going to also need to introduce a while loop here
that says something like while empty.
OK, so this is fine.
It seems like it fixes our problem.
But there is a little bit of a limitation
to this approach, which is that now what you see
is suppose these two threads are running.
And they are being sort of scheduled in round robin;
they are being scheduled one after the other.
Now this thread runs.
And when it runs, suppose that the queue is empty.
Suppose the producer hasn't put anything on the queue yet.
Now when this guy runs, he's going to sit here,
and for the whole time that it's scheduled,
it's just going to check this while
loop to see if the thing is empty over,
and over, and over, and over, and over again, right?
so that's all whole lot of wasted time
that the processor could and should
have been doing something else useful like perhaps
letting the producer run so that it could enqueue some data.
So in order to do this, so in order to fix this problem,
we introduce this notion of sequence coordination
operators.
And what sequence coordination operators do
is they allow a thread to declare
that it wants to wait until some condition is true.
And they allow other threads to signal that that condition has
now become true, and sort of allow
threads that are waiting for that condition
to become true to run.
So, we have these two operations:
wait on some variable until some condition is true,
and signal on that variable.
OK, so we're basically out of time now.
So what I want to do is I'll come back
and I'll finish going through this example for the next time.
But what you should do is think about,
suppose you had these sequence coordination operators.
How could we sort of modify these two while loops
that we have for these two operators in order
to be able to take advantage of the fact
that in order to be able to make this
so we don't sit in this spin loop forever and execute.
So that's it.
Today we saw how to get these multiple processes to run
on one computer.
And next time we'll talk about sort making computer programs
run more efficiently, getting good performance out
of this sort of architecture we've sketched.