The following content is provided under a Creative
Commons license.
Your support will help MIT OpenCourseWare continue to
offer high quality educational resources for free.
To make a donation or view additional materials from
hundreds of MIT courses, visit MIT OpenCourseWare at
ocw.mit.edu.
PROFESSOR: The last time we talked about nondeterministic
programming.
And I think actually the mic is up pretty high.
If we can tone that down just a little bit.
We talked about nondeterministic programming.
And as you recall, the rule with nondeterministic
programming is you should never do it
unless you have to.
Today we're going to talk about
synchronizing with locks.
And it goes doubly that you should never synchronize
without locks unless you have to.
There's some good reasons for synchronizing without
locks as we'll see.
But it, once again, becomes even more difficult to test
correctness and to ensure that the program that you think
you've written is, in fact, the program
you meant to write.
So we're going to talk about a bunch of
really important topics.
The first is memory consistency.
And then we'll talk a little bit about lock free protocols
and one of the problems that arises called the AVA problem.
And then we're going to talk about a technology that we're
using in the Cilk++ system, which tries to make an end run
around some of these problems and allows you to do
synchronization without locks, with low overhead.
But it only works in certain context.
So we're going to start with memory consistency.
So here is a very simple parallel program.
So initially a and b are both 0.
And processor zero moves a 1 into a.
And then it moves whatever is in location b
into the EBX register.
Processor one does something complementary.
It moves a 1 into b.
And then it moves whatever is in a into the EAX register.
Into the EAX register as opposed to the EBX register.
And the question is what are the final possible values of
EAX and EBX after both processors have executed.
Seems like a straightforward enough question.
What values can EAX and EBX have, depending upon-- there
may be scheduling of when things happen and so forth.
So it's not always going to give the same answer.
But the question is what's the set of answers
that you can get?
Well, it turns out you can't just answer this question for
any particular machine without knowing the
machine's memory model.
So it depends upon how memory operations behave in the
parallel computer system.
And different machines have different memory models.
And we'll give you different answers for this code.
There'll be some answers that you get on some machines,
different answers on different machines.
So probably the bedrock of memory models is a model
called sequential consistency.
And this is intuitively what you might think you want.
So Lamport in 1979 said, "The result of any execution is the
same as if the operations of all the processors were
executed in some sequential order, and the operations of
each individual processor appear in this sequence in the
order specified by its program."
So what does that mean?
So what it says is that if I look at the processor's
program and the sequence of operations that are issued by
that processor's program, they're interleaved with the
corresponding sequences defined by the other
processors to produce a global linear order.
So the first thing is that there's a global linear order
that consists of all of these processors' instructions being
interleaved.
In this linear order, whenever you perform a load from memory
into register, it receives the value that was stored by the
most recent store operation in that linear order to that
location, i.e. it's memory.
So you don't get something if you have in this linear order
that two processors wrote.
Well, one of them came last.
The most recent one before you read, that's the
one that you get.
Now, there may be many different
interleavings and so forth.
And you could get any of the values that correspond to any
of those interleavings.
But the point is that you must get a value that is
represented by some interleaving.
The hardware can then do anything it wants, but for the
execution to satisfy the sequential consistency model,
for it to be sequentially consistent, must appear as if
the loads and storage obey some global linear order.
So let's be concrete about that with the problem that I
gave before.
So initially, we have a and b are 0.
And now, we have these instructions executed.
So what I have to do is say, I get any possible outcome based
on interleaving these instructions in this order.
So if I look at it, I've got two instructions
here, two over here.
So that there are six possible interleavings because 4 choose
2 is 6 for those people who've taken 6042.
So there are six possible interleavings.
So for example, if I execute first move a 1 into a, and
then I execute move load into register, the value of b, and
then I move 1 into b, and then I load the value of a, I get a
value of 1 for EAX and a value of 0 for EBX.
For this particular interleaving of those
instructions.
That's what happens if I execute these two
before these two.
If I execute these two instructions here before these
two here, I get the order 3412.
And essentially, the opposite thing happens.
EAX gets 0 and EBX gets 1.
And then, if I interleave them in some way, where 1 and 3
somehow come first before I do the 2 and 4, then I'll get a
value of 11 for each of them.
Those are the middle cases.
So what don't I get?
AUDIENCE: 00
PROFESSOR: You never gets 00 in a
sequentially consistent execution.
Sequential consistent implies that no execution--
whoops, that should be EAX.
That EAX equals EBX equals 0.
I don't ever get that outcome.
If I did, then I would say my machine wasn't sequentially
consistent.
So now let me take a detour a little bit to look at mutual
exclusion again.
And understand what happens to mutual exclusion algorithms in
the context of memory consistency.
So everybody understood what sequential consistency is.
I simply look at my program as if I'm interleaving
instructions.
So most implementations of mutual exclusion, as I showed
previously, employ some kind of atomic read-modify-write.
So the example I gave you last time was using the exchange
operation to atomically exchange a value in a register
with a value in memory.
People remember that?
To implement a lock?
So in order to implement a lock, I
atomically switch two values.
So we, in particular, use the exchange one.
And there are a bunch of other commands that people can use.
Test-and-set, compare-and-swap,
load-linked-store-conditional, which essentially do some kind
of read-modify-write on memory.
These tend to be expensive instructions, as I mentioned.
They usually tend to cost something
like an L2 cache hit.
Now, the question is can mutual exclusion be
implemented with only atomic loads and stores?
Do you really need one of these heavyweight operations
to implement mutual exclusion?
What if I don't use our read-modify-write?
Is that possible to do it?
And in fact, the answer is yes.
So Dekker and Dijksra show that it can as long as the
computer system is sequentially consistent.
So as long as you have sequential consistency, you in
fact, can implement a mutual exclusion with
read-modify-write.
We're actually not going to use either the Dekker or
Dijksra algorithms, although you can read about those in
the literature.
We're going to look at what is probably the simplest such
algorithm that's been devised to date, which
is devised by Peterson.
And I'm going to illustrate it with these two smileys.
That's a she.
And that's a he.
And they want to operate on widget x.
And she wants to frob it.
And he wants to borf it.
And we want to preserve the property that we are not
frobbing and borfing at the same time.
So how do we do that?
Well, here's the code.
So we're going to set up some things before we start he and
she operating.
So we're going to have our widget x.
That's our protected variable.
And we're going to have a Boolean set initially to false
that says whether she wants to frob it.
So we don't want to make them frob it unless they
want to frob it.
And we don't want him to borf it unless he wants to borf it.
And we're going to have an extra auxiliary variable,
which is whose turn it is.
So they're going to sort of do a take turn.
But that only is going to come into account if the other one
doesn't have a conflict.
If they don't have a conflict, then one of them is going to
be able to go.
So here's what she basically does.
She initially sets that she wants to
operate on the widget.
And then, what she does is she sets the turn to be his.
And then, while he wants it, and the turn is his, she's
going to just spin.
Notice that you're not frobbing it in while loop.
The body of the while loop is empty.
So this is a spinning solution.
So while he wants it, and it's his turn, you're just going to
sit there, continually testing the variables he wants and
turn equals his until one of them ends up being false.
So if he doesn't want it, or it's not his turn, then she
gets to frob it.
And when she's done, she sets she wants to false.
And he does a similar thing.
He sets it to true, says it's her turn.
And then, while she wants it, and the turn is hers, just
sits there waiting, continually re-executing this
until finally, one of these turns out to be false.
And then he borfs it.
And he sets it to false.
And then, they're doing both of these things sort of in a
loop, periodically coming back and executing it.
And what you want to do is you don't want to make it so that
it's forced.
That it's one turn, then the other because maybe he never
wants to borf it.
And then, she would be stuck not being able to frob it,
even though he doesn't want.
So if you think about this-- let's think about why this
always is going to give you mutual exclusion.
So basically, what's happening here is if he wants it--
by the way, these things are not easy to reason about.
And usually, as much as I can talk and talk in class, what
you really need to do is go home, and sit down with this
kind of thing.
And study it for 10 minutes.
And then, you'll understand what the subtleties are as
what's going on.
But basically, what we're doing is we're making it so
that it's not going to be the case that both she's setting
it and she wants it.
And the turn is his.
And then, if there's a race where he wants it also, then
that's going to preclude both of them from going into it at
the same time.
And then whichever one sets the turn, one of those is
going to occur first.
And one is going to occur second.
And whoever ends up coming second, the other
one gets to go ahead.
So it's very subtle how that is actually working to make
sure that each one is gating the other to allow them to go.
But the way to reason about this is to reason about it is
what are the possible interleavings?
And the important interleavings here as you can
see are what happens when setting these things.
And once they're set, what happens in
testing these things?
And especially because when you go around the loop and so
forth, you have to imagine that an arbitrarily long
amount of time is gone.
So for example, between the time that you check that the
turn is his, he may have already gone around this loop.
And so you have to worry about--
even though, it may look like one instruction from this
processors point of view for correctness purpose, you have
to imagine that an arbitrary amount of computation could
occur between any two instructions.
So any question about this code?
People see how it preserves mutual exclusion and how you
use sequential consistency to reason about it by asking what
are the possible interleavings?
Questions?
Yeah.
AUDIENCE: So, I don't know if I got it right.
So basically, sets the [UNINTELLIGIBLE]
to give him a chance before she goes to loop.
So basically, she waits there until he has been able to go?
That's why on the [UNINTELLIGIBLE].
PROFESSOR: So on this third line--
AUDIENCE: Both of them.
Either before actually frobbing or borfing.
And before that while you always give the turn to the
other to give them a chance to go.
PROFESSOR: Yeah.
So there are two things you want to show.
One is that they can't both be stalled on
the while loop there.
And that can't happen because the turn can't be
simultaneously his and hers.
So you know that they're not both going to deadlock in
trying to do this by sitting there waiting for the other
because of this.
And now, the question is well, how do you know that one can't
get through while the other is also going through?
And for that, you have to look and say, oh well, if you go
through, then you know that it is either he doesn't want it,
or it's not his turn.
And in which case, if he doesn't want it,
it's not his turn.
If he does change it to that he wants it, then in fact,
it's going to be your turn.
Question?
AUDIENCE: This only works for exactly two threads, right?
PROFESSOR: This only works for exactly two threads.
This does not work for three, but there are extensions of
this sort of thing to end threads in an
arbitrary large number.
However, the data structures to implement this kind of
mutual exclusion for end threads end up taking space
proportional to n.
And so one of the advantages of the built in atomics--
the compare-and-swap, or the atomic exchange, or whatever--
is they work for an arbitrary number of threads with only a
bounded amount of resource.
You don't require extra data structures and so forth.
So that's why they put those things in the architecture
because in the architecture you can build things that will
solve this problem much more simply than
this sort of thing.
However, there are going to be lessons here that you may want
to use in your programming, depending
on what you're doing.
So now, it turns out that no modern day processor
implements sequential consistency.
There have been machines that were built-- actually quite
good machines--
that implemented sequential consistency.
But today, nobody implements it.
They all implement some form of what's called relaxed
consistency, where the hardware may reorder
instructions.
And so you have things executing
not in program order.
And the compilers may reorder instructions as well.
So both the hardware and the software are going in there.
So let's take a look at that.
So here's the program order for one of the things.
We move 1 into a, and then move the value of b into EBX
to do a load.
Here's the program order.
Most modern hardware will switch these and execute it in
this order.
Why do you suppose?
Even if you write it this way, the instruction level
parallelism within the processor will, in fact,
execute it in the opposite order most of the time.
Yeah?
AUDIENCE: Because loading takes longer.
PROFESSOR: Yeah.
Because loading takes longer.
Loading is going to take latency.
I can't complete the load from the processor's point of view
until I get an answer.
So if I load, and I wait for it to go out to the memory
system and back into the processor,
and then I do a store--
well, as soon as I've done the store, I can move on.
Even if the store takes a while to get out
to the memory system.
But if I do it in the opposite order.
I do the store first, and then I do the load, I've ended up
wasting essentially one cycle, the cycle to do the store,
when I could have been overlapping that with the time
it took to do the load.
So people follow that?
So if I execute the load first, I can go right on to
execute the store.
I can issue the load, go right on to execute the store
without having to wait for the load to complete if I have a
multi-issue CPU in the processor core.
So you get higher instruction level parallelism.
Now when is it safe for the hardware compiler to perform
this reordering?
Can it always switch instructions like this to put
loads before stores?
When would this be a bad idea to put a load before a store?
Yeah?
AUDIENCE: You're loading the variable you just stored.
PROFESSOR: Yeah, if you're loading the
variable you just stored.
Suppose you say store into x and then load from x.
That's different from if I load from x, and then
I store into x.
So if you're going to the same location, then that's not a
safe thing to do.
So basically, in this case, if a is not equal to b, then this
is safe to do.
But if a equals b, this is not safe to do.
Because it's going to give you a different answer.
However, it turns out that there's another time when this
is not safe to do.
So this would have been the end of the story if we were
running on one processor.
The other time that it's not safe to do it is--
if it's safe, the other assumption is that there's no
concurrency.
If there is concurrency, you can run into trouble as well.
And the reason is because another processor may be
changing the value that you're planning to read.
And so if you read things out of order, you may violate
sequential consistency.
Let me show you what's going on in the hardware so you have
an appreciation of what the issue is here.
So here's 30,000 feet of hardware reordering.
So the processor is going to issue memory operations to the
memory system.
And results of memory operations are
going to come back.
But they really only have to come back when?
If they're loads.
If they're stores, they don't have to come back.
So the processor, in fact, can issue stores faster than the
network can handle them.
And the memory system can handle them.
So the processors are generally very fast.
The memory systems are relatively slow.
But the processor is not generally issuing a store on
every cycle.
It may do store, it may do some additions, it may do
another store, et cetera.
So rather than waiting for the memory system to do every
store, they create a store buffer.
And the memory system pulls things out of the store buffer
as fast as it can.
And the processor shoves stuff into the store buffer up to
the point that the store buffer gets full, in which
case it would have to stall.
But for most many codes, it never has to stall because
there is a sufficient frequency of other operations
going on that you don't have to wait.
So when a store occurs, it doesn't occur immediately on
the store buffer.
Now along comes a load operation.
And the load operation, if it's to a different address,
you want to have that take priority because the processor
can be waiting.
It's next instructions may be waiting on the result.
So you want that to go as fast as possible.
They have a passing lane here where the fast cars or the
important cars, the ambulances, et cetera, in this
case loads, can scoot by all the other things in traffic
and get to the memory system first.
But as we said, we don't want to do that if the last thing
that I stored was to the same address.
So in fact, there is content addressable memory here, which
matches the address that is being loaded with everything
in the store buffer.
And if it does match, it gets satisfied immediately by the
store buffer.
And only does it make it out to the network if it's not in
the store buffer.
But what you can see here is that this mechanism, which
works great on one processor, violates sequential
consistency because I may have operations going to two
different memory locations, where the order, in fact,
matters to me.
So let's see how that works out.
So first of all, let me tell you what the memory can-- so a
load can bypass a store to different address.
First of all, any questions about this mechanism?
So this accounts for a whole bunch of understanding of what
happens in concurrency in systems.
This one understanding of store buffers.
It's absolutely crucial.
And I have talked, by the way, with lots of experts who don't
understand this.
That this is what's going on for why we don't have
sequential consistency in our computers.
It's because they made the decision to allow this
optimization, even though it doesn't preserve sequential
consistency.
There were machines in the past that did support
sequential consistency.
And what they did was they used speculation to allow the
processor to assume that it was sequentially consistent.
And if that turned out to be wrong, they were able to roll
back the processor's state to the point before
the access was done.
In fact, the processor is already doing that for
branches, where it makes branch predictions and
executes down a line.
But it's wrong, it has to flush the
pipeline and so forth.
Why they don't do the same thing for hardware is an
interesting--
for loads of stores-- is an interesting question.
Because at some level there's no reason
they couldn't do this.
Instead, it's sort of been a thing where the software
people say, yeah we can handle it.
And the hardware people say, OK.
You're willing to handle it.
We won't worry about it then.
When in fact, it just makes life complicated for everybody
that you don't have sequential consistency.
AUDIENCE: [INAUDIBLE]
you have to do speculation across both [INAUDIBLE].
PROFESSOR: Well here, you only have to do speculation over
what actually is coming out of your memory system.
And if it doesn't match, you could roll back.
The issue, in part, is how many machine states are you
ready to roll back to.
Loads come more frequently than branches.
That's one thing.
So no doubt, there are good reasons for why
they're doing it.
Nevertheless, definitely loss of sequential consistency
becomes a headache for a lot of people in doing a
concurrent program.
We had a question here?
Yes, Sara?
AUDIENCE: So this does not preserve sequential
consistency?
But as long as there's only one processor, it should have
the same effect, right?
PROFESSOR: But sequential consistency for one processor
is easy because all you do is execute them--
AUDIENCE: Yeah, I'm just saying--
PROFESSOR: It should have the same effect, exactly.
So on one processor, this works perfectly well.
If there's no concurrency, this is going to give you the
same behavior.
And yet, you've now got this optimization that loads can
bypass stores.
And therefore, you can do a store and a load and be able
to overlap their execution.
So this definitely wins for serial execution.
Yep, good.
Any other questions about this mechanism?
You could reason about it on the quiz.
That kind of thing, right?
Yeah, OK?
So here's the x86 memory consistency model.
For many years, Intel was unwilling to say what their
memory consistency model was for fear that people would
then rely on it.
And then, they would be forced into it.
But recently, they've started being more explicit about it.
And this is the large part of it.
I haven't put up all the things because there are a
whole bunch of instructions, such as locking instructions
and so forth, for which for some of them, it's more
complicated.
But this is the basics.
So loads are not reordered with loads.
So if you add a load to one location, a load to another
location, they always execute in the same order.
Stores are not reordered with stores.
If you have store and then a subsequent store, those two
stores always go in that order.
Stores are not reordered with prior loads.
So if you do a store after a load--
if you do a load and then a store, they're going to go in
that order.
However, a load--
and this is what we just talked about--
may be reordered with a prior store to a different location
but not with a prior store to the same location.
So that's exactly what we just talked about on
the previous slide.
Then, loads and stores are not reordered with lock
instructions.
So a certain set of instructions are called lock
instructions.
And they include all the atomic updates, the exchanges,
comparisons-and-swaps, and a variety of other atomic
operations that the hardware supports.
The stores to the same location always respect a
global order.
Everybody sees the store to a location in
exactly the same order.
And the lock instructions respect a global total order.
So that everybody sees that this thread, or processor, got
a lock before that one.
You don't have two different processors disagreeing on what
the order was that somebody acquired a lock or whatever.
And then, memory ordering preserves transitive
visibility, which is sort of like
saying it obeys causality.
In other words, if after doing a, if you had some effect, and
then you did b, it should look like to other people like a
and then b happened.
Like there's a causality going on.
But that's not sequential consistency, mainly
because of four here.
So what's the impact of reordering?
So here, we have our example from the beginning for the
memory bottle, where I'm storing a 1 into a and then
loading whatever is in b.
And similarly, over here the opposite.
So what happens if I'm allowed to do reordering?
What can happen to these two instructions?
Yeah.
They can execute in the opposite order.
Similarly, these two guys can execute in the opposite order.
So they can actually execute in this order where we do the
load and then the stores.
So it executes as if this were the order.
Did I do this right?
Executes as if this were the order.
So I could do 1, 2, 3, 4.
So if then, I do the ordering 2, 4, 1, 3.
AUDIENCE: [INAUDIBLE]
PROFESSOR: I got the screwed up, I think.
Didn't I?
AUDIENCE: [INAUDIBLE]
PROFESSOR: Because I should be swapping these guys, right?
AUDIENCE: Swapped the wrong [INAUDIBLE].
PROFESSOR: Ugh.
OK.
So if I did this one 2, 1, 4, 3.
So ignore this thing.
Suppose I do the order 2.
So basically, I load b.
Then, I load a.
Then, I store a.
And then, I store b.
What's the result value that are in EAX and EBX?
You get 00.
Remember 00 wasn't the legal value from sequential
consistency.
But in this case, the Intel architecture and many other
architectures out there will give you the wrong value for
the execution of these instructions.
Any question about that?
So it doesn't preserve sequential consistency.
So that's kind of scary in some way because you got to
reason about this.
Let's see what happens in Peterson's algorithm if you
don't have sequential consistency.
So here we go.
We have the code where she wants is true,
turn is his, et cetera.
How is this going to fail?
What could happen here?
Where will the bug arise?
What's going to happen?
What's the reordering that might happen?
AUDIENCE: On the while you do loads, right?
[INAUDIBLE] the he_wants and turn is.
PROFESSOR: Sorry?
AUDIENCE: On the while statement,
you do a load, right?
Because [INAUDIBLE].
PROFESSOR: Right.
He_wants is a load.
AUDIENCE: And so that will get reordered.
PROFESSOR: Where could that be reordered to?
That could be reordered all the way to the top.
Similarly, this one can be reordered all
the way to the top.
So the loads could be ordered all the way to the top.
And now, what's going to happen is you're going to set
that she_wants is true but get a value of he_wants
that might be old.
And so they won't see each other's values.
And so then, both threads can now enter the critical section
simultaneously.
Yeah, Reid?
AUDIENCE: If you swap the order of the loads, does the
[INAUDIBLE]?
PROFESSOR: If you swap the order of the loads--
AUDIENCE: If you swap-- put the turn equals his on the
left, [INAUDIBLE] on the right.
Because according to--
PROFESSOR: Put the turn equals his over here?
AUDIENCE: Because the he_wants can't cross the load.
PROFESSOR: Yeah, but that's not what you want to do.
AUDIENCE: Then you can't [INAUDIBLE].
PROFESSOR: The whole idea here is that when you're saying you
want to do something, you give the other one a turn so that
whoever ends up winning the race allows just one of them
to go through.
Yeah?
AUDIENCE: I think the point is that if you put turn equals
his and he_wants--
PROFESSOR: You're saying this stuff here.
AUDIENCE: Swap those two [UNINTELLIGIBLE] turn equals
his will not be reordered before the store that--
PROFESSOR: You might be right.
Let me think about that.
AUDIENCE: You both reorder the same [? word. ?]
AUDIENCE: But you just stored turn, right?
PROFESSOR: Yeah.
So if do turn equals his--
I see what you're saying.
Do this turn equals his.
I was looking at this turn equals his.
AUDIENCE: You mean turn equals equals his.
AUDIENCE: So the Boolean expression [INAUDIBLE].
PROFESSOR: Yeah.
OK, I hadn't thought about that.
Let me just think about that a second.
So if we do the turn equals his--
AUDIENCE: [INAUDIBLE]
and you won't reorder those two [INAUDIBLE]?
PROFESSOR: Then the--
Yeah.
You got to be--
I have to think about that.
I don't know about you folks, but I find this stuff really
hard to think about.
And so do most people, I think.
This is one of these things where I don't think I can do
without sitting down for 10 minutes and
thinking about it deeply.
But it's an interesting thought that if you did it the
other direction that maybe there would be
a requirement there.
I'm skeptical that that is true because to my knowledge
to do the mutual exclusion, you pretty much have to do
what I'm going to talk about next.
But it would be interesting if is true.
Because you also have to worry about this guy getting
reordered with respect to this one.
AUDIENCE: The loads can't be reordered with
respect to each other.
PROFESSOR: So he_wants and turn equals his.
Yeah.
So the loads won't be reordered.
Yeah.
So that looks OK.
And then, you're saying and then therefore, it can't go
forward because this one won't get reordered with that one.
You might be right.
That'd be cute.
So I have to update the slides for next year if that's true.
So one way out of this quandary is to use what's
called a memory fence or memory barrier.
And it's a hardware action that enforces an ordering
constraint between the instructions before
and after the fence.
So a memory fence says don't allow the processor to reorder
these things.
So why would you not want to do a memory fence?
Then we'll talk about why you do it.
Yeah?
AUDIENCE: To force a hardware slowdown?
PROFESSOR: Yeah.
You're forcing the hardware slowdown.
You're also forcing compiler because the compiler has to
respect that, too.
You're not letting the compiler do optimizations
across the fence.
So generally, fences slow things down.
In addition, it turns out that they have
some significant overhead.
So you can issue a memory fence explicitly as an
instruction.
So the mfence instruction sets a memory fence.
There's also, it turns out, on x86 an lfence and an sfence,
which allow loads to go over but not stores and
stores but not loads.
And this one is basically both.
From the point of view of what we're using it for, we're only
going to worry about the fences.
They're done by the explicit one.
But it also turns out all the locking instructions
automatically put a fence in.
One of the humorous things in recent memory is major
manufacturers for whom the lock instruction was actually
faster than doing a memory fence, which is kind of weird
because a lock instruction does a memory fence.
So how do you think that sort of thing comes about?
So when you looked at performance it would be like--
for this particular machine I'm thinking about--
it was 30 cycles to do a lock instruction.
And it was on the order of 50 cycles to do a memory fence.
And so if you want to do a memory fence,
what should you do?
AUDIENCE: Do a lock.
PROFESSOR: Do a lock instruction
instead to get the effect.
But why do you suppose that came up in the hardware?
Why is it that one instruction would be--
It's a social reason why this sort of thing happens.
So I don't know for sure.
But I know enough about engineering to understand how
these things come about.
So here's what goes on.
They do studies of traces of programs.
And how often do you think lock instructions occur?
And how often do you think fence instructions occur?
Turns out lock instructions occur all the time, whereas
fences, they don't occur so often because usually it's
somebody who really knows what they're doing who's using a
memory fence.
So then, they say to the engineering team, we're going
to make our code go faster.
And lock instructions are going really fast.
So they put a top engineer on making lock
instructions go fast.
They put a second-rate engineer on making memory
fence operations go fast because they're not used as
often, without sort of recognizing that, gee, what
you do for one is the same problem.
You can do the same thing for the other.
So it ends up you'll see things in architecture that
are really quite humorous like that, where things are sort of
like, wait a minute, how come this is slower when well, it
probably has to do with the engineering team that built
the system.
And actually now I'm aware of two architectures where they
did the same kind of thing by different manufacturers.
Where they got these memory fences.
It should be at least as fast because the one is doing--
anyway.
Interesting story there.
Now, you can actually access a memory fence using a built in
function called sync synchronize.
And in fact, there whole set of atomics--
I've put the information here for where you can go and look
at the atomic operations that include memory fences and so
forth to using in the compiler.
It turns out when I was trying to get this going last night,
I couldn't get it to work.
And it turns out that's because our compiler had a bug
where this instruction was compiling to nothing.
There's a compiler bug.
And so I messed around for far too much time and then finally
sent out a help message to the T.A.s.
And then, John figured out that there was a bug.
And he's patched all the compilers so that you
guys all have it.
But anyway, it was like, how come this isn't working?
AUDIENCE: What compiler are we using?
PROFESSOR: This was GCC.
I was trying 4 1, and I tried 4 3.
And so the one that we're using in class for the most
part, is 4 3.
So anyway, John put the patch in.
So now, when you use these
instructions, they're all there.
And then, the last thing is that the typical cost of a
memory fence operation is comparable to that of an L2
cache access.
So memory fences tend to be on our machine--
and I haven't actually measured in our machine.
I meant to do that, and I didn't get around to it.
It's probably on the order of 10, or 15 cycles, or
something, which is not bad.
If it's less than 20, it's pretty good.
So here's Peterson's algorithm with memory fences.
You just simply sticky in the memory fence there to prevent
the reordering.
And it's interesting if there's a way that we can play
the game with the instruction stream to do the same thing
because that would make this code go, generally, a lot
faster in terms of overhead.
And so using memory fences, you can restore consistency.
Now, memory fences are like data races.
If you don't have them, how do you know that
you don't have them.
It's very difficult to regression test for them,
which is one reason I think there was a
bug in the GCC compiler.
How do you know that some piece of code is failing
because most of the time it will work correctly.
It's just occasionally, they'll be some reordering,
and timing, and race condition that causes
it not to work out.
In this case, you both have to have the race and the
reordering happening at the same time for Peterson's
algorithm, for example.
So compilers can be very difficult
for things like this.
Really, the way to do it, which is what I was doing, was
do an objdump and search for is fence in there.
And in this case, it wasn't in there.
AUDIENCE: And also compiler's self-analyzers, by itself.
And that's this instruction that basically can take code.
PROFESSOR: Right.
It's not doing anything.
Right.
So it says, oop, get out of it.
Yep.
Good.
So any questions about consistency.
So what turns out to be most of the time when you're
designing things where you want to synchronize through
memory directly, rather than using locks or what have you.
The methodology that I found works pretty well.
Work it out for sequential consistency, and then figure
out where you have to put the fences in.
And that's a pretty good methodology
for working out where--
here's sequential consistency.
Now, what reorderings do I need to ensure in order to
make sure that it works properly.
And that can be error prone.
So once again, big skull and cross bones on whether you
actually try this in practice.
It really better make a difference.
Now, the fact that you can synchronize directly through
memory has led to a lot of protocols that are called
lock-free protocols, which have some advantages, even
though, in particular, because they don't use locks.
And so I want to illustrate some of those because you'll
see these in certain places.
So recall the summing problem from last time.
So here we have an array.
And what we're going to do is run through all the elements
in the array, computing something on every element,
and adding into result.
And we wanted to parallelize that.
So we parallelize that with a Cilk 4.
And what was the problem when we parallelize this?
We get a race.
So there's the race.
We get a race on result because we've got two parallel
instructions both trying to update
results at the same time.
So we can solve that with a lock.
And I showed you last time that we could
solve this for lock.
By declaring a mutex, and then locking before we update the
results, and then unlock.
And of course, as we argued yesterday, that could cause
severe contention.
Now, contention can be an issue.
But if it turns out that the compute here, which I've moved
outside the lock notice.
I've put it into temp and then added temp in so I can lock
for the shortest possible time.
If this compute is sufficiently large, there may
be contention.
But it may not be a significant contention in your
execution because the update here could be very, very short
compared with the time it takes to compute.
So for example, if computing on array i cost you more than
say order n time, then the fact that you have contention
there isn't going to matter, generally, because the total
amount of time that you're going to be locking is just
small compared to the total execution time.
Still in a multiprogram setting, there may be other
problems that you can get into, even when you have this
and even if you think that contention
is going to be minimal.
So can anybody think of what the issues might be?
Why could this be problematic even if contention
is not a big issue?
And the hint here is it's in a multiprogram setting.
So what happens in a multiprogram setting.
Yeah.
AUDIENCE: [INAUDIBLE]
PROFESSOR: Because the resolve is--
AUDIENCE: [INAUDIBLE PHRASE]
PROFESSOR: It actually doesn't have to do with resolve here.
It has to do with locking explicitly.
It's a problem with locking in a multiprogrammed environment.
What happens in a multiprogrammed environment?
What do I mean by multiprogrammed environment?
AUDIENCE: [INAUDIBLE]
PROFESSOR: Have multiple jobs running, right?
And what happens to the processor when there are
multiple jobs running?
AUDIENCE: [INAUDIBLE]
PROFESSOR: Contact switches.
So now, what can go wrong here?
What can be really bad here?
Yeah.
AUDIENCE: You aquire the lock and then the
contacts switch out.
PROFESSOR: Yeah.
You acquire the lock.
And then, the operating system contact switches you out.
And so what happens?
You hold the lock while some other job is running.
And what are those guys doing.
They go and spin and wait on the lock.
Now, this is a good time where you'd rather not have a
spinning lock.
You'd rather have a yielding lock.
But even so, suddenly you're talking about something that's
operating at the level of 100 times a second, 10
milliseconds, versus something that is operating on a
nanosecond level.
So you're talking six orders of magnitude of performance
difference if you end up getting switched out while you
hold a lock.
That's the issue.
What happens.
And then, if that happens, all the other loop
iterations must wait.
AUDIENCE: [INAUDIBLE]
in the large program here [UNINTELLIGIBLE PHRASE].
I don't have a mic.
If one [UNINTELLIGIBLE] crashes or one
[UNINTELLIGIBLE PHRASE]
the lock [UNINTELLIGIBLE PHRASE].
AUDIENCE: Can you specify whether those are yielding
locks or spinning locks?
PROFESSOR: Usually, the mutex type will tell you.
I'm just using a simple name of mutex.
I probably should have been using the ones that--
we were using one called Cilk mutex.
And I probably should've used that here rather than just
simple mutex.
AUDIENCE: Are they yielding?
PROFESSOR: There's a good question.
I used to know the answer to this.
I believe that those spin for a while, are competitive.
They spin for a while and then yield.
But I'm not sure.
They may just spin.
They don't just automatically yield.
They're either competitive, or they'll spin and yield.
I believe they spin and yield.
And I believe there's actually a switch where
you can tell it--
if you're doing timing measurements--
make it so that it purely spins so that you can get
better benchmark results.
AUDIENCE: So my question is does the colonel have power to
switch out a spinning lock or not?
PROFESSOR: Yeah.
Well, the colonel, the scheduler, can come in at any
moment and say, whip you're out.
You're out.
That's it.
And wherever it is, it interrupts it at
that moment in time.
So one solution to this problem is to
use a lock-free method.
And one of the common ways of doing that is with what's
called compare-and-swap instruction.
So this is what's called a locking instruction, meaning
it's one of these ones that goes out to L2, in terms of
timing and so forth.
And what it does is it does the following thing.
It has an address of a location.
And it's got the old value that was stored in the
location and a new value.
And it says if the value that is there is the old value,
well, then stick the new value in there.
And then return essentially true.
Otherwise return false.
So it's basically saying what you tend to do is you first
look to see what's the value.
You then update the value.
And then you say, if it hasn't changed, stick it back in and
return true.
If it has changed, return false.
So it's only swaps the value if it is true.
There's actually two versions.
One which says bool and one which says val.
And if you do the bool version, it returns a flag.
If you do the val version, it actually returns the value
that was in there.
So it's more like a compare-and-swap.
The main thing about this is this code essentially executes
atomically with the single instruction, which is called--
The instruction is cmpxchg.
Is it up there?
Oh, there it is.
Yeah.
So the cmpxchg instruction on x86.
So when you compile this, you should find on your assembly
output that instruction somewhere unless the compiler
figures out a better way to optimize that.
But generally, you should find that.
Also, one of the things about this is it works on values
that are sort of integer type values.
But it doesn't work on floating point numbers, in
particular.
So you can't compare-and-swap a value, which is a floating
point value.
You can only do it with energy type values.
So let's take a look at how we can use the compare-and-swap
for the summing problem.
So what we do is we have the same sort of code.
And now, what I'm going to do is compute my temporary value.
And then, what I'll do is I'll read the value
of result into old.
I'll then update my new value for what I think I want the
result to be the result plus the thing that I computed.
And now, what I do is I attempt to compare-and-swap as
long as the old value is what I read it to be.
Swap in the new value.
If the old value turns out to be different from what is
currently in the result location,
then it returns false.
And I redo this again.
Then, I have to redo the whole loop again.
So this is a do-while loop.
Do-while is like a while loop, except you do the body first.
And then you test the condition.
So if this fails, I go back.
I then get a new value for the result and so forth.
So let me show you how that works.
Let's see.
So first, I'll show you how this works.
Actually, I'll show it on a more interesting example how
this works.
So what happens if I get swapped out in the middle of a
loop iteration?
All I do is when I do the compare-and-swap it fails.
So no other instructions can wait.
They can all march ahead and do the thing they need to do.
And then, the one that got swapped out, eh.
It gets some old value.
It discovers that and has to re-execute the loop.
So is that fine?
So what this means is that the amount work that's going on,
however, could in fact, be greater, depending upon how
much contention there is.
If there's a lot of contention, you could end up
having these guys fighting and not re
executing a lot of code.
But that's really not much worse than them spinning is
what it comes down to.
Any questions?
Let's do a more interesting example.
Here's a lock-free stack.
So what we're going to do is we're going to have a node,
which has a next pointer and some data.
All we really care about is the next pointer.
And we have a stack, which has basically a head pointer.
So we have a linked list here.
We want to basically be able to insert things at the front
and take things out of the front.
So here's a lock-free push.
So remember, this could be concurrent.
So these guys want to operate on it at a time.
We saw last time how in doing very simple updates on link
structures, you could get yourself into a mess if you
didn't properly synchronize when we did the insertion in
the hash table.
So here's my push [? up ?] code.
Well let's walk through it.
It says, basically, here's my node that I want to insert.
It says, first of all, make node.next point to the head.
So we basically have it pointing to 77.
So then what we say is OK.
Let's compare-and-swap to make the head point to the node but
only if the value of the head has not changed.
It's still the value of the node.next.
And if so, it does the swap.
Question?
AUDIENCE: You say compare-and-swap.
But you compare it to what?
PROFESSOR: PROFESSOR: In this case it's comparing to the--
so this is basically the location that you're doing the
compare-and-swap on, the old value that you expect to see
in that location, and the new value.
So here, what it says--
when we're at this point here--
we're saying before you do the compare-and-swap, we're
saying, I only want you to set that pointer to go to here if
this value is still pointing to there.
So only move this here if this value is still 77.
In other words, if somebody else came in--
well, I'll do an example in a second that shows what happens
when we have concurrency, and one of them might fail.
But if it is true, then it basically sets it.
And now I'm home free.
So let's take a look at what happens when we have
contention.
So I have two guys.
So 33 says, OK I'll come in.
Let me set my next pointer to the head.
But then comes 81.
And it says, OK.
Let me try to set my pointer to also be 77 because I look
at what the head is, and that's where
it's supposed to go.
So now, what happens is we do the
compare-and-swap operation.
And they both are going to try to do it.
And one of them is going to, essentially, do it first
because the hardware preserves that the compare-and-swaps,
their locking operations, they will happen in
some definite order.
So in this case, 81 got in there and did its
compare-and-swap first.
When it looked, 77 was still a value that it said.
So it allowed that pointer to be changed.
But now what happens when 33 tries.
33 tries to do the compare-and-swap.
And the compare-and-swap fails because it's saying, I want to
swap 33 in as long as the value of head is the pointer
to 70, the node was 77.
The value is no longer the pointer to the node of 77.
It's now the pointer to the value of the node with 81.
So the compare-and-swap fails.
People follow that?
And so what does 33 have to do?
It's got to start again.
So it goes back around the loop, and now it sets it to
81, which is now the head.
And now, it can compare-and-swap in the value.
And they both get in there perfectly well.
Question?
AUDIENCE: What if there's [INAUDIBLE]?
What if two nodes have--
PROFESSOR: Well, notice here, it's not looking at the value
of the data.
Nowhere does data appear here.
It's actually looking at the address of
this chunk of memory.
There is a similar problem, which I will
raise in just a moment.
There is still a problem.
Yeah, question?
AUDIENCE: So I'm confused about the interface.
So you give it the address of where you want to
compare the value of.
And you're giving it what you're pointing at and--
PROFESSOR: And here's the value that I expect to be
stored in this location.
The value I expect to be in there is node dot next.
So if I go back a couple things.
Here.
Here, the guy says, the value I expect to be
there is in this case.
the address of this chunk of memory here.
He expects the address of the node containing 77 is going to
be in this location.
It's not.
What's in this location is the address of
this chunk of memory.
But you're saying, if it's equal to this, then you can go
ahead and do the swap.
Otherwise you're going to fail.
And the swap consists of now sticking this value into--
conditionally sticking it in there.
So you either do it or you don't do it.
So let's now do a pop.
So pop you can also do with things.
So here, I'm going to want to extract an element.
And what I'm going to do is create a current value that I
want to make point to the element that gets eliminated.
So what I do is I say, well, the element that I want is
that guy there.
And now, what I want to do is make the head jump around and
point to 94.
So what I do is I say, well, as long as the--
and I want to do that unless I get down to the fact that I
have an empty list.
So basically, I say, if the head still has
the value of current--
so they're pointing to the same place--
then, I want to move in current arrow next.
And then I'm done.
Otherwise, I want to set current to head, reset it, and
go back to the beginning and try to pop again.
And I'm going to keep doing that until I get my pop to
succeed or until current points to nil.
If it ended up at the end, then I don't want to keep
popping if the list ended up being empty.
So basically, it sets that one to jump over.
And now, once it's done that, I can go, and I can clean up,
I can get rid of this pointer, et cetera.
But nobody else who's coming in to use this link list, can
see 15 now because I'm the only one with a pointer to it.
So people understand that?
So where's the bug?
Turns out this has a but after all that work.
Each of these individually does what it's supposed to do.
But here's the bug.
And it's a famous problem because you see it all the
time when people are synchronizing through memory
with lock-free algorithms.
It's called the ABA problem.
So here's the problem.
And it's similar to what some people were
concerned about earlier.
So here's the ABA problem.
Thread 1 begins to pop 15.
So imagine that what it does is it sets its current there,
and then it reads the value here, and starts to set the
head here using the compare-and-swap.
But it doesn't complete the compare-and-swap yet.
The compare-and-swap hasn't executed.
It's simply gotten this value, and it's
about to swap it here.
So then, thread 2 comes along.
And it says, oh, I want to pop something as well.
So it comes in.
And it turns out it's faster, and manages to pop 15 off, and
set up its pointers.
Now, what would normally happen here is if this
completed, what would happen?
The compare-and-swap instruction would discover
that this pointer is no longer the pointer to the head.
And so it would fail.
We'd be all hunky dory.
No problem.
But what could actually happen here?
Thread 2 keeps going on.
It says, oh, let me pop 94.
So it does the same thing.
So thread 1 is still stalled here, not having completed its
compare-and-swap.
It swaps 94.
Then, thread 2 goes on and says, oh,
let's put 15 back on.
So it puts 15 back on because after all, it had 15.
So now, what happens here?
Thread 1 now looks, and it now completes, and does its
compare-and-swap, it resumes, splicing out 15, which it
thinks it has.
But it doesn't realize that other stuff has gone on.
And now, we've got a mess.
So this is the ABA problem because what happened was we
were checking to see whether the value was still the same
value, the same chunk of memory.
It got popped off.
But it got popped back on.
But now, it could be in any configuration.
We don't know what it is.
And now, the code is thinking, that oh, nothing happened.
But in fact, something happened.
So it's ABA because basically, you've got 15 there.
It goes away.
Then, 15 comes back.
Question?
AUDIENCE: Can you compare two things and then swap because
that would solve this, right?
PROFESSOR: That's called a double compare-and-swap.
And we'll talk about it in a second.
So the classic way to solve this problem is to use
versioning.
So what you do is you pack a version number with each
pointer in the same atomically updatable word.
So that when 15 comes back, you've got the pointer.
But you also have a version on that pointer so that the value
has to be the same as the version you
had and not the value.
What you do is you increment the version number every time
the pointer is changed.
So you just do an increment.
But you do the compare-and-swap on the
version number and the pointer at the same time.
Now, it turns out that some architectures actually have
what's called a double compare-and-swap, which will
do compare-and-swap on two distinct locations.
And that simplifies things even more because it means you
don't have to pack and make sure that
things fit in one word.
You can keep versioning elsewhere.
And there are a whole bunch of other places where you can, in
fact, optimize and get even tighter code than you could if
you have to pack.
So that's generally the way you solve this.
And, of course, you can see this gets--
as I say, this week has been skull and cross bones lecture.
It's appropriate it comes right after Halloween because
really, you do not want to play these games
unless you have to.
But you should know about them because you will find times
where you need this, or you need to understand somebody's
code that they've written in a lock-free way.
Because remember lock-free has the nice property that hey,
the operating system swaps something out, it just keeps
running nice and jolly if it's correct.
So the other issue is that version numbers may need to be
very large.
So if you have a version number, how many bits to you
assign to that version number.
Well, 64 bits, that's no problem.
You never run out of 64 bits.
2 to the 64th is a very, very, very big number.
And you'll never run out of 2 to the 64th.
We did that calculation at the beginning of the term.
How big did we say it was?
It's pretty big, right?
It's like this big.
Or is it this big?
My two-year-old is this big.
So anyway, it's pretty big.
So is it bigger than--
no, it's not bigger than the number of particles in the
universe, right?
That's 10 to the 80th, which is much bigger
than 2 to the 64th.
But it's still a big number.
I think it's like more than there are atoms in the earth
or something.
It's still pretty big.
You never get through it if you calculate it.
I think we calculated it and it was
hundreds of years or whatever.
Anyway, it's a long time.
Many, many, many years at the very fastest, updating with
biggest supercomputers, and the most
processors, et cetera.
Never run out of 64 bits.
32 bits.
Four billion.
Maybe you run out.
Maybe you don't.
So that's one of the issues.
You have to say, well, how often do I have to do that.
Really, you only have to worry about this.
You can wraparound.
But you've got to make sure that then you never have a
situation where something could be swapped out for long
enough that it would come back and bite you because you're
coming around and then eating your tail.
And you've got to make sure you wouldn't have things
overlap and get a [? thing. ?]
So that might be a risk you're willing to take.
You can do an analysis and say, what are the odds my
system crashes from this reason or
from a different reason?
That can be reasonable engineering trade-off.
So there's an alternative to compare-and-swap.
One is the double compare-and-swap.
Another one is some machines have what's called a
load-linked, store conditional instruction.
What those are actually is a pair of instructions.
One is load-linked.
When you load-linked, it basically says, let's set a
bit, essentially, in that word.
And if that word ever changes when you do store conditional,
it will fail.
So even if some other processor changes it to the
exact same value, it's keeping track of whether anybody else
wrote it using the memory consistency mechanism.
The MSI type protocol that we talked about.
It's using that kind of mechanism to make sure that if
it changes.
And so this is actually much more reliable as a mechanism.
x86 does not have load-linked, store conditional.
I'm not sure why.
I don't know if there's a patent on it or
what's going on.
But they don't have it.
Final topic is reducers.
So once again, recall the summing problem.
In Cilk++, they have a mechanism called reducer
hyperobjects, which lets you do an end run around some of
these synchronization problems.
And the basic idea behind it is we actually could code this
fairly easily as we talked about last time by just doing
divide and conquer on the array.
We add up the first half of the elements, add up the
second half of the elements, when they
return, add them together.
But the problem is that coding that is a pain to do.
So the hyper object mechanism sort of does that
automatically for you.
What you can do is declare result to be an integer, which
is going to have the operation add performed on it.
And what happens then is you can just go ahead and add the
values up like this.
And basically, what it does is essentially adds things
locally and will combine them on an as needed basis.
So you don't actually have to do any synchronization at all.
In the end, you have to get the result
by doing a get value.
So let me show you a little bit more what's going on in
this situation.
So the first thing here is we're saying result is a
summing reducer over int.
The updates are resolved automatically without races or
contention because they're basically doing it by keeping
local values and copying them.
And then, at the end, you can get the underlying value.
So the way this works is that when you declare the variable,
you're declaring it as a reducer over some associative
operation, such as addition.
So it only works cleanly if your operation is associative.
And there are a lot of associative operations.
Addition, multiplication, logical, and, list
concatenation.
I can concatenate two lists.
So what does associative mean?
I think I have a slide on this in a minute.
It means a times b times c.
I can parenthesize it any way I want and
get the same answer.
Associative, right?
It's not associative like
associative memory or whatever.
So now, the individual strands in the computation can update
x as if it were an ordinary non-local variable.
But in fact, it's maintained as a set of different copies
called views.
The Cilk++ runtime system coordinates the views and
combines them when appropriate.
And when only one view remains, now you can get the
actual value.
So for example, you may have a summing reducer where the
actual value at this point in time is 89.
But locally, each processor may only see a different value
whose sum is 89.
But locally, I could do something like increment this.
And this guy can independently increment his view and has the
effect that it increments whatever the total sum is.
And then, the runtime system manages to combine everything
at the end to make it be the value when there's no more
parallelism associated with that reducer.
So here's the conceptual behavior.
Imagine I have this code.
I set x equal to 0.
I then add 3.
I then increment.
I had 4, increments at 5.
Fa da da da da.
At the end, I get some value, which I
don't think I put down.
Another way I could do this is the following.
Let me do exactly the same here but with a local view
that I'll call x1.
For this set of operations, let me start a new view that I
start out with the identity for addition, which is 0 and
add those guys up.
And then, at the end, let me add x1 and x2.
It should give me the same answer if addition is
associative.
In particular, these now can operate in
parallel with no races.
So if you don't actually look at the intermediate values--
if all I'm doing is updating them, but I'm not actually
looking to see what the absolute value of the thing
is, I should get the same answer at the end.
The answer to the result is then determinant.
It's not deterministic because it's going to get done in a
different way with different memory state.
But it's determinant, meaning the output answer is going to
give you the same no matter how it executes, even if the
resulting computation is nondeterministic.
So this is a way of encapsulating, if you will,
nondeterminism.
And it worked because addition is associative.
It didn't matter which order I did it.
And once again, I could have broken it here instead of
there, and I still get the same answer.
It doesn't matter.
So the idea is as these things are work stealing around.
they're accumulating things locally but combining them in
a way that maintains the invariant that the final value
is going to be the sum.
So there's a lot of other related work where people do
reduction types of things, but they're all tied to specific
control or data structures.
And the neat thing about the Cilk++ version is that it is
not tied to anything.
You can name it anywhere.
You can write recursive programs.
You can update locally your reducer wherever you want, and
it figures out exactly how to combine them in order to get
your final answer.
So the algebraic framework for this is that we have a monoid,
which is a set, an operator, and an identity, where the
operator is an associative binary operator.
And the identity is, in fact, the identity.
So here are some examples.
Integers with plus and 0, the real numbers with times and 1,
true and false, Booleans with and, where true is the
identity, strings over some alphabet with concatenation,
where the empty string is the identity.
You can do MAX with minus infinity as the
operation, and so forth.
And you can come up with your own.
It's easy to come up with examples of monoids.
So what we do in Cilk++ is we represent a monoid over a set
t by a C++ class that inherits from this base class that's
predefined for you, which is parameterized using templates
with the types.
So the set that we're going to use is, in fact,
going to be a type.
And the member function reduced--
this monoid has to have a member function reduced that
implements the binary operator times.
And it also has an identity member function.
So we set up the algebraic framework.
So here's, for example, how I could define a sum monoid.
I inherit from the base with int, for example, here.
And I define my reduced function.
And it actually turns out to be important, you always do
the right one into the left.
Otherwise, you won't have it be associative.
And then, you have an identity, which gives you in
this case a new element, which is 0.
And so you can now define the reducer as so.
You just say Cilk reducer, the sum monoid
you've defined and x.
And now, the local view of x can be accessed as x open
close parenthesis.
Now, in the example I showed you, you didn't need to do the
open close parenthesis.
And the way you get rid of those open close parenthesis
is you define a wrapper class.
So it's generally inconvenient to replace every access with x
over brown.
That's one issue.
The other thing is accesses aren't safe.
Nothing prevents a programmer from writing x times equals 2,
even though the reducer was defined over plus.
And that will screw up the logic of this code if
somewhere he's multiplying when, in fact, it's only
supposed to be combined with addition.
So the way you solve that is with a wrapper class.
You can do a wrapper class that will protect all of the
operations inside and export things that you can just refer
to the variable.
And it will actually call that.
For most of what you're doing, you probably don't need to
write a wrapper class.
You'll do fine just operating with the extra parentheses.
In addition, there's a whole bunch of
commonly use reducers.
Lists, appends, max, min, adds, an output stream, and
some strings, and also you can roll your own using things.
One issue with addition is that, in fact,
this doesn't preserve--
for floating point addition-- does not
preserve the same answer.
And the reason is because floating point numbers are not
associative.
If I had a to b and add that to c, I can get something
different because of round off error from adding a to the
result of b and c.
So generally, floating point operations don't give you--
they'll give you something that is close enough for most
things, but it's not actually associative.
So you will get different answers.
A quick example.
I'm sorry to run over a little bit here.
I hope people have a couple minutes.
Here's a real world example.
A company had a mechanical assembly represented a tree of
assemblies down to individual parts.
A pickup truck has all these parts and all of these extra
subparts all the way down to some geometric description of
what the part is.
And what they want to do is the so-called collision
detection problem, which has nothing to do
with colliding autos.
What they're doing is saying, find collisions between the
assembly and a target object.
And that object might be something like a half space
because they're computing a cutaway.
Tell me all the things that fall within this.
Or maybe, here's an engine compartment, and does the
engine fit in with it?
So here's a code that does that.
Basically, it does a recursive walk, where it looks to see
whether it's an internal node or a leaf.
If it's a leaf, it says, oh, let me check to see whether
the target collides with a particular
element of the tree.
And if so, add that object to the end of a list.
So this is the standard a C++ library for putting something
on the end of the list.
If it's an internal node, then go through all of the children
recursively.
And walk the children recursively.
So basically, you're going to look through the whole tree.
Does it intersect this particular object, x?
So how do we parallelize this?
We can parallelize the recursion.
We turn the 4 loop here into a Cilk 4.
So it goes through all the children at the same time.
They all can do their comparisons
completely the same.
Oops, but we have a bug.
What's the bug?
AUDIENCE: Is it push back?
PROFESSOR: Yeah.
The push back here.
We have a race here because they're all trying to push on
to this output list at the same time.
So we could resolve it with a lock or whatever.
But it turns out it's much better to resolve it with a--
so we could do this, right?
But now, you've got lock contention.
And also, the list ends up getting produced
in a jumbled order.
So it turns out if you use a reducer, you declare this to
be a reducer with list append.
And what happens then is turns out list concatenation is
associative.
If I concatenate a to b, and then concatenate c, that's the
same as concatenating a to the concatenation of b and c.
And I can concatenate lists in constant time by keeping a
pointer to the head and tail of each list.
So if you do that, and that turns out to be one of the
built in functions, then, in fact, this code operates
perfectly well with no contention and so forth.
And in fact, produces the output in the same order as
the original C++.
It runs fast.
And there's a little description of how it works.
The actual protocol is kind of tricky.
And we'll put the paper--
let's make sure we get this paper up on the web.
I think it was there from last year.
So we should be able to find it.
If you're interested in how the details work.
Here's the important thing to know from a
programmer point of view.
So typically, the cost--
it turns out the reduce operations you're only calling
when there's actually a steal.
It's actually a return from a steal.
But since stealing occurs relatively infrequently the
load balance, the number of times you actually do one of
these reduce operations is small.
The most of the cost is actually accessing the reducer
to do the updates.
And it's never worse than a hash table lookup the way it's
implemented.
If the reducer is accessed several times within a region
of code, the compiler can optimize the lookups using
common subexpression elimination.
And in the common case, then, what happens is it basically
has an access cost equal to one additional level of
indirection, which is typically an L1 cache hit.
So the overhead of actually updating one of these things
is really just like an extra L1 cache hit for most of these
things, for most of the time.
If you have the case that you're accessing a reducer
several times within the same block of code.
Otherwise, at the very worst, you have to actually do a hash
table lookup.
And that tends to be a little bit more like a function call
overhead just in terms of order of magnitude.
Sorry for running over.