The following content is provided under a Creative
Commons license.
Your support will help MIT OpenCourseWare continue to
offer high quality educational resources for free.
To make a donation or view additional materials from
hundreds of MIT courses, visit MIT OpenCourseWare@mit.edu.
PROFESSOR: Any issues with the project so far?
Have you gotten your repositories set up and the
compiler works and all this?
So I'm glad to see a lot of people are
getting a early start.
In fact, if you haven't, a lot of other people in the class
are getting an early start so you're behind already.
So go there, get the project started, because these
performance projects can run for long time.
And in fact, last year a lot of people asking, when is my
project done?
Because a lot of times when you have a project there's a
certain set of--
we ask you to implement.
Once that's implemented, the project is over.
But when is a performance project done?
And the only answer I L when the deadline comes because you
can always shave 10% here, you can get a better data
structure, you can keep like kind of
tweaking for a long time.
And a lot of you will do.
In fact, to give you a warning, last year, the
performance difference between the best project won, versus
the worst one, even the worst one got about 50% better than
the code we gave.
But the best one got 1,000x better than the worst one
because they figured out right data structures, they changed
algorithms, they did some precomputation, they did a lot
of cool stuff, and so there's a lot to get if you do this
project right.
So just don't be happy if you just got 10%, 50%, and say,
yeah, I got some performance.
Think carefully.
There's a lot of room in this project for you to
gain, so work hard.
I'm not saying this yields 1,000x.
I don't know.
I haven't exactly seen all the things you can do, but there
could be a huge thing you can win.
So think through not just small, low hanging fruit, but
there could be interesting things you can do that can
have impact.
So today what I am going to do is talk a little bit about
basics of performance engineering.
Kind of things you guys would probably
know, rules to follow.
So to start that if you look at performance engineering, it
can fit into a couple of classes.
One is you want to get a maximum out of the compiler
you have, the processor you have, the system you have, you
want them running efficiently.
So if you look at the matrix multiple examples we did this
in the first class it's saying, OK, we have a system,
we have this very well known algorithm, very long piece of
code, and we want to get the best out of that thing, how do
you go about doing that?
That's mainly the focus there.
Then when you we looked at the bithacking stuff that
Professor Leiserson did last time, it's mainly about really
changing the algorithms.
We basically said, OK, look, if we're thinking about this
as bits there might be fundamental different ways to
approach this problem.
By doing that we can actually get really good performance.
OK, we'll be doing more of that.
Today we are going to focus on this two middle criteria.
What we are going to do is we are going to stay within the
spirit of the program that was given.
Even though we are going to change the program a little
bit, we are not going to fundamentally change what the
program does.
So the same computations would be done basically on the same
day but what we will do is we will change certain aspects
how it's being done to gain some performance advantages.
So the nice thing about this one is you don't have to
really understand the algorithm, you don't have to
understand lot of what's going on in, what's implementation.
This is more--
you can do this as a mechanism sub out.
The unfortunate thing is there's not theory in these
type of engineering.
There's not something we can prove, here's the
optimal you can do.
[UNINTELLIGIBLE] nice there.
So normally performance programming basically knows
that you need to understand everything that's in modern
performance.
That means all the way from high level algorithms, every
layer in the systems, the compiler, the hardware, the
disks, all those things can impact performance.
So it's normally when in performance to understand
everything that [UNINTELLIGIBLE].
This is very different from a lot of what you
guys are used to.
It is staying within one layer.
You say, OK, this is my layer, I don't care
what's on any of it.
I don't know care much about--
I'm going to work on that.
And the thing that normally comes and bite you is the
other layers that you just take for granted.
In performance it's basically going through end to end.
That's the normal thing.
And normally, if you are an experienced performance
engineer, you know when the problems show up.
And you can kind of have a basic understanding, here are
the kind of places where they have performance problems.
And so a good performance this is where experience comes in.
So instead of saying looking at the code and say I have no
idea what's going on, you can say, let me think about this,
this is where normally something goes wrong.
And then you have to have the skill to zoom in and identify
the problem fast.
And most of performance engineering is basically a
good dose of common sense.
So have kind of a good person who can
do really good debugging.
Is you have kind of a built in algorithm how do you go about
doing something.
And a lot of times you know your strengths, kind of
mistakes you make, you look at those things first, and the
same way performance engineering there has to be
kind of a back of your head algorithm.
And hopefully at the end of this class you're going to
develop that.
And in this lecture what I'm trying to do is come up with a
set of rules, that kind of patterns that occur regularly,
mistakes lot of people make, and that can have a
substantial performance impact, or performance gain if
you do that.
If you have done the design patterns lectures in 005, yes,
you do, you will see some similar ideas.
What you're doing is kind of say, OK, here's a bunch of
patterns in there, what we can do.
So what I'm doing is I am following some very
interesting set of rules that was developed by this guy
named John Bentley.
He wrote a book called Programming Pearls, I guess.
It's a very old book, last published in 1990.
It talks about a lot of the small, interesting things you
can do to get really good performance.
It's out of print.
This is probably one reason to probably
go to library probably.
Libraries have this copy but you can't get it on Amazon or
online so that's unfortunate.
But what I'm going to do is I'm going to kind of read
through it and some of these rules, modified to suit the
current program and practices.
So he looked at it and said there are basically two kinds
of things you can do, we can modify data,
you can modify code.
And then went down more into details, if you modify data
you can do three things.
You can pay one against other, you can pay by space and gain
time, you can pay from time and gain space, and there's
sometimes, if you're lucky, you have a win-win situation
when you can get both space and time advantage.
So let's go through a couple of those things.
So if you [UNINTELLIGIBLE]
by space for time, there are four things.
I'm going to go through all this four data structure
augmentation, storing precomputed results, caching,
and lazy evaluation.
So what's data structure augmentation?
So here what you want to do is you have a data structure that
you keep the data.
You add some extra information to your data structures.
That can mean the common operations run much quicker.
OK?
And when this is good, because what do you want to do is this
adding this information has a clear benefit.
Calculating additional information has to
be cheap and easy.
So you don't want a situation where we have this
information, you've spent all this time calculating this
additional information and you're not using it that much,
or the use doesn't give you that much advantage.
So you have amortize the cost in here.
And then also keeping that information current can't be
too difficult, because sometimes you can add
information into a data structure that makes it very
hard to make sure that that information is kept up to
date, and then you're spending more time and you're
[UNINTELLIGIBLE]
that.
So a couple of quick examples in here.
Something like if you have a single link list, if you are
doing a lot of deletion it's a very expensive proposition
because you might had to walk through the entire list to
delete an element.
But if you are in a doubly linked list, a division
operation is basically order one instead of
[UNINTELLIGIBLE].
So suddenly you realize, I'm doing a lot of deletions,
suddenly I can say I'm adding this additional information to
my link list, I'm maintaining point as to both directions,
make this one operation run much faster.
This is a very classic example of adding additional
information.
Another one is what we call reference counting.
So assume I have objects somewhere and I want to figure
out who points to me.
In terms of garbage collection you want to see if anybody
points to me.
The normal way to do that is look for every other thing
saying is anybody pointing to that person, had to walk
through every possible object out there to find if anybody's
pointing to this object.
That's what normal garbage garbage collectors do.
But there's something easier, what you
call reference counting.
That means if each object keep a count on the amount of
pointing coming to itself then to ask that question, is this
really true?
And you look and say, OK, what's my count?
If there's count to zero nobody's pointing to me.
But of course every time you point to that object to update
the counter, every time you point away or take away an
object that points you have to subtract the counters, you had
to keep that information additional.
But by doing that discussion about asking whether anybody
points to you becomes a very trivial question.
So here's a case you add a little bit more information
and that makes some questions, some things you want to know
trivial and very easy to do, and you have to figure out
where the [UNINTELLIGIBLE]
is.
So here's one interesting thing.
Go a little bit further, one thing you can do is
precompute, result and storage.
What you are doing is you want some calc computation and
instead of every time you want that in recalculating it, say,
OK, I'm going to compute them early and I'm going to store
it somewhere, and then every time I want then I
just do a look up.
So when is this going to be useful?
So this is--
the rest of my slides are going to have this kind of
form, I'm going to give you a problem, tell you when it's
going to be useful, and use some examples.
First of all, the function you are calculating has to be
somewhat expensive, otherwise if it is just a simple single
operation look up is probably too slow to compile.
And the function has to be very heavily used, because if
you are calling it many, many times it's worth storing it.
Another very important thing is the argument
space has to be small.
You might be asking for calling the function millions
and millions of time, but every time you call you give a
different set of arguments.
If the arguments are very large it can be many, many
things here, and then it's not easy to store everything.
And it might take more time to find all the solutions for all
the arguments.
So if the arguments have to be small.
And of, course, a couple additional things, results
only depends on the arguments.
So assume the function results actually look time of day to
the computation.
OK, that doesn't work, because then every time you call the
function, you get different result.
That doesn't work that much.
And functions shouldn't have side effects.
What that means is if you call the function, if it is
modifying some global variable, global state.
OK.
Then if you keep the call, if you don't call the function
that might not happen, so you want to make sure that this
function is something can not call and give the results and
the state of the system won't change.
And another interesting, nice thing to have is function
determinants, that means every time you call the function you
get the same four arguments, you get the same results.
So this is a long list.
Some of the things I will go, so for an example, OK, so
instead of me getting there getting all the things I want,
you [INAUDIBLE]
when I give a prescription like the thing, what kind of
things can I do in this way that might be helpful?
Can anybody come up with an interesting example?
[UNINTELLIGIBLE]
would precompute and save.
AUDIENCE: Anytime you do dynamic programming.
PROFESSOR: Anytime you dynamic program.
This is dynamic programming, it's a very good way, later we
probably go get a little bit into that.
Where you precompute and save results.
OK, that's very sophisticated answer.
Good.
Any other things you can think of?
That's almost a meta answer that's in that class of things
you can do.
Any problem there that you have encountered that you'd
rather precompute and save that would be a big, big win.
AUDIENCE: Operations with large primes that [INAUDIBLE].
PROFESSOR: Operations with?
AUDIENCE: Large primes.
PROFESSOR: Large primes, operations in
large primes, yes.
If you are doing that, especially if this fits into
this argument space small apart.
If the large primes are, any large primes then you might
not call it multiple times.
So here I want to give you kind of a template that what
happens here.
So what you want to do is at initialization time I get this
function initialized, what I do is I precompute for all
arguments what the results are stored at, and then every time
you go apply the function I just basically, instead of
calling the function, I just look at this table.
OK.
So then would this be a bad idea?
AUDIENCE: Because you know this early ahead of time you
know that you're going to be calling the function for all
of your arguments.
PROFESSOR: So if the argument space is very large or if you
don't call the function that many times, or as I said, it
might only call for very small number of arguments,
[UNINTELLIGIBLE]
this might not be a good idea.
OK?
So we will go.
As we improve, you'll see some other things that actually
address that.
So here's an interesting way of doing a use case in here.
So what I have here is Pascal's Triangle.
How many people know Pascal's Triangle?
OK, good, so I don't have to explain that.
So here is a simple computation
for Pascal's Triangle.
So what I'm doing is in order to calculate this value I am
calling these two values and adding that.
OK?
But the problem is that this will call- so it kind of call
exponentially more and more doing that.
So you'd say exponential rhythm.
However, what I can do in Pascal's Triangle is I can
basically compute it at the beginning by storing the
previous role, or previous set of values in and then previous
role, and every time I'm calling I just basically have
to look up the two elements in the previous role.
Trying to get my mouse, OK.
Only have to look at the two elements on the previous role.
I added up, I will never have to compute beyond that.
So, in fact, by just doing this one I have taken
exponential algorithm to basically lead a path through
the data, basically.
This actually become a square, because if you have some
depth, I had to do i squared--
become a square, pass through the data.
If I'm looking for n, I don't know how to do
exponential look up.
Everybody saw this?
So this is a nice way, simple way I'm just doing a simple
storing this data, however in here I had to calculate, I
need to know how much maximum of that, and calculate and I
just look up.
And here the [UNINTELLIGIBLE]
if you calculate a little bit extra, who cares, I'm OK.
And another interesting thing is Fibonnaci.
And again, here, if you do keeping the previous value, in
here, you're actually changing from exponential algorithm, to
in here basically a linear calculation.
OK?
So this is almost an algorithmic change here.
I kind of lied when I said it's just a
data structure change.
But just a simple data structure change made an
algorithm difference.
So the other interesting thing you can do is caching.
So what that means is if you have a heavily used function
you can keep some number of previous basically results
from calling that function.
This has a lot of caveats so let's go
through this carefully.
Again the function has to be expensive, otherwise it's not
worth doing all that work.
Functions have to be heavily used, otherwise you're
calculating and if you don't reuse it it's not worth doing
all the worth.
Here the arguments space can be large, OK, because you are
not calculating everything, you can have a large argument
space, and that's OK.
And then there's another thing called temporal locality.
Anybody knows what temporal locality means?
OK what.
AUDIENCE: The same arguments are called [UNINTELLIGIBLE].
PROFESSOR: Yeah, so a lot of times we realize that if you
go into the caching and stuff like that.
We really get into that in hardware when you talk about
hardware lecture.
So what happens is yes, we have observed that if I have a
set of arguments and if I calculate most of the times I
will have the same arguments very quickly time-wise.
So if I calculate something, then for a short time I might
reuse those arguments again and again, so that's called
temporal locality.
So caching works when there's a very good temporal locality.
And then there's two other things we want to do because
we won't have hash function that means because a lot of
arguments from that we only get one value to look up where
I store my cache, have to have a hash function, but to
calculate it by argument.
Sometimes if the arguments are too complicated you might not
be able to calculate a good hash function and the hash
function has to be good.
What do you mean by a good hash function?
AUDIENCE: [INAUDIBLE].
PROFESSOR: Yeah, no [UNINTELLIGIBLE].
That means they tend to be a good distribution.
So given most of the arguments it shouldn't be ending up as
the same value again, so you should have good distribution
when you call with different arguments.
And the final results only depend on the arguments that's
again otherwise you have a problem and the function
[UNINTELLIGIBLE]
if your [UNINTELLIGIBLE] this important.
And there's another thing called coherence.
So what that means is I keep some
precalculated values in here.
And sometime this if results are only dependent on the
arguments you don't have that much of an issue.
But sometimes you might in the situation that the results are
dependent on the arguments and some other things like the
time of the day or whatever.
And if you want full coherence, what you want to
make sure is--
many of you know that the results might be varied.
I need to understand that, I need to
invalidate all the cache.
And suddenly the data or the user changed so that thing I
memorized only works for User A, now went to User B, the
results are wrong and I had to get rid of all the results.
Or there are some cases where you could say, yeah, little
bit of things I can tolerate, like for example, a really far
fetched example is something like if you're looking at a
web page on a search.
OK, you don't have a good guarantee that all the web
pages in the world are in that search.
If you're a little bit late getting a new web
page, then you're OK.
So you can do tolerate sometimes, a lot of times, if
you can tolerate these kind of things you can have huge
performance impact.
You don't have to be exact.
So some of these projects we look we look at these kind of
ideas, of tolerate a little bit of a slack and getting a
good performance out of that.
So here's the kind of code, I put a lot of code in my slides
because not I won't explain everything but since these
online if you want to figure out what the right template is
you can look at it.
I have no idea why this is moving on me.
OK, OK, here we go.
So what you first have to say is every time you cache a
value you have to know, I had to know old arguments and
results, I had to store that, and then I'm keeping this many
precomputed values in here.
And so every time I call the function with arguments first
I have to see I had to get the get the has with these
arguments because that might be the place, this buffer disk
just might be the place I have stored a previously computed
[UNINTELLIGIBLE].
Then you say, OK look, is my [UNINTELLIGIBLE] stored, has
the same argument.
If that means I already have that precomputed, at that
point I just return it, I have no issue in there.
Otherwise you're to call the function because I haven't
stored it, and then I will go about storing the value
because I calculated I need a storage because the next time
I call these arguments I want to get the same results, and
then I return the result.
OK.
Is this kind of clear, how this caching works?
OK, what's a good place for caching versus precomputing?
Any idea what can make a good scenario where actually
caching can have a big impact?
You have heard a lot of things that have the word cache
associated with it, and you can say, OK, what that means.
AUDIENCE: Web browsing.
PROFESSOR: Web browsing OK, you web browse actually that's
a cache, because there's a good probability that if you
look at a page and you might be refreshing that page or you
might be looking at that page, so a lot of times what you do
is you keep a copy locally, and the next time you ask for
that and if you have a local copy you can give
that local copy back.
And then of course there's some issues about how to make
the local companies stay, the coherence issues and stuff
like that it can get complicated but that's a
situation where actually you can use caching.
So finally, in this category, we are going to talk about
lazy evaluation.
So lazy evaluation means basically a lot of times you
might ask for value but you might not really need it at
that point.
Just keep the computation to the side without doing it and
then do the computations only when you really, really need
the result.
So basically this is useful, a lot of times you might try to
compute a lot of values but only a certain of that
actually is used.
And at that point you just basically further that
calculation and only use it when it's actually done.
The nice part is when can be done in a function call, and
also results can be calculated kind of incrementally.
And also, the arguments or what's needed to calculate the
results can be nicely packaged up so it can be
recalculated later.
So here what you do here is in the precomputing is something
like precomputing but you don't precompute early, at the
beginning nothing is precomputed.
When you only ask to apply a function then if there's no
precomputed value you'll recompute, otherwise you just
return the value.
So if you compare this one Pascal's Triangle beforehand I
just computed, there's a bunch of values in here and kept it.
OK assuming OK up to PDMAX I will have
the results in there.
But you might not do that.
PDMAX might be very large and you might be looking only for
small Pascal Triangle.
So instead of doing that, what you can do is you can do this
way, you can keep the array for results, assume everything
is normally in [UNINTELLIGIBLE]
global array.
and then what happens is you look at the results if the
results is greater than zero we are what you're looking at,
that means, ah-ha.
I have calculated it.
I have a new value, I just iterated it.
I'm happy.
Otherwise I will call Pascal by recalculating values
because I don't have the current results.
And once I get it I will store it.
Once I get the value I will basically, I will calculate
the Pascal, I will store it and I will return the value.
So what that means is this value of the Pascal yx will be
only calculated once fully, and afterward every other time
you lose that you actually use the previous one.
This works very nice, this works both very nice about
precomputation but you don't pay that extra overhead.
So you are doing the minimum amount of computation.
You can never exceed the computation of the normal
computation.
So if you only ask for Pascal very small one, that's exactly
the work we [UNINTELLIGIBLE] do.
But it will also really use this kind of exponential blow
up that happens in the normal computation.
So this is kind of the you can have the cake and eat it too
in this work.
OK good.
So now let's switch from space for time, for time for space.
OK?
So here, one thing you can do is if you have a huge amount
of data what you can do is you can store the data in some
kind of reduced form and then what you can do is then you
can get a big compression of data basically and then as you
need the data you can do some more computation to get the
data you want.
So what you're doing is you are really reducing the space
you need but now you are paying by time to get access
to that data.
So this is why [UNINTELLIGIBLE]
storage is at a premium.
If you look at systems before 1980s everybody had to do it
because storage was a premium and all the memory and disk
space was so complicated everything got
compacted in there.
Now, most your laptop and desktops, you don't care, you
have enough space.
However, if you got things like embedded devices and
things like that, this still becomes a issue.
Or if you have a very large data set it becomes an issue.
And what you can do is by compressing, drastically
reduce the data size.
And you can do it in different ways, sometimes you
can do it in batch.
That means you keep the data compressed, expand it all, and
process, hopefully you have room for that.
Or sometimes if you don't have room you do it by streams,
that means you're looking at a small amount of data expanded,
work at it and then either throw it or again compress it
and put it back in.
And hardest thing is if you actually do random access.
That means you'll randomly go to some data, expand and
understand that sometimes there's some complications.
So there are a bunch of packing methods and some
simplest things are use a small data size.
So right now you're dealing with 64 bit words, you can say
wait a minute, why does this data only need 8 bits?
Why do you keep storing everything 64-bit if my data
maximum can be only 0 to 256.
I can use 8 bits.
Or I can look at that and I can use small data size,
that's something.
And a lot of people realize a lot of data we are storing are
just zeros.
There are many different data structures, you have huge data
structures and most of it's zeroes.
And eliminating zeroes in many instances can you get a long
way in many, many, many data storage.
And something like LZ77 eliminates repetition.
So a lot of times what you find in there you have
repetitive patterns you can eliminate that and then, of
course, you can go into a lot more complicated.
heavy-weight compression.
So to just give you a feel for what compression is like I
found this cute animation so I will show it to you.
So here is the input stream, but it has this corrector LA,
and then it has some notion about how much a repetition it
has, corrector O and 0 3.
So what happens if it's just a character?
It will get taken in, so these two correctors go.
Now these two forces from the point I point in here go to
inwards, and repeat it four times.
What that means is basically in here it will get repeated
four times because that 2 4.
So that means instead of having these four correctors
in there, what it had was it had two things, and then all
get repeated three times and then it'd get shifted in here.
So instead of having ten correctors before, we had 2,
3, 4, 5, 6, 7.
Seven, out of seven correctors you compress you stored
information off ten correctors because of repetition.
OK, so this is a very simple scheme that people use to do
this LZ77 compression.
More complicated thing you can do a lot of times is build an
interpreter.
What that means is you have some complex data, instead of
storing the data you have a summary of the data stored.
OK.
And then what you can do is you can do this abstract
representation.
I mean you were already tested.
Like, for example, you [UNINTELLIGIBLE]
exactly doing that.
What you do is you start off storing all the beats of
machine instructions.
You store it in abstraction called add register names or
something like in smd.
What that means is you're building a very simple
abstraction which that would be more complex sometimes to
store than the actual data.
So in here, things like if you look at a java byte cord,
that's a nice interpreter.
So instead of storing all this complex instructions you
interpret, you stored some higher level set of
instructions, that byte code, that is much more compressed
and that you run you map it into the machine instructions.
There's a lot of advantages other than just compressing
storage, it's a nice abstraction, you can change it
at the highly level easily.
OK, so those are something that you can do, you give up
one thing for the other.
And if you're lucky you can have the cake and eat it too.
Which is you get both space and time at the same time.
So we looked at it in last lecture sometime.
This is bithack.
Bithacks kind of gave you that because instead of having 64
bit word to represent you got 64 of them into one big
stream, if you're only representing zeroes and ones.
And by doing that you get a lot more compressed storage.
Also now you can operate that entire 64 in parallel.
We can generalize it a little bit, so instead of just doing
64 Boolean, sometimes you can do in a 64 bit word, you can
store two 32 bit words, or four 16 bit words,
so eight 8 bit words.
OK.
In [UNINTELLIGIBLE]
is called a Memex SSE extensions.
Normally it's called SIMDs, a single instruction multiple
data, because what you do to that, all the bits, are the
same thing for everybody.
So if you're adding you're doing same add
for both of the data.
And because of that you basically going to get both
the compression.
Because now for 64 bit you are keeping two thing, or four
things, or eight things.
Also, now you're operating.
In a single operation you can get the same operation
happening to all the data.
So you're getting both storage and fast operations in here.
Of course if you only look at one you have to do a little
bit more work, you take it out of that data word, so if
you're just looking at [UNINTELLIGIBLE]
thing can be expensive.
So when is it viable?
You'll do the same operations all the data.
OK, if each day you do something different it's not
that great and items can be stored in contiguous memory,
so when you load you can say I'm getting nice in one word,
if I'm getting two 32 bit, so four 16 bit, so I can do that.
And you don't end up picking each operations and doing
individual things too much.
So if you are to pick each individual elements a lot,
then you have to actually now pick apart that word and that
can be multiple operations and expensive.
So here's a simple example.
This, I think, from after last bithacks class, this is a
little bit, you probably know, can come up with enough
examples here.
What I'm doing is I'm doing a Battleship board game.
So you represent the board, and the interesting thing in
this board is to know whether that location has something or
not, it's empty or full, that's all I need to know.
And so normally if I do this board game I can have two
boards in here.
But what I'm doing is having this overlap calculations,
saying of the two boards, how many things overlap.
I can just have each location represented by integer, but
then only the representing one value is not good enough.
One thing I can do is I can just put each row into one 64
bit word, and then instead of doing the two loops I can, in
one loop, I can just do the ending off each row, and I get
the result.
So after the the bithack lecture, this should be kind
of trivial.
OK, everybody get it?
Anybody have questions so far?
Are we all on the same page?
OK, good.
There's a lot of blank faces so I want to help, I'll start
asking more questions or something.
OK, so that's modifying data.
Now we're going to modifying code, where you will change
the programs to actually get performance and that can be
done in loops, logic rules, procedures, expression, and
parallelism rules.
When we get to parallelism rules we will actually do a
lot more on them in future lectures, but I just want to
address some basic stuff now.
Bunch of different loop rules.
I will go each of them individually.
So first of all, why loops?
Why do you think loops are so important?
OK, somebody who hasn't answered the question.
Why is loops so important?
Why are we like really focused on loops all of the time?
Anyone want to answer here, this side?
OK.
AUDIENCE: [INAUDIBLE] in terms of maintainability,
[INAUDIBLE]?
PROFESSOR: Yeah, so loops give you a nice, simple
abstractions it's easier to maintain instead of having
multiple computed times.
Yes, instead of having same thing repeated millions of
times for the same thing, if I write a loop, it's much more
concise representation.
That's why I think we do a lot of loops, because it makes
programs easier.
In fact, assume this word, loops didn't exist, OK?
If loops doesn't exist that means you're going at each
instruction get executed only one time.
If you have a 3 gigahertz machine that requires even if
you do one instruction per cycle in 32 bit instruction
you need 12 gigabytes of instruction per second, if you
only execute one instruction once.
That means if you have 100 gigabyte disk full of a
program you go through each instruction the entire 100
gigabytes in 8 seconds.
OK.
Of course this can never be done, that means you cannot
feed 100 gigabytes into the processor in 8 seconds.
That won't work because of the disk access time,
caches, and all those.
Even if it is doable, what that means is the entire
reason your Pentium is running at 3 gigahertz and actually
showing like it's running as 3 gigahertz is because you are
using instructions many, many times.
You have loops that run millions of times.
If you don't have loops you can use whatever the data
machine you build in '004 and that probably as fast you
could get because of this instruction thing.
So loops are critical, and loops are what makes our
programs run fast.
Repeated execution of same instruction again
and again and again.
OK.
So in the world, in that sense, if you think about
this, unless there are some instruction that's run
millions of times, you can never keep this ferocious
beast's appetite full because there will always be new
instructions.
So because of that looking at loops, if you want to get good
performance just looking at the most important loops, the
inner loops, and just doing only that we'll probably get
you most of way in many, many cases.
And basically I could even say 99 of the program execution
time is in 10% of the code, even, more than
that in many cases.
Because otherwise [UNINTELLIGIBLE]
back of the envelope calcuation you can do, you
realize how much code you need to feed the beast to keep it
active otherwise.
So first loop optimization, loop and
invariant code motion.
So normally in a loop you run millions of times, and if you
do the same thing millions of times and get the same results
you are just doing useless computation.
And so here the key thing is most of the time the compiler,
a good compiler will come help you.
So in this class you're not trying
to replace the compiler.
We, as humans, we want to the least amount of work.
The compilers [UNINTELLIGIBLE] dumb things sitting in there,
it can do most of the hard work.
The key thing is first trying to get the
compiler to do the work.
Only do things yourself if the compiler cannot do.
So most of the time compiler, if it can analyze the code,
and prove the results are the same in each iteration, it's
going to get rid of it, it will do that for you.
But there are cases it's not possible because sometimes the
computation might be too costly and also there might be
cases the computation is too cheap if you hoist it out, you
have to keep the results in some [UNINTELLIGIBLE]
and if you keep too many things in too many registers,
that cost might be too expensive.
So there might be case that you might not do, but most of
the time what happens because the
compiler can't do anything.
So here's a good example.
I'm doing, I don't know why I did that, but square root and
take exponential off that.
Compiler look at that as functions.
I have no idea what the function are most of the time.
The compiler can't tell what each function does because the
function call.
It can do any arbitrary thing as far as the compiler knows.
It says, OK, function call, I don't know what happened
inside so I'm just giving up and I'm just leaving it, but
you, on the other hand, knows that the square root function,
exponential function doesn't have any what you call the
side effects.
That means those functions doesn't go
changing anything else.
It just calculates [UNINTELLIGIBLE] the value,
you'll know this basically calculate the same value again
and again and you can just basically take it out, do it
twice, happy.
So lot of times make sure the compiler can do it.
If the compiler cannot do then you have to go in and
interfere at that point.
So here's another interesting thing the compilers cannot do.
A lot of times you're going through this array or data
structure looking for some value.
OK, once you've found the value, you will say OK, found
it, I return.
OK.
Normally what that means is when you're going to add it,
there's two things.
First of all you had to test whether you found the value
that you're trying to exit.
Second, you have to make sure that you don't overrun on the
data structure, you're not overrunning that.
you have to make sure you're not at the end of that because
that value might not be in that area.
If that's the case, you run to the end of that.
So pretend I don't find it.
So you do two tests.
One thing you can do is you start doing the two test at
the end of that array, how about you add the value you
are testing for?
So you know when you reach to the end you will always find
the value because you added there, so you
know it's there already.
So by doing that you can reuse the test by just looking for
the value because now when it comes to the end, you don't
have to do anymore tests because the value you're
looking is already there and you found that, you go out.
But you should be able to add that value to the end of that.
So what you do is this is your normal test that's what you're
doing is you're, if I can get my mouse, you're going through
this array, looking for a value.
A value I return otherwise I return minus 1 in here.
Now I do two tests.
I do this test to make sure that I'm not at
the end of the array.
I'm [UNINTELLIGIBLE] this one.
So, in fact, [UNINTELLIGIBLE]
loop I just can put the two tests next to each other so
this is what you're doing.
But instead of doing that one thing I can do is put that at
the end of the array, after all the data that I care
about, I can put the value I'm looking for and then I just
only do one test.
I check for that value.
So if you find the real value in the area, at
that point I'm done.
Or I ran out of the array, and after I ran out the real data
I just found the value because I, myself, put it there.
And so I have to find it, and the minute I find it, if I
realize that's the case I say, OK, that's the value I put and
I [UNINTELLIGIBLE].
Here I have done something which is not really good
programming practice.
Can anybody put your 005 hat and say, that's bad
programming?
AUDIENCE: You modified [INAUDIBLE].
PROFESSOR: I modified my input array.
I know.
That's not that great.
I mean if I did it really well, if I want to make sure
what I should have done is kept the old value of the n
and at the end, it back so it looks the
pristine thing I got.
I mean, if I was really careful I would have done
that, and then my invariant that my input array haven't
changed would hold nicely. because here I have mucked
with my input array.
So even though you are constantly doing performance
programming you should still have a good eye for doing good
correct programming and keeping nice programming
variants around than mucking it all over the place.
So if you wonder putting a new hat in here, you should still
have the 005 hat, at least partially.
What else can we do?
A lot of times what you find is we have these loops, and
loops have a couple of features.
First of all, you have to keep checking the loop condition,
that's expensive.
And if the loop body is very small, the compiler doesn't
have that much opportunity to do anything useful in the
body, just basically compile as is.
So one thing you want to do is if the loop bodies are too
small and the number of iterations is small, you want
to unroll the loop.
So in here, instead of doing this calculation again and
again, I want to actually do a computation unrolled, and this
gives you two things.
First of all I get rid of this disk, so I actually
[UNINTELLIGIBLE] in here.
And second, this computation can be done much faster, the
compiler can do this much faster than this one.
Do you see why?
Can anybody see why this computation would happen much
faster than this doing in the loop iteration?
AUDIENCE: Because you don't have to write the final result
of the variable until all the numbers [INAUDIBLE]?
PROFESSOR: So that's one thing.
Even the final results in the register what happens in here?
So that's a good assertion but that's not
exactly what's back there.
AUDIENCE: [INAUDIBLE].
PROFESSOR: OK, he got 90% of the way there.
First of all here I had increment i.
And I have it check [UNINTELLIGIBLE]
the loop condition.
That's another point in here.
AUDIENCE: [INAUDIBLE].
PROFESSOR: The one on right can be run in parallel.
In here I am doing sum equals--
some plus A I, so that means I have to calculate
[UNINTELLIGIBLE].
Once that is done and we get the results only then I can go
add the two, only then I can add day three.
So the last [UNINTELLIGIBLE]
using, you can show 16 instructions at every cycle.
OK what six things you can do in here?
Nothing.
Because you're just waiting.
You're twiddling your thumb for the first one to finish,
to do that.
In here, you can say, look, I don't have to add A1A to it.
I can just do these things in parallel.
The first cycle I can add is 0 to A of 1.
that's one instruction 2 3 2 4 5 3 6 7.
I can add all those things together as a tree.
I can do tree addition instead of one after another, that can
happen in one go.
And I can get a six editions done and then the next
instruction I add the [UNINTELLIGIBLE] and at the
later stages do not have too much to do but I could get a
lot of [UNINTELLIGIBLE].
So the compiler can actually [UNINTELLIGIBLE]
all of that.
So that works very nicely if I have small number of
iterations.
But if you have a large number of iterations, you can still
do some stuff.
So in here what you can do is it goes to one and you can
say, OK, I don't know.
I don't know how to unroll fully because I don't know
what n is, but I can unroll partially.
So here what I can do is say, look, I have been running it n
times, I will unroll my loop four times.
OK.
So I at least get enough of unrolling here.
That way you can choose.
That is, you unroll enough that the meshing can get busy.
It's not just crawling.
And then I haven't blown up my code like crazy in here.
And of course you had to clean up and then you have to know
how many full iterations.
So how do you choose how much to unroll?
What impacts my number of unrolls?
I unroll four times.
You can get greedy, and say, I'm going to unroll 8 times,
16, 32, 64, 128.
Because a lot of time these are not zero-sum some gains.
Some stuff impact some other way.
So what should I use normally for my amount of
iterations to unroll?
Anybody want to take a guess what helps?
AUDIENCE: It's probably the number [UNINTELLIGIBLE].
PROFESSOR: OK, so in one sense you can look and say number of
instructions I can execute in one cycle, that's a good thing
to keep, unroll enough to keep the process of
[UNINTELLIGIBLE] most of the time.
But sometimes we can say oh that's hard to figure that one
out, I don't know that because I'm at the compiler.
I'm at high level.
I don't know what happens at low level.
I'm going to unroll 1,000 times.
So why shouldn't I unroll a huge amount?
What happens if I unroll too much?
AUDIENCE: [INAUDIBLE].
PROFESSOR: You might--
AUDIENCE: You might to be [INAUDIBLE].
PROFESSOR: OK, define it more closely.
What do you mean?
AUDIENCE: Perhaps there's some [UNINTELLIGIBLE].
PROFESSOR: That's the one interesting thing.
Assume your number of iterations
are not always millions.
Sometimes you might [UNINTELLIGIBLE]
hundred times, or 200 times.
So if you unroll 500 times, you would never get to the
unroll loop.
You end up in this clean up loop.
And so clean up loop, of course, is not unrolled and so
you're back to your square one.
And so a lot of times if you unroll too much and if your
number of iterations doesn't fit so you end up in here,
that's not good.
Another thing is when you unroll loop here there's a
huge amount of instructions now.
That instructions might be too much, it might not fit in the
cache, it might just overwhelm the memory system.
So you have already now created a loop that doesn't
fit in the instruction cache and that's not good either.
So that will limit down roll, so don't get too greedy, just
sometimes these are the things you kind of play with in the
problems you're looking at and some of it is
architecture dependent.
Some of them are things like too much unroll where the
number of iterations not that high is problem dependent.
Another thing you can do there are many places if you have
two different very same looking loops we can put them
together into one loop and that gives [UNINTELLIGIBLE]
all the loop tests and stuff and also the array access.
At least done once, so that can give you
a benefit in here.
Here's interesting one.
Here you have this weird loop that the j loop run from I to
J in minus I, I to n minus I. How can I improve this one?
Can anybody tell me what might happen in here?
So I have outer loop I goes from 0 to N, inner loop goes
from I to N minus I. What happen?
Somebody's getting it.
OK.
AUDIENCE: [INAUDIBLE].
PROFESSOR: Yes, what happens is because what the loop is at
the beginning you're going from basically 0 to N, and
then the lower bound is going like this, upper bound is
going like this.
After N over 2, this is N, N over 2, there's
no iterations left.
OK, you're trading and trading and trading and after it
iterates, so you're running outer loop, N over 2 to end
with nothing running on the inner loop.
And so if you have this kind of funky loops you can just
look at it a little bit careful and say, wait a
minute, that's not useful.
So therefore I can just get rid of this new situations.
And then sort of free trading to nothing.
So that's loops rules.
Loops are very important.
also you can do a lot of interesting logic rules.
So first of all, can anybody shout out what each of these
things can be done?
Square root of x greater than 0.
Square of [UNINTELLIGIBLE]
zero, sorry.
x [UNINTELLIGIBLE].
Actually, not [UNINTELLIGIBLE]
sorry.
x not equal to zero.
Because x squared of x squared non-zero.
OK.
Here's something your compiler probably doesn't know about
the transformation.
We can do that.
How about the next one?
How can I get to input the next one?
Square root of base climb, looking at the distance
between x and Y and AB.
AUDIENCE: [INAUDIBLE].
PROFESSOR: Eliminate the square root.
It doesn't help anything you do except do too much
computation.
OK, next one.
Yeah, I can basically multiply and then do take a one
[UNINTELLIGIBLE]
final one.
AUDIENCE: [INAUDIBLE].
PROFESSOR: One.
OK, and then you should go and find that program who wrote
that, but believe me you'll find these kind of code
sitting somewhere because you didn't think through, you just
start writing and suddenly you have this kind of nice axiom.
So when do that this is something compiler can do, of
for these kind of patterns.
That was a stupid mistake but a lot of other things are
things that a compiler cannot do but you know more than the
compiler does.
OK.
Other interesting thing in here is if you're looking
something like [UNINTELLIGIBLE]
increasing function- OK, let me look at this carefully.
OK, this is very similar to what we talked before with the
[INAUDIBLE]
ending.
So one thing you can do is I can put the cut off at the end
and then I don't have to take check both the bounds and
basically [UNINTELLIGIBLE]
through something.
Basically this is the [UNINTELLIGIBLE]
check we did done a little bit differently.
OK.
And here's another interesting thing.
Reordering test.
So what can you do here?
What do I do here?
I'm looking at is, I am looking at two different a
circles, see whether they are [UNINTELLIGIBLE]
in here.
This is expensive test because I had to do something in a
square root and check whether it's less
than or equal to radius.
So here's what I had to do.
How can I make this faster?
What's the [UNINTELLIGIBLE] in here?
So assume I have this bunch of balls running around in my
graphics program.
I want to make sure that the balls haven't collided.
They are actually moving with no collision with other balls
and I'm doing this test.
What's a good [UNINTELLIGIBLE] in here?
AUDIENCE: [INAUDIBLE].
PROFESSOR: So he's getting there because most of the time
what happens is if you're looking at two balls collision
most times these balls are farther apart and if you just
found the square around the ball, testing these two
squares are colliding it's very cheap, OK?
And then if the squares are all happy, then and only then
you might want to check if actually all the
balls are all happening.
OK and by doing this square test you can do very expensive
operations in here.
You can basically do some very simple thing, in fact I have
even reduced it more by just doing first doing the x side
and then if x is [UNINTELLIGIBLE]
I don't even have to check for Y. And only if those two
checks fail that I realize, look, the square might be now
there's a good chance it's actually the color id.
And there's a really good chance most of the test, after
doing the first two things, will be done.
You'll never have to go through this very expensive
operations.
So here's something you have a little bit of intuition on
what's going on out there in the problem, and use that
intuition to look like more work, but in reality really
reduce the amount of work you had to do because the test is
very expensive.
OK, here what I have done, OK this is a little bit of a
contrived example, what I want to do is assume I have
basically a 4 bit word in here, OK.
And I am checking- this would be 8 I think, so I have 8 bit
word in here.
OK, I'm checking whether the 8 bit word is a palindrome.
So the way I check the word is a palindrome is first of all I
set up some bits in here, bit masks, OK?
And I have a bit mask for the two ends.
I said, OK, these two are same, then these two same,
these two same, these two same, I just kind of go down
the bit mask.
Did everybody see how this work?
And then in these two I kind of shift my bit mask to the
side, these two, basically shifting.
So I'm checking the most two bits are the same, the next
two bits are the same, next two bits are the same, going
on [UNINTELLIGIBLE].
OK, so every time I do that I have [UNINTELLIGIBLE]
this bit mask do this operation and stuff like that.
Assume I am checking whether my [UNINTELLIGIBLE]
correctors.
If I call this a lot of times what can I do?
Kind of out of the box thinking.
So the one thing is if I am only looking for correctors,
how many different correctors are there?
AUDIENCE: [INAUDIBLE].
PROFESSOR: Yeah, because all the correctors in everything I
look up, I have 8 bits, two to date.
What's two to date?
[UNINTELLIGIBLE].
OK, so you have 256 possibilities in here, OK.
So I'm doing this to 256 why don't I precompute and store?
So if I just calculate for 256 different possibilities, we
see the palindrome or not, I just store a bit for each of
them, saying it's a palindrome or not, and then my palindrome
test is very simple.
I just look up my look up table and say, hey, is this a
palindrome, no?
Yes?
No?
Done.
So if I'm doing this computation a lot of times I
suddenly realize I can precompute the kind of things
we did before.
I am very safe, I can do that.
So sometimes you have this programs that has very
complicated control structures but when you look at this here
you realize [UNINTELLIGIBLE]
use multiple times in here.
There's S1, if v S2, else S3.
S4 again, if v S5.
And so there re a lot of conditions going on in here.
And something like that you can sometimes look at that and
say, like OK, look, because my conditions are simple.
I mean in here, I only do one, I just do that one and I will
create the path depending on what the
Boolean expressions are.
And even if you have two or three conditions you can
create a couple of parts depending on that.
So I'm replicating some stuff, like S1 is in
both, S4 is in both.
So I have expanded my code a lot, but now instead of having
this spaghetti code that my compiler and my architecture
is going to stop all the time because [UNINTELLIGIBLE]
prediction and stuff, as Professor Leiserson pointed
out, can have issues.
I created this nice single branch, lot of code to
execute, lot of chances for my compiler to do a lot of
optimization.
I can say things like that.
And compiler can do it most of the time and sometimes you
might want to help the compiler.
Procedure rules.
So one interesting thing is a lot of times when you have a
lot of small procedure calls.
A couple of things.
OK, let me ask you, what's the problem with having a lot of
small procedure calls?
You cause a lot of small methods all over the place.
What's the problem with small methods?
AUDIENCE: [INAUDIBLE].
PROFESSOR: [UNINTELLIGIBLE]
yes, because every time you do a method call you have to
basically go change everything [UNINTELLIGIBLE], put it back
in memory, stack, and then go do that.
I have a basically calling [UNINTELLIGIBLE]
expensive part.
OK, so if I'm doing a very small thing your calling
context might have a lot more than overhead than what the
procedure does, method does.
What else?
What's a more subtle thing that can happen?
So if you're thinking more, I'm a compiler guy so I think
like compilers, what would the compiler do at this point?
AUDIENCE: [INAUDIBLE].
PROFESSOR: Compiler [UNINTELLIGIBLE]
oops, barrier, I have no idea what's happening in here.
I had retrieved this as just bit stop.
I will do a bunch of things above that, I will do a bunch
things below that.
But I can't go through that kind of thing, so the compiler
basically treats that as, I can't touch that barrier.
And that [UNINTELLIGIBLE] means the amount of stuff that
the compiler can do is [UNINTELLIGIBLE].
So one way to do that is basically inline.
On a max function, it's pretty simple to inline.
Basically you can define a macro that the CE front end
will go inline for you.
So this is simple.
Or you can define inline method in here and put it in
the same file, and then we will tell the compiler, look,
go and try doing inline listening, and by doing that
we are forcing the compiler, or telling it the ability to
inline, and they're not only getting rid of the overhead of
the method calls, but also you're letting the compiler a
lot more opportunity to go and do something interesting.
So here's something you'll find in many
streaming type programs.
What you do is you run through a loop, reading through a
bunch of data, do some processing
and writing it back.
And then you want to get the data you wrote back, read it
again, do some processing and write it back.
The problem in that this one is the first loop which
stretches through all data.
So by the time I'm done with the first loop I have written
all the data and the thing I wrote early is probably all
the way down in my cache hierarchy.
Because I wrote a huge amount and it doesn't count it in my
cache so it goes down, down, down, and the next guy has to
read it all the way from bottom up.
So [UNINTELLIGIBLE]
means instead of doing that what you can do is run for a
small amount of data.
So don't run the entire data set.
Run a small amount of data, write it into a buffer in the
middle that we know sit in the cache, and the disk guys picks
up the buffer and finish it up and write it back.
So by keeping the data in the buffers what they have done is
between these two I actually had some nice locality.
I can keep the data up.
So as we go into memory systems and stuff like that
these kind of patterns come up a lot.
Ah ha.
Tail recursion elimination.
So here there are many functions that the last thing
you do is call itself.
OK?
This is pretty expensive because what normally happens
is when you call a function you set up a calling contest,
you do all this context [UNINTELLIGIBLE]
put things into the stack and go, but you don't need to keep
instead because you're basically done with the
previous guy.
You're just calling just to go create another stack.
In a lot of these cases you can reformulate the problem
into nice loop that has no function calls.
So, in here, when you're doing factorial, you can actually
formulate the factorial in a way that you don't have to do
a function call.
So a lot of times if you end up in a situation that the
last thing you do in some part is the call itself and if
you're not doing anything afterwards, you can always end
up in a situation where you don't know how do that, you
can actually convert it into [UNINTELLIGIBLE] loop.
And by doing this I got rid of all my calling overhead and
stuff that I'm just starting a nice tight loop.
OK?
So if you have patterns that look like that you can think,
OK, look, this is nice, I thought through
[UNINTELLIGIBLE].
I got this there.
But now I can [UNINTELLIGIBLE] transform it to more iterative
way of executing.
In many cases you will find a way of doing that.
OK, then there are a bunch of expression rules.
So a lot of times you have this computation that if you
think carefully you can let the preprocessor do in
something like calculate something like that, if I know
what [UNINTELLIGIBLE] are, stuff like that, I can just
put it in as a constant and voila I don't even have to pay
any cost to compute it because by the time it's compiled all
the computing is done.
And there are a lot of good cases you can just get rid of
work by giving it to the preprocessor.
I'm just calling sin(a) and sin(a) twice in here, and the
sin(a) squared I just basically, the compiler
probably won't know that the sine every time you give it
the same number, you give it the same same input, it's a
function, I think the compiler doesn't know that, and this is
something you probably have to do for the compiler.
Especially when you know the function has no side effects.
This is a little bit more funky, so assume I do two
similar computations, you might want to do them together
as a single computation that gives the compiler more
opportunity, but that means you relay the function like a
sine, cosine, s of 1 function.
It's a little bit of funky thing that in this list of
interesting things to do.
But I put it there because there might be cases you might
find that's interesting, like if you're doing taking min and
max, it might be calling min and max separated.
You might be able to call it together.
So the last set of rules is parallelism rules.
Of course, we are going to visit parallelism like crazy
in this class, but today we will just go through some very
high level stuff to give you a little bit of feel for what
can be done.
So I have this rule, this program.
So what's wrong with this program?
Why would this program run very slow?
Can anybody tell me?
Why can't, in this program, I can't do the full utilization
of my machine.
The problem in this program is every iteration until this
condition is calculated I can't start calculating this.
And so every iteration I had of it, this is finished,
condition calculated, and [UNINTELLIGIBLE]
and new calculate that.
And once that's done, you go to the next iteration.
We go to next iteration.
So by doing this I can't do too much work.
On the other hand, if I can unroll the loop like two times
and have two different conditions, now I can do this
condition and these conditions, and these two can
be done in parallel and when these two get resolved I can
do these two operations.
So instead of doing xmax, I'm operating two different
variables and at the end I can resolve that.
So what I'm doing is I'm calculating
two things in there.
And they go to four things, whatever, to
keep the machine busy.
And each of them can be calculated independently.
OK, so I can get parallelism and at the end of the day I
can do that to help when I'm summing something or whatever.
Do you see this?
So to get the six instruction issued at a time, something
like that will give you more abilities because there's more
things that can run in parallel.
So most of the time it will be good to see whether you can
eliminate these kind of chains like here xmax is a chain
because xmax--
until the previous is calculated I can't calculate
the next one, until that so I have this chain that until
this is calculated I can't do this, until this is done I
can't do the next iteration, so xmax is kind of keeping me
very sequential.
By doing that I have two things I can calculate.
So other interesting things is if you are doing through this
kind of data structure link list.
Sometimes if you have I believe the bypass have
another point that points couple of things ahead then I
can keep multiple of them going on.
So I can do sort of waiting for the next one I can
basically switch two elements, do that so the next one I can
get some parallelism in here so I can do things like that.
So to get some basic parallel performance going on.
And this is basic kind of a little bit of a data structure
augmentation.
You add additional data make it run faster.
And you want to get all your loops vectorized.
That means you want to run things like assembly
instruction.
If you're using 32 bits, you want to put a couple of them
together, run in one SIMD unit.
But I don't want any of you guys to go write
assembly to do this.
It's going to be a big pain to write all the SIMD assembly.
We are not talking about assembly but most the
compiler, [UNINTELLIGIBLE] compiler is somewhat
good at doing it.
So what this means is you're kind of really nudge the
compiler, look at bunch of compiler frags to get the
compiler to do it.
Sometimes it's--
when we go more into it, we'll figure out how we can do it
looking at given the right flags to compiler, sometimes
chaining the program a little bit makes it
easier for the compiler.
So sometimes just wrestling with the compiler, that's much
more easier, believe me, than trying to do it by yourself.
By doing that you can get some interesting performance.
And we will talk a lot more about this coarse grain
parallelism and we have a bunch of lectures on that.
Trying to figure out how to get multicores, each core do
something parallel, and get all your [UNINTELLIGIBLE]
in the machine working for at a given time
then only one guy.
And so here's an interesting situation.
So I have this problem here.
Can I run this parallel?
So what I'm doing is I'm doing these two loops and I'm adding
all the elements of an array to total.
Is this parallel?
Can I run anything or any of these things parallel?
Somebody says, yes, no, whatever, what do you think?
Take a stand.
OK, what do you think?
AUDIENCE: I think yes.
PROFESSOR: Somebody says yes, good.
Yes, takers, no takers.
Yes, everybody says yes.
[UNINTELLIGIBLE]
is parallel.
But look at here, what happens is previous total is needed to
calculate the next total, so how can I run this parallel?
AUDIENCE: [INAUDIBLE]
for loop jumps every two.
PROFESSOR: OK, so she's somewhat onto it, so let me
show you what we can do.
One thing you can do is instead of calculating one
total for each basically column, you can
calculate its own total.
OK?
So what happens is so for this i's row, j's column.
So for each row, so each row gets a different total, a temp
total, and so what that means is you can calculate all the
columns total separately.
You can total all the columns, and then total up those values
to get the full total.
So totalling up columns can be done parallel because now all
of these things can be done parallel because it's
totalling up two different, multiple different values.
So it's kind of what she said but you do it for each row
separately.
And so here's something, you made the program a little bit
more complicated.
Actually for a sequence it run a little bit slower but at the
end of the day you actually get the program
parallelizable.
So a lot of times when you're parallelizing is to also
looking at that algorithm little bit changes to the
program that at the beginning might look like you're slowing
down the program but ending up getting good parallel
performance.
So a lot of parallelism looks like this.
So here's the entire list of things we talked about.
It's out there so you can go look at them and see if you
understand them better.
And I'm leaving you with this example that I'm not going to
have time to go through. which is my program of a traveling
salesman problem.
So what you're doing is you're looking at a way for a
tradesman to travel to all of the city's
with a minimum cost.
Of course the minimum shortest thing is exponential algorithm
so you don't do that.
There's a good heuristic, that the greedy heuristic every
time you go to the city you go to the cheapest city that's
not visited and you just jump around and that's seem to be
pretty good heuristic.
And so I call this up, and I will go to the end and then
just finish it up.
Oops, I went too far.
So what I did was I have eight different stages basically,
each stage I did some changes and then I got this program to
run about five times faster than from where I started.
So you will see at each stage what I did.
So this is basically out of the menu of things I showed
you I found an instance I can do and in this program I got
five times faster.
This is more normal than the 300 [UNINTELLIGIBLE]
I got [UNINTELLIGIBLE]
and that's a real extreme case, but enjoy
looking at this code.