Given, the data set X(mxn matrix) where m is the number of measurement types and n is the number of samples.Mathematically ,the goal is to find some orthonormal matrix P in Y=PX such that is a diagonal matrix.The rows of P are the principal components of X.

Let’s state some elementary linear algebra theorems which will be helpful in formulating the above formula in a more meaningful way.

- The inverse of an orthogonal matrix is its transpose.
- For any matrix , , are symmetric.
- A matrix is symmetric if and only if it is orthogonally diagonalizable.
- A symmetric matrix is diagonalized by a matrix of its orthonormal eigenvectors i.e ( where D is the diagonal matrix and E is the matrix with eigenvectors of A arranged as columns)

Now,we select matrix P s.t each row is an eigenvector of

Therefore,the diagonal value of is the variance of **X** along

- In practice,computing PCA of a data set
**X**entails:subtracting off the mean of each measurement type .(PCA doesn’t necessarily give a unique answer.For example if the data matrix is of Temperature,then it might give a different P for Celsius and Fahrenheit measurements.To minimize this error ,mean is subtracted from each measurement.In time series analysis we normally z square the data to achieve the same goal i.e subtrat the mean and divide by std. deviation) - and computing the eigenvectors of

Advertisements

Principal Component Analysis is a standard tool in modern data analysis in diverse fields from neuroscience to computer graphics because it’s a simple, non parametric method for extracting relevant information from confusing data sets.

PCA is mainly used for dimensionality reduction.Its operation can be thought of as revealing the internal structure of the data in a way that best explains the variance in the data. If a multivariate dataset is visualised as a set of coordinates in a high dimensional data space, PCA can supply the user with a lower-dimensional picture, a “shadow” of this object when viewed from its most informative viewpoint. The main objective of PCA is to convert a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables called principal components**. **This transformation is defined in such a way that the first principal component has the largest possible variance and each succeeding component in turn has the highest variance possible under the constraint that it be orthogonal to (i.e., uncorrelated with) the preceding components.

Mathematically,Given a data matrix X, where every sample is a m dimensional vector where m is the number of measurement types(variables), we want to project X into lower dimensional space while preserving as much information as possible which can be achieved by finding an orthogonal basis that spans the column space of X. In particular, choose projection that minimizes squared error in reconstructing original data

PCA makes one stringent but powerful assumption,linearity.Linearity vastly simplifies the problem by restricting the set of potential bases.With this assumption PCA is now limited to reexpressing data as a linear combination of its basis vectors.

Let X be the original recorded data set and y is a new representation of the data set.Our goal is to find a linear transformation matrix P such as PX = Y s.t rows of P are row of basis vectors for expressing the columns of X.these row vectors will then become the principal components of X.

Now comes the question, what does “best express the data” mean .To extract valuable information from a signal measurement noise in any data should be low.A common measure for noise which quantifies it in terms of signal is SNR(Signal to noise ratio) :

SNR = ** / **where is the variance.

A high SNR indicates a high precision measurement whereas a low SNR indicates very noisy data.Thus , by assumption the dynamics of interest exist along directions with largest variance and presumably highest SNR.

Another key factor is the redundancy of data.For example,in a 2-dim case,for two arbitrary measurement types and , was it really necessary to record two variables or it would be more meaningful to just have recored a single variable,not both .Can we predict from or vice versa using the best fit line.This is the central idea behind dimensional reduction.

Identifying the redundant cases is easy for a 2-dim case (find the slope of the best-fit line and judge the quality of the fit).For higher dimensions this can be achieved by determining the covariances between the variables and defining the covariance matrix for X as where diagonal elements are the variance of particular measurement types,where large values denote interesting structure and off-diagonal terms are the covariance between different measurement types where large values correspond to high redundancy.We have to manipulate our covariance matrix to say to achieve following two goals:

- to minimize redundancy which is meaured by the magnitude of the covariance and
- maximize the signal,measured by the variance.

This can be achieved by diagonalizing i.e to make all off-diagonal elements to be zero (Y is decorrelated) and by rank ordering each successive dimension in Y according to variance.

While there are many ways for diagonalizing ,PCA selects the easiest method i.e to assume that all basis vectors {,,……} are orthonormal .terefore,P is an orthonormal matrix.The true benefit of this assumption is that there exists an efficient ,analytical solution to this problem.

Advertisements

This is undoubtedly one of the best explanation of higgs boson discovery I have seen so far.With this discovery, Large Hadron Collider has accomplished its primary mission of confirming the existence of the hypothesised and heretofore elusive subatomic particle, the Higgs Boson.

Higgs Boson discovery was probably the biggest news in the Physics world in the last decade or so.

P.S …an R chart appears at the 00:28 mark.

Advertisements

**You can answer many seemingly difficult questions quickly****.**But you are not very impressed by what can look like magic, because you know the trick. The trick is that your brain can quickly decide if question is answerable by one of a few powerful general purpose “machines” (e.g., continuity arguments, the correspondences between geometric and algebraic objects, linear algebra, ways to reduce the infinite to the finite through various forms of compactness) combined with specific facts you have learned about your area. The number of fundamental ideas and techniques that people use to solve problems is, perhaps surprisingly, pretty small — see http://www.tricki.org/tricki/map for a partial list, maintained by Timothy Gowers.**You are often confident that something is true long before you have an airtight proof for it (this happens especially often in geometry).**The main reason is that you have a large catalogue of connections between concepts, and you can quickly intuit that if X were to be false, that would create tensions with other things you know to be true, so you are inclined to believe X is probably true to maintain the harmony of the conceptual space. It’s not so much that you can imagine the situation perfectly, but you can quickly imagine many other things that are logically connected to it.**You are comfortable with feeling like you have no deep understanding of the problem you are studying.**Indeed, when you do have a deep understanding, you have solved the problem and it is time to do something else. This makes the total time you spend in life reveling in your mastery of something quite brief. One of the main skills of research scientists of any type is knowing how to work comfortably and productively in a state of confusion. More on this in the next few bullets.**Your intuitive thinking about a problem is productive and usefully structured, wasting little time on being aimlessly puzzled.**For example, when answering a question about a high-dimensional space (e.g., whether a certain kind of rotation of a five-dimensional object has a “fixed point” which does not move during the rotation), you do not spend much time straining to visualize those things that do not have obvious analogues in two and three dimensions. (Violating this principle is a huge source of frustration for beginning maths students who don’t know that they shouldn’t be straining to visualize things for which they don’t seem to have the visualizing machinery.) Instead…**When trying to understand a new thing, you automatically focus on very simple examples that are easy to think about, and then you leverage intuition about the examples into more impressive insights.**For example, you might imagine two- and three-dimensional rotations that are analogous to the one you really care about, and think about whether they clearly do or don’t have the desired property. Then you think about what was important to the examples and try to distill those ideas into symbols. Often, you see that the key idea in the symbolic manipulations doesn’t depend on anything about two or three dimensions, and you know how to answer your hard question.As you get more mathematically advanced, the examples you consider easy are actually complex insights built up from many easier examples; the “simple case” you think about now took you two years to become comfortable with. But at any given stage, you do not strain to obtain a magical illumination about something intractable; you work to reduce it to the things that feel friendly.- To me, the
**biggest misconception that non-mathematicians have about how mathematicians think is that there is some mysterious mental faculty that is used to crack a problem all at once**. In reality, one can ever think only a few moves ahead, trying out possible attacks from one’s arsenal on simple examples relating to the problem, trying to establish partial results, or looking to make analogies with other ideas one understands. This is the same way that one solves problems in one’s first real maths courses in university and in competitions. What happens as you get more advanced is simply that the arsenal grows larger, the thinking gets somewhat faster due to practice, and you have more examples to try, perhaps making better guesses about what is likely to yield progress. Sometimes, during this process, a sudden insight comes, but it would not be possible without the painstaking groundwork [http://terrytao.wordpress.com/ca… ].Indeed, most of the bullet points here summarize feelings familiar to many serious students of mathematics who are in the middle of their undergraduate careers; as you learn more mathematics, these experiences apply to “bigger” things but have the same fundamental flavor. **You go up in abstraction, “higher and higher”. The main object of study yesterday becomes just an example or a tiny part of what you are considering today.**For example, in calculus classes you think about functions or curves. In functional analysis or algebraic geometry, you think of spaces whose*points*are functions or curves — that is, you “zoom out” so that every function is just a point in a space, surrounded by many other “nearby” functions. Using this kind of zooming out technique, you can say very complex things in short sentences — things that, if unpacked and said at the zoomed-in level, would take up pages. Abstracting and compressing in this way allows you to consider extremely complicated issues while using your limited memory and processing power.**The particularly “abstract” or “technical” parts of many other subjects seem quite accessible because they boil down to maths you already know. You generally feel confident about your ability to learn most quantitative ideas and techniques.**A theoretical physicist friend likes to say, only partly in jest, that there should be books titled “______ for Mathematicians”, where _____ is something generally believed to be difficult (quantum chemistry, general relativity, securities pricing, formal epistemology). Those books would be short and pithy, because many key concepts in those subjects are ones that mathematicians are well equipped to understand. Often, those parts can be explained more briefly and elegantly than they usually are if the explanation can assume a knowledge of maths and a facility with abstraction.Learning the domain-specific elements of a different field can still be hard — for instance, physical intuition and economic intuition seem to rely on tricks of the brain that are not learned through mathematical training alone. But the quantitative and logical techniques you sharpen as a mathematician allow you to take many shortcuts that make learning other fields easier, as long as you are willing to be humble and modify those mathematical habits that are not useful in the new field.

**You move easily between multiple seemingly very different ways of representing a problem**. For example, most problems and concepts have more algebraic representations (closer in spirit to an algorithm) and more geometric ones (closer in spirit to a picture). You go back and forth between them naturally, using whichever one is more helpful at the moment.Indeed, some of the most powerful ideas in mathematics (e.g., duality, Galois theory [http://en.wikipedia.org/wiki/Gal…, algebraic geometry [http://en.wikipedia.org/wiki/Alg…) provide “dictionaries” for moving between “worlds” in ways that,*ex ante*, are very surprising. For example, Galois theory allows us to use our understanding of symmetries of shapes (e.g., rigid motions of an octagon) to understand why you can solve any fourth-degree polynomial equation in closed form, but not any fifth-degree polynomial equation. Once you know these threads between different parts of the universe, you can use them like wormholes to extricate yourself from a place where you would otherwise be stuck. The next two bullets expand on this.**Spoiled by the power of your best tools, you tend to shy away from messy calculations or long, case-by-case arguments unless they are absolutely unavoidable.**Mathematicians develop a powerful attachment to elegance and depth, which are in tension with, if not directly opposed to, mechanical calculation. Mathematicians will often spend days figuring out why a result follows easily from some very deep and general pattern that is already well-understood, rather than from a string of calculations. Indeed, you tend to choose problems*motivated*by how likely it is that there will be some “clean” insight in them, as opposed to a detailed but ultimately unenlightening proof by exhaustively enumerating a bunch of possibilities. (Nevertheless, detailed calculation of an example is often a crucial part of beginning to see what is really going on in a problem; and, depending on the field,*some*calculation often plays an essential role even in the best proof of a result.)In*A Mathematician’s Apology*[http://www.math.ualberta.ca/~mss…, the most poetic book I know on what it is “like” to be a mathematician], G.H. Hardy wrote:”In both [these example] theorems (and in the theorems, of course, I include the proofs) there is a very high degree of unexpectedness, combined with inevitability and economy. The arguments take so odd and surprising a form; the weapons used seem so childishly simple when compared with the far-reaching results; but there is no escape from the conclusions. There are no complications of detail—one line of attack is enough in each case; and this is true too of the proofs of many much more difficult theorems, the full appreciation of which demands quite a high degree of technical proficiency. We do not want many ‘variations’ in the proof of a mathematical theorem: ‘enumeration of cases’, indeed, is one of the duller forms of mathematical argument. A mathematical proof should resemble a simple and clear-cut constellation, not a scattered cluster in the Milky Way.”[…]“[A solution to a difficult chess problem] is quite genuine mathematics, and has its merits; but it is just that ‘proof by enumeration of cases’ (and of cases which do not, at bottom, differ at all profoundly) which a real mathematician tends to despise.”

**You develop a strong aesthetic preference for powerful and general ideas that connect hundreds of difficult questions, as opposed to resolutions of particular puzzles.**Mathematicians don’t really care about “the answer” to any particular question; even the most sought-after theorems, like Fermat’s Last Theorem [http://en.wikipedia.org/wiki/Fer…, are only tantalizing because their difficulty tells us that we have to develop very good tools and understand very new things to have a shot at proving them. It is what we get in the process, and not the answer*per se*, that is the valuable thing. The accomplishment a mathematician seeks is finding a new dictionary or wormhole between different parts of the conceptual universe. As a result, many mathematicians do not focus on deriving the practical or computational implications of their studies (which can be a drawback of the hyper-abstract approach!); instead, they simply want to find the most powerful and general connections. Timothy Gowers has some interesting comments on this issue, and disagreements within the mathematical community about it [http://www.dpmms.cam.ac.uk/~wtg1… ].

**Understanding something abstract or proving that something is true becomes a task a lot like building something.**You think: “First I will lay this foundation, then I will build this framework using these familiar pieces, but leave the walls to fill in later, then I will test the beams…” All these steps have mathematical analogues, and structuring things in a modular way allows you to spend several days thinking about something you do not understand without feeling lost or frustrated. (I should say, “without feeling unbearably lost and frustrated; some amount of these feelings is inevitable, but the key is to reduce them to a tolearable degree.)Andrew Wiles, who proved Fermat’s Last Theorem, used an “exploring” metaphor:

“Perhaps I can best describe my experience of doing mathematics in terms of a journey through a dark unexplored mansion. You enter the first room of the mansion and it’s completely dark. You stumble around bumping into the furniture, but gradually you learn where each piece of furniture is. Finally, after six months or so, you find the light switch, you turn it on, and suddenly it’s all illuminated. You can see exactly where you were. Then you move into the next room and spend another six months in the dark. So each of these breakthroughs, while sometimes they’re momentary, sometimes over a period of a day or two, they are the culmination of—and couldn’t exist without—the many months of stumbling around in the dark that proceed them.” [http://www.pbs.org/wgbh/nova/phy… ]

**In listening to a seminar or while reading a paper, you don’t get stuck as much as you used to**in youth because you are good at modularizing a conceptual space, taking certain calculations or arguments you don’t understand as “black boxes”, and considering their implications anyway. You can sometimes make statements you know are true and have good intuition for, without understanding all the details. You can often detect where the delicate or interesting part of something is based on only a very high-level explanation. (I first saw these phenomena highlighted by Ravi Vakil, who offers insightful advice on being a mathematics student: http://math.stanford.edu/~vakil/… .)**You are good at generating your own definitions and your own questions in thinking about some new kind of abstraction.**

One of the things one learns fairly late in a typical mathematical education (often only at the stage of starting to do research) is how to make good, useful definitions. Something I’ve reliably heard from people who know parts of mathematics well but never went on to be professional mathematicians (i.e., write articles about new mathematics for a living) is that they were good at proving difficult propositions that were stated in a textbook exercise, but would be lost if presented with a mathematical structure and asked to find and prove some interesting facts about it. Concretely, the ability to do this amounts to being good at making definitions and, using the newly defined concepts, formulating precise results that other mathematicians find intriguing or enlightening.This kind of challenge is like being given a world and asked to find events in it that come together to form a good detective story. Unlike a more standard detective, you have to figure out what the “crime” (interesting question) might be; you’ll you have to generate your own “clues” by building up deductively from the basic axioms. To do these things, you use analogies with other detective stories (mathematical theories) that you know and a taste for what is surprising or deep. How this process works is perhaps the most difficult aspect of mathematical work to describe precisely but also the thing that I would guess is the strongest thing that mathematicians have in common.**You are easily annoyed by imprecision in talking about the quantitative or logical.**This is mostly because you are trained to quickly think about counterexamples that make an imprecise claim seem obviously false.**On the other hand, you are very comfortable with intentional imprecision or “hand-waving” in areas you know, because you know how to fill in the details.**Terence Tao is very eloquent about this here [http://terrytao.wordpress.com/ca… ]:”[After learning to think rigorously, comes the] ‘post-rigorous’ stage, in which one has grown comfortable with all the rigorous foundations of one’s chosen field, and is now ready to revisit and refine one’s pre-rigorous intuition on the subject, but this time with the intuition solidly buttressed by rigorous theory. (For instance, in this stage one would be able to quickly and accurately perform computations in vector calculus by using analogies with scalar calculus, or informal and semi-rigorous use of infinitesimals, big-O notation, and so forth, and be able to convert all such calculations into a rigorous argument whenever required.) The emphasis is now on applications, intuition, and the ‘big picture’. This stage usually occupies the late graduate years and beyond.”In particular, an idea that took hours to understand correctly the first time (“for any arbitrarily small epsilon I can find a small delta so that this statement is true”) becomes such a basic element of your later thinking that you don’t give it conscious thought.- Before wrapping up, it is worth mentioning that mathematicians are not immune to the limitations faced by most others. They are not typically intellectual superheroes. For instance, they often become resistant to new ideas and uncomfortable with ways of thinking (even about mathematics) that are not their own. They can be defensive about intellectual turf, dismissive of others, or petty in their disputes. Above, I have tried to summarize how the mathematical way of thinking feels and works at its best, without focusing on personality flaws of mathematicians or on the politics of various mathematical fields. These issues are worthy of their own long answers!
**You are humble about your knowledge because you are aware of how weak maths is, and you are comfortable with the fact that you can say nothing intelligent about most problems**. There are only very few mathematical questions to which we have reasonably insightful answers. There are even fewer questions, obviously, to which*any given*mathematician can give a good answer. After two or three years of a standard university curriculum, a good maths undergraduate can effortlessly write down hundreds of mathematical questions to which the very best mathematicians could not venture even a tentative answer. (The theoretical computer scientist Richard Lipton lists some examples of potentially “deep” ignorance here: http://rjlipton.wordpress.com/20…) This makes it more comfortable to be stumped by most problems; a sense that you know roughly what questions are tractable and which are currently far beyond our abilities is humbling, but also frees you from being very intimidated, because you do know you are familiar with the most powerful apparatus we have for dealing with these kinds of problems.

Advertisements

Advertisements