Orthogonality & Uncorrelatedness

Recently I was pondering over orthogonality... a simple concept in the domain of geometry, characterizing two vectors in Euclidian space with 90° angle spanned between them. The orthogonality means that the inner product of two vectors has to be equal to zero.

Our TiU Econometrics grad course is drawing heavily on this notion, since it proves very handy in explaining mechanics underlying the Ordinary Least Squares (OLS) regression. And that's how I got tangled in the concept.

So without further ado, here's a small puzzle:

Advanced econometrics textbooks equate the orthogonality with uncorrelatedness. The two concepts are hence considered equivalent,

Othogonal <=> uncorrelated.

However, we could easily come up with examples which break one part of the equivalence whilst sustaining the other. A straightforward illustration is a pair of individual-specific dummies. These are orthogonal by construction, however negatively correlated with each other!

Similarly, the opposite case of the equivalence violation could arise by taking again an individual-specific dummy vector, and a vector of constant terms. This pair of vectors is uncorrelated, however when plotted into the Euclidean space, the angle between the two is just 45°!

So, that is quite confusing, right? Well fear not - we do not have to lose faith in OLS. It turns out that the notion of orthogonality in geometry and in econometrics are actually two slightly different beasts. To satisfy the textbook equivalence of uncorrelatedness, the geometrical orthogonality has to be applied not to the original vectors, but to their de-meaned forms! The orthogonality of the original vectors has in most cases no consequence for uncorrelatedness of a & b.

This is in fact very natural, since correlation is based on covariance, which is a result of integration over a product of de-meaned realizations of two random variables.

The culprit of this semantic difference between the geometric and econometric interpretation of orthogonality is actually the constant term, which acts as a de-meaning agent in the context of OLS regression. But since the constant is almost always present in the econometric models, the de-meaning becomes somewhat implicit. The econometric orthogonality turns into something what I would probably call "centered orthogonality".

This identification has also straightforward implication for the inner product of the original vectors. These two vectors are (econometrically) orthogonal, when their inner product is equal to n-times the product of their means, with n being the number of their scalar components.



Oh, the semantics...

Referee's dilemma

Do you sometimes find yourself in a situation where people expect you to undertake a very daunting task despite your lack of prior experience?
Well, I know I do. And although it would be tempting to pull that joke - no, this isn't a cheeky metaphor for my PhD studies. At least not an intentional one. Instead, I am alluding to something much more specific: the mysterious process of becoming a referee.

As usual, these things happen to me somewhat out of the blue, and so, two months after commencing my PhD studies, I opened my mailbox just to see that I was selected to file my first referee report! I felt really excited, but this feeling was quickly overridden by the sense of utter insecurity as I had no idea whether I was sufficiently qualified for the job. Sure, in my prior work I used similar estimation techniques as the ones in the questioned paper, but still - could I rigorously asses merits of another person's work? Even more so if it is a Professor? Clearly, the seed of doubt had been planted....

Nevertheless, I am always up for a challenge, so I agreed to write the report. The whole task was however still very obscure to me, and therefore I started gathering information about the do's and dont's of refereeing. Apart from some referee reports I had in hand, I made a good use of advices provided by JME, IAAE, and this wonderful piece by Dave Harrison. If you're in a similar position like me, I strongly recommend reading through these works carefully.

After absorbing all the recommendations, I have begun with the assessment itself. First read-through revealed that the paper has some potential, but handful of considerable flaws in empirical analysis marred the overall picture. I guess this happens way too often, but still - such an outcome was not what I was hoping for. Unlike with extremely good (or bad) papers, you cannot just write an exalted acceptance letter (or a resolute rejection pamphlet). You have to ponder whether the paper is good enough to be let through (after some minor/major changes), or whether the author would be better off starting from a scratch, or maybe switching to a more fitting journal.
Sounds hard? Well don't forget that this dilemma is further obfuscated by the fact that you cannot check validity of either the data or the codes which back up the analysis in question.

...boy, deciding between the recommendation statements is tough! I was contemplating either a revision or a weak rejection, and you can imagine that my initial sense of insecurity was now beyond belief... assessing the relevance of one's paper is one thing, but claiming it is not worth publishing? That's harsh.

Now, I am not going to disclose my eventual decision, but I will share a piece of advice which helped me to make up my mind:

If you're not sure whether the paper can make it to the publishing phase, try to imagine how much time it would take to transform it into an acceptable manuscript (provided all the results remain valid). If it is approaching the time you deem necessary to write up a similar, yet publishable paper, then it's probably better to reject. If, however, the author could be better off retaining his project and changing it here and there, suggest which changes are bound to be made and ask for a revision.

Closing thoughts? Refereeing is a daring task, especially if you want to do your job well. Given the time it takes, it is hardly a surprise that many academics turn the requests down and consider the whole business as nothing but a tiresome nuisance. However, every coin has two sides and I believe that the refereeing can be very enlightening, especially for PhD students. Firstly, by assessing the paper you are bound to develop better sense of critical thinking. Secondly, scrutinizing the paper's subject matter will broaden your research scope. But most importantly, the whole experience will improve your own journal submissions.
After finishing the first report, I browsed through my older works and realized that I see them in an entirely different light. I could pinpoint many of the little flaws which were bound to displease prospective referees and tilt their decision towards rejection.

Obviously, I have tried my best to correct these flaws, and my fist submitted paper is currently under review...so let's see whether that refereeing gig paid off!

Hello world.

Here I am, blogging once again... Let's see how long it'll last this time :)