The Myth of the Free ALUs

One of the myths which arised with the new generation of consoles was that arithmetic operations (aka ALUs) were from now on literally free. If long ago we used cube map LUTs to normalize vectors, on GCN it costs nothing. Sure, they do have some cost, but with 6-7 ALU cycles per byte read from VRAM recommended at the top memory bandwidth (which you even almost never get), they should be all hidden beyond the memory fetches latency.

Well, it turns out, they are not. And here are some of the reasons why.

Some ALUs are more expensive than others. 

If an extra mad would unlikely (I won quite significantly one day by optimizing out a redundant matrix multiplication from a tight loop) make a change, divisions or even trigonometry would. Even in an SSAO shader could benefit up to 13% performance increase when arithmetic is optimized.

There are no free lunches now. 

Everything you could have relied hardware to do before (cube map texcoord calculation, attribute interpolation etc.) is now done with ALUs. You can use nointerpolation to avoid, if needed.

The scalar/vector ALUs are unbalanced.

In case your shader utilizes too many SALUs, not only you may become scalar register bound, but also your VALU units might stall while waiting for a scalar op result. An example here is excessive lane swizzling: when I tried to use it as a "cheap shared memory replacement" for blur filter, I got a ridiculous SALU/VALU ratio.

Computer Architecture - How To?

Some time ago I remember asking a more senior team member: how does one learn about how hardware operates on the low level? I was answered that this could be learned only from practice.
While I don't think that this is a completely invalid approach - since practical experience is always more valuable than a raw theory - I was searching for some theoretical basis I could get before diving into practice. I think I have something now what I can recommend.
Firstly, there is this course on hardware architecture, which is really good: https://www.coursera.org/course/comparch
Beware though, it is a grad-level course, so the level is really very demanding. You should know what are caches, associativity, memory hierarchy etc. They recommend the following book:

I really recommend you start reading it from appendix, which serves a sort of a primer. The book is also very up-to-date, even more than the course itself.
The second book recommended (http://www.amazon.com/Modern-Processor-Design-Fundamentals-Superscalar/dp/1478607831/) provides a lot of low-level details on reservation stations/register renaming, very in-depth on a pair of particular architectures, and a comprehensive description of branch prediction algorithms. Despite it has been just republished, it is a bit outdated (e.g., Intel P6 microarchitecture is used as example), so it is really up to you to buy or to pass.
I hope this has been helpful, and as previously, feel free to comment and discuss.