I am compiling links and notes to papers about JIT bytecode techniques. Eventually, this collection will become a full-fledged review paper.
Tracing JIT compilers
Good discussion and links here. Highlights: LuaJit, Dynamo
Interesting hybrid approach.
Not extremely JIT related, but interesting nonetheless. Instead of applying peephole optimizations, a model is constructed which represents all programs which do the same thing as the input. Then the problem is treated as an actual comb.opt problem and the answer is searched/solved for.
Dynamo is a JIT translator from a native code to itself, optimizing in the process. Most gains are probably from turning complicated control flow into straight-line code. Transmeta Crusoe was a hardware-accelerated version of a similar concept.