You wrote: "... there may have been motivation for Magnetar to pressure the underwriters to include particularly toxic mortgages in the CDOs they sponsored." No, I doubt it. Magnetar pressured the CDO managers (*not* the underwriter, which were the banks) to put in HIGHER YIELDING assets -- now, higher yielding assets are probably crappier, but they didn't care about the credit quality per say, only the high coupon with which they can achieve the positive carry -- i.e., getting paid while making the trade.
Furthermore, after reading that Magnetar asked Janet to provide advice, I must say, these guys aren't that sophisticated. Janet's books are great, but are NOT technical books and are horrible places for seeking trade ideas! You're better off asking the research desk of any investment bank. Please see the attached CDS primer from Lehman circa 2005, which shows you the pricing discrepancies that exist in the market, and how you can profit from it (page 10, under "valuation considerations"). They even provide a nice table (see figure 11 on page 12) on the returns given different assumptions of leverage and borrowing cost. This is just one document however, and plenty of other guides available that time will tell you how to hedge the default risk by shorting a more senior tranche - which implies that you believe correlation is high (you're long correlation -- see page 8 on the correlation trading guide from Merrill in 03 - this is for corp bond CDOs, but whatever, same concept).
So yeah, I doubt these guys knew enough to be able to figure out which loans were crappy even if they tried. If they were so good at picking out bad loans, they'd have just taken a bet (trading a view) on the crappy loans via an ABS-CDS on that crappy bond pool. Why go through all the trouble of setting up a CDO?
So in summary - these guys didn't purposefully set up crappy CDOs because
1) they frankly weren't that smart and didn't seem that knowledgeable (they made money because they were lucky - Howie Hubler blew up and lost $8 billion doing a correlation trade, and he was the smartest bond trader at Morgan Stanley), and therefore
2) couldn't have picked out which bonds were crappy even if they tried.
Here is the Lehman CDS primer (20 pages) and here is a Merrill primer on correlation trading (37 pages).
Hmm... what else could these brainy guys have been doing instead of working on "financial innovation"?
See A reallocation of human capital and The bubble algorithm for human capital allocation.
The Magnetar trade part one, two, three.