In 2009, IRS Commissioner Doug Shulman announced that the IRS had created a new task force to audit people he called "global high-wealth individuals." These new IRS targets, Shulman explained, have tens of millions of dollars in income and assets and "make use of sophisticated financial, business, and investment arrangements with complicated legal structures and tax consequences." They often have an intricate web of related business entities like S-corporations and other pass-through entities they control, along with various off-shore accounts and business entities. In other words, they're people like Mitt Romney.

There aren't that many people in the "global high-wealth individual" group—only about 8,000 taxpayers a year who have more than $10 million in annual income—and Romney is "exactly the kind of taxpayer the program was designed to look at," says Rebecca Wilkins, a lawyer at the Center for Tax Justice who used to work with rich clients as a CPA. It's possible for someone with a lot of income, like a corporate CEO, to have a fairly simple tax return, Wilkins says. But Romney's return from 2010 (the only completed one he's released so far) weighs in at 203 pages; 55 pages are simply devoted to disclosing the existence of a host of foreign transactions in tax havens like the Cayman Islands. In his domestic portfolio, there's Romney’s IRA, to which he was legally able to contribute only around $30,000 a year but which is now mysteriously worth between $21 and $102 million. It's the sort of stuff that the new IRS unit is supposed to vet.

Romney's campaign has said he's never been audited. "I pay all the taxes that are legally required and not a dollar more," Romney said during a January primary debate. But IRS audits on Romney's fellow elite global high-wealth individuals have turned up quite a bit of extra money for the government. Out of the 36 high-wealth individuals audited in fiscal 2011 and the first five months of fiscal 2012, the IRS discovered an extra $47 million in taxes that should have been paid by 24 people in that group. Those figures suggest that some mega-rich people using aggressive-but-legal tax avoidance schemes are still not paying all they owe. There's no telling whether Romney deserves an audit or would pass one. But as long as he doesn't release his tax returns, speculation will continue about what's on those returns and whether they need a good look.