It's getting even harder for low-income renters to find housing they can afford -- not that it was ever easy.

In a new report, Harvard's Joint Center for Housing Studies says there were 6.8 million more U.S. households among the renting poor in 2009 than the number of units priced for that demographic and not occupied by higher-income tenants. That's up from 5.6 million in 2003, its report says.

That change is driven by more demand plus less supply -- an increasing number of renters with very low incomes even as the number of available units dropped by 200,000. And the problem is poised to get worse.

"With little new supply of multifamily units in the pipeline, rents could rise sharply as demand increases," the center warns. "Regardless, affordability is likely to deteriorate further over the next few years as persistently high unemployment limits renter income gains."

Renters as a whole, not just the lowest-income tenants, have felt the pressure. Renter income dropped in the last decade, hitting a level in 2009 that was actually lower -- in real terms -- than what renters were earning in 1980. But the cost of monthly rent is up.

Rising energy costs aren't helping, either. They accounted for 15 percent of all renters' housing costs in 2009, up from less than 11 percent at the beginning of the decade, the center says.