Zillow's home-value estimates criticized
You've probably checked out Zillow.com's "Zestimates" at some point -- those free estimates of a home's value -- so you might have given some thought to how accurate they are. That's the subject of much debate, actually.
The latest volley comes from professors Daniel R. Hollas, Ronald C. Rutherford and Thomas A. Thomson, arguing in The Appraisal Journal that they're really Overzestimates:
The results indicate that Zillow overestimates value for approximately 80% of the houses in the sample by at least 1%. ... The average overestimation is 11.66% or $13,576, with a median of $9,717 or 7.92%. Zillow’s magnitude of overestimation is marginally higher than the value overestimation by recent homebuyers reported in the literature.
The authors -- from the University of Texas at San Antonio -- pitted Zestimates vs. 2006 sale prices of about 2,000 houses in Arlington, Texas. "Zillow indicates that this market is one where its data has its highest accuracy rating," they wrote, adding later: "The likelihood is that in [a] more ... volatile market with a lower accuracy rating Zillow would misprice at a higher rate and larger amount."
Zillow roared back with a response:
"In addition to being limited to only one city in the U.S., the study does not compare sales and Zestimate values during the same time period; it looks at sales in 2006 compared to Zestimate values in January and February 2007 – apples and oranges as it’s two separate periods of time," Zillow spokeswoman Jill Simmons wrote in an email to me. "It’s unfortunate that this study has been structured in such a misleading, and limited fashion."
Here's how the authors describe their methodology:
The sale price data, housing characteristics, and location data are obtained from the MLS for the city of Arlington, Texas (Tarrant County), for sales during the last six months of 2006. The sale price data was acquired in January 2007. Next, data from Zillow.com was obtained for each of the MLS sales to acquire Zestimates during the last week of January 2007 and the first two weeks of February 2007.
And here's how Zillow describes (for a non-mathematician audience) how it arrives at its figures:
When our statisticians developed the model to determine home values, they explored how homes in certain areas were similar (i.e., number of bedrooms and baths, and a myriad of other details) and then looked at the relationships between actual sale prices and those home details. These relationships form a pattern, and they used that pattern to develop a model to estimate a market value for a home.
How does the Zestimate of your home -- or homes near you -- match up with your sense of value?







Comments
My problem with Zillow was not necessarily related to home prices... The oddest thing was that during the period when I was purchasing my home, zillow never had it listed. The estimates for the homes in my neighborhood was a little high but nothing out of the range of possibilities. I would peg it to the fact that the housing market was collapsing rapidly and no one really knew what the value of any home was.
Posted by: bryanintowson | February 26, 2010 8:38 AM
I don't put a lot of faith in the zestimate. I just look at recent sales (last 6 months) in the area.
Posted by: jfg | February 26, 2010 9:05 AM
The estimate for my home on zillow is exactly the same as a recent appraisal that was done.
Posted by: zsizzle | February 26, 2010 9:26 AM
So, this is what our universities are researching/dealing with? My doctoral thesis is surely more interesting and academically scientific: The Role Of Non-Verbal Transgressive Random Price Shaping (with the goal of pocket filling) At The Average Real-Estate Agency In West Carolina, In The Light Of The Onset Of The X-Crisis Year Of The 21st Century.
Whoever gives grants for such research should consider charitable giving.
Posted by: Stanley | February 26, 2010 9:35 AM
I've come to hate Zestimates. The bankers in charge of short sales seem to be using Zestimates sometimes as an indicator of how much money they should be able to get. However, Zestimates are computed using comparables that are wildly inappropriate. When I tried to buy a tiny townhouse in Russett Zillow was using single family homes, bigger townhouses, and properties with many more bathrooms and bedrooms as comparables. The result was a valuation that had no basis in reality.
Since my current property is so new there is no Zestimate available yet. In fact, it's not even on Google maps yet! I think Zillow has done more harm to the industry than good though. We rely too much on computer models these days and not enough on our own intuition!
Posted by: BigDragon | February 26, 2010 1:28 PM
For what it's worth, I've noticed the appraisals on eppraisal and cyberHomes are much lower than zillow's estimates.
Posted by: Matt | February 26, 2010 1:52 PM
This is NOT new. I wrote a blog entry myself, three years ago, that cited a study in the Wall Street Journal that called Zillow to task.
To test Zillow against the real estate market, the Journal used direct market data from regional multiple lists for 1,000 transactions that had just closed – too early for that data to show up in Zillow’s database. The overall accuracy of Zillow’s estimates in the Journal study had a median price difference of 7.8%, above or below the actual sales price. Zillow itself lists its median ‘margin of error’ at 7.2%. Not bad, you say? Well, remember that on a lowly $200,000 purchase 7% adds up to $14,000. Not only will that pay a Realtor’s commission and more, it can also exhaust the negotiating room that many Sellers build into their asking prices, or that Buyers build into their opening offer. But that’s just the median.
Zillow was more than 25% off target in 11% of the transactions surveyed. And in 34 of the 1,000 transactions, Zillow was more than 50% off. In one spectacular example of computer wizardry, Zillow underestimated the value of a house by more than $2 million. (For the original Journal article, see the Personal Finance section of the paper, Wednesday February 14, 2007.)
There are too many variations, allowances, etc. that need to be made in the process of estimating the value of a property. A person, a qualified real estate agent or appraiser (depending upon the reason you want the estimate) is still your best bet.
Posted by: Wayne Curtis | February 26, 2010 1:54 PM
I know, Wayne -- that's why I noted that this is just the latest volley in the debate. Thanks for summing up the WSJ piece, which was an interesting analysis.
Posted by: Jamie Smith Hopkins | February 26, 2010 2:17 PM
Don't get me started! When we were getting well on into the housing slump, Zillow's zestimates kept rising and stayed way too high for an inexcusably long period of time.. then WHOMP, all of a sudden they plummeted and now often appear unreasonably low, in relation to comparable sales for the Baltimore neighborhoods I track. They don't even list my house in the correct neighborhood, so the comps they use for it are completely off. Maybe zsizzle's appraiser used Zillow... now that's scary!
Posted by: lisa | February 26, 2010 2:25 PM
Jamie, I have written multiple posts on my blog about how far off Zestimates are in the Towson area. I was able to, thanks to twitter, get in touch with some of the higher ups there to get a better understanding of their values.
Another way they determine price is 1) What you bought it for, which is completely irrelevant, and 2) What your taxes are, which can be very off as well if you have lived in your home for a very long time.
The people at Zillow also stated that the area in question must have very differing homes. Essentially, if the homes are not cookie cutter, their Zestimates will not be close.
Perfect example: Zillow had a Zestimate on a home of $948,000. It sold for $2.1 million. A little off?
Posted by: Marney Kirk | February 26, 2010 5:35 PM
When doing research on a property, I have previously used Zillow but found that the data provided for the area I was looking in was not clearly accurate. For my last investment purchase, I contacted a realtor who was able to provide me with MLS sheets on comparable properties in the area that had sold within the last year. Using this information plus doing research on remodeling costs, I was able to present a negotiable price with the seller and landed a sweetheart deal and a great place for my tenant. Being diligent in your own interests means not relying on outside sources to predetermine pricing. Getting out there and policing your interests gets better results.
Posted by: Charschi | February 26, 2010 7:27 PM
And speaking of cookie cutter houses -- In Baltimore, what I found when viewing homes was that virtually identical houses on the exact same street, e.g. built in the 1920's by the same builder, can be dramatically different in terms of the shape they are in. Some may have been let go with shoddy maintenance/upkeep and minimal rehab and others may be nearly pristine and extremely well cared for over the years -- and that's not even taking total rehabs into consideration. Even in this market, two such homes would garner very different prices. As Charschi mentions, only actual sales comps, rather than quant methods can give you a better feel, but seeing and feeling a home is above all the best means.
Posted by: lisa | February 27, 2010 3:30 AM
I don't live anywhere near Baltimore, but Zillow's estimates are awful for my area (in Ann Arbor, MI).
Zillow says our house is worth $134,000 (and within the past month went as high as $144,000). We just sold our house for the most our Realtor thought it would appraise for -- $100,000. That's a hefty margin of error on somebody's part! Our appraisal is Tuesday. Guess we'll find out who's right and who's wrong then ... but my money's not on Zillow for accuracy.
Also -- in the neighborhood where we're moving, it has mistaken the taxable value for the value of the house. In Michigan, the taxable value (SEV) is 50% of the value of the house.
Posted by: Jen | February 27, 2010 7:13 AM
Zillow is free! Why complain? Most of the people commenting here probably use Zillow from time to time to check comparable sales if nothing else.
Zillow is the only website I know of that's free without having to sign up for anything.
Posted by: JB | February 28, 2010 10:56 AM
I think Zillow is pretty much a random number generator. Sadly, many sellers also view it as some kind of a price setting engine. They need to understand that Zillow cannot take into consideration the condition of a particular property and many other important details.
I admit that I do look at the numbers, but so far I saw only 2 homes where the numbers were somewhat close to reality. Most of the time they're overestimated. I only use them as a "ballpark" estimate and also to check how the property stacks up against others on the same street, for example.
As already mentioned, it is free, so you get what you paid for, I guess.
Posted by: Jelena | March 2, 2010 2:35 PM
I lived in a house in San Gabriel CA. I personally knew the broker because we'd worked together at a commercial MLS. The house was very small, barely over 1K sq.ft. The inside doors were all outdated, skeleton key doors whose locks didn't work. The front bedroom was tiny. It had a big back yard. The broker had it listed for $465K, which was over the median for that area, even though my friend insisted that it was what the market would bear. Zillow completed mistated the details of the house by a factor of 5. They stated it had 5 bedrooms and that it had 5K sq.ft. They inflated the price to over $1M. I will never recommend Zillow. I work at a construction cost publisher. Zillow is not reliable, but if you don't care because it's free, have at it.
Posted by: Al | March 3, 2010 11:18 AM
Zillow is a complete joke. The "zestimate" really should be ignored. What they do provide are recent sales in the neighborhood. The lag time from sales data obviously affects the "zestimate". The best thing anyone can do is check to see sales in the last 6 months. If you see a home down the street sell for $200k and another for $210k, but the "zestimate" is $250k, you know the $250k is inaccurate. I also would not necessarily trust Zillow with the sales prices they list. You can verify the sales price on mdlandrec.net or the real property search at http://sdatcert3.resiusa.org/rp_rewrite/
Once you see on Zillow which properties sold and their addresses, you can type it in the real property search to confirm. Do not go by assessed value by the county/state. If you don't want to go through the hassle and all the work on your own, then you can always ask a realtor to send you a free CMA Report. Make sure to ask the realtor to include shortsales and REO's. Foreclosures from auction will not appear in the MLS so those sales will not be available.
Posted by: Frank Rizzo | March 4, 2010 10:17 AM
Online home valuation sites can be misleading to sellers and borrowers looking to buy or refinance. They would be better served by a real estate agent, lender, or appraiser with access to a database of comparable home sales that can be used to determine a value estimate.
Posted by: Ditech Home Loans | March 10, 2010 5:17 PM
We bought our home in April of 2007. I checked Zillows last year. It said we paid $158,000 for the house when we actually paid $125,000. Big difference in my eyes.
Posted by: Texas homeowner | March 16, 2010 10:26 PM
Zillow can't see into a home, so home improvements are not included in Zillow's estimates.
Zillow is UNDER estimating my home by about $11,000.
Posted by: sally | August 1, 2010 12:03 PM
Waterfront is clearly and blatantly overlooked by Zillow. In the area where I live, all waterfront properties are premium priced, but Zillow does not account for this. They only provide a small checkbox among other much less substantial factors and they only use comparable sales to come up with their zestimates. At least that's how it appears to me. What I can say is that if you own a waterfront home and most of the other homes in your neighborhood are not waterfront, your home will be vastly under-zestimated, because in their secret formula they base the value of your home on comparables in the immediate neighborhood. When I had a realtor do a CMA, she had to go pretty far out of my neighborhood for comps, and she admitted that was an unusual thing for her to have to do, but there wasn't anything in my neighborhood to compare my home to because it was the only one on the water, and my home has about 500 feet of water frontage, compared with pretty much zero for all the other homes in the neighborhood. Zillow fails to deal with this at all. This is a problem.
Posted by: Stanley | August 25, 2010 11:25 PM
The Zillow data is poorly accumulated in many instances. One area in Dublin, OH the builder I work for built many homes, they list half the homes as 3 BR 1.5 bath. These are 4/5 BR and 3 to 4 bath homes. The Zestimates are about 75% of a few months ago in a area that has actually trended slightly upward over the last 6 mos. Garbage in = Garbage out. sad thing is, many people actually think their zestimates are close to accurate.
Posted by: JIM | September 19, 2010 5:13 PM
I agree with Stanley above that Zillow totally underestimates the value of waterfront property.In my experience, it valued the houses across the street from ours (they have no waterfront access) at almost the same price as our waterfront home.
And it valued some run down waterfront houses, with less waterfront property, at more than our house.
It seems to focus on square feet of house, number of bedrooms and baths.With waterfront,most of the value, sometimes 80 percent, is the value of the property.Zillow's methodology seems to ignore that.
I wish I could buy waterfront property for the values they are assigning!
Posted by: TG | December 5, 2010 4:23 PM
With so many variables to consider, I would think that an estimate within 10% of actual value would be fairly remarkable. There are just too many things to consider when estimating value, including debt and lending markets. I can't imagine even the most sophisticated models could develop consistently accurate results.
Posted by: Plumber Plano Tx | January 10, 2011 3:11 PM
Zillow is a joke. Their numbers are dramatically off and never at the purported proper level of valuation. The lag in data, combined with their weak association data model is laughable. Anyhow, here is a great personal example:
My wife and I bought a foreclosed home that sits on a 1/4 acre lot off of the bank in San Diego, CA for $335K. There was an appraisal on the home by the bank's approved appraiser at the time of the sale for $390K. We then gutted and remodeled the house in its entirety. The home was then re-appraised and it came in at $440K. Meanwhile, Zillow claims our home is worth $330K. Not to mention that is has the wrong SF, number of bedrooms, number of bathrooms, and lot size listed. And to boot, if I look into the comparables they are using to evaluate it, it is cross referenced against condos and townhomes instead of single family residences.
I ask you this: with no real knowledge of condition and finishes, lagged data, and a weak association model, how could Zillow possibly have a proper valuation?
Posted by: Davey Blue | February 4, 2011 6:57 PM
Hi,
For my area, Zillow has my home's value listed as very low. I am sort of happy right now because I'm going through a divorce. So, there's nothing for my husband to try to get, if he tried (god forbid). He didn't pay anything toward the house, but he can still try to sue for equity.
Anyway, I checked out Zillow's "recent sales" listings for my area. They had MUCHOS mistakes. Townhouses accidentally listed with hundreds of acres, condos and townhouses listed as single family homes and vice versa. There were also discrepancies in the number of bedrooms and baths. I actually emailed the webmaster using their link to report errors several months ago, but they haven't corrected any of the errors.
If they're using their own data to automatically calculate the values, then of course they're incorrect. Zillow probably has some software automatically calculating values based on the data in their system. But if the data is not updated, is incorrect etc. Of course all their valuations will be wrong. The "zestimate" will not replace a qualified appraiser anytime soon.
Posted by: JuanitaBeasley | February 8, 2011 1:31 PM
I reviewed the Zillow site.
THEY ARE SO OUT OF WHACK IN VALUATION, IT"S NOT FUNNY.
On my street, 5 of the Same size House, Same exact style of House, Same Lot Size, Same Backyard View!
5 DIFFERENT VALUES ($70,000. Difference). Yeah,,they have down to a science,,,"WEIRD SCIENCE"!
Posted by: K George Lucas | January 18, 2012 6:31 PM