Open methodology
15 publications, based in the United States, the United Kingdom and Germany.
Every day, a program reads each publication's public feeds and sitemaps and picks out new phone reviews. It identifies itself as BestSmartphoneBot, obeys every site's robots.txt, and fetches one page at a time. For each review it records the publisher's own score and any award, plus a short summary in our words of the verdict, strengths and weaknesses. The text of the review is not stored or republished; every review is linked.
Publishers use different scales, and they use them differently: some give most phones four stars out of five, others spread their scores widely. A simple average would let the most generous publisher decide the rankings. So each score is first converted to 0–100, then compared with that publisher's own history of scores. A phone that scores well above a publisher's typical score counts for more than one that matches it. While a publisher has few scores on record, this adjustment is small.
The consensus score is the average of those adjusted scores, with more recent reviews weighted more heavily (a review loses half its weight after a year). It needs scored reviews from at least 2 independent publishers. Publications that give picks rather than scores count towards a phone's review and award totals.
One owner, one voice. Some publications belong to the same company — Tom's Guide and TechRadar (Future plc), for example, or Android Police and Pocket-lint (Valnet). Sister publications share editorial standards and sometimes reviewers, so they aren't independent opinions. Their scores are averaged into a single voice before the consensus is calculated, and agreement is measured across independent voices. Two sister sites giving the same score therefore count once.
Next to every score we show how far apart the reviewers are. “Reviewers agree” means the adjusted scores sit within about five points of each other; “Reviewers split” means they are more than ten points apart, and that phone is worth reading about in more than one place.
A phone joins the rankings once at least 2 independent publishers have reviewed it (sister publications count as one). It leaves them when it has had no new review for about eighteen months; its page stays up.
Specs — launch price, form factor, display, cameras, battery and so on — come from the spec boxes in the reviews we cite. Each review is read separately; where they differ, we show the value most of them give, and implausible values are discarded. Prices are US launch prices for the base storage model, not today's street price.
Lists such as “best phones under $500” are filters over the same consensus ranking. The camera and battery lists are different: they rank by the camera or battery scores publishers give in their reviews, because a spec like megapixels or battery capacity says little about how a phone actually performs. A list is published only once it has at least three phones in it.
“Most popular” means the phone pages read most on this site over the last seven days. Each visit to a phone page adds one to that phone's count for the day — nothing else. We use no cookies for this and store nothing about you: no address, no identifier. Bots, link previews and quick reloads aren't counted. Popularity never affects a phone's score.
Anything the program is unsure about — an unfamiliar name that might be an existing phone, a score that doesn't match the publisher's own page data, a large change in a publisher's score — is held back until a person has looked at it.