Hotel Tech Report Updates Center logo

Updates Center

Empowering the next generation of hoteliers to embrace their greatest asset, technology.

Subscribe to Updates

Labels

  • All Posts
  • premium
  • PPC
  • lead generation
  • promo opportunities
  • rankings/ht score
  • review manager
  • other product updates
  • Awards
  • bug fixes

Jump to Month

  • May 2025
  • March 2025
  • February 2025
  • January 2025
  • December 2024
  • November 2024
  • September 2024
  • August 2024
  • July 2024
  • June 2024
  • May 2024
  • March 2024
  • January 2024
  • November 2023
  • October 2023
  • September 2023
  • August 2023
  • July 2023
  • June 2023
  • May 2023
  • April 2023
  • March 2023
  • February 2023
  • January 2023
  • November 2022
  • October 2022
  • August 2022
  • July 2022
  • June 2022
  • May 2022
  • April 2022
  • March 2022
  • February 2022
  • January 2022
  • December 2021
  • November 2021
  • October 2021
  • September 2021
  • August 2021
  • July 2021
  • June 2021
  • May 2021
  • April 2021
  • March 2021
  • February 2021
  • January 2021
  • November 2020
  • October 2020
  • September 2020
  • August 2020
  • June 2020
  • May 2020
  • April 2020
  • March 2020
  • February 2020
  • January 2020
  • December 2019
  • November 2019
  • October 2019
  • September 2019
  • August 2019
  • July 2019
  • June 2019
  • May 2019
  • April 2019
  • February 2019
  • November 2018
  • October 2018
  • September 2018
  • August 2018
  • July 2018
  • April 2018
  • October 2017
Release notes byAnnounceKit

Create yours, for free!

rankings/ht score
7 years ago

Minimum Review Threshold Update

The minimum review threshold is a variable in the HT Score algorithm that serves to level the playing field both for startups (via a ceiling on the impact of review quantity) and large companies (via a floor on the amount of reviews required to be considered a credible/trusted provider within the category):

  • Review quantity ceiling: Limit to the impact of review quantity to level the playing field for smaller companies who have great products that hoteliers love to shine (so bigger isn't always better)
  • Review quantity floor: Ensures that a company can't just get a handful of perfect scores from their friends/early clients and all of a sudden boost to the top of the rankings ahead of established and trusted products in the category with lots of customers who vouch for the product

Update August 2021: Minimum review threshold now includes a decay factor on old reviews per the HT Score Poseidon Algorithm Update (Q3 '21)

TL;DR

  1. The 'Minimum Review Threshold' is just a fancy way of saying 'the average number of reviews' for products in a given category 
  2. The algorithm only looks at products with enough reviews to be considered significant and trusted by buyers (30 reviews+)
  3. It protects against scenarios with limited sample sizes (ex. a company who gets 1x 5-star review won't be able to outrank a company with 200 x 4.5-star reviews)
  4. It creates a level playing field and protects against larger companies always being top rated (bigger isn't always better)
  5. Quantity is a factor in the HotelTechScore but only to the extent that a given product needs to meet the avg. # of reviews in the respective category to be able to be compared side-by-side with other established companies

Related articles: How is the HotelTechScore calculated?




__

Let's dive into the details, rationale and an example of how the Minimum Review Threshold works, shall we...

Reminder of what the minimum review threshold is in the scoring algorithm
If you recall from thearticle on how the HotelTechScore is calculatedone critical element of the score is the minimum review threshold.  The minimum review criteria is quite simply a check and balance that is meant to ensure that there is a sufficient amount of customer feedback to help inform the score.  So for example, if a product has all 5* ratings but only one customer review...they shouldn't have a high score relative to a company that has 4* and 5* ratings but 20 customer reviews.  

This is where the minimum review threshold came in.  It was a way to ensure that products without enough information/insights weren't jumping ahead of companies with lots of high quality reviews to help buyers.  The way we did that initially was to take the average number of reviews for all of the companies that had claimed profiles on Hotel Tech Report and essentially set that as the bar for how many reviews a company needed at a minimum and if a product didn't meet that minimum threshold, their score would get docked to ensure that they didn't move past companies with lots of high quality reviews (for more detail please revisit thearticle).

 - The Problem -
The problem that was brought to our attention by a few top rated HTR vendors (thanks Mackenzie and Alex!) is that in some categories this still leaves certain companies who don't have many reviews looking as good as companies with lots of reviews (and sometimes even passing them).

When we looked into these specific cases we realized that the issue lied in the fact that the minimum review threshold was being calculated site wide and that in several circumstances the problem that we were trying to fix with the minimum review threshold criteria was still happening in certain scenarios.

 - Why it was happening -
The minimum review threshold used to be site wide so in some more developed categories with companies with lots of reviews, the threshold was relatively low which meant getting a few reviews could boost your score a lot very quickly.

 - The Solution -
Change the minimum review threshold to be by category.  After the update, the part of the algorithm criteria that is the minimum review threshold will switch from all HTR profiles, to being calculated within each category.

 - How will this impact you? - 

(a) If your company has a similar amount of reviews relative to others in your category, you will likely see your score go up
(b) For companies who seem to have a high score currently but only a few reviews, your score will probably go down

 - What if I'm in the latter (aka. camp 'b')? -
If you thought you could sneak away with a high score and very few reviews...sorry to rain on your parade...but this is a good thing, we promise! Plus, if you want to prevent your score from decreasing, all you have to do is reach out to customers and get some more reviews! (note: the update goes live Friday 10/27)


Let's look at an example to see how this works...

Angela works at a hypothetical company called Guestly who makes guest happiness software (category also hypothetical...).  Now for Angela to hit the minimum review threshold, she will need to be above the average amount of reviews for profiles in her category with more than 1 review.  So here are the review counts for them:

  1. Happy Guest- 27 reviews
  2. GuestSatis - 13 reviews
  3. Better Toguester - 3 reviews
  4. Guestly - 5 reviews

To meet the criteria for the minimum review threshold we'll take the average of these review counts (27+13+3+5)/4 companies=12.

So in this case, only GuestSatis and Happy Guest meet the minimum review threshold so Guestly will need to get 7 more reviews to check this box and bump their score along this criteria (again, the minimum review threshold is one of several criteria...but it is an important one).  Remember, Hotel Tech Report doesn't rank based on who the best company is (that's a subjective measure) but ranks the companies that provide the most information and transparency and who have the most customers that love their products and services enough to share their experience with others.


Still have questions?

If you have any feedback, ideas or thoughts on how we can continue to improve the HotelTechScore ranking (or anything else!) like Mackenzie and Alex, please reach out any time via the live chat on Hotel Tech Report or by posting on our communityfeature request board.


May 2020 Update: Average for the minimum review threshold calculation now only includes companies with >30 reviews Learn more

Avatar of authorAdam Hollander