You need to validate an RMS before you put one in. Fact. The problem is - there is no commercial tool to help you do this today and there is little know-how. There is no standardised methodology to even begin to do it and what does exist is a range of imperfect - and that’s being generous - comparisons performed ad hoc.
Ultimately, hoteliers need to figure out what the return on investment (ROI) is for any given RMS. Only big corporate chains with deep pockets have been able to tackle the conundrum thus far. But for smaller chains and independents the sensational marketing of the revenue management vendors is too often dictating the decision making process. Plenty of salivating numbers are plastered all over RMS sales pitches. They look exciting. They are aggressive. The system is the best system ever.
So an executive team gets sold on the idea and signs off on all the contracts. No testing. No validation. Now a lack of trust in validation has led to the RM vendor space becoming a horse trading bazaar of sorts.
For example, a hotelier may decide to choose a ‘cheaper’ RMS system because they think they are saving money but, in fact, they are overlooking the fact that performance is EVERYTHING. A difference in revenue gains of one or two percent between one RMS and another is a HUGE difference, so much so that the price paid for the system itself becomes almost irrelevant. Mathematically, you should pay more for the RMS that gets you the better performance, even if it’s just a 1% boost. The better ROI will more than offset the extra cost. Every time.
“ A difference in revenue gains of one or two percent between one RMS and another is a HUGE difference, so much so that the price paid for the system itself becomes almost irrelevant”
But smaller brands don’t have the scale or the resources and too often ignore this statistical truism even though they shouldn’t. Ironically, it’s the smaller groups that will benefit the most from getting an RMS. Most hotels still do not use an RMS at all and it’s the pricing capability of the solution which has the biggest impact and not the inventory controls and other trimmings championed by many software companies. It all starts with actually being able to determine what the performance of an RMS actually was and hoteliers need an affordable tool to help with that.
Field experiments are everything
To validate an RMS properly you need to do robust AB testing or, in the academic parlance, ‘field-experimentation’. Field experiments means controlled experiments and the best analogy to use in a post-Covid world is that it’s like testing a vaccine for efficacy. The RMS you are testing is the equivalent of the vaccine being trialled and, as most of us now know, there are reasons why you don’t just simply give the vaccine out to a bunch of people and test if it works. Technically, that’s just not how a proper trial works. You need to do a controlled experiment that includes inoculating people with a placebo and so on and so forth.
So we can agree that you need to test different revenue strategies in the field. Amazon, Uber and Netflix run these experiments all the time. Even AirBnB is doing AB testing. Meanwhile, the vast majority of hotels never test their revenue strategies in the real world. Field experimentation is possible and we must start doing it…
Hoteliers are already testing
Few field experiments have been done in hospitality for revenue management and the little that has been done has not been well documented publicly. There’s one here and another one here.
But there is hope! At the corporate end of the hospitality industry, the Hiltons and Marriotts will always do a controlled experiment to validate an RMS before they upgrade or switch. The problem is that this involves hiring teams of data scientists who run a statistical appraisal over several months! A DIY tool doesn’t currently exist so hiring an entire consulting team of ten people to run the tests internally is what ends up happening.
Absent the same resources, smaller hotels are left to run simple and crude tests on the efficacy of an RMS. They may use RevPar…but that’s not ideal as there could be market impacts. Or perhaps they go a step further to remove any influence the market may have by looking at RGI? Once again, this would not be academically prudent as there is no way of knowing if the numbers are being influenced by a competitor’s new refurb or their new marketing hire. And so the search continues…
A promising reality is that hotel marketing departments already do field experiments. CRM systems often use them and serving customers different emails to see what garners the best reaction is, technically, a field experiment. If hotels are already used to doing AB testing on their ancillary pricing or on promotions or on marketing, then they are more likely to demand it of their RMS providers in the future. But right now, field experimentation is not something near the top of what is required when choosing an RMS…it should be!
What we would like to see
The entire hospitality industry should have access to the know-how and have access to affordable tools to independently validate the performance of a revenue management strategy. Currently, we do not even have a clear way in the hospitality industry to do an ROI calculation for an RMS. This must change.
Share the knowledge! Big corporate groups should publish their experiments as it would lead to more know-how in the RM community. What can smaller hotels and properties learn from these market leaders in field experimentation?
Impact Analysis. This would mean you could validate an RMS affordably and perhaps one data scientist, instead of Hilton’s team of nine - five data scientists, three IT guys and a project lead.
There are challenges but it’s more than possible.
How to perform Impact Analysis
What kind of designs and strategies do we imagine a good tool would incorporate? To get the ball rolling, we designed three complementary ways to start testing any given RM strategy. They become increasingly granular but with the trade off of becoming more complex in terms of design and implementation.
Property Splits. This is what the big chains do when they run RM tests and the rough idea is that you put one half of your properties on one strategy and the other half on another and test the results. It’s good for looking at a whole group but its not so granular and for smaller groups and individual properties it’s not that relevant.
Alternating Periods. We can start to get a bit more granular and look at the property itself by running different strategies on different reservation nights. You can also combine it with multiple periods. What should be the length of the test period? One week? Two? Three months? Seasonality effects are an issue but you can control for that.
- High Frequency Price Updates. This is the most complex one to pull off but it allows much more granularity as you can start to look at single stay nights. In short, you could allocate half of your inventory to one algorithmic strategy and the other half to another strategy, which allows you to answer more granular questions and you can change it rapidly (hourly).
The first design - Property Splits - could be implemented manually and an RMS could help with the data visualization with custom-built tools. In terms of analytics, one would need to build custom dashboards targeting experiment design and analysis. That’s where a customisable business intelligence (BI) tool comes into play - hello Pace Analytics!
We could start with those hotels that have the right PMS (cloud, two way integrations etc.) and start to provide some technical solutions that will help reduce the cost so that hoteliers can start validating their RM strategy. Pace even has a way of offering a system to do the second design - Alternating Periods - but it needs improvement.
Hoteliers can also play their part by adding an initial step to their RMS shopping process - demanding validation from their vendors and asking for help with that validation. We need to work together and build a solution that helps push the RM industry forward and putting some fire under our toes will help.
However, we need to go beyond customisable BI and to a future where these kinds of dashboards are an industry standard and accepted throughout the industry. Wider adoption and bigger changes will take several years but we must start now as, ultimately, it will be transformational for hospitality. It boils down to what the return on investment of a given RMS is. Because if the ROI is 5 or 6% then the price tag shouldn’t matter.
You can read more about our approach to Impact Analysis in a recently published white paper. Or learn more in the below online presentation.