Testing whether weather changes how people search for hotels
A search result page experiment for the Expedia Hotels conversion rate optimization team, designed so that every variant would teach us something whether it won or lost.
| My role | Designer on the test. I owned variant design, the design exploration, and the specs handed to developers. |
|---|---|
| Team | Hotels conversion rate optimization: product manager, analysts, content strategy, engineering. |
| Funnel stage | Consideration, inside hotel search results. |
| Platform | Responsive web, mobile first. |
| Tools | Design toolkit components, responsive specs, [add testing platform]. |
The problem
Travelers search for beach and ski destinations on dates when the weather will not cooperate. They see the results, something feels wrong, and they leave without changing their search. The search box is right there, but nothing prompts them to use it again.
The team framed this as a search error that the product never tells you about. The user has made a reasonable request and received a technically correct answer that will disappoint them in person, months later.
Constraints that shaped the design
This was not an open design problem. The test had to be:
- Mobile-up responsive
- Isolated along one vector, so content, interaction, or UI changed, but not several at once
- Visually consistent with the existing UI toolkit
- Workable under localization
- Compliant with content strategy guidelines
Isolating a single vector is what makes a result readable. If a variant changes the copy and the layout and the interaction, a lift tells you almost nothing about why.
What we found in discovery
- The behavior only mattered for traditional beach and ski destinations, so the trigger needed to be narrow rather than global.
- The message belonged in the search area, because functionally it behaves like a search error.
- Historical weather data could be compared against the customer's chosen location and dates, so the trigger was technically feasible.
- Any element added had to preserve the peek of the first search result above the fold on mobile, which is a known driver of scrolling.
Hypothesis
If we show historical weather conditions alongside an open search box when a traveler searches a beach or ski destination in an off season window, then more travelers will revise their search instead of abandoning it, because the page gives them a reason and an obvious next action at the same moment.
Design exploration
Work included new icons, several ways to signal prominence, and testing across breakpoints. I explored iconography, temperature data, and plain language as competing ways to make the message register.
As is often the case when a test carries this many constraints, the final design came down to copy. The visual options kept colliding with localization or with the toolkit, and the words were the one variable we could move cleanly.
The variants
Control
Standard results list with the collapsed search box, no weather signal.
Variant 2
Weather messaging with no call to action, paired with an uncollapsed search box so revising the search needs no extra tap.
Leaving the call to action out of the second variant was deliberate. If the messaging alone moves behavior, the insight generalizes to other search errors. If it needs an explicit prompt, that is also worth knowing, and it is a different design problem.
What we set out to learn
The learning goals were written before the test ran:
- Is showing weather conditions alongside an open search box enough on its own to produce new search behavior?
- When travelers do run a new search, do they change the location or the date range?
The second question mattered more than the first. Location changes and date changes imply different products: one is a destination recommendation problem, the other is a flexible dates problem.
Outcome
To add: the test result, or a plain statement that the outcome is not mine to share. Growth leaders will ask, and a losing result with a clear reading is worth more than silence. If the numbers are unavailable, say so here and describe what the team did next.
What I took from it
Heavy constraints push the decision toward copy. That felt like a limitation at the time and now reads as a finding: when the layout, the toolkit, and localization are all fixed, words are the cheapest variable in the system, and they are usually under tested.
Writing the learning goals as questions, before the variants existed, is the habit I kept. It makes the uninteresting result impossible, because every outcome answers something.