Scraping Hotel Data with Python (2026 Guide)

TL;DR

Hotel data scraping turns public prices, listings, amenities, and availability into dated records. The tested Python workflow below extracts one Expedia search and The Hoxton Rome's official room profile, validates both with Pydantic, and exports a four-row market snapshot. It keeps nightly prices, stay totals, taxes, dates, occupancy, source URLs, and retrieval times separate because a valid number without that context is unreliable.

What is hotel data scraping?

Hotel data scraping is the extraction of facts visibly published on hotel, online travel agency, review, and comparison pages. A useful row may contain a property name, room label, displayed price, availability message, amenities, review score, cancellation text, source URL, and retrieval time. The search inputs belong beside the result: destination, check-in and checkout dates, guests, rooms, locale, and currency.

Those details decide whether the row means anything. A hotel price can describe one night or the whole stay. Taxes may be included, excluded, or listed separately. A promotional rate may require membership. Inventory can change before the next page load. Drop those distinctions and a clean CSV can still tell the wrong story.

The example stays on public sources. It does not sign in, inspect a traveler's booking, call private endpoints, or reproduce a booking flow. Expedia Rapid and Booking.com Demand are separate official products for approved integrations. Only the ScrapeGraphAI code below was executed.

Which hotel fields are worth collecting?

Start with the decision, not an oversized hotel object. A revenue analyst comparing visible rates needs different data from a researcher mapping amenities. Keep the observation narrow and retain source text until its meaning is clear.

For price research, collect:

• property name and visible location;
• check-in, checkout, adults, children, and rooms;
• the complete displayed price text;
• nightly price only when the page labels it as nightly;
• total-stay price only when the page labels it as a total;
• taxes and fees as their own field;
• room type, meal plan, cancellation terms, and promotion conditions;
• display currency, locale, source URL, and UTC retrieval time.

For listing and property research, the useful fields shift toward address, room categories, amenities, contact page, public phone number, check-in policy, and a short source-backed description. Aggregate review scores and counts may help a market study, but reviewer names and account data usually do not.

Avoid a giant catch-all schema. It encourages the model to fill a field because the field exists, even when the page does not show a value. Incomplete but honest data is easier to review than a polished guess.

Choose the source before the extraction method

Hotel information is split across several kinds of pages. Pick the one that actually publishes the fact you need.

Source	What it is good at	Main limitation
OTA search page	Comparable visible offers for one destination and stay	Results, prices, experiments, and inventory change quickly
Direct hotel website	Room names, amenities, policies, contact details, and direct offers	Hard to normalize across different site structures
Review site	Aggregate reputation and category scores	Creative review text and personal data need extra care
Official partner API	Contracted inventory, stable fields, support, and booking workflows	Approval, commercial terms, and product constraints apply
Hotel data scraping service	Managed collection and normalization	You still own source rights, field definitions, and quality checks

An OTA search can put several properties under the same dates and occupancy. A direct hotel site is the better source for its own room names or contact details. Mixing both is fine, but the rows are not interchangeable. The CSV below marks offers and the property profile with explicit record_type values.

If Booking.com is the only source in scope, use the focused Booking.com scraping guide. For airline and metasearch access, the KAYAK API guide covers a different partner-gated product. This page owns the broader problem: choosing, validating, and normalizing general hotel-market data.

Official hotel APIs versus public-page extraction

If customers will search or book inside your product, start with an official API. That is the route built for licensed availability, stable identifiers, contracted usage rights, and a supported booking journey.

Expedia Group's Rapid Lodging Shopping documentation describes an approved partner product for shopping lodging properties and rooms. Booking.com's Demand API accommodation documentation covers accommodation search and related travel-product integration for eligible partners. Your account, contract, and provisioned documentation determine what you can call and how the data may be used.

There is no official API request code in this guide. Public documentation does not tell us what credentials, markets, commercial allowances, or production conditions a specific reader has. Inventing an endpoint or quota would make the article look more complete while making it less useful.

Public-page extraction has a narrower job: record what an accessible page displayed for a defined search at a recorded time. That can serve research, visible-price monitoring, catalog checks, or a small hospitality data extractor. Contracted inventory and booking operations belong elsewhere.

Requirement	Official partner API	Public-page extraction
Customer booking flow	Preferred	Wrong boundary
Stable product identifiers and fields	Contract-dependent, but designed for integrations	Page structure can change
Small visible market snapshot	May be more access than the job needs	Often a practical fit when permitted
Authenticated or private inventory	Use only through approved access	Never attempt to obtain it
Source-linked observation	Add provenance in your system	Natural fit when URL and time are retained

ScrapeGraphAI does not grant API entitlement or permission to collect a page. It also does not make a challenge, sign-in wall, or refusal something to work around. If a source stops exposing the required public fields, the workflow should stop or use a permitted fallback.

Design the schema before the prompt

Before touching the SDK, define one row. Here it is a visible Rome hotel offer for September 15 to 16, 2026, two adults, and one room. Property name, displayed price, and three-letter currency are required. Nightly price, stay total, taxes, review score, and policy text remain optional because a card may omit them.

Prices remain strings. That preserves $326 nightly and $376 total instead of flattening both to a number with no meaning. A production pipeline can parse amounts later, after it has established the price type, currency, tax treatment, and promotion rules.

The direct-source profile requires a hotel name, address, at least one amenity, and at least one named room type. Check-in time stays optional. The Hoxton room page did not publish one in the extraction context, so the code stores None instead of borrowing a time from another page.

source_url and retrieved_at are excluded from the extraction schema. The script adds the input URL and the UTC clock after model_validate. A language model should not invent provenance that the application already knows.

Set up the tested Python workflow

On July 29, 2026, the final run used a clean Python 3.12.12 environment with scrapegraph-py==2.1.0, Pydantic 2.13.4, and pandas 2.3.3. The ScrapeGraphAI key existed only as the process-level SGAI_API_KEY.

python -m pip install "scrapegraph-py==2.1.0" "pydantic==2.13.4" "pandas==2.3.3"
export SGAI_API_KEY="your-key"

The Python SDK documentation is the source for the current client and FetchConfig interface. The companion Colab asks for the key with getpass, so it is neither echoed nor stored in notebook output.

Start with the imports, inputs, and Pydantic models.

import json
import time
from datetime import datetime, timezone
from typing import Any
 
import pandas as pd
from pydantic import BaseModel, Field, ValidationError, field_validator
from scrapegraph_py import FetchConfig, ScrapeGraphAI
 
MISSING_TEXT = {"N/A", "No content available", "Not available", "XXX"}
CHECKIN = "2026-09-15"
CHECKOUT = "2026-09-16"
DESTINATION = "Rome, Italy"
ADULTS = 2
ROOMS = 1
 
SEARCH_SOURCES = [
    (
        "Expedia",
        "https://www.expedia.com/Hotel-Search?destination=Rome%2C%20Lazio%2C%20Italy&startDate=2026-09-15&endDate=2026-09-16&adults=2&rooms=1",
    ),
    (
        "Hotels.com",
        "https://www.hotels.com/Hotel-Search?destination=Rome%2C%20Lazio%2C%20Italy&startDate=2026-09-15&endDate=2026-09-16&adults=2&rooms=1",
    ),
    (
        "Booking.com",
        "https://www.booking.com/searchresults.en-gb.html?ss=Rome%2C+Italy&checkin=2026-09-15&checkout=2026-09-16&group_adults=2&no_rooms=1&group_children=0&selected_currency=EUR",
    ),
]
PROFILE_URL = "https://thehoxton.com/italy/rome/rooms/"
 
 
class HotelOffer(BaseModel):
    property_name: str = Field(min_length=1)
    location: str | None = None
    room_type: str | None = None
    displayed_price: str = Field(min_length=1)
    nightly_price: str | None = None
    stay_total: str | None = None
    taxes_and_fees: str | None = None
    currency: str = Field(min_length=3, max_length=3)
    review_score: float | None = Field(default=None, ge=0, le=10)
    cancellation_terms: str | None = None
    availability_note: str | None = None
    property_url: str | None = None
 
    @field_validator(
        "location", "room_type", "nightly_price", "stay_total",
        "taxes_and_fees", "cancellation_terms", "availability_note",
        "property_url", mode="before",
    )
    @classmethod
    def clean_missing_text(cls, value: Any) -> Any:
        if isinstance(value, str) and value.strip() in MISSING_TEXT:
            return None
        return value
 
 
class HotelSearchResults(BaseModel):
    offers: list[HotelOffer] = Field(min_length=1, max_length=5)
    destination: str = ""
    checkin: str = ""
    checkout: str = ""
    adults: int = 0
    rooms: int = 0
    source_name: str = ""
    source_url: str = ""
    retrieved_at: datetime | None = None
 
 
class HotelProfile(BaseModel):
    hotel_name: str = Field(min_length=1)
    description: str | None = None
    address: str = Field(min_length=1)
    amenities: list[str] = Field(min_length=1, max_length=20)
    room_types: list[str] = Field(min_length=1, max_length=20)
    check_in_time: str | None = None
    check_out_time: str | None = None
    contact_url: str | None = None
    phone: str | None = None
    source_url: str = ""
    retrieved_at: datetime | None = None
 
    @field_validator(
        "description", "check_in_time", "check_out_time",
        "contact_url", "phone", mode="before",
    )
    @classmethod
    def clean_missing_text(cls, value: Any) -> Any:
        if isinstance(value, str) and value.strip() in MISSING_TEXT:
            return None
        return value

Use one guarded JavaScript retry

Make one normal request. If it fails or Pydantic rejects the required fields, make one retry with FetchConfig(mode="js", stealth=True, wait=2000). Two failures end that source. The retry renders a public page; it is not a license to keep pressing a blocked one.

Known placeholder strings become None. Offers without the three required strings are discarded. The cleanup also rejects malformed property URLs observed during testing rather than publishing them as links.

def schema_without(model: type[BaseModel], fields: set[str]) -> dict[str, Any]:
    schema = model.model_json_schema()
    for field in fields:
        schema["properties"].pop(field, None)
    schema["required"] = [
        field for field in schema.get("required", []) if field not in fields
    ]
    return schema
 
 
def parse_json_data(response: Any) -> Any:
    data = response.data.json_data
    return json.loads(data) if isinstance(data, str) else data
 
 
def keep_usable_offers(payload: Any) -> Any:
    if not isinstance(payload, dict) or not isinstance(payload.get("offers"), list):
        return payload
 
    usable = []
    for offer in payload["offers"]:
        if not isinstance(offer, dict):
            continue
        property_url = offer.get("property_url")
        if isinstance(property_url, str) and (
            "%22" in property_url or "\\" in property_url
        ):
            offer = {**offer, "property_url": None}
        required = (
            offer.get("property_name"),
            offer.get("displayed_price"),
            offer.get("currency"),
        )
        if any(
            not isinstance(value, str)
            or not value.strip()
            or value.strip() in MISSING_TEXT
            for value in required
        ):
            continue
        usable.append(offer)
    return {**payload, "offers": usable[:5]}
 
 
def extract_with_retry(
    sgai,
    *,
    prompt,
    url,
    schema,
    model,
    clean_payload=False,
):
    errors = []
    for attempt in range(2):
        options = {"url": url, "schema": schema}
        if attempt == 1:
            options["fetch_config"] = FetchConfig(
                mode="js", stealth=True, wait=2000
            )
        response = sgai.extract(prompt, **options)
        try:
            if response.status != "success":
                raise RuntimeError(response.error or response.status)
            payload = parse_json_data(response)
            if clean_payload:
                payload = keep_usable_offers(payload)
            validated = model.model_validate(payload)
            return validated, {
                "attempts": attempt + 1,
                "javascript_retry": attempt == 1,
                "errors_before_success": errors,
            }
        except (RuntimeError, ValidationError, ValueError, TypeError) as error:
            errors.append(f"{type(error).__name__}: {error}")
            if attempt == 1:
                raise RuntimeError(
                    f"Extraction failed after two attempts: {errors}"
                ) from error
    raise RuntimeError("unreachable")

Extract public hotel offers for one stay

The source ladder starts with a public Expedia Rome search. Hotels.com comes next, followed by the previously tested Booking.com search. The first source with a valid offer after the allowed attempts wins.

search_prompt = (
    "Extract up to five hotel offers visibly listed for Rome, Italy, for "
    "check-in 2026-09-15, check-out 2026-09-16, two adults and one room. "
    "For each offer copy the property name, location, room type, the complete "
    "displayed price text, nightly price only when explicitly labeled, total "
    "stay price only when explicitly labeled, taxes and fees, three-letter "
    "currency, review score, cancellation terms, availability note, and "
    "property URL. Do not calculate prices or infer missing values."
)
search_schema = schema_without(
    HotelSearchResults,
    {
        "destination", "checkin", "checkout", "adults", "rooms",
        "source_name", "source_url", "retrieved_at",
    },
)
failures = []
 
with ScrapeGraphAI() as sgai:
    for source_name, source_url in SEARCH_SOURCES:
        try:
            raw_search, search_meta = extract_with_retry(
                sgai,
                prompt=search_prompt,
                url=source_url,
                schema=search_schema,
                model=HotelSearchResults,
                clean_payload=True,
            )
            search = raw_search.model_copy(
                update={
                    "destination": DESTINATION,
                    "checkin": CHECKIN,
                    "checkout": CHECKOUT,
                    "adults": ADULTS,
                    "rooms": ROOMS,
                    "source_name": source_name,
                    "source_url": source_url,
                    "retrieved_at": datetime.now(timezone.utc),
                }
            )
            break
        except RuntimeError as error:
            failures.append({"source": source_name, "error": str(error)})
    else:
        raise RuntimeError(f"All search sources failed: {failures}")

The exact published blocks ran together in the clean environment at 20:19 UTC on July 29. Expedia returned three offers: The Sanctuary Urban Retreat at $307 nightly and $352 total, Ostia Antica Park Hotel at $81 nightly and $103 total, and LF Borgo Trastevere at $157 nightly and $186 total. Each card included a low-availability message. No tax string was visible in the extracted fields, so taxes_and_fees remained empty.

Public Expedia search used by the workflow, captured July 29, 2026. Result cards are dynamic, so the image documents the search context rather than a permanent ranking. Open the live search.

The manual browser check found the same three property names and all six extracted price strings on the rendered source. The dates, occupancy, nightly labels, and total labels also matched. An earlier rehearsal had returned a different trio only 25 minutes before. That is why the CSV keeps the timestamp and never treats the order as durable.

Extract a hotel profile from its official website

OTA cards answer the market-search question. For property-controlled details, the second request reads The Hoxton Rome rooms page. Comparative prices are deliberately outside that prompt.

profile_prompt = (
    "Extract the hotel name, a concise factual description, street address, "
    "amenities, named room types, check-in and check-out times, contact page "
    "URL, and public phone number from this official hotel website. Copy only "
    "facts visibly published on the page. Do not infer missing values."
)
 
with ScrapeGraphAI() as sgai:
    raw_profile, profile_meta = extract_with_retry(
        sgai,
        prompt=profile_prompt,
        url=PROFILE_URL,
        schema=schema_without(
            HotelProfile, {"source_url", "retrieved_at"}
        ),
        model=HotelProfile,
    )
profile = raw_profile.model_copy(
    update={
        "source_url": PROFILE_URL,
        "retrieved_at": datetime.now(timezone.utc),
    }
)

The final profile contained the address at Largo Benedetto Marcello 220, the public Rome phone number, 18 amenity or policy strings, and eight room labels: Shoebox, Roomy, Cosy, Cosy • Up, Roomy • Balcony, Roomy • Terrace, Biggy, and Biggy • Terrace. The room labels, address, phone, and Flexy Time text matched the rendered source.

One extracted string did not clear review. Free cancellation up to 2pm the day before arrival landed in amenities, while the refreshed page exposed only a broader flexible-cancellation link in the reviewed DOM. The row stays in the raw snapshot for reproducibility, but that field needs quarantine rather than publication as a verified hotel policy.

Official room categories from The Hoxton Rome, captured July 29, 2026. View the live rooms page.

Build a normalized multi-source market snapshot

Keep the source boundary in the table. Each Expedia result becomes an offer row, while the Hoxton page becomes one profile row. A column that does not apply stays empty. It never inherits a value from another source just to complete the table.

rows = []
for offer in search.offers:
    rows.append(
        {
            "record_type": "offer",
            "hotel_name": offer.property_name,
            "destination": search.destination,
            "checkin": search.checkin,
            "checkout": search.checkout,
            "adults": search.adults,
            "rooms": search.rooms,
            "currency": offer.currency.upper(),
            "room_type": offer.room_type,
            "displayed_price": offer.displayed_price,
            "nightly_price": offer.nightly_price,
            "stay_total": offer.stay_total,
            "taxes_and_fees": offer.taxes_and_fees,
            "location": offer.location,
            "review_score": offer.review_score,
            "availability_note": offer.availability_note,
            "cancellation_terms": offer.cancellation_terms,
            "amenities": None,
            "check_in_time": None,
            "check_out_time": None,
            "contact_url": offer.property_url,
            "source_url": search.source_url,
            "retrieved_at": search.retrieved_at.isoformat(),
        }
    )
 
rows.append(
    {
        "record_type": "profile",
        "hotel_name": profile.hotel_name,
        "destination": DESTINATION,
        "checkin": None,
        "checkout": None,
        "adults": None,
        "rooms": None,
        "currency": None,
        "room_type": " | ".join(profile.room_types),
        "displayed_price": None,
        "nightly_price": None,
        "stay_total": None,
        "taxes_and_fees": None,
        "location": profile.address,
        "review_score": None,
        "availability_note": None,
        "cancellation_terms": None,
        "amenities": " | ".join(profile.amenities),
        "check_in_time": profile.check_in_time,
        "check_out_time": profile.check_out_time,
        "contact_url": profile.contact_url,
        "source_url": profile.source_url,
        "retrieved_at": profile.retrieved_at.isoformat(),
    }
)
 
frame = pd.DataFrame(rows)
csv_path = "hotel_market_snapshot.csv"
frame.to_csv(csv_path, index=False)
 
reloaded = pd.read_csv(csv_path)
assert len(search.offers) >= 1
assert len(reloaded) == len(search.offers) + 1
assert reloaded["hotel_name"].notna().all()
assert reloaded["source_url"].str.startswith("https://").all()
assert reloaded["retrieved_at"].notna().all()
assert not reloaded.astype(str).apply(
    lambda column: column.str.contains("sgai-", case=False, regex=False).any()
).any()

The checked file has four rows: three offers and one profile. All four have a hotel name, HTTPS source, and UTC retrieval time. The export test also scans for the credential prefix because a notebook must not leak its key into its own dataset.

This is a compact market snapshot, not a unified hotel catalog. A real dataset would also need source-specific identifiers, deduplication rules, locale tracking, raw response retention, change history, and a review queue for ambiguous rows.

A valid schema can still be semantically wrong

Pydantic verifies structure. It can require a nonempty price string, keep a score between zero and ten, and reject an empty offer list. It cannot see whether $376 total was mapped into nightly_price, whether Rome City Centre is a hotel name, or whether a price belongs to a different card.

The expensive mistakes are usually the rows that look tidy:

• a stay total lands in the nightly field;
• a tax line is attached to the next property;
• an old price and a new availability message share one row;
• a neighborhood becomes the property name;
• a promotion requiring membership is stored as a public rate;
• the three-letter currency is inferred from the traveler's location rather than the page.

Review a sample against the rendered source. Keep the exact displayed text and screenshot or source capture allowed by your policy. Quarantine ambiguous rows. Do not silently “fix” them from general knowledge because that removes the evidence needed to understand the error.

Expedia returned different properties during two valid runs 25 minutes apart. Python saw no schema error in either result. The Hoxton extraction also put a cancellation statement inside an amenities list. Query context, retrieval time, and manual review are what make both failures visible.

Production checks for hotel price data

Once this runs on a schedule, a few controls become non-negotiable.

Price meaning. Store nightly_price, stay_total, and taxes_and_fees separately. Do not divide a total by the number of nights unless the product explicitly wants a calculated field and labels it as calculated. Preserve the source string beside parsed numeric values.

Search identity. Make dates, occupancy, rooms, currency, locale, residency, device context, and applied filters part of the record key. Two searches for “Rome” are not comparable when one is for a weekend, one uses a member rate, or one displays taxes differently.

Availability drift. Hotel inventory can change between calls. A low-availability badge is a dated observation, not a durable property attribute. Track changes as new rows rather than overwriting history.

Promotions and policies. Free cancellation needs a deadline and rate context. Breakfast may be included for one room and absent for another. Member, mobile, package, and pay-later conditions should remain attached to the exact offer.

Failure handling. Distinguish an empty result from a blocked page, consent screen, timeout, and validation error. One justified rendering retry is enough for this example. Add exponential backoff for transient errors, but stop when access is refused.

Quality review. Set field-level acceptance rules and anomaly checks. A sudden currency change, impossible review score, missing date, or 90 percent price swing should enter a review queue. Sample the source regularly even when automated validation stays green.

For monitoring design, continue with the price scraping guide. The market research scraping guide goes deeper on source triangulation. If a notebook is too small for the job, the web scraping API guide covers an application-facing architecture.

Robots rules, terms, privacy, and the booking boundary

Technical access is not permission. Check the source's terms, robots directives, licenses, and applicable law. Limit collection to fields needed for the stated purpose. Use conservative request rates and a clear retention policy. The web scraping legality guide explains the questions to take to counsel for a commercial project.

Avoid authenticated pages, traveler details, payment information, loyalty data, reviewer identities, and other personal data. Aggregate ratings are usually enough for hotel-market analysis. If personal data is truly necessary, define the lawful basis, minimization, security, retention, and deletion process before collecting it.

Do not reproduce hotel descriptions, photographs, or reviews at scale simply because the page is public. Factual fields and short source-linked observations are a narrower dataset than copying creative content.

This workflow ends before booking. It cannot quote a guaranteed rate, reserve inventory, take payment, or modify a reservation. Cross that line only through an approved partner integration with the required support and contractual controls.

Run the hotel data workflow in Colab

The public Hotel Data Scraping with ScrapeGraphAI notebook contains the pinned installation, hidden key input, schemas, source ladder, guarded retry, Expedia search, Hoxton profile, CSV assertions, download cell, and production limitations.

It is shared as anyone-with-link viewer. The published 15-cell file was anonymously downloaded over HTTP after sharing, parsed successfully, and matched the local notebook byte for byte. It contains no saved output and no credential material. Run the cells in order, then compare a sample with the live pages before using the CSV.

Can hotel prices be scraped with Python?

Yes, when the page is public, the collection is permitted, and the workflow accepts that prices and availability are volatile. Define dates, guests, currency, and price types explicitly. Validate the response, record the source and time, and stop if the page refuses access.

How do I scrape hotel listings without mixing up prices?

Use one model per offer, require the property name and displayed price, and keep nightly, total, and tax fields separate. Limit the number of cards, reject placeholders, and manually compare a sample with the rendered page. A valid schema alone cannot prove card alignment.

Is web scraping Expedia the same as using Expedia Rapid?

No. Expedia Rapid is an official partner product governed by provisioned access and commercial terms. Scraping a public Expedia page records what that page visibly showed for a specific search. It does not create Rapid access, licensed inventory, booking rights, or service guarantees.

Should I buy hotel data scraping services or build the workflow?

A managed service can help with collection operations, but it cannot choose your legal basis, define the right price semantics, or guarantee that a valid field means what your analysis assumes. Build a small source-checked pilot first. Buy operational scale only after the schema and review process are stable.

What should a hospitality data extractor store?

At minimum, store the property, source, retrieval time, search dates, occupancy, currency, displayed price text, price type, taxes, and applicable room or policy context. Add property amenities and contact details as separate profile records rather than copying them into every offer.

Related Articles

• Booking.com Scraping Guide for Hotel Data
• KAYAK API Documentation and Python Guide
• Price Scraping Guide
• Market Research Scraping
• Web Scraping API Guide
• Is Web Scraping Legal?