--- title: "Web Scraping with GLM: Text and Visual Data Extraction" slug: web-scraping-with-glm date: 2026-09-16T08:27:42+00:00 modified: 2026-09-16T08:28:06+00:00 permalink: https://brightdata.com/blog/ai/web-scraping-with-glm type: blog --- [ Blog ](https://brightdata.com/blog "Blog") / [AI](https://brightdata.com/blog/ai) [AI](https://brightdata.com/blog/ai) # Web Scraping with GLM: Text and Visual Data Extraction Use GLM-5.3 with Bright Data’s Web Unlocker for AI-driven web scraping. Automatically extract text and visual data, bypassing common barriers. 5 min read [ ](https://brightdata.com/blog/authors/antonello-zanini) [Antonello Zanini Technical Writer ](https://brightdata.com/blog/authors/antonello-zanini) ![Web Scraping with GLM](https://media.brightdata.com/2026/09/Web-Scraping-with-GLM.png) In this article, you will discover: - Why GLM-5.3 and GLM-5.3-Flash represent a significant advancement for LLM-powered web scraping tasks. - The core obstacles in building web data extraction systems and how the Bright Data Web Unlocker API addresses them. - A step-by-step implementation of AI-powered web scraping using GLM in Python. - How to perform vision-based web scraping with GLM-5.3’s multimodal capabilities. Let’s dive in! ## Advantages of GLM Models for Web Data Extraction Modern LLMs have revolutionized the approach to web scraping. Rather than implementing manual parsing routines to process raw HTML markup, you can leverage an AI model to handle the extraction automatically. This eliminates the fragility inherent in [CSS selector-based approaches and XPath expression matching](/blog/web-data/xpath-vs-css-selectors). All that you need is a well-crafted prompt that specifies the data elements you need to retrieve. Within just a few code statements, you can instruct the LLM to identify and extract organized data from any web page. This AI-driven methodology produces web scrapers that are substantially more robust when page structures change. In this context, [GLM-5.3](https://z.ai/blog/glm-5.3) is exceptionally proficient at text-based information retrieval and structured data generation. Then, [GLM-5.3-Flash](https://z.ai/blog/glm-5.3-flash) comes with vision functionality, permitting you to analyze web page screenshots and graphical elements. For additional context, consult these related resources: 1. General principles of [AI-driven web scraping methodology](/blog/web-data/ai-web-scraping). 2. Application of [AI for visual data extraction via GPT Vision](/blog/ai/web-scraping-with-gpt-vision). ## Primary Challenges in AI Web Scraping AI-based extraction reduces sensitivity to changes in page markup. Still, the main technical challenges encountered in AI web scraping are largely consistent with those found in [traditional web scraping](/blog/web-data/web-scraping-challenges). The main issues remain JavaScript rendering, anti-bot detection systems, device fingerprinting techniques, request rate limitations, CAPTCHAs, and IP reputation issues. After all, before an LLM can process a web page, you must first acquire its content. This requires handling the various protective measures designed to prevent programmatic access. Thus, AI-powered extraction does not eliminate the fundamental difficulty of obtaining web page content at scale. What is certain is that adding AI to the process introduces additional considerations. In particular, token consumption becomes a critical factor. Transmitting extensive HTML documents to advanced models like [GLM-5.3 can quickly accumulate considerable costs](https://openrouter.ai/z-ai/glm-5.3). The optimal solution involves transforming raw HTML into an LLM-optimized Markdown format. Converting web pages into structured Markdown preserves essential information and layout hierarchy while [dramatically reducing unnecessary token consumption](https://www.kaggle.com/code/brightdataml/benchmarking-ai-on-different-data-formats). This transformation keeps headers, enumerated lists, image URLs and captions, and hyperlinks in a format that LLMs can process efficiently. For more details, read our [tutorial on HTML to Markdown conversion](/blog/web-data/scrape-websites-to-markdown). ### Bright Data Web Unlocker API: The Solution [Bright Data’s Web Unlocker API](/products/web-unlocker) delivers an all-in-one solution to the underlying challenges of web scraping. That is true whether you want to apply traditional parsing or AI-powered parsing. Web Unlocker API works as an endpoint that accepts a target URL and returns its rendered content, while handling all anti-bot protections. In particular, it [bypasses CAPTCHA protections](/products/web-unlocker/captcha-solver), processes JavaScript-intensive pages, circumvents browser fingerprinting detection, rotates IPs for you, and more. Learn more about [Bright Data’s approach to anti-bot bypass](https://docs.brightdata.com/concepts/how-bright-data-handles-blocking). On top of that, Web Unlocker API provides the flexibility to return retrieved data in multiple output formats, including: 1. [LLM-optimized Markdown representation](https://docs.brightdata.com/scraping-automation/web-unlocker/features#scrape-as-markdown) 2. [Screenshot captures of the rendered page](https://docs.brightdata.com/scraping-automation/web-unlocker/features#return-a-screenshot) As a result, the response data from Web Unlocker API requests serves as the ideal input for subsequent text-based or vision-based data extraction with LLMs, such as GLM-5.3 and GLM-5.3-Flash. **Note**: The Web Unlocker API is included within [Bright Data’s free tier](https://docs.brightdata.com/general/account/billing-and-pricing/free-tier), which grants 5,000 free extraction requests per calendar month. What sets Web Unlocker API apart from [other web scraping APIs](/blog/web-data/best-web-scraping-apis) is the underlying enterprise-grade infrastructure provided by Bright Data. This includes [400+ million residential IP addresses](/proxy-types/residential-proxies), unlimited concurrency, 99.99% uptime, and a 99.95% success rate. Such an infrastructure enables reliable, highly scalable programmatic data extraction. ## How to Build an AI-Powered Web Scraper with GLM-5.3 in Python In this guided section, you will learn how to develop a Python script that leverages GLM-5.3 for AI-powered data parsing. The script focuses on collecting data from [this specific IKEA product page](https://www.ikea.com/us/en/p/kallax-shelf-unit-white-20631663/): Yet, the key advantage of AI-based data parsing is that the same code can be applied across all IKEA product pages. That is true regardless of their individual layouts or design variations. The workflow will be: 1. The Bright Data Web Unlocker API retrieves the target IKEA product page and outputs its content as LLM-optimized Markdown. 2. GLM-5.3 receives the Markdown content through the OpenAI-compatible Z.ai API and extracts the product information. 3. The resulting output, which conforms to a predefined JSON schema, is exported to disk. **Note**: For production systems that require structured IKEA product data at scale, consider using [Bright Data’s IKEA Scraper API](/products/web-scraper/ikea). Follow the instructions below! ### Prerequisites Ensure the availability of: - [Python 3.10 or later installed on your system](https://www.python.org/downloads/). - A funded GLM account with an [API key configured](https://docs.z.ai/guides/overview/quick-start). - A [Bright Data account](/) with a Web Unlocker API and an API key set up. – Consult the [Web Unlocker setup documentation](https://docs.brightdata.com/scraping-automation/web-unlocker/quickstart). – Follow the [official guide to retrieve your Bright Data API key](https://docs.brightdata.com/api-reference/authentication#how-do-i-generate-a-new-api-key). Throughout this guide, we will assume your Web Unlocker API zone is named `web_unlocker`: Also, your Z.ai API key will look like this: ### Step #1: Initialize a Python Project Establish a new directory for your GLM scraping project: ```none mkdir glm-scraper ``` Move into this directory and add a Python [virtual environment](https://docs.python.org/3/library/venv.html) inside it: ```none cd glm-scraper python -m venv .venv ``` Load the project directory in your preferred Python editor, such as [Visual Studio Code](https://code.visualstudio.com/docs/languages/python) or [PyCharm](https://www.jetbrains.com/pycharm/download/#section=windows). Within the project root, add a `scraper.py` file: ```none glm-scraper ├─── .venv/ └─── scraper.py # <----- ``` This file will contain the Python code for AI-powered data extraction using GLM models. Next, enable the virtual environment. On Linux or macOS systems, run: ```none source .venv/bin/activate ``` On Windows platforms, execute: ```none .venv\Scripts\activate ``` With activation complete, install all necessary project dependencies: ```none pip install dotenv requests openai pydantic ``` These required packages are: - [`dotenv`](https://pypi.org/project/python-dotenv/)`: Environment variable management for storing API keys. - [`requests`](https://pypi.org/project/requests/): HTTP client for communicating with the Bright Data Web Unlocker API. - [`openai`](https://pypi.org/project/openai/): OpenAI-compatible client for accessing the OpenAI-compatible Z.ai APIs for GLM access. - [`pydantic`](https://pypi.org/project/pydantic/): Data model definition and validation framework for the output schema. Well done! Your Python project is now ready for web scraping with GLM. ### Step #2: Add Environment Variable Reading Your GLM scraper requires authentication with external services, including Bright Data and Z.ai’s APIs. These integrations demand API keys, which you should never expose within your source code for security reasons. Instead, maintain your API keys in an environment configuration file and load them during execution. The `python-dotenv` package simplifies this process by letting you read environment variables from a `.env` file. To get started with `python-dotenv`, add this code to the beginning of `scraper.py`: ```none from dotenv import load_dotenv # Load the environment variables from the .env file load_dotenv() ``` Then, create a `.env` file at the project root: ```none glm-scraper ├─── .venv/ ├─── .env # <----- └─── scraper.py ``` **Tip**: In a real-world repository, include `.env` in your `.gitignore` configuration to prevent accidental credential exposure. Populate it with your API credentials: ```none BRIGHT_DATA_API_KEY="" ZAI_API_KEY="" ``` Replace: - `` with your Bright Data API credentials. - `` with your Z.ai API key. Next, load these environment variables into your script using Python’s [`os.getenv()`](https://docs.python.org/3/library/os.html#os.getenv): ```none import os # Loading the required secrets from the envs ZAI_API_KEY = os.environ.get("ZAI_API_KEY") BRIGHT_DATA_API_KEY = os.environ.get("BRIGHT_DATA_API_KEY") ``` Fantastic! You now have secure access to your API keys in your Python script. ### Step #3: Get the Target Web Page in Markdown Format with Web Unlocker API The first phase of AI-powered scraping involves obtaining the target page, ideally in LLM-ready Markdown format. Bright Data’s Web Unlocker API provides the simplest way to accomplish that. For more guidance on Web Unlocker integration, [consult the official documentation](https://docs.brightdata.com/scraping-automation/web-unlocker/send-your-first-request#direct-api-access-recommended) or examine [Bright Data’s Python integration examples](https://github.com/luminati-io/bright-data-web-unlocker-python-project). Utilize the `requests` library to send an HTTP `POST` request to Web Unlocker and retrieve the target page rendered as Markdown: ```none import requests # Set up the authentication headers for Web Unlocker headers = { "Content-Type": "application/json", "Authorization": f"Bearer {BRIGHT_DATA_API_KEY}" } # Define the request payload for the Web Unlocker API call payload = { "zone": "web_unlocker", # Replace with your Web Unlocker zone name "url": "https://www.ikea.com/us/en/p/kallax-shelf-unit-white-20631663/", # The IKEA product page URL you want to scrape "format": "raw", "country": "us", "data_format": "markdown" # Request Markdown-formatted output } # Retrieve the unlocked content of the target page web_unlocker_response = requests.post( "https://api.brightdata.com/request", json=payload, headers=headers ) markdown_page = web_unlocker_response.text ``` If you are not familiar with the Python HTTP syntax, review [our comprehensive Requests guide](/blog/web-data/python-requests-guide). Notice the `data_format="markdown"` parameter in the body. This configuration directs the Web Unlocker API to [output the web page in Markdown format](https://docs.brightdata.com/scraping-automation/web-unlocker/features#scrape-as-markdown) rather than supplying raw HTML. Print `markdown_page`, and you will see: The returned output represents the Markdown-converted version of your target IKEA product page. Observe how the Web Unlocker API takes care of the obstacles of web page retrieval: 1. Accessing the target IKEA infrastructure through Bright Data’s residential proxy network. 2. Managing typical anti-bot countermeasures, such as IKEA’s anti-scraping systems. 3. Obtaining the fully rendered page markup. 4. Transforming the HTML into LLM-optimized Markdown and returning the result. You now have page information in a perfect format for GLM-5.3 text-based data parsing. Before proceeding, establish the output structure that GLM will generate! ### Step #4: Define the Output JSON Schema GLM-5.3, like most other LLMs, produces text output by default. This is suboptimal for data extraction workflows, where the objective is converting raw web content into structured data. Fortunately, the GLM-5.3 API supports structured output through [JSON mode](https://docs.z.ai/guides/capabilities/struct-output). This functionality allows the model to return a plain JSON string. **Note**: While you cannot specify the schema directly in the `json_schema` argument [as you can with OpenAI’s structured outputs](https://developers.openai.com/api/docs/guides/structured-outputs?api-mode=responses), you can mention the desired response structure in the prompt. This will mandate the model to produce a JSON string that conforms to that format. The most convenient method for defining structured [JSON schemas](https://json-schema.org/) in Python is through [Pydantic](https://pydantic.dev/), the most popular data modeling framework. Pydantic helps you represent data structures via Python classes, which can be converted into JSON Schema format for the Z.ai API. Start by examining the target IKEA page to identify required information fields. For this example, define a Pydantic model capturing IKEA product details: ```none from pydantic import BaseModel, Field from typing import List, Optional class IkeaProduct(BaseModel): title: Optional[str] = Field(None, description="Full product title as displayed on the IKEA website") product_number: Optional[str] = Field(None, description="IKEA product article number, such as 304.247.75") product_type: Optional[str] = Field(None, description="General type of IKEA product, such as armchair, shelf unit, cabinet, or table") price: Optional[float] = Field(None, description="Current one-time purchase price") currency: Optional[str] = Field(None, description="Currency used for the displayed price, such as USD") color: Optional[str] = Field(None, description="Product color or finish, such as birch veneer or dark gray") dimensions: Optional[str] = Field(None, description="Overall product dimensions as displayed by IKEA, including width, depth, and height") materials: Optional[List[str]] = Field(None, description="Main materials used to manufacture the product") rating: Optional[float] = Field(None, description="Average customer rating, typically on a 5-point scale") review_count: Optional[int] = Field(None, description="Total number of customer reviews") series: Optional[str] = Field(None, description="IKEA product series the item belongs to, such as POÄNG") assembly_required: Optional[bool] = Field(None, description="Whether the product requires assembly") is_in_stock: Optional[bool] = Field(True, description="Whether the product is currently available for purchase") ``` This model organizes IKEA product information into [Pydantic fields](https://pydantic.dev/docs/validation/2.0/usage/types/types_fields/) with explicit typing. GLM-5.3 will retrieve matching values from the page content and deliver them as a JSON string response respecting this schema. **Pro tip**: Pass the Markdown retrieved from the page by the Web Unlocker API to an LLM and ask it to generate a Pydantic model containing all the data points you are interested in. Wonderful! You now possess all necessary components for GLM web scraping. ### Step #5: Use GLM-5.3 for Automated Data Parsing The Z.ai API is compatible with the OpenAI Chat Completions and Responses interfaces, permitting [access via the OpenAI Python SDK](https://docs.z.ai/guides/develop/openai/python#openai-python-sdk). Configure the OpenAI client with your Z.ai API credentials and the appropriate API base URL. ```none from openai import OpenAI # Initialize the OpenAI client for interacting with Z.ai API glm_client = OpenAI( api_key=ZAI_API_KEY, base_url="https://api.z.ai/api/paas/v4/", # OpenAI Chat Completions-compatible GLMM API base URL ) ``` Then, employ the [Chat Completions endpoint](https://docs.z.ai/api-reference/llm/chat-completion) to transmit the extracted Markdown to GLM-5.3: ```none import json # Prepare the scraping prompt with the output schema scraping_prompt = f""" Extract the IKEA product information from the given Markdown document and return it as structured data that conforms to the provided JSON schema. SCHEMA: {json.dumps(IkeaProduct.model_json_schema(), indent=2)} MARKDOWN: {markdown_page} """ # Perform web data parsing with GLM-5.3 response = glm_client.chat.completions.create( model="glm-5.3", messages=[ { "role": "user", "content": [ { "type": "text", "text": scraping_prompt, }, ], } ], response_format={"type": "json_object"} ) # Get the parsed data in a string matching the defined schema format product_data = response.choices[0].message.content ``` Note that the prompt needs to include: 1. Your web data extraction instructions. 2. The target JSON schema. 3. The Markdown content from Bright Data’s Web Unlocker API. The `response_format` parameter enables JSON mode with `{"type": "json_object"}`. Remember that GLM-5.3 supports `json_object` format rather than `json_schema`. That is a critical distinction from some alternative models, such as when performing [web scraping with Kimi](/blog/ai/web-scraping-with-kimi). **Note**: The [optional `thinking` and `reasoning_effort` parameters](https://docs.z.ai/guides/capabilities/thinking-mode) activate GLM-5.3’s extended reasoning capabilities. Set `reasoning_effort` to `"max"` to enable deep reasoning for highly complex data parsing tasks. This is particularly valuable when pages contain ambiguous or non-standard layouts. The result is `product_data`, containing a JSON string with the extracted information structured according to your `IkeaProduct` model. Print it, and you will see: Cool! The only remaining step is to persist this JSON data to disk. ### Step #6: Save the Scraped Data to Disk Before storing `product_data` to a `product.json` file, validate the response against the `IkeaProduct` Pydantic model: ```none # Get the parsed data in the defined schema format product_data = response.choices[0].message.content # Validate the JSON string against the IkeaProduct schema product = IkeaProduct.model_validate_json(product_data) ``` The [`model_validate_json()`](https://pydantic.dev/docs/validation/dev/api/pydantic/base_model/#pydantic.BaseModel.model_validate_json) method verifies that the LLM response conforms to the expected Pydantic schema. If the response contains invalid data or does not match the defined field types, Pydantic raises a validation error instead of silently saving malformed data. Then, persist it to disk: ```none # Export the validated IkeaProduct object to a JSON file with open("product.json", "w", encoding="utf-8") as f: f.write(product.model_dump_json(indent=2)) ``` This snippet uses `product.model_dump_json()` to serialize the validated `IkeaProduct` object and then writes it to `product.json`. The resulting file will contain the structured IKEA product data extracted by GLM-5.3. Mission complete! ### Step #7: Put It All Together Below is the complete implementation of your GLM web scraper for IKEA product pages: ```none # pip install dotenv requests openai pydantic from dotenv import load_dotenv import os import requests from pydantic import BaseModel, Field from typing import List, Optional from openai import OpenAI import json # Load the environment variables from the .env file load_dotenv() # Loading the required secrets from the envs BRIGHT_DATA_API_KEY = os.environ.get("BRIGHT_DATA_API_KEY") ZAI_API_KEY = os.environ.get("ZAI_API_KEY") # The output schema for the product information class IkeaProduct(BaseModel): title: Optional[str] = Field(None, description="Full product title as displayed on the IKEA website") product_number: Optional[str] = Field(None, description="IKEA product article number, such as 304.247.75") product_type: Optional[str] = Field(None, description="General type of IKEA product, such as armchair, shelf unit, cabinet, or table") price: Optional[float] = Field(None, description="Current one-time purchase price") currency: Optional[str] = Field(None, description="Currency used for the displayed price, such as USD") images: Optional[List[str]] = Field(None, description="URLs of product images from the IKEA website") color: Optional[str] = Field(None, description="Product color or finish, such as birch veneer or dark gray") dimensions: Optional[str] = Field(None, description="Overall product dimensions as displayed by IKEA, including width, depth, and height") materials: Optional[List[str]] = Field(None, description="Main materials used to manufacture the product") visual_features: Optional[List[str]] = Field(None, description="Physical characteristics visible in the supplied product images, such as shape, components, finish, or configuration") rating: Optional[float] = Field(None, description="Average customer rating, typically on a 5-point scale") review_count: Optional[int] = Field(None, description="Total number of customer reviews") series: Optional[str] = Field(None, description="IKEA product series the item belongs to, such as POÄNG") designer: Optional[str] = Field(None, description="Designer credited by IKEA for the product") max_load: Optional[str] = Field(None, description="Maximum supported load, such as maximum load on a seat, shelf, or tabletop") care_instructions: Optional[List[str]] = Field(None, description="IKEA-recommended care and cleaning instructions") package_count: Optional[int] = Field(None, description="Number of packages included with the product") package_dimensions: Optional[List[str]] = Field(None, description="Dimensions of the product packaging") package_weight: Optional[str] = Field(None, description="Weight of the packaged product") assembly_required: Optional[bool] = Field(None, description="Whether the product requires assembly") assembly_instructions_url: Optional[str] = Field(None, description="URL to the official IKEA assembly instructions") safety_information: Optional[List[str]] = Field(None, description="Important safety information, such as wall anchoring or tipping requirements") good_to_know: Optional[List[str]] = Field(None, description="Additional IKEA product information and usage recommendations") compatible_series: Optional[List[str]] = Field(None, description="IKEA product series or furniture lines that the product is described as coordinating with") accessories: Optional[List[str]] = Field(None, description="Accessories or complementary IKEA products shown or recommended on the product page") category_path: Optional[List[str]] = Field(None, description="IKEA category breadcrumb path leading to the product") available_colors: Optional[List[str]] = Field(None, description="Other color or finish options available for the same product") is_in_stock: Optional[bool] = Field(True, description="Whether the product is currently available for purchase") # Set up the authentication headers for the Web Unlocker API headers = { "Content-Type": "application/json", "Authorization": f"Bearer {BRIGHT_DATA_API_KEY}" } # Define the request payload for the Web Unlocker API scraping request payload = { "zone": "web_unlocker", # Replace with your Web Unlocker zone name "url": "https://www.ikea.com/us/en/p/kallax-shelf-unit-white-20631663/", # The IKEA product page URL you want to scrape "format": "raw", "country": "us", "data_format": "markdown" # To get the web page in Markdown } # Retrieve the Markdown content of the unlocked target page web_unlocker_response = requests.post( "https://api.brightdata.com/request", json=payload, headers=headers ) markdown_page = web_unlocker_response.text # Initialize the OpenAI client for Z.ai models glm_client = OpenAI( api_key=ZAI_API_KEY, base_url="https://api.z.ai/api/paas/v4/", ) # Prompt to LLM-powered web data parsing scraping_prompt = f""" Extract the IKEA product information from the given Markdown document and return it as structured data that conforms to the provided JSON schema. SCHEMA: {json.dumps(IkeaProduct.model_json_schema(), indent=2)} MARKDOWN: {markdown_page} """ # Perform web data scraping with GLM-5.3 response = glm_client.chat.completions.create( model="glm-5.3", messages=[ { "role": "user", "content": [ { "type": "text", "text": scraping_prompt, }, ], } ], response_format={"type": "json_object"}, ) # Get the parsed data in the defined schema format product_data = response.choices[0].message.content # Convert the JSON string into a validated IkeaProduct object product = IkeaProduct.model_validate_json(product_data) # Export the IkeaProduct object to a JSON file with open("product.json", "w", encoding="utf-8") as f: f.write(product.model_dump_json(indent=2)) ``` In your activated virtual environment, execute the Python script using: ```none python scraper.py ``` Page retrieval via the Web Unlocker API and subsequent processing through GLM-5.3 take time. So, be patient while the `product.json` file is produced. Once it appears in your project’s folder, open it, and you will see: Notice the alignment between extracted data and the information available on the original IKEA product page. Et voilà! The Web Unlocker API handled IKEA’s anti-bot protections and delivered the LLM-optimized Markdown content, while GLM-5.3 successfully processed that content and transformed it into well-organized, structured data. ## How to Perform Vision-Based Web Data Extraction with GLM-5.3-Flash GLM-5.3 does not natively support multimodal input, meaning it can only process text-based inputs. However, [other models in the GLM family support multimodal input](https://docs.z.ai/guides/overview/overview#vision-models), including GLM-5.3-Flash (as well as GLM-4.6V and others). Instead of supplying the LLM with textual page content, you can provide a screenshot or image and ask GLM-5.3-Flash to extract structured information from it. The scraping procedure remains largely the same: 1. The Bright Data Web Unlocker API captures a screenshot of the target IKEA page. 2. GLM-5.3-Flash receives the screenshot through the Z.ai APIs and extracts product information according to a predefined schema. 3. The extracted data is validated and saved as structured JSON. **Note**: The core workflow changes very little, so this section focuses on the modifications needed for visual data extraction. All prerequisite and setup steps described earlier remain valid. ### Step #1: Retrieve the Page Screenshot Adjust the Web Unlocker API request to obtain a page screenshot instead of Markdown: ```none # Set up the authentication headers for Web Unlocker headers = { "Content-Type": "application/json", "Authorization": f"Bearer {BRIGHT_DATA_API_KEY}" } # Define the request payload for the Web Unlocker API call payload = { "zone": "web_unlocker", # Replace with your Web Unlocker zone name "url": "https://www.ikea.com/us/en/p/kallax-shelf-unit-white-20631663/", # The IKEA product page URL you want to visually scrape "format": "raw", "country": "us", "data_format": "screenshot" # Get a screenshot of the web page } # Retrieve the screenshot of the unlocked target page web_unlocker_response = requests.post( "https://api.brightdata.com/request", json=payload, headers=headers ) page_screenshot = web_unlocker_response.content ``` The essential modification is the `data_format` field, now configured to `"screenshot"`. This [instructs Web Unlocker to return a page screenshot](https://docs.brightdata.com/scraping-automation/web-unlocker/features#return-a-screenshot). The API response will comprise screenshot image data in PNG format, representing the completely rendered page. Export the screenshot to disk: ```none with open("screenshot.png", "wb") as f: f.write(page_screenshot) ``` This optional step proves beneficial for visual inspection of the retrieved page and troubleshooting extraction workflows. The output will be a `screenshot.png` file: Note how Bright Data’s Web Unlocker API captured a screenshot of the full IKEA product page. Proceeding forward! ### Step #2: Update the Pydantic Model Through analysis of a page screenshot, GLM-5.3-Flash can identify visual information unavailable in the page’s Markdown representation. For instance, it can interpret text embedded in visual labels (e.g., the “Best seller” tag): Simultaneously, a screenshot does not deliver all the page information. For instance, fields like `images` or `assembly_instructions_url`, which were accessible through the Markdown-based extraction workflow, cannot be populated from visual data. Accordingly, adapt the `IkeaProduct` Pydantic model for visual processing: ```none class IkeaProduct(BaseModel): title: Optional[str] = Field(None, description="Full product title as displayed on the IKEA website") product_type: Optional[str] = Field(None, description="General type of product, such as an armchair, shelf unit, cabinet, or table") price: Optional[float] = Field(None, description="Current displayed numerical price") currency: Optional[str] = Field(None, description="Currency used for the displayed price, such as USD") is_best_seller: Optional[bool] = Field(False, description="Whether a best-seller or top-seller badge is displayed") badge_label: Optional[str] = Field(None, description="Promotional or product badge displayed on the page, such as Top Seller") delivery_options: Optional[List[str]] = Field(None, description="Delivery or fulfillment options displayed on the page") color: Optional[str] = Field(None, description="Selected product color or finish, such as white or birch veneer") visual_aspects: Optional[str] = Field(None, description="Description of the physical characteristics visible in the product images, such as shape, components, finish, or configuration") dimensions: Optional[str] = Field(None, description="Overall product dimensions as displayed on the IKEA website") rating: Optional[float] = Field(None, description="Average customer rating, typically on a 5-point scale") review_count: Optional[int] = Field(None, description="Total number of customer reviews displayed") ``` Observe the introduction of the `visual_aspects` field. This field directs GLM to document physical characteristics visible within product images, enabling extraction of information unavailable through text-based parsing. Also, focus on the new `is_best_seller` and `badge_label` fields. Great! Send the updated Pydantic model and the web page screenshot to the Z.ai API for visual web scraping with GLM-5.3-Flash. ### Step #3: Scrape Data from an Image with GLM-5.3 Vision Just like [OpenAI models with vision capabilities](https://developers.openai.com/api/docs/guides/images-vision), GLM-5.3-Flash accepts images via the `image_url` field. This accepts either publicly accessible image URLs or [Base64-encoded image representations](https://developer.mozilla.org/en-US/docs/Glossary/Base64). Since the screenshot exists locally as image bytes, encoding it as Base64 is required. To do so, employ Python’s [`base64` module](https://docs.python.org/3/library/base64.html): ```none import base64 screenshot_base64 = base64.b64encode(page_screenshot).decode("utf-8") ``` Now, transmit the Base64-encoded screenshot to GLM-5.3-Flash alongside your extraction instructions: ```none # Perform visual web data parsing with GLM-5.3-Flash response = glm_client.chat.completions.create( model="glm-5.3-flash", messages=[ { "role": "user", "content": [ { "type": "image_url", "image_url": { "url": f"data:image/png;base64,{screenshot_base64}" }, }, { "type": "text", "text": f""" Extract the IKEA product information from the provided webpage screenshot and return it as structured data that conforms to the provided JSON schema. SCHEMA: {json.dumps(IkeaProduct.model_json_schema(), indent=2)} """, }, ], } ], response_format={"type": "json_object"}, ) # Get the parsed data in the defined schema format product_data = response.choices[0].message.content ``` The request to the Z.ai API comprises two components: 1. The page screenshot image. 2. A text instruction directing GLM-5.3 to retrieve product information using the designated Pydantic schema. **Note**: This time, the target model is `glm-5.3-flash`, not `glm-5.3` as before. This change is required as GLM-5.3 does not have vision capabilities. The result is a `product_data` variable storing a JSON string with the visually scraped data. Store it to disk as described previously. Impressive! ### Step #4: Test the Visual-Based GLM Scraping Script The final code of your visual GLM web scraping script will be: ```none # pip install dotenv requests openai pydantic from dotenv import load_dotenv import os import requests import base64 import json from pydantic import BaseModel, Field from typing import List, Optional from openai import OpenAI # Load the environment variables from the .env file load_dotenv() # Loading the required secrets from the envs BRIGHT_DATA_API_KEY = os.environ.get("BRIGHT_DATA_API_KEY") ZAI_API_KEY = os.environ.get("ZAI_API_KEY") class IkeaProduct(BaseModel): title: Optional[str] = Field(None, description="Full product title as displayed on the IKEA website") product_type: Optional[str] = Field(None, description="General type of product, such as an armchair, shelf unit, cabinet, or table") price: Optional[float] = Field(None, description="Current displayed numerical price") currency: Optional[str] = Field(None, description="Currency used for the displayed price, such as USD") is_best_seller: Optional[bool] = Field(False, description="Whether a best-seller or top-seller badge is displayed") badge_label: Optional[str] = Field(None, description="Promotional or product badge displayed on the page, such as Top Seller") delivery_options: Optional[List[str]] = Field(None, description="Delivery or fulfillment options displayed on the page") color: Optional[str] = Field(None, description="Selected product color or finish, such as white or birch veneer") visual_aspects: Optional[str] = Field(None, description="Description of the physical characteristics visible in the product images, such as shape, components, finish, or configuration") dimensions: Optional[str] = Field(None, description="Overall product dimensions as displayed on the IKEA website") rating: Optional[float] = Field(None, description="Average customer rating, typically on a 5-point scale") review_count: Optional[int] = Field(None, description="Total number of customer reviews displayed") # Set up the authentication headers for Web Unlocker headers = { "Content-Type": "application/json", "Authorization": f"Bearer {BRIGHT_DATA_API_KEY}" } # Define the request payload for the Web Unlocker API call payload = { "zone": "web_unlocker", # Replace with your Web Unlocker zone name "url": "https://www.ikea.com/us/en/p/kallax-shelf-unit-white-20631663/", # The IKEA product page URL you want to visually scrape "format": "raw", "country": "us", "data_format": "screenshot" # Get a screenshot of the web page } # Retrieve the screenshot of the unlocked target page web_unlocker_response = requests.post( "https://api.brightdata.com/request", json=payload, headers=headers ) page_screenshot = web_unlocker_response.content # Export the screenshot to a PNG file with open("screenshot.png", "wb") as f: f.write(page_screenshot) # Convert the screenshot to Base64 for vision processing screenshot_base64 = base64.b64encode(page_screenshot).decode("utf-8") # Initialize the OpenAI-compatible client for Z.ai glm_client = OpenAI( api_key=ZAI_API_KEY, base_url="https://api.z.ai/api/paas/v4/", ) # Perform visual web data parsing with GLM-5.3-Flash response = glm_client.chat.completions.create( model="glm-5.3-flash", messages=[ { "role": "user", "content": [ { "type": "image_url", "image_url": { "url": f"data:image/png;base64,{screenshot_base64}" }, }, { "type": "text", "text": f""" Extract the IKEA product information from the provided webpage screenshot and return it as structured data that conforms to the provided JSON schema. SCHEMA: {json.dumps(IkeaProduct.model_json_schema(), indent=2)} """, }, ], } ], response_format={"type": "json_object"}, ) # Get the parsed data in the defined schema format product_data = response.choices[0].message.content # Convert the JSON string into a validated IkeaProduct object product = IkeaProduct.model_validate_json(product_data) # Export the IkeaProduct object to a JSON file with open("visual_product.json", "w", encoding="utf-8") as f: f.write(product.model_dump_json(indent=2)) ``` Launch the script, and this will generate a `visual_product.json` file containing: Notice the successful retrieval of product information from the page screenshot, including visual aspects and the “Best seller” label. Amazing! ## Conclusion In this tutorial, you learned how to build an LLM-powered web data extraction system using GLM-5.3 and GLM-5.3-Flash. The main challenges of AI web scraping (i.e., accessing web content without getting blocked and converting it into an LLM-ready format) have been easily addressed with Bright Data’s Web Unlocker API. As demonstrated here, combining GLM-5.3 and GLM-5.3-Flash with the Web Unlocker API allows you to extract structured information from both web pages and their visual representations. This is just one of many use cases supported by [Bright Data’s AI-compatible web access solutions](/ai/web-access). Register with Bright Data today and start experimenting with our web scraping APIs for free! Contact usStart free trial No credit card required [ ](https://www.linkedin.com/in/antonello-zanini/) Antonello Zanini Technical Writer 5.5 years experience Antonello Zanini is a technical writer, editor, and software engineer with 5M+ views. Expert in technical content strategy, web development, and project management. Expertise Web Development Web Scraping AI Integration [ View all articles ](https://brightdata.com/blog/authors/antonello-zanini) Table of Contents Data for AI Supercharge your AI with instant and reliable access to web data. No blockers. No hassle. Talk to an expert Bright Data MCP Get started with Bright Data’s Web MCP Server today with 5000 free monthly requests and unlock your AI’s full potential. Start free now [ ](https://news.ycombinator.com/submitlink?t=Web+Scraping+with+GLM%3A+Text+and+Visual+Data+Extraction&u=https://brightdata.com/blog/ai/web-scraping-with-glm) [ ](https://www.linkedin.com/shareArticle?mini=true&title=Web+Scraping+with+GLM%3A+Text+and+Visual+Data+Extraction&url=https://brightdata.com/blog/ai/web-scraping-with-glm) [ ](http://www.reddit.com/submit?title=Web+Scraping+with+GLM%3A+Text+and+Visual+Data+Extraction&url=https://brightdata.com/blog/ai/web-scraping-with-glm) ## You might also be interested in [ ](https://brightdata.com/blog/web-data/containers-as-a-service "Containers as a Service (CaaS): How to Use It for Web Data Pipelines and AI Agents") [Web Data Antonello Zanini Technical Writer ### Containers as a Service (CaaS): How to Use It for Web Data Pipelines and AI Agents How Containers as a Service works, and how to scale a Bright Data web scraping pipeline across containers for AI agents. 16-Sep-2026 12 min read ](https://brightdata.com/blog/web-data/containers-as-a-service) [ ](https://brightdata.com/blog/ai/openhuman-with-bright-data "Production-Ready Web Access in OpenHuman Through the Bright Data CLI") [AI Antonello Zanini Technical Writer ### Production-Ready Web Access in OpenHuman Through the Bright Data CLI Integrate Bright Data CLI with OpenHuman to enable production-ready web access and data collection for AI agents. 09-Sep-2026 12 min read ](https://brightdata.com/blog/ai/openhuman-with-bright-data) [ ](https://brightdata.com/blog/ai/minimax-m3-with-bright-data "Giving self-hosted MiniMax M3 agents live web access with Bright Data") [AI Satyam Tripathi Technical Writer ### Giving self-hosted MiniMax M3 agents live web access with Bright Data Self-hosted MiniMax M3 agents get live web access using Bright Data’s search and scraping tools. Bypass blocks and CAPTCHAs. 09-Sep-2026 54 min read ](https://brightdata.com/blog/ai/minimax-m3-with-bright-data)