Building a Web Data Extraction App with Lovable, Supabase, and Bright Data

User Input
   ↓
Authentication
   ↓
Database Logging
   ↓
Edge Function
   ↓
Bright Data Web Unlocker (Bypasses anti-bot protection)
   ↓
Raw HTML
   ↓
Turndown (HTML → Markdown)
   ↓
Clean structured text
   ↓
Google Gemini AI (Natural language processing)
   ↓
Structured JSON
   ↓
Database Storage
   ↓
Frontend Display
   ↓
User Export

Build a modern web data extraction app using React and Tailwind CSS. The UI should include:
- A gradient background with card-style layout
- An input field for the webpage URL
- A textarea for the extraction prompt (e.g., "Extract product title, price, and ratings")
- A display area to render structured JSON output
- Responsive styling with hover effects and proper spacing

Set up complete Supabase authentication:
- Sign up and login forms using email/password
- Session management and auto-persistence
- Route protection for unauthenticated users
- Sign out functionality
- Create user profile on signup
- Handle all auth-related errors

Create Supabase tables for storing extractions and results:
- extractions: stores URL, prompt, user_id, status, processing_time, error_message
- extraction_results: stores parsed JSON output
Apply RLS policies to ensure each user can only access their own data

Create an Edge Function called 'extract-web-data' that:
- Fetches the target page using Bright Data's Web Unlocker
- Converts raw HTML to Markdown using Turndown
- Sends the Markdown and prompt to Google Gemini AI (gemini-2.0-flash-001)
- Returns clean structured JSON
- Handles CORS, errors, and response formatting
- Requires GEMINI_API_KEY and BRIGHT_DATA_API_KEY as Edge Function secrets

Below is a reference implementation that handles HTML fetching using Bright Data, markdown conversion with Turndown, and AI-driven extraction with Gemini:

import { GoogleGenerativeAI } from '@google/generative-ai';
import TurndownService from 'turndown';

interface BrightDataConfig {
  apiKey: string;
  zone: string;
}

// Constants
const GEMINI_MODEL = 'gemini-2.0-flash-001';
const WEB_UNLOCKER_ZONE = 'YOUR_WEB_UNLOCKER_ZONE';

export class WebContentExtractor {
  private geminiClient: GoogleGenerativeAI;
  private modelName: string;
  private htmlToMarkdownConverter: TurndownService;
  private brightDataConfig: BrightDataConfig;

  constructor() {
    const geminiApiKey: string = 'GEMINI_API_KEY';
    const brightDataApiKey: string = 'BRIGHT_DATA_API_KEY';

    try {
      this.geminiClient = new GoogleGenerativeAI(geminiApiKey);
      this.modelName = GEMINI_MODEL;
      this.htmlToMarkdownConverter = new TurndownService();
      this.brightDataConfig = {
        apiKey: brightDataApiKey,
        zone: WEB_UNLOCKER_ZONE
      };
    } catch (error) {
      console.error('Failed to initialize WebContentExtractor:', error);
      throw error;
    }
  }

  /**
   * Fetches webpage content using Bright Data Web Unlocker service
   */
  async fetchContentViaBrightData(targetUrl: string): Promise<string | null> {
    try {
      // Append Web Unlocker parameters to the target URL
      const urlSeparator: string = targetUrl.includes('?') ? '&' : '?';
      const requestUrl: string = `${targetUrl}${urlSeparator}product=unlocker&method=api`;

      const apiResponse = await fetch('https://api.brightdata.com/request', {
        method: 'POST',
        headers: {
          'Authorization': `Bearer ${this.brightDataConfig.apiKey}`,
          'Content-Type': 'application/json'
        },
        body: JSON.stringify({
          zone: this.brightDataConfig.zone,
          url: requestUrl,
          format: 'raw'
        })
      });

      if (!apiResponse.ok) {
        throw new Error(`Web Unlocker request failed with status: ${apiResponse.status}`);
      }

      const htmlContent: string = await apiResponse.text();

      return htmlContent && htmlContent.length > 0 ? htmlContent : null;

    } catch (error) {
      console.error('Failed to fetch webpage content:', error);
      return null;
    }
  }

  /**
   * Converts HTML to clean Markdown format for better AI processing
   */
  async convertToMarkdown(htmlContent: string): Promise<string | null> {
    try {
      const markdownContent: string = this.htmlToMarkdownConverter.turndown(htmlContent);
      return markdownContent;
    } catch (error) {
      console.error('Failed to convert HTML to Markdown:', error);
      return null;
    }
  }

  /**
   * Uses Gemini AI to extract specific information from markdown content
   * Uses low temperature for more consistent, factual responses
   */
  async extractInformationWithAI(markdownContent: string, userQuery: string): Promise<string | null> {
    try {
      const aiPrompt: string = this.buildAIPrompt(userQuery, markdownContent);

      const aiModel = this.geminiClient.getGenerativeModel({ model: this.modelName });

      const aiResult = await aiModel.generateContent({
        contents: [{ role: 'user', parts: [{ text: aiPrompt }] }],
        generationConfig: {
          maxOutputTokens: 2048,
          temperature: 0.1,
        }
      });

      const response = await aiResult.response;
      return response.text();

    } catch (error) {
      console.error('Failed to extract information with AI:', error);
      return null;
    }
  }

  private buildAIPrompt(userQuery: string, markdownContent: string): string {
    return `You are a data extraction assistant. Below is some content in markdown format extracted from a webpage.
Please analyze this content and extract the information requested by the user.

USER REQUEST: ${userQuery}

MARKDOWN CONTENT:
${markdownContent}

Please provide a clear, structured response based on the user's request. If the requested information is not available in the content, please indicate that clearly.`;
  }

  /**
   * Main extraction workflow: fetches webpage → converts to markdown → extracts with AI
   */
  async extractDataFromUrl(websiteUrl: string, extractionQuery: string): Promise<string | null> {
    try {
      const htmlContent: string | null = await this.fetchContentViaBrightData(websiteUrl);

      if (!htmlContent) {
        console.error('Could not retrieve HTML content from URL');
        return null;
      }

      const markdownContent: string | null = await this.convertToMarkdown(htmlContent);

      if (!markdownContent) {
        console.error('Could not convert HTML to Markdown');
        return null;
      }

      const extractedInformation: string | null = await this.extractInformationWithAI(markdownContent, extractionQuery);

      return extractedInformation;
    } catch (error) {
      console.error('Error in extractDataFromUrl:', error);
      return null;
    }
  }
}

/**
 * Example usage of the WebContentExtractor
 */
async function runExtraction(): Promise<void> {
  const TARGET_WEBSITE_URL: string = 'https://example.com';
  const DATA_EXTRACTION_QUERY: string = 'Extract the product title, all available prices, ...';

  try {
    const contentExtractor = new WebContentExtractor();

    const extractionResult: string | null = await contentExtractor.extractDataFromUrl(TARGET_WEBSITE_URL, DATA_EXTRACTION_QUERY);

    if (extractionResult) {
      console.log(extractionResult);
    } else {
      console.log('Failed to extract data from the specified URL');
    }

  } catch (error) {
    console.error(`Application error: ${error}`);
  }
}

// Execute the application
runExtraction().catch(console.error);

Connect the frontend to the Edge Function:
- On form submission, call the Edge Function
- Log the request in the database
- Update status (processing/completed/failed) after the response
- Show processing time, status icons, and toast notifications
- Display the extracted JSON with loading states

Create a history view that:
- Lists all extractions for the logged-in user
- Displays URL, prompt, status, duration, and date
- Includes View and Delete options
- Expands rows to show extracted results
- Uses icons for statuses (completed,failed,processing)
- Handles long text/URLs gracefully with a responsive layout

Polish the interface:
- Add toggle between "New Extraction" and "History"
- Create a JsonDisplay component with syntax highlighting and copy button
- Fix responsiveness issues for long prompts and URLs
- Add loading spinners, empty states, and fallback messages
- Include feature cards or tips at the bottom of the page