--- title: "Finding the web’s hidden robot training moments in public web video data" slug: robot-training-data-in-public-web-video date: 2026-08-25T06:33:18+00:00 modified: 2026-08-25T07:40:34+00:00 permalink: https://brightdata.com/blog/ai/robot-training-data-in-public-web-video type: blog --- [ Blog ](https://brightdata.com/blog "Blog") / [AI](https://brightdata.com/blog/ai) [AI](https://brightdata.com/blog/ai) # Finding the web’s hidden robot training moments in public web video data Research: Bright Data’s Video Search API uncovered 10× more physical-action moments by searching what videos show instead of relying on titles, descriptions, and tags. 18 min read [ ](https://brightdata.com/blog/authors/raz-kaplan) [Raz Kaplan AI GTM Lead ](https://brightdata.com/blog/authors/raz-kaplan) ![Finding the web’s hidden robot training moments in public web video data](https://media.brightdata.com/2026/08/Finding-the-webs-hidden-robot-training-moments-in-public-web-video-data.png) Video data can help train robots, including humanoids, to understand and perform physical tasks. This covers VLA models, which connect visual observations and language instructions to actions, and World models, which learn how an environment changes over time. The public web holds millions of videos of people doing that work, and finding a specific action inside them is hard, because a title describes a video’s general topic while leaving the actions in the footage unnamed. Descriptions and tags are usually no better. For this study we used Bright Data’s [Video Search API](https://brightdata.com/ai/video-data/vla). Video search api is a tool that enables you to visually search moments inside web videos. You hand it a natural-language description of an action, it searches the visual content of public videos rather than the text attached to them, and it returns candidate videos together with timestamps for where the action may appear. Conventional keyword search runs the other way around, matching your words against a video’s title, description, and tags, so the two approaches end up reaching very different footage. We sent it **141 physical actions** and got back **10,828 moments** across **7,549 videos**. For each retained moment, we checked whether any word of the action’s name appeared in its video’s title, description, or tags. In **90.8%** of cases, none did. If you collect training footage by keyword, that 90.8% is the data you miss, and it makes the API’s harvest 10× larger than what text search could reach. Those extra moments show the same actions you asked for, inside videos whose titles talk about something else, and they stay invisible to a text pipeline no matter how many queries you run against these fields. We matched literal substrings against title, description, and tags, and we counted inside the corpus the API itself returned, so the ratio holds under those two conditions. Semantic search, multilingual search, transcript search, and YouTube’s own ranking sit outside what we tested. --- ### The result in three numbers **10.9×**more moments than literal matchcontent search returned 10,828; 992 carry any word of the action’s name in their metadata**90.8%**named nowhere9,836 of 10,828 moments carry no mention of the action in their title, description or tags**56.1%**no clue in the title at alla weaker test than search. We asked google/gemini-2.5-flash whether anything in the title relates to the action at all, however faintly, including clues far too vague to search on. It found nothing in 6,071 of 10,828Where these three numbers come fromThe first two are the same division: 9,836 of 10,828 moments carry no word of the action in any field, and the 992 that do are what a literal matcher reaches, 10,828 ÷ 992 = 10.9. The four buckets (title, description, tags, none) are mutually exclusive and add up to the checked total. Both are counted per match row; collapse them to the 8,849 distinct video-seconds instead and the ratio comes out at 8.9×. The third counts the titles the model rated unrelated: 6,071 of 10,828, one row per moment across 7,549 distinct videos. The timestamps carry a second consequence. Allow each moment at most 10 seconds of action and the **5,066.5 h** of source video these matches sit inside becomes **24.58 h** of clip, which cuts what has to be moved, stored and reviewed by 206.1×. **What retrieval actually returned.** Retrieval being confident is not the same as retrieval being right, so we pulled a real frame from almost every match and had a vision model grade all of them. Section 3.5 has the rates. This audit surfaces candidate footage for discovery and curation, and no policy, world model or robot was trained on it. ### Sample of six videos and their titles on YouTube What a vision model saw, shown the image without the title, against the video’s own title, translated into English where a source language is named. An AI judge read every title in the corpus for any hint of its action, and it rated these six unrelated: a bare date, a Quran verse, an economy news bulletin. We confirmed each frame by eye and each video is live. Click a frame to play the video at that second. The momentWhat the frame showsWhat the video is called[](https://www.youtube.com/watch?v=Q6T5qNlN8k0&t=41s) `mix_cement`Person shoveling cement in mixing tubTawbah surah verse 26. “But (even so) where are you going?” *(from tr)*[](https://www.youtube.com/watch?v=Uk0HDyKdMM8&t=149s) `refuel`Hand inserting fuel nozzle into car fillerBangladesh economy reform challenges and IMF message | Bangladesh Economy | IMF *(from bn)*[](https://www.youtube.com/watch?v=MVabAXq-o40&t=374s) `fold`Two hands folding green paper in halfDecember 24, 2023 *(from zh)*[](https://www.youtube.com/watch?v=O_4ODEXyEs0&t=304s) `cut_knife`Hand slicing avocado on chopping boardFood Grown in Moon Soil… Scientists Got Scared? *(from hi)*[](https://www.youtube.com/watch?v=1Na9zoYj2C4&t=14s) `emergency_stop`Hand striking red mushroom emergency stop button(Cheongju City/Main Line) Woojin Sanjeon Apollo 1100 Electric Ultra-Low-Floor Model PEV Standard 2024 June Edition 2027 Dongil Transportation 712 *(from ko)*[](https://www.youtube.com/watch?v=KKiSvsIFetg&t=13273s) `chisel_carve`Hand guiding chisel into wood carvingadapt and overcome 🌱--- ### 90.8% of retrieved moments are named nowhere in their metadata That share is the gap between what content retrieval reached on this corpus and what literal metadata matching would have reached. The obvious objection is language, since we test English action words against titles that are often not English. Restricting to model-confirmed English titles still leaves the figure at 85.8%; Section 3.3 shows every slice. #### Where the action is named, over all 10,828 retrieved moments Table formatBucketMomentsShareWhat it means for a text pipelinenamed in the title3863.6%literal matcher finds itdescription only5314.9%only if you read descriptionstags only750.7%only if you read tags**named nowhere****9,836****90.8%**literal matcher does not find itOne real moment from each bucket, because “531 moments are named only in the description” is a claim and an actual title with the matched word marked is evidence: BucketActionVideo title (opens at the matched second)Matchnamed in the title`pour`[Teaching a beginner how to pour latte art patterns (Expert Barista Guide)](https://www.youtube.com/watch?v=2P4d7Vq4GSc&t=404s)matched on “pour”description only`take`[GENTLE SOY SAUCE SOUP RAMEN (SHOYU RAMEN) | A Real Japanese Dad Cooks | Japanese cooking](https://www.youtube.com/watch?v=c8Kf9xlqVc0&t=216s)matched on “take”tags only`fold`[ورقة واحدة .. طيّات قليلة .. بدون لاصق.. وعالمٌ هرميٌ كامل!](https://www.youtube.com/watch?v=hljmkQn7uD8&t=89s)matched on “fold”named nowhere`take`[\# viral video in social media # viral koriyaan # sad story of koriyan romance](https://www.youtube.com/watch?v=v3JvHty7O_w&t=40s)no word of “take” appears anywhere in its metadata#### Invisible share, by domain Numbers in table formatDomainNamed nowhereMoments checkedTabletop (DROID)98.1%1,836Outdoor95.9%217Whole body95.4%866Household92.0%402Construction91.1%987Industrial89.6%4,612Automotive83.8%1,908Every matched video in every domain carried scraped metadata, so each n is that domain’s full retrieved set. Nobody titles a video “putting a block in a bowl”; repair videos do announce what they fix, which is why automotive is the one domain a literal matcher partly reaches. --- ### 1 What a timestamp removes from the pipeline Say you want ten seconds of someone soldering a joint. The usual route takes five steps, and every one of them runs on a whole video file: find a video by its title, extraction all of it, search your own copy for the moment, cut that moment out, then label what it shows. Step one misses 90.8% of these moments outright. Searching the footage instead hands back the video *and* the second inside it, and one of the five steps stops existing. There is nothing to search within the extracted video, since you already know where the moment is. Cutting still has to happen, since [Media Data API](https://docs.brightdata.com/api-reference/media-data-api/media-data-api.json) delivers whole videos, audio, captions and storyboards and takes no timestamp, but you cut against a known offset instead of hunting for one. Five steps become four. Offline collection pipelines commonly split that work across several systems: moving video into cloud storage, annotating it frame by frame either manually or semi-automatically, and converting between robotics formats such as ROS or MCAP and general-purpose video formats. Each handoff is a place where throughput is lost and where the timing alignment between camera streams and motor commands can drift. We measured none of that here. It is the context that makes a returned timestamp worth having. **Conventional route** (five steps, paid on whole videos): 1. **Find videos** · keyword search over titles 2. **Extract** · 5,066.5 h of source video *(the expensive step, measured from the scraped duration of every matched video)* 3. **Run a search** · a second tool, over your own copy 4. **Cut** · trim each clip to the moment 5. **Label** · annotate what you cut **With moment retrieval** (four steps, paid on seconds): 1. **Find scenes** · content search returns the second 2. **Retrieve** · ~24.58 h around the returned offsets *(est: assumes each moment is at most 10 seconds of action)* 3. **Cut** · trim locally, against a known offset 4. **Label** · one frame per moment --- ### 2 Method We wrote one sentence describing each of the 141 actions and sent them to [Video Search API](/ai/video-data/vla). The soldering one read: > **one of the 141 sentences we sent** > > close-up of a hand holding a soldering iron to a circuit board joint while feeding solder wire with the other hand, bench lamp lighting We then scraped the title, description, and tags of every video that came back. Bright Data documents the index as matching on sampled frames rather than on title, description or tags, so the fields we graded should not be what found the footage. We did not test that independently. We drew 22 action names from [DROID](https://huggingface.co/datasets/lerobot/droid_1.0.1)‘s task strings, which we read for verb frequency, and added the remaining 119 as hand-authored actions to broaden the audit beyond DROID’s coverage. --- ### 3 Results #### 3.1 What each route costs Everything here is in hours over this corpus, with durations summed from the scraper, so the source figure is a count rather than an estimate. We extracted no video, so we claim no byte count. Two counts sit behind the **206.1×**. The 5,066.5 h is a real sum over 7,549 videos. The clip figure counts each (video, second) once, over 8,849 distinct seconds, and multiplies by the 10-second window. The ratio rests on two choices, how long a clip is and whether the long tail counts, and the long tail is the bigger lever of the two: 483 videos of three hours or more are ~6% of the set and carry ~60% of the hours. The full sensitivity table: Numbers in table formatDrop videos longer thanVideosCorpus hoursDistinct secondsClip hours (10 s)Reduction1 h6,5541,113.57,60521.1252.7×2 h6,8731,563.88,01722.2770.2×3 h7,0662,031.38,24722.9188.7×6 h7,3643,272.88,60923.91136.9×12 h7,5465,012.38,84624.57204.0×keep all (published)7,5495,066.58,84924.58206.1×The same corpus, obtained either way: 10,828 match rows falling on 8,849 distinct video-seconds. Counting every match instead of each distinct second would give 30.08 clip hours and a 168.4× reduction. The same ratio applies to review time only if the alternative reviews every source video in full, review time scales linearly with duration, and stepping through 8,849 separate clips carries no per-clip overhead. #### 3.2 The matching rule, and what tightening it does For each retained candidate moment, we checked the video’s title, description, and tags, the main metadata fields available in this audit. A moment counts as named if any word of the action’s name longer than two characters appears as a case-insensitive substring, so “change” alone marks `change_tire_wrench` and “car” is found inside “carpet”. That is the most permissive rule we could have picked, which makes the headline a floor. Tighten it and it climbs: Every rule stricter than the published one moves the figure up, which is what makes 90.8% a floor rather than a best case: 395 of the 992 moments the published rule counts as named owe that to a fragment inside a longer word. Every value is measured, four real passes over the corpus (`supply_gap.py _variants()`). #### 3.3 Much of it is titled in another language We read a title language for every one of the 10,828 matched moments, so this is the whole corpus rather than a sample. Of them, **4,863 (~45%)** sit in a video titled in one of **77 named non-English languages**, plus one title the model could not place. The same English action descriptions returned 423 moments from Hindi-titled videos and 509 from Portuguese-titled ones. No geographic field was collected, so this says nothing about where the videos were made. *Title language of every matched moment, across all 79 languages found.* That mix is why we sliced the corpus toward English before trusting the headline number. Restricting to titles we know are English moves the figure by a few points and leaves most of it standing, so part of the gap is a language mismatch and the rest is a description failure that survives after the mismatch is removed: The same numbers as a tableSliceNamed nowherenall moments**90.8%**10,828plain-ASCII titles87.6%5,560model-confirmed English titles85.8%5,965confirmed English audio95.0%317 (too few to lean on)#### 3.4 Per action Sorted by the share with no literal action term in the available metadata. 17 actions have no retained moment whose metadata names the target action. Across all 141 actions with at least 8 checked moments the share runs 50.0% to 100.0%, median 94.1%. The 12 least hidden: Numbers in table formatActionDomainFoundSays so in its textNamed nowhere`wash_car`Automotive723650.0%`wash_car_panel`Automotive783653.8%`car_door_open`Automotive923858.7%`car_door_close`Automotive913660.4%`car_door`Automotive933661.3%`pack_box`Industrial732861.6%`grinder_use`Industrial692366.7%`hand_saw_cut`Industrial772567.5%`jack_up_car`Automotive782469.2%`tape_box`Industrial752369.3%`sit_stand`Whole body28871.4%`impact_driver`Industrial701972.9%**All 141 actions** · found / says-so / named-nowhere, plus the index-wide extrapolation per actionActionDomainFoundSays soNamed nowhereEst. moments index-wideUsable rateRate basis`climb_ladder`Whole body930100.0%188,79026.1%action`crane_hook`Industrial840100.0%170,4369.5%action`flip`Tabletop (DROID)870100.0%176,52326.7%action`lift`Tabletop (DROID)890100.0%180,58124.7%action`load_cart`Industrial820100.0%166,4609.8%action`pick_up`Tabletop (DROID)720100.0%146,08847.9%action`push`Tabletop (DROID)910100.0%184,63923.3%action`push_heavy_cart`Whole body700100.0%142,10024.3%action`push_trolley`Industrial650100.0%131,95020.0%action`refuel`Automotive910100.0%184,63926.4%action`refuel_nozzle`Automotive880100.0%178,55227.6%action`seatbelt_buckle`Automotive870100.0%176,52329.9%action`slide`Tabletop (DROID)820100.0%166,37832.1%action`sort_parcel`Industrial820100.0%166,46037.8%action`stack`Tabletop (DROID)650100.0%136,95821.9%action`unfold`Tabletop (DROID)790100.0%160,29157.0%action`ziptie_bundle`Industrial820100.0%166,46048.8%action`carry_upstairs`Whole body87198.9%176,61025.6%action`hang`Tabletop (DROID)92198.9%181,59612.1%action`pull`Tabletop (DROID)87198.9%176,52318.4%action`pull_trolley`Whole body92198.9%186,76022.8%action`close`Tabletop (DROID)83198.8%168,40741.5%action`emergency_stop`Industrial85198.8%172,46522.6%action`handbrake_pull`Automotive82198.8%166,37831.7%action`place`Tabletop (DROID)84198.8%170,43639.3%action`remove`Tabletop (DROID)86198.8%174,49448.8%action`turn_on`Tabletop (DROID)85198.8%172,46529.4%action`count_stock`Industrial76198.7%154,28032.4%action`reach_overhead`Whole body79198.7%160,37046.8%action`wipe`Tabletop (DROID)77198.7%180,58128.0%action`crouch_down`Whole body65198.5%131,95034.9%action`seat_chip`Industrial65198.5%131,95046.9%action`plaster_wall`Construction88297.7%178,55225.3%action`plug_in`Household87297.7%176,52336.5%action`turn`Tabletop (DROID)86297.7%174,49427.9%action`unplug_cable`Industrial88297.7%178,64042.5%action`label_parcel`Industrial85297.6%172,55045.9%action`move`Tabletop (DROID)82297.6%166,37834.2%action`press`Tabletop (DROID)83297.6%168,40742.2%action`spirit_level`Industrial83297.6%168,40712.5%action`conveyor_pick`Industrial79297.5%160,29135.4%action`grout_joint`Construction81297.5%164,34928.4%action`open_hood`Automotive80297.5%162,32053.8%action`trowel_mortar`Construction79297.5%160,29128.2%action`balance_load`Whole body77297.4%156,31021.1%action`shovel_load`Outdoor74297.3%150,22031.5%action`scan_barcode`Industrial71297.2%144,05927.1%action`desolder_remove`Industrial63296.8%127,89051.6%action`pull_wire`Construction61296.7%123,76930.0%action`take`Tabletop (DROID)90396.7%182,61037.8%action`forklift_controls`Industrial59296.6%119,71127.1%action`open_gate`Whole body88396.6%178,64021.6%action`pour`Tabletop (DROID)89396.6%183,62456.2%action`plug_in_socket`Industrial85396.5%172,55036.5%action`shrink_wrap`Industrial81396.3%164,43022.5%action`wipe_windshield`Automotive82396.3%166,37834.6%action`control_dial`Industrial72395.8%146,08831.9%action`steering`Automotive72395.8%146,08859.7%action`open_carton`Industrial69395.7%140,07031.9%action`put`Tabletop (DROID)91495.6%184,63941.8%action`plant_seed`Outdoor67395.5%136,01028.8%action`steering_turn`Automotive43295.3%87,24760.5%action`pipe_fitting`Industrial83495.2%168,40744.6%action`open`Tabletop (DROID)79494.9%173,98744.2%action`dig_soil`Outdoor76494.7%154,28044.6%action`sweep`Household75494.7%152,17529.3%action`route_cable`Industrial74494.6%150,22043.8%action`gear_shift`Automotive73494.5%148,11737.0%action`stack_pallet`Industrial70494.3%142,10012.9%action`thermal_paste`Industrial69494.2%140,07019.1%action`ev_charge_plug`Automotive85594.1%172,46526.2%action`carry_box`Household84594.0%170,43625.0%action`open_trunk`Automotive84594.0%170,43628.6%action`socket_ratchet`Industrial82593.9%166,37831.7%action`top_up_fluid`Automotive82593.9%166,37837.0%action`press_machine_button`Industrial79593.7%160,29131.6%action`valve_turn`Industrial79593.7%160,29119.0%action`insert_usb`Industrial78593.6%158,34046.0%action`caulk_gun`Industrial86693.0%174,49434.1%action`adjust_mirror`Automotive85692.9%172,46520.0%action`sand_surface`Industrial85692.9%172,46555.3%action`cut_pipe`Construction68592.6%137,97241.8%action`machine_unload`Industrial68592.6%137,97230.9%action`unload_truck`Industrial68592.6%138,04035.3%action`machine_lever`Industrial80692.5%162,32021.2%action`assemble_snap_fit`Industrial77692.2%156,31051.3%action`measure_tape`Industrial76692.1%154,20442.1%action`chisel_carve`Industrial49491.8%99,42142.9%action`wrench_bolt`Industrial85791.8%172,46534.1%action`hammer_nail`Industrial84791.7%170,43620.2%action`solder_pipe`Construction70691.4%142,03037.1%action`step_over`Whole body70691.4%142,10028.6%action`gauge_read`Industrial78791.0%158,26235.9%action`fold`Tabletop (DROID)77790.9%158,76957.1%action`tile_place`Construction83890.4%168,40730.1%action`solder`Industrial55689.1%111,59545.5%action`screwdriver`Industrial72888.9%146,08843.1%action`bend_lift`Whole body62788.7%125,86032.3%action`power_drill_hole`Industrial70888.6%142,03040.9%action`solder_joint`Industrial61788.5%123,83055.7%action`mix_cement`Construction68888.2%137,97243.3%action`nail_gun`Construction68888.2%137,97229.8%action`drywall_screw`Construction72987.5%146,08828.2%action`lay_brick`Construction801087.5%162,32027.5%action`clamp_workpiece`Industrial62887.1%125,79835.5%action`machine_load`Industrial62887.1%125,79819.4%action`cut_vegetable`Household831186.7%168,40742.2%action`torque_wrench`Industrial831186.7%168,40731.3%action`cut_knife`Household731086.3%148,11734.2%action`tire_pressure`Automotive731086.3%148,11747.9%action`weld_seam`Industrial871286.2%176,52331.4%action`pliers_grip`Industrial721086.1%146,08850.7%action`paint_brush`Construction901385.6%182,61022.2%action`wire_cut`Industrial761185.5%154,20434.2%action`paint_roller`Construction791383.5%160,29115.2%action`breadboard_wire`Industrial651183.1%131,95044.6%action`change_tire`Automotive691282.6%140,00150.0%action`kneel_work`Whole body551081.8%111,65045.5%action`change_tire_wrench`Automotive711381.7%144,05946.4%action`assemble_part`Industrial701381.4%142,03064.3%action`check_oil_dipstick`Automotive801581.2%162,32043.8%action`screw_panel`Industrial811680.2%164,43037.5%action`lathe_operate`Industrial511178.4%103,47931.4%action`engine_bay`Automotive771777.9%156,23356.6%action`probe_test`Industrial631477.8%127,89054.8%action`machine_control`Industrial781876.9%158,26236.4%action`screwdriver_drive`Industrial782074.4%158,26239.5%action`weld`Industrial862274.4%174,49431.4%action`power_drill`Industrial701874.3%142,03048.6%action`impact_driver`Industrial701972.9%142,03035.3%action`sit_stand`Whole body28871.4%56,84035.7%action`tape_box`Industrial752369.3%152,25026.4%action`jack_up_car`Automotive782469.2%158,26227.3%action`hand_saw_cut`Industrial772567.5%156,23332.9%action`grinder_use`Industrial692366.7%140,00142.6%action`pack_box`Industrial732861.6%148,11750.0%action`car_door`Automotive933661.3%188,69720.4%action`car_door_close`Automotive913660.4%184,63926.4%action`car_door_open`Automotive923858.7%186,66822.8%action`wash_car_panel`Automotive783653.8%158,26241.6%action`wash_car`Automotive723650.0%146,08845.1%actionPer-action findability. “Says so” counts moments whose own title, description or tags carry any word of the action’s name under the published substring rule. The extrapolation columns multiply each action’s measured per-shard yield by the index’s shard count and divide by the measured cross-action overlap; “rate basis” says whether the usable rate was measured on that action’s own frames, inherited from its domain, or inherited from the corpus-wide rate. #### 3.5 Getting the frames To check that retrieval was right at all, we pulled a real frame from each matched video and looked at it. What YouTube exposes without extracting the video is the storyboard, a sprite sheet of thumbnails whose sampling interval widens with runtime: we measured 1 s on videos up to 109 s, 2 s from 175 s to 275 s, 5 s from 454 s to 892 s, and 10 s from 1,366 s upward. The step boundaries between those bands were not resolved by the 12 videos we probed. The image we judged is the nearest sample at or before the matched second, so it can be up to one interval early, 10 s in the worst case and 5 s or 10 s for a little over half the frames. Measured by `storyboard_probe.py`; the per-video specs are in `data/storyboard_spec.json`. A vision model judged all 10,750 of 10,828 frames we obtained, so every rate here is measured on 99.3% of the corpus rather than on a sample. **93.8%** are real footage rather than animation, a game, or a screen recording; **25.7%** caught the action visibly underway and 57.4% showed it underway or set up; **34.5%** scored 3 or better on a usability scale of 0 to 5. That last figure is a judgement about a downstream training process the model cannot observe, and it behaves like one: on a 60-frame sample, four runs of the identical configuration spanned 3.4 points, and re-wording the rubric moved it 20. The action-visible share moved 1.7 across that same rubric change, which makes it the figure to compare across runs. Frame acquisition: Numbers in table formatOutcomeMomentsShareFrames obtained10,750 / 10,82899%No parseable storyboard750.7%Sprite fetch failed30.03%Frame acquisition. We fetched storyboards directly and hit rate limiting on 863 of the 10,828 moments, 856 of which a later pass recovered, so none of the 78 lost moments ended on a 429; [Media Data API](https://docs.brightdata.com/api-reference/media-data-api/media-data-api.json) is the supported route for this step and returns storyboards directly. This audit made no call to it. Those moments sit on 7,549 videos, and a video costs one watch-page fetch plus at least one sprite sheet, cached per video. --- ### 4 The moments, playable Matches shown against the nearest storyboard frame at or before the second they were matched. Ranking is not curated. 10,750 of the 10,828 moments have a frame of their own, and the full gallery holds 10,761 rows because 11 more share an image with another action matched at the same second. Those rows rest on 8,792 distinct images, because the filename names a video and a second rather than an action. Below are the top-scoring matched moments, one per action, straight off the ranking. **Click any frame to play the source video at the matched second.** [](https://www.youtube.com/watch?v=dsOefN6Mny8&t=166s)[](https://www.youtube.com/watch?v=cKl6h0O2n80&t=699s)[](https://www.youtube.com/watch?v=3FQl-uhcrYI&t=136s)`measure_tape` · 0.391 · useful 4/5 Hand using tape measure on wooden board *Shootout review and a work story!*`cut_pipe` · 0.388 · useful 2/5 Hands holding copper pipe cutter tool *How Raw Copper Turns into a Masterpiece with Artistic Hammer Strikes⚒️🔥🇮🇷*`machine_load` · 0.386 · useful 2/5 Pink gloved hands arranging white metal small parts *THK rail | Installation guide | THK rail | Rail removal and installation | 0363.*[](https://www.youtube.com/watch?v=bqZxQKw5WEU&t=340s)[](https://www.youtube.com/watch?v=dNi7ayBp2nk&t=575s)[](https://www.youtube.com/watch?v=_RtV-k39--4&t=494s)`forklift_controls` · 0.383 · useful 5/5 Hands on forklift controls and steering wheel *JUST DRIVE : Forklift #38*`solder` · 0.382 · useful 3/5 Hands near circuit board, unclear soldering setup *⚡ Simple DIY Repair Old Bike Battery And Install Separate USB Charging Port For* `steering_turn` · 0.378 · useful 5/5 Both hands gripping steering wheel, turning *1995 Volvo 850 T5-R Wagon – POV Driving Impressions*[](https://www.youtube.com/watch?v=Acq-JaCaZ4s&t=234s)[](https://www.youtube.com/watch?v=NIUlXM8eO-c&t=439s)[](https://www.youtube.com/watch?v=aDD3H8-GLw8&t=885s)`tape_box` · 0.376 · useful 1/5 Hand near tape rolls and packing supplies *Cell 32140 high discharge line – battery pack accessories 0824457979*`wipe` · 0.374 · useful 3/5 Hands manipulating white fabric on surface *Washed Notebook ASMR 📘✨ Radiator‑Dried Pages With Intense Crinkles*`machine_unload` · 0.373 · useful 1/5 Gloved hands near industrial machine setup *Transmission Assembly – 1929 Ford Model A*[](https://www.youtube.com/watch?v=qugpJY_vgYQ&t=729s)[](https://www.youtube.com/watch?v=Wlw81mG4t74&t=861s)[](https://www.youtube.com/watch?v=GKP-rqZoDuo&t=21082s)`nail_gun` · 0.371 · useful 4/5 Hands positioning nail gun on timber frame *Building a deck for garden raised bed – Part 1*`sit_stand` · 0.371 · useful 5/5 Person rising from chair using armrests *Flexibility and Mobility Senior Chair Exercise*`count_stock` · 0.371 · useful 2/5 Hand writing in notebook near clipboard *\[26.4.16.Thu.\] Study with me | Studying together | Thursday meet-up! | Firefight*[](https://www.youtube.com/watch?v=BsuRceDcme4&t=1358s)[](https://www.youtube.com/watch?v=2aPPTbhPz3w&t=11811s)[](https://www.youtube.com/watch?v=oATd18wPPe8&t=65s)`probe_test` · 0.371 · useful 5/5 Two hands holding multimeter probes on circuit *How to Measure Electronic Components with a Multimeter (Complete Guide)*`open_carton` · 0.369 · useful 2/5 Hands holding bottles near box opening, no cutting *Oddly Satisfying Video-How to Cutting New Wooden Fruits and Vegetables ASMR – Cu*`kneel_work` · 0.369 · useful 3/5 Person prone on floor in work position *Lupimorphie, animal-performance by Régis Moulu, Capsule 114, dance*[](https://www.youtube.com/watch?v=Rc0zu29h0Rg&t=23s)[](https://www.youtube.com/watch?v=KKiSvsIFetg&t=13273s)[](https://www.youtube.com/watch?v=uF8evG-9QH8&t=744s)`push_heavy_cart` · 0.367 · useful 4/5 Person pushing heavy cart with both hands *2321FI”2-Step Multi-Functional Aluminum Alloy Step Trolley: Comprehensive Usage* `chisel_carve` · 0.366 · useful 5/5 Hand guiding chisel into wood carving *adapt and overcome 🌱*`lathe_operate` · 0.362 · useful 5/5 Hand turning lathe cross-slide handwheel with spinning part *Unbelievable Damage! Caterpillar Cylindrical Roller Bearing Destroyed 😱*Each cell: the action it was matched for and its relevance score, then what the vision model saw in the frame (shown the image without the title), then the video’s own title in English. Timestamps open two seconds early so the moment plays into view. --- ### 5 Related work Retrieving a moment by description is an established task: temporal grounding on [Charades-STA](https://arxiv.org/abs/1705.02101), highlight detection on [QVHighlights](https://arxiv.org/abs/2107.09609), natural language queries in [Ego4D](https://arxiv.org/abs/2110.07058). We introduce no new method here. The construct has precedent. [TVR](https://arxiv.org/abs/2001.09099) (2020) labelled its 108,965 queries by whether video, subtitles or both answered them, reporting **74.2% video-only**. What differs is the corpus and the purpose: open web video rather than six TV shows, physical actions, a verified subsample, and a cost ratio. Release format follows [AVA](https://research.google.com/ava/download.html) and [HowTo100M](https://www.di.ens.fr/willow/research/howto100m/): identifier, timestamp and label, with no video redistributed. Retrieval throughout is [Video Search API](/ai/video-data/vla). Frames here were obtained by parsing the YouTube storyboard off the watch page; no clip was extracted at any point. For a production run the supported route is [Media Data API](https://docs.brightdata.com/api-reference/media-data-api/media-data-api.json). What the audit establishes is narrower than a replacement for robot demonstrations. The web holds a large and poorly indexed layer of physical-action footage, and timestamped visual search makes that layer considerably easier to reach. Whether footage curated this way measurably improves a robotics or world-model system is the next question, and answering it needs a training experiment this audit did not run. --- ### 6 Citation ```none @misc{webmoments2026, title = {Finding the Web's Hidden Robot Training Moments in Public Web Video Data}, author = {Bright Data}, year = {2026}, howpublished = {Technical report}, note = {141 physical actions, 10,828 retrieved match rows over 8,849 distinct video-second moments, and 10,750 frames reviewed by a vision model} } ``` Contact usStart free trial No credit card required [ ](https://www.linkedin.com/in/razkaplan/) Raz Kaplan AI GTM Lead [ View all articles ](https://brightdata.com/blog/authors/raz-kaplan) Table of Contents Data for AI Supercharge your AI with instant and reliable access to web data. No blockers. No hassle. Talk to an expert Bright Data MCP Get started with Bright Data’s Web MCP Server today with 5000 free monthly requests and unlock your AI’s full potential. Start free now [ ](https://news.ycombinator.com/submitlink?t=Finding+the+web%E2%80%99s+hidden+robot+training+moments+in+public+web+video+data&u=https://brightdata.com/blog/ai/robot-training-data-in-public-web-video) [ ](https://www.linkedin.com/shareArticle?mini=true&title=Finding+the+web%E2%80%99s+hidden+robot+training+moments+in+public+web+video+data&url=https://brightdata.com/blog/ai/robot-training-data-in-public-web-video) [ ](http://www.reddit.com/submit?title=Finding+the+web%E2%80%99s+hidden+robot+training+moments+in+public+web+video+data&url=https://brightdata.com/blog/ai/robot-training-data-in-public-web-video) ## You might also be interested in [ ](https://brightdata.com/blog/ai/openhuman-with-bright-data "Production-Ready Web Access in OpenHuman Through the Bright Data CLI") [AI Antonello Zanini Technical Writer ### Production-Ready Web Access in OpenHuman Through the Bright Data CLI Integrate Bright Data CLI with OpenHuman to enable production-ready web access and data collection for AI agents. 09-Sep-2026 12 min read ](https://brightdata.com/blog/ai/openhuman-with-bright-data) [ ](https://brightdata.com/blog/ai/minimax-m3-with-bright-data "Giving self-hosted MiniMax M3 agents live web access with Bright Data") [AI Satyam Tripathi Technical Writer ### Giving self-hosted MiniMax M3 agents live web access with Bright Data Self-hosted MiniMax M3 agents get live web access using Bright Data’s search and scraping tools. Bypass blocks and CAPTCHAs. 09-Sep-2026 54 min read ](https://brightdata.com/blog/ai/minimax-m3-with-bright-data) [ ](https://brightdata.com/blog/web-data/multimodal-web-scraping-with-minimax "Multimodal Web Scraping with MiniMax") [Web Data Antonello Zanini Technical Writer ### Multimodal Web Scraping with MiniMax Pair Bright Data Web Unlocker with MiniMax M3 vision to extract structured data from images and web page screenshots. 09-Sep-2026 4 min read ](https://brightdata.com/blog/web-data/multimodal-web-scraping-with-minimax)