Skip to content

scraper: update VirtualRealPorn for the new site - #2244

Open
DarXV05 wants to merge 1 commit into
xbapps:masterfrom
DarXV05:scraper/virtualrealporn-site-update
Open

scraper: update VirtualRealPorn for the new site#2244
DarXV05 wants to merge 1 commit into
xbapps:masterfrom
DarXV05:scraper/virtualrealporn-site-update

Conversation

@DarXV05

@DarXV05 DarXV05 commented Aug 8, 2026

Copy link
Copy Markdown

Builds on #2233 by @ShadowOps, which covers the selector migration. Credited via Co-authored-by.

Problem

The site moved to Laravel Livewire and every selector the scraper used is gone. Checked against a live scene page:

selector matches
script#virtualreal_video-streaming-js-extra (scene ID) 0
script#virtualreal_download-links-js-extra (filenames) 0
div.metaSingleData a span (tags) 0
figure[itemprop="associatedMedia"] a (gallery) 0

Scene ID comes from the first, and the scraper ends with if sc.SceneID != "" { out <- sc } — so it emits nothing and returns no error. Silent failure.

Additional fixes beyond #2233

Found while verifying that PR against the live site:

Performer names carry a " VR" suffix on the new site. Stored verbatim, they create a second actor record alongside the existing one. Stripped.

Trailers resolve zero sources. The PR keeps TrailerType: "scrape_json" pointed at trailer.contentUrl in the JSON-LD, but the VideoObject no longer has a trailer key at all — its keys are @context, @type, actor, description, duration, genre, interactionStatistic, name, thumbnailUrl, uploadDate, url. The stream sources are on the player element instead, which is the shape badoink.go already handles: scrape_html over dl8-video source. Verified to return 5 sources with quality labels.

Covers silently rejected in some builds. image.Decode validates covers, but pkg/scrape registers no image format decoder. It works in the shipped binary only because pkg/server/heatmapproxy.go happens to import image/jpeg — nothing in pkg/scrape guarantees it, and a build excluding pkg/server rejects every cover. Registers gif/jpeg/png explicitly.

Verification

Single-scene scrape against the live site:

scene_id   virtualrealporn-<id>    duration  42 min
title      <ok>                    released  2026-02-13
cast       2 (suffix stripped)     tags      8
covers     1                       filenames 25
gallery    7                       trailer   scrape_html / dl8-video source

Listing pagination also verified: /videos/?page=1 yields 20 scene links via a.data-title.

The site moved to Laravel Livewire and every selector the scraper used is gone,
so it matched nothing, left SceneID empty and returned without emitting a scene
or an error. Verified against a live scene page: the old scene-ID, tag, gallery
and download-link selectors all return zero elements.

Builds on xbapps#2233 by ShadowOps, which covers the selector migration, with three
further fixes found while verifying it against the live site:

- Performer names carry a " VR" suffix on the new site. Stored verbatim they
  create a second actor record beside the existing one, so strip the suffix.

- The JSON-LD VideoObject no longer has a "trailer" object, so the trailer
  config pointed at a field that no longer exists and resolved zero sources.
  The stream sources are on the dl8 player element instead, which is the shape
  badoink.go already handles: scrape_html over `dl8-video source`.

- image.Decode is used to validate covers, but this package registers no image
  format decoder. It worked only because pkg/server happens to import
  image/jpeg; nothing in pkg/scrape guaranteed it, and a test binary that omits
  pkg/server rejected every cover. Register gif, jpeg and png explicitly.

Co-authored-by: ShadowOps <1622411+ShadowOps@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant