Log in

hypeScraper

hypeScraper

Elgg 3.0

A tool for scraping, caching and embedding remote resources.

Features

  • Scrapes URLs and turns them in responsive preview cards
  • Aggressive caching of scraped resources for enhanced performance
  • Linkifies #hashtags, @usernames, links and emails

Card viewCard mobilePlayer

Developer notes

Card

To display a URL card with an image preview, title and brief description, use output/card view:

echo elgg_view('output/card', array(
    'href' => 'https://www.youtube.com/watch?v=Dlf1_vuIR4I',
));

Player

To dipslay a rich media player use output/player view:

echo elgg_view('output/player', array(
    'href' => 'https://www.youtube.com/watch?v=Dlf1_vuIR4I',
));

Linkify

To linkify all URLs, usernames, emails and hashtags that are not wrapped in html tags, use output/linkify view. Pass your text in a value parameter. You can use parse_ flags to skip certain qualifiers.

$text = '@someone needs to #linkify this article http://example.com and email it to someone@example.com';
if (elgg_view_exists('output/linkify')) {
    $text = elgg_view('output/linkify', array(
        'value' => $text,
        //'parse_urls' => false,
        //'parse_hashtags' => false,
        //'parse_usernames' => false,
        //'parse_emails' => false,
    ));
}

To generate a preview for multiple URLs extracted from text, use output/url_preview view. Pass your text as a value parameter. The view will parse all URLs and generate previews.

$text = 'This video is really cool https://vimeo.com/channels/staffpicks/116498390';
if (elgg_view_exists('output/url_preview')) {
    $text = elgg_view('output/url_preview', array(
        'value' => $text,
    ));
}

Package Releases

VersionRelease Date
hypeScraper 6.2.16.2.1
hypeScraper 6.2.06.2.0
hypeScraper 6.1.16.1.1
hypeScraper 6.1.06.1.0
hypeScraper 6.0.36.0.3
hypeScraper 6.0.26.0.2
hypeScraper 6.0.16.0.1
hypeScraper 6.0.06.0.0

Changelog

6.2.0 (2018-06-29)

Bug Fixes

  • dev: use service facade (da4f97f)
  • extract: make sure html entities are decoded before extracting tokens (ef93384)
  • fields: sync with new fields API (d9fa8d0)
  • river: fix attachment (5321107)

Features

  • core: align with latest core changes (9acff29)
  • core: updates and fixes (cc41b48)

6.1.1 (2018-05-07)

Bug Fixes

6.1.0 (2018-04-25)

Features

  • core: align with latest core changes (9acff29)

6.0.3 (2018-04-25)

Bug Fixes

  • extract: make sure html entities are decoded before extracting tokens (ef93384)

6.0.2 (2018-04-17)

Bug Fixes

6.0.1 (2018-03-28)

Bug Fixes

  • fields: sync with new fields API (d9fa8d0)

6.0.0 (2018-03-19)

5.5.3 (2018-03-19)

Bug Fixes

  • embedding: first return value from getoEmbedDomains now correct (38f7f95)

Features

  • releases: upgrade for Elgg 3 (56c6e1e)

5.5.2 (2017-10-17)

Bug Fixes

  • scraper: set session cookie to allow scraping session-bound local resources (dafe27c)

5.5.1 (2017-10-17)

Bug Fixes

  • oembed: correctly split multiline oembed domain config (8570b9b), closes #52

5.5.0 (2017-06-26)

Features

5.4.1 (2017-06-26)

Bug Fixes

  • images: add a hotfix to replace image URLs after datastore migration (15a1885)

5.4.0 (2017-05-31)

Features

  • admin: add option to clear all cached URLs by domain/w (2164593)

5.3.0 (2017-05-12)

Features

  • flush: add an option to flush scraped URL data (5930a12), closes #49

5.2.2 (2017-04-19)

Bug Fixes

  • card: avoid class name collision (942b7a4)

5.2.1 (2017-04-16)

Bug Fixes

  • card: avoid class name collision (942b7a4)

5.2.0 (2017-03-22)

Features

  • card: make card title an anchor link (d20acff)
  • cards: admins can now edit card title, desc, player and image (1ee69de)
  • css: slightly reworks the card styles (c7d0d94)
  • security: implement a whitelist of oembed domains (0608b91)

5.1.7 (2017-03-13)

5.1.6 (2017-02-28)

Bug Fixes

5.1.5 (2017-02-28)

5.1.4 (2017-02-26)

Bug Fixes

  • cache: no longer throw exception on invalid url (50dbaec)
  • parsing: better url filtering, no longer throw exception on fail (9968595)

5.1.3 (2017-02-18)

Bug Fixes

  • memory: set upper threshold for parseable image width (60ca20d)

5.1.2 (2017-02-18)

Bug Fixes

  • memory: ensure thumbnails are images and resizable (9e5ce74), closes #42

5.1.1 (2017-02-06)

Bug Fixes

  • scraper: minimize the risk of fatal impact (0e7fdf7)

5.1.0 (2017-02-03)

Features

  • admin: adds an admin interface for previewing cards (e53f92e)

5.0.3 (2016-12-08)

5.0.2 (2016-12-08)

5.0.1 (2016-12-08)

Bug Fixes

  • linkify: exclude HTML tag attribute names and values from token matches (56caba3)

5.0.0 (2016-11-16)

Bug Fixes

  • cache: adds filesize sanity checks before generating thumbnails (c7e5d0c)
  • css: make sure nested flex cards preserve their padding values (e9cddb7)
  • embed: only allow iframe, video and audio tags in embed html (cd23401), closes #14
  • scraper: ensure that urls are valid before scraping them (bdf060a)
  • thumbs: fix class namespace (e054447)
  • upgrade: normalize and filter URLs during upgrade (0ec03f8)

Features

  • cache: add restrictions on parsing and caching of images (5a6ec64)
  • core: update requirements (e339264)
  • elgg: now requires Elgg 2.3 (a8f750c)
  • images: use new image resize library (a458f1d)
  • player: add an option to disable player fallback (e9cef19)
  • release: 5.0 RC release (4fb1b5a)
  • releases: upgrade to latest Elgg, drop hypeApps requirement (d9c5491)
  • thumbs: improve thumbs handling (5d3f816)
  • upgrades: add a memory-safe upgrade script (a19da0a)

BREAKING CHANGES

  • elgg: Now requires Elgg 2.3
  • release: Most of the API has been rewritten to reduce complexity. The hooks will continue working, but hypeScraper() function has been removed along side all of the DI services. Support for embed.ly and iframely have been removed. All the legacy and deprecated functions and views have been removed.
  • releases: Now requires Elgg 2.2 or higher. Drops requirement for hypeApps and subsequently affects all APIs that rely on its functionality
  • thumbs: Drops /server/thumbs.php Drops \hypeJunction\Scraper\ThumbServer class
  • core: Now requires Elgg 2.1

4.2.2 (2016-01-06)

Bug Fixes

  • thumbs: thumb URLs are now normalized to site root (8105d17)

About

Dependencies

Requires
Elgg version >= 3.0
PHP extension: curl
PHP extension: xml
Suggests
Plugin: hypeShortcode Any