Homepage

v1.0.0-beta8.3

Highlights

  • Support node 16 & 18
  • Drop node 12
  • Support generating and running typescript projects. More info here
  • New headless chromium detection patches that pass all current tests
  • New commands to update subcomponents

    • update-chrome Improved chromium downloader. See command options here
    • update-ua Updates the builtin database of user agent strings
    • update-stealth Updates the headless chromium detection patches
  • Documentation moved to github pages here: https://ayakashi-io.github.io

Updating

Update to the latest version by running:

npm update -g ayakashi

A new @ayakashi/types package is also available. It can be updated in existing projects by running npm --save-dev @ayakashi/types

Highlights

  • add support for node 14
  • drop support for node 8, 10 and 13
  • dependency updates

Updating

Update to the latest version by running:

npm update -g ayakashi

A new recommended Chromium revision is also available. It can be updated by running ayakashi update-chrome

Highlights

  • support node 13

    • LTS versions (10 or 12) are still the recommended versions
  • trackMissingChildren() will now cascade to nested child props
  • some type improvements

Updating

Update to the latest version by running:

npm update -g ayakashi

A new @ayakashi/types package is also available. It can be updated in existing projects by running npm --save-dev @ayakashi/types

A new recommended Chromium revision is also available. It can be updated by running ayakashi update-chrome

A maintenance release focusing mainly on bug fixes, small improvements and test coverage.

Highlights

  • support node 12
  • evaluate()

    • parameter and return types are now fully inferred
    • improved serialization allows passing and returning complex objects and functions
    • evaluateAsync() now supports async functions
  • support default exports on actions, preloaders, extractors and props
  • type improvements
  • new integration test suite
  • many bug fixes

Updating

Update to the latest version by running:

npm update -g ayakashi

A new @ayakashi/types package is also available. It can be updated in existing projects by running npm --save-dev @ayakashi/types

  • Cookies are now automatically shared between page loads, renderless loads and HTTP requests. New cookie methods for manual manipulation have also been added. Read more here
  • Persistent sessions have been completely revamped. They now allow chromium contexts, cookies and other scraper settings to be shared between multiple steps of a pipeline. Read more here
  • User agents have been improved and they now include more settings to tweak. The options have been moved to the emulatorOptions section of the config
  • New workers and workerConcurrency config options have been added to allow manual control of the thread count and concurrency of the system
  • domQL data attribute querying is now more flexible. Read more here
  • Simple mode is now available for renderlessScrapers and apiScrapers with the --simpleRenderless and --simpleApi CLI flags
  • Bug fixes and improvements

Update to the latest version by running:

npm update -g ayakashi

A new @ayakashi/types package is also available. It can be updated in existing projects by running npm --save-dev @ayakashi/types

  • The extraction API has been revamped

    • extract() will now return just the extracted values instead of wrapper objects. Learn more in the updated documentation
    • HTML attributes and data-attributes are now supported
    • Multiple extractions in the same extract() call and nested extractions have been removed
    • New methods extractFirst() and extractLast() can be used to extract a single value of the first or the last match respectively
    • New method join() can be used to properly group multiple sets of related data
    • Inferred return types have been greatly improved
  • The builtin scripts have been updated to reflect the new extraction semantics
  • Primitive values can now be yielded as output from scrapers or scripts without the need to be wrapped in objects
  • Random protocol and bridge ports can be used by setting them to 0 in the config
  • Multiple projects can now be run at the same time without interfering with each other
  • Multiple resumable sessions of the same project can exist and run (even at the same time) by using the --sessionKey CLI argument

Update to the latest version by running:

npm update -g ayakashi

A new @ayakashi/types package is also available. It can be updated in existing projects by running npm --save-dev @ayakashi/types

Update to the latest version by running:

npm update -g ayakashi

A new @ayakashi/types package is also available. It can be updated in existing projects by running npm update.

Update to the latest version by running:

npm update -g ayakashi
  • Each scrapper invocation now runs in its own browser context (like an incognito window) unless persistentSession is on or not running in headless mode.
  • Add support for renderless scrappers. Learn more here: https://ayakashi.io/docs/guide/renderless-scrappers.html
  • Add support for API scrappers. Learn more here: https://ayakashi.io/docs/guide/api-scrappers.html
  • saveToSQL script will now re-use the database connection and won't create a new one each time it is run.
  • Add recursiveYield() and recursiveYieldEach() methods, which allow to recursively re-run the scrapper by yielding the extracted data to itself.
  • domQL tagName queries will now accept both lowercase and uppercase tag names.

Update to the latest version by running:

npm update -g ayakashi

A new @ayakashi/types package is also available. It can be updated in existing projects by running npm update.

  • use a new, more efficient chromium downloader
  • start using a global store folder
  • use a recommended chromium revision instead of downloading the latest one
  • chromium will now be downloaded only once in the global store folder (instead of a local .chromium directory)
  • local .cache and .session directories are also moved to the global store
  • show a notification if there is a new update or chromium revision

Update to the latest version by running:

npm update -g ayakashi