Changelog
Changelog
All notable changes to this project will be documented in this file.
2.0.1 (2024-10-25)
🚀 Features
- Add standby URL, change default standby port (#287) (8cd2f2c) by @jirimoravcik
- Add crawlee version to system info print (#304) (c28f38f) by @vdusek
🐛 Bug Fixes
- Adjust tests of scrapy user data (#284) (26ffb15) by @janbuchar
- Use HttpHeaders type in Scrapy integration (#289) (3e33e91) by @vdusek
- Allow empty timeout_at env variable (#303) (b67ec98) by @janbuchar, closes #596
2.0.0 (2024-09-10)
- Check the Upgrading to v2.0 guide.
🚀 Features
- Better Actor API typing (#256) (abb87e7) by @janbuchar, closes #243
- Expose Request from Crawlee (#266) (1f01278) by @vdusek
- Automatically configure logging (#271) (1906bb2) by @janbuchar
🐛 Bug Fixes
- Make apify.log public again (#249) (22677f5) by @janbuchar
- Dataset list response handling (#257) (0ea57d7) by @janbuchar
- Ignore deprecated platform events (#258) (ed5ab3b) by @janbuchar
- Possible infinity loop in Apify-Scrapy proxy middleware (#259) (8647a94) by @vdusek
- Hotfix for batch_add_requests batch size limit (#261) (61d7a39) by @janbuchar
Refactor
- Preparation for v2 release (#210) (2f9dcc5) by @janbuchar, closes #135, #137, #138, #147, #149, #237
1.7.2 (2024-07-08)
🚀 Features
- Add actor standby port (#220) (6d0d87d) by @jirimoravcik
1.7.1 (2024-05-23)
🐛 Bug Fixes
- Set a timeout for Actor cleanup (#206) (cfed57d) by @janbuchar, closes #200
1.7.0 - 2024-03-12
Added
- Add a new way of generating the
uniqueKey
field of the request, aligning it with the Crawlee.
Fixed
- Improve error handling for
to_apify_request
serialization failures - Scrapy's
Request.dont_filter
works.
1.6.0 - 2024-02-23
Fixed
- Update of Scrapy integration, fixes in
ApifyScheduler
,to_apify_request
andapply_apify_settings
.
Removed
- Removed
ApifyRetryMiddleware
and stay with the Scrapy's default one
1.5.5 - 2024-02-01
Fixed
- Fix conversion of
headers
fields in Apify <--> Scrapy request translation
1.5.4 - 2024-01-24
Fixed
- Fix conversion of
userData
andheaders
fields in Apify <--> Scrapy request translation
1.5.3 - 2024-01-23
Added
- Add
apply_apify_settings
function to Scrapy subpackage
1.5.2 - 2024-01-19
Internal changes
- Create a new subpackage for Scrapy pipelines
- Remove some noqas thanks to the new Ruff release
- Replace relative imports with absolute imports
- Replace asserts with custom checks in Scrapy subpackage
Fixed
- Add missing import check to
ApifyHttpProxyMiddleware
1.5.1 - 2024-01-10
Internal changes
- Allowed running integration tests from PRs from forks, after maintainer approval
- Do not close
nested_event_loop
in theScheduler.__del__
1.5.0 - 2024-01-03
Added
- Added
ApifyHttpProxyMiddleware
1.4.1 - 2023-12-21
Fixed
- Resolved issue in
ApifyRetryMiddleware.process_exception()
, where requests were getting stuck in the request queue
Internal changes
- Fixed type hint problems for resource clients
1.4.0 - 2023-12-05
Internal changes
- Migrate from Autopep8 and Flake8 to Ruff
1.3.0 - 2023-11-15
Added
- Added
scrapy
extra
1.2.0 - 2023-10-23
Added
- Added support for Python 3.12
Internal changes
- Fix lint error (E721) in unit tests (for instance checks use
isinstance()
)
1.1.5 - 2023-10-03
Added
- Update the Apify log formatter to contain an option for adding the logger name
Internal changes
- rewrote documentation publication to use Docusaurus
- removed PR Toolkit workflow
1.1.4 - 2023-09-06
Fixed
- resolved issue with querying request queue head multiple times in parallel
Internal changes
- fixed integration tests for Actor logger
- removed
pytest-randomly
Pytest plugin - unpinned
apify-client
andapify-shared
to improve compatibility with their newer versions
1.1.3 - 2023-08-25
Internal changes
- unified indentation in configuration files
- update the
Actor.reboot
method to use the new reboot endpoint
1.1.2 - 2023-08-02
Internal changes
- started importing general constants and utilities from the
apify-shared
library - simplified code via
flake8-simplify
- started using environment variables with prefix
ACTOR_
instead of some with prefixAPIFY_
- pinned
apify-client
andapify-shared
to prevent their implicit updates from breaking SDK
1.1.1 - 2023-05-23
Fixed
- relaxed dependency requirements to improve compatibility with other libraries
1.1.0 - 2023-05-23
Added
- option to add event handlers which accept no arguments
- added support for
is_terminal
flag in status message update - option to set status message along with
Actor.exit()
Fixed
- started enforcing local storage to always use the UTF-8 encoding
- fixed saving key-value store values to local storage with the right extension for a given content type
Internal changes
- switched from
setup.py
topyproject.toml
for specifying project setup
1.0.0 - 2023-03-13
Internal changes
- updated to
apify-client
1.0.0 - started triggering base Docker image builds when releasing a new version
Fixed
- fixed
RequestQueue
not loading requests from an existing queue properly
0.2.0 - 2023-03-06
Breaking changes
- fixed local
MemoryStorageClient
not handling the difference between storageid
andname
correctly
Added
- Added the
KeyValueStore.get_public_url()
method to get public URLs of key-value store records
Fixed
- fixed parsing messages from the platform events websocket when they have no event data
- fixed
EventManager
not waiting for platform events websocket connection during initialization - fixed local
RequestQueueClient
not respecting theforefront
argument - fixed local
RequestQueueClient
not counting thehandledRequestCount
property - fixed local storage operations possibly running in parallel
- stopped calling
sys.exit()
in a nested asyncio loop - stopped purging storages by default
Internal changes
- started running unit tests in CI on Windows runners in addition to Linux
- added unit tests for environment variables handling
- added unit tests for the
Configuration
class - added unit tests for the
EventManager
class - added more Flake8 plugins and fixed issues they reported
0.1.0 - 2023-02-09
Initial release of the package.