aljaz/Tutorial-Codebase-Knowledge
1
0
Fork 0
mirror of https://github.com/aljazceru/Tutorial-Codebase-Knowledge.git synced 2026-01-03 23:04:26 +01:00
Code Issues Packages Projects Releases Wiki Activity

update readme

Browse Source
This commit is contained in:
zachary62
2025-04-04 17:29:26 -04:00
parent 0d1b43574e
commit 180e0d9f67
18 changed files with 36 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/Crawl4AI/index.md
View File

@@ -7,6 +7,8 @@ has_children: true
# Tutorial: Crawl4AI
> This tutorial is AI-generated! To learn more: https://github.com/The-Pocket/Tutorial-Codebase-Knowledge
`Crawl4AI` is a flexible Python library for *asynchronously crawling websites* and *extracting structured content*, specifically designed for **AI use cases**.
You primarily interact with the `AsyncWebCrawler`, which acts as the main coordinator. You provide it with URLs and a `CrawlerRunConfig` detailing *how* to crawl (e.g., using specific strategies for fetching, scraping, filtering, and extraction).
It can handle single pages or multiple URLs concurrently using a `BaseDispatcher`, optionally crawl deeper by following links via `DeepCrawlStrategy`, manage `CacheMode`, and apply `RelevantContentFilter` before finally returning a `CrawlResult` containing all the gathered data.