Skip to content

Entry

Trafilatura

Appears in 4 awesome lists

Python & Command-line tool to gather text and metadata on the Web - Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

Open github.comadbar/trafilatura

Found in these lists

Awesome Data Analysis

Section: Tools · A Python & command-line tool to gather text and metadata on the web.

FreshScore 79

awesome-python

Section: Scraping and Crawling · Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

FreshScore 79

Awesome AI Agents: Tools, Resources, and Projects

Section: Tools · Python & Command-line tool to gather text and metadata on the Web - Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

SlowScore 66

Awesome Python

Section: Web Scraping · A tool for gathering text and metadata from the web, with built-in content filtering.

FreshScore 100

browser-use

Open-source browser agent (Python and TypeScript) whose loop injects serialized DOM state each step, exposes browser actions as registered tools, and runs the same code against a local Chrome or a cloud browser. It has grown from a minimal reference into a full framework (~70k lines of Python…

In 11 listsDetails

Selenium

Portable software testing framework for web applications. License: Apache 2 , .

In 9 listsDetails

Awesome LangChain

😎 Awesome list of tools and projects with the awesome LangChain framework

In 4 listsDetails

DemoGPT

DemoGPT enables you to create quick demos by just using prompt. It applies ToT approach on Langchain documentation tree.

In 9 listsDetails

AI News

Collection of AI-related utilities. Welcome to submit issues and pull requests /收藏AI相关的实用工具,欢迎提交issues 或者pull requests

In 3 listsDetails

playwright-python

Python version of the Playwright testing and automation library.

In 6 listsDetails

scrapy

Fast, high-level web crawling and scraping framework for Python. Extract structured data from websites at scale with built-in support for handling common challenges like pagination, cookies, and concurrent requests. BSD-3-Clause licensed.

In 6 listsDetails

Crawl4AI

Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key. Join our Discord https://discord.gg/VuxzM8mtf

In 5 listsDetails