Skip to content

AdguardTeam/tsurlfilter

Repository files navigation

Extensions libraries

badge-open-issues badge-closed-issues badge-license

This mono-repository contains a collection of TypeScript libraries which are used in AdGuard browser extensions and other projects.

Packages

The following packages are available in this repository:

Package Name Description
logger Logging library for AdGuard extensions.
css-tokenizer A fast, spec-compliant CSS tokenizer for standard and Extended CSS.
agtree Universal adblock filter list parser which produces a detailed AST.
tsurlfilter A library that enforces AdGuard's blocking rules logic.
tswebextension Wraps the web extension API for use with tsurlfilter.
adguard-api Manages filter lists and ad filtering via tswebextension.
adguard-api-mv3 MV3 compatible version of adguard-api.
dnr-rulesets Utility to load prebuilt AdGuard DNR rulesets for mv3 extensions.
examples/manifest-v2 Example using Manifest V2.
examples/manifest-v3 Example using Manifest V3.
examples/tswebextension-example Example for tswebextension.
examples/tswebextension-mv3 Example for tswebextension using Manifest V3.

Detailed information on each package is available in the ./packages directory.

Development

Prerequisites

Ensure that the following software is installed on your computer:

Note

For development, our team uses macOS and Linux. It may be possible that some commands not work on Windows, so if you are using Windows, we recommend using WSL or a virtual machine.

Environment Setup

Install dependencies with pnpm: pnpm install.

Note

If you want to use another linked packages in monorepo workspace, link it in root folder.

This repository uses pnpm workspaces and Lerna to manage multiple packages in a single repository.

Catalogs

This repository also uses pnpm catalogs to manage dependencies. It ensures that common dependencies have the same version for all packages, which reduces version conflicts and simplifies dependency maintenance.

All common dependencies are listed in pnpm-workspace.yaml, so if any update is needed, you should update it there.

Development Commands

  • Runs tests in all packages: npx lerna run test
  • Lint all packages: pnpm lint
  • Remove node_modules from all packages and root package: pnpm clean
  • Builds the packages in the current repo: npx lerna run build
  • Builds a specific package: npx lerna run build --scope=<package-name>
    • For example, to build the tswebextension package: npx lerna run build --scope=@adguard/tswebextension. This command also builds @adguard/tsurlfilter first as it is required for @adguard/tswebextension.
  • Increment a specific package: pnpm run increment <package-name>. This command increments the patch or prerelease version.

Note

You can find Lerna commands in the following link: Lerna Commands.

Linking packages from this monorepo to other projects

When linking packages from this monorepo to projects that don't use pnpm, you need to be careful about package hoisting. pnpm uses a nested node_modules structure by default, which may not be compatible with other package managers that use flat structures.

If you need to link packages to a non-pnpm environment, you can force pnpm to use a flat structure by using the --shamefully-hoist flag when installing dependencies:

pnpm install --shamefully-hoist

For projects using pnpm, you can use pnpm link to connect the packages locally. For more details, please check the pnpm documentation.

Notice about zod package versions

Within this monorepo, zod is utilized for data validation. There are instances where a zod schema is exported from one package for use in another. However, this can potentially lead to issues if the zod versions across these packages differ. For more context, refer to this issue.

To prevent this problem, the same zod version must be used across all packages in the monorepo. That's why we use pnpm catalogs.

Sample extensions

Source code of sample extensions can be found in ./packages/examples directory.

You can build them using the following commands:

  • MV2 sample extension: npx lerna run build --scope tswebextension-mv2
  • MV3 sample extension: npx lerna run build --scope tswebextension-mv3
  • AdGuard API example: npx lerna run build --scope adguard-api-example

To test if this extension works correctly you can use the following test pages:

Test pages:

VSCode Workspace

If you're using Visual Studio Code for development, it may be easier to work with the monorepo if you use the workspace functionality. To do this, create a tsurlfilter.code-workspace file in the monorepo root directory.

{
    "folders": [
        { "path": "packages/logger" },
        { "path": "packages/css-tokenizer" },
        { "path": "packages/agtree" },
        { "path": "packages/tsurlfilter" },
        { "path": "packages/tswebextension" },
        { "path": "packages/dnr-rulesets" },
        { "path": "packages/adguard-api" },
        { "path": "packages/adguard-api-mv3" },
        { "path": "packages/examples/adguard-api" },
        { "path": "packages/examples/adguard-api-mv3" },
        { "path": "packages/examples/tswebextension-mv2" },
        { "path": "packages/examples/tswebextension-mv3" }
    ]
}

Dependencies and dependent packages

In this monorepo, packages can depend on each other using workspace references instead of specific versions. This is indicated by workspace:^ in the package.json dependencies section. When a package depends on another package in the workspace, changes to the dependency may require updates to the dependent package's changelog.

Independent Packages

These packages do not depend on other monorepo packages:

  • @adguard/css-tokenizer
  • @adguard/logger

Dependency Tree

Below is the dependency relationship between packages:

  • @adguard/agtree depends on:

    • @adguard/css-tokenizer
  • @adguard/tsurlfilter depends on:

    • @adguard/agtree
    • @adguard/css-tokenizer
  • @adguard/tswebextension depends on:

    • @adguard/agtree
    • @adguard/logger
    • @adguard/tsurlfilter
  • @adguard/dnr-rulesets depends on:

    • @adguard/logger
    • @adguard/tsurlfilter as devDependency
  • @adguard/api depends on:

    • @adguard/tswebextension
  • @adguard/api-mv3 depends on:

    • @adguard/tswebextension

When making changes to a package, consider updating the changelogs of all dependent packages that might be affected by your changes. For example, if you make changes to @adguard/agtree, you should also update the changelog of @adguard/tsurlfilter and @adguard/tswebextension.

To summarize the dependency tree, here is a scheme of the dependency tree:

┌───────────────┐ ┌───────────────┐
│ css-tokenizer │ │    logger     │
└──┬────────────┘ └────────────┬──┘
   │   ┌────────────────┐      │
   ├──►│     agtree     ├───┐  │
   │   └────────────────┘   │  │
   │   ┌────────────────┐   │  │
   └──►│                │   │  │
       │  tsurlfilter   │◄──┤  │
   ┌───┤                │   │  │
   │   └────────────────┘   │  │
   │   ┌────────────────┐   │  │
   │   │                │◄──┘  │
   ├──►│ tswebextension │◄─────┤
   │   │                ├───┐  │
   │   └────────────────┘   │  │
   │   ┌────────────────┐   │  │
   └──►│  dnr-rulesets  │◄──┼──┘
       └────────────────┘   │
       ┌────────────────┐   │
       │      api       │◄──┤
       └────────────────┘   │
       ┌────────────────┐   │
       │    api-mv3     │◄──┘
       └────────────────┘
Examples packages

There are also some example packages which are needed for development and testing.

  • packages/examples/adguard-api depends on:

    • @adguard/api
    • @adguard/tswebextension as devDependency
  • packages/examples/adguard-api-mv3 depends on:

    • @adguard/api-mv3
    • @adguard/dnr-rulesets as devDependency
    • @adguard/tswebextension as devDependency
  • packages/examples/tswebextension-mv2 depends on:

    • @adguard/tswebextension
    • @adguard/tsurlfilter as devDependency
  • packages/examples/tswebextension-mv3 depends on:

    • @adguard/logger
    • @adguard/tswebextension
    • @adguard/tsurlfilter as devDependency