tokstools – Tools for manipulating tokens

This package is still in the experimental stage. Therefore, its behavior is not guaranteed to match the description in the manual, and bugs of varying severity may occur.

Additionally, some features, as well as their syntax, may be subject to change.

It requires any modern engine that supports etex primitives and can therefore be used with either TeX or LaTeX.

The tokstools package provides tools for manipulating tokens and implements “PEG”-style grammars in TeX. Predefined elementary patterns can be combined to form grammars that search for matches within tokens.

Two categories of macros are provided:

• those operating on individual tokens (\toksdo, \tokscount, and \printtoks);
• those operating on tokens that have matched a grammar (\ifpegmatch, \pegcount, and \pegreplace)

Sources	/macros/generic/tokstools
Documentation	README.md Package documentation Package documentation
Version	0.2
Licenses	The LaTeX Project Public License 1.3c
Copyright	2026 Christian Tellechea
Maintainer	Christian Tellechea
Contained in	TeX Live as tokstools MiKTeX as tokstools
Topics	Parser

Download the contents of this package in one zip archive (481.7k).