# Tokenizer

Parses a query string into its individual tokens

****

- Since [5.1.0](https://github.com/getkirby/kirby/releases/tag/5.1.0)
- Full class name: `Kirby\Query\Parser\Tokenizer`
- Source: <https://github.com/getkirby/kirby/tree/5.5.3/src/Query/Parser/Tokenizer.php#L19>

<unstable-notice>
`Tokenizer` has been **marked as unstable**. It might be changed in a future Kirby major or minor release without being considered a breaking change. Use with caution.
</unstable-notice>

****

### Tokenizer

- [new Tokenizer()](https://getkirby.com/docs/reference/objects/query-parser/tokenizer/__construct.md)
- [Tokenizer::match()](https://getkirby.com/docs/reference/objects/query-parser/tokenizer/match.md)
- [Tokenizer::token()](https://getkirby.com/docs/reference/objects/query-parser/tokenizer/token.md)
- [$tokenizer->tokens()](https://getkirby.com/docs/reference/objects/query-parser/tokenizer/tokens.md)