token_get_all

Description

token_get_all() parses the given source string into PHP language tokens using the Zend engines lexical scanner. The function returns an array of token descriptions. Each array element itself is either a one character string or itself an array containing a token id and the string representation of that token in the source code.

<?php
  $tokens = token_get_all(";"); // => array(";");
  $tokens = token_get_all("foreach") // => array(T_FOREACH => "foreach");
  $tokens = token_get_all("/* comment */") // => array(T_ML_COMMENT => "/* comment */");
?>