PHP Tokenization
Subject: Web development using PHP and MySQL
Tokenization is the process of breaking up a long string into words. PHP offers a special funtion for this purpose, called strtok()
strtok
strtok() function takes two arguments, the string or text to be broken up into tokens AND string containing all the delimeters.
- in strtok() delimiters are characters that count as boundary between tokens e.g blank space " " , comma"," , semi-colon";" , hyphen "-", a, i, s, 2, 7 etc.
Sytax
strtok("text to be broken up","delimeter-character")
strtok() example
<![CDATA[
<?php
//space " " in-between each word is use as a delimiter to break the text
$stoken = strtok("Tokenization is the process of breaking up a long string"," ");
while($stoken){
print($stoken."<br/>");
//"this help us to exit the loop"
$stoken = strtok(" ");
}
?>
]]>
Output:
Tokenization
is
the
process
of
breaking
up
a
long
string
Explode
explode() function takes two arguments, string containing all the delimeters AND a string or text to be broken up into tokens and the string to be separated.
- in explode() delimiters are full-fledged string e.g AND, WHERE, HOW, A, WHEN etc.
- in explode() the substring or broken text are returned in an array.
Sytax
explode("delimeter-string", "text to be broken up")
explode() example
<![CDATA[
<?php
$expl_token = explode("big","some big boys and big girls are not rich");
$array_len = count($expl_token);
for($i=0; $i<$array_len; $i++){
print $expl_token[$i]."<br/>";
}
?>
]]>
Output:
some
boys and
girls are not rich
Since the result are kept in array, you can equally access individual element as well, instead of accessing all the array element using loop.
<![CDATA[
<?php
print $expl_token[1];
?>
]]>
Output:
boys and
By:
Benjamin Onuorah
Login to comment or ask question on this topic
Previous Topic Next Topic
