Java StringTokenizer Example

Here is full  code of our Java StringTokenizer Example. You can copy paste this code into your favourite IDE and run it straight-away. It doesn't require any third party library like Apache commons or Google Guava. All you need to do is create a Java source file with same name as public class of this example, then IDE will take care of compiling and running this example. Alternatively you can also compile and execute this example from command prompt as well. If you look at the first example, we have a String where words are separated by a white-space, and to get each word from that String, we have created a StringTokenizer object by passing that String itself, notice we have not provided any delimiter, because by default StringTokenizer uses white-space as token separator.


In order to get each token, in our case word, you just need to loop, until hasMoreTokens() return false. Now to get the word itself, just call nextToken() method of StringTokenizer. This is similar to Iterating over Java Collection using Iterator, where we use hasNext() method as while loop condition and next() method to get next element from Collection. Second example is more interesting, because here our text is a web address, which has protocol and IP address. Here we are passing multiple delimiter to split http string e.g. //(double slash), :(colon) and .(dot), Now StringTokenizer will create token if any of this is found in target String.  Third example shows you how to get total number of tokens from StringTokenizer, quite useful if you want to copy tokens into array or collection, as you can use this number to decide length of array or size of respective collection. 

import java.util.StringTokenizer;

/**

 * Java program to show how to use StringTokenizer for breaking a delimited

 * String into tokens. StringTokenizer allows you to use multiple delimiters as

 * well. which means you can split String containing comma and colon in one call.

 *

 * @author Javin Paul

 */

public class StringTokenizerDemo{

   

    public static void main(String args[]) {

        // Example 1 - By default StringTokenizer breaks String on space

        System.out.println("StringTokenizer Example in Java, split String on whitespace");

        String word = "Which one is better, StringTokenizer vs Split?";

        StringTokenizer tokenizer = new StringTokenizer(word);

        while (tokenizer.hasMoreTokens()) {

            System.out.println(tokenizer.nextToken());

        }

        // Example 2 - StringTokenizer with multiple delimiter

        System.out.println("StringTokenizer multiple delimiter Example in Java");

        String msg = "http://192.173.15.36:8084/";

        StringTokenizer st = new StringTokenizer(msg, "://.");

        while (st.hasMoreTokens()) {

            System.out.println(st.nextToken());

        }

       

       

        // Example 3 - Counting number of String tokens

        System.out.println("StringTokenizer count Token Example");

        String records = "one,two,three,four,five,six,seven";

        StringTokenizer breaker = new StringTokenizer(records, ",");

        System.out.println("Total number of tokens : " + breaker.countTokens());

    }

}

Output:

StringTokenizer Example in Java, split String on whitespace

Which

one

is

better,

StringTokenizer

vs

Split?

StringTokenizer multiple delimiter Example in Java

http

192

173

15

36

8084

StringTokenizer count Token Example

Total number of tokens : 7