Published: 14 Feb 2025 | Reading Time: 6 min read
Tokens in Java are the smallest units of a Java program that the compiler recognises as meaningful. These form the foundation of every Java program as these little elements are read by the Java compiler and understood to execute your code. If you want to write clear and error-free Java programs, you need to know what tokens are and how they work.
In this article, we will tell you the concept of tokens in java in simple parts. We'll cover the different types of tokens, their uses, and examples to help you understand them better.
In Java, tokens are the smallest individual units of a program that hold meaning to the compiler. Just like words and punctuation marks form sentences in a language, tokens define the structure and syntax of Java code. They are sequences of characters grouped together to create meaningful instructions that the Java compiler can analyse and execute.
int number = 10;
This line is made up of multiple tokens, these are:
Each of these tokens in java has a specific role and helps the compiler understand what the programmer is trying to perform. Without tokens, the program would be a mix of characters with no structure or meaning.
Java tokens are classified into five main types, each performing a specific purpose in a program's structure and functionality. Understanding these types is important for writing clear and error-free code. These types are:
Each token has a specific function that helps the compiler understand the structure and operations within the code. Here's a how we can explain java tokens:
Keywords in Java are reserved words with predefined meanings or more simply put, these are words defined in Java that represent a set of instructions.
Keywords help the compiler understand the operations and flow of the program. They define the syntax and indicate the type of behavior or action that needs to be performed.
public class Example {
public static void main(String[] args) {
if (true) {
System.out.println("This is a keyword example.");
}
}
}
In this code, public, class, static, void, and if are keywords. They define the program's structure and logic.
Identifiers in Java are the names used to identify variables, methods, classes, or objects. They are important for creating readable, maintainable, and functional programs, as they act as labels that the compiler and developers can use to reference specific elements within the code.
For example, in the declaration int age = 25; the word age is an identifier representing the variable storing the value 25.
Java implements specific rules to confirm identifiers are valid and work correctly:
Starting Character: Identifiers must start with a letter (A-Z or a-z), an underscore (_), or a dollar sign ($).
Valid: myVariable, _tempValue, $amount
Invalid: 1variable, @name
Allowed Characters: After the first character, identifiers can include letters, digits (0-9), underscores, and dollar signs. No spaces or special characters (like #, @, &) are allowed.
Boolean Literals: It should not be Boolean literal, that is, true or false.
Null Literal: It should not be null literal.
No Keywords: Identifiers cannot use reserved Java keywords, such as class, int, or return.
No Length Restrictions: Technically, there is no limit to the length of an identifier, but it is recommended to keep them brief and defining for readability.
public class IdentifierExample {
public static void main(String[] args) {
int age = 25; // 'age' is an identifier for a variable
String name = "John"; // 'name' is another identifier
System.out.println("Age: " + age);
System.out.println("Name: " + name);
}
}
The word 'age' is an identifier for a variable, and 'name' is another identifier in the above example.
Literals in Java are fixed values written directly in the code, representing constants that do not change during program execution. These values are used to assign specific data to variables or define constants in the program. For example, in the declaration int num = 100; the number 100 is a literal representing an integer value. The value may be a number, whether a whole number or decimal or it could also be a sequence of characters, which is called String literal, Boolean type, etc.
Literals are fundamental in programming as they act as the primary building blocks of data manipulation. Java provides several types of literals that provide different kinds of data requirements, such as numbers, text, and logical values.
public class LiteralExample {
public static void main(String[] args) {
int num = 100; // Integer literal
double pi = 3.14159; // Floating-point literal
char grade = 'A'; // Character literal
String greeting = "Hello"; // String literal
boolean isPassed = true; // Boolean literal
System.out.println("Number: " + num);
System.out.println("Pi: " + pi);
System.out.println("Grade: " + grade);
System.out.println("Greeting: " + greeting);
System.out.println("Passed: " + isPassed);
}
}
Number: 100
Pi: 3.14159
Grade: A
Greeting: Hello
Passed: true
The code demonstrates the use of different types of literals in Java. The output prints the values of an integer literal (100), a floating-point literal (3.14159), a character literal (A), a string literal (Hello), and a boolean literal (true). Each line corresponds to the respective literal and displays it as part of a message.
The time complexity of this code is O(1), as it only performs a constant number of operations: variable initialization and printing outputs.
There are five types of Literals on Java:
Integer Literals in Java refer to fixed values used in the code, representing whole numbers. They can be written in various number systems such as decimal, hexadecimal, octal, and binary.
Decimal Integer Literals are sequences of decimal digits (0-9) and are the most common way to represent integers. Examples include values like 6, 453, or 34789.
Hexadecimal Integer Literals use a sequence of hexadecimal digits (0-9, A-F) to represent values. These numbers are prefixed with 0x or 0X to indicate that they are in base 16. Examples include 0x56ab, 0X6AF2, etc.
Octal Integer Literals are sequences of digits from 0 to 7 and are prefixed with a 0 to denote base 8. Examples of octal literals include 07122, 04, and 043526.
Binary Integer Literals consist of binary digits (0 and 1) and are prefixed with 0b or 0B to indicate base 2. Examples include 0b0111001, 0b101, and 0b1000.
Floating-point literals represent numbers with decimal points or in exponential notation. These are used for specific values, such as measurements or calculations with fractions. Float literals are suffixed with f or F, while double literals use d or D. Examples include 3.14, -0.5, and 2.5e3 (scientific notation).
Character literals represent a single character in single quotes ('). They can include letters, digits, symbols, or escape sequences like '\n' (newline) and '\t' (tab). These literals are useful for handling individual characters in text processing. Examples include 'A', 'z', and '#'.
String literals represent sequences of characters in double quotes ("). They are used for handling text data, such as messages, user input, or file content. Examples include "Hello", "Java Programming", and "12345".
Boolean literals represent logical values, either true or false. They are used in conditional statements and logical operations to control program flow. For example, if (isPassed), the variable isPassed could be assigned a Boolean literal, such as true. Examples are true and false.
There is only one value of Null Literal, that is, null.
Operators in Java are symbols used to perform operations on variables, values, or expressions. They allow developers to manipulate data, make comparisons, and assign values to variables.
Java offers many types of operators which classified based on the type of operation they perform.
Operators are important in almost every program as they enable the manipulation of data, control the flow of execution, and help in assessing conditions.
public class OperatorExample {
public static void main(String[] args) {
int a = 10;
int b = 20;
int sum = a + b; // Addition operator
boolean isGreater = a > b; // Comparison operator
boolean isEqual = a == b; // Comparison operator
System.out.println("Sum: " + sum); // 30
System.out.println("Is a greater than b? " + isGreater); // false
System.out.println("Is a equal to b? " + isEqual); // false
}
}
Sum: 30
Is a greater than b? false
Is a equal to b? false
There are four types of operators in java:
Arithmetic operators are used to perform basic mathematical operations like addition, subtraction, multiplication, division, and modulus (remainder).
| Operator | Description | Example |
|---|---|---|
| + | Addition: Adds two operands. | a + b |
| - | Subtraction: Subtracts the second operand from the first | a - b |
| * | Multiplication: Multiplies two operands. | a * b |
| / | Division: Divides the first operand by the second. | a / b |
| % | Modulus: Returns the remainder when the first operand is divided by the second. | a % b |
Example:
int a = 10;
int b = 20;
int sum = a + b; // sum will be 30
int remainder = b % a; // remainder will be 0
Comparison operators are used to compare two values and return a boolean result (true or false). These operators help assess conditions and make decisions in a program.
| Operator | Description | Example |
|---|---|---|
| == | Equal to: Checks if two operands are equal. | a == b |
| != | Not equal to: Checks if two operands are not equal. | a != b |
| < | Less than: Checks if the left operand is less than the right operand. | a < b |
| > | Greater than: Checks if the left operand is greater than the right operand. | a > b |
| <= | Less than or equal to: Checks if the left operand is less than or equal to the right operand. | a <= b |
| >= | Greater than or equal to: Checks if the left operand is greater than or equal to the right operand. | a >= b |
Example:
int a = 10;
int b = 20;
boolean isEqual = a == b; // false
boolean isGreater = a > b; // false
boolean isLessThanOrEqual = a <= b; // true
Logical operators are used to combine multiple boolean expressions and return a single boolean value. They help in making complex decisions and controlling program flow.
Example:
boolean isAdult = true;
boolean hasLicense = false;
boolean canDrive = isAdult && hasLicense; // false
boolean isNotAdult = !isAdult; // false
Assignment operators are used to assign values to variables. In addition to simple assignment, Java provides compound assignment operators that combine assignment with an arithmetic operation.
| Operator | Description | Example |
|---|---|---|
| = | Simple assignment operator: Assigns the right operand to the left operand. | a = b |
| += | Adds the right operand to the left operand and assigns the result to the left operand. | a += b |
| -= | Subtracts the right operand from the left operand and assigns the result to the left operand. | a -= b |
| *= | Multiplies the left operand by the right operand and assigns the result to the left operand. | a *= b |
| /= | Divides the left operand by the right operand and assigns the result to the left operand. | a /= b |
| %= | Computes the remainder of the left operand divided by the right operand and assigns the result to the left operand. | a %= b |
Example:
int a = 10;
a += 5; // a becomes 15 (a = a + 5)
a -= 3; // a becomes 12 (a = a - 3)
a *= 2; // a becomes 24 (a = a * 2)
a /= 4; // a becomes 6 (a = a / 4)
a %= 3; // a becomes 0 (a = a % 3)
| Operator | Description | Example |
|---|---|---|
| ++ | Increment by one | a++ |
| -- | Decrement by one | a-- |
| Operator | Description | Example |
|---|---|---|
| & | Bitwise AND | a & b (bits are set to 1 only if both bits are 1) |
| ~ | Bitwise complement (NOT) | ~a (inverts the bits of a) |
| ^ | Bitwise exclusive OR (XOR) | a ^ b (bits are set to 1 if the bits are different) |
| >> | Shift Right | a >> 2 (shifts bits of a to the right, filling with 0) |
| << | Shift Left | a << 2 (shifts bits of a to the left by 2 positions) |
| >>> | Shift right with zero fill | a >>> 2 (shifts bits of a to the right, filling with 0) |
The ? operator is used to construct a conditional expression, also known as the ternary operator, that evaluates a condition and returns one of two values based on the result.
Separators in Java are symbols that help define the structure of your code and organize different parts of a program. These symbols denote boundaries, group elements together, and separate different components within the program. Without separators, the program would lack clarity and structure, making it hard for both the compiler and developers to understand the flow of the code. Java provides several types of separators that each serve a specific purpose, helping break down code into manageable parts, control execution flow, and manage collections of data.
public class SeparatorExample {
public static void main(String[] args) {
int[] numbers = {1, 2, 3}; // {} for array initialization and [] for array declaration
System.out.println(numbers[0]); // [] for array access
}
}
| Separator | Description |
|---|---|
| () (Parentheses) | Used to group expressions or define method arguments, controlling the order of operations. |
| {} (Curly Braces) | Used to define code blocks like the body of methods, loops, or conditional statements. |
| ; (Semicolon) | Ends individual statements, marking where one instruction ends and another begins. |
| , (Comma) | Separates items in a list, such as arguments in a method or elements in an array. |
| . (Dot) | Used to access members of a class or object, such as methods and fields, and for referencing classes in imports. |
In Java, comments are used to provide explanatory notes or descriptions within the code. These comments are ignored by the compiler and do not affect the execution of the program.
There are two types of comments in Java:
Single line comments are used for brief explanations, typically for one line of code.
Example: // This is a single-line comment
Multi-line comments are used for longer explanations spanning multiple lines.
The process of compiling Java code involves breaking down the source code into these tokens, analyzing them, and confirming that they form a valid and executable program. Here's how Java tokens work:
Java tokenization is the first step in the compilation process. During this phase, the compiler reads the source code character by character and breaks it into individual tokens. Tokenize java string can represent various elements such as keywords, identifiers, literals, operators, and separators.
int x = 10;
The compiler identifies int, x, =, 10, and ; as separate tokens.
Once the code is tokenized, the next step is syntax analysis. In this phase, the compiler checks if the sequence of tokens follows the rules of Java syntax. Syntax rules define how tokens can be combined to form valid statements or expressions. If the tokens are arranged incorrectly, the compiler will generate a syntax error.
After syntax analysis, the compiler performs semantic analysis. This step confirms that the tokens make logical sense in the context of the program. The compiler checks that the program's structure is valid in terms of variable declarations, data types, and operations.
int x = "hello";
Here, the semantic analysis will detect an error because:
Here's a program to see how different types of tokens come together:
public class TokenDemo {
public static void main(String[] args) {
// Declaring variables
int num1 = 15;
int num2 = 20;
// Using an operator
int sum = num1 + num2;
// Printing the result
System.out.println("The sum is: " + sum);
}
}
A JSON Web Token (JWT) is a compact, self-contained way to securely transmit information between two parties, such as a client and a server. It is a string composed of three parts—header, payload, and signature—separated by dots.
The header contains metadata about the token, like the algorithm used for signing, while the payload holds the actual information (claims) being transmitted. The signature ensures that the token hasn't been altered, providing authenticity and integrity.
A Bearer Token is a type of token used in authentication and authorization mechanisms. It is a part of the HTTP header used in API requests, commonly for passing authentication information like JWT (JSON Web Tokens). The bearer token is included in the Authorization header of HTTP requests to authenticate users or applications.
If you're working with JWT in Java, you'll need libraries to generate and verify the JWTs. The commonly used library is jjwt. Add the dependency in your pom.xml file.
<dependency>
<groupId>io.jsonwebtoken</groupId>
<artifactId>jjwt</artifactId>
<version>0.11.5</version>
</dependency>
To generate a bearer token, you'll need to create a JWT. Here's how you can create java token generator, a simple JWT token using RS256 (RSA encryption) in Java.
import io.jsonwebtoken.Jwts;
import io.jsonwebtoken.SignatureAlgorithm;
import java.util.Date;
public class JWTGenerator {
private static final String SECRET_KEY = "your-256-bit-secret"; // Ideally, store this securely
public static String generateToken(String username) {
return Jwts.builder()
.setSubject(username)
.setIssuedAt(new Date())
.setExpiration(new Date(System.currentTimeMillis() + 3600000)) // 1 hour expiration
.signWith(SignatureAlgorithm.HS256, SECRET_KEY)
.compact();
}
public static void main(String[] args) {
String token = generateToken("user123");
System.out.println("Bearer Token: " + token);
}
}
HS256 is used for signing the JWT. You can replace it with RS256 if you're using RSA encryption.
Expiration: The token expires in 1 hour.
Once you have the token, it's time to send it in an HTTP request header. Here is a sample of bearer token java code using HttpURLConnection to send a GET request with the bearer token.
import java.net.HttpURLConnection;
import java.net.URL;
public class BearerTokenRequest {
public static void sendRequest(String token) throws Exception {
// Create URL and open connection
URL url = new URL("https://your-api.com/protected-endpoint");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
// Add Authorization header with Bearer token
connection.setRequestProperty("Authorization", "Bearer " + token);
// Get response
int responseCode = connection.getResponseCode();
System.out.println("Response Code: " + responseCode);
}
public static void main(String[] args) throws Exception {
String token = "your-jwt-token"; // Use the generated token
sendRequest(token);
}
}
On the server side, you'll need to verify the token. Here's an authorization bearer token java example of how to verify a JWT token using jjwt java:
import io.jsonwebtoken.Jwts;
import io.jsonwebtoken.Claims;
import io.jsonwebtoken.SignatureException;
public class JWTVerifier {
private static final String SECRET_KEY = "your-256-bit-secret";
public static Claims verifyToken(String token) throws SignatureException {
return Jwts.parser()
.setSigningKey(SECRET_KEY)
.parseClaimsJws(token)
.getBody();
}
public static void main(String[] args) {
String token = "your-jwt-token";
try {
Claims claims = verifyToken(token);
System.out.println("Token is valid. User: " + claims.getSubject());
} catch (SignatureException e) {
System.out.println("Invalid token.");
}
}
}
This code checks the signature of the token using the secret key. If the token is valid, the server can extract claims (such as the subject or username) and proceed.
Java tokens form the foundational building blocks of any Java program which enable both the compiler and the developer to understand the structure and functionality of the code. These tokens, which include keywords, identifiers, literals, operators, and separators, help organize and define the instructions that a program executes. Keywords set the structure and flow, identifiers give meaning to variables and functions, literals provide constant values, operators perform necessary operations, and separators break down the code into comprehensible units.
A type token in Java refers to the classification of a token that represents a specific data type, such as an integer, floating-point number, or boolean. It helps the compiler understand what kind of value a variable can hold.
In Java, punctuators and separators are symbols that help structure and organize the code. Punctuators include operators and symbols like +, -, *, =, while separators are symbols that help define boundaries or groupings, such as parentheses (), curly braces {}, semicolons ;, and commas ,.
In a Java string, a token refers to a sequence of characters that can be parsed as a meaningful unit. These tokens can be words, symbols, or other structures that the Java compiler identifies within the string for processing, such as keywords, identifiers, and literals.
Tokens are important because they help the Java compiler break down the source code into manageable parts for analysis and execution. Understanding tokens is important for writing syntactically correct Java code and for debugging purposes.
The Java compiler identifies tokens through a process called tokenization or lexical analysis. During this process, the compiler scans the source code, breaks it into individual tokens, assigns meanings to each token, and prepares them for further analysis and compilation.
Yes, tokens in Java are case-sensitive. This means that uppercase and lowercase letters are treated as distinct characters. For example, Variable and variable would be considered two different identifiers.
In Java, you cannot create custom tokens as they are predefined by the language specification. However, you can define your own identifiers (such as variable names and method names) within the constraints of the language rules.
Prime Number Program in Java: Explained with Examples - Learn how to write a prime number program in Java with clear logic, optimized algorithms, examples, and interview-ready explanations. (04 Jan 2026, 8 min read)
Why Encapsulation in Java Matters: Learn with Code Snippets - Understand encapsulation in Java, a key object-oriented principle used to restrict direct access and protect internal data from unauthorized changes. (04 Jan 2026, 5 min read)
Master Binary Search in Java: Fast and Efficient Searching Explained - Learn Binary Search in Java with clear logic, easy code examples, and real-world applications. Boost your coding skills with this step-by-step guide! (03 Jan 2026, 5 min read)
Learn Bubble Sort in Java: Easy Sorting Technique Explained - Get a complete guide on bubble sort in Java, including coding examples and tips to understand this basic but important sorting algorithm. (02 Jan 2026, 6 min read)
Best Java Training Institutes in Hyderabad: Your Guide to Career Success - Find the best Java training institutes in Hyderabad for hands-on learning, placement support, and industry-recognized certifications. (02 Jan 2026, 5 min read)
Understanding Inheritance in Java: Key Concepts & Benefits Explained - Learn all about inheritance in Java with clear examples. Understand types, benefits & how it supports code reusability in object-oriented programming. (02 Jan 2026, 8 min read)