porytiles::Token Class Reference

Represents a lexical token from C source code. More...

#include <token.hpp>

Public Member Functions
	Token (TokenType type, std::string text, SourcePosition position)
	Constructs a token with the given type, text, and position.
	Token (std::string text, std::int64_t int_value, SourcePosition position)
	Constructs an integer literal token with a parsed value.
TokenType	type () const
	Returns the token type.
const std::string &	text () const
	Returns the raw text of the token.
const SourcePosition &	position () const
	Returns the source position where the token starts.
std::int64_t	int_value () const
	Returns the integer value for integer_literal tokens.
bool	is (TokenType type) const
	Checks if this token is of the specified type.
template<typename... Types>
bool	is_any_of (Types... types) const
	Checks if this token is any of the specified types.

Detailed Description

Represents a lexical token from C source code.

Token captures the type, textual representation, parsed value (if applicable), and source position of a lexical unit. For integer literals, the parsed numeric value is stored in int_value_. For identifiers and string literals, the text representation is used.

Invariant

type_ is always set to a valid TokenType

position_ contains valid 1-based line and column numbers

Definition at line 106 of file token.hpp.

Constructor & Destructor Documentation

Token() [1/2]

porytiles::Token::Token ( TokenType  type, std::string  text, SourcePosition  position  )

inline

Constructs a token with the given type, text, and position.

Parameters

type	The token type
text	The raw text of the token
position	The source position where the token starts

Definition at line 115 of file token.hpp.

Token() [2/2]

porytiles::Token::Token ( std::string  text, std::int64_t  int_value, SourcePosition  position  )

inline

Constructs an integer literal token with a parsed value.

Parameters

text	The raw text of the token
int_value	The parsed integer value
position	The source position where the token starts

Definition at line 127 of file token.hpp.

Member Function Documentation

int_value()

std::int64_t porytiles::Token::int_value ( ) const

inline

Returns the integer value for integer_literal tokens.

Precondition

type() == TokenType::integer_literal

Returns

The parsed integer value

Definition at line 168 of file token.hpp.

is()

bool porytiles::Token::is ( TokenType  type ) const

inline

Checks if this token is of the specified type.

Parameters

type	The token type to check against

Returns

True if this token's type matches

Definition at line 180 of file token.hpp.

is_any_of()

template<typename... Types>

bool porytiles::Token::is_any_of ( Types...  types ) const

inline

Checks if this token is any of the specified types.

Template Parameters

Types

Variadic token types

Parameters

types

The token types to check against

Returns

True if this token's type matches any of the given types

Definition at line 193 of file token.hpp.

position()

const SourcePosition & porytiles::Token::position ( ) const

inline

Returns the source position where the token starts.

Returns

A const reference to the source position

Definition at line 157 of file token.hpp.

text()

const std::string & porytiles::Token::text ( ) const

inline

Returns the raw text of the token.

Returns

A const reference to the token text

Definition at line 147 of file token.hpp.

type()

TokenType porytiles::Token::type ( ) const

inline

Returns the token type.

Returns

The token type

Definition at line 137 of file token.hpp.

---

The documentation for this class was generated from the following file:

• porytiles/include/porytiles/utilities/c_parser/token.hpp