porytiles::Parser Class Reference

Parser for C preprocessor constructs. More...

#include <parser.hpp>

Public Member Functions
	Parser (gsl::not_null< const TextFormatter * > format, std::vector< Token > tokens)
	Constructs a parser for the given token stream.
	Parser (gsl::not_null< const TextFormatter * > format, std::vector< Token > tokens, const CParserContext *context)
	Constructs a parser with a context for rich error formatting.
ChainableResult< std::vector< DefineStatement > >	parse_defines ()
	Parses all #define statements from the token stream.
ChainableResult< std::vector< EnumDeclaration > >	parse_enums ()
	Parses all enum declarations from the token stream.
ChainableResult< std::vector< ArrayDeclaration > >	parse_pointer_arrays ()
	Parses all pointer array declarations from the token stream.
ChainableResult< std::vector< FunctionDefinition > >	parse_functions ()
	Parses function definitions from the token stream.
ChainableResult< std::vector< StructVariableDeclaration > >	parse_struct_variables ()
	Parses struct variable declarations from the token stream.
ChainableResult< std::vector< StructInitializerDeclaration > >	parse_struct_initializers ()
	Parses struct variable declarations with their designated initializer fields.
ChainableResult< std::vector< IncbinDeclaration > >	parse_incbin_arrays ()
	Parses INCBIN array declarations from the token stream.

Detailed Description

Parser for C preprocessor constructs.

Parser analyzes a token stream and extracts structured information. The initial implementation focuses on parsing #define statements, extracting macro names and evaluating constant expressions.

The parser uses the Shunting Yard algorithm to evaluate arithmetic expressions in #define values, supporting operators: +, -, *, /, %, &, |, ^, ~, <<, >>

Example usage:

Lexer lexer{source_content};
auto tokens_result = lexer.lex();
if (tokens_result.has_value()) {
    Parser parser{std::move(tokens_result).value()};
    auto defines_result = parser.parse_defines();
    if (defines_result.has_value()) {
        for (const auto& def : defines_result.value()) {
            // process defines
        }
    }
}

Future extensions will add methods like:

• parse_functions() for function declarations/definitions
• parse_structs() for struct definitions

Definition at line 54 of file parser.hpp.

Constructor & Destructor Documentation

Parser() [1/2]

porytiles::Parser::Parser ( gsl::not_null< const TextFormatter * >  format, std::vector< Token >  tokens  )

inline

Constructs a parser for the given token stream.

This constructor creates a parser without a CParserContext. Errors will be formatted as simple "line:col: message" strings without source context highlighting.

Parameters

format	The text formatter for styled output
tokens	The tokens to parse (typically from Lexer::lex())

Definition at line 66 of file parser.hpp.

Parser() [2/2]

porytiles::Parser::Parser ( gsl::not_null< const TextFormatter * >  format, std::vector< Token >  tokens, const CParserContext *  context  )

inline

Constructs a parser with a context for rich error formatting.

This constructor creates a parser with a CParserContext that enables rich error formatting with source context highlighting via FileHighlightPrinter. The context must outlive the parser.

Parameters

format	The text formatter for styled output
tokens	The tokens to parse (typically from Lexer::lex())
context	The parser context for rich error formatting (non-owning)

Definition at line 82 of file parser.hpp.

Member Function Documentation

parse_defines()

ChainableResult< std::vector< DefineStatement > > porytiles::Parser::parse_defines

(

)

Parses all #define statements from the token stream.

Scans through the token stream looking for #define directives. For each one found, parses the macro name and evaluates the value expression if present. Non-define preprocessor directives and other code constructs are skipped.

Supports:

• Simple integer defines: #define FOO 123
• Hex/octal/binary literals: #define BAR 0xFF
• Arithmetic expressions: #define BAZ (1 << 4)
• String defines: #define MSG "hello"
• Flag defines: #define DEBUG
• References to previously defined macros: #define B A (where A was defined earlier)

When constructed with a CParserContext, errors include source code context with highlighted error locations. When constructed without a context, errors are simple "line:col: message" format.

Returns

A vector of DefineStatement on success, or an error on failure

Definition at line 135 of file parser.cpp.

parse_enums()

ChainableResult< std::vector< EnumDeclaration > > porytiles::Parser::parse_enums

(

)

Parses all enum declarations from the token stream.

Scans through the token stream looking for enum declarations. For each one found, parses the optional enum name and all members with their values. Supports both implicit counter-based values and explicit value assignments.

Supports:

• Anonymous enums: enum { ... }
• Named enums: enum Name { ... }
• Simple members: FOO,
• Explicit values: FOO = 10,
• Expression values: FOO = (1 << 4),
• References to previously defined macros: FOO = SOME_DEFINE,

When constructed with a CParserContext, errors include source code context with highlighted error locations. When constructed without a context, errors are simple "line:col: message" format.

Returns

A vector of EnumDeclaration on success, or an error on failure

Definition at line 166 of file parser.cpp.

parse_functions()

ChainableResult< std::vector< FunctionDefinition > > porytiles::Parser::parse_functions

(

)

Parses function definitions from the token stream.

Scans through the token stream looking for function definitions matching the pattern:

[static] TYPE IDENTIFIER ( params ) { body }

This is used by AnimCodeParser to extract queue and driver functions like:

static void QueueAnimTiles_General_Flower(u16 timer) {
    AppendTilesetAnimToBuffer(..., TILE_OFFSET_4BPP(12), 4 * TILE_SIZE_4BPP);
}

The parser captures the function name and all tokens within the body braces for later pattern matching.

Returns

A vector of FunctionDefinition on success, or an error on failure

Definition at line 863 of file parser.cpp.

parse_incbin_arrays()

ChainableResult< std::vector< IncbinDeclaration > > porytiles::Parser::parse_incbin_arrays

(

)

Parses INCBIN array declarations from the token stream.

Scans through the token stream looking for array declarations using INCBIN macros:

// Single path arrays:
[const] TYPE IDENTIFIER [] = INCBIN_MACRO("path");
 
// Multi-path arrays (palettes):
[const] TYPE IDENTIFIER [][SIZE] = { INCBIN_MACRO("path1"), INCBIN_MACRO("path2"), ... };

Examples from pokeemerald:

const u32 gTilesetTiles_General[] = INCBIN_U32("data/tilesets/primary/general/tiles.4bpp");
 
const u16 gTilesetPalettes_General[][16] = {
    INCBIN_U16("data/tilesets/primary/general/palettes/00.gbapal"),
    INCBIN_U16("data/tilesets/primary/general/palettes/01.gbapal"),
    ...
};

The parser extracts the variable name, INCBIN macro name, and path(s) from each declaration.

Returns

A vector of IncbinDeclaration on success, or an error on failure

Definition at line 1171 of file parser.cpp.

parse_pointer_arrays()

ChainableResult< std::vector< ArrayDeclaration > > porytiles::Parser::parse_pointer_arrays

(

)

Parses all pointer array declarations from the token stream.

Scans through the token stream looking for pointer array declarations matching the pattern:

[const] TYPE * [const] IDENTIFIER [] = { element1, element2, ... };

This is used by AnimCodeParser to extract animation frame arrays like:

const u16 *const gTilesetAnims_General_Flower[] = {
    gTilesetAnims_General_Flower_Frame0,
    gTilesetAnims_General_Flower_Frame1
};

The parser extracts the array name and all identifier elements from the initializer list.

Returns

A vector of ArrayDeclaration on success, or an error on failure

Definition at line 753 of file parser.cpp.

parse_struct_initializers()

ChainableResult< std::vector< StructInitializerDeclaration > > porytiles::Parser::parse_struct_initializers

(

)

Parses struct variable declarations with their designated initializer fields.

Scans through the token stream looking for struct variable declarations matching the pattern:

[const] struct TYPE IDENTIFIER = { .field1 = value1, .field2 = value2, ... } [;]

This is used by ProjectTilesetMetadataProvider to extract tileset metadata from headers.h files:

const struct Tileset gTileset_General = {
    .isCompressed = TRUE,
    .isSecondary = FALSE,
    .tiles = gTilesetTiles_General,
    .palettes = gTilesetPalettes_General,
    .metatiles = gMetatiles_General,
    .metatileAttributes = gMetatileAttributes_General,
    .callback = InitTilesetAnim_General,
};

Unlike parse_struct_variables(), this method also parses the designated initializer fields, enabling extraction of field values like isSecondary and variable references.

Returns

A vector of StructInitializerDeclaration on success, or an error on failure

Definition at line 1019 of file parser.cpp.

parse_struct_variables()

ChainableResult< std::vector< StructVariableDeclaration > > porytiles::Parser::parse_struct_variables

(

)

Parses struct variable declarations from the token stream.

Scans through the token stream looking for struct variable declarations matching the pattern:

[const] struct TYPE IDENTIFIER = { ... } [;]

This is used by ProjectTilesetMetadataProvider to extract tileset names from headers.h files like:

const struct Tileset gTileset_General = {
    .isCompressed = TRUE,
    .isSecondary = FALSE,
    ...
};

The parser captures the struct type and variable name; the initializer body is skipped.

Returns

A vector of StructVariableDeclaration on success, or an error on failure

Definition at line 939 of file parser.cpp.

---

The documentation for this class was generated from the following files:

• porytiles/include/porytiles/utilities/c_parser/parser.hpp
• porytiles/lib/utilities/c_parser/parser.cpp