Token Struct Reference

A single token returned by the tokenizer. More...

#include <tokenize.h>

Data Fields
uint32_t	column
	1-based column number
size_t	length
	Length in bytes.
uint32_t	line
	1-based line number
const char *	start
	Pointer into source buffer.
int	type
	Token type code (keyword code or generic)

Detailed Description

A single token returned by the tokenizer.

Definition at line 34 of file tokenize.h.

Field Documentation

column

uint32_t Token::column

1-based column number

Definition at line 39 of file tokenize.h.

length

size_t Token::length

Length in bytes.

Definition at line 37 of file tokenize.h.

line

uint32_t Token::line

1-based line number

Definition at line 38 of file tokenize.h.

start

const char* Token::start

Pointer into source buffer.

Definition at line 36 of file tokenize.h.

type

int Token::type

Token type code (keyword code or generic)

Definition at line 35 of file tokenize.h.

---

The documentation for this struct was generated from the following file:

• include/tokenize.h