multi_grammar.h File Reference

Runtime disambiguation for overlapping loaded grammars (Tiers 1-3). More...

#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#include "parse_context.h"

Go to the source code of this file.

Data Structures
struct	LimeForkCandidate
	One candidate grammar for a collision. More...
struct	LimeForkRank
	Per-candidate ranking detail (optional output of lime_mg_resolve). More...

Typedefs
typedef struct LimeBayesStore	LimeBayesStore
	Opaque Beta-Bernoulli store keyed by (state, token, ext_id).
typedef struct LimeDialectRegistry	LimeDialectRegistry
	Opaque name -> composed-snapshot registry.

Functions
LimeBayesStore *	lime_bayes_create (void)
	Create an empty store (entries allocated lazily).
LimeBayesStore *	lime_bayes_deserialize (const void *buf, size_t buflen)
	Rebuild a store from a blob produced by lime_bayes_serialize.
void	lime_bayes_destroy (LimeBayesStore *s)
	Destroy a store.
void	lime_bayes_observe (LimeBayesStore *s, uint16_t state, uint16_t token, uint32_t ext_id, bool success)
	Record that ext_id won at (state, token) and the parse success-ceeded – updates that arm's Beta posterior.
int	lime_bayes_rank (LimeBayesStore *s, uint16_t state, uint16_t token, const uint32_t *ext_ids, uint32_t n, float *out_confidence)
	Rank ext_ids at (state, token) by posterior mean.
size_t	lime_bayes_serialize (const LimeBayesStore *s, void *buf, size_t buflen)
	Serialize the store to a flat blob for host-side persistence.
bool	lime_dialect_parse_sigil (const LimeDialectRegistry *reg, const char *input, ParserSnapshot **out_snap, size_t *out_offset)
	Parse a leading @dialect sigil at the start of input.
bool	lime_dialect_register (LimeDialectRegistry *reg, const char *name, ParserSnapshot *snap)
	Register a composed snapshot under a dialect name.
LimeDialectRegistry *	lime_dialect_registry_create (void)
	Create an empty dialect registry.
void	lime_dialect_registry_destroy (LimeDialectRegistry *reg)
	Destroy the registry.
ParserSnapshot *	lime_dialect_select (const LimeDialectRegistry *reg, const char *name)
	Look up the composed snapshot for a dialect name.
int	lime_mg_resolve (const LimeForkCandidate *cands, uint32_t ncands, const int *lookahead, uint32_t nlook, struct LimeBayesStore *bayes, uint16_t bayes_state, uint16_t bayes_token, LimeForkRank *out_ranks)
	Resolve a collision by simulating each candidate over the upcoming token stream and ranking the outcomes (Tier 1).

Detailed Description

Runtime disambiguation for overlapping loaded grammars (Tiers 1-3).

Three layered mechanisms for resolving collisions when several grammar dialects are loaded into one composed parser. Each tier is explicit about its guarantee level – this is deliberately NOT a promise that any union of dialects "always works", which LR theory forbids when two dialects share a production with different meaning.

Tier 1 – full-statement fork-resolve (lime_mg_resolve): On a collision admissible in >=2 candidate (snapshot, token) pairs, simulate each candidate over the real upcoming token stream (lime_simulate_parse) and prefer the one that reaches accept / gets furthest with fewest errors. CORRECT when the candidates diverge within the statement (the common case for "90% overlap" dialects: LIMIT vs ROWNUM, hint syntax, etc.).

Tier 2 – Bayesian tie-break (LimeBayesStore): When Tier 1 ties (candidates trial identically – e.g. truly identical productions), pick the historically-likelier dialect for this (state, token) and learn from confirmed parses. HEURISTIC: it does not make an ambiguous parse correct, it makes it resolve consistently and improve with feedback.

Tier 3 – dialect mode selection (LimeDialectRegistry): For genuinely mutually-ambiguous full dialects that no parse-time method can separate, select one dialect per session or per statement (a named composed snapshot, or a leading @dialect sigil). EXACT, but this is mode selection, not disambiguation.

See also

docs/MULTI_GRAMMAR.md

parse_context.h for lime_simulate_parse and the admissibility oracle.

Definition in file multi_grammar.h.

Typedef Documentation

LimeBayesStore

typedef struct LimeBayesStore LimeBayesStore

Opaque Beta-Bernoulli store keyed by (state, token, ext_id).

Definition at line 112 of file multi_grammar.h.

LimeDialectRegistry

typedef struct LimeDialectRegistry LimeDialectRegistry

Opaque name -> composed-snapshot registry.

Definition at line 156 of file multi_grammar.h.

Function Documentation

lime_bayes_deserialize()

LimeBayesStore * lime_bayes_deserialize	(	const void *	buf,
		size_t	buflen
	)

Rebuild a store from a blob produced by lime_bayes_serialize.

Returns

New store, or NULL on bad/corrupt input.

lime_bayes_rank()

int lime_bayes_rank	(	LimeBayesStore *	s,
		uint16_t	state,
		uint16_t	token,
		const uint32_t *	ext_ids,
		uint32_t	n,
		float *	out_confidence
	)

Rank ext_ids at (state, token) by posterior mean.

Returns

Index into ext_ids of the highest-posterior arm, or -1 on bad args. out_confidence (optional) receives the winning posterior mean (0.5 with no evidence).

lime_bayes_serialize()

size_t lime_bayes_serialize	(	const LimeBayesStore *	s,
		void *	buf,
		size_t	buflen
	)

Serialize the store to a flat blob for host-side persistence.

Returns

Bytes written; pass buf == NULL to query the size needed. Returns 0 on failure (buffer too small).

lime_dialect_parse_sigil()

bool lime_dialect_parse_sigil	(	const LimeDialectRegistry *	reg,
		const char *	input,
		ParserSnapshot **	out_snap,
		size_t *	out_offset
	)

Parse a leading @dialect sigil at the start of input.

Recognizes an optional @name prefix (ASCII letters/digits/_ after '@'), looks the name up in reg, and reports the snapshot plus the byte offset where the real statement begins (past the sigil and any following whitespace). The host owns the actual scanning of the rest of the input; this is just the per-statement dialect selector.

Parameters

	reg	Dialect registry.
	input	Statement text (NUL-terminated).
[out]	out_snap	Selected snapshot, or NULL if no recognized sigil (caller falls back to a default dialect).
[out]	out_offset	Byte offset of the statement body in input.

Returns

true if a recognized @dialect sigil was consumed; false if there was no sigil (out_offset = 0) or the name was unknown.

lime_dialect_register()

bool lime_dialect_register	(	LimeDialectRegistry *	reg,
		const char *	name,
		ParserSnapshot *	snap
	)

Register a composed snapshot under a dialect name.

Acquires a reference to snap (released on destroy / re-register). Re-registering a name replaces the previous snapshot.

Returns

true on success, false on bad args / allocation failure.

lime_dialect_registry_destroy()

void lime_dialect_registry_destroy

(

LimeDialectRegistry *

reg

)

Destroy the registry.

Releases the snapshot reference held for each registered dialect.

lime_dialect_select()

ParserSnapshot * lime_dialect_select	(	const LimeDialectRegistry *	reg,
		const char *	name
	)

Look up the composed snapshot for a dialect name.

Returns

Borrowed snapshot pointer (NOT acquired; valid while registered), or NULL if the name is unknown. Pass to parse_begin (which acquires its own reference).

lime_mg_resolve()

int lime_mg_resolve	(	const LimeForkCandidate *	cands,
		uint32_t	ncands,
		const int *	lookahead,
		uint32_t	nlook,
		struct LimeBayesStore *	bayes,
		uint16_t	bayes_state,
		uint16_t	bayes_token,
		LimeForkRank *	out_ranks
	)

Resolve a collision by simulating each candidate over the upcoming token stream and ranking the outcomes (Tier 1).

Each candidate is run through lime_simulate_parse against the SAME lookahead buffer (the real upcoming tokens, external codes). The winner is chosen by, in order:

1. reached_accept (true preferred),
2. fewest errors,
3. most tokens consumed (got furthest),
4. lowest priority value,
5. Tier-2 Bayesian posterior (if bayes != NULL),
6. lowest ext_id (deterministic final tie-break).

Parameters

cands	Candidate grammars (>= 1).
ncands	Number of candidates.
lookahead	Upcoming external token codes to simulate against.
nlook	Number of lookahead tokens.
bayes	Optional Tier-2 store consulted only on a Tier-1 tie (NULL to skip).
bayes_state,bayes_token	Key for the Bayesian tie-break.
out_ranks	Optional array of ncands entries filled with per-candidate detail (NULL to skip).

Returns

Index of the winning candidate, or -1 if all failed / bad args.