Home

ANTLR
Reference Manual

Credits

Project Lead
Terence Parr

Support from
MageLang Institute's jGuru Java Training

Help with initial coding
John Lilly, Empathy Software

C++ code generator by
Peter Wells

Substantial intellectual effort donated by
John Mitchell
Scott Stanchfield
Jim Coker
Monty Zukowski (Streams)
Chapman Flack (UNICODE, Streams)

ParseView parser debugger by
Scott Stanchfield

Document Version 2.6.0
May 31, 1999

ParseView debugger is now available.

Check out the Bug Database

Check out http://www.ANTLR.org !!

If you are looking for the C++ version (1.33) of PCCTS rather than the Java version (2.xx), see Getting started with PCCTS.

Download ANTLR 2.6.0.

ANTLR 2.6.0 release notes

ANTLR Meta-Language

• Meta-Language Vocabulary
• Header Section
• Parser Class Definitions
• Lexical Analyzer Class Definitions
• Tree-parser Class Definitions
• Options Section
• Tokens Section
• Grammar Inheritance
• Rule Definitions
• Atomic Production elements
• Simple Production elements
• Production Element Operators
• Token Classes
• Predicates
• Element Labels
• EBNF Rule Elements
• Interpretation Of Semantic Actions
• Semantic Predicates
• Syntactic Predicates
  • Fixed depth lookahead and syntactic predicates
• ANTLR Meta-Language Grammar

Lexical Analysis with ANTLR

• Lexical Rules
  • Return values
  • Skipping characters
  • Distinguishing between lexer rules
  • Definition order and lexical ambiguities
  • Keywords and literals
  • Common prefixes
  • Token definition files
  • Character classes
  • Token Attributes
  • Lexical lookahead and the end-of-token symbol
  • Scanning Binary Files
• Manipulating Token Text and Objects
  • Manipulating the Text of a Lexical Rule
  • Token Object Creation
  • Heterogeneous Token Object Streams
• Filtering Input Streams
• Lexical States
• Case sensitivity
• Ignoring whitespace in the lexer
• Tracking Line Information
• Tracking Column Information
• Using Explicit Lookahead
• A Surprising Use of A Lexer: Parsing
• But...We've Always Used Automata For Lexical Analysis!

ANTLR Tree Parsers

• What's a tree parser?
• What kinds of trees can be parsed?
• Tree grammar rules
  • Syntactic predicates
  • Semantic predicates
  • An Example Tree Walker
• Transformations
• An Example Tree Transformation
• Examining/Debugging ASTs

Token Streams

• Introduction
• Pass-Through Token Stream
• Token Stream Filtering
• Token Stream Splitting
  • Example
  • Filter Implementation
  • How To Use This Filter
  • Tree Construction
  • Garbage Collection Issues
  • Notes
• Token Stream Multiplexing (aka "Lexer states")
  • Multiple Lexers
  • Lexers Sharing Same Character Stream
  • Parsing Multiplexed Token Streams
  • The Effect of Lookahead Upon Multiplexed Token Streams
  • Multiple Lexers Versus Calling Another Lexer Rule
• The Future

Token Vocabularies

• Introduction
  • How does ANTLR decide which vocabulary symbol gets what token type?
  • Why do token types start at 4?
  • What files associated with vocabulary does ANTLR generate?
  • How does ANTLR synchronize the symbol-type mappings between grammars in the same file and in different files?
• Grammar Inheritance and Vocabularies
• Recognizer Generation Order
• Tricky Vocabulary Stuff

Error Handling and Recovery

• Modifying Default Error Messages With Paraphrases
• Parser Exception Handling
• Specifying Parser Exception-Handlers

Java Runtime Model

• Programmer's Interface
• What ANTLR generates
• Multiple Lexers/Parsers With Shared Input State
• Parser Implementation
• Parser Class
• Parser Methods
• EBNF Subrules
• Production Prediction
• Production Element Recognition
• Standard Classes
• Lexer Implementation
• Lexer Form
• Creating Your Own Lexer
• Lexical Rules
• Token Objects
• Token Lookahead Buffer

ANTLR Tree Construction

• Notation
• Controlling AST construction
• Grammar annotations for building ASTs
• Leaf nodes
• Root nodes
• Turning off standard tree construction
• Tree node construction
• AST Action Translation
• Invoking parsers that build trees
• AST Factories
• AST enumerations
• A few examples
• Labeled subrules
• Reference nodes
• Required AST functionality and form

Grammar Inheritance

• Introduction and motivation
• Functionality
• Where Are Those Supergrammars?
• Error Messages

Options

• File, Grammar, and Rule Options
  • Options supported in ANTLR
  • language: Setting the generated language
  • k: Setting the lookahead depth
  • importVocab: Initial Grammar Vocabulary
  • exportVocab: Naming Output Vocabulary
  • testLiterals: Generate literal-testing code
  • defaultErrorHandler: Controlling default exception-handling
  • codeGenMakeSwitchThreshold: controlling code generation
  • codeGenBitsetTestThreshold: controlling code generation
  • buildAST: Automatic AST construction
  • ASTLabelType: Setting label type
  • charVocabulary: Setting the lexer character vocabulary
  • warnWhenFollowAmbig
• Command Line Options