Main Functions of Lexical Analyzer
Introduction
- The lexical analyzer is one of the first phases of a compiler.
- It is used to transform a high-level program into tokens.
Function of Lexical Analyzer
- Input: Source program (high-level language)
- Output: Stream of tokens.
- The lexical analyzer reads the program character by character.
Types of Tokens
- Tokens can be divided into different categories:
Parser and Tokens
- The parser (Syntax Analyzer) receives tokens from the lexical analyzer.
- Various types of information are needed to identify tokens.
- A parse tree is generated as output.
Symbol Table
- The symbol table is a data structure that stores information about all identifiers and their types.
Task Details
- Example:
a = b + c
- Tokens:
id = id + id
- Here
id
represents identifiers.
Other Important Tasks
- Remove white spaces and tabs.
- Remove comments.
- Identify errors if any and provide the line number of the error.
Conclusion
- The lexical analyzer plays a significant role for a compiler.
- Watch more videos to learn more about it.
- Video content is available for related topics.
These notes provide a summary of the main functions and procedures of the lexical analyzer, allowing students to easily understand how it works.