BaseScanner

Base class for scanner objects. Recognizes identifiers, numbers and single character tokens. Gets input from cin.

Quick Index

Class Summary

class BaseScanner
{

public:
	BaseScanner() ;
	~BaseScanner() ;
void	Reset(void) ;
const char*	Lexeme(void) const
TokenType	Lookahead(void) const ;
int	AsInt(void) const ;
float	AsFloat(void) const ;
void	Match(TokenType token);
protected:
	BaseScanner(const BaseScanner& s) ;
void	FAdd(char ch) ;
void	FClearLexeme(void) ;
char	FGetChar(void) ;
void	FSkipWS(void);
void	FAdvance(void);
void	FCollectId(void);
void	FCollectNumber(void);
void	FCollectString(void);

}; // BaseScanner

Back to the top of BaseScanner

DESCRIPTION

The BaseScanner class can be used to recognize simple tokens in an input stream. The tokens recognized by the Scanner are:

identifiers A letter followed by zero or more underscores, letters, or digits

numbers A floating point or integer value

The single character tokens @ % ^ = , : and the newline character.

A string of text appearring between double quote characters.

The BaseScanner class skips over comments in the source file. When a semi-colon is encountered the BaseScanner repeatedly calls FGetChar until a newline or eof is encounted.

Back to the top of BaseScanner

DERIVING_CLASSES

The BaseScanner's sole interaction with the input stream is through the protected virtual function FGetChar. By default FGetChar simply reads a character from the standard input. Derived classes can scan arbitrary input streams by overriding the FGetChar method.

Derived classes must always call the Reset method before scanning an input stream. Reset causes the scanner to read the first character and advance to the next token in the input stream. Calling Reset at other times may cause the scanner to lose input.

The protected member function FCollectId has also been declared as virtual to allow the scanner to interface easily with various symbol tables. See the class TVIScanner for an example.

The BaseScanner class also provides the virtual observer function Line which can be used to return the line number for file based input streams.

Back to the top of BaseScanner

EXCEPTIONS

When an unrecoverable error is encounter by a Scanner object it will throw an object of type ScanError which is a subclass of the Exception base class.

Back to the top of BaseScanner

BaseScanner(const BaseScanner& s) ;

Copy Constructor

Note that the copy constructor is protected and can not be instantiated directly.

	BaseScanner(const BaseScanner& s)      
;

Function is currently defined inline.

Back to the top of BaseScanner

void FAdd(char ch) ;

Add a character to the current lexeme.

Parameters

in ch: The character to be added to the lexeme.

Preconditions

There is enough room in the lexeme for the character and the terminating NULL.

Postcoditions

The character ch has been added to the end of the lexeme buffer

The lexeme buffer has been NULL terminated.

The internal pointer fLexPos is incremented

	void FAdd(char ch)
	                                                          
;

Function is currently defined inline.

Back to the top of BaseScanner

void FClearLexeme(void) ;

Clears the lexeme by setting fLexPos to zero and writing a NULL character to the first element of the lexeme buffer

	void FClearLexeme(void)
	                                        
;

Function is currently defined inline.

Back to the top of BaseScanner

char FGetChar(void) ;

Fetches a single character from the input stream.

	virtual char FGetChar(void)                                  
;

Function is currently defined inline.

Back to the top of BaseScanner

void FSkipWS(void);

Skips over whitespace characters in the input buffer. Since the newline character is a token in TVI code these are NOT skipped.

	void FSkipWS(void);

Back to the top of BaseScanner

void FAdvance(void);

Fetch the next token from the input stream. Calls FCollectId or FCollectNumber as appropriate.

Preconditions

The file is open.

The current character is either the first character in the file or the first character following the previous token.

Postconditions

The next token has been scanned from the file.

fLine has been incremented if the token is a newline.

fLexeme contains the surface string of the token just scanned.

fToken has been set to the TokenType of the token.

fCurrentChar contains the first character following the token just scanned.

Throws

a ScanError object if an invalid character is encountered.

	void FAdvance(void);

Back to the top of BaseScanner

void FCollectId(void);

Fetches an identifier from the input stream.

Preconditions

The file is open

The first charact of the identifier has already been added to the lexeme.

Postconditions

The surface string has been copied into fLexeme.

fCurrentChar contains the character immediatedly following the identifier.

fToken is set to T_id

	virtual void FCollectId(void);

Back to the top of BaseScanner

void FCollectNumber(void);

Fetches an integer or floating point number from the input stream

Preconditions

The file is open and the first digit of the number has already been added to the lexeme.

Postconditions

fLexeme contains a copy of the surface string.

fToken is set to T_int if the number was an integer or fToken is set to T_float if the number was a floating point value.

fCurrentChar contains the first character immediately following the number

Throws

A ScanError object if the format of the number is incorrect.

	void FCollectNumber(void);

Back to the top of BaseScanner

void FCollectString(void);

Collects all the text between a pair of double quote characters. Quoted strings are not allowed to span lines.

A quoted string may contain \t, \n for tab and newline characters respectively. If a double quote character must appear inside the string then it must also be escaped with the backslash,

I.E.

	        STR = quote = " This is a \"quote\"..."

Preconditions

The file is open.

fLexeme contains the double quote character.

fCurrentChar contains the first character of the quoted string.

Postconditions

fLexeme contains a copy of the quoted string with the quotes removed.

fToken is set to T_string.

fCurrentChar contains the first character following the quoted string.

Throws

A ScanError object if and illegal escape

	void FCollectString(void);

Back to the top of BaseScanner

BaseScanner() ;

Initialize all members to default values.

	BaseScanner() ;

Function is currently defined inline.

Back to the top of BaseScanner

~BaseScanner() ;

The destructor closes the file if it is open.

	virtual ~BaseScanner()    
;

Function is currently defined inline.

Back to the top of BaseScanner

void Reset(void) ;

Resets the read pointer of the ifstream to the beginning of the file. If the ifstream isn't open no actions are taken.

Postconditions

If the file is open the read pointer is positioned to the beginning of the file.

Throws

Throws an object of type ScanError if the ifstream could not be reset. void Reset(void);

	virtual void Reset(void)
	                                               
;

Function is currently defined inline.

Back to the top of BaseScanner

const char* Lexeme(void) const

Returns the surface string of the next token in the input stream. I.E. if the next token is the opcode move then the TokenType will be T_opcode and the surface string with be the string "move".

	const char* Lexeme(void) const  { return fLexeme; }

Back to the top of BaseScanner

TokenType Lookahead(void) const ;

Returns the TokenType for the next token in the input stream. Returns T_eof if there are no tokens remaining in the stream.

	TokenType Lookahead(void) const                   
;

Function is currently defined inline.

Back to the top of BaseScanner

int AsInt(void) const ;

Converts the lexeme into an integer and returns the result.

Preconditions

this->Lookahead() returns T_int

	int AsInt(void) const                                    
;

Function is currently defined inline.

Back to the top of BaseScanner

float AsFloat(void) const ;

Converts the lexeme into a floating point value and returns the result.

Preconditions

this->Lookahead() return T_float

	float AsFloat(void) const                                
;

Function is currently defined inline.

Back to the top of BaseScanner

void Match(TokenType token);

Compares the paramters token witht the next token in the input stream. If the tokens are the same the scanner advances to the next token in the stream. Throws an exception if the tokens do not match.

Throws

Throws an object of type ScanError if fToken != token when match is called.

	void Match(TokenType token);

Back to the top of BaseScanner

All Members

public:
void	Reset(void) ;
const char*	Lexeme(void) const
TokenType	Lookahead(void) const ;
int	AsInt(void) const ;
float	AsFloat(void) const ;
void	Match(TokenType token);
protected:
void	FAdd(char ch) ;
void	FClearLexeme(void) ;
char	FGetChar(void) ;
void	FSkipWS(void);
void	FAdvance(void);
void	FCollectId(void);
void	FCollectNumber(void);
void	FCollectString(void);

Back to the top of BaseScanner

Ancestors

Class does not inherit from any other class.

Back to the top of BaseScanner

Descendants

Scanner
- TVIScanner

Back to the top of BaseScanner

Generated from source by the Cocoon utilities on Mon Apr 9 01:38:29 2001 .

Report problems to jkotula@vitalimages.com