Puma::Tokenizer Class Reference

#include <Puma/Tokenizer.h>

Inheritance diagram for Puma::Tokenizer:

Description

Tokenizers read input from a Source to form tokens which are stored in a Unit.

Public Member Functions
virtual	~Tokenizer ()
	Destructor. More...
virtual void	fill_unit (Source &in, Unit &unit)=0
	Fill a unit from the given source. More...

Constructor & Destructor Documentation

~Tokenizer()

virtual Puma::Tokenizer::~Tokenizer ( )

inlinevirtual

Destructor.

Member Function Documentation

fill_unit()

virtual void Puma::Tokenizer::fill_unit ( Source &  in, Unit &  unit  )

pure virtual

Fill a unit from the given source.

Parameters

in	The input source.
unit	The output unit.

Implemented in Puma::CScanner.