boost::locale::boundary::token_iterator< IteratorType, ValueType > Class Template Reference
[Boundary Analysis]

token_iterator is an iterator that returns text chunks between boundary positions More...

#include <boost/locale/boundary.hpp>

List of all members.

Public Types

typedef std::iterator_traits
< IteratorType >::value_type 
char_type
typedef IteratorType base_iterator
typedef mapping
< token_iterator< IteratorType,
ValueType > > 
mapping_type

Public Member Functions

 token_iterator ()
token_iterator const & operator= (IteratorType p)
 token_iterator (mapping_type const &map, bool begin, unsigned mask)
 token_iterator (token_iterator const &other)
token_iterator const & operator= (token_iterator const &other)
ValueType operator* () const
token_iteratoroperator++ ()
token_iteratoroperator-- ()
token_iterator operator++ (int unused)
token_iterator operator-- (int unused)
bool full_select () const
void full_select (bool fs)
bool operator== (token_iterator const &other) const
bool operator!= (token_iterator const &other) const
unsigned mark () const


Detailed Description

template<typename IteratorType, typename ValueType = std::basic_string<typename std::iterator_traits<IteratorType>::value_type>>
class boost::locale::boundary::token_iterator< IteratorType, ValueType >

token_iterator is an iterator that returns text chunks between boundary positions

Token iterator may behave in two different ways: select specific tokens in only tide way and select them widely. For tide selection (default) it would not return text chunks that do not fit the selection mask. For example, for word iteration with mask "word_letters" for text "I met him at 7" it would return "I", "met", "him", "at" ignoring white spaces punctuation and numbers, But sometimes, you need to perform full selection of almost entry text for example for sentence boundaries and sentence_term mask you may want to specify full_select(true), So "Hello! How<LF>are you?" would return you biggest possible chunks "Hello!", " How<LF>are you?".

Examples:

boundary.cpp, and wboundary.cpp.


Member Typedef Documentation

template<typename IteratorType, typename ValueType = std::basic_string<typename std::iterator_traits<IteratorType>::value_type>>
typedef std::iterator_traits<IteratorType>::value_type boost::locale::boundary::token_iterator< IteratorType, ValueType >::char_type

The character type of the text

template<typename IteratorType, typename ValueType = std::basic_string<typename std::iterator_traits<IteratorType>::value_type>>
typedef IteratorType boost::locale::boundary::token_iterator< IteratorType, ValueType >::base_iterator

Underlying iterator that is used to iterate original text.

template<typename IteratorType, typename ValueType = std::basic_string<typename std::iterator_traits<IteratorType>::value_type>>
typedef mapping<token_iterator<IteratorType,ValueType> > boost::locale::boundary::token_iterator< IteratorType, ValueType >::mapping_type

The type of mapping that iterator can iterate over it


Constructor & Destructor Documentation

template<typename IteratorType, typename ValueType = std::basic_string<typename std::iterator_traits<IteratorType>::value_type>>
boost::locale::boundary::token_iterator< IteratorType, ValueType >::token_iterator (  )  [inline]

Default constructor

template<typename IteratorType, typename ValueType = std::basic_string<typename std::iterator_traits<IteratorType>::value_type>>
boost::locale::boundary::token_iterator< IteratorType, ValueType >::token_iterator ( mapping_type const &  map,
bool  begin,
unsigned  mask 
) [inline]

Create token iterator for mapping map with location at begin or end according to value of flag begin, and a mask mask

It is strongly recommended to use map.begin(), map.end() instead.

template<typename IteratorType, typename ValueType = std::basic_string<typename std::iterator_traits<IteratorType>::value_type>>
boost::locale::boundary::token_iterator< IteratorType, ValueType >::token_iterator ( token_iterator< IteratorType, ValueType > const &  other  )  [inline]

Copy constructor


Member Function Documentation

template<typename IteratorType, typename ValueType = std::basic_string<typename std::iterator_traits<IteratorType>::value_type>>
token_iterator const& boost::locale::boundary::token_iterator< IteratorType, ValueType >::operator= ( IteratorType  p  )  [inline]

set position of the token iterator to the location of underlying iterator.

This operator sets the token iterator to first token following that position. For example:

For word boundary with "word_any" mask:

  • "to| be or ", would point to "be",
  • "t|o be or ", would point to "to",
  • "to be or| ", would point to end.

p - should be in range of the original mapping.

template<typename IteratorType, typename ValueType = std::basic_string<typename std::iterator_traits<IteratorType>::value_type>>
token_iterator const& boost::locale::boundary::token_iterator< IteratorType, ValueType >::operator= ( token_iterator< IteratorType, ValueType > const &  other  )  [inline]

Assignment operator

template<typename IteratorType, typename ValueType = std::basic_string<typename std::iterator_traits<IteratorType>::value_type>>
ValueType boost::locale::boundary::token_iterator< IteratorType, ValueType >::operator* (  )  const [inline]

Return the token the iterator points it. Iterator must not point to the end of the range. Throws std::out_of_range exception

Note, returned value is not lvalue, you can't use this iterator to assign new values to text.

template<typename IteratorType, typename ValueType = std::basic_string<typename std::iterator_traits<IteratorType>::value_type>>
token_iterator& boost::locale::boundary::token_iterator< IteratorType, ValueType >::operator++ (  )  [inline]

Increment operator

template<typename IteratorType, typename ValueType = std::basic_string<typename std::iterator_traits<IteratorType>::value_type>>
token_iterator& boost::locale::boundary::token_iterator< IteratorType, ValueType >::operator-- (  )  [inline]

Decrement operator

template<typename IteratorType, typename ValueType = std::basic_string<typename std::iterator_traits<IteratorType>::value_type>>
token_iterator boost::locale::boundary::token_iterator< IteratorType, ValueType >::operator++ ( int  unused  )  [inline]

Increment operator

template<typename IteratorType, typename ValueType = std::basic_string<typename std::iterator_traits<IteratorType>::value_type>>
token_iterator boost::locale::boundary::token_iterator< IteratorType, ValueType >::operator-- ( int  unused  )  [inline]

Decrement operator

template<typename IteratorType, typename ValueType = std::basic_string<typename std::iterator_traits<IteratorType>::value_type>>
bool boost::locale::boundary::token_iterator< IteratorType, ValueType >::full_select (  )  const [inline]

Get full selection flag, see description of token_iterator

template<typename IteratorType, typename ValueType = std::basic_string<typename std::iterator_traits<IteratorType>::value_type>>
void boost::locale::boundary::token_iterator< IteratorType, ValueType >::full_select ( bool  fs  )  [inline]

Set full selection flag, see description of token_iterator

template<typename IteratorType, typename ValueType = std::basic_string<typename std::iterator_traits<IteratorType>::value_type>>
bool boost::locale::boundary::token_iterator< IteratorType, ValueType >::operator== ( token_iterator< IteratorType, ValueType > const &  other  )  const [inline]

Compare two iterators. They equal if they point to same map, have same position and same mask

template<typename IteratorType, typename ValueType = std::basic_string<typename std::iterator_traits<IteratorType>::value_type>>
bool boost::locale::boundary::token_iterator< IteratorType, ValueType >::operator!= ( token_iterator< IteratorType, ValueType > const &  other  )  const [inline]

Opposite of ===

template<typename IteratorType, typename ValueType = std::basic_string<typename std::iterator_traits<IteratorType>::value_type>>
unsigned boost::locale::boundary::token_iterator< IteratorType, ValueType >::mark (  )  const [inline]

Return the mark that token iterator points at. See description of mapping class and various boundary flags


The documentation for this class was generated from the following file:

Generated on Thu Mar 18 23:02:03 2010 for Boost.Locale by doxygen 1.5.6