[fpc-pascal] Split stream into words

[Michael Van Canneyt]

On Tue, 3 Jul 2018, Michael Van Canneyt wrote:

>
> Hi,
>
> What's the easiest way to split a stream into words ?
> Words are just that: words, but - here is the caveat - they must support 
> unicode.
> So Michael and Michaël are both words.
>
> Tried regexpr unit (the obvious choice), but that does not seem to do the 
> trick:

Correction, regexp can handle it if you compile for unicode, and use the
correct regexp...

Michael.