[fpc-pascal] Split stream into words

[Michael Van Canneyt]

On Tue, 3 Jul 2018, Marco van de Voort wrote:

> In our previous episode, Michael Van Canneyt said:
>> 
>> > In our previous episode, Michael Van Canneyt said:
>> >> 
>> >> What's the easiest way to split a stream into words ?
>> >
>> > Doesn't strutils have some word extraction and count functions?
>> 
>> It does: WordCount,ExtractWord, but they are very inefficent.
>
>
> function splitstring(const s:string;c:char):TStringList;
>
> var i,i2,j : integer;
>    x : string;
> begin
>  result:=TStringlist.create;
>  i:=0;
>  repeat
>    j:=PosEx(c,s,i+1);
>    i2:=j;
>    if i2=0 then i2:=length(s)+1;
>    x:=trim(copy(s,i+1,i2-i-1));
>    result.add(x);
>    i:=j;
>  until j=0;
> end;
>
> Afaik I also must have a variant with posset somewhere. In another variant
> I use a class around a array of string, which keeps a count of valid
> entries. This avoids setlengths on repeated use. All fairly trivial.

Trivial indeed, till you need more fine-grained control.
e.g. C needs to be an array of chars that mark word boundaries etc.

But I managed to solve the problem with regexps...

Michael.