In c#:
public static int CountTokens(string input)
{
string pattern = @“/”“(?\.|[^”“\])*”“|'(?:[st]|re|ve|m|ll|d)| ?\p{L}+| ?\p{N}+| ?[^s\p{L}\p{N}]+|\s+(?!\S)|\s+”;
Regex regex = new Regex(pattern, RegexOptions.Compiled | RegexOptions.Multiline);
return regex.Matches(input).Count;
}
This uses the simpler regex provided by raymonddavey – Do you get billed extra when echo=true - #4 by curt.kennedy
Can’t say it’s perfect, but for my purposes, it works.