Overview

Simple SOQL tokenizer Tokens returned are either: - SOQL keyworkds (select, from, where, having, group by, order by, limit, offset) - top level parenthesized expression - top level single quoted expression - strings without white spaces - white spaces