Class Token


  • public final class Token
    extends java.lang.Object
    A single lexical token: its type, source offset, and payload.
    • Constructor Summary

      Constructors 
      Constructor Description
      Token​(TokenType type, int offset, java.lang.String text)  
      Token​(TokenType type, int offset, java.lang.String text, byte[] bytes)  
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      byte[] getBytes()
      Raw payload for string tokens (already escape/hex decoded).
      int getOffset()  
      java.lang.String getText()
      Textual payload: the number's digits, a name's characters, a keyword, etc.
      TokenType getType()  
      java.lang.String toString()  
      • Methods inherited from class java.lang.Object

        equals, getClass, hashCode, notify, notifyAll, wait, wait, wait
    • Constructor Detail

      • Token

        public Token​(TokenType type,
                     int offset,
                     java.lang.String text)
      • Token

        public Token​(TokenType type,
                     int offset,
                     java.lang.String text,
                     byte[] bytes)
    • Method Detail

      • getOffset

        public int getOffset()
      • getText

        public java.lang.String getText()
        Textual payload: the number's digits, a name's characters, a keyword, etc.
      • getBytes

        public byte[] getBytes()
        Raw payload for string tokens (already escape/hex decoded).
      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object