Interface ITokenStringifier

  • All Known Implementing Classes:
    StandardStringifier, StandardTokeniser

    public interface ITokenStringifier
    Interface to define the final step in file preprocessing, it turns the list of tokens into a list of strings paired with their respective line numbers.

    Standard implementations include formatted code (output in same format as input file) and tokenising.

    • Method Summary

      All Methods Instance Methods Abstract Methods 
      Modifier and Type Method Description
      java.util.List<IndexedString> processTokens​(java.util.List<? extends org.antlr.v4.runtime.Token> tokens, org.antlr.v4.runtime.Vocabulary vocab)
      Transform list of tokens into a list of indexed strings, where each string is a line of the original file, and the index of the string is the line number
    • Method Detail

      • processTokens

        java.util.List<IndexedString> processTokens​(java.util.List<? extends org.antlr.v4.runtime.Token> tokens,
                                                    org.antlr.v4.runtime.Vocabulary vocab)
        Transform list of tokens into a list of indexed strings, where each string is a line of the original file, and the index of the string is the line number
        Parameters:
        tokens - the list of preprocessed tokens
        vocab - lexer vocabulary (for tokenising)
        Returns:
        list of line number indexed strings