Package COM.INFORMATIMAGO.COMMON-LISP.PARSER.SCANNER

An abstract scanner class.

A method to the SCAN-NEXT-TOKEN generic function needs to be provided.


License:

    AGPL3

    Copyright Pascal J. Bourguignon 2004 - 2016

    This program is free software: you can redistribute it and/or modify
    it under the terms of the GNU Affero General Public License as published by
    the Free Software Foundation, either version 3 of the License, or
    (at your option) any later version.

    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU Affero General Public License for more details.

    You should have received a copy of the GNU Affero General Public License
    along with this program.
    If not, see <http://www.gnu.org/licenses/>

ACCEPT

ADVANCE-LINE

BUFFERED-SCANNER

DO:             This macro generates a simple scanner.  This defines
                subclass of BUFFERED-SCANNER named `NAME'.

RETURN:         NAME

SUPERCLASS:     The name of the superclass of the class NAME. Should
                be a subclass of SCANNER.

TOKEN-CLASS-NAME:
                The name of the class of tokens. Should TOKEN or a
                subclass of TOKEN.

TERMINALS:      A list of couples (name-of-terminals
                regexp-of-terminal) or strings containing the literal terminal..

ALPHANUMERICS:  Characters that may be present in identifiers and keywords,
                and as such, shall be eaten greedily when scanning for keywords.

SKIP-SPACES:    When NIL, the spaces are not eaten between token, but
                they must be dealt with by definiting a terminal
                covering them.

GENERATE-SCANNER

RETURN: The next character in the PEEK-STREAM, advancing.

Makes an instance of the TOKEN class or a subclass
thereof, filled with the current information in the scanner object.

DO:             Just like CL:PEEK-CHAR.

                If peek-type is not supplied or NIL, peek-char returns
                the next character to be read from input-stream,
                without actually removing it from input-stream. The
                next time input is done from input-stream, the
                character will still be there.

                If  peek-type is T, then peek-char skips over
                whitespace[2] characters, but not comments, and then
                performs the peeking operation on the next
                character. The last character examined, the one that
                starts an object, is not removed from  input-stream.

                If peek-type is a CHARACTER, then peek-char skips over
                input characters until a character that is char= to
                that character is found; that character is left in
                input-stream.

RETURN:         The same character that will be returned by the next
                (getchar self).

NOTE:           There's no conforming way to determine whether a
                character has the whitespace[2] syntax in the current
                *readtable*.  Therefore we use instead the
                PEEK-STREAM-SPACES method to get the list of spaces.

PRINT-PARSER-ERROR

PRINT-SCANNER-ERROR

RETURN: A string containing the read line.

DO:           Scans a new token and store it into (scanner-current-token scanner)
PARSER-DATA:  Some parsers give information to the scanner.
RETURN:       (scanner-current-token scanner).

An abstract scanner.

Class precedence list: SCANNER SLOTED-OBJECT STANDARD-OBJECT T

Class init args: SOURCE FILE LINE COLUMN STATE SPACES TAB-WIDTH TOKEN-KIND-PACKAGE

SCANNER-BUFFER

The number of the current column.

Text of the current token

The last token read.

SCANNER-END-OF-LINE-P

SCANNER-END-OF-SOURCE-P

A scanner error.

Class precedence list: SCANNER-ERROR SIMPLE-ERROR SIMPLE-CONDITION ERROR SERIOUS-CONDITION CONDITION STANDARD-OBJECT T

Class init args: FORMAT-CONTROL FORMAT-ARGUMENTS FILE LINE COLUMN STATE CURRENT-TOKEN SCANNER FORMAT-CONTROL FORMAT-ARGUMENTS

The column on which the scanner error was detected.

The scanner token where error was detected.

The error message format control arguments.

The error message format control string.

An invalid character scanner error.

Class precedence list: SCANNER-ERROR-INVALID-CHARACTER SCANNER-ERROR SIMPLE-ERROR SIMPLE-CONDITION ERROR SERIOUS-CONDITION CONDITION STANDARD-OBJECT T

Class init args: FORMAT-CONTROL FORMAT-ARGUMENTS FILE LINE COLUMN STATE CURRENT-TOKEN SCANNER FORMAT-CONTROL FORMAT-ARGUMENTS INVALID-CHARACTER

The invalid character that made the scanner error.

The line on which the scanner error was detected.

The scanner that detected the error.

The scanner state when error was detected.

The namestring of the current file being scanned.

The number of the current line.

The source can be a PEEK-STREAM, a STREAM, or a STRING.

A string containing the characters considered space by SKIP-SPACES.

The state of the scanner.

TAB aligns to column number modulo TAB-WIDTH.

SCANNER-TOKEN-KIND-PACKAGE

DO: Skips over the spaces in the input stream. Updates line and column slots.
RETURN: line; column

A syntactic element.

Class precedence list: TOKEN SLOTED-OBJECT STANDARD-OBJECT T

Class init args: KIND TEXT COLUMN LINE

Returns the column of the first character of the token.

TOKEN-END-OF-SOURCE-KIND

TOKEN-END-OF-SOURCE-P

Returns the kind of the token.

Returns the line where the token was found.

Returns the literal text the token.

DO: Unread the character CH from the PEEK-STREAM.

WORD-EQUAL