Syntax of the Core

This chapter defines the lexical structure of Nex and the grammar of its core: the expressions and statements from which the body of every routine is built. The structure peculiar to classes and modules is deferred to Chapter 3.

The grammar is presented in a variant of Backus–Naur form. Nonterminals are set in slanted type and terminals—reserved words and symbols—in typewriter type. Angle brackets \(\langle\,\cdot\,\rangle\) enclose an optional phrase; a phrase followed by a superscript asterisk may be repeated zero or more times, and one followed by a superscript plus, one or more times. Where a production is a derived form, explained by translation into a simpler one, it is so marked and the translation appears in Appendix C.

2.1Reserved Words

The following are the reserved words of Nex. They may not be used as identifiers.

A few words are contextual: they carry meaning in one position and are ordinary identifiers everywhere else, so they are absent from the list above. where introduces a refinement predicate (Section 3.5.1), and alias introduces the operator binding of a routine (Section 3.4). Each remains available as the name of a variable, field, parameter, or routine. A program may call a field alias and still bind an operator with alias, since no other phrase may appear where the binding does.

union introduces a union declaration (Section 3.2.2). It is contextual in a narrower sense: it stays usable after a . to call an existing member, as in a set’s union method. But a program cannot declare a field, method, constructor, variable, parameter, or routine of its own named union — no rule outside that one member-access position accepts it in place of an identifier.

The boolean constants true and false and the built-in type names Integer, Real, Char, Boolean, String, and Function are also recognised by the lexer. They are not, strictly, keywords usable in arbitrary positions; they denote particular constants and classes of the standard environment (Appendix B), and a program may not redefine them.

The standard environment has further scalar classes that the lexer does not single out: Byte, Integer16, and Integer32, the fixed-width integers, and the name Integer64, which is another spelling of Integer. They are written as ordinary identifiers, and it is the standard environment, not the lexer, that gives them their meaning (Appendix B.2).

The identifier result is reserved within the body of a routine that declares a return type: it names the cell whose final contents become the routine’s result (Section 3.4). The identifier exception is bound, within a rescue block, to the value being handled (Section 5.7).

2.2Special Constants

A special constant is an integer, fixed-width integer, real, character, string, or boolean literal, or the constant nil.

Integer constants

An integer constant is a non-empty sequence of decimal digits, or a based literal introduced by 0b (binary), 0o (octal), or 0x (hexadecimal). An underscore may appear between two digits as a visual separator and has no other significance. There is no sign: a leading minus is the unary operator of Section 2.6, not part of the constant — with the one exception of the signed fixed-width constants below.

A decimal integer constant has type Integer. An Integer is a signed 64-bit integer on every platform, with range \([-2^{63},\, 2^{63}-1]\); a constant outside that range is rejected.

The value spaces of the scalar types are set out in Appendix B, §B.3: the 64-bit, checked Integer, the IEEE 754 Real, and the Unicode character model of Char and String.

Fixed-width integer constants

An integer constant followed immediately by a suffix is a fixed-width integer constant. The suffix u8 makes a Byte, i16 an Integer16, and i32 an Integer32. The digits, bases and separators are those of an integer constant. The suffix is u8 rather than a single letter because b is itself a hexadecimal digit, and it forms one token with the constant by the longest-match rule of Section 2.5.

A byte constant has type Byte and must lie in \([0,\, 255]\); an int16 constant has type Integer16 and must lie in \([-2^{15},\, 2^{15}-1]\); an int32 constant has type Integer32 and must lie in \([-2^{31},\, 2^{31}-1]\). A constant outside its range is rejected.

For the two signed types a leading minus is part of the constant: -5i16 is one constant of type Integer16, not the negation of one, and -32768i16 is a valid constant although 32768i16 is not. The unary minus of Section 2.6, applied to any other operand of a fixed-width type, yields an Integer (Section 4.4); so -5u8 is the Integer \(-5\), a Byte never being negative.

There is no implicit conversion from an integer constant to a fixed-width type. An unsuffixed constant is always an Integer, whatever type the context expects: 200 is not a Byte, and 200u8 must be written (Section 4.3).

Real constants

A real constant has an optional integer part, a mandatory decimal point, a mandatory fractional part of at least one digit, and an optional exponent. Thus 4.5, 10.0, .5, and 12.0e-3 are real constants, while 10. and 12.e-3 are not: a real constant may not end at the point. A real constant has type Real.

Character constants

A character constant is a # followed either by a single character other than a digit or whitespace, by a decimal code point, or by one of the named characters #nul, #space, #newline, #tab, #return. Thus #A, #b, and #65 are character constants. A character constant has type Char.

String constants

A string constant is a sequence of characters enclosed in double quotes "…" or single quotes '…'. A string constant has type String. The two delimiters differ in how a backslash is treated: a double-quoted string interprets escape sequences, while a single-quoted string is raw—every character between the quotes, backslash included, stands for itself.

2.3Comments

A comment is introduced by two hyphens -- and extends to the end of the line. Comments do not nest—there being no closing delimiter—and are discarded by the lexer. A comment is equivalent to a single space.

2.4Identifiers

An identifier begins with a letter or underscore and continues with letters, underscores, and digits. Identifiers are case-sensitive. By convention—and only by convention—class names begin with an upper-case letter and the names of variables, fields, routines, and parameters with a lower-case letter; the grammar does not enforce this.

An identifier that coincides with a reserved word is not an identifier but that reserved word. The longest-match rule (Section 2.5) applies, so classes is a single identifier and not the keyword class followed by es.

2.5Lexical Analysis

The character stream is converted to a stream of tokens by the following rules. Whitespace—spaces, tabs, carriage returns, and line feeds—and comments separate tokens but are otherwise insignificant; Nex is not an indentation-sensitive language. Each token is the longest sequence of characters that can begin a token at the current position (the maximal munch rule). Thus <= is one token and not two, and := is the assignment operator and not a colon followed by an equals sign.

The reserved words, the operator and punctuation symbols, the special constants, and the identifiers are the tokens of the language. The symbol #{ is a single token opening a set display (Section 3 and Appendix C); it must be distinguished by maximal munch from the character constant introducer #. (This means the character literal { must be represented using its numeric code #123).

2.6Operators and Precedence

Nex has a fixed set of infix and prefix operators; the programmer cannot declare new ones or alter their precedence. The operators are listed below from lowest to highest binding power. All binary operators are left-associative, exponentiation ^ included: 2 ^ 3 ^ 2 denotes (2 ^ 3) ^ 2 and evaluates to 64. (Mathematical convention writes the power tower right-associated; Nex trades that convention for a single associativity rule, and a right-nested power must be parenthesised: 2 ^ (3 ^ 2).) The prefix operators are unary minus - and logical not.

Operators are not first-class and do not denote methods directly at the surface. An expression such as a + b is evaluated by the rules of Chapter 5, which appeal to the arithmetic of the operands’ classes. Application (a call or member access) and parentheses bind more tightly than any operator.

2.7The Grammar of Expressions

An expression denotes a value. The expression grammar is layered by the precedence of Section 2.6; we present it here in collapsed form, taking the precedence and associativity as given, and exhibit the layered productions in Appendix A.

The argument list and the collection displays have the obvious forms:

The empty braces {} denote the empty map; the empty set is written #{}. These displays are derived forms, expanded into creations and a sequence of insertions in Appendix C.

2.8The Grammar of Statements

A statement is executed for its effect. A block is a sequence of statements, executed in order. An expression standing alone is a statement, executed for its effect and its value discarded.

select, with its optional timeout clause, is the one concurrency form listed above, and it is a statement. spawn is not: it is an expression, listed among the expression forms in Section 2.7. The timeout clause is not a statement in its own right either, only an optional part of select. The meaning of both is given in Chapter 6.

The host block with is explained in Chapter 5; the loop invariant and variant, in Chapter 3.

A match clause dispatches on the runtime class of the subject. Besides binding the whole value with as, it may also take the value apart. A variant pattern V(…) matches an instance of V and, through its field patterns, binds or constrains the variant’s payload fields by name: a bare name id binds that field to a local of the same name, id: id' renames it, _ ignores it, a literal requires the field to equal that constant, and a nested pattern applies recursively to the field’s value.

A leading _ pattern is a catch-all, equivalent to else. An optional if guard is a boolean tested after the structural match succeeds, with its bindings in scope; when the guard is false, control falls through to the next clause.

Because a guarded, literal, or nested clause may fail after its class matches, such a clause does not on its own cover its variant for the exhaustiveness check of Section 4.4. That check treats a variant named by its qualified name (Section 3.6.1) as covering the same variant a bare-named clause would, not a distinct one.

Patterns and guards are a derived form, translated to the bare id as dispatch of the core in Appendix C. Pure literal-value dispatch remains the role of case.

2.9Syntactic Restrictions

Certain conditions, though expressible in the grammar, are forbidden. They are checked before elaboration and a violation is a compile-time error.

• No two parameters of one routine, nor two fields of one class, nor two local declarations in scope at the same point, may bind the same identifier. An inner let may, however, shadow an outer binding of the same name in a nested block.
• The expression old e may occur only within an ensure clause, and every name e reads must there be a field of the current object (Section 3.4): a parameter, a local, or result beneath old is a compile-time error.
• The identifier result may be read or assigned only within a routine that declares a return type.
• The cursor variable of an across loop, the bound variable of a match clause, and the local of a convert are in scope only within the body they introduce.
• A routine has at most one require clause and at most one ensure clause; multiple conditions are written as named assertions within the single clause (Section 3.4).
• retry may occur only within a rescue block.

Further restrictions that depend on types or on the class hierarchy—the exhaustiveness of a match on a sealed class, the conformance of an overriding routine, the immutability of a once field—belong to the static semantics and are stated in Chapter 4.