Hansl Primer
Hansl Primer
Allin Cottrell
Department of Economics
Wake Forest University
November, 2020
Permission is granted to copy, distribute and/or modify this document under the terms of the
GNU Free Documentation License, Version 1.1 or any later version published by the Free Software
Foundation (see http://www.gnu.org/licenses/fdl.html).
Contents
1 Introduction 1
I Without a dataset 5
3 Hello, world! 6
3.1 Manipulation of scalars . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.2 Manipulation of strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4 Matrices 11
4.1 Matrix indexing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
4.2 Matrix operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
4.3 Matrix pointers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
5 Nice-looking output 15
5.1 Formatted output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
5.2 Graphics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
7 Numerical methods 23
7.1 Numerical optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
7.2 Numerical differentiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
8 Control flow 28
8.1 The if statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
8.2 Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
8.3 The catch modifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
8.4 The quit statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
9 User-written functions 36
9.1 Parameter passing and return values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
9.2 Recursion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
i
Contents ii
II With a dataset 41
10 What is a dataset? 42
10.1 Creating a dataset from scratch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
10.2 Reading a dataset from a file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
10.3 Saving datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
10.4 The smpl command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
10.5 Dataset accessors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
12 Estimation methods 49
12.1 Canned estimation procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
12.2 Generic estimation tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
12.3 Post-estimation accessors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
12.4 Formatting the results of estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
12.5 Named models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
III Reference 52
14 Operators 54
14.1 Precedence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
14.2 Assignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
14.3 Increment and decrement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
15 Greek-letter identifiers 57
Chapter 1
Introduction
Other resources
If you are serious about learning hansl then after working through this primer — or in the process
of doing so—you’ll want to take a look at the following additional resources.
• The Gretl Command Reference. This contains a complete listing of the commands and built-in
functions available in hansl, with a full account of their syntax and options. Examples of usage
1 Hansl is in fact Turing-complete.
1
Chapter 1. Introduction 2
are provided in some instances. This is available in an “online” version for handy reference as
well as in PDF, both accessible via the Help menu in the gretl GUI.
• The Gretl User’s Guide. Chapters 10 to 16, in particular, go into more detail on some of the
programming topics discussed here (data types, loops, the definition and use of functions). In
addition Part II of the Guide, on Econometric Methods, gives many examples of hansl usage.
The Guide is available via gretl’s Help menu; the latest version can also be found online at
http://sourceforge.net/projects/gretl/files/manual/.
• Sample scripts. The gretl package comes with a large number of sample or practice scripts,
which can be found under the menu item /File/Script files/Practice file. Many of these are
simple replication exercises for textbook problems but you will find some more interesting
examples under the Gretl tab.
• Function packages. Relatively ambitious examples of hansl coding can be found in the various
contributed “function packages”. You can download these packages via the gretl menu item
/Tools/Function packages/On server. Once a package is downloaded it appears in the listing
under /Tools/Function packages/On local machine; in that context you can right-click and
select View code to examine the hansl functions.
• The gretl-users mailing list. Most well-considered questions get answered quite quickly and
fully. See https://gretlml.univpm.it/postorius/lists/.
Chapter 2
OK, so you’re impatient. Then perhaps you’re thinking “Do I really need to go through the whole
thing? After all, I’ve been coding econometric stuff for a while, and I’m pretty confident I can pick
a new scripting language if it’s not too obscure. I just need a few tips to get me started”. If that’s
not what you’re thinking at all, we suggest you move along to chapter 3; but if it is, well then, we’ll
give you a hansl script which exemplifies a hefty share of the topics discussed in the rest of this
primer. We will use for our example a Vector AutoRegressive model, or VAR for short.
As you probably know, a finite-order VAR can be estimated via conditional maximum likelihood
simply by applying OLS to each equation individually. That amounts to solving a least-squares
problem and its solution can be easily written, in matrix notation, as Π̂ = (X 0 X)−1 X 0 Y , where Y con-
tains your endogenous variables and X contains their lags plus other exogenous terms (typically, a
constant term at least). But of course, you may choose to find the maximum of the concentrated
likelihood L = −(T /2) ln |Σ̂| numerically if you so wish.
The following example contains a hansl script which performs these actions:
/* in-house VAR */
3
Chapter 2. For the impatient 4
/* by iterated OLS */
/* via matrices */
scalar n = cols(Y)
scalar k = cols(X)
scalar T = rows(Y)
matrix C = mshape(param, k, n)
matrix E = Y - X*C
matrix Sigma = E’E
print A B C1 C2 C3 D
If you were able to follow the script above in all its parts, congratulations. You probably don’t need
to read the rest of this document (though we don’t mind if you do). But if you find the script too
scary, never fear: we’ll take things step by step. Read on.
Part I
Without a dataset
5
Chapter 3
Hello, world!
We begin with the time-honored “Hello, world” program, the obligatory first step in any program-
ming language. It’s actually very simple in hansl:
# First example
print "Hello, world!"
There are several ways to run the above example: you can put it in a text file first_ex.inp and
have gretl execute it from the command line through the command
gretlcli -b first_ex.inp
or you could just copy its contents in the editor window of a GUI gretl session and click on the
“gears” icon. It’s up to you; use whatever you like best.
From a syntactical point of view, allow us to draw attention on the following points:
1. The line that begins with a hash mark (#) is a comment: if a hash mark is encountered,
everything from that point to the end of the current line is treated as a comment, and ignored
by the interpreter.
2. The next line contains a command (print) followed by an argument; this is fairly typical of
hansl: many jobs are carried out by calling commands.
3. Hansl does not have an explicit command terminator such as the “;” character in the C lan-
guage family (C++, Java, C#, . . . ) or GAUSS; instead it uses the newline character as an implicit
terminator. So at the end of a command, you must insert a newline; conversely, you can’t put
a newline in the middle of a command — or not without taking special measures. If you need
to break a command over more than one line for the sake of legibility you can use the “\”
(backslash) character, which causes gretl to ignore the following line break.
Note also that the print command automatically appends a line break, and does not recognize
“escape” sequences such as “\n”; such sequences are just printed literally. The printf command
can be used for greater control over output; see chapter 5.
Let’s now examine a simple variant of the above:
/*
Second example
*/
string foo = "Hello, world"
print foo
In this example, the comment is written using the convention adopted in the C programming lan-
guage: everything between “/*” and “*/” is ignored.1 Comments of this type cannot be nested.
1 Each type of comment can be masked by the other:
• If /* follows # on a given line which does not already start in ignore mode, then there’s nothing special about /*,
it’s just part of a #-style comment.
• If # occurs when we’re already in comment mode, it is just part of a comment.
6
Chapter 3. Hello, world! 7
In this line, we assign the value “Hello, world” to the variable named foo. Note that
2. The name of the variable (its identifier) must follow the following convention: identifiers can
be at most 31 characters long and must be plain ASCII. They must start with a letter, and can
contain only letters, numbers and the underscore character.2 Identifiers in hansl are case-
sensitive, so foo, Foo and FOO are three distinct names. Of course, some words are reserved
and can’t be used as identifiers (however, nearly all reserved words only contain lowercase
characters).
In hansl, a variable has to be of one of these types: scalar, series, matrix, list, string, bundle
or array. As we’ve just seen, string variables are used to hold sequences of alphanumeric charac-
ters. We’ll introduce the other ones gradually; for example, the matrix type will be the object of
the next chapter.
The reader may have noticed that the line
implicitly performs two tasks: it declares foo as a variable of type string and, at the same time,
assigns a value to foo. The declaration component is not strictly required. In most cases gretl
is able to figure out by itself what type a newly introduced variable should have, and the line
foo = "Hello, world" (without a type specifier) would have worked just fine. However, it is
more elegant (and leads to more legible and maintainable code) to use a type specifier at least the
first time you introduce a variable.
In the next example, we will use a variable of the scalar type:
scalar x = 42
print x
string a = "zoo"
a = 3.14 # no, no, no!
If you try running the example above, an error will be flagged. However, it is acceptable to destroy
the original variable, via the delete command, and then re-declare it, as in
scalar X = 3.1415
delete X
string X = "apple pie"
2 Actually one exception to this rule is supported: identifiers taking the form of a single Greek letter. See chapter 15
for details.
Chapter 3. Hello, world! 8
There is no “type-casting” as in C, but some automatic type conversions are possible (more on this
later).
Many commands can take more than one argument, as in
scalar x = 42
string foo = "not bad"
print x foo
In this example, one print is used to print the values of two variables; more generally, print can
be followed by as many arguments as desired. The other difference with respect to the previous
code examples is in the use of the two set commands. Describing the set command in detail
would lead us to an overly long diversion; suffice it to say that this command is used to set the
values of various “state variables” that influence the behavior of the program; here it is used as a
way to silence unwanted output. See the Gretl Command Reference for more on set.
The eval command is useful when you want to look at the result of an expression without assigning
it to a variable; for example
eval 2+3*4
will print the number 14. This is most useful when running gretl interactively, like a calculator, but
it is usable in a hansl script for checking purposes, as in the following (rather silly) example:
scalar a = 1
scalar b = -1
# this ought to be 0
eval a+b
in which we assume that x, m and s2 are pre-existing scalars. The example above contains two
noteworthy points:
• The usage of the exp (exponential) and sqrt (square root) functions; it goes without saying
that hansl possesses a reasonably wide repertoire of such functions. See the Gretl Command
Reference for the complete list.
• The usage of $pi for the constant π . While user-specified identifiers must begin with a letter,
built-in identifiers for internal objects typically have a “dollar” prefix; these are known as
accessors (basically, read-only variables). Most accessors are defined in the context of an open
dataset (see part II), but some represent pre-defined constants, such as π . Again, see the Gretl
Command Reference for a comprehensive list.
Hansl does not possess a specific Boolean type, but scalars can be used for holding true/false
values. It follows that you can also use the logical operators and (&&), or (||), and not (!) with
scalars, as in the following example:
a = 1
b = 0
c = !(a && b)
Chapter 3. Hello, world! 9
In the example above, c will equal 1 (true), since (a && b) is false, and the exclamation mark is the
negation operator. Note that 0 evaluates to false, and anything else (not necessarily 1) evaluates to
true.
A few constructs are taken from the C language family: one is the postfix increment operator:
a = 5
b = a++
print a b
the second line is equivalent to b = a, followed by a++, which in turn is shorthand for a = a+1,
so running the code above will result in b containing 5 and a containing 6. Postfix subtraction is
also supported; prefix operators, however, are not supported. Another C borrowing is inflected
assignment, as in a += b, which is equivalent to a = a + b; several other similar operators are
available, such as -=, *= and more. See the Gretl Command Reference for details.
The internal representation for a missing value is NaN (“not a number”), as defined by the IEEE 754
floating point standard. This is what you get if you try to compute quantities like the square root
or the logarithm of a negative number. You can also set a value to “missing” directly using the
keyword NA. The complementary functions missing and ok can be used to determine whether a
scalar is NA. In the following example a value of zero is assigned to the variable named test:
scalar not_really = NA
scalar test = ok(not_really)
if x == NA ... # wrong!
because a missing value is taken as indeterminate and hence not equal to anything. This last
example, despite being wrong, illustrates a point worth noting: the test-for-equality operator in
hansl is the double equals sign, “==” (as opposed to plain “=” which indicates assignment).
string s = "endogenous"
string pet = substr(s, 3, 5)
which would result to assigning the value dog to the variable pet.
The following are useful operators for strings:
• the closely related ~= operator, which acts as an inflected assignment operator (so a ~= "_ij"
is equivalent to a = a ~ "_ij");
• the offset operator +, which yields a substring of the preceding element, starting at the given
character offset. An empty string is returned if the offset is greater than the length of the
string in question.
A noteworthy point: strings may be (almost) arbitrarily long; moreover, they can contain special
characters such as line breaks and tabs. It is therefore possible to use hansl for performing rather
complex operations on text files by loading them into memory as a very long string and then
operating on that; interested readers should take a look at the readfile, getline, strsub and
regsub functions in the Gretl Command Reference.5
For creating complex strings, the most flexible tool is the sprintf function. Its usage is illustrated
in Chapter 5.
5 We are not claiming that hansl would be the tool of choice for text processing in general. Nonetheless the functions
mentioned here can be very useful for tasks such as pre-processing plain text data files that do not meet the requirements
for direct importation into gretl.
Chapter 4
Matrices
matrix X = zeros(2,3)
X[2,1] = 4
print X
produces
X (2 x 3)
0 0 0
4 0 0
1. In case the matrix has only one row (column), the column (row) specification can be omitted,
as in x[3].
2. Including the comma but omitting the row or column specification means “take them all”, as
in x[4,] (fourth row, all columns).
3. For square matrices, the special syntax x[diag] can be used to access the diagonal.
4. Consecutive rows or columns can be specified via the colon (:) character, as in x[,2:4]
(columns 2 to 4). But note that, unlike some other matrix languages, the syntax [m:n] is
illegal if m > n.
5. It is possible to use a vector to hold indices to a matrix. E.g. if e = [2, 3, 6], then X[,e]
contains the second, third and sixth columns of X.
scalar a = 2*3
matrix A = { 1, 2, 3 ; 4, 5, a }
The matrix is defined by rows; the elements on each row are separated by commas and rows are
separated by semicolons. The whole expression must be wrapped in braces. Spaces within the
braces are not significant. The above expression defines a 2 × 3 matrix.
11
Chapter 4. Matrices 12
Note that each element should be a numerical value, the name of a scalar variable, or an expression
that evaluates to a scalar. In the example above the scalar a was first assigned a value and then
used in matrix construction. (Also note, in passing, that a and A are two separate identifiers, due
to case-sensitivity.)
Other noteworthy matrix operators include ˆ (matrix power), ** (Kronecker product), and the “con-
catenation” operators, ~ (horizontal) and | (vertical). Readers are invited to try them out by running
the following code
matrix A = {2,1;0,1}
matrix B = {1,1;1,0}
matrix KP = A ** B
matrix PWR = A^3
matrix HC = A ~ B
matrix VC = A | B
print A B KP PWR HC VC
Note, in particular, that A3 = A · A · A, which is different from what you get by computing the cubes
of each element of A separately.
Hansl also supports matrix left- and right-“division”, via the \ and / operators, respectively. The
expression A\b solves Ax = b for the unknown x. A is assumed to be an m × n matrix with full
column rank. If A is square the method is LU decomposition. If m > n the QR decomposition is
used to find the least squares solution. In most cases, this is numerically more robust and more
efficient than inverting A explicitly.
Element-by-element operations are supported by the so-called “dot” operators, which are obtained
by putting a dot (“.”) before the corresponding operator. For example, the code
A = {1,2; 3,4}
B = {-1,0; 1,-1}
eval A * B
eval A .* B
1 In fact, in this case an optimized algorithm is used; you should always use a’a instead of a’*a for maximal precision
and performance.
Chapter 4. Matrices 13
produces
1 -2
1 -4
-1 0
3 -4
It’s easy to verify that the first operation performed is regular matrix multiplication A · B, whereas
the second one is the Hadamard (element-by-element) product A B. In fact, dot operators are
more general and powerful than shown in the example above; see the chapter on matrices in the
Gretl User’s Guide for details.
Dot and concatenation operators are less rigid than ordinary matrix operations in terms of con-
formability requirements: in most cases hansl will try to do “the obvious thing”. For example, a
common idiom in hansl is Y = X ./ w, where X is an n × k matrix and w is an n × 1 vector. The
result Y is an n × k matrix in which each row of X is divided by the corresponding element of w.
In proper matrix notation, this operation should be written as
Y = hwi−1 X,
where the h·i indicates a diagonal matrix. Translating literally the above expression would imply
creating a diagonal matrix out of w and then inverting it, which is computationally much more
expensive than using the dot operation. A detailed discussion is provided in the Gretl User’s Guide.
Hansl provides a reasonably comprehensive set of matrix functions, that is, functions that produce
and/or operate on matrices. For a full list, see the Gretl Command Reference, but a basic “survival
kit” is provided in Table 4.1. Moreover, most scalar functions, such as abs(), log() etc., will
operate on a matrix element-by-element.
Function(s) Purpose
rows(X), cols(X) return the number of rows and columns of X, respectively
zeros(r,c), ones(r,c) produce matrices with r rows and c columns, filled with zeros
and ones, respectively
mshape(X,r,c) rearrange the elements of X into a matrix with r rows and c
columns
I(n) identity matrix of size n
seq(a,b) generate a row vector containing integers from a to b
inv(A) invert, if possible, the matrix A
maxc(A), minc(A), meanc(A) return a row vector with the max, min, means of each column of
A, respectively
maxr(A), minr(A), meanr(A) return a column vector with the max, min, means of each row of
A, respectively
mnormal(r,c), muniform(r,c) generate r ×c matrices filled with standard Gaussian and uniform
pseudo-random numbers, respectively
The following piece of code is meant to provide a concise example of all the features mentioned
above.
# now do estimation
matrix X = 1 ~ Z # concatenation operator
matrix beta_hat1 = inv(X’X) * (X’y) # OLS by hand
matrix beta_hat2 = mols(y, X) # via the built-in function
matrix beta_hat3 = X\y # via matrix division
Nice-looking output
where X, Y and Z are placeholders for values not known at the time of writing the script; they will
be filled out as the values of variables or expressions when the script is run. Let’s say that at run
time the replacements in the sentence above should come from variables named vname (a string),
b (a scalar value) and se (also a scalar value), respectively.
Across the spectrum of programming languages there are basically two ways of arranging for this.
One way originates in the C language and goes under the name printf. In this approach we (a)
replace the generic placeholders X, Y and Z with more informative conversion specifiers, and (b)
append the variables (or expressions) that are to be stuck into the text, in order. Here’s the hansl
version:
The value of vname replaces the conversion specifier “%s,” and the values of b and se replace the
two “%g” specifiers, left to right. In relation to hansl, here are the basic points you need to know:
“%s” pairs with a string argument, and “%g” pairs with a numeric argument.
The C-derived printf (either in the form of a function, or in the form of a command as shown
above) is present in most “serious” programming languages. It is extremely versatile, and in its
advanced forms affords the programmer fine control over the output.
In some scripting languages, however, printf is reckoned “too difficult” for non-specialist users.
In that case some sort of substitute is typically offered. We’re skeptical: “simplified” alternatives
to printf can be quite confusing, and if at some point you want fine control over the output, they
either do not support it, or support it only via some convoluted mechanism. A typical alternative
looks something like this (please note, display is not a hansl command, it’s just illustrative):
display "The coefficient on ", vname, "is ", b, ", with standard error ", se, "\n"
That is, you break the string into pieces and intersperse the names of the variables to be printed.
The requirement to provide conversion specifiers is replaced by a default automatic formatting of
the variables based on their type. By the same token, the command line becomes peppered with
mutiple commas and quotation marks. If this looks preferable to you, you are welcome to join one
of the gretl mailing lists and argue for its provision!
Anyway, to be a bit more precise about printf, its syntax goes like this:
The format is used to specify the precise way in which you want the arguments to be printed.
15
Chapter 5. Nice-looking output 16
will print
• The first special sequence is %d: this indicates that we want an integer at that place in the
output; since it is the leftmost “percent” expression, it is matched to the first argument, that
is 5.
• The second special sequence is %6.4f, which stands for a decimal value with 4 digits after the
decimal separator1 and at least 6 digits wide; this will be matched to the second argument.
Note that arguments are separated by commas. Also note that the second argument is neither
a scalar constant nor a scalar variable, but an expression that evaluates to a scalar.
• The format string ends with the sequence \n, which inserts a newline.
The conversion specifiers in the square-root example are relatively fancy, but as we noted earlier
%g will work fine for almost all numerical values in hansl. So we could have used the simpler form:
The effect of %g is to print a number using up to 6 significant digits (but dropping trailing zeros); it
automatically switches to scientific notation if the number is very large or very small. So the result
here is
The escape sequences \n (newline), \t (tab), \v (vertical tab) and \\ (literal backslash) are recog-
nized. To print a literal percent sign, use %%.
Apart from those shown in the above example, recognized numeric formats are %e, %E, %f, %g, %G
and %x, in each case with the various modifiers available in C. The format %s should be used for
strings. As in C, numerical values that form part of the format (width and or precision) may be
given directly as numbers, as in %10.4f, or they may be given as variables. In the latter case, one
puts asterisks into the format string and supplies corresponding arguments in order. For example,
scalar width = 12
scalar precision = 6
printf "x = %*.*f\n", width, precision, x
If a matrix argument is given in association with a numeric format, the entire matrix is printed
using the specified format for each element. A few more examples are given in table 5.1.
1 The decimal separator is the dot in English, but may be different in other locales.
Chapter 5. Nice-looking output 17
Command effect
printf "%12.3f", $pi 3.142
printf "%12.7f", $pi 3.1415927
printf "%6s%12.5f%12.5f %d\n", "alpha",
3.5, 9.1, 3 alpha 3.50000 9.10000 3
printf "%6s%12.5f%12.5f\t%d\n", "beta",
1.2345, 1123.432, %11 beta 1.23450 1123.43200 11
printf "%d, %10d, %04d\n", 1,2,3 1, 2, 0003
printf "%6.0f (%5.2f%%)\n", 32, 11.232 32 (11.23%)
Output to a string
A closely related effect can be achieved via the sprintf function: instead of being printed directly
the result is stored in a named string variable, as in
Output to a file
Hansl does not have a file or “stream” type as such, but the outfile command can be used to
divert output to a named text file. To start such redirection you must give the name of a file;
by default a new file is created or an existing one overwritten but the --append can be used to
append material to an existing file. Only one file can be opened in this way at any given time. The
redirection of output continues until the command end outfile is given; then output reverts to
the default stream.
Here’s an example of usage:
printf "One!\n"
outfile "myfile.txt"
printf "Two!\n"
end outfile
printf "Three!\n"
outfile "myfile.txt" --append
printf "Four!\n"
end outfile
printf "Five!\n"
After execution of the above the file myfile.txt will contain the lines
Two!
Four!
Three special variants on the above are available. If you give the keyword null in place of a real
filename along with the write option, the effect is to suppress all printed output until redirection
is ended. If either of the keywords stdout or stderr are given in place of a regular filename the
effect is to redirect output to standard output or standard error output, respectively.
This command also supports a --quiet option: its effect is to turn off the echoing of commands
and the printing of auxiliary messages while output is redirected. It is equivalent to doing
before invoking outfile, except that when redirection is ended the prior values of the echo and
messages state variables are restored.
5.2 Graphics
The primary graphing command in hansl is gnuplot which, as the name suggests, in fact provides
an interface to the gnuplot program. It is used for plotting series in a dataset (see part II) or
columns in a matrix. For an account of this command (and some other more specialized ones, such
as boxplot and qqplot), see the Gretl Command Reference.
Chapter 6
Hansl possesses two kinds of “structured data type”: associative arrays, called bundles and arrays
in the proper sense of the word. Loosely speaking, the main difference between the two is that
in a bundle you can pack together variables of different types, while arrays can hold one type of
variable only.
6.1 Bundles
Bundles are associative arrays, that is, generic containers for any assortment of hansl types (in-
cluding other bundles) in which each element is identified by a string. Python users call these
dictionaries; in C++ and Java, they are referred to as maps; they are known as hashes in Perl. We
call them bundles. Each item placed in the bundle is associated with a key which can used to retrieve
it subsequently.
To use a bundle you first either “declare” it, as in
bundle foo
These two formulations are basically equivalent, in that they both create an empty bundle. The
difference is that the second variant may be reused — if a bundle named foo already exists the
effect is to empty it—while the first may only be used once in a given gretl session; it is an error to
declare a variable that already exists.
To add an object to a bundle you assign to a compound left-hand value: the name of the bundle
followed by the key. The most common way to do this is to join the key to the bundle name with a
dot, as in
foo.matrix1 = m
which adds an object called m (presumably a matrix) to bundle foo under the key matrix1. The key
must satisfy the rules for a gretl variable name (31 characters maximum, starting with a letter and
composed of just letters, numbers or underscore)
An alternative way to achieve the same effect is to give the key as a quoted string literal enclosed
in square brackets, as in
foo["matrix1"] = m
When using the more elaborate syntax, keys do not have to be valid as variable names — for example,
they can include spaces—but they are still limited to 31 characters.
To get an item out of a bundle, again use the name of the bundle followed by the key, as in
matrix bm = foo.matrix1
# or using the long-hand notation
matrix m = foo["matrix1"]
19
Chapter 6. Structured data types 20
Note that the key identifying an object within a given bundle is necessarily unique. If you reuse an
existing key in a new assignment, the effect is to replace the object which was previously stored
under the given key. It is not required that the type of the replacement object is the same as that
of the original.
A quicker way, introduced in gretl 2017b, is to use the defbundle function, as in
in which every odd-numbered argument must evaluate to a string (key) and every even-numbered
argument must evaluate to an object of a type that can be included in a bundle.
Note that when you add an object to a bundle, what in fact happens is that the bundle acquires
a copy of the object. The external object retains its own identity and is unaffected if the bundled
object is replaced by another. Consider the following script fragment:
bundle foo
matrix m = I(3)
foo.mykey = m
scalar x = 20
foo.mykey = x
After the above commands are completed bundle foo does not contain a matrix under mykey, but
the original matrix m is still in good health.
To delete an object from a bundle use the delete command, with the bundle/key combination, as
in
delete foo.mykey
delete foo["quoted key"]
This destroys the object associated with the key and removes the key from the hash table.1
Besides adding, accessing, replacing and deleting individual items, the other operations that are
supported for bundles are union and printing. As regards union, if bundles b1 and b2 are defined
you can say
bundle b3 = b1 + b2
to create a new bundle that is the union of the two others. The algorithm is: create a new bundle
that is a copy of b1, then add any items from b2 whose keys are not already present in the new
bundle. (This means that bundle union is not necessarily commutative if the bundles have one or
more key strings in common.)
If b is a bundle and you say print b, you get a listing of the bundle’s keys along with the types of
the corresponding objects, as in
? print b
bundle b:
x (scalar)
mat (matrix)
inside (bundle)
Bundle usage
To illustrate the way a bundle can hold information, we will use the Ordinary Least Squares (OLS)
model as an example: the following code estimates an OLS regression and stores all the results in
a bundle.
1 Internally, gretl bundles in fact take the form of GLib hash table.
Chapter 6. Structured data types 21
my_model.betahat = b
my_model.s2 = s2
my_model.vcv = V
my_model.stderr = sqrt(diag(V))
The bundle so obtained is a container that can be used for all sort of purposes. For example, the
next code snippet illustrates how to use a bundle with the same structure as the one created above
to perform an out-of sample forecast. Imagine that k = 4 and the value of x for which we want to
forecast y is
x0 = [10 1 − 3 0.5]
The formulae for the forecast would then be
ŷf = x0 β̂
q
sf = σ̂ 2 + x0 V (β̂)x
CI = ŷf ± 1.96sf
where CI is the (approximate) 95 percent confidence interval. The above formulae translate into
6.2 Arrays
A gretl array is a container which can hold zero or more objects of a certain type, indexed by
consecutive integers starting at 1. It is one-dimensional. This type is implemented by a quite
“generic” back-end. The types of object that can be put into arrays are strings, matrices, bundles
and lists; a given array can hold only one of these types.
Array operations
The following is, we believe, rather self-explanatory:
strings S1 = array(3)
matrices M = array(4)
strings S2 = defarray("fish", "chips")
S1[1] = ":)"
S1[3] = ":("
M[2] = mnormal(2,2)
print S1
eval inv(M[2])
Chapter 6. Structured data types 22
S = S1 + S2
print S
The array() takes an integer argument for the array size; the defarray() function takes a variable
number of arguments (one or more), each of which may be the name of a variable of the given type
or an expression which evaluates to an object of that type. The corresponding output is
0.52696 0.28883
-0.15332 -0.68140
In order to find the number of elements in an array, you can use the nelem() function.
Chapter 7
Numerical methods
where f (x; a) is a function of x, whose shape depends on a vector of parameters a. The objective
function f (·) is assumed to return a scalar real value. In most cases, it will be assumed it is also
continuous and differentiable, although this need not necessarily be the case. (Note that while
hansl’s built-in functions maximize the given objective function, minimization can be achieved
simply by flipping the sign of f (·).)
A special case of the above occurs when x is a vector of parameters and a represents “data”. In
these cases, the objective function is usually a (log-)likelihood and the problem is one of estimation.
For such cases hansl offers several special constructs, reviewed in section 12.2. Here we deal with
more generic problems; nevertheless, the differences are only in the hansl syntax involved: the
mathematical algorithms that gretl employs to solve the optimization problem are the same.
The reader is invited to read the “Numerical methods” chapter of the Gretl User’s Guide for a
comprehensive treatment. Here, we will only give a small example which should give an idea of
how things are done.
# ----------------------------------------------------------------------
set max_verbose 1
23
Chapter 7. Numerical methods 24
matrix theta1 = { 0, 0 }
y1 = BFGSmax(theta1, "Himmelblau(theta1)")
matrix theta2 = { 0, -1 }
y2 = NRmax(theta2, "Himmelblau(theta2)")
We use for illustration here a classic “nasty” function from the numerical optimization literature,
namely the Himmelblau function, which has four different minima; f (x, y) = (x 2 + y − 11)2 + (x +
y 2 − 7)2 . The example proceeds as follows.
1. First we define the function to optimize: it must return a scalar and have among its arguments
the vector to optimize. In this particular case that is its only argument, but there could have
been other ones if necessary. Since in this case we are solving for a minimum our definition
returns the negative of the Himmelblau function proper.
2. We next set max_verbose to 1. This is another example of the usage of the set command; its
meaning is “let me see how the iterations go” and it defaults to 0. By using the set command
with appropriate parameters, you control several features of the optimization process, such
as numerical tolerances, visualization of the iterations, and so forth.
4. Invoke the BFGSmax function; this will seek the maximum via the BFGS technique. Its base
syntax is BFGSmax(arg1, arg2), where arg1 is the vector contining the optimization variable
and arg2 is a string containing the invocation of the function to maximize. BFGS will try
several values of θ1 until the maximum is reached. On successful completion, the vector
theta1 will contain the final point. (Note: there’s much more to this. For details, be sure to
read the Gretl User’s Guideand the Gretl Command Reference.)
5. Then we tackle the same problem but with a different starting point and a different optimiza-
tion technique. We start from θ2 = [0, −1] and use Newton–Raphson instead of BFGS, calling
the NRmax() function instead if BFGSmax(). The syntax, however, is the same.
Table 7.1 on page 27 contains a selected portion of the output. Note that the second run converges
to a different local optimum than the first one. This is a consequence of having initialized the
algorithm with a different starting point. In this example, numerical derivatives were used, but
you can supply analytically computed derivatives to both methods if you have a hansl function for
them; see the Gretl User’s Guide for more detail.
The optimization methods hansl puts at your disposal are:
• BFGS, via the BFGSmax() function. This is in most cases the best compromise between perfor-
mance and robustness. It assumes that the function to maximize is differentiable and will try
to approximate its curvature by clever use of the change in the gradient between iterations.
You can supply it with an analytically-computed gradient for speed and accuracy, but if you
don’t, the first derivatives will be computed numerically.
• Newton–Raphson, via the NRmax() function. Actually, the name is misleading. It should be
called something like “curvature-based”, since it relies on the iterations
where g(x) is the gradient and C(xi ) is some measure of curvature of the function to op-
timize; if C(x) is the Hessian matrix, you get Newton–Raphson. Again, you can code your
own functions for g(·) and C(·), but if you don’t then numerical approximations to the gra-
dient and the Hessian will be used, respectively. Other popular optimization methods (such
Chapter 7. Numerical methods 25
as BHHH and the scoring algorithm) can be implemented by supplying to NRmax() the appro-
priate curvature matrix C(·). This method is very efficient when it works, but is rather fragile:
for example, if C(xi ) happens to be non-negative definite at some iteration convergence may
become problematic.
• Derivative-free methods: the only method that hansl offers presently is simulated annealing,
via the simann() function, but an implementiation of the Nelder–Mead algorithm (also known
as the “amoeba” method) should be just a matter of time. These methods work even when
the function to maximize has some form of disconinuity or is not everywhere differentiable;
however, they may be very slow and CPU-intensive.
a = 3.5
b = 2.5
returns
Details on the algorithm used can be found in the Gretl Command Reference. Suffice it to say
here that you have a fdjac_quality setting that goes from 0 to 2. The default value is to 0,
which gives you forward-difference approximation: this is the fastest algorithm, but sometimes
may not be precise enough. The value of 1 gives you bilateral difference, while 2 uses Richardson
extrapolation. As you go up, you gain in accuracy, but the method becomes considerably more
CPU-intensive.
Chapter 7. Numerical methods 27
? matrix theta1 = { 0, 0 }
Replaced matrix theta1
? y1 = BFGSmax(theta1, "Himmelblau(11, theta1)")
Iteration 1: Criterion = -170.000000000
Parameters: 0.0000 0.0000
Gradients: 14.000 22.000 (norm 0.00e+00)
...
Function evaluations: 39
Evaluations of gradient: 16
Replaced scalar y1 = -1.83016e-28
? matrix theta2 = { 0, -1 }
Replaced matrix theta2
? y2 = NRmax(theta2, "Himmelblau(11, theta2)")
Iteration 1: Criterion = -179.999876556 (steplength = 1)
Parameters: 1.0287e-05 -1.0000
Gradients: 12.000 2.8422e-06 (norm 7.95e-03)
...
y1 = -1.8301573e-28
y2 = -1.0581385e-07
theta1 (1 x 2)
3 2
theta2 (1 x 2)
3.5844 -1.8481
Control flow
The primary means for controlling the flow of execution in a hansl script are the if statement
(conditional execution), the loop statement (repeated execution), the catch modifier (which enables
the trapping of errors that would otherwise halt execution), and the quit command (which forces
termination).
if <condition>
...
elif <condition>
...
else
...
endif
Points to note:
• The <condition> can be any expression that evaluates to a scalar: 0 is interpreted as “false”,
non-zero is interpreted as “true”; NA generates an error.
• Following if, “then” is implicit; there is no then keyword as found in, e.g., Pascal or Basic.
• The elif and else clauses are optional: the minimal form is just if . . . endif.
Example:
scalar x = 15
28
Chapter 8. Control flow 29
Note, from the example above, that the elif keyword can be repeated, making hansl’s if statement
a multi-way branch statement. There is no separate switch or case statement in hansl. With one
or more elifs, hansl will execute the first one for which the logical condition is satisfied and then
jump to endif.
+ Stata users, beware: hansl’s if statement is fundamentally different from Stata’s if option: the latter se-
lects a subsample of observations for some action, while the former is used to decide if a group of statements
should be executed or not; hansl’s if is what Stata calls “branching if”.
scalar ax = x>=0 ? x : -x
Of course, in the above case it would have been much simpler to just write ax = abs(x). Consider,
however, the following case, which exploits the fact that the ternary operator can be nested:
This example deserves a few comments. We want to compute the number of days in a month, coded
in the variable m. The value we assign to the scalar days comes from the following pathway.
1. First we check if the month is February (m==2); if so, we set days to 28 and we’re done.2
2. Otherwise, we compute a matrix of zeros and ones via the operation m.={4,6,9,11} (note
the use of the “dot” operator to perform an element-by element comparison — see section 4.2);
if m equals any of the elements in the vector, the corresponding element of the result will be
1, and 0 otherwise;
3. The maxr function gives the maximum of this vector, so we’re checking whether m is any one
of the four values corresponding to 30-day months.
4. Since the above evaluates to a scalar, we put the right value into days.
The ternary operator is more flexible than the ordinary if statement. With if, the <condition>
to be evaluated must always come down to a scalar, but the query operator just requires that the
condition is of “suitable” type in light of the types of the operands. So, for example, suppose you
have a square matrix A and you want to switch the sign of the negative elements of A on and above
its diagonal. You could use a loop,3 and write a piece of code such as
1 Some readers may find it helpful to note that the conditional assignment operator works in exactly the same way as
the =IF() function in spreadsheets.
2 OK, we’re ignoring leap years here.
3 The loop keyword is explained in detail in the next section.
Chapter 8. Control flow 30
matrix A = mnormal(4,4)
matrix B = A
loop r = 1 .. rows(A)
loop c = r .. cols(A)
if A[r,c] < 0
B[r,c] = -A[r,c]
endif
endloop
endloop
By using the ternary operator, you can achieve the same effect via a considerably shorter (and
faster) construct:
matrix A = mnormal(4,4)
matrix B = upper(A.<0) ? -A : A
+ At this point some readers may be thinking “Well, this may be as cool as you want, but it’s way too
complicated for me; I’ll just use the traditional if”. Of course, there’s nothing wrong with that, but in some
cases the ternary assignment operator can lead to substantially faster code, and it becomes surprisingly
natural when one gets used to it.
8.2 Loops
The basic hansl command for looping is (doh!) loop, and takes the form
In other words, the pair of statements loop and endloop enclose the statements to repeat. Of
course, loops can be nested. Several variants of the <control-expression> for a loop are sup-
ported, as follows:
1. unconditional loop
2. while loop
3. index loop
4. foreach loop
5. for loop.
Unconditional loop
This is the simplest variant. It takes the form
loop <times>
...
endloop
where <times> is any expression that evaluates to a scalar, namely the required number of itera-
tions. This is only evaluated at the beginning of the loop, so the number of iterations cannot be
changed from within the loop itself. Example:
Chapter 8. Control flow 31
# triangular numbers
scalar n = 6
scalar count = 1
scalar x = 0
loop n
scalar x += count
count++
print x
endloop
yields
x = 1.0000000
x = 3.0000000
x = 6.0000000
x = 10.000000
x = 15.000000
x = 21.000000
Note the usage of the increment (count++) and of the inflected assignment (x += count) operators.
Index loop
The unconditional loop is used quite rarely, as in most cases it is useful to have a counter variable
(count in the previous example). This is easily accomplished via the index loop, whose syntax is
loop <counter>=<min>..<max>
...
endloop
The limits <min> and <max> must evaluate to scalars; they are automatically turned into integers if
they have a fractional part. The <counter> variable is started at <min> and incremented by 1 on
each iteration until it equals <max>.
The counter is “read-only” inside the loop. You can access either its numerical value through the
scalar i or use the accessor $i, which will perform string substitution: inside the loop, the hansl
interpreter will substitute for the expression $i the string representation of the current value of
the index variable. An example should made this clearer: the following input
scalar a_1 = 57
scalar a_2 = 85
scalar a_3 = 13
loop i=1..3
print i a_$i
endloop
i = 1.0000000
a_1 = 57.000000
i = 2.0000000
a_2 = 85.000000
i = 3.0000000
a_3 = 13.000000
In the example above, at the first iteration the value of i is 1, so the interpreter expands the
expression a_$i to a_1, finds that a scalar by that name exists, and prints it. The same happens
through the rest of the iterations. If one of the automatically constructed identifiers had not been
defined, execution would have stopped with an error.
Chapter 8. Control flow 32
While loop
Here you have
where <condition> should evaluate to a scalar, which is re-evaluated at each iteration. Looping
stops as soon as <condition> becomes false (0). If <condition> becomes NA, an error is flagged
and execution stops. By default, while loops cannot exceed 100,000 iterations. This is intended as
a safeguard against potentially infinite loops. This setting can be overridden if necessary by setting
the loop_maxiter state variable to a different value.
Foreach loop
In this case the syntax is
where <catalogue> can be either a collection of space-separated strings, or a variable of type list
(see section 11.2). The counter variable automatically takes on the numerical values 1, 2, 3, and so
on as execution proceeds, but its string value (accessed by prepending a dollar sign) shadows the
names of the series in the list or the space-separated strings; this sort of loop is designed for string
substitution.
Here is an example in which the <catalogue> is a collection of names of functions that return a
scalar value when given a scalar argument.
scalar x = 1
loop foreach f sqrt exp ln
scalar y = $f(x)
print y
endloop
y = 1.0000000
y = 2.7182818
y = 0.0000000
For loop
The final form of loop control emulates the for statement in the C programming language. The
syntax is loop for, followed by three component expressions, separated by semicolons and sur-
rounded by parentheses, that is
1. Initialization (<init>): this must be an assignment statement, evaluated at the start of the
loop.
Chapter 8. Control flow 33
2. Continuation condition (<cont>): this is evaluated at the top of each iteration (including the
first). If the expression evaluates as true (non-zero), iteration continues, otherwise it stops.
3. Modifier (<modifier>): an expression which modifies the value of some variable. This is
evaluated prior to checking the continuation condition, on each iteration after the first.
Here’s an example, in which we find the square root of a number by successive approximations:
y = 128.5000000000, d = 16256.3
y = 65.2461089494, d = 4001.05
y = 34.5848572866, d = 940.112
y = 20.9934703720, d = 184.726
y = 16.5938690915, d = 19.3565
y = 16.0106268314, d = 0.340172
y = 16.0000035267, d = 0.000112855
y = 16.0000000000, d = 1.23919e-11
Number of iterations: 8
sqrt(256) = 16
Be aware of the limited precision of floating-point arithmetic. For example, the code snippet below
will iterate forever on most platforms because x will never equal exactly 0.01, even though it might
seem that it should.
However, if you replace the condition x!=0.01 with x>=0.01, the code will run as (probably) in-
tended.
Loop options
Two options can be given to the loop statement. One is --verbose. This has simply the effect of
printing extra output to trace progress of the loop; it has no other effect and the semantics of the
loop contents remain unchanged.
The --progressive option is mostly used as a quick and efficient way to set up simulation studies.
When this option is given, a few commands (notably print and store) are given a special, ad hoc
meaning. Please refer to the Gretl User’s Guide for more information.
an example in which we use the while variant of the loop statement to perform calculation of the
square root in a manner similar to the example above, using break to jump out of the loop when
the job is done.
scalar x = 256
scalar y = 1
loop while 1
d = y*y - x
if abs(d) < 1.0e-7
break
else
y -= d/(2*y)
printf "y = %15.10f, d = %g\n", y, d
endif
endloop
matrix a = floor(2*muniform(2,2))
catch ai = inv(a)
scalar err = $error
if err
printf "The matrix\n%6.0f\nis singular!\n", a
else
print ai
endif
Note that the catch keyword cannot be used before if, elif or endif. In addition, it should not be
used on calls to user-defined functions; it is intended for use only with gretl commands and calls to
“built-in” functions or operators. Suppose you’re writing a function package which includes some
subsidiary functionality which may fail under certain conditions, and you want to prevent such
failure from aborting execution. In that case you should use catch within the particular function
in question, and if an error condition is detected, signal this to the caller by returning a suitable
“invalid” value—say, NA (for a function that returns a scalar) or an empty matrix. For example:
else
printf "Calculated x = %g\n", x
endif
end function
User-written functions
Hansl natively provides a reasonably wide array of pre-defined functions for manipulating variables
of all kinds; the previous chapters contain several examples. However, it is also possible to extend
hansl’s native capabilities by defining additional functions.
Here’s what a user-defined function looks like:
The opening line of a function definition contains these elements, in strict order:
2. type, which states the type of value returned by the function, if any. This must be one of
void (if the function does not return anything), scalar, series, matrix, list, string or
bundle.
3. funcname, the unique identifier for the function. Function names have a maximum length
of 31 characters; they must start with a letter and can contain only letters, numerals and the
underscore character. They cannot coincide with the names of native commands or functions.
Type Description
bool scalar variable acting as a Boolean switch
int scalar variable acting as an integer
scalar scalar variable
series data series (see section 11.1)
list named list of series (see section 11.2)
matrix matrix or vector
string string variable or string literal
bundle all-purpose container
matrices array of matrices
bundles array of bundles
strings array of strings
1 If you have to pass many parameters, you might want to consider wrapping them into a bundle to avoid syntax
cluttering.
36
Chapter 9. User-written functions 37
Each element in the listing of parameter must include two terms: a type specifier, and the name by
which the parameter shall be known within the function.
The function body contains (almost) arbitrary hansl code, which should compute the return value,
that is the value the function is supposed to yield. Any variable declared inside the function is
local, so it will cease to exist when the function ends.
The return command is used to stop execution of the code inside the function and deliver its
result to the calling code. This typically happens at the end of the function body, but doesn’t have
to. The function definition must end with the expression end function, on a line of its own.
+ Beware: unlike some other languages (e.g. Matlab or GAUSS), you cannot directly return multiple outputs
from a function. However, you can return a bundle and stuff it with as many objects as you want.
In order to get a feel for how functions work in practice, here’s a simple example:
x−1
f (x) = 2 · ,
x+1
which provides a decent approximation to the natural logarithm in the neighborhood of 1. Some
comments on the code:
1. Since the function is meant to return a scalar, we put the keyword scalar after the intial
function.
2. In this case the parameter list has only one element: it is named x and is specified to be a
scalar.
3. On the next line the function definition begins; the body includes a comment and an if block.
5. The lines below the function definition give a simple example of usage. Note that in the
printf command, the two functions ln() and quasi_log() are indistinguishable from a
purely syntactic viewpoint, although the former is native and the latter is user-defined.
In ambitious uses of hansl you may end up writing several functions, some of which may be quite
long. In order to avoid cluttering your script with function definitions, hansl provides the include
command: you can put your function definitions in a separate file (or set of files) and read them in
as needed. For example, suppose you saved the definition of quasi_log() in a separate file called
quasilog_def.inp: the code above could then be written more compactly as
Chapter 9. User-written functions 38
include quasilog_def.inp
scalar x = 3
f(x)
print x
x = 6.0000000
x = 3.0000000
The first print statement is executed inside the function, and the displayed value is 6 because the
input x is doubled; however, what really gets doubled is simply a copy of the original x: this is
demonstrated by the second print statement. If you want a function to modify its arguments, you
must use pointers.
Pointers
Each of the type-specifiers, with the exception of list, may be modified by prepending an asterisk
to the associated parameter name, as in
This indicates that the required argument is not a plain matrix but rather a pointer-to-matrix, or in
other words the memory address at which the variable is stored.
This can seem a bit mysterious to people unfamiliar with the C programming language, so allow
us to explain how pointers work by analogy. Suppose you set up a barber shop. Ideally, your
customers would walk into your shop, sit on a chair and have their hair trimmed or their beard
shaved. However, local regulations forbid you to modify anything coming in through your shop
door. Of course, you wouldn’t do much business if people must leave your shop with their hair
untouched. Nevertheless, you have a simple way to get around this limitation: your customers can
come to your shop, tell you their home address and walk out. Then, nobody stops you from going
to their place and exercising your fine profession. You’re OK with the law, because no modification
of anything took place inside your shop.
While our imaginary restriction on the barber seems arbitrary, the analogous restriction in a pro-
gramming context is not: it prevents functions from having unpredictable side effects. (You might
be upset if it turned out that your person was modified after visiting the grocery store!)
Chapter 9. User-written functions 39
In hansl (unlike C) you don’t have to take any special care within the function to distinguish the
variable from its address,2 you just use the variable’s name. In order to supply the address of a
variable when you invoke the function, you use the ampersand (&) operator.
An example should make things clearer. The following code
scalar x = 0
scalar y = 1000000
swap(&x, &y)
print x y
x = 1000000.0
y = 0.0000000
2. For some parameter types, there is a special syntax construct for ensuring that its value is
bounded; for example, you can stipulate a scalar argument to be positive, or constrained
within a pre-specified range.
A thorough discussion is too long to fit in this document, and the interested reader should refer
to the “User-defined functions” chapter of the the Gretl User’s Guide. Here we’ll just show you a
simple, and hopefully self-explanatory, example which combines features 2 and 3. Suppose you
have a function for producing smileys, defined as
2 In C, this would be called dereferencing the pointer. The distinction is not required in hansl because there is no
loop times
printf "%s ", s
endloop
printf "\n"
end function
Then, running
smileys()
smileys(2, 1)
smileys(4)
produces
:-)
:-( :-(
:-) :-) :-) :-)
9.2 Recursion
Hansl functions can be recursive; what follows is the obligatory factorial example:
loop i=0..6
printf "%d! = %d\n", i, factorial(i)
endloop
Note: this is fun, but in practice, you’ll be much better off using the pre-cooked gamma function
(or, better still, its logarithm).
Part II
With a dataset
41
Chapter 10
What is a dataset?
A dataset is a memory area designed to hold the data you want to work on, if any. It may be
thought of a big global variable, containing a (possibly huge) matrix of data and a hefty collection
of metadata.
R users may think that a dataset is similar to what you get when you attach a data frame in R. Not
really: in hansl, you cannot have more than one dataset open at the same time. That’s why we talk
about the dataset.
When a dataset is present in memory (that is, “open”), a number of objects become available for
your hansl script in a transparent and convenient way. Of course, the data themselves: the columns
of the dataset matrix are called series, which will be described in section 11.1; sometimes, you will
want to organize one or more series in a list (section 11.2). Additionally, you have the possibility of
using, as read-only global variables, some scalars or matrices, such as the number of observations,
the number of variables, the nature of your dataset (cross-sectional, time series or panel), and so
on. These are called accessors, and will be discussed in section 10.5.
You can open a dataset by reading data from a disk file, via the open command, or by creating one
from scratch.
For more details see the Gretl User’s Guide, and the Gretl Command Reference for the nulldata
and setobs commands. The only important thing to say at this point, however, is that you can
resize your dataset and/or change some of its characteristics, such as its periodicity, at nearly any
point inside your script if necessary.
Once your dataset is in place, you can start populating it with series, either by reading them from
files or by generating them via appropriate commands and functions.
42
Chapter 10. What is a dataset? 43
The open command can also be used to read stuff off the Internet, by using a URL instead of a
filename, as in
open http://someserver.com/somedata.csv
The Gretl User’s Guide describes the requirements on plain text data files of the “CSV” type for
direct importation by gretl. It also describes gretl’s native data formats (XML-based and binary).
The append and join commands can be used to add further series from file to a previously opened
dataset. The join command is extremely flexible and has a chapter to itself in the Gretl User’s
Guide.
1. Selecting a contiguous subset of observations: this will be mostly useful with time-series
datasets. For example:
smpl 4 122 # select observations for 4 to 122
smpl 1984:1 2008:4 # the so-called "Great Moderation" period
smpl 2008-01-01 ; # form January 1st, 2008 onwards
2. Selecting observations on the basis of some criterion: this is typically what you want with
cross-sectional datasets. Example:
Note that, in this context, restrictions go “on top of” previous ones. In order to start from
scratch, you either reset the full sample via smpl full or use the --replace option along
with --restrict.
3. Restricting the active dataset to some observations so that a certain effect is achieved au-
tomatically: for example, drawing a random subsample, or ensuring that all rows that have
missing observations are automatically excluded. This is achieved via the --no-missing,
--contiguous, and --random options.
In the context of panel datasets, some extra qualifications have to be made; see the Gretl User’s
Guide.
1 Users with a Stata background may find the hansl way of doing things a little disconcerting at first. In hansl, you first
restrict your sample through the smpl command, which applies until further notice, then you do what you have to. There
is no equivalent to Stata’s if clause to commands.
Chapter 10. What is a dataset? 44
In addition there are a few more specialized accessors: $obsdate, $obsmajor, $obsminor, $obsmicro
and $unit. These are specific to time-series and/or panel data, and they all return series. See the
Gretl Command Reference for details.
Chapter 11
Scalars, matrices and strings can be used in a hansl script at any point; series and lists, on the other
hand, are inherently tied to a dataset and therefore can be used only when a dataset is currently
open.
• If lngdp belongs to a time series or panel dataset, then the syntax lngdp(-1) yields its first
lag, and lngdp(+1) its first lead.
The rules for assigning values to series are just the same as for other objects, so the following
examples should be self-explanatory:
+ In hansl, you don’t have separate commands for creating series and modifying them. Other popular
packages make this distinction, but we still struggle to understand why this is supposed to be useful.
45
Chapter 11. Series and lists 46
• To turn series into matrices, you use the curly braces syntax, as in
matrix MACRO = {outputgap, unemp, infl}
where you can also use lists; the number of rows of the resulting matrix will depend on your
currently selected sample.
• To turn matrices into series, you can just use matrix columns, as in
series y = my_matrix[,4]
But note that this will work only if the number of rows in my_matrix matches the length of
the dataset (or the currently selected sample range).
Also note that the lincomb and filter functions are quite useful for creating and manipulating
series in complex ways without having to convert the data to matrix form (which could be compu-
tationally costly with large datasets).
Here we assume that employed is a dummy series coding for employee status. Its value will be
tested for each observation in the current sample range and the value assigned to worker_income
at that observation will be determined accordingly. It is therefore equivalent to the following much
more verbose formulation (where $t1 and $t2 are accessors for the start and end of the sample
range):
series worker_income
loop i=$t1..$t2
if employed[i]
worker_income[i] = income[i]
else
worker_income[i] = 0
endif
endloop
list xlist = 1 2 3 4
list reglist = income price
list W = null
or simply by bare declaration. Some more special forms (for example, using wildcards) are de-
scribed in the Gretl User’s Guide.
The main idea is to use lists to group, under one identifier, one or more series that logically belong
together somehow (for example, as explanatory variables in a model). So, for example,
list xlist = x1 x2 x3 x4
ols y 0 xlist
is an idiomatic way of specifying the OLS regression that could also be written as
ols y 0 x1 x2 x3 x4
Note that we used here the convention, mentioned in section 11.1, by which a series can be identi-
fied by its ID number when used as an argument to a command, typing 0 instead of const.
Lists can be concatenated, as in as in list L3 = L1 L2 (where L1 and L2 are names of existing
lists). This will not necessarily do what you want, however, since the resulting list may contain
duplicates. It’s more common to use the following set operations:
Operator Meaning
|| Union
&& Intersection
- Set difference
So for example, if L1 and L2 are existing lists, after running the following code snippet
list UL = L1 || L2
list IL = L1 && L2
list DL = L1 - L2
the list UL will contain all the members of L1, plus any members of L2 that are not already in L1; IL
will contain all the elements that are present in both L1 and L2 and DL will contain all the elements
of L1 that are not present in L2.
To append or prepend variables to an existing list, we can make use of the fact that a named list
stands in for a “longhand” list. For example, assuming that a list xlist is already defined (possibly
as null), we can do
Another option for appending terms to, or dropping terms from, an existing list is to use += or -=,
respectively, as in
xlist += cpi
zlist -= cpi
A nice example of the above is provided by a common idiom: you may see in hansl scripts some-
thing like
Chapter 11. Series and lists 48
list C -= const
list C = const C
which ensures that the series const is included (exactly once) in the list C, and comes first.
the matrix A will contain the ID numbers of the four series as a row vector. This operation goes
both ways, so the statement
list C = seq(7,10)
is perfectly valid (provided, of course, that you have at least 10 series in the currently open dataset).
If instead you want to create a data matrix from the series which belong to a given list, you have to
enclose the list name in curly brackets, as in
matrix X = {L}
Estimation methods
You can, of course, estimate econometric models via hansl without having a dataset (in the sense
in which we’re using that term here) in place — just as you might in Matlab, for instance. You’ll
need data, but these can be loaded in matrix form (see the mread function in the Gretl Command
Reference), or generated artificially via functions such as mnormal or muniform. You can roll your
own estimator using hansl’s linear algebra primitives, and you also have access to more specialized
functions such as mols (see section 4.2) and mrls (restricted least squares) if you need them.
However, unless you need to use an estimation method which is not currently supported by gretl,
or have a strong desire to reinvent the wheel, you will probably want to make use of the built-
in estimation commands available in hansl. These commands are series-oriented and therefore
require a dataset. They fall into two main categories: “canned” procedures, and generic tools that
can be used to estimate a wide variety of models based on common principles.
• The user is presented with a fairly simple interface. A few inputs must be specified, and
perhaps a few options selected, then the heavy lifting is done within the gretl library. Full
results are printed (parameter estimates plus numerous auxiliary statistics).
where parameters usually takes the form of a listing of series: the dependent variable followed by
the regressors.
The line-up of procedures can be crudely categorized as follows:
Don’t let names deceive you: for example, the probit command can estimate ordered models,
random-effect panel probit models, . . . The hansl “house style” is to keep to a relatively small
number of command words and to distinguish variants within a class of estimators such as Probit
by means of options, or the character of the data supplied.
49
Chapter 12. Estimation methods 50
Simultaneous systems (SUR, FIML and so on) constitute the main exception to the syntax summary
above; these require a system block— see the chapter on Multivariate models in the Gretl User’s
Guide.
Each of these commands takes the form of a block of statements (e.g. nls . . . end nls). The user
must supply a function to compute the fitted dependent variable (nls), the log-likelihood (mle), or
the GMM residuals (gmm). With nls and mle, analytical derivatives of the function in question with
respect to the parameters may (optionally) be supplied.
The most widely used of these tools is probably mle. Hansl offers several canned ML estimators,
but if you come across a model that you want to estimate via maximum likelihood and it is not
supported natively, all you have to do is write down the log-likelihood in hansl’s notation and run
it through the mle apparatus.
--silent option. Consult the Gretl Command Reference to determine which commands accept this option.
Chapter 12. Estimation methods 51
diff y x
ADL <- ols y const y(-1) x(0 to -1)
ECM <- ols d_y const d_x y(-1) x(-1)
# the following two values should be equal
ssr_a = ADL.$ess
ssr_e = ECM.$ess
Part III
Reference
52
Chapter 13
Programming languages differ in their rules regarding the use of white space in a program. Here
we set out the rules in hansl. The rules differ somewhat between commands on the one hand and
function calls plus assignment on the other.
1. The individual elements just mentioned must always be separated by at least one space, and
where one space is required you are free to insert as many as you like.
2. Whenever a parameter is supplied with an option flag, the parameter must be attached to the
flag with an equals sign, with no intervening space:
ols y 0 x --cluster=clustvar # correct
ols y 0 x --cluster =clustvar # broken!
# set 1
y = sqrt(x)
y=sqrt(x)
# set 2
c = cov(y1, y2)
c=cov(y1,y2)
c = cov(y1 , y2)
1. When an assignment starts with a type keyword such as series or matrix, this must be
separated from what follows by at least one space, as in
2. In a function call, the opening parenthesis that marks the start of the argument list must be
attached to the name of the function without intervening space:
c = cov(y1, y2) # correct
c = cov (y1, y2) # broken!
53
Chapter 14
Operators
14.1 Precedence
Table 14.1 lists the operators available in gretl in order of decreasing precedence: the operators on
the first row have the highest precedence, those on the second row have the second highest, and so
on. Operators on any given row have equal precedence. Where successive operators have the same
precedence the order of evaluation is in general left to right. The exceptions are exponentiation
and matrix transpose-multiply. The expression a^b^c is equivalent to a^(b^c), not (a^b)^c, and
similarly A’B’C’ is equivalent to A’(B’(C’)).
() [] . {}
! ++ -- ^ ’
* / % \ **
+ - ~ |
> < >= <= ..
== !=
&&
||
?:
In addition to the basic forms shown in the Table, several operators also have a “dot form” (as in
“.+” which is read as “dot plus”). These are element-wise versions of the basic operators, for use
with matrices exclusively; they have the same precedence as their basic counterparts. The available
dot operators are as follows.
Each basic operator is shown once again in the following list along with a brief account of its
meaning. Apart from the first three sets of grouping symbols, all operators are binary except
where otherwise noted.
() Function call
[] Subscripting
. Bundle membership (see below)
{} Matrix definition
! Unary logical NOT
++ Increment (unary)
-- Decrement (unary)
^ Exponentiation
’ Matrix transpose (unary) or transpose-multiply (binary)
54
Chapter 14. Operators 55
* Multiplication
/ Division, matrix “right division”
% Modulus
\ Matrix “left division”
** Kronecker product
+ Addition
- Subtraction
~ Matrix horizontal concatenation
| Matrix vertical concatenation
> Boolean greater than
< Boolean less than
>= Greater than or equal
<= Less than or equal
.. Range from–to (in constructing lists)
== Boolean equality test
!= Boolean inequality test
&& Logical AND
|| Logical OR
?: Conditional expression
The interpretation of “.” as the bundle membership operator is confined to the case where it is
immediately preceded by the identifier for a bundle, and immediately followed by a valid identifier
(key).
Details on the use of the matrix-related operators (including the dot operators) can be found in the
chapter on matrices in the Gretl User’s Guide.
14.2 Assignment
The operators mentioned above are all intended for use on the right-hand side of an expression
which assigns a value to a variable (or which just computes and displays a value — see the eval
command). In addition we have the assignment operator itself, “=”. In effect this has the lowest
precedence of all: the entire right-hand side is evaluated before assignment takes place.
Besides plain “=” several “inflected” versions of assignment are available. These may be used only
when the left-hand side variable is already defined. The inflected assignment yields a value that is
a function of the prior value on the left and the computed value on the right. Such operators are
formed by prepending a regular operator symbol to the equals sign. For example,
y += x
The new value assigned to y by the statement above is the prior value of y plus x. The other
available inflected operators, which work in an exactly analogous fashion, are as follows.
-= *= /= %= ^= ~= |=
In addition, a special form of inflected assignment is provided for matrices. Say matrix M is 2 × 2. If
you execute M = 5 this has the effect of replacing M with a 1 × 1 matrix with single element 5. But
if you do M .= 5 this assigns the value 5 to all elements of M without changing its dimensions.
Chapter 14. Operators 56
However, they can also be embedded in more complex expressions, in which case they first yield the
original value of the variable in question, then have the side-effect of incrementing or decrementing
the variable’s value. For example:
scalar i = 3
k = i++
matrix M = zeros(10, 1)
M[i++] = 1
After the second line, k has the value 3 and i has value 4. The last line assigns the value 1 to
element 4 of matrix M and sets i = 5.
Warning: as in the C programming language, the unary increment or decrement operator should be
not be applied to a variable in conjunction with regular reference to the same variable in a single
statement. This is because the order of evaluation is not guaranteed, giving rise to ambiguity.
Consider the following:
This is supposed to assign the value of i to M[i], but is it the original or the incremented value?
This is not actually defined.
1 The C programming language also supports prefix versions of ++ and --, which increment or decrement their operand
before yielding its value. Only the postfix form is supported by gretl.
Chapter 15
Greek-letter identifiers
As mentioned in chapter 3, the sole exception to the requirement that hansl identifiers must be
plain ASCII is that they may take the form of a single Greek letter. Here are the details.
• This exception applies only to names of variables other than series; the names of series must
always be ASCII.
• The supported Greek characters are the 24 (unaccented) letters in the basic Greek alphabet,
minus omicron, which is indistinguishable from the Latin ‘o’.
• These letters may be used in lower or upper case (constituting distinct identifiers, as usual in
hansl), except for the several upper-case letters which are indistinguishable in the Latin and
Greek alphabets (‘A’, ‘B’, ‘E’, ‘K’, ‘M’, ‘N’, . . . ).
In gretl’s graphical interface (script editor and GUI “console”), the acceptable Greek letters can be
entered by typing a Latin letter while the Alt key is depressed. The mappings from Latin to Greek
are shown below: lower case first, then upper case.
57
Chapter 15. Greek-letter identifiers 58
Latin Greek
D ∆ Delta
F Φ Phi
G Γ Gamma
J Ψ Psi
L Λ Lambda
P Π Pi
Q Θ Theta
S Σ Sigma
U Υ Upsilon
W Ω Omega
X Ξ Xi
Y Υ Upsilon
A word of advice: it’s probably not a good idea to employ Greek-letter identifiers in hansl scripts
that you intend to share via the internet, since one cannot assume that text encodings are preserved
unchanged. This warning applies in particular if you, or any of the intended recipients of your
scripts, work on MS Windows, since Windows does not natively support UTF-8, the mandatory
encoding of such identifiers for use with gretl.