User Friendly Functional Programming
User Friendly Functional Programming
Section 3.2 shows the functional program that this editing ses-
sion created.
( LAM )
s, λ(x0 . . . xn ).e.x ⇓ (s, λ(x0 . . . xn ).e.x)
( APP )
s, e ⇓ (v 0 : s0 , λ(x0 . . . xn ).e00 .x)
s, e0 ⇓ v 00 (x0 7→ v0000 ) ∈ v 00 . . . (xn 7→ vn000 ) ∈ v 00 (v 0 n (x0 7→ v0000 )) n . . . n (xn 7→ vn000 ))) : s0 , e00 ⇓ v 0000 v 0000 : s, x ⇓ v
s, e(e0 ) ⇓ v
evaluation makes programs run slowly, the complex evaluation be- marked as such and arguments explicitly listed, the user interface
havior makes performance hard to predict, and programmers often attempts to hide this from users as much as possible.
have to battle with space leaks due to long chains of lazy thunks. Although MashMaker is dynamically typed, the Widget meta-
MashMaker, however, is not a conventional programming lan- data for a function includes a flag for each argument saying whether
guage. We believe that the unusual application domain that Mash- it is a closure. If an argument is a closure then the forms UI inter-
Maker works in makes lazy evaluation highly appropriate. In par- prets text entered for that argument a little differently than for non-
ticular: closure arguments. Any text entered is assumed to be the body of a
lambda expression, and any variables in the expression that are not
• In the case of web mashups, the bookkeeping cost of remember- bound at the current location are assumed to be lambda arguments.
ing how to evaluate something is tiny compared to the massive Advanced users can tell that this alternative text handling is in use
cost of fetching and scraping a web site, thus it is only neces- by noticing a λ icon next to the argument text box.
sary for a very small number of expressions to be unneeded for
the bookkeeping cost to be more than paid back. 3.6 Live Data
• Even if fetching a web site was cheap, it is important for us to All data in MashMaker is live, meaning that it may change over
minimize the number of queries we make to a remote server, to time and will react to changes in other parts of the data tree. If a
avoid overwhelming a server (Section 3.7). tree is the result of a web query, then this tree will update over
• Typical mashup programs work with relatively small amounts time, as the source web site changes 9 .
of data that are not directly presented to the user, and so space The MashMaker language is designed to handle changing data
leaks are far less of a problem. well. In particular, since MashMaker overlays extensions over gen-
erated data (Section 3.2), rather than modifying it in-place, these
• Many web sites are already essentially lazy. For example when extensions will be automatically applied to new versions of the
one makes a search using Google, it does not return all results underlying data. Also, since the MashMaker extension construct
in one page, but instead produces results lazily as one presses automatically adds new properties to all children of a node, these
the “next” buttons. properties will also apply to any new children that are added to the
tree.
MashMaker’s lazy evaluation works largely as one would ex-
pect. The value of a node is only evaluated when it is either de- 3.7 Throttling
manded by the evaluation of another node, or the user attempts to
One important practical issue that has to be dealt with whenever
view it through the graphical interface.
one creates a mashup is the need to avoid placing too much load on
the web sites supplying data. If one has an agreement with the web
3.5 Lambda Expressions in the User Interface site provider then it is likely to specify a maximum load, and if one
The MashMaker user interface has somewhat unusual treatment
of lambda expressions. While the syntax in the underlying core 9 Either by polling the web site at a fixed frequency, or waiting until the user
language is fairly conventional, with lambda expressions explicitly asks for a refresh.
does not have an explicit agreement then placing too much load on Abstraction gra- What are the minimum and maximum lev-
a server could cause the owners to block the mashup system’s IP dient els of abstraction? Can fragments be encap-
address. sulated?
As a result of this, it is necessary for MashMaker to throttle the Consistency When some of the language has been learnt,
rate at which requests can be made to external web sites. Indeed this how much of the rest can be inferred?
rate is one of the primary issues that determines the performance of Error-proneness Does the design of the notation induce
a Mashup, since if a mashup needs to make too many requests, then ‘careless mistakes’?
it will have to slow itself down in order to avoid sending requests Hidden depen- Is every dependency overtly indicated in
too rapidly. This performance restriction has motivated MashMaker dencies both directions? Is the indication perceptual
not only to use lazy evaluation (Section 3.4), but also to use a or only symbolic?
number of other tricks (not discussed in this paper) to minimize the Premature com- Do programmers have to make decisions
number of requests that need to be made to external web servers. mitment before they have the information they need?
Progressive Can a partially-complete program be exe-
3.8 When Websites turn Bad evaluation cuted to obtain feedback on “how am I do-
One limitation of MashMaker, as with most other tools that scrape ing”?
information from web sites, is that mashups can break if the under- Role- Can the reader see how each component of
lying websites change. If a website changes the structure of the data expressiveness a program relates to the whole?
it produces, or changes its HTML such that the current scraper no Viscosity How much effort is required to perform a
longer understands it, then mashups that depend on this data will single change?
no longer function correctly. In the long term, we hope this this Visibility and Is every part of the code simultaneously vis-
problem will become less severe as websites increasingly publish juxtaposability ible (assuming a large enough display), or
semantic information in well defined data formats. is it at least possible to compare any two
More generally, MashMaker is not intended to be used for parts side-by-side at will? If the code is dis-
“mission critical” applications where data integrity is essential. persed, is it at least possible to know in what
Instead, its focus is on applications where it is more important to order to read it?
be able to produce interesting data than to be certain that the data
is correct. Figure 8. Cognitive Dimensions (taken from Peyton Jones et al.
2003)
4. Evaluating Usability
Following Peyton Jones et al. (2003), we evaluate the usability of
MashMaker using the Cognitive Dimensions of Notations (CDs) then this does not affect the behavior of non-dependent parts of
framework (Blackwell et al. 2001). CDs provide a vocabulary that the program.
enumerates concepts important to users who are engaged in pro- • Viscosity: MashMaker’s support for user-defined functions, au-
gramming tasks. While evaluation against cognitive dimensions is tomatic synchronization of property definitions across multiple
subjective, and is not a substitute for thorough user testing, these children, and its general preference for linking of data rather
concepts have been shown over time to be important to human than copying, make it easy to make widespread changes.
problem solving and it is important to consider each when design- • Visibility and Juxtaposability: Unlike conventional program-
ing a usable interface. We list the cognitive dimensions in Figure 8 ming languages, MashMaker juxtaposes program and data to-
and evaluate MashMaker against these dimensions below: gether, so the programmer can easily see the effects of evaluat-
• Abstraction Gradient: MashMaker can be used at a number ing their expressions. While MashMaker does not allow one to
of different levels of abstraction, allowing use by users rang- view multiple forms or multiple view panels in the same win-
ing from complete beginner to experienced programmer (Sec- dow, MashMaker does allow one to view the same data store
tion 2.5). with multiple browser windows, allowing one to put arbitrary
information side-by-side.
• Consistency: New widgets are created using the same mecha-
nism as creating simple expressions. All work in MashMaker Based on this analysis, and also our personal experiences using
is done using the same simple mechanism of applying widgets MashMaker, we believe that our design is fundamentally sound.
and setting form parameters. However, in order to demonstrate this objectively, we need to per-
• Error Proneness: Unlike normal spreadsheets, MashMaker
form a proper user study, and indeed we intend to do this in the near
future.
automatically ensures that when a user adds a property to a set
of children the defining expression is identical for all children.
5. Related Work
• Hidden Dependencies and Role Expressiveness: When a
node is selected, all dependent or source nodes are automat- In this section, we explain how MashMaker relates to previous
ically highlighted to make it clear that there is a dependency. work on Mashup creation and end-user programming in general.
In prior work, we described the basic MashMaker tool from a
• Premature Commitment: Users do not have to decide in ad- database angle (Ennals and Garofalakis 2007).
vance what they are looking for, but instead can wander aim-
lessly, looking for something useful. If they find something they 5.1 Mashup Creation Tools
like, they can bookmark it as a new function widget, but they Mashups are an increasingly hot topic, and thus there have been
need not decide in advance that this is what they are going to many efforts to simplify their creation. Relative to MashMaker,
do. these previous tools generally fall into two groups: those which
• Progressive Evaluation: There is no requirement that a pro- are easy to use, but can only create a limited family of mashups;
gram be in any sense “complete” in order for the user to look at and those which are relatively difficult to use, but can create a wide
its result. Similarly, like a spreadsheet, if some evaluations fail range of mashups.
Google MyMaps 10 and MapCruncher 11 make it easy for end TreeSheet (Leonard 2004) represents data as an XML tree, but
users to create mashups involving maps. Swivel.com makes it very unlike MashMaker, programming is done using imperative scripts,
easy for end users to create graph mashups from multiple data rather than functional overlays.
tables. However, while each of these tools is easy to use, and Within the database community, SEMEX (Cai et al. 2005) and
excellent at producing mashups of a specific type, none of them DataSpaces (Franklin et al. 2005) have looked at data-integration –
is as general purpose as MashMaker. how to get transform various data sources into a suitable structure
Yahoo Pipes 12 is a powerful tool that allows users to process so that they can be combined with queries.
data from RSS feeds. While, at the time of writing, the small
set of operations available in Pipes makes it less flexible than 5.2 End-User Programming Tools
MashMaker, it seems likely that the tool will be extended to give it MashMaker also bears some similarity to a number of end-user pro-
equivalent expressive power. The key difference between Pipes and gramming tools that have not been used for creating web mashups:
MashMaker is that, unlike MashMaker, pipes presents the program Programmable Structured Documents (PSDs) (Takeichi et al.
as an explicit graphical dataflow graph, rather than mixing it with 2003; Hu et al. 2004; Liu et al. 2005) allow one to extend a stan-
the data being browsed. dard XML document by embedding elements in the tree that are
Marmite (Wong and Hong 2006) takes a pipeline-based ap- computed from other elements. An expression defining an XML
proach, similar to Apple’s Automator 13 . The data from a web site node can refer to other nodes using XPath expressions and then pro-
is routed through a sequence of pipeline stages, each of which is cess the nodes using arbitrary Haskell functions. Like MashMaker,
configurable, and can produce data of a different type. Like Yahoo PSDs are based on a functional language (in this case Haskell) and
Pipes, the program is separated from the data and presented as a are evaluated lazily. Unlike MashMaker PSDs deal with static XML
graph, rather than being embedded in the data like a spreadsheet. documents, rather than live data. Indeed, since PSDs include ex-
Like MashMaker, Marmite will automatically suggest operations to pressions directly in a document, rather that overlaying changes on
apply to data. Unlike MashMaker, these suggestions are based on top of generated data in a way that can be automatically re-applied,
the type of the data (similar to Jungloids (Mandelin et al. 2005)), they could not be used to add additional properties to live data with-
rather than based on the behavior of previous users. Anthracite 14 out changes to the model (Section 3.6).
is similar to Marmite, but requires that the user be familiar with Subtext (Edwards 2005) is a programming tool that allows one
complex concepts such as HTML and regular expressions. to look at a program together with the results of its evaluation. In
Creo (Faaborg and Lieberman 2006) augments web pages with Subtext, every node in the data tree corresponds to the execution of
additional links that can obtain additional information about items a single program line with specific data and is annotated with the
on a web page. Like MashMaker, Creo will automatically make value produced. Function calls are expanded as subtrees and func-
suggestions and can learn by example from things that users do tion definitions contain example arguments that the programmer
with their data. Unlike MashMaker, Creo is limited to adding addi- can adjust to interactively see how their program will behave. Like
tional hyperlinks to web pages and cannot perform bulk data pro- MashMaker, subtext allows programmers to easily see how their
cessing tasks. program will behave when applied to particular arguments. Unlike
Google Mashup Editor, Plagger.org, Ning.com, Javascript Dataflow MashMaker, the Subtext interface is program-centric, rather than
Architecture (Lim and Lucas 2006), and Web Mashup Scripting data-centric — meaning that data is layered on top of a program,
Language (Sabbouh et al. 2007) are powerful tools for creating rather than overlaying a program on top of data.
mashups, but they require that the user write code. MashMaker’s function creation system is influenced by the
ClearSpring.com, Widsets.com, WidgetBox.com, and Apple’s work of Peyton Jones et al. (2003) in extending Microsoft Excel to
Dashboard 15 allow users to write small graphical web widgets support user-defined functions. Like MashMaker, they allow one
and then lay them out together on a screen. DataMashups.com to define a new function by selecting a result cell and then using a
additionally allows users to connect these widgets together (e.g. graphical interface to specify which other cells are arguments.
the output of this widget is the input to that widget), but complex MashMaker’s suggestion system is influenced by Jungloids (Man-
tasks require considerable programmer skill. delin et al. 2005) and Google Suggest 16 . Like Jungloids, Mash-
HunterGatherer (Schraefel et al. 2002) and Internet Scrap- Maker suggests operations that are appropriate to the data one has
book (Sugiura and Koseki 1998) allow users to extract parts of at hand. Like Google Suggest, MashMaker learns from the behav-
multiple web sites and composite them together, but are not able to ior of other users.
perform complex processing on these sites and extract collections More generally, MashMaker draws on past work on Program-
of data. ming by Example (Cypher et al. 1993; Lieberman 2001), and pre-
Like MashMaker, C3W (Fujima et al. 2004) uses a spreadsheet vious work on programming approaches for beginners (Kelleher
metaphor. C3W uses a standard flat two-dimensional spreadsheet and Pausch 2005).
to connect web sites together. If a user defines values for a web
site’s input cells, then it will produce results in its output cells, 6. Conclusions
clipped from the web page. Unlike MashMaker, C3W uses a flat
two dimensional grid, rather than a tree. This prevents one writing We have presented MashMaker, a tool that allows end-users to
mashups that produce nested data, such as producing a list of easily create web mashups. While MashMaker is, at its core, a
restaurants for each of several apartments. functional language, it contains a number of deviations from the
standard functional paradigm. By taking ideas from such popular
tools as file systems, spreadsheets, and web browsers, we have
produced a tool that we believe is well suited to the task of mashup
10 http://maps.google.com creation.
11 http://research.microsoft.com/mapcruncher This project is interesting as a programming language research
12 http://pipes.yahoo.com project both because it approaches an application domain for which
13 http://www.apple.com/ programming languages have not historically been seen as the solu-
14 http://www.metafly.com/products/anthracite
tion, and also because, in the process of fitting our language to this
15 http://www.apple.com/ 16 http://labs.google.com/suggest/
domain, we have produced a language that has many differences Caitlin Kelleher and Randy Pausch. Lowering the barriers to programming:
from previous functional languages. A taxonomy of programming environments and languages for novice
We have implemented MashMaker as an AJAX web applica- programmers. ACM Comput. Surv., 37(2):83–137, 2005. ISSN 0360-
tion, currently made available within our organization as part of 0300. doi: http://doi.acm.org/10.1145/1089733.1089734.
a closed beta program. We plan to make it publically available in Thomas Leonard. Tree-Sheets and Structured Documents. PhD thesis,
the near future. In the long term, the success of MashMaker will University of Southampton, 2004.
be judged based on the extent to which real users adopt it, and the Henry Lieberman. Your wish is my command: programming by example.
scale of the benefit they are able to obtain from it. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2001.
For more information on MashMaker, and access to the public ISBN 1-55860-688-2.
beta when it opens, please go to the following url: Seung Chan Slim Lim and Peter Lucas. Jda: a step towards large-scale
http://berkeley.intel-research.net/rennals/mashmaker/ reuse on the web. In OOPSLA ’06: Companion to the 21st ACM SIG-
mashmaker.html PLAN conference on Object-oriented programming systems, languages,
and applications, pages 586–601, New York, NY, USA, 2006. ACM
Press. ISBN 1-59593-491-X. doi: http://doi.acm.org/10.1145/1176617.
Acknowledgments 1176631.
This work has benefited from the input of many people. Particular Dongxi Liu, Zhenjiang Hu, and Masato Takeichi. An environment for main-
taining computation dependency in XML documents. In DocEng ’05:
thanks should go to Minos Garofalakis, Eric Paulos, and Ian Smith. Proceedings of the 2005 ACM symposium on Document engineering,
pages 42–51, New York, NY, USA, 2005. ACM Press. ISBN 1-59593-
240-2. doi: http://doi.acm.org/10.1145/1096601.1096616.
References David Mandelin, Lin Xu, Rastislav Bodik, and Doug Kimelman. Jungloid
Alan F. Blackwell, Carol Britton, Anna Louise Cox, Thomas R. G. Green, mining: helping to navigate the API jungle. In PLDI ’05: Proceedings of
Corin A. Gurr, Gada F. Kadoda, Maria Kutar, Martin Loomes, Chrysto- the 2005 ACM SIGPLAN conference on Programming language design
pher L. Nehaniv, Marian Petre, Chris Roast, Chris Roe, Allan Wong, and implementation, pages 48–61, New York, NY, USA, 2005. ACM
and Richard M. Young. Cognitive dimensions of notations: Design tools Press. ISBN 1-59593-056-6. doi: http://doi.acm.org/10.1145/1065010.
for cognitive technology. In CT ’01: Proceedings of the 4th Interna- 1065018.
tional Conference on Cognitive Technology, pages 325–341, London,
Simon Peyton Jones. Wearing the hair shirt: a retrospective on Haskell
UK, 2001. Springer-Verlag. ISBN 3-540-42406-7.
(invited talk). In ACM SIGPLAN Conferenge on Principles of Program-
Yuhan Cai, Xin Luna Dong, Alon Halevy, Jing Michelle Liu, and Jayant ming Languages (POPL’03), 2003a.
Madhavan. Personal information management with SEMEX. In SIG-
Simon Peyton Jones, editor. Haskell 98 Language and Libraries: the
MOD ’05: Proceedings of the 2005 ACM SIGMOD international con-
Revised Report. Cambridge University Press, may 2003b.
ference on Management of data, pages 921–923, New York, NY, USA,
2005. ACM Press. ISBN 1-59593-060-4. doi: http://doi.acm.org/10. Simon Peyton Jones, Alan Blackwell, and Margaret Burnett. A user-centred
1145/1066157.1066289. approach to functions in Excel. In ICFP ’03: Proceedings of the eighth
ACM SIGPLAN international conference on Functional programming,
Allen Cypher, Daniel C. Halbert, David Kurlander, Henry Lieberman,
pages 165–176, New York, NY, USA, 2003. ACM Press. ISBN 1-58113-
David Maulsby, Brad A. Myers, and Alan Turransky, editors. Watch
756-7. doi: http://doi.acm.org/10.1145/944705.944721.
what I do: programming by demonstration. MIT Press, Cambridge, MA,
USA, 1993. ISBN 0-262-03213-9. Marwan Sabbouh, Jeff Higginson, Danny Gagne, and Salim Semy. Web
mashup scripting language (poster). In 16th International World Wide
Jonathan Edwards. Subtext: uncovering the simplicity of programming. In
Web Conference, 2007.
OOPSLA ’05: Proceedings of the 20th annual ACM SIGPLAN confer-
ence on Object oriented programming, systems, languages, and applica- M. C. Schraefel, Daniel Wigdor, Yuxiang Zhu, and David Modjeska. Hunter
tions, pages 505–518, New York, NY, USA, 2005. ACM Press. ISBN gatherer: within-web-page collection making. In CHI ’02: CHI ’02
1-59593-031-0. doi: http://doi.acm.org/10.1145/1094811.1094851. extended abstracts on Human factors in computing systems, pages 826–
827, New York, NY, USA, 2002. ACM Press. ISBN 1-58113-454-1. doi:
Robert Ennals and Minos Garofalakis. Mashmaker : Mashups for the
http://doi.acm.org/10.1145/506443.506617.
masses (demo paper). In Proceedings of the 2007 ACM SIGMOD
International Conference on Management of Data (SIGMOD’2007), Michael Schrenk. Webbots, Spiders, and Screen Scrapers. No Starch Press,
2007. 2007.
Alexander Faaborg and Henry Lieberman. A goal-oriented web browser. In Atsushi Sugiura and Yoshiyuki Koseki. Internet scrapbook: automating web
CHI ’06: Proceedings of the SIGCHI conference on Human Factors in browsing tasks by demonstration. In UIST ’98: Proceedings of the 11th
computing systems, pages 751–760, New York, NY, USA, 2006. ACM annual ACM symposium on User interface software and technology,
Press. ISBN 1-59593-372-7. doi: http://doi.acm.org/10.1145/1124772. pages 9–18, New York, NY, USA, 1998. ACM Press. ISBN 1-58113-
1124883. 034-1. doi: http://doi.acm.org/10.1145/288392.288395.
Michael Franklin, Alan Halevy, and David Maier. From databases to datas- Masato Takeichi, Zhenjiang Hu, Kazuhiko Kakehi, Yashushi Hayashi, Shin-
paces: A new abstraction for information management. In SIGMOD Cheng Mu, and Keisuke Nakano. TreeCalc: towards programmable
Record, 2005. structured documents. In Japan Society for Software Science and Tech-
nology, 2003.
Jun Fujima, Aran Lunzer, Kasper Hornbæk, and Yuzuru Tanaka. Clip, con-
nect, clone: combining application elements to build custom interfaces Jeffrey Wong and Jason Hong. Marmite: end-user programming for the
for information access. In UIST ’04: Proceedings of the 17th annual web. In CHI ’06: CHI ’06 extended abstracts on Human factors in
ACM symposium on User interface software and technology, pages 175– computing systems, pages 1541–1546, New York, NY, USA, 2006. ACM
184, New York, NY, USA, 2004. ACM Press. ISBN 1-58113-957-8. doi: Press. ISBN 1-59593-298-4. doi: http://doi.acm.org/10.1145/1125451.
http://doi.acm.org/10.1145/1029632.1029664. 1125733.
Zhenjiang Hu, Shin-Cheng Mu, and Masato Takeichi. A programmable
editor for developing structured documents based on bidirectional trans-
formations. In PEPM ’04: Proceedings of the 2004 ACM SIGPLAN
symposium on Partial evaluation and semantics-based program manip-
ulation, pages 178–189, New York, NY, USA, 2004. ACM Press. ISBN
1-58113-835-0. doi: http://doi.acm.org/10.1145/1014007.1014025.