WebDSL is a domain-specific language for modeling web applications with a rich data model.
see WebDSL in Eclipse.
The plugin includes a new project wizard which will help you get started using WebDSL:
clean-project.xml
to clean the project’s generated files before committing to version controlbin/catalina.sh run
(Mac/Linux) or bin/catalina.bat run
(Windows).Use the ‘Convert to a WebDSL Project’ wizard to regenerate the project build files (this will overwrite the old files, including application.ini).
If you encounter issues when running the plugin, here are a few things that you should check or try:
http://yellowgrass.org/project/WebDSL
. You can also subscribe to the mailing https://mailman.st.ewi.tudelft.nl/listinfo/webdsl
and report your issue, or go to the #webdsl channel on irc.freenode.netWebDSL can be invoked from the command-line by using the compiler supplied with the plugin (http://webdsl.org/selectpage/Download/WebDSLplugin, recommended) or downloading the stand-alone compiler: http://webdsl.org/selectpage/Download/WebDSLJava.
Mac/Linux users can start WebDSL using the webdsl
script at:
eclipse/plugins/webdsl.editor_[version]/webdsl-template/webdsl
and Windows users can start WebDSL using the webdsl.bat
script at:
eclipse/plugins/webdsl.editor_[version]/webdsl-template/webdsl.bat
Where [version] is your installed version of the plugin. For convenience, you can add the directory to your path or make an alias for the script.
The quickest way to get an application running is to execute:
webdsl run appname
This will generate an application.ini file with default settings, then compile the application, and start a Tomcat instance on port 8080 with the application deployed.
If there is already an application.ini file with settings that have to be used, execute:
webdsl run
This will also build and run, using the settings in the existing application.ini file.
To create just the war file instead, use:
webdsl war
The installation of WebDSL will result in a webdsl script and a directory with templates being added to your install location. The script is used to invoke the compilation and deployment of WebDSL applications.
In your console, go to the location of the main .app file and invoke the webdsl script with
webdsl build
The script uses an application.ini file for configuration. If an application.ini file is not in the current directory, the script will offer an interactive way to generate it. If the application.ini is available it will be used to configure the application with e.g. database connection settings. The compilation begins by creating a .servletapp directory to which the WebDSL template, the application files, and the static resources are copied. Then the actual WebDSL compiler, webdslc, is invoked. This will either produce an error and halt, or it will produce the source code of a java web application. Upon a successful run of the webdsl compiler, the script will compile the java code, and build a war file. This war file can be copied manually to the tomcat /webapps dir, or it can be uploaded through the web deploy interface of tomcat. If the tomcat path is set in application.ini, then
webdsl deploy
will copy the war file to the /webapps directory.
If you have updated webdsl and need to copy the new WebDSL template in .servletapp use
webdsl cleanall
to remove the .servletapp directory (or simply delete it with rm) and then do a build.
The script commands can be combined, e.g.
webdsl cleanall build deploy
to clean the generated directory and its contents, regenerate, and deploy.
1 create a hello.app file
hello.app:
application test
define page root(){
"Hello world"
}
create or generate application.ini:
backend=servlet
tomcatpath=**path to your tomcat directory e.g. /Apps/tomcat/**
appname=hello
dbserver=localhost
dbuser=**mysql user account, e.g. root**
dbpassword=**password**
dbname=webdsldb
dbmode=create-drop
smtphost=localhost
smtpport=25
smtpuser=
smtppass=
2 create the database
mysql -u root -p
create database webdsldb;
exit
3 start tomcat in another shell:
catalina.sh run (stop with cmd/ctrl+c)
or in the background
catalina.sh start (stop with catalina.sh stop)
4 compile and deploy WebDSL app
webdsl cleanall deploy
5 open browser and go to http://localhost:8080/hello
WebDSL application are organized in _*.app_ files. Each .app file has a header, that either declares the name of a module or the name of an application. The declared name should be identical to the filename. Each application needs an .app file that declares the name of the application. This is the name refered to in the application.ini file (see below).
An application can be organized in different modules. In a typical .app file the header is followed by a list of import statements, which contain a path to other modules (without extension). In this way your application can be separated over several files, and modules can be reused.
Within .app files one can define sections. A section is merely a label to identify the structure of a file. Most section names have no influence on the program itself, some have however, for example in styling definitions.
The real contents of a .app file are a list of definitions. This might be page-, template-, action- or entity definitions. Other kinds of definitions might be introduced by WebDSL modules. A module might either refer to a module of an WebDSL application, or to an module of the WebDSL compiler itself. In this case the latter one is refered to. Those definitions will be examined in detail in the next chapters.
A very simple application might look like:
HelloWorld.app:
application HelloWorld
imports MyFirstImport
section pages
define page root () {
"hello world"
IAmImported()
}
MyFirstImport.app:
module MyFirstImport
define IAmImported() {
spacer
"I am imported from a module file"
}
In the second file the section declaration is omitted, since an application may start with a list of declarations as well. The page that is shown when no page is specified (e.g. when visiting http://localhost:8080/yourapp) is named “root” and has no arguments.
In the application.ini file compile-, database- and deployment information is stored. Executing the webdsl command in a certain directory will look for a application.ini file to obtain compilation information. If no such file was found, it will start a simple wizard to create one.
Example application.ini:
backend=servlet
tomcatpath=/opt/tomcat
appname=hello
dbserver=localhost
dbuser=webdsluser
dbpassword=webdslpassword
dbname=webdsldb
dbmode=update
smtphost=localhost
smtpport=25
smtpuser=
smtppass=
backend The back-end target platform of the application. Currently, the servlet back-end is only up-to-date.
appname The name of the application to compile. The compiler will look for a APPNAME.app file to compile. This name will also become the servlet name and show up as part of the URL. By renaming the generated APPNAME.war file to ROOT.war and then deploying it, the application name will not be in the URL.
tomcatpath This field should contain the root directory of the Tomcat installation. For example /opt/tomcat. It is used when executing ‘webdsl deploy’.
dbmode This field indicates if the application should try to create tables in a database, or try to sync it with the existing schema to avoid loss of data. Valid values are create-drop, update, and false. Update can lead to unpredictable results if data model is changed too much, if data needs to be properly migrated, use Acoda instead. For production deployment use ‘export DBMODE=false’.
dbserver Location of the Mysql server, which will be used in the connection URL, e.g. ‘localhost’.
dbuser User to be used for connecting to the MySQL database.
dbpassword Password for the specified user.
dbname Database name, note that the database needs to exist when the application is run. The ‘webdsl’ script will try to create the database in the wizard, but manually creating it via command-line or MySQL Administrator is also possible.
db Set db=h2
to enable H2 Database Engine instead of the default MySQL.
dbfile H2 database file, an empty file will be populated with tables automatically, when using ‘create-drop’ or ‘update’ db modes.
dbmode Same as for MySQL.
db Set db=h2mem
to enable in-memory H2 Database Engine instead of the default MySQL.
dbmode Same as for MySQL, although effectively the tables are always dropped after a restart with in-memory database
db Set db=jndi
to retrieve a JDBC resource from the application server, rather than providing the configuration in the web application.
dbjndipath JNDI path to the JDBC resource. On Apache Tomcat this is typically prefixed by ‘java:comp/env’. An example may be: ‘java:comp/env/jdbc/mydatabase’
dbmode Same as for MySQL.
Apart from settings in the application.ini, also a Context XML file must be provided for Apache Tomcat. An example may be:
<Context>
<Resource name="jdbc/mydatabase"
auth="Container"
type="javax.sql.DataSource"
driverClassName="com.mysql.jdbc.Driver"
maxActivate="100" maxIdle="30" maxWait="10000"
username="root" password="dbpassword"
url="jdbc:mysql://localhost:3306/mydatabase?useServerPrepStmts=false&characterEncoding=UTF-8&useUnicode=true&autoReconnect=true" />
</Context>
This XML file must be stored in: $TOMCAT_BASE/conf/Catalina/localhost/<appname>.xml
smtphost SMTP host for sending email, e.g. smtp.gmail.com
smtpport SMTP port for sending email, e.g. 465
smtpuser SMTP username
smtppass SMTP password
smtpprotocol smtpprotocol=smtps
[smtp/smtps] Use smtp or smtps as protocol.
smtpauthenticate smtpauthenticate=true
[true/false] Authenticate with a username and password.
indexdir set the index directory, default is /var/indexes.
searchstats Enable/disable search statistics, which can be displayed using template showSearchStats(). Default is false.
rootapp rootapp=true
will deploy the application as root application, it will not have the application name prefix in the URL.
wikitext-hardwraps wikitext-hardwraps=true
will enable so-called hard wraps in markdown. This way, each newline which isn’t followed by 2 white spaces is also rendered as new line. Default is false. See http://yellowgrass.org/issue/WebDSL/818
appurlforrenderwithoutrequest (as of WebDSL 1.3.0) Sets the URL to be used when links to pages are to be rendered outside a request. Normally, WebDSL will construct links using the request URL as a base. In case pages or templates with links are to be rendered outside a request (e.g. using a background task), WebDSL will use this property value as the base url.
sessiontimeout Sets the session timeout, specified in minutes.
javacmem javacmem=3G
set javac max memory for compilation of generated Java classes
debug debug=true
will show queries and Java exception stacktraces in the log.
verbose verbose=2
will show more info during compilation, mainly for developers.
fastpp fastpp=true
will make the compiler write Java code faster (writing files stage), however, it also becomes less readable. (only for C-based back-end of the WebDSL compiler)
For the webdsl tomcatdeploy
and webdsl tomcatundeploy
commands to work, a user has to be configured in Tomcat (tomcat/conf/tomcat-users.xml). For example:
<tomcat-users>
<role rolename="manager"/>
<user username="tomcat" password="tomcat" roles="manager"/>
</tomcat-users>
The tomcat manager URL and username and password can be set in the application.ini file (defaults are listed as examples):
tomcatmanager tomcatmanager=http:\\localhost:8080\manager
URL to Tomcat manager
tomcatuser tomcatuser=tomcat
manager user declared in tomcat/conf/tomcat-users.xml
tomcatpassword tomcatpassword=tomcat
password for that user
We use the following settings for Tomcat on our production server (NixOS/Linux):
-Xms350m
-Xss8m
-Xmx8G
-Djava.security.egd=file:/dev/./urandom
-XX:MaxPermSize=512M
-XX:PermSize=512M
-XX:-UseGCOverheadLimit
-XX:+UseCompressedOops
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/var/tomcat/logs/heapdump.hprof
-Xmx8G
The most important setting, maximum heap space, value depends on size/number of applications, but the default setting is usually too low. If this setting is too high for your JVM, it won’t start at all.
-XX:MaxPermSize=512M
This allows redeploying the application without running into permgenspace errors too quickly.
-Djava.security.egd=file:/dev/./urandom
The default implementation for random can be too slow (java.util.UUID.randomUUID is used for entity identifiers, including RequestLogEntry) see http://stackoverflow.com/questions/137212/how-to-solve-performance-problem-with-java-securerandom
Use jvisualvm
to inspect the Tomcat process, this allows you to look at the heap and running threads and create dumps for later inspection. A heap dump can also be created using:
jmap -F -dump:format=b,file=<filename> <process id>
The Eclipse Memory Analyzer can be used to inspect this file, get it from http://www.eclipse.org/mat/
If the tomcat process becomes unresponsive try
kill -3 <process id>
to generate a thread dump in the catalina.out log.
See MySQL section.
See Lightweight VPS section, also contains step-by-step installation.
WebDSL applications can be deployed on a light-weight server or VPS. Whether performance is acceptable depends on many factors such as complexity of the application, number of users, capacity of the server. Currently, the ram usage is usually the limiting factor. The JVM halts or gets stuck when the max heap space limit is crossed (-Xmx setting). 512mb ram, typically the lowest VPS option, can run a simple WebDSL application, but getting 1gb or 2gb is recommended.
In the rest of this section is a walkthrough of the minimal steps required for installation of a WebDSL application on an Ubuntu Server (this was for a VPS with 1gb ram).
Update packages library (run all apt-get commands as root or with sudo):
apt-get update
Install MySQL:
apt-get install mysql-server
Enter a password for the mysql root account.
Install Java, Tomcat, and other requirements for running the WebDSL compiler:
apt-get install ant unzip openjdk-7-jdk tomcat7
Get WebDSL compiler:
wget http://hydra.nixos.org/job/webdsl/trunk/buildJavaZip/latest/download/1/webdsl-java.zip
unzip webdsl-java.zip
chmod +x webdsl/bin/webdsl
export PATH=$PATH:/[path]/webdsl/bin/
Add the export PATH line to your ~/.bashrc file to make the ‘webdsl’ command work the next time you log in as well.
Install mail SMTP server:
apt-get install postfix
Choose the internet configuration, test locally with ‘sendmail’ command. If something is wrong in the configuration, change it with:
sudo dpkg-reconfigure postfix
/etc/init.d/postfix reload
Configure WebDSL application, create application.ini:
appname=myapp
backend=servlet
tomcatpath=/var/lib/tomcat7/
httpport=8080
httpsport=8443
dbmode=update
indexdir=/var/indexes/
dbserver=localhost
dbname=mydb
dbuser=myuser
dbpassword=mypass
smtphost=localhost
smtpport=25
smtpprotocol=smtp
smtpauthenticate=false
rootapp=true
If using a gmail account to send mail instead of local SMTP server, use:
smtphost=smtp.gmail.com
smtpport=465
smtpuser=blabla
smtppass=thepass
Create database and mysql user:
mysql -u root -p
create database mydb;
grant all privileges on mydb.* to myuser@'localhost' identified by 'mypass';
flush privileges;
quit
Open up the indexes directory (can be placed anywhere):
mkdir /var/indexes
chown -R tomcat7 /var/indexes
Compile application (in this application.ini myapp.app is the main file) and deploy:
webdsl build deploy
Check what’s going on in Tomcat using:
tail -f /var/lib/tomcat7/logs/catalina.out
Set Tomcat’s heap higher:
nano /etc/default/tomcat7
Change
JAVA_OPTS="-Djava.awt.headless=true -Xmx128m -XX:+UseConcMarkSweepGC"
to
JAVA_OPTS="-Djava.awt.headless=true -Xmx768m -XX:+UseConcMarkSweepGC"
Restart Tomcat:
/etc/init.d/tomcat7 restart
Check Tomcat’s current JVM arguments:
ps aux | grep tomcat
Tomcat will run on port 8080 instead of 80, a quick fix to get it to work on port 80 is the following:
iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8080
Users interact with web applications through the browser. This process consists of request and response strings being exchanged between the web server and the browser. A form is defined by a response string, which is interpreted by the browser to produce components that allow user interaction. A user can fill in data in a text field, and press the submit button. The browser first collects the data from the form input fields, and constructs a request string to send to the web server, which receives the request string and parses it. Values from input fields can be accessed separately but are represented as strings. A web application bears the responsibility of converting these strings to actual types to be used in further processing of the request. In WebDSL, the conversion of request parameters is done automatically. This is the first phase of the request processing lifecycle. The request processing lifecycle consists of the following phases:
Request parameter conversion is not possible if the incoming value is not well-formed. For example, a value of “3f” cannot be converted to an integer. Since a failed conversion invalidates any input this triggers re-rendering the page with error messages.
In the first phase, parameters are decoded from strings. In the ‘Update Model Values’ phase, these parameters are automatically inserted in data model entities. WebDSL supports such data binding through input elements. For example, the element
input(u.email)
declares that an input field should be displayed with the current contents of the email property of variable u of type User. Furthermore, when a user submits the containing form with a new value in the email field, the new value will be assigned to u.email.
Data binding requires assignments to and collection operations on entity properties which trigger validation checks defined in the entity. When a property is validated each validation rule defined on that property is checked, possibly producing multiple error messages. When at least one validation fails during this phase, further processing is disabled and errors are displayed.
When the model is updated and entity validations are checked, there can still be validation rules in pages which need to be enforced. The form validation phase traverses the form that is submitted and checks any validation it encounters. An invalid result prevents any action from executing and produces an error in the page.
When all validation checks in previous phases have succeeded, the selected action is executed. During the execution of an action there can be action assertions that validate the data in the current execution state of the action. Moreover, data invariants are still checked during this phase and can produce validation errors as well. If any validation check fails, the entire action is cancelled (clearing all changes made during that request).
Validation messages produced in the previous phases result in a re-render of the same page with error messages inserted. If all validations succeed, the action results in a redirect to the same or a different page.
Notes:
Data models in WebDSL are defined using entity definitions. An entity definition consists of the entity’s name, possibly a super-entity from which it inherits, 0 or more properties and 0 or more entity functions:
entity User {
name :: String (length = 25)
email :: Email
password :: Secret
homepage :: URL
pages -> Set<Page>
function checkPassword(s : String) : Bool {
return password.check(s);
}
predicate sameUser(u:User){ this == u }
}
A property consists of 4 parts:
The difference between reference and composite property kinds is that composite indicates that the referred entity is part of the one referring to it. The only effect this currently has is that composite cascades delete (deleting the entity will also delete the referred entity).
For a complete overview of the available types, see Types.
An example data model for a blogging site:
entity Author {
name :: String
email :: Email
password :: Secret
posts -> Set<Post> (inverse=Post.author)
}
entity Post {
author -> Author
title :: String
text :: Text
comments -> Set<Comment> (inverse=Comment.post)
}
entity Comment {
post -> Post
author :: String
text :: Text
}
Instantiating new entity objects is done with the following expression:
Entity{ [property := value]* }
The entity name followed by an optional list of property assignments between curly brackets.
Example:
User{}
User{ name := "Alice" }
User{ name := "Bob" age := 34 }
Default initialization (what you would put into the constructor of an object in e.g. the Java programming language), can be added by extending the constructor function that is implicitly called.
Example:
entity A : B{
extend function A(){
name := name +"A";
}
}
entity B{
extend function B(){
name := name +"B";
}
}
test constructors {
var t := A{};
assert(t.name == "BA");
}
Creating an empty entity which doesn’t call the constructor extensions can be done using createEmptyEntity, e.g. createEmptyUser()
The ‘name’ property is special, it is declared for each entity. By default it is a derived property that simply returns the id of the entity (which is also a special property declared for each entity, id:UUID is set automatically). The name can be customized by declaring a real name property:
name :: String
Or derived name property:
name :: String := firstname + lastname
Or by declaring a property as the name using an annotation:
someproperty :: String (name)
The name property is used in input
and select
template elements to refer to an entity. Example:
application exampleapp
init{
var u := User{};
u.save();
u := User{};
u.save();
u := User{};
u.save();
}
entity User{}
entity UserList{
users -> List<User>
}
var globalList := UserList{}
define page root(){
for(u:User in globalList.users){
output(u.name) //there is always a name property
}
form{
input(globalList.users) //this will show three UUIDs as options
submit("save",action{})
}
}
If the name is not a real property, you cannot create an input for it or assign to it.
The allowed
annotation for entity properties provides a way to restrict the choices the user has when the property is used in an input:
entity Person{
friends -> Set<Person> (allowed=from Person as p where p != this)
}
var p1 := Person{}
define page root(){
form{
input(p1.friends)
submit action{} {"save"}
}
}
The allowed collection can be accessed through an entity function with name allowed[PropertyName]
, e.g. p1.allowedFriends()
Entities can inherit properties and functions from other entities, like subclassing in Object-Oriented programming.
Example:
entity Sub : Super {
str :: String
}
entity Super {
i :: Int
}
function test(){
var e1 := Sub{ i := 1 str := "sdf" };
var e2 := Super{ i := 1 };
}
Subclass entities can be passed whenever an argument of one of its super types is expected.
Example:
function test(){
var e1 := Sub{ i := 1 str := "sdf" };
test(e1);
}
function test(s:Super){
log(s.i);
}
Checking the dynamic type of an entity can be done using isa
and casting is performed using as
.
Example:
function test(s:Super){
if(s isa Sub){
var su :Sub := s as Sub;
log(su.str);
}
}
When specifically want to call a function from the Superclass, use the ‘super’ keyword.
Example:
entity Sub : Super {
function foo() : Int {
return super.foo();
}
}
entity Super {
function foo() : Int {
return 42;
}
}
For defined entities, a number of properties are automatically generated.
id :: UUID
The id property is used in the database as key for the objects. The property is can only be read.
version :: Int
The version property is a hibernate property which auto-increases for an object that is dirty when it is written to the database.
created :: DateTime
The created property is a generated property which is set on the save of an object also with cascaded saves.
modified :: DateTime
The modified property is a generated property which is automatically set on flush of an dirty object.
For defined entities, a number of global functions are automatically generated. Replace Entity with the defined entity name below.
If the Entity has an id annotation on a property, the following functions are generated (idtype is the type of the id property):
getUniqueEntity
getUniqueEntity(id : idtype) : Entity
If the Entity with the given id already exists, it is returned. If it did not exist, it is created once and a flush to the database is performed (this will commit any changes made to the entities in memory, e.g. the changes from data binding of input fields), repeated calls to this function with the same argument will keep returning that created Entity.
isUniqueEntity
isUniqueEntity(ent : Entity) : Bool
This function returns false when the value of the id property of ent is already taken. The function returns true when the id property is not taken, but will do so only once, subsequent calls with different entities but the same id will then return false (which makes this function suitable for processing a batch of entities in an action).
isUniqueEntityId
isUniqueEntityId(id : idtype, ent : Entity) : Bool
This function returns false when the entity would not be unique when given the id argument. The function returns true when the entity would be unique, but will do so only once for a given id, checking a different entity with the same id will return false in the rest of the action handling.
isUniqueEntityId(id : idtype) : Bool
This function returns false when the given id is not available for the Entity type. The function will return true only once, to cope with batch processing.
Note that these functions use one collection per entity to determine whether an id is available, so a call to isUniqueUserId(id) can influence the result of isUniqueUser(ent).
findEntity
findEntity(id : idtype) : Entity
This function returns the Entity with the given id value, null if it does not exist.
For each String property in an Entity, a find function is generated (repace Property with the property name):
findEntityByProperty
findEntityByProperty(val : String) : List<Entity>
This function returns a list of all Entitys with the exact given Property value, an empty list if there are none.
findEntityByPropertyLike
findEntityByPropertyLike(val : String) : List<Entity>
This function returns a list of all Entitys with the given Property value as substring, an empty list if there are none.
Every entity has a name, which is always a string. This name can be retrieved by the automatically generated getName() function.
The name of an entity is determined as follows:
If a property of the entity has the name annotation, the name of the entity equals this property. This property must be of type String.
If a property of the entity is called ‘name’ and is of type String, this property determines the entity name.
Otherwise, the id of the entity (converted to its string-value) is used.
A typical scenario where these functions come in handy is a create/edit page for an entity. In the following example the isUniquePage function is used to verify that the new page has a unique identifier property:
entity Page {
identifier :: String (id, validate(isUniquePage(this), "Identifier is taken")
}
define page createPage(){
var p := Page{}
form{
label("Identifier"){input(p.identifier)}
action("save",save())
action save(){
p.save();
message("New page created.");
return home();
}
}
}
You can quickly generate basic pages for creating, reading, updating and deleting entities using derive CRUD -entityname-
. It will create pages that allows creating and deleting such entities, and editing of all entities of this type in the database.
Example:
application test
entity User {
username :: String
}
derive CRUD User
//application global var
var u_1 := User{username:= "test"}
define page root(){
navigate(createUser()){ "create" } " "
navigate(user(u_1)){ "view" } " "
navigate(editUser(u_1)){ "edit" } " "
navigate(manageUser()){ "manage" }
}
As the navigates indicate, the pages that are created are:
view:
define page entity(arg:Entity){...}
create:
define page createEntity(){...}
edit:
define page editEntity(arg:Entity){...}
manage (delete):
define page manageEntity(){...}
These pages are particularly useful when you’re just constructing the domain model, because the generated pages are usually too generic for a real application.
Storing data in the session context on the server is done using session entities. Example:
session shoppingcart {
products -> List<Product>
}
A session entity name is a globally visible variable in the application code. The entity object is automatically instantiated and saved, one for each browser session accessing the application.
Typically, session data is used for keeping track of authentication state, but it can also be used for temporarily storing data for anonymous users. A common oversight with session data is that it is shared between tabs in a browser.
Declaring an access control principle, e.g. principal is User with credentials name,password
, automatically creates a securityContext
session entity. For more information about access control see the Access Control section.
Session entities can also be extended with extra properties. Example:
extend session shoppingcart{
lastSearchQuery :: String
}
Session data times out by default, this timeout length can be adjusted in the application.ini
file, e.g. sessiontimeout=10080
. This time is specified in minutes. More information about application settings is shown on the Application Configuration page.
This section lists all the built-in types available in WebDSL.
There are multiple types that are equivalent to String. These types can have different validation rules, functions, inputs, and outputs. Converting between these types can be done with casts, e.g.
var : Secret := url("123") as Secret;
The String compatible types are:
Similarly, the Date times are equivalent as well:
Enumeration types are like enum
in Java and other languages. You define them as follows:
enum Gender {
maleGender("Male"),
femaleGender("Female")
}
You can use them as follows:
entity User {
gender -> Gender
}
define page somePage() {
var u : User;
input(u.gender) // shows a drop-down
output(u.gender.name) // shows either Male or Female
}
Or, in action code:
function setMale(u : User) {
u.gender := maleGender;
}
Represents a string of characters. Example:
var s : String := "Hello world";
The default value for String properties and variables is "".
contains(s: String):Bool
Tests whether s is a substring of this string.
length():Int
Returns the length of this string.
parseInt():Int
Returns the Int value in this string. If this string does not contain a valid Int value, this function returns null.
parseUUID():UUID
Returns the UUID value in this string. If this string does not contain a valid UUID value, this function returns null.
toUpperCase():String
Returns this string in uppercase.
toLowerCase():String
Returns this string in lowercase.
split():List
Returns the characters in this string as separate strings in a list.
split(separator:String):List
Returns a list of strings produced by splitting this string around matches of separator.
makePatch(new : String):Patch
Creates a Patch from this String to the new String, see Patch type.
diff(new : String):List
Creates a List describing the differences between this String and the new String.
trim(): String
concat():String
Concatenates the strings in this list of strings.
concat(separator:String):String
Concatenates the strings in this list of strings, separated by separator.
Represents an integer number. Example:
var i : Int := 3;
The default value for Int properties and variables is 0.
floatValue():Float
Converts this value to a Float.
toString():String
Converts this value to a String.
Represents a floating point number. Example:
var f : Float := 3.5;
The default value for Float properties and variables is 0f.
round():Int
Rounds this value to the nearest Int value.
floor():Int
Returns the largest Int that is less than or equal to this value.
ceil():Int
Returns the smallest Int that is greater than or equal to this value.
toString():String
Converts this value to a String.
random():Float
Produces a random Float between 0 and 1.
Represents a truth value. Either true or false. Example:
var b : Bool := true;
The default value for Bool properties and variables is false.
toString():String
Converts this value to a String.
Represents an ordered list of items of a certain type. Example:
var l : List<Int> := [1, 2, 3, 4];
Sorted output of lists can be created using the for
loop filter in templates or actions:
for(u:User in [u1,u2,u3] order by u.name desc){
output(u)
}
length
Gives the number of items in the list.
List()
Creates an empty list of type Entity.
List(…, Entity, …)
Creates a list of type Entity with the elements resulting from the comma separated argument expressions.
Example:
var list := List<User>(User{},SubSubUser{},SubUser{})
[Entity, …]
Creates a list of type Entity (type of first element) with the elements resulting from the comma separated expressions between the [ ].
Example:
var list := [User{ name := "test" },SubUser{},uservar]
add(Entity)
Adds the entity to this list.
remove(Entity)
Removes the first occurence of Entity in this list.
clear()
Removes all entities in this list.
addAll(List/Set)
Adds all entities of the List/Set to this list.
set() : Set
Creates a Set containing the unique elements in this list.
indexOf(Entity) : Int
Returns the index of the first occurence of Entity in this list. Returns -1 if the Entity is not in this list.
get(Int)
Returns the element at location Int in this list.
set(Int,Entity)
Sets the list element at Int to Entity. If the Int is not within bounds, nothing is set, and a warning is shown in the log.
insert(Int,Entity)
Inserts the Entity at location Int in this list. If the Int is not within bounds, nothing is set, and a warning is shown in the log.
removeAt(Int)
Removes the element at location Int in this list. If the Int is not within bounds, nothing is set, and a warning is shown in the log.
subList(from:Int,to:Int):List
Returns a portion of this list between the specified from, inclusive, and to, exclusive.
Represents an unordered collection of unique items of a certain type. Example
var s : Set<Int> := {1, 2, 3, 4};
Sorted output of sets can be created using the for
loop filter in templates or actions.
for(u:User in [u1,u2,u3] order by u.name desc){
output(u)
}
length
Gives the number of items in the set.
Set()
Creates an empty set of type Entity.
Set(…, Entity, …)
Creates a set of type Entity with the elements resulting from the comma separated argument expressions.
Example:
var set := Set<Person>(Person{},SubSubPerson{},SubPerson{})
{Entity, …}
Creates a set of type Entity (type of first element) with the elements resulting from the comma separated expressions.
Example:
var set : Set<Person> := {Person{ name := "test" },personvar}
add(Entity)
Adds the entity to this set.
remove(Entity)
Removes Entity in this set.
clear()
Removes all entities in this set.
addAll(List/Set)
Adds all entities of the List/Set to this set.
list() : List
Creates a List containing the elements in this set.
Represents a secret string (usually a password). The page input for a Secret is a masked textfield. The page output for a Secret is “********”.
Example:
var pass : Secret := "123";
The default value for Secret properties and variables is "".
The Secret type is compatible with the String type, all the String functions can be used, and String literals can be assigned to Secret typed vars. A Secret can be cast to a String, this is necessary when calling functions or templates with String arguments. For example:
function test1(s:String){}
function test2(pass:Secret){
test1(pass as String);
}
A String can also be cast to a Secret:
assert(pass == "123" as Secret)
all String functions
Secret is compatible with String.
check(input:Secret):Bool
Checks the input Secret (not digested) against the digest version contained in this Secret.
Example:
if (user.password.check(password)) {
securityContext.principal := us;
securityContext.loggedIn := true;
}
digest():Secret
Generates a digest of the clear-text password contained in this Secret.
Example:
var s : Secret := "123";
s := s.digest();
assert(s.check("123" as Secret));
Represents an e-mail address as a string. If you are interested in sending email from your application, have a look at the SendEmail page. The page input for an Email is a textfield with the following validation:
validate(/[a-zA-Z0-9_\-\.]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([a-zA-Z0-9\-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?/.match(this), "Not a valid email address")
The page output for Email is the same as String output.
Example:
var address : Email := "webdslorg@gmail.com";
The default value for Email properties and variables is "". Add the ‘not empty’ annotation (or a custom validation) to an Email type property in an entity to disallow the empty string.
The Email type is compatible with the String type, all the String functions can be used, and String literals can be assigned to Email typed vars. An Email can be cast to a String, this is necessary when calling functions or templates with String arguments. For example:
function test1(s:String){}
function test2(address:Email){
test1(address as String);
}
A String can also be cast to an Email:
assert(address == "123" as Email)
all String functions
Email is compatible with String.
Represents a large string. The page input for a Text is a textarea. The page output for Text is the same as String output.
Example:
var t : Text := "123";
The default value for Text properties and variables is "".
The Text type is compatible with the String type, all the String functions can be used, and String literals can be assigned to Text typed vars. A Text can be cast to a String, this is necessary when calling functions or templates with String arguments. For example:
function test1(s:String){}
function test2(t:Text){
test1(t as String);
}
A String can also be cast to a Text:
assert(t == "123" as Text)
all String functions
Text is compatible with String.
Represents a large string with Markdown syntax support and internal page links. Internal page links in WikiText can be created using caption (remove the spaces).
The page input for WikiText is a textarea. The page output for WikiText processes the Markdown and page links and produces html elements.
Example:
var t : WikiText := "123";
The default value for WikiText properties and variables is "".
The WikiText type is compatible with the String type, all the String functions can be used, and String literals can be assigned to WikiText typed vars. A WikiText can be cast to a String, this is necessary when calling functions or templates with String arguments. For example:
function test1(s:String){}
function test2(t: WikiText){
test1(t as String);
}
A String can also be cast to a WikiText:
assert(t == "123" as WikiText)
all String functions
WikiText is compatible with String.
Represents a patch. The page input for a Patch is the same as for Text. The page output for Patch is the same as for Text.
Example:
var p : Patch := "12345".makePatch("24");
The default value for Patch properties and variables is "".
The Patch type is compatible with the String type, all the String functions can be used, and String literals can be assigned to Patch typed vars. A Patch can be cast to a String, this is necessary when calling functions or templates with String arguments. For example:
function test1(s:String){}
function test2(p:Patch){
test1(p as String);
}
A String can also be cast to a Patch:
assert(p == "123" as Patch);
all String functions
Patch is compatible with String.
applyPatch(arg: String):String
Applies this patch to the arg String.
Example:
var s1 : Patch := "12345".makePatch("24");
assert(s1.applyPatch("12345") == "24");
Represents both a date and a time. The page input for a DateTime is a textfield, the expected format is dd/MM/yyyy H:mm. The page output for a DateTime shows the DateTime formatted with dd/MM/yyyy H:mm. Use the format function to customize the output format.
The default value for DateTime properties and variables is null.
The DateTime type is compatible with the Time and Date types. A DateTime can be cast to these types.
DateTime(String):DateTime
Dates can be constructed using the Date constructor (expected format dd/MM/yyyy H:mm):
var dt : DateTime := DateTime("22/06/1983 22:08");
DateTime(String, String):DateTime
var dt : DateTime := DateTime("12:12 05-1994-06", "mm:H MM-yyyy-dd");
The second parameter represents the date/time formatting string.
now():DateTime
Creates a DateTime containing the current time and day.
format(formatstring:String):String
Format this DateTime using the formatstring. See Java SimpleDateFormat class documentation for formatstring syntax.
before(arg:Date/Time/DateTime):Bool
Tests whether this date and time is before the arg date and time.
after(arg:Date/Time/DateTime):Bool
Tests whether this date and time is after the arg date and time.
addSeconds(amount:Int)
Adds seconds, amount
may be negative.
addMinutes(amount:Int)
Adds minutes, amount
may be negative.
addHours(amount:Int)
Adds hours, amount
may be negative.
addDays(amount:Int)
Adds days, amount
may be negative.
addMonths(amount:Int)
Adds months, amount
may be negative.
addYears(amount:Int)
Adds years, amount
may be negative.
getSecond():Int
Gets the second.
getMinute():Int
Gets the minute.
getHour():Int
Gets the hour.
getDay():Int
Gets the day of the month.
getDayOfYear():Int
Gets the day of the year.
getMonth():Int
Gets the month.
getYear():Int
Gets the year.
Represents a date (not including a time). The page input for a Date is a textfield, the expected format is dd/MM/yyyy. The page output for a Date shows the Date formatted with dd/MM/yyyy. Use the format function to customize the output format.
The default value for Date properties and variables is null. Note that all Date types are DateTime at run-time.
The Date type is compatible with the DateTime and Time types. A Date can be cast to these types.
Date(String):Date
Dates can be constructed using the Date constructor (expected format dd/MM/yyyy):
var d : Date := Date("04/09/2009");
Date(String, String):Date
The second parameter represents the date formatting string.
var d1 : Date := Date("12-20-1990", "MM-dd-yyyy");
today():Date
Creates a Date containing the current day and time 00:00.
age(Date):Int
Gets the age from a date of birth.
format(formatstring:String):String
Format this DateTime using the formatstring. See Java SimpleDateFormat class documentation for formatstring syntax.
before(arg:Date/Time/DateTime):Bool
Tests whether this date and time is before the arg date and time.
after(arg:Date/Time/DateTime):Bool
Tests whether this date and time is after the arg date and time.
addSeconds(amount:Int)
Adds seconds, amount
may be negative.
addMinutes(amount:Int)
Adds minutes, amount
may be negative.
addHours(amount:Int)
Adds hours, amount
may be negative.
addDays(amount:Int)
Adds days, amount
may be negative.
addMonths(amount:Int)
Adds months, amount
may be negative.
addYears(amount:Int)
Adds years, amount
may be negative.
Represents a time (not including a date). The page input for a Time is a textfield, the expected format is H:mm. The page output for a Time shows the Time formatted with H:mm. Use the format function to customize the output format.
The default value for Time properties and variables is null. Note that all Date types are DateTime at run-time.
The Time type is compatible with the DateTime and Date types. A Time can be cast to these types.
Time(String):Time
Time can be constructed using the Time function (expected format H:mm):
var t : Time := Time("22:08");
Time(String, String):Time
The second parameter represents the date formatting string.
var t1 : Time := Time("59:08", "mm:H");
format(formatstring:String):String
Format this DateTime using the formatstring. See Java SimpleDateFormat class documentation for formatstring syntax.
before(arg:Date/Time/DateTime):Bool
Tests whether this date and time is before the arg date and time.
after(arg:Date/Time/DateTime):Bool
Tests whether this date and time is after the arg date and time.
addSeconds(amount:Int)
Adds seconds, amount
may be negative.
addMinutes(amount:Int)
Adds minutes, amount
may be negative.
addHours(amount:Int)
Adds hours, amount
may be negative.
addDays(amount:Int)
Adds days, amount
may be negative.
addMonths(amount:Int)
Adds months, amount
may be negative.
addYears(amount:Int)
Adds years, amount
may be negative.
Represents a hyperlink. The page input for a URL is the same as for String. The page output for URL is a hyperlink.
Example:
var u : URL := "http://webdsl.org";
The default value for URL properties and variables is "".
The URL type is compatible with the String type, all the String functions can be used, and String literals can be assigned to URL typed vars. A URL can be cast to a String, this is necessary when calling functions or templates with String arguments. For example:
function test1(s:String){}
function test2(t: URL){
test1(t as String);
}
A String can also be cast to a URL:
assert(t == "http://webdsl.org" as URL);
url(arg : String):URL
Casts the arg String to a URL type, equivalent to ‘arg as URL’.
You can also use this in a navigate templatecall in order to provide an absolute URL instead of an internal link.
Example:
navigate(url("http://webdsl.org")){ "powered by WebDSL" }
all String functions
URL is compatible with String.
Represents an (uploaded) file. The page input for a File is a file upload component. The page output is a file download link.
fileName():String
Returns the name of this file.
getContentAsString():String
Returns the content of this file as String.
download()
The current action will result in a download of this file.
Represents an (uploaded) image. The page input for an Image is a file upload component. The page output shows the image.
fileName():String
Returns the name of this image file.
getContentAsString():String
Returns the content of this file as String.
download()
The current action will result in a download of this image file.
getWidth()
Returns the calculated width of this image.
getHeight()
Returns the calculated height of this image.
resize(maxWidth : Int, maxHeight : Int)
Resizes the image to the set dimensions. Note that currently, images can only be downscaled.
crop(x : Int, y : Int, width : Int, height : Int)
Crops the image to the specified size and coordinates.
clone() : Image
Makes a copy of the image and returns it.
Pages in WebDSL can be defined using the following construct:
define page [pagename]( [page-arguments]* ){ [page-elements]* }
There are basic output elements for structure and layout of the page, such as title
and header
.
Example:
define page root() {
title { "Page title" }
section {
header{ "Hello world." }
"Greetings to you."
}
}
Pages can have parameters, and output
is used for inserting data values.
Example:
define page user(u : User) {
"The name of this user is " output(u.name)
}
The form
element in combination with submit
is used for submitting data. input
elements perform automatic data binding upon submit. For more information about forms, go to the Form page.
Example:
define page editUser(u:User){
form{
input(u.name)
submit action{} { "save" }
}
}
Pages can be made reusable by declaring them as template, and calling them from other pages or templates.
Example:
define common(){
header{ "my page" }
}
define page root(){
common()
}
The output(<expression>)
template call is used to display a value in a page. It can also be used with Entity type expressions, and collections.
Example:
define page user(u:User){
output(u)
}
The output
template can be customized for each entity type.
Example:
define output(u:User){
"user with name: " output(u.name)
}
navigate <page call> { <page element*> }
Link to a page. For example:
page news() { "News" }
Declares the title of the current page.
title { element* }
Declares the description of a page (not visible, added as description meta tag in the head section of a page). This data is often viewed in search result snippets. Introduced in WebDSL 1.3.0.
description { element* }
section { element* }
Indicate sections in a document; may be nested. May include a
header { element* }
element that indicates the section title.
image ( <string with relative or absolute path to image> )
Displays an image. Images placed in an “images” folder in the root directory of your application will be automatically copied during deployment.
Example:
define page root(){
image("http://webdsl.org/webdslorg/images/WebDSL-small.png")
image("/images/WebDSL-small.png")
}
Lists can be created with the list
and listitem
elements.
Example:
list {
listitem { "Milk" }
listitem { "Potatoes" }
listitem { "Cheese (lots)" }
}
Tables can be created with the table
, row
, and column
elements.
Example:
table {
row { column{ "Username" } column{ output(user.name) } }
row { column{ "Password" } column{ "it's a secret" } }
}
block{ <page element*> }
block(String){ <page element*> }
Groups text; optionally defines a class for referencing in CSS. Results in a <div> element in HTML.
Templates enable reuse of page elements. For example, a template for a footer could be:
define footer() { All your page are belong to us. }
This template can be included in a page with a template call:
define page example(){
footer
}
Like pages, templates can be parameterized.
define edit(g:Group){
form {
input(g.members)
action("save",action{})
}
}
define page editGroup(g:Group){
edit(g)
}
While pages must have unique names, templates can be overloaded. The overloading is resolved compile-time, based on the static types of the arguments.
define edit(g:Group){...}
define edit(u:User){...}
define page editGroup(g:Group){
edit(g)
}
Template definitions can be redefined locally in a page or template, to change their meaning in that specific context. All uses are replaced in templates called from the redefining template.
define main{
body()
}
define body(){
"default body"
}
define page root(){
main
define body(){
"custom body"
}
}
Iterating a collection of entities or primitives can be done using a for loop. There are three types of for loops for templates:
for(id:t in e){ elem* }
This type of for loop iterates the collection produced by expression e, which must contain elements of type t. The elements in the collection are accessible through identifier id.
The collection can be filtered:
for(id:t in e filter){ elem* }
This for loop iterates all the entities in the database of type t. These can also be filtered. Note that it is more efficient to retrieve the objects using a filtering query and use the regular for loop above for iteration.
for(id:t){ elem* }
for(id:t filter){ elem* }
This for loop iterates the numbers from e1 to e2-1.
for(id:Int from e1 to e2){ elem* }
All three template for loops can be followed by a separated-by declaration, which will separate the outputs from the for loop with the declared elem*.
separated-by{ elem* }
The filter part of a for loop can consist of four parts:
where e1
e1 is a boolean expression which needs to evaluate to true for the element to be iterated.
order by e2 asc/desc
e2 is an expression that needs to produce a primitive type such as String or Int, which will be used to order the elements ascending or descending.
limit e3
e3 is an Int expression which will limit the number of elements that get iterated.
offset e4
e4 is an Int expression which will offset the starting element of the iteration.
Each of the four parts is optional, but they have to be specified in this order. The filtering is done in the application, so use queries instead of filters to optimize the WebDSL application.
XML fragments can be embedded directly in templates. This allows easy reuse of existing XHTML fragments and CSS. For example:
define main() {
<div id="pagewrapper">
<div id="header">
header()
</div>
<div id="footer">
<p />"powered by " <a href="http://webdsl.org">"WebDSL"</a><p />
</div>
</div>
<some:tag />
}
While the name and attribute names are fixed, the attribute values can be any WebDSL expression that produces a string:
define test(i : Int) {
<div id="page" + "wrapper" + i />
}
The includeCSS(String)
template call allows you to include a CSS file in the resulting page. CSS files can be included in your project by placing them in a stylesheets/ directory in the project root.
Example 1:
define page root() {
includeCSS("dropdownmenu.css")
}
It is also possible to include a CSS file using an absolute URL.
Example 2:
define page root() {
includeCSS("http://webdsl.org/stylesheets/common_.css")
}
The media attribute can be set by passing it as second argument in includeCSS(String,String)
Example 3:
define page root(){
includeCSS("http://webdsl.org/stylesheets/common_.css","screen")
}
The includeJS(String)
template call allows you to include a javascript file in the resulting page. Javascript files can be included in your project by placing them in a javascript/ directory in the project root.
Example 1:
define page root() {
includeJS("sdmenu.js")
}
It is also possible to include a Javascript file using an absolute URL.
Example 2:
define page root() {
includeJS("http://ajax.googleapis.com/ajax/libs/jquery/1.2.6/jquery.min.js")
}
When an invalid URL is being requested from a WebDSL application, the default response is to give a 404 error. To customize this error page, define a pagenotfound
page in your application.
Example:
define page pagenotfound() {
title{ "myapp / page not found (404)" }
main()
define body() {
par{ "That page does not exist!" }
par{ "Maybe you can find what you are looking for using the search page." }
}
}
By default, any template content will be escaped, if you want to include a string directly in the page source use rawoutput.
Example:
rawoutput{" "}
Setting HTML element attributes is supported for calls to built-in templates, the syntax is as follows:
templatename(...)[attrname=e, ...]{ ... }
attrname
is an attribute name, and e
is a webdsl expression such as "foo"
or "foo"+bar
.
Example:
define page root(){
var somevalue := "lo"
image("/images/logosmall.png")[alt = somevalue+"go", longdesc = "blablabla"]
navigate root()[title = "root page"]{ "root" }
}
Template and page definitions can be overridden using the override
modifier, e.g. to override a built-in page such as pagenotfound
:
define override page pagenotfound(){
"page does not exist!"
}
add ?logsql
after the URL of a page to get a log of all the SQL queries executed to render that page
for applications with access control enabled, accessing this log is disabled by default, it can be enabled using an access control rule:
rule logsql { check }
e.g.
rule logsql { principal.isAdmin }
The form
element enables user input, and should include submit
or submitlink
elements to handle that user input. When pressing such a submit button/link, data binding will be performed for all inputs in the form.
form {
var name : String
var pass : Secret
label("Username:"){ input(name) }
label("Password:"){ input(pass) }
submit save() { "save" }
}
action save(){
User{
username := name
password := pass.digest()
}.save();
}
input(<expression>)
creates an input form element. Can be applied directly to the properties of an entity (e.g., input(user.name)) or to page variables.
Input widgets are determined by the type of the property passed to the input template call:
For example, to get a checkbox, use:
define root(){
var x : Bool := false
form{
input(x)
submit action{ log(x); } { "log result" }
}
}
or:
entity TestEntity {
x :: Bool
}
define editTestEntity (e:TestEntity){
form{
input(e.x)
submit action{ } { "update entity" }
}
}
Actions define targets for form submits. The body of an action contains statements See action code.
Example:
define page edituser(u : User) {
form {
"Edit this user"
label("Name:"){ input(u.name) }
label("Group:){ input(u.group) }
submit saveUser() {"save"}
}
}
action saveUser() {
u.save();
}
Actions may be declared inline with the submit
element
Example:
define page edituser(u : User) {
form {
"Edit this user"
label("Name:"){ input(u.name) }
label("Group:){ input(u.group) }
submit action{u.save();} {"save"}
}
}
Submits for actions may be declared as properties on template elements, using the same DOM events as for Javascript, such as onclick
, onblur
, onkeyup
(http://www.w3schools.com/jsref/dom_obj_event.asp).
Example:
define page edituser(u : User) {
form {
"Edit this user"
label("Name:"){ input(u.name) }
label("Group:){ input(u.group) }
image("images/save.png")[onclick := action{u.save();}]
}
}
Page and template definitions can contain variables. This example displays “Dexter”:
define page cat() {
var c := Cat { name := "Dexter" }
output(c.name)
}
entity Cat {
name :: String
}
These variables are necessary when constructing a page that creates a new entity instance. The instance can be created in the variable and data binding can be used for the input page elements. The next example allows new cat entity instances to be created, and the default name in the input form is “Dexter”:
define page newCat() {
var c := Cat { name := "Dexter" }
form{
label("Cat's name:"){ input(c.name) }
action("save",action{ c.save(); return showCat(c); })
}
}
It is possible to initialize such a page/template variable with arbitrary statments using an ‘init’ action:
define page newCat() {
var c
init {
c := Cat{};
c.name := "Dexter";
}
form{
label("Cat's name:"){ input(c.name) }
action("save",action{ c.save(); return showCat(c); })
}
}
Be aware that these type of variables (and the init blocks) are handled separately from the other elements. They do not adhere to template control flow constructs like ‘if’ and ‘for’; they are extracted from the definition. However, you can express such functionality in the ‘init’ block. For example:
error:
define page bad() {
if(someConditionFunction()){
var c := Cat{}
}
else {
var c := Cat{ name := "Dexter" }
}
output(c.name)
}
ok:
define page good() {
var c
init{
if(someConditionFunction()){
c := Cat{}
}
else {
c := Cat{ name := "Dexter" }
}
}
output(c.name)
}
select(x, y)
can be used as input for an entity variable or for a collection of entities variable, where x
is the variable or fieldaccess and y
is the collection of options. It will create a dropdown box/select or a multi-select respectively. The name
property of an entity is used to describe the entity in a select, see name property.
input(x)
for an entity reference property or a collection property is the same as select
, with as options all entities of its type that are in the database.
Example:
entity User {
username :: String (name)
teammate -> User
group -> Set<Group>
}
entity Group {
groupname :: String (name)
}
init{ //application init
var u := User { username := "Alice" };
u.save();
u := User { username := "Bob"};
u.save();
var g := Group { groupname := "group 1" };
g.save();
g := Group { groupname := "group 2" };
g.save();
}
define page root(){
form{
table{
for(u:User){
output(u.username)
input(u.teammate)
input(u.group)
}
}
submit("save",action{})
}
}
input(u.teammate)
is a dropdown/select with options null
, “Alice”, “Bob”. input(u.group)
is a multi-select with options “group 1” and “group 2”.
Example 2:
define page root(){
var teammates := from User
var groups := from Group
form{
table{
for(u:User){
output(u.username)
select(u.teammate, teammates)
select(u.group, groups)
}
}
submit("save",action{})
}
}
Equivalent to the previous example, but using explicit selects instead.
Example 3:
var u3 := User { username:="Dave" }
var g3 := Group { groupname:="group 3" }
define page root(){
var teammates := [u3]
var groups := {g3}
form{
table{
for(u:User){
output(u.username)
select(u.teammate, teammates)
select(u.group, groups)
}
}
submit("save",action{})
}
}
Options are restricted in this example, null
and “Dave” for select(u.teammate, teammates)
and only “group 3” for select(u.group, groups)
The null
option for a select can be removed either by a not null
annotation on the property:
teammate -> User (not null)
Or by setting [not null]
on the input
or select
itself:
input(u.teammate)[not null]
select(u.teammate, teammates)[not null]
The possible options can also be determined using an annotation on the property:
group -> Set<Group> (allowed = {g3})
In this case just using input(u.group)
will only show “group 3”
Radio buttons can be used as an alternative to select
for selecting an entity from a list of entities. The name
property, or the property with name
annotation, will be used as a label for the corresponding radio button.
entity Person{
name :: String
parent -> Person
}
define page editPerson(p:Person){
radio(p.parent, getPersonList())
}
The captcha
element creates a fully automatic CAPTCHA form element.
Example:
define page root(){
var i : Int
form{
input(i)
captcha()
submit action{ Registration{ number := i }.save(); } {"save"}
}
}
This section describes the expressions and statements available in WebDSL.
literals
A number of literals are supported:
operators
The following operators are supported:
binary operators
Example:
if(!(b is a String) && (b in [8, 5] || b + 3 = 7)) {
// ...
}
variables
Variables can be accessed by use of their identifiers and their properties using the . notation. Example: person.lastName
indexed access List elements can be retrieved and assigned using index access syntax:
var a := list[0]; list[2] := “test”;
Functions can be defined globally and as methods in entities:
function sayHello(to : String) : String {
return "Hello, " + to;
}
entity User {
name :: String
function showName() : String {
return sayHello(name + "!");
}
}
As of August 2016, entity functions without arguments can also be preceded with the cache
keyword. This cache operates at the request level, i.e. it only calculates its value once per request. This is useful for cases where a more expensive function is repeatedly invoked (e.g. for access control).
entity SubForum {
name : String
managers : [User]
...
cached function isManager() : String {
return loggedIn()
&& ( principal() in managers || parentForum.isManager() )
}
}
Variables can be defined globally, in pages (see PageVariables), and in code blocks.
Syntax:
var <identifier> : <Sort>;
This defines a variable within the current scope with name identifier and type Sort.
Variable declarations can also have an expression that initializes the value:
var <identifier> [: <Sort>] := expression;
The sort is optional in this case, if the Sort is not declared, the var will receive the type resulting from the expression (also known as local type inference).
Global variables always need an expression for initialization, they are added to the database once (when the first page is loaded, the database is checked to see whether all global vars have been created already). Global variables can be edited, but removing them can cause problems when there are explicit references to those variables.
Global variables can be further initialized using a global init{} block, e.g.
var defaultUser := User{name:="default"}
init{
defaultUser.someInitializeFunction();
}
define page root(){
output(defaultUser.name)
}
Global inits are also performed only once after database creation (if the dbmode is create-drop each new deploy will recreate the globals and execute inits, see ApplicationConfiguration).
The ; is optional for global and page variable declarations.
The syntax of an assignment:
<variable> := <value expression>;
Example:
p.lastName := "Doe";
The if-statement has the following syntax:
if(<expression>) {
<block executed if true>
} [else {
<block executed if false>
}]
If the expression is true the first block of code is executed, if it’s false, the second block is executed. The else block is optional. Example:
if(user.lastName = "Doe") {
msg := "You are unkown";
}
If can also be used in an expression, using the following syntax:
if(e1) e2 else e3
Example:
if(p.visible) p.name else ""
Syntax:
return <expression>;
Example:
function test():String{
return p.lastName;
}
In the context of a entity function this returns the expression as the result of that function. In the context of an action or page init definition, it redirects the user to the page specified in the expression.
Example:
action done(){ return root(); }
Iterating a collection of entities or primitives can be done using a for loop. There are three types of for loop statements:
This type of for loop iterates the collection produced by expression e, which must contain elements of type t. The elements in the collection are accessible through identifier id.
The collection can be filtered:
for(id:t in e){ stat* }
for(id:t in e filter){ stat* }
This for loop iterates all the entities in the database of type t. These can also be filtered. Note that it is more efficient to retrieve the objects using a filtering query and use the regular for loop above for iteration.
for(id:t){ stat* }
for(id:t filter){ stat* }
This for loop iterates the numbers from e1 to e2-1.
for(id:Int from e1 to e2){ stat* }
The filter part of a for loop can consist of four parts:
where e1
e1 is a boolean expression which needs to evaluate to true for the element to be iterated.
order by e2 asc/desc
e2 is an expression that needs to produce a primitive type such as String or Int, which will be used to order the elements ascending or descending.
limit e3
e3 is an Int expression which will limit the number of elements that get iterated.
offset e4
e4 is an Int expression which will offset the starting element of the iteration.
Each of the four parts is optional, but they have to be specified in this order. The filtering is done in the application, so use queries instead of filters to optimize the WebDSL application.
List comprehensions are a combination of mapping, filtering and sorting.
[ e1 | id : t in e2 ]
e2 produces a collection of elements with type t, e1 is an expression that allows transformation of the elements using identifier id.
Filters are also allowed:
[ e1 | id : t in e2 filter ]
Example:
[e.title | e : BlogEntry in b.entries
where e.created > date
order by e.created desc]
This expression returns all titles (e.title) from b.entries where the time created (e.created) is greater than a certain date, ordered by e.created in descending order. Both the where and order by clauses are optional. An ordering is either ascending (asc) or descending (desc).
And [ e1 | id : t in e2 ]
If e1 produces a boolean, the list comprehension can be preceded by “And” to create the conjunction of the elements produced by the list comprehension.
Or [ e1 | id : t in e2 ]
If e1 produces a boolean, the list comprehension can be preceded by “Or” to create the disjunction of the elements produced by the list comprehension.
Besides for loops, iteration can also be performed using the while statement.
while(e){ stat* }
This will repeat stat* while e evaluates to true.
The case-statement has the following syntax:
case(<expression>) {
[case <expr-1> {
<block executed if true>
}] *
[default {
<block executed if no cases match>
}]
}</verbatim>
Any number of cases and optionally one default case can be specified.
Example:
case(formatNumber) {
1 {
// format is one
}
2 {
// format is two
}
default {
// format is neither one nor two
}
}
Examples in codefinder
The rendertemplate
function can be used to render template contents to a String.
rendertemplate(TemplateCall):String
Example:
define test(a:Int){ output(a) "!" }
function showContent(i:Int){
log(rendertemplate(test(i)));
}
Native Java classes can be declared in a WebDSL application in order to interface with existing libraries and code. If you want to use just one native function, see native function interface.
The supported elements are properties, (static) methods, and constructors. The supported types are
Both the primitive type and the object types such as int and Integer can be produced by the WebDSL call (so overloading between these types is a problem here).
Add Java classes to a nativejava/ dir next to your app file and jar files in lib/.
Example:
native class nativejava.TestSub as SubClass : SuperClass {
prop :String
getProp():String
setProp(String)
constructor()
}
native class nativejava.TestSuper as SuperClass {
getProp():String
static getStatic(): String
returnList(): List<SubClass>
}
define page root() {
var d : SuperClass := SubClass()
output(d.getProp())
var s : SubClass := SubClass()
init{
s.setProp("test");
}
output(s.prop)
output(SuperClass.getStatic())
for(a: SubClass in d.returnList()){
output(a.prop)
}
}
(Example taken from compiler tests, source
https://svn.strategoxt.org/repos/WebDSL/webdsls/trunk/test/succeed/native-classes.app
https://svn.strategoxt.org/repos/WebDSL/webdsls/trunk/test/succeed/nativejava/TestSub.java
https://svn.strategoxt.org/repos/WebDSL/webdsls/trunk/test/succeed/nativejava/TestSuper.java
)
If you want an instance of your defined native java class to be passed as page (or ajax template) argument, the class should be serializable for WebDSL. From WebDSL version 1.3.0 and on, support is added for doing this by implementing the following 2 methods in your java class. (If you use a class defined in a library, you may need to extend this class with the following methods)
public static YourClass fromParamMap( Map<String,String> paramMap )
public final Map<String,String> toParamMap()
note: The keys in the returned Map may only consist of character classes [A-Z][a-z][0-9], values may hold any value as they get filtered. On deserialization, the static fromParamMap
method is invoked and its result is cast to the type as defined in the page/java template definition.
Examples can be found (notice the link ;)) in WebDSL’s source code itself.
Checking user inputs and providing clear feedback is essential for the usability of web applications. WebDSL allows declarative specification of such input validation rules using the validate feature.
Validation rules in WebDSL are of the form validate(e,s) and consist of a Boolean expression e to be validated, and a String expression s to be displayed as error message. Any globally visible functions or data can be accessed as well as any of the properties and functions in scope of the validation rule context.
Value well-formedness checks (e.g. whether the user enters a valid integer in an Int input) are added automatically to each input field.
Validation can be specified on entities in property annotations:
entity User {
username :: String (id, validate(isUniqueUser(this), "Username is taken"))
password :: Secret (validate(password.length() >= 8, "Password needs to be at least 8 characters")
, validate(/[a-z]/.find(password), "Password must contain a lower-case character")
, validate(/[A-Z]/.find(password), "Password must contain an upper-case character")
, validate(/[0-9]/.find(password), "Password must contain a digit")
email :: Email))
}
extend entity User {
username(validate(isUniqueUser(this),"Username is taken"))
password(validate(password.length() >= 8, "Password needs to be at least 8 characters")
,validate(/[a-z]/.find(password), "Password must contain a lower-case character")
,validate(/[A-Z]/.find(password), "Password must contain an upper-case character")
,validate(/[0-9]/.find(password), "Password must contain a digit"))
}
Validation can be specified directly in pages:
define page editUser(u:User) {
var p: Secret;
form {
group("User") {
label("Username") { input(u.username) }
label("Email") { input(u.email) }
label("New Password") {
input(u.password)
}
label("Re-enter Password") {
input(p) {
validate(u.password == p, "Password does not match")
}
}
action("Save", action{ } )
}
}
}
Validation can be specified in actions:
define page createGroup() {
var ug := UserGroup {}
form {
group("User Group") {
label("Name") { input(ug.name) }
label("Owner") { input(ug.owner) }
action("Save", save()) } }
action save() {
validate(email(newGroupNotify(ug)),"Owner could not be notified by email");
return userGroup(ug);
}
}
Validation output can be customized by overriding the templates used to display validation messages. Currently, there are 4 global validation templates:
define ignore-access-control errorTemplateInput(messages : List<String>)
Displays validation message related to an input.
define ignore-access-control errorTemplateForm(messages : List<String>)
Displays validation message for validation in a form.
define ignore-access-control errorTemplateAction(messages : List<String>)
Displays validation message for validation in an action.
define ignore-access-control templateSuccess(messages : List<String>)
Displays validation message for success messages.
When overriding these validation templates, use an elements
templatecall to refer to the element being validated.
Example:
define ignore-access-control errorTemplateInput(messages : List<String>){
elements
for(ve: String in messages){
output(ve)
}
}
WebDSL provides input components that validate the inputs using ajax.
built-in value types:
inputajax(String/Secret/URL/Email/Text/WikiText)
inputajax(Int/Bool/Float/Long)
reference types:
inputajax(Entity/List<Entity>/Set<Entity>)
selectajax(Entity/Set<Entity>)
radioajax(Entity)
provide selection options:
inputajax(Entity/List<Entity>/Set<Entity>,List<Entity>)
selectajax(Entity/Set<Entity>,List<Entity>)
radioajax(Entity,List<Entity>)
Selection options can also be provided using the allowed
annotation on an entity property. Example:
entity Person{
parent -> Person (allowed=from Person as p where p != this)
}
A minimal access control example is shown here.
The Access Control sublanguage is supported by a session entity that holds information regarding the currently logged in user. This session entity is called the securityContext and is configured as follows:
principal is User with credentials name, password
This states that the User entity will be the entity representing a logged in user. The credentials are not used in the current implementation (the idea is to derive a default login template). The resulting generated session entity will be:
session securityContext
{
principal -> User
loggedIn :: Bool := this.principal != null
}
Note that this principal declaration is used to enable access control in the application.
It will also generate authentication()
(both login and logout), login()
, and logout()
templates, and an authenticate
function that takes the credentials as arguments and, if they are correct, returns true and sets the principal (only String/Email/Secret-type credential properties are allowed).
Authentication can be added manually, instead of using the generated authentication templates. Here is a small example application with custom authentication:
principal is User with credentials name, password
entity User{
name :: String
password :: Secret
}
define login(){
var username := ""
var password : Secret := ""
form{
label("Name: "){ input(username) }
label("Password: "){ input(password) }
captcha()
submit login() {"Log In"}
}
action login(){
validate(authenticate(username,password),
"The login credentials are not valid.");
message("You are now logged in.");
}
}
define logout(){
"Logged in as " output(securityContext.principal)
form{
submitlink logout() {"Log Out"}
}
action logout(){
securityContext.principal := null;
}
}
define page root(){
login()
" "
logout()
}
init{
var u1 : User :=
User{ name := "test" password := ("test" as Secret).digest() };
u1.save();
}
access control rules
rule page root(){
true
}
When storing a secret property you need to create a digest of it:
newUser.password := newUser.password.digest();
This makes sure the secret property is stored encrypted. A digest can be compared with an entered string using the check method:
us.password.check(enteredpassword)
The default policy is to deny access to all pages and ajax templates, the rules determine what the conditions for allowing access are. Regular templates are accessible by default, however, you can add additional access control rules on templates to limit their accessibility.
A simple rule protecting the editUser page to be only accessable by the user being edited looks like this:
access control rules
rule page editUser(u:User){
u == principal
}
An analysis of this rule:
section some description
.Matching can be done a bit more freely using a trailing * as wildcard character, both in resource name and arguments:
rule page viewUs*(*){
true
}
When more fine-grained control is needed for rules, it is possible to specify nested rules. This implies that the nested rule is only valid for usage of that resource inside the parent resource. The allowed combinations are page - action, template - action, page - template. The next example shows nested rules for actions in a page:
rule page editDocument(d:Document){
d.author == principal
rule action save(){
d.author == principal
}
rule action cancel(){
d.author == principal
}
}
This flexibility is often not necessary, and it is also inconvenient having to explicitly allow all the actions on the page, for these reasons some extra desugaring rules were added. When specifying a check on a page or template without nested checks, a generic action rules block with the same check is added to it by default. For example:
rule page editDocument(d:Document){
d.author == principal
}
becomes
rule page editDocument(d:Document){
d.author == principal
rule action *(*)
{
d.author == principal
}
}
Predicates are functions consisting of one boolean expression, which allows reusing complicated expressions, or simply giving better structure to the policy implementation. An example of a predicate:
predicate mayViewDocument (u:User, d:Document){
d.author == principal
|| u in d.allowedUsers
}
rule page viewDocument(d:Document){
mayViewDocument(principal,d)
}
rule page showDocument(d:Document){
mayViewDocument(principal,d)
}
Pointcuts are groups of resources to which conditions can be specified at once. Especially the open parts of the web application are easy to handle with pointcuts, an example:
pointcut openSections(){
page home(),
page createDocument(),
page createUser(),
page viewUser(*)
}
rule pointcut openSections(){
true
}
Pointcuts can also be used with parameters:
pointcut ownUserSections(u:User){
page showUser(u),
page viewUser(u),
page someUserTask(u,*)
}
rule pointcut ownUserSections(u:User){
u == principal
}
Note that each parameter must be used in each pointcut element, this indicates the value to be used as argument for the pointcut check. A wildcard * can follow to indicate that there may be additional arguments.
A disabled page or action redirects to a very simple page stating access denied. Since this is not very user friendly, the visibility of navigate links and action buttons/links are automatically made conditional using the same check as the corresponding resource. An example conditional navigate:
if(mayViewDocument(securityContext.principal,d)){
navigate(viewDocument(d)){ "view " output(d.title) }
}
When using conditional forms it is often more convenient to put the form in a template, and control the visibility by a rule on the template.
Access Control policies that rely on extra data can create new or extend existing properties. An example of extending an entity is adding a set of users property to a document representing the users allowed access to that document:
extend entity Document{
allowedUsers -> Set<User>
}
Administration of Access Control in WebDSL is done by the normal WebDSL page definitions. All the data of the Access Control policy is integrated into the WebDSL application. An option is to incorporate the administration into an existing page with a template. This example illustrates the use of a template for administration:
define allowedUsersRow(document:Document){
row{ "Allowed Users:" input(document.allowedUsers) }
}
The template call for this template is added to the editDocument page:
table{
row{ "Title:" input(document.title) }
row{ "Text:" input(document.text) }
row{ "Author:" input(document.author) }
allowedUsersRow(document)
}
By using a template the Access Control can be disabled easily by not including the access control definitions and the template. The unresolved template definitions will give a warning but the page will generate normally and ignore the template call.
WebDSL supports a subset of HQL. To escape to WebDSL inside a query, prefix the expression with ~.
// select all updates
var allUpdates : List<Update> := from Update;
// Select all users whose username is "zef"
var username : String := "zef";
var users : List<User> := from User as u where u.username = ~username;
// A paginated for, showing 10 items per page and prefetch the "user" column (in a template)
var page : Int := 0;
for(update : Update in from Update as u left join fetch u.user order by u.date desc limit 10*page, 10) {
output(update.text)
}
Tests for the entities and functions operating on those entities can be defined in test blocks.
Example:
test capitalizeTest {
var u := User{ name := "alice" };
u.capitalizeName();
assert(u.name == "Alice");
}
entity User {
name :: String
function capitalizeName(){
var temp := name.explodeString();
temp.set(0,temp.get(0).toUpperCase());
name := temp.concat();
}
}
The ‘webdsl test appname’ command builds the app and runs the tests, an error will be returned when any of the tests fail. This command will create a new application.ini and uses an in-memory H2 Database Engine.
webdsl test myapp
If the settings in the existing application.ini can be used for testing, run the ‘webdsl check’ command instead.
webdsl check
Multiple test blocks can be defined in an application, each test will run with a fresh initialization of the database, the global variables and global init blocks are handled before each test.
Use assert calls to verify the correctness of results. When the assert argument evaluates to false, the test run will show the location of the failing assert. If the assert consists of one == or != comparison, the two compared results will also be printed upon failure. The assert function has an optional second argument, a String which can be used to pass a message that will be shown when the assert fails.
Signatures:
assert(Bool)
assert(Bool, String)
Testing example: tests for built-in Text type: text.app.
The resulting web application can also be automatically tested by using
webdsl test-web myapp
or
webdsl check-web
where ‘test-web’ will automatically create an application.ini and ‘check-web’ will use an existing one.
These commands start up tomcat and run the tests. Tests are currently expressed by calls to WebDriver with HTMLUnit behind it (a better abstraction specific to WebDSL tests is planned, current web testing is mainly to support testing the compiler). The interface is described here (see Native Class):
https://svn.strategoxt.org/repos/WebDSL/webdsls/trunk/src/org/webdsl/dsl/languages/test/native-classes.str
Example:
test datavalidation {
var d : WebDriver := HtmlUnitDriver();
d.get(navigate(root()));
assert(!d.getPageSource().contains("404"), "root page may not produce a 404 error");
var elist : List<WebElement> := d.findElements(SelectBy.tagName("input"));
assert(elist.length == 4, "expected 4 <input> elements");
elist[1].sendKeys("123");
elist[2].sendKeys("111");
elist[3].click();
var pagesource := d.getPageSource();
var list := pagesource.split("<hr/>");
assert(list.length == 3, "expected two occurences of \"<hr/>\"");
assert(list[1].contains("inputcheck"), "cannot find inputcheck message");
assert(list[2].contains("formcheck"), "cannot find formcheck message");
}
Source:
https://svn.strategoxt.org/repos/WebDSL/webdsls/trunk/test/succeed-web/data-validation/validate-in-elements.app
WebDSL provides ajax operations which allow you to easily replace an element or group of elements in a page without reloading the entire page. These operations can be used as statements inside actions or functions.
Ajax Operations
replace(target, templatecall);
append (target, templatecall);
clear (target);
restyle (target, String);
relocate(pagecall);
refresh();
visibility(target, show/hide/toggle);
runscript(String);
The most commonly used operation is replace, for example:
action someaction() {
replace(body, showUser(user));
}
This will replace the contents of an element or placeholder (see ajax targets section below) that has id ‘body’ with the output of the templatecall showUser(user)
.
runscript
provides a way to interface with arbitrary Javascript code. Put .js files in a javascript directory in the root of your project and include it using includeJS
in a template, e.g. includeJS("sdmenu.js")
.
Example: Moving a div around using JQuery animate:
runscript("$('"+id+"').animate({left:'"+x+"', top:'"+y+"'},1000);");
The refresh action is the default action; when no other interface changing operations are executed the browser will just refresh the current page. For example an input form which submits to a completely empty action results in the data being saved (default behavior) and the page being refreshed (default behavior).
Ajax Targets
There are three ways to target an ajax operation. target can either be
Placeholder with non-ajax default content:
placeholder leftbar { "default content" /* non-ajax default elements */ }
Placeholder with ajax default content:
placeholder leftbar ajaxTemplateCall() /* call to ajax template*/
Id attribute example:
table[id := myfirsttable] { /* non-ajax default elements */ }
When a template is used as target in an ajax operation, it must be declared with the ajax
modifier.
Example:
define testtemplate(p:Person){
placeholder testph{ "no details shown" }
submit("show details",show())[ajax]
action show(){
replace(testph,showDetails(p));
}
}
define ajax showDetails(person:Person){
" name: " output(person.name)
}
Since an ajaxtemplate results in an extra entry point at the server, it must be explicitly opened when access control is enabled:
rule ajaxtemplate showDetails(p:Person){true}
DOM Event Handling
To invoke actions when an HTML event is fired, for example when pressing a key, event attributes can be defined on elements. The syntax of such an attribute is:
<event name> := <action call>
W3schools.com provides an overview of all available events.
Example:
"quicksearch: "
input(search)[onkeyup := updatesearch(search)]
The result is that the updatesearch
action is invoked on the server.
Forms and Ajax
Typically you should not make a form cross an ajax placeholder. The server considers ajax templates as self-contained components similar to pages.
Example of proper usage:
define demo(){
placeholder test()
}
define ajax test(){
form{ input(someGlobal.name) submit action{} {"save"} }
}
Example of incorrect usage (the submit will be contained in a form on the client but not on the server):
define demo(){
form{
placeholder test()
}
}
define ajax test(){
input(someGlobal.name) submit action{} {"save"}
}
In some cases interaction between a regular form and ajax operations is not an issue, e.g. when the ajax template does not contain any input elements. The most common case is rendering validation messages in the form, this behavior is provided in the WebDSL library, see next section.
Ajax Input Validation
There are prebuild library components for creating inputs with ajax validation responses.
A simple example:
define demo(){
var s := "test"
form{
inputajax(s)
submit action{ log(s); } {"log"}
}
}
See Ajax Validation
Styling of WebDSL pages is done using CSS. In the application directory add the following directory and file:
stylesheets/common_.css
This CSS file will be automatically included when deploying the application. Other CSS files can be included using includeCSS
(in this example the included file is located at stylesheets/jquery-ui.css):
includeCSS("jquery-ui.css")
When the application is deployed in the Eclipse plugin you can edit the CSS file directly in the tomcat directory (don’t forget to also save the CSS file back to your project):
WebContent/stylesheets/common_.css
For deployment to external tomcat this directory is:
tomcat/webapps/appname/stylesheets/common_.css
The Firebug add-on for Firefox can be very helpful in figuring out the page structure, other browsers have similar development tools.
Explicit hooks for CSS can be added using the XML embedding:
define someTemplate(){
<div class="mydiv">
"content of mydiv"
</div>
}
Note that the “mydiv” is a WebDSL expression, so this could also be stored in an entity and retrieved using a field access:
<div class=user.cssclass>
Classes for styling can also be added to a template call (separate from the regular arguments):
input[class="mynameinput"](u.name)
If you want to define your own template that takes such extra arguments, use all attributes
:
define page root(){
someOtherTemplate[class="importantdiv"]{ "content" }
}
define someOtherTemplate(){
<div all attributes>
elements
</div>
}
The span
template modifier adds a span around a template, which can then be used as a hook for CSS:
define span spanTemplate(){ "span around me" }
WebDSL supports simple search capabilities through Lucene. Entity properties can be marked as “searchable” to subsequently be indexed:
entity Message {
subject :: String (searchable)
text :: Text (searchable)
sender -> ForumUser
}
The searchable can be applied to the following built-in WebDSL types: String, Text, WikiText, Int, Long, Float and date types. Properties of user defined entity types are currently not supported to be searchable (i.e. sender in the previous example).
If one or more properties of an entity are marked as searchable, a set of searchEntity
functions are generated, in this case:
function searchMessage(query : String) : List<Message>
function searchMessage(query : String, limit : Int)
: List<Message>
function searchMessage(query : String, limit : Int,
offset : Int) : List<Message>
Which can be used from anywhere. For instance on a search page:
define page search(query : String) {
var newQuery : String := query;
action doSearch() {
return search(newQuery);
}
title { "Search" }
form {
input(newQuery)
submit("Search", doSearch())
}
for(m : Message in searchMessage(query, 50)) {
output(m)
}
}
The query syntax adheres to Lucene’s query syntax as does the scoring.
All data to be indexed (properties marked as “searchable”) and queries are analyzed using the default analyzer of Lucene. This means that punctuation and stop words (commonly used words like ‘the’, ‘to’, ‘be’) are stripped from text and text is transformed to tokens in lowercase.
WebDSL stores its search index in the /var/indexes/APPNAME
directory. This is currently not configurable. Make sure this directory is readable and writeable for the user that runs tomcat.
When the searchable
annotations or search mappings are added/changed when data is already in the database, the search index has to be recreated. When your application has been deployed , go to the directory in which it was deployed, for instance /var/tomcat/webapps/yourapp. In this directory you will find a webdsl-reindex
script (for *nix only) which will invoke ant reindex and fixes permissions of the index directory. By default, the reindex task completely reindexes all searchable entities. As of WebDSL version 1.2.9 it also accepts entity names as command line argument (separated by whitespace) to reindex a subset of entities.
Reindex all entities
_*nix_:
# sudo sh webdsl-reindex
Windows
as of 1.3.0:
# ant reindex
before 1.3.0:
# ant reindex -f build.reindex.xml
Reindex a subset of entities
_*nix_:
# sudo sh webdsl-reindex Entity1 Entity2 Etc
Windows:
as of 1.3.0:
# ant reindex -Dentities="Entity1 Entity2 Etc"
before 1.3.0:
# ant reindex -f build.reindex.xml -Dentities="Entity1 Entity2 Etc"
Note: Don’t start your application during reindexing, it will crash because it can’t initialize the directory provider. So reindexing should be done before starting or when already running your application.
A demo of the search functionality can be seen on Reposearch.
note: Some syntax changes and additional features are expected for WebDSL 1.3.0. The search language will become more structured. This manual will be updated/completed upon 1.3.0 release
WebDSL offers full text search engine capabilities based on Apache Lucene and Hibernate Search. Current implementation supports:
Using search in WebDSL starts by marking which entities need to be searchable. If one property is marked searchable, the entity can be searched. For each entity property one or more search fields can be specified. There are 2 ways to specify these: using search mappings or using searchable annotations. For simple search functionality, searchable annotations will suffice, but for cleaner code we recommend using search mappings.
A search mapping starts with the name of the property to be indexed, optionally followed by mapping specifications:
name
Override the default search field name
. Default: property name
analyzer
Indexed using analyzer analyzer
instead of the default analyzer.
Float
|^Float
Search field is boosted to Float
at index time (default 1.0).
Indicate that this search field can be used for spell checking/autocompletion.
entity
In case marking an reference/composite property as searchable, you might want to make only a specific subclass of the property type searchable.
Int
|with depth Int
In case marking an reference/composite property as searchable, you can specify the depth of the ‘embedded’ path, 1 is the default.
mapping specification
Prefix a mapping specification
with the plus sign if you want this search field to be used by default at query time. If no default search field is specified, all search fields are used by default.
Search mappings belong to an entity and can be placed inside an entity declaration, or somewhere else by adding the entity name. Names of the search fields are scoped to entities, so different entities may share the same names for search fields.
//Embedded search mapping
entity Message {
subject :: String
text :: Text
category:: String
sender -> User
search mapping {
+subject
+text using snowballporter as textSnowBall
text
category
+sender for subclass ForumUser
}
}
//External search mapping
entity ForumUser : User {
forumName :: String
forumPwd :: Secret
messages -> Set<Message> (inverse=Message.sender)
}
...
search mapping ForumUser {
forumName using none
}
Search fields can also be specified using property annotations:
//Using searchable annotations
entity Message {
subject :: String (searchable)
text :: Text (searchable, searchable(name=textSnowBall, analyzer=snowballporter)
category:: String (searchable)
sender -> ForumUser (searchable())
}
The above code marks the entity Message
searchable, and it has 3 search fields: subject
, text
using the default analyzer, and textSnowball
, which uses the snowball porter analyzer. Searchable annotations have no restriction w.r.t. search mappings, and both can be used interchangeably (not recommended since it’s less transparent). The following table shows the annotation equivalent of specifications in search mappings.
search mapping | <-> | searchable annotation |
subject | <-> | searchable searchable() |
subject as sbj | <-> | searchable(name = sbj) |
subject using defaultNoStop | <-> | searchable(analyzer = defaultNoStop) |
subject^2.0 | <-> | searchable()^2.0 |
subject boosted to 2.0 | <-> | searchable(boost = 2.0) |
subject as sbjTriGram using trigram boosted to 0.5 | <-> | searchable(analyzer = trigram, name = sbjTriGram)^0.5 |
subject as sbjUntokenized using none | <-> | searchable(analyzer = none, name = sbjUntokenized) |
message as sbjAC using kwAnalyzer (autocomplete) | <-> | searchable(analyzer = kwAnalyzer, name = sbjAC, autocomplete) |
user as forumuser for subclass ForumUser | <-> | searchable(name = forumuser, subclass = ForumUser) |
user with depth 2 | <-> | searchable(depth=2) |
+ text as txt | <-> | searchable(name = txt, default) |
Properties of any type can be made searchable, although there are some notes to make.
These properties don’t contain any text or value by themselves, but hold references to other entities. Therefore, the properties themselves cannot be indexed, but the searchable properties of the referred entity/entities will be indexed in the scope of the current entity. For example if you want to be able to search for Message
entities by the name of the sender (in the above example), the property forumName
of ForumUser
needs to be indexed in the scope of Message
. This can be done by marking the sender
property as searchable. All search fields from ForumUser
will then be available for Message
, and searchfields are prefixed with ‘propertyName.
’ by default (or different name if specified using as
in search mappings). The search field from the example becomes : sender.forumName
.
Note: Searchable reference/composite properties need to be part of an inverse relation to keep the index of the owning entity updated with changes in its reference entity/entities. The mapping options available for reference properties are restricted to name
and subclass
.
In case no analyzer is specified for a numeric property search field, it will be indexed as numeric fields, which is a special type of field in Lucene. It enables efficient range queries and sorting on this field.
Derived properties are currently only indexed when the entity owning this property is saved/changed.
By default, textual properties will use the default analyzer from Lucene, which is optimized for the English language. In the specification of a search field (in search mapping or searchable annotation), a different analyzer can be assigned to it like is done for the textSnowBall
search field. A custom analyzers can be declared, each containing:
The range of tokenizers and filters that are supported can be found here and here (with more information about specific analyzers). You don’t need to use the factory keyword at the end. Useful analyzers definitions are already included in a new WebDSL project under ./search/searchconfiguration.app. The default analyzer can be overwritten by adding the default
keyword before analyzer
. More advanced analysis may require different behavior at search and query time. Using the index { ... }
and query { ... }
block, the analyzers may be specified different for indexing and query time (see the synonym analyzer).
For each indexed entity, search functions and a searcher class are automatically generated. For simple searches, the generated functions will suffice. For more advanced searches, the magic is in the generated entity searcher(s).
For the example entity Message
, the following search functions are generated.
function searchMessage(query : String) : List<Message>
function searchMessage(query : String, limit : Int)
: List<Message>
function searchMessage(query : String, limit : Int,
offset : Int) : List<Message>
The limit and offset parameters can be used for paginated results. It only loads at most the limit
number of results from the database (for efficiency/faster pageloading). These functions use the default search fields when searching, and the specified analyzers are applied for each search field.
More features are available when using WebDSL’s search language designed to perform search operations. The language let you interact with the generated Searcher object for the targeted entity. A reference to (or initialization of) a searcher is followed by one or more constructs in which search criteria can be declared.
//matches Messages with "tablet", but without "ipad"
var msgSearcher := search Message matching +"tablet", -"ipad";
//enable faceting on an existing searcher
msgSearcher := ~msgSearcher with facets sender.forumName(20), category(10)
List of search language constructs:
var searcher := search Book matching author: "dahl";
var results := searcher.results(); //returns List<Book>;
Calling .results() on a searcher returns the search results. Calling .count() on a searcher returns the total number of results.
searcher := search Entity matching title: "user interface";
searcher := search Entity matching title, description: userQuery;
searcher := search Entity matching "user interface";
searcher := search Entity matching title: +userQuery, -"case study";
searcher := search Entity matching ranking:4 to 5, title:-"language";
Declares a searcher that matches a simple or boolean query. Fields are optional: if the query expression is not preceded by a field constraint, the default search fields are used (i.e. all search fields if no default fields are defined, see …). qExp can be any String compatible WebDSL expression or a range expression optionally prefixed with a boolean operator (+ for must, - for mustnot, nothing for should).
searcher := search Entity matching rating: {1 to 3}
searcher := search Entity matching rating: [startDate to endDate]
searcher := search Entity matching rating: -[* to sinceDate]
Range expressions are in the form [minExp to maxExp] (including min and max value) or {minExp to maxExp} (exludes min and max, where both expressions can be any expression of a simple WebDSL builtin type. An open range is specified with an asterisk : [* to “A”} for example.
var searcher := search Book matching author: "dahl" start 20 limit 10
With the start and limit keywords, you can control which results to be retrieved.
searcher := search Entity on title: q [no lucene, strict matching];
Declare the searcher’s options. Available options are:
searcher := search Entity matching title: "graph"
with filter hidden:false;
Specify a filter constraint. A filter constraint is a field-value expression. Be aware that when using a filter, a bitset is constructed and cached to accelerate future queries using the same filter. Filters are not considered in result ranking. Thus, only use field-value filters if you expect the same filtering to occur frequently.
Example:
searcher := search Entity matching title: "graph" with facet author(10);
searcher := search Entity matching title: "graph" with facets author(10), rating([* to 1],[2 to 3},[3 to 4},[4 to *]);
Specify enabled facets. These can be discrete or range facets
facets := author facets from s;
Returns a list: List with the facets for the specified field. Facet objects have the following boolean functions available, for example to apply different styling on the variety of facet states:
searcher := ~searcher with filter(s) selectedDateFacet.must(), selectedPriceFacet.must();
Previously returned facets can be used to narrow the search results. The behaviour of the facet (must, should, mustnot) can be set on the facet object itself (should by default).
searcher := search Entity matching title: "graph" in namespace "science";
When using search namespaces, restricting a search to a single namespace is done using the in namespace construct followed by a String-compatible expression.
The searcher class that is created for the example Message
entity is MessageSearcher
. The first advantage of using this searcher instead of the generated functions is the ability to interact with the searcher, for further refinements to the search query, or to get information like the total number of results, or time that was needed to perform the search.
define page searchPage(query : String) {
var searcher := MessageSearcher().query(query);
var results := searcher.results();
var searchTime := searcher.searchTime(); //String
"You searched for '" output(searcher.query()) "', " output(searcher.count()) " results found in " output(searchTime) "."
if(searcher.count() > 0) {
showResults(results)
}
}
define showResults(results : List<Message>) {
//code to view results
}
The available searcher functions generated for each searchable entity are:
(see here)
allowLuceneSyntax(allow : Bool) : EntitySearcher
OR is the default.
defaultAnd() : EntitySearcher
defaultOr() : EntitySearcher
addFieldFilter(field : String, value : String) : EntitySearcher
getFieldFilterValue(field : String) : String
getFilteredFields() : List<String>
removeFieldFilter(field : String)
clearFieldFilters()
The field(s) parameters specify which search field(s) to use for suggestions. ‘limit’ controls the max number of suggestions to retrieve. Additionally the namespace can be specified, if used. For spell suggestions the accuracy [0..1] can be set
static autoCompleteSuggest(toComplete : String, field : String, limit : Int) : List<String>
static autoCompleteSuggest(toComplete : String, namespace : String, field : String, limit : Int) : List<String>
static autoCompleteSuggest(toComplete : String, fields : List<String>, limit : Int) : List<String>
static autoCompleteSuggest(toComplete : String, namespace : String, fields : List<String>, limit : Int) : List<String>
static spellSuggest(toCorrect : String, fields : List<String>, accuracy : Float, limit : Int) : List<String>
static spellSuggest(toCorrect : String, namespace : String, fields : List<String>, accuracy : Float, limit : Int) : List<String>
static spellSuggest(toCorrect : String, field : String, accuracy : Float, limit : Int) : List<String>
static spellSuggest(toCorrect : String, namespace : String, field : String, accuracy : Float, limit : Int) : List<String>
boost(field : String, boost : Float) : EntitySearcher
The max
parameter defines the maximum facets to collect for that field. For range facets, the ranges are encoded as String in the same format as range queries. Multiple ranges can be specified concatenated, optionally seperated with a symbol like white space or comma but that’s not required."
enableFaceting(field : String, max : Int) : EntitySearcher
enableFaceting(field : String, rangesAsString : String) : EntitySearcher
getFacets(field : String) : List<Facet>
addFacetSelection(facet : Facet) : EntitySearcher
addFacetSelection(facets : List<Facet>) : EntitySearcher
getFacetSelection() : List<Facet>
getFacetSelection(field : String) : List<Facet>
removeFacetSelection(facet : Facet) : EntitySearcher
clearFacetSelection() : EntitySearcher
clearFacetSelection(field : String) : EntitySearcher
field(field : String) : EntitySearcher
fields(fields : List<String>)] : EntitySearcher
setOffset(offset : Int) : EntitySearcher
setLimit(limit : Int) : EntitySearcher
Highlight matched tokens using the analyzer from the specified search field in a given text, optionally specifying a pre- and posttag (bold by default), number of fragments, fragment length and fragment separator. There are 4 types of highlight methods. Replace highlight with the version that is suitable for you:
highlight(field : String, toHighlight : String) : String
highlight(field : String, toHighlight : String, preTag : String, postTag : String) : String
highlight(field : String, toHighlight : String, preTag : String, postTag : String, nOfFrgmts : Int, frgmtLength : Int, frgmtSeparator : String) : String
Just like an ordinary query, first specify the fields using the field(s)
function
moreLikeThis(text : String) : EntitySearcher
Note: Query text from the first specified query is returned in case multiple queries are combined using boolean queries.
getQuery() : String
query(queryText : String) : EntitySearcher
sortDesc(field : String) : EntitySearcher
sortAsc(field : String) : EntitySearcher
clearSorting() : EntitySearcher
range(start, end) : EntitySearcher
range(start, end, includeMin : Bool, includeMax : Bool) : EntitySearcher
setNamespace(ns : String) : EntitySearcher
getNamespace() : String
removeNamespace() : EntitySearcher
results() : List<Entity>
count() : Int
searchTime() : String
searchTimeMillis() : Int
searchTimeSeconds() : Float
Filters are an efficient way to filter search results, because they are cached. If you expect to perform many queries using the same filter (like only showing Message
s in a specific category), using a filter is the way to go:
MessageSearcher.query(userQuery).addFieldFilter("category","humor")
or
search Message matching userQuery with filter category:"humor"
To get the value of a previously added field filter, use the getFieldFilterValue(field : String)
method.
Search namespaces become usefull if you want to allow searches on entities with some specific property value. For example searching Message
s by category in the above example. Namespaces have some advantages over using field filters. An index is created for each namespace separately, instead of one for all entities of that type. Since the indexes are used as input for auto completion and spell checking, the use of namespaces enables suggestion services scoped to one, or all, namespace(s).
Facets can be displayed in many contexts. For example, when displaying a list of products, you want the product categories to be displayed as facets. Any searchable property can be used for faceting. The values, as they appear in the search index, are used for faceting. So if you use the default analyzer for the category property of Product, categories containing white spaces are not treated as single facet value. For this to work you need to define an additional field which doesn’t tokenize the value of the property, for example by indexing this property untokenized:
entity Product{
name :: String
categories -> Set<Category> (inverse=Category.products)
search mapping{
name
categories
}
}
entity Category {
name::String
products -> Set<Product>
search mapping{
name using none //or 'name using no' in v1.2.9.0
}
}
Facets can be retrieved through the use of a searcher. You first need to specify the facets you want to use by enabling them in the searcher. A typical example is to display facets in the search results:
(updated April 5th)
define searchbar(){
var query := "";
form {
input(query)
submit action{
//construct a searcher and enable faceting on tags.name, limited to 20 top categories
//more facets can be enabled by separating the field(topN) facet definitions by a comma
var searcher := search Product matching query with facets categories.name(20);
return search(searcher);} {"search"}
}
}
define page search(searcher : ProductSearcher){
var results : List<Product> := results from searcher;
var facets : List<Facet> := categories.name facets from searcher;
header{"Filter by product category:"}
for(f : Facet in facets){
facetLink(f, searcher)
}separated-by{" "}
showResults(results)
}
define facetLink(facet: Facet, searcher: ProductSearcher){
submitlink narrow(facet){ if(facet.isSelected()){"+"} output(facet.getValue()) }"(" output(facet.getCount()) ")"
action narrow(facet : Facet){
if (facet.isSelected()) { searcher.removeFacetSelection(facet); } else { ~searcher matching facet.must(); }
goto search(searcher);
}
}
Collaborative filtering recommendations are supported in the WebDSL language using the Mahout framework.
Collaborative filtering systems are content agnostic. These systems focus on the relationships between users and items and not directly on the content properties of their corresponding entities. From here onwards we refer to users and items as general entity types that will be related, this is further discussed below.
One example is a book store where consumers buy books, so in this case the consumer is the user and the book is the item. Users have preferences for items, but its up to you whether you take those into account. The preference value can be expressed on a scale that matches your needs, for example 0 to 1 or 1 to 5, as long as it’s a number.
entity Customer {
name :: String(id)
books -> List<BookPurchase>
}
entity Book {
isbn :: String(id)
author :: String
title :: String
}
entity BookPurchase {
by -> Customer
book -> Book
pref :: Int
}
recommend BookPurchase {
user = by
item = book
value = pref
}
For Mahout these preferences indicate which books are very popular and which are less. Such preferences are not obligatory but if implemented can be used to order the recommendations based on popularity.
Another example are friend relations where you can recommend users to users based on their current network of users. In this case the item is also the user type. The opposite holds as well. You could have books that recommend other books. This is useful if you do not know the users preferences, for example with an unidentified visitor. In this case you talk about item to item relations. These relations change less often and therefore require less calculations for Mahout on a regular basis.
Users need to have items in common in order for there to be recommendations. Otherwise in statistical terms the precision of the relationships can not be calculated. Based on solely the link between users and items you could already express preferences. If you don’t know the user, it is still possible to obtain recommendations based on the current item that is shown. In this case it is an item to item relation. Otherwise if you do know the user and he or she has one or more relations to items, you could generate a list of recommendations based on his or her preferences.
In order to express the relation between a user and his or her items you need to have a special entity. This entity holds both the original user and the item and might have a preference value. As a developer you could add other information too, for example the date when the relation was created. A list of relation items is stored in the user entity, as shown in the book example above. If the preference value is not added to the relation entity, the recommendation system automatically expects a boolean relation.
The name of the relation entity is used as the name to call the recommendation system.
recommend RelationEntityName {
user = by
item = book
value = pref
algorithm = Loglikelihood
neighborhoodalg = NUser
neighborhoodsize = 9
type = Both
schedule = 1 weeks
}
The recommendation block has two obligatory values. These specify which variable in the relation entity holds the reference to the user and the item instance. So in above’s book store example the name of the recommendation block matches the name of the relation, which is BookPurchase. The user and item variables are set to by and book respectively. The user and item values:
user: The variable name that references the User instance inside the relation entity.
item: The variable name that references the Item instance inside the relation entity.
The other optional values are listed here:
algorithm: Can be of value: Euclidean, Pearson, Loglikelihood, or Tanimoto. This specifies which algorithm the Mahout framework should use in order to build up a list of recommendations. The default value is: Loglikelihood. Please be aware that the precision of the recommendation is largely determined by the algorithm that is chosen, go to the website of the Mahout framework to read more about the benefits and drawbacks of these algorithms. Note: In case you want to test the precision of your choice and determine the related speed have a look at the test procedures discussed further below.
neighborhood size: Can be any non-null numeric value. This specifies the size of the neighborhood that Mahout needs to look into find other recommended items.
The default value is: 9.
neighborhood algorithm: Can be of value: NUser or Threshold. This specifies which algorithm should be used to determine the neighborhood in the dataset. This can have a major effect on the precision of the recommendations returned by the recommendation system.
The default value is: NUser
type: Can be of value: User, Item, or Both. This specifies whether you want to use the recommendations for User-to-Item relation, Item-to-Item relation, or even both. It is highly recommended to set the flag on the most limited form that you need, this will drastically decrease the amount of time required to compute the recommendations.
The default value is: Both
schedule: This can be any time interval value as defined fully at Recurring Tasks. This defines at what interval the recommendation system should rebuild the mahout index in the background.
NOTE: Do not set this value less than the time it would take to compute the index once.
The default value is: 1 weeks
NOTE on scheduling Scheduling through the definition of the recommendation is not supported yet, instead use the default method of WebDSL to schedule the reconstruct function every x days. For example:
invoke RelationEntity.reconstructRecommendationCache() every 2 days
After configuring the recommendation system you can implement it in the code using the name of the relation entity. In the above mentioned book store example this is the BookPurchase entity name. All the recommendation functions for this relation are accessible through the corresponding relation entity as if it were a static function of that entity class.
There are two ways to get recommendations, which method you need depends on the situation you are in. In the most common situation you know who the user is, and you want to get recommendations based on the items he / she already likes. In this case we talk about the user-to-item recommendations. In case you do not know the user, but you know which item they are interested in you could generate a list of recommendations too. The latter case is called item-to-item recommendation. These two methods require a different function call to the Recommendations system.
The user-to-item recommendations are accessible by calling to the method:
RelationEntity.getUserRecommendations(user : UserType, maxNum : Int) : List<ItemType>
RelationEntity.getUserRecommendations(user : UserType) : List<ItemType>
This method returns a List as its result and could be filtered further or simply looped through as any normal list. The user argument refers to the user for which you want to obtain recommendations. The maxNum argument is the maximum number of recommendations that you want to obtain, this argument is optional and by default set to 10.
The item-to-item recommendations are very similar to the user-to-user recommendation methods. The major difference between these function calls is the fact that you supply an item to the function so you could get a list of recommendations based on that single item. The item-to-item methods are:
RelationEntity.getItemRecommendations(item : ItemType, maxNum : Int) : List<ItemType>
RelationEntity.getItemRecommendations(item : ItemType) : List<ItemType>
Just like the user-to-item recommendations these functions result in a List<ItemType . The item argument refers to the item that you want to use as a reference to obtain recommendations. The maxNum argument is the maximum number of recommendations that you want to obtain, this argument is optional and by default set to 10.
Determining the recommendations takes a while to compute, this is dependent on several factors including the size of the data set, the type of relations (binary or value based), if there are duplicate relations possible, and several configuration options. The configuration options that play a major role here are the algorithm type, the neighborhood size, and its related neighborhood algorithm. With the following function call you can determine how long the last operation of the recommendation system took in milliseconds:
RelationEntity.getLastExecutionTime() : Int
The precision and recall test is used to determine the precision of the recommendations. In other words, with this test you can determine the quality of the recommendations that will be given back. The precision and recall performance of the recommendation system depends on the algorithm used and the type of network you want it to analyze.
NOTE: Do not use this function on production systems, as it takes a while to compute and does not add any value on a live production machine. It should only be used to determine the type of algorithm that performs best during the development phase.
REQUIREMENT: In order for the precision and recall method to return valuable results you need to test a network that is filled with data as it would be when used in production. Compared to a real data set a random data set could falsely state that the performance of algorithm x is very good, while you should use algorithm y.
RelationEntity.evaluateIRStats() : String
The precision and recall float values are both encapsulated in a string returned by this function, for example: “0.2714285714285714 :: 0.2857142852714785”
With the schedule parameter of the recommend configuration you can set when the mahout index should be rebuilt as discussed before. However, if you want to build the mahout index manually that is possible too. The function you need to call in order to reconstruct the index is:
NOTE: Do not use the manual function on production systems as it can take several hours to compute and would block all the open connections to the website in the meanwhile.
RelationEntity.reconstructRecommendationCache() : void
This page describes how to create an email template and send email from your application. Make sure the email settings are configured in application.ini, see Application Configuration. If you are interested in storing email addresses in an entity, have a look at the Email type page.
Defining an email template:
define email testemail(us : User) {
to(us.mail)
from("webdslorg@gmail.com")
subject("Test Email")
par{ "Dear " output(us.name) ", " }
par{
"Look at your profile page: "
navigate(user(us)){"go"}
//navigate will become an absolute link in the email
}
}
Sending email:
email testemail(someuser);
The actual sending happens asynchronously, if there are issues while the application is trying to send an email, it will retry that email after 3 hours. If necessary, you can inspect and influence this email queue through the QueuedEmail
entity:
entity QueuedEmail {
body :: String (length=1000000)
//Note: default length for string is currently 255
to :: String (length=1000000)
cc :: String (length=1000000)
bcc :: String (length=1000000)
replyTo :: String (length=1000000)
from :: String (length=1000000)
subject :: String (length=1000000)
lastTry :: DateTime
}
Recurring task allow you to execute a certain function in set interval, e.g. every minute, 5 hours or every week. For this WebDSL uses the following syntax (which is subject to change):
function someFunction() {
log("I was executed!");
}
invoke someFunction() every 5 minutes
If the called function returns anything, this value is discarded. Functions invoked in this manner have access to entities and global variables, but not session data (because the function is not invoked by a user).
Syntax of the time intervals:
TimeInterval = TimeIntervalPart*
TimeIntervalPart = Exp "weeks"
TimeIntervalPart = Exp "days"
TimeIntervalPart = Exp "hours"
TimeIntervalPart = Exp "minutes"
TimeIntervalPart = Exp "seconds"
TimeIntervalPart = Exp "milliseconds"
So valid time intervals are:
1 hours // note the plural
1 hours 10 minutes // repeat every 70 minutes
2 weeks 10 milliseconds
note: this section is outdated, it is recommended to configure HTTPS on the Nginx or Apache Httpd in front of tomcat, using HSTS policy to force all traffic over HTTPS
Using https requires some extra configuration when deploying to an external tomcat server, the tomcat instance used in the plugin and command-line test and run commands is already configured (note: this uses a dummy configuration which should not be used in production deployment of the app). Follow these steps to configure Tomcat 6:
Run this command and follow the instructions (note down the password):
%JAVA_HOME%\bin\keytool -genkey -alias tomcat -keyalg RSA
Then, in tomcat/conf/server.xml add (use the password entered in the keytool):
<Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"
maxThreads="150" scheme="https" secure="true"
keystoreFile="${user.home}/.keystore" keystorePass="--password--"
clientAuth="false" sslProtocol="TLS" />
Read more about this topic here: http://tomcat.apache.org/tomcat-6.0-doc/ssl-howto.html
WebDSL includes a simple way to define string and JSON-based webservices.
The JSON interface is defined as follows:
native class org.json.JSONObject as JSONObject {
constructor()
constructor(String)
get(String) : Object
getBoolean(String) : Bool
getDouble(String) : Double
getInt(String) : Int
getJSONArray(String) : JSONArray
getJSONObject(String) : JSONObject
getString(String) : String
has(String) : Bool
names() : JSONArray
put(String, Object)
toString() : String
toString(Int) : String
}
native class org.json.JSONArray as JSONArray {
constructor()
constructor(String)
get(Int) : Object
getBoolean(Int) : Bool
getDouble(Int) : Double
getInt(Int) : Int
getJSONArray(Int) : JSONArray
getJSONObject(Int) : JSONObject
getString(Int) : String
length() : Int
join(String) : String
put(Object)
remove(Int)
toString() : String
toString(Int) : String
}
Example use in WebDSL:
function myJsonFun() : String {
var obj := JSONObject("{}");
obj.put("name", "Pete");
obj.put("age", 27);
return obj.toString();
// Will return '{"name": "Pete", "age": 27}'
}
A service is simply a WebDSL function that uses the service
keyword instead of function
, you don’t have to specify a return type, it will convert anything you return to a string (using .toString()
):
entity Document {
title :: String (id, name)
text :: Text
}
service document(id : String) {
if(getHttpMethod() == "GET") {
var doc := findDocument(id);
var json := JSONObject();
json.put("title", doc.title);
json.put("text", doc.text);
return json;
}
if(getHttpMethod() == "PUT") {
var doc := getUniqueDocument(id);
var json := JSONObject(readRequestBody());
doc.text := json.getString("text");
return doc.title;
}
}
services are mapped to /serviceName
, e.g. /document
. Here’s a few sample requests to test (note, these are services part of an application called “hellojson”):
$ curl -X PUT 'http://localhost:8080/hellojson/document/my-first-doc' \
-d '{"text": "This is my first document"}'
my-first-doc
$ curl http://localhost:8080/hellojson/document/my-first-doc
{"text":""This is my first document","title":"my-first-doc"}
But, like pages, services can also have entities as arguments:
service documentJson(doc : Document) {
var obj := JSONObject();
obj.put("title", doc.title);
obj.put("text", doc.text);
return obj;
}
The following zip file contains a WebDSL and a Mobl project, the WebDSL application provides data through a web service to the Mobl application:
http://webdsl.org/examples/mobl-webdsl-example.zip
How to use this example:
0 Get the Eclipse zip from the WebDSL site (WebDSL in Eclipse), which contains the WebDSL and Mobl plugins.
1 Import both projects using: ‘file -> import -> exising projects into workspace’ (the testapp WebDSL project will automatically compile and deploy)
2 Open testmobl.mobl, add a space somewhere and save the file to trigger a compilation of the Mobl project.
3 In the WebDSL project, right-click import-mobl.xml and select ‘run as->ant build’. This simply copies the content of the Mobl project www directory to the WebContent directory of the WebDSL project, which contains the deployed application.
4 Go to the browser and click the link ‘show mobile page’.
Note that you will need to update the project names in ‘import-mobl.xml’ when copying it to another project, and you should copy the ‘projectname import-mobl.xml.launch’ file as well.
WebDSL provides possibility to generate code for a synchronization framework combined with mobl. Nevertheless, the webservices are open to be used by other applications.
The steps that are required for the synchronization framework are the following:
The framework is meant as an extension to a WebDSL application. This means that it requires at least a full model in the application to generate a working framework.
The synchronization framework requires and allows some adaption by additional settings. Those settings can be added in the synchronization settings for each entity.
entity Example{
synchronization configuration{
}
}
The following settings can be configured:
The data synchronization requires a Toplevel Entity to enable the data partitioning. This is simple a flag that specifies that objects of this type represent a data partition. Additionally, this setting requires a String property that can be used to represent this object.
entity Car{
registrationIdentifier :: String
synchronization configuration{
toplevel name property : registrationIdentifier
}
}
The data synchronization framework enable external sources to read and modify data on the server with the web application. The framework allows control over which data can be accessed by who. This can only be specified when the a principal is defined in the web application. There are three different levels that can be specified for each entity: read, write and create. It is recommended to specify those rules for each entity.
entity Dummy{
name :: String
synchronization configuration{
access read: true
access write: Logedin()
access create: principal().isAdmin()
}
}
The last setting that can be configured is that of restricted properties. It allows to simplify the data model that you want to use on synchronization. The properties that are specified in this configuration are removed from the shared data and also for the calculation of data partitioning.
entity Person{
surName :: String
firstName :: String
fullName :: String
synchronization configuration{
restricted properties : surName, firstName
}
}
Generation of the framework is easy. After specifying the settings, open the main application file in the IDE. Then select the generate synchronization framework from the Transform menu.
The framework is generated in the folder webservices. To enable the synchronization framework inside the web application you need to include the main file of the framework.
application TestApp
imports webservices/services/interface
The framework generates code for mobl that enable synchronization in a mobl application. However, it still needs a full mobile application.
Other applications can use the available webservices to synchronize with the application.
The framework generates a lot of files, but what does it contain:
The core of the synchronization contains functions that overlook the main functionality of the synchronization. Identification, detection and resolution of updates.
The webservices are used for communication with mobl of other applications. This is a layer on the core of the synchronization. All services are called by post request to the url:
http://<websiteurl>/webservice/<webservicename>
The following services are available and should be used in that order:
The mappers are meant for mapping the updates to local values. The also have some additional statements for checking validity of the input. There are two mappers, one for modification and one for creation. Currently, they contain the same code. This is done so they can be overwritten separately.
The values in the database are not in a format that can be send through webservices. Therefor, the framework has 3 functions for each entity
The synchronization framework uses data partitioning to reduce the amount of data for mobile applications. This solution chooses to use object relations to determine if objects are linked to the TopLevel entity. This requires that each entity has a function to calculate the related objects.
The main function of data partitioning gets a closure of a data partitioning by calling the related functions until there are no new objects any more.
As mentioned before you can specify three rules for the access control of objects. Those rules are turned into functions named:
The access control requires that remote applications can login to the application. To improve security a device can register itself and get a devicekey. Which then can be used to authenticate instead of using the password. Those keys are stored as an additional property of the principal and if removed the device is de-authenticated.
A big part of the data synchronization is about the model. The model is basically a copy of that of WebDSL only with other mobl types. It should also be used for developers that try to understand what model is expected for the webservices. The following sections are additional notes to the creation of the model.
Mobl has a more restricted set of types. This let to the choice to not support all types. properties with the following types are removed:
Mobl does not support class Hierarchy. To support all entities from the application the synchronization framework has flatten the hierarchy. The influence can be found in the renamed properties that now have a prefix of there original class name. And a additional property that tells the actual type: Typefield
The data partitioning requires some additional information. This is stored in the property sync and lastSynced.
The search annotations in mobl are expensive and better can be removed from the model.
Mobl also needs some mappers of the values. However the limited difference between mobile and JSON representation, allows it to use the function generated by the mobl compiler.
There are some integration functions for mobl that can be used to call synchronization processes. It can be seen as the core of the synchronization for mobl applications.
The authentication are some functions to enable the devicekey setup. It has the following functions:
The logging out of the device also cleans the database for security reasons.
As a start the generated framework has a data browser included to have easy start with the application.
It has a page for every entity, namely: showSimple
Those pages allow to click through the data stored locally.
The send version numbers of each objects can be used to change the protocol of resolution of outdated objects. Giving it an high number will interpret that the object is newer than that of the system.
mobl doesn’t have difference between set and lists, it only supports collections. The biggest problem is that the ordering can not be trust.
This section gives some practical tips on working with MySQL.
mysqldump -u root --single-transaction dbname > mydump.sql
Optionally exclude less important tables:
mysqldump -u root myapplication > dump.sql --single-transaction --ignore-table=myapplication._SecurityContext --ignore-table=myapplication._RequestLogEntry
mysql -u root dbname < mydump.sql
When loading large MySQL dumps (for local testing), convert them to MyISAM (lack of transactions makes it unusable for production db, but it loads a lot faster due to lack of foreign key checks).
Before loading the dump, increase these settings in /etc/my.cnf and restart MySQL to load the changed settings:
key_buffer_size=1024m
max_allowed_packet=1024m
mysqladmin -u root shutdown
mysqld -u root &
Then run
cat mydump.sql | sed s/ENGINE=InnoDB/ENGINE=MyISAM/ | mysql -u root
Or, if you want to create an intermediate file with the MyISAM dump first (slower):
sed s/ENGINE=InnoDB/ENGINE=MyISAM/ mydump.sql > mydump.sql.myisam
mysql -u root dbname < mydump.sql.myisam
Show status of InnoDB:
mysql -u root
show innodb status;
See what MySQL is doing (e.g. expensive query):
show processlist;
Check the current structure of a table, including foreign key constraints. This can be helpful in resolving issues caused by db mode ‘update’, which only adds columns but will not change an existing column:
show create table _Alias;
We use the following settings for MySQL on our production server (NixOS/Linux):
[mysqld]
key_buffer_size = 256M
max_allowed_packet = 64M
sort_buffer_size = 2M
read_buffer_size = 2M
myisam_sort_buffer_size = 64M
query_cache_size = 128M
max_connections = 250
[mysqldump]
max_allowed_packet = 16M
[isamchk]
key_buffer = 256M
sort_buffer_size = 256M
[myisamchk]
key_buffer = 256M
sort_buffer_size = 256M
See Download section.
Below are larger examples of WebDSL applications than those found on the other manual pages.
For explanation of access control elements, see Access control manual section
application minimalac
entity User {
name :: String
password :: Secret
}
init{
var u := User{ name := "1" password := ("1" as Secret).digest() };
u.save();
}
define page root(){
authentication()
" "
navigate protectedPage() { "go" }
}
define page protectedPage(){ "access granted" }
principal is User with credentials name, password
access control rules
rule page root(){true}
rule page protectedPage(){loggedIn()}
A simple application that shows the public timeline of Twitter using Twitter4J. Example project code is available here
Screenshot of result:
Project files:
WebDSL application with native class interface declaration:
application exampleapp
define page root() {
output(TwitterReader.read(null,null))
}
native class nativejava.TwitterReader as TwitterReader {
static read(String,String):List<String>
}
Implementation of TwitterReader.java:
package nativejava;
import java.util.*;
import twitter4j.Status;
import twitter4j.Twitter;
import twitter4j.TwitterException;
import twitter4j.TwitterFactory;
public class TwitterReader{
public static List<String> read(String twitterID,String twitterPassword){
//The factory instance is re-useable and thread safe.
Twitter twitter = new TwitterFactory().getInstance(twitterID,twitterPassword);
List<String> result = new LinkedList<String>();
List<Status> statuses;
try {
statuses = twitter.getPublicTimeline();
for (Status status : statuses) {
result.add(status.getUser().getName() + ":" + status.getText());
}
} catch (TwitterException e) {
e.printStackTrace();
}
return result;
}
}
This example will show a complex form that uses Ajax for custom data validation.
The full project source of this example is located here:
https://svn.strategoxt.org/repos/WebDSL/webdsls/trunk/test/succeed-web/manual/ajax-form-validation/
Inverse annotations can cause problems due to save cascading in WebDSL, if an inverse is made with an entity in the database, then your temporary entity will be automatically saved in the database as well.
Don’t use WebDSL’s data validation described on the Validation page in combination with this example, because validation is done with custom code here. Data validation will be integrated with ajax to more easily get the result that is implemented in this example.
We’re going to create an edit page for a Person
entity:
entity Person {
fullname :: String
username :: String (name)
parents -> Set<Person>
children -> Set<Person>
}
The name annotation indicates that the username
is used to refer to the Person entity in select inputs, such as those for the parents
and children
property, see name property page.
The root
page includes a personedit
template and passes it a new Person
object.
define page root() {
main()
define body() {
personedit(Person{})
}
}
The personedit
template provides the form that checks values whenever changes occur. The various placeholder
elements provide a location to insert error messages. The actual checks are encapsulated in functions, this allows the save action to easily do a server-side check before saving the new Person object.
define personedit(p:Person){
form{
par{
label("username: "){ input(p.username)[onkeyup := action{ checkUsername(p); checkUsernameEmpty(p); checkFullname(p); }] }
placeholder pusernameempty { }
placeholder pusername { }
}
par{
label("fullname: "){ input(p.fullname)[onkeyup := action{ checkFullname(p); checkFullnameEmpty(p); } ] }
placeholder pfullnameempty { }
placeholder pfullname { }
}
par{
label("parents: "){ input(p.parents)[onchange := action{ checkParents(p); } ] }
placeholder pparents { }
}
par{
label("children: "){ input(p.children)[onchange := action{ checkParents(p); } ] }
}
submit save() [ajax] {"save"} //explicit ajax modifier currently necessary in non-ajax templates to enable replace. A warning is shown in the log if this is missing.
}
action save(){
// made an issue requesting & operator :)
var checked := checkUsernameEmpty(p);
checked := checkUsername(p) && checked;
checked := checkFullname(p) && checked;
checked := checkParents(p) && checked;
checked := checkFullnameEmpty(p) && checked;
if(checked){
p.save();
return root();
}
}
}
The function definitions perform the check, and also update the placeholders (note that placeholder names are currently global in the application). They also return the result as a boolean, so the functions can be reused in the save action. replace
calls perform the insertion of templates into placeholders, in this case the templates are just creating messages.
function checkUsernameEmpty(p:Person):Bool{
if(p.username != ""){
replace(pusernameempty, empty());
return true;
}
else {
replace(pusernameempty, mpusernameempty());
return false;
}
}
function checkUsername(p:Person):Bool{
if((from Person as p1 where p1.username = ~p.username).length == 0)){
replace(pusername, empty());
return true;
}
else {
replace(pusername, mpusername(p.username));
return false;
}
}
function checkFullnameEmpty(p:Person):Bool{
if(p.fullname != ""){
replace(pfullnameempty, empty());
return true;
}
else {
replace(pfullnameempty, mpfullnameempty());
return false;
}
}
function checkFullname(p:Person) :Bool{
if(p.username != p.fullname) {
replace(pfullname, empty());
return true;
}
else{
replace(pfullname, mpfullname());
return false;
}
}
The templates for the messages are shown below. An errorclass template is used to wrap all the errors in the same div tag with special error class, to provide a hook for CSS styling. Templates that are used in replace actions have to be declared as ajax
template. When access control is enabled the ajax templates can be protected with the ajaxtemplate
rule type.
define errorclass(){
<div class="error"> elements() </div>
}
define ajax empty(){ "" }
define ajax mpusername(name: String){ errorclass{ "Username " output(name) " has been taken already" } }
define ajax mpusernameempty(){ errorclass{ "Username may not be empty" } }
define ajax mpfullname(){ errorclass{ "Username and fullname should not be the same" } }
define ajax mpfullnameempty(){ errorclass{ "Fullname may not be empty" } }
define ajax mpparents(pname:String,names : List<String>){
errorclass{
"Person"
if(names.length > 1){"s"}
" "
for(name: String in names){
output(name)
} separated-by {", "}
" cannot be both parent and child of " output(pname)
}
}
This app includes some CSS for top-aligned labels (http://css-tricks.com/label-placement-on-forms/), the errors are shown on new lines and in red.
label {
clear:both;
float:left;
margin:10px 0 2px 0;
}
input, select {
clear:both;
float:left;
}
#errorclass {
color: red;
clear: both;
float: left;
margin:2px 0 0 0;
}
input[type="button"]{
clear: both;
float: left;
margin: 20px 0 10px 0;
}
Since the example used a new Person entity, the save
controls whether the object is persisted. If the entity was already in the database, and this is an edit page, then the save wouldn’t be necessary to persist changes. Unfortunately this has the side-effect that all intermediate submits (on every change) already persist the changes automatically. One way to work around this issue create a copy of the entity and use that for data binding. Instead of the save()
call, the code needs to put the changes back into the real persisted entity.
The editpage, in this case a global entity var is passed in, to demonstrate changing an entity that is persisted.
define page edit(){
main()
define body() {
personedit(pAlice)
}
}
The template is shown below, unchanged parts are left out. A template var that copies the original values is used for data binding, in the save action the changes are placed in the real person object. The save
call is not necessary for edits, but now the template works correctly for both edit and create actions.
define personedit(realp:Person){
var p := Person{ username := realp.username fullname := realp.fullname children := realp.children parents := realp.parents};
form{
par{
label("username: "){ input(p.username)[onkeyup := action{ checkUsername(p,realp); checkUsernameEmpty(p); checkFullname(p); }] }
...
action save(){
// made an issue requesting & operator :)
var checked := checkUsernameEmpty(p);
checked := checkUsername(p,realp) && checked;
checked := checkFullname(p) && checked;
checked := checkParents(p) && checked;
checked := checkFullnameEmpty(p) && checked;
if(checked){
realp.username := p.username;
realp.fullname := p.fullname;
realp.parents := p.parents;
realp.children := p.children;
realp.save(); // does nothing in the case of an update
return root();
}
}
}
One of the checks needs to change, because the entity might be already in the database now.
function checkUsername(p:Person, realp:Person):Bool{
var matches := from Person as p1 where p1.username = ~p.username;
if(matches.length == 0 || (matches.length == 1 && matches[0] == realp)){
replace(pusername, empty());
return true;
}
else {
replace(pusername, mpusername(p.username));
return false;
}
}
The full project source of this example is located here:
https://svn.strategoxt.org/repos/WebDSL/webdsls/trunk/test/succeed-web/manual/ajax-form-validation/
Tutorial application: Event Planner
Tutorial PDF: http://webdsl.org/tutorial-event-planner-files/webdsl-tutorial.pdf
Files: http://webdsl.org/tutorial-event-planner-files/tutorial-base-files.zip
Solution: http://webdsl.org/tutorial-event-planner-files/tutorial-solution.zip
This section contains information for developers of WebDSL.
When running an application entirely in the Eclipse environment, you can choose to start debug mode in the ‘Servers’ view.
On the command-line, ‘webdsl run’ will set up the remote debugger interface, on the usual port 8000. Then set it up in eclipse: Run menu -> debug configurations… -> click on remote java applications -> new (icon top left) -> add source dirs of your project -> press ‘debug’