JSONiq for JSON users

XQuery for JSON


This tutorial introduces the JSONiq language, which declaratively manipulates JSON data.

Why don't you go ahead can try the queries of this document on our online demo interface?

 http://jsoniq.zorba-xquery.com/


JSON

Elevator Pitch

And here you go

Like Monsieur Jourdain, you already knew some JSONiq

JSONiq basics

The real JSONiq Hello, World!

Numbers and arithmetic operations

Logical operations

Strings

Sequences

A bit more in depth

Variables

Iteration

Conditions

Composability of Expressions

JSON Navigation

Objects

Arrays

Pairs

Relational Algebra

Access external data

I want more

Even more

More is not enough

JSON

As explained on http://www.json.org/, JSON is a lightweight data-interchange format designed for humans as well as for computers. It supports as values:

  1. objects (string-to-value map)
  2. arrays (ordered sequence of values)
  3. strings
  4. numbers
  5. booleans (true, false)
  6. null

JSONiq provides declarative querying and updating capabilities on JSON data.

Elevator Pitch

JSONiq is based on XQuery, which is a W3C standard (like XML and HTML). XQuery is a very powerful declarative language that manipulates XML data, but that is also a very good fit for JSON. Here is an appetizer before we start the tutorial from scratch.

let $stores :=

[

  { "store number" : 1, "state" : "MA" },

  { "store number" : 2, "state" : "MA" },

  { "store number" : 3, "state" : "CA" },

  { "store number" : 4, "state" : "CA" }

]

let $sales := [

   { "product" : "broiler", "store number" : 1, "quantity" : 20  },

   { "product" : "toaster", "store number" : 2, "quantity" : 100 },

   { "product" : "toaster", "store number" : 2, "quantity" : 50 },

   { "product" : "toaster", "store number" : 3, "quantity" : 50 },

   { "product" : "blender", "store number" : 3, "quantity" : 100 },

   { "product" : "blender", "store number" : 3, "quantity" : 150 },

   { "product" : "socks", "store number" : 1, "quantity" : 500 },

   { "product" : "socks", "store number" : 2, "quantity" : 10 },

   { "product" : "shirt", "store number" : 3, "quantity" : 10 }

]

let $join :=

  for $store in jn:values($stores), $sale in jn:values($sales)

  where $store("store number") = $sale("store number")

  return {

    "nb" : $store("store number"),

    "state" : $store("state"),

    "sold" : $sale("product")

  }

return [$join]

[

  { "nb" : 1, "state" : "MA", "sold" : "broiler" },

  { "nb" : 1, "state" : "MA", "sold" : "socks" },

  { "nb" : 2, "state" : "MA", "sold" : "toaster" },

  { "nb" : 2, "state" : "MA", "sold" : "toaster" },

  { "nb" : 2, "state" : "MA", "sold" : "socks" },

  { "nb" : 3, "state" : "CA", "sold" : "toaster" },

  { "nb" : 3, "state" : "CA", "sold" : "blender" },

  { "nb" : 3, "state" : "CA", "sold" : "blender" },

  { "nb" : 3, "state" : "CA", "sold" : "shirt" }

 ]

And here you go

Like Monsieur Jourdain, you already knew some JSONiq

The first thing you need to know is that every well-formed JSON document is a JSONiq query as well.

This means that you can copy-and-paste a JSON document into a query. The following are JSONiq queries that are "idempotent" (they just output themselves):

{ "pi" : 3.14, "sq2" : 1.4 }

{ "pi" : 3.14, "sq2" : 1.4 }

[ 2, 3, 5, 7, 11, 13 ]

[ 2, 3, 5, 7, 11, 13 ]

{

  "operations" : [

    { "binary" : [ "and", "or"] },

    { "unary" : ["not"] }

  ],

  "bits" : [

    0, 1

  ]

}

{

  "operations" : [

    { "binary" : [ "and", "or" ] },

    { "unary" : [ "not" ] }

  ],

  "bits" : [

    0, 1

  ]

}

This works with objects, arrays (even nested), strings, numbers, booleans, null. The exceptions to this rule (but we are working on it!) are that:

(i) if a pair has a true, false or null value, there must be a space on at least one side of the colon (this will be fixed).

(ii) empty objects are not recognized

(iii) characters escaped with the \ in JSON strings are not recognized

It also works the other way round: if your query outputs an object or an array, you can readily use it as a JSON document.

JSONiq is a declarative language. This means that you only need to say what you want - the compiler will take care of the how. In the above queries, you are basically saying: I want to output this JSON content, and here it is.

JSONiq basics

The real JSONiq Hello, World!

Wondering what a hello world program looks like in JSONiq? Here it is:

"Hello, World!"

Hello, World!

Not surprisingly, it outputs the string "Hello, World!". Again, just say what you want and you will get it.

Numbers and arithmetic operations

Okay, so, now, you might be thinking: "What is the use of this language if it just outputs what I put in?" Of course, JSONiq can more than that. And still in a declarative way. Here is how it works with numbers:

2 + 2

will surprisingly output:

4

whereas

(38 + 2) div 2 + 11 * 2

will output

42

(mind the division operator which is the "div" keyword. The slash has other semantics).

Like JSON, JSONiq works with decimals, too:

6.022e23 * 42

2.52924E25

Logical operations

JSONiq supports boolean operations.

true and false

false

(true or false) and (false or true)

true

The unary not is a function (yes, JSONiq has functions, and they are in the same way as in many languages):

not(true)

false

Strings

JSONiq is capable of manipulating strings as well, using functions:

concat("Hello ", "Captain Kirk")

Hello Captain Kirk

substring("Mister Spock", 8, 5)

Spock

JSONiq comes up with a rich string function library out of the box inherited from its base language. These functions are listed here (actually, you will find even many more for numbers, etc.)

Sequences

Until now, we only handled single values (an object, an array, a number, a string, a boolean). JSONiq supports sequences of values. You can build a sequence using commas:

1, 2, 3, 4, 5, 6, 7, 8, 9, 10

1 2 3 4 5 6 7 8 9 10

1, true, 4.2e1, "Life"

1 true 42 Life

{ "Question" : "Ultimate" }, ["Life", "The universe", "and everything"]

{ "Question" : "Ultimate" }[ "Life", "The universe", "and everything" ]

The "to" operator is very convenient, too:

1 to 100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40

41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80

81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100

Some functions even work on sequences:

sum(1 to 100)

5050

string-join(("These", "are", "some", "words"), "-")

These-are-some-words

count(10 to 20)

11

avg(1 to 100)

50.5

A bit more in depth

Variables

You can bind a sequence of values to a (dollar-prefixed) variable, like so:

let $x := "Bearing 3 1 4 Mark 5. "

return concat($x, "Engage!")

Bearing 3 1 4 Mark 5. Engage!

let $x := ("Kirk", "Picard", "Sisko")

return string-join($x, " and ")

Kirk and Picard and Sisko

You can bind as many variables as you want:

let $x := 1

let $y := $x * 2

let $z := $y + $x

return [$x, $y, $z]

[ 1, 2, 3 ]

and even reuse the same name:

let $x := 1

let $x := $x + 2

let $x := $x + 3

return $x

6

Iteration

In a way very similar to let, you can iterate over a sequence of values with the "for" keyword. Instead of binding the entire sequence of the variable, it will bind each value of the sequence in turn to this variable.

for $i in 1 to 10

return $i * 2

2 4 6 8 10 12 14 16 18 20

More interestingly, you can combine fors and lets like so:

let $sequence := 1 to 10

for $value in $sequence

let $square := $value * 2

return $square

2 4 6 8 10 12 14 16 18 20

and even filter out some values:

let $sequence := 1 to 10

for $value in $sequence

let $square := $value * 2

where $square < 10

return $square

2 4 6 8

Conditions

You can make the output depend on a condition with an if-then-else construct:

for $x in 1 to 10

return if ($x < 5) then $x

                   else -$x

1 2 3 4 -5 -6 -7 -8 -9 -10

Composability of Expressions

Now that you know of a couple of elementary JSONiq expressions, you can combine then in more elaborate expressions. For example, you can put any sequence of values in an array:

[ 1 to 10 ]

[ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ]

Or you can dynamically compute the value of object pairs:

{

  "Greeting" : (let $d := "Mister Spock"

                return concat("Hello, ", $d)),

  "Farewell" : string-join(("Live", "long", "and", "prosper"),

                           " ")

}

{ "Greeting" : "Hello, Mister Spock", "Farewell" : "Live long and prosper" }

You can also dynamically generate pairs:

concat("Integer ", 2) : 2 * 2

"Integer 2" : 4

and then wrap lots of them in an object:

{

  for $i in 1 to 10

  return concat("Square of ", $i) : $i * $i

}

{

"Square of 1" : 1,

"Square of 2" : 4,

"Square of 3" : 9,

"Square of 4" : 16,

"Square of 5" : 25,

"Square of 6" : 36,

"Square of 7" : 49,

"Square of 8" : 64,

"Square of 9" : 81,

"Square of 10" : 100

}

JSON Navigation

Up to now, you learnt how to compose expressions so as to do some computations and to build pairs, objects and arrays. It also works the other way round: if you have some JSON data, you can access it and navigate.

All you need to know is: JSONiq views

  1. an array as a sequence of values,
  2. an object as a set of name/value pairs (we call them just: pairs)
  3. a pair as a (name, value) tuple.

Objects

If you use an object as a functor and provide it with a string, it will return the pair named after its parameter:

let $person := {

  "first name" : "Sarah",

  "age" : 13,

  "gender" : "female",

  "friends" : [ "Jim", "Mary", "Jennifer"]

}

return $person("first name")

"first name" : "Sarah"

Arrays

If you use an array as a functor and provide it with an integer, it will return the corresponding entry:

let $friends := [ "Jim", "Mary", "Jennifer"]

return $friends(2)

Mary

Pairs

Given a pair, you can get its name and value using the functions jn:name and jn:value:

let $pair := "name of the pair" : "value of the pair"

return {  "name" : jn:name($pair), "value" : jn:value($pair) }

{ "name" : "name of the pair", "value" : "value of the pair" }

This functions also exist in plural form for objects:

let $person := {

  "name" : "Sarah",

  "age" : 13,

  "gender" : "female",

  "friends" : [ "Jim", "Mary", "Jennifer"]

}

return { "names" : [ jn:names($person)],

         "values" : [ jn:values($person)]

       }

{

"names" : [ "name", "age", "gender", "friends" ],

"values" : [ "Sarah", 13, "female", [ "Jim", "Mary", "Jennifer" ] ]

}

jn:values can also be used for an array:

let $person := {

  "name" : "Sarah",

  "age" : 13,

  "gender" : "female",

  "friends" : [ "Jim", "Mary", "Jennifer"]

}

return jn:values($person("friends"))

[ "Jim", "Mary", "Jennifer" ]

Relational Algebra

Remember last century's SELECT FROM WHERE statements? Well, JSONiq didn't throw out the baby with the bathwater. It has selection, projection and join capability.

let $stores :=

[

  { "store number" : 1, "state" : "MA" },

  { "store number" : 2, "state" : "MA" },

  { "store number" : 3, "state" : "CA" },

  { "store number" : 4, "state" : "CA" }

]

let $sales := [

   { "product" : "broiler", "store number" : 1, "quantity" : 20  },

   { "product" : "toaster", "store number" : 2, "quantity" : 100 },

   { "product" : "toaster", "store number" : 2, "quantity" : 50 },

   { "product" : "toaster", "store number" : 3, "quantity" : 50 },

   { "product" : "blender", "store number" : 3, "quantity" : 100 },

   { "product" : "blender", "store number" : 3, "quantity" : 150 },

   { "product" : "socks", "store number" : 1, "quantity" : 500 },

   { "product" : "socks", "store number" : 2, "quantity" : 10 },

   { "product" : "shirt", "store number" : 3, "quantity" : 10 }

]

let $join :=

  for $store in jn:values($stores), $sale in jn:values($sales)

  where $store("store number") = $sale("store number")

  return {

    "nb" : $store("store number"),

    "state" : $store("state"),

    "sold" : $sale("product")

  }

return [$join]

[

{ "nb" : 1, "state" : "MA", "sold" : "broiler" },

{ "nb" : 1, "state" : "MA", "sold" : "socks" },

{ "nb" : 2, "state" : "MA", "sold" : "toaster" },

{ "nb" : 2, "state" : "MA", "sold" : "toaster" },

{ "nb" : 2, "state" : "MA", "sold" : "socks" },

{ "nb" : 3, "state" : "CA", "sold" : "toaster" },

{ "nb" : 3, "state" : "CA", "sold" : "blender" },

{ "nb" : 3, "state" : "CA", "sold" : "blender" },

{ "nb" : 3, "state" : "CA", "sold" : "shirt" }

 ]

Access external data

Our implementation even supports collections of JSON objects or arrays:

dml:collection("my:data")

{ "foo" : "Your" }

{ "foo" : "Collection" }

{ "foo" : "of" }

{ "foo" : "JSON" }

{ "foo" : "objects" }

It is also possible to get JSON content with an HTTP request, or by parsing it from a string. Functions that handle this are available.

I want more

JSONiq supports JSON updates. You can declaratively update your JSON data. JSONiq provides functions that produce a list of updates. The list of updates that is eventually output by your program is then applied to your JSON data.

let $my-data := jn:json("persons.json")

let $john := jn:value($my-data("John"))

let $mary := jn:value($my-data("Mary"))

return (jn:replace-value($john, "status", "married"),

        jn:replace-value($mary, "status", "married"))

Even more

JSONiq can do way more that what is presented here. Here are a couple of highlights:

- JSONiq is a strongly typed language, but is smart enough to not bother you with types when unnecessary. It potentially supports static typing as well to detect errors before you even execute your program.

- You can define your own functions and modules.

- JSONiq provides you with loads of available modules.

- JSONiq has tons of further features such as switch, typeswitch and try-catch expressions, universal/existential quantifiers, path expressions, filtering expressions, functors, mappings, grouping, windowing.

More is not enough

- JSONiq supports XML. Yes: you can manipulate JSON and XML with the same language! JSONiq is actually a superset of XQuery, a W3C standard, and extends its data model to support JSON.

- JSONiq supports scripting. If you need to write a full-fledged, side-effecting Web application, scripting is for you.