The code to accompany this series is available here.

In the previous part of this series, we saw how to use lattice semantics to calculate all our degrees of separation within the Game of Thrones dataset in one pass. But we didn’t quite get to our final solution because we weren’t yet calculating counts of the number of characters at each degree.

In this part, we’ll tie off that loose end, and discover why lattice semantics are “lattice” semantics in the process.

Of course, we could just count up our values in Flix (not Datalog) code, and that would be a perfectly reasonable approach. But as we’ll see Datalog gives us a really elegant solution by leveraging *partial ordering*.

So far we’ve been working with simple integers, and integers are (surprise!) *ordered*. So 1 is less than 2 and 3 is less than 126 and so on.

In fact, not only are integers ordered, they *totally* ordered. You can pick any pair of numbers `a`

and `b`

, and at least one of `a <= b`

or `b <= a`

will be true (they might both be true if `a`

equals `b`

).

But, not every set of values is totally ordered. A real world example of a partial ordering is ancestry; given two people a and b, it might easily be the case that *neither* “a is an ancestor of b” *nor* “b is an ancestor of a” is true.

For example, “Eddard Stark is an ancestor of Arya Stark” is true. But neither “Cersei Lannister is an ancestor of Tyrion Lannister” nor “Tyrion Lannister is an ancestor of Cersei Lannister” are true.

However, we can ask “who is the most recent common ancestor of Cersei and Tyrion” (in this case Tywin Lannister). We generally refer to this as the *least upper bound*.

In part 3, we said that lattice semantics chose the “maximum” value from all possible values. This was a simplification: they actually chose the least upper bound.

For integers, which we were working with in part 3, the least upper bound *is* the maximum value. But for other types, those that are partially ordered (as we saw with ancestry above), the least upper bound could be something else.

An interesting example is sets. It may not be the case that either “set a is a subset of set b” or “set b is a subset of set a” is true. But they will always have a least upper bound which is the *union* of a and b.

For example, the least upper bound of `Set#{"Tyrion Lannister", "Cersei Lannister"}`

and `Set#{"Tyrion Lannister", "Arya Stark"}`

is `Set#{"Tyrion Lannister", "Cersei Lannister", "Arya Stark"}`

.

This is where the “lattice” in lattice semantics comes from: a partially ordered set which defines a least upper bound is called a

latticein mathematics.

Happily, this is exactly what we need to solve our degrees of separation problem.

As a reminder, here were the rules we used in part 3 of this series:

```
Degree(x; Down(0)) :- Root(x).
Degree(x; n + Down(1)) :- Degree(y; n), Related(y, x).
```

Here are the new rules we’re going to add:

```
AggregatedDegree(n; Set#{x}) :- fix Degree(x; n).
DegreeCount(n, Set.size(s)) :- fix AggregatedDegree(n; s).
```

We start by inferring new `AggregatedDegree`

facts from the `Degree`

facts we calculated previously. We’re using the fact that the least upper bound of a collection of sets is the union of those sets, so the value on the right side of the semicolon will be the union of all of the character names in `Degree`

.

You might be wondering what the

`fix`

is for in our new rules? If you look at the two sides of the rule (either side of the`:-`

) you can see that on the left we’re using`n`

as a normal value (it’s on the left of the semicolon), whereas on the right it’s a lattice value (it’s on the right of the semicolon). Flix requires that we use`fix`

if we ever mix a value this way; it ensures that we completely calculate all the`Degree`

facts before starting to create`AggregateDegree`

facts.

So now we have a number of `AggregatedDegree`

facts, one for each degree of separation, where the right hand side is a set of all the characters of that degree. The final step is to convert those facts into `DegreeCount`

facts by finding the size of each set.

Here’s the whole thing. As you can see, it’s even more elegant than the solution we came up with in part 2 (which was already much simpler than we could have achieved without using Datalog).

```
def main(): Unit \ IO =
let relationshipTypes = "parents" :: "parentOf" :: "killed" :: "killedBy" ::
"serves" :: "servedBy" :: "guardianOf" :: "guardedBy" :: "siblings" ::
"marriedEngaged" :: "allies" :: Nil;
let relationships = Json.getRelationships(relationshipTypes);
let related = inject relationships into Related;
let rules = #{
Degree(x; Down(0)) :- Root(x).
Degree(x; n + Down(1)) :- Degree(y; n), Related(y, x).
AggregatedDegree(n; Set#{x}) :- fix Degree(x; n).
DegreeCount(n, Set.size(s)) :- fix AggregatedDegree(n; s).
};
let root = inject "Tyrion Lannister" :: Nil into Root;
query rules, related, root select (d, c) from DegreeCount(d, c) |>
List.foreach(match (d, c) -> println("Separated by degree ${d}: ${c}"))
```

And, for completeness, here’s what it outputs:

```
Separated by degree 6: 2
Separated by degree 5: 14
Separated by degree 4: 60
Separated by degree 3: 104
Separated by degree 2: 56
Separated by degree 1: 6
Separated by degree 0: 1
```

That’s it for our journey through Datalog and Flix. Please do experiment with other problems which can leverage Datalog: I’d love to see what you come up with!

]]>The code to accompany this series is available here.

It’s very common that we want to find the “best” solution to a problem. Perhaps we want the shortest route between two points? Or the fastest? Or the most efficient? Or…?

Flix’s implementation of Datalog provides a feature known as “lattice semantics” (we’ll see why “lattice” later) which allows us to solve exactly these kinds of problems. To illustrate, consider the following function:

```
def foo(): List[(String, Int32)] =
let rules = #{
Foo("x", 1).
Foo("x", 2).
Foo("x", 5).
Foo("y", 2).
Foo("y", 3).
};
query rules select (x, y) from Foo(x, y)
```

This, unsurprisingly, just returns exactly the facts that we specified:

```
(x, 1) :: (x, 2) :: (x, 5) :: (y, 2) :: (y, 3) :: Nil
```

But now, let’s modify it so that it uses lattice semantics:

```
def foo(): List[(String, Int32)] =
let rules = #{
Foo("x"; 1).
Foo("x"; 2).
Foo("x"; 5).
Foo("y"; 2).
Foo("y"; 3).
};
query rules select (x, y) from Foo(x; y)
```

All we’ve done is replace the commas (`,`

) in our original `Foo`

rules with semicolons (`;`

). What it now returns is:

```
(x, 5) :: (y, 3) :: Nil
```

Switching to semicolons instructs Datalog to aggregate all the facts that have the same values on the left of the semicolon into a single fact, where the value on the right is the “maximum” of all the facts that are being combined. In our example above, the maximum value is 5 when the LHS is “x”, and 3 when it’s “y”.

There is some subtlety in what we mean when we say “maximum”, as we’ll see later, but for simple integers then what we get is exactly the same as if we had used

`Int32.max`

.

It seems like a small change, but it’s a small change that dramatically increases the power at our fingertips.

Finding maximum values is great, but what if we want to find minimums instead? Flix allows us to reverse the sorting order of any type by using `Down`

, so `Down[Int32]`

sorts in the opposite order to `Int32`

:

```
flix> 1 < 3
true
flix> 4 < 2
false
flix> Down(1) < Down(3)
false
flix> Down(3) < Down(2)
true
```

Using `Down`

in Datalog lattice semantics has exactly the effect you might imagine, allowing us to find minimum values instead of maximum values:

```
def bar(): List[(String, Down[Int32])] =
let rules = #{
Bar("x"; Down(1)).
Bar("x"; Down(2)).
Bar("x"; Down(5)).
Bar("y"; Down(2)).
Bar("y"; Down(3)).
};
query rules select (x, y) from Bar(x; y)
```

Which returns:

```
(x, 1) :: (y, 2) :: Nil
```

So much for theory, let’s see how this helps us solve a real problem.

In part 2 of this series, we used Datalog to calculate degrees of separation between characters in Game of Thrones. The combination of logic programming and functional programming gave us a very simple solution. Now, we’re going to use lattice semantics to make it even better.

As a refresher, here’s the code from part 2 which creates our `Related`

facts:

```
let relationshipTypes = "parents" :: "parentOf" :: "killed" :: "killedBy" ::
"serves" :: "servedBy" :: "guardianOf" :: "guardedBy" :: "siblings" ::
"marriedEngaged" :: "allies" :: Nil;
let relationships = Json.getRelationships(relationshipTypes);
let related = inject relationships into Related;
```

And here is code which uses lattice semantics to calculate the degree of separation of every character from some root character (in this case Tyrion) in a single pass:

```
let rules = #{
Degree(x; Down(0)) :- Root(x).
Degree(x; n + Down(1)) :- Degree(y; n), Related(y, x).
};
let root = inject "Tyrion Lannister" :: Nil into Root;
println(query rules, related, root select (x, d) from Degree(x; d))
```

Before we go through this to see how it works, here’s the (truncated) output:

```
(Aegon Targaryen, 3) :: (Aeron Greyjoy, 3) :: (Aerys II Targaryen, 2) ::
(Akho, 3) :: (Alliser Thorne, 3) :: (Alton Lannister, 2) :: ...
```

To see how this works, imagine what would happen if we wrote the above *without* lattice semantics:

```
Degree(x, 0) :- Root(x).
Degree(x, n + 1) :- Degree(y, n), Related(y, x).
```

This can be interpreted as saying: Character `x`

is related to our root character by degree `n + 1`

if we can find any character `y`

which is related by degree `n`

, and where `y`

is related to `x`

.

So Cersei, for example, is related to Tyrion by degree 1 because Tyrion is related to Tyrion by degree 0, and Tyrion is related to Cersei.

But … Cersei is also related to Tyrion by degree 2 because Tywin is related to Tyrion by degree 1, and Tywin is related to Cercei. And Cersei is related to Tyrion by degree 3 because Jaime is related to Tyrion by degree 2, and Jaime is related to Cersei. And so on.

By contrast, Arya is not related to Tyrion at degree 1 or 2, but is related to him by degree 3, because Eddard Stark is related to Tyrion at degree 2, and Eddard is related to Arya. And Arya is related to Tyrion by degree 4 because Walder Frey is related to Tyrion at degree 3, and Walder is related to Arya. And so on.

So, as you can probably already see, with non-lattice semantics this will never terminate; we’ll just keep calculating higher and higher degrees of separation until we run out of either memory or patience.

But, with lattice semantics (and given that we’re using `Down`

to reverse the ordering) we can constrain all of our degrees (the numbers on the right of the semicolon) to a single minimum value (whichever is the lowest they can ever take), which means:

- Our function terminates (always helpful!), and
- The degree calculations always return the length of the “shortest path” from Tyrion to every other character.

So we’re almost there: we now have a list of every character along with their minimum degree of separation from Tyrion. But if we’re going to duplicate what we created in part 2, we need to go further and count how many characters there are at each degree of separation. We’ll see how to do that in the next part of this series.

]]>The code to accompany this series is available here.

In part 1 of this series, we used Datalog rules to infer new facts about characters in Game of Thrones.

But where do our initial facts come from? In part 1 we simply included them as part of our Datalog, but this isn’t a scalable approach. We don’t want to have to manually re-enter all the details of all of the characters in Game of Thrones!

What’s more likely is that we’ll read these facts from some external source; perhaps a database or a JSON file. This is where the power of Flix integration comes into play; we can use Flix, a powerful general purpose language, to read our data and get it into the right format. We then *inject* it into Datalog facts.

So instead of writing this (from part 1):

```
def main(): Unit \ IO =
let got = #{
ParentOf("Tywin Lannister", "Cersei Lannister").
ParentOf("Tywin Lannister", "Jaime Lannister").
ParentOf("Cersei Lannister", "Joffrey Baratheon").
ParentOf("Cersei Lannister", "Myrcella Baratheon").
SiblingOf(x, y) :- ParentOf(p, x), ParentOf(p, y), if x != y.
};
let siblings = query got select (x, y) from SiblingOf(x, y);
println(siblings)
```

We can instead write:

```
def main(): Unit \ IO =
let parentList = ("Tywin Lannister", "Cersei Lannister") ::
("Tywin Lannister", "Jaime Lannister") ::
("Cersei Lannister", "Joffrey Baratheon") ::
("Cersei Lannister", "Myrcella Baratheon") :: Nil;
let parents = inject parentList into ParentOf;
let got = #{
SiblingOf(x, y) :- ParentOf(p, x), ParentOf(p, y), if x != y.
};
let siblings = query got, parents select (x, y) from SiblingOf(x, y);
println(siblings)
```

Or, more realistically, we read our data from some external file (in this case, Jeffrey Lancaster’s amazingly detailed Game of Thrones dataset):

```
def main(): Unit \ IO =
let parents = inject Json.getParents() into ParentOf;
let got = #{
SiblingOf(x, y) :- ParentOf(p, x), ParentOf(p, y), if x != y.
};
let siblings = query got, parents select (x, y) from SiblingOf(x, y);
println(siblings)
```

Which gives us the following (truncated!) output:

```
(Aegon Targaryen, Jon Snow) :: (Aegon Targaryen, Rhaenys Targaryen) ::
(Arya Stark, Bran Stark) :: (Arya Stark, Rickon Stark) ::
(Arya Stark, Robb Stark) :: (Arya Stark, Sansa Stark) ::
(Benjen Stark, Brandon Stark) :: (Benjen Stark, Eddard Stark) :: ...
```

The ability to move data back and forth between Datalog and Flix makes it very easy to use Datalog as part of solving a larger problem. To illustrate this, let’s consider working out whether Game of Thrones obeys the “six degrees of separation” rule (the idea that all people are six or fewer connections away from each other; you might have come across it from the “Six Degrees of Kevin Bacon” game, or a mathematician’s Erdős number).

As well as parentage, the dataset we’ve been using includes all kinds of other relationships between characters such as alliances, marriages and engagements, who serves whom, who killed whom, etc. Taking all of these relationships together, how many degrees of separation does it take from a major character (say Tyrion) to cover the entire cast of characters?

(You can find the complete source here).

First, we extract our long list of relationships from our dataset:

```
let relationshipTypes = "parents" :: "parentOf" :: "killed" :: "killedBy" ::
"serves" :: "servedBy" :: "guardianOf" :: "guardedBy" :: "siblings" ::
"marriedEngaged" :: "allies" :: Nil;
let relationships = Json.getRelationships(relationshipTypes);
```

We then inject that list into Datalog as a series of `Related`

facts, just like we’ve seen above:

```
let related = inject relationships into Related;
```

So now we have a set of facts like `Related("Tyrion Lannister", "Shae")`

because Tyrion killed Shae, and `Related("Arya Stark", "Nymeria")`

because Arya is guarded by Nymeria, and so on.

Next, here is a Datalog rule which, given a set of characters we’ve already found, finds the characters with the next degree of separation:

```
let rules = #{
NextDegree(x) :- AlreadyFound(y), Related(y, x), not AlreadyFound(x).
};
```

So a character `x`

is in the next degree of separation if there is at least one character `y`

who we have already and where `y`

is related to `x`

. Finally we exclude characters that we’ve already found.

Here’s a function which uses the above to repeatedly calculate degrees of separation until we’ve exhausted all our characters:

```
def degreesOfSeparation(deg: Int32, cs: List[String]): Unit \ IO = {
let alreadyFound = inject cs into AlreadyFound;
let nextDegree = query rules, alreadyFound, related select (x) from NextDegree(x);
match List.length(nextDegree) {
case count if count > 0 =>
println("Separated by degree ${deg}: ${count}");
degreesOfSeparation(deg + 1, cs ::: nextDegree)
case _ =>
println("Nobody separated by degree ${deg}")
}
};
```

This function takes `deg`

(a number representing the current degree of separation we’re looking at) and `cs`

a list of characters we’ve already found.

We start by creating our `Root`

facts from this list of characters, and then use the Datalog `rules`

we defined above, along with our `Root`

and `Related`

facts to generate the next set of `DirectlyRelated`

characters.

`List.length(directlyRelated)`

returns the length of this list, and we use `match`

to determine whether this number is greater than zero.

If it is, then we output the number of characters we just found, and recursively call `degreesOfSeparation`

with `deg`

increased by one, and a new set of characters which includes both our original root set and the characters we just found.

Finally, we kick the whole thing off by calling `degreesOfSeparation`

with our initial root character, Tyrion:

```
degreesOfSeparation(1, "Tyrion Lannister" :: Nil)
```

Here’s what we get when we run it:

```
Separated by degree 1: 6
Separated by degree 2: 56
Separated by degree 3: 104
Separated by degree 4: 60
Separated by degree 5: 14
Separated by degree 6: 2
Nobody separated by degree 7
```

So yes, it does look like Game of Thrones does obey the six degrees of separation rule.

Here’s the whole thing. I suggest trying to implement this in your favourite language to see just how easy the combination of Flix and Datalog makes this:

```
def main(): Unit \ IO =
let relationshipTypes = "parents" :: "parentOf" :: "killed" :: "killedBy" ::
"serves" :: "servedBy" :: "guardianOf" :: "guardedBy" :: "siblings" ::
"marriedEngaged" :: "allies" :: Nil;
let relationships = Json.getRelationships(relationshipTypes);
let related = inject relationships into Related;
let rules = #{
NextDegree(x) :- AlreadyFound(y), Related(y, x), not AlreadyFound(x).
};
def degreesOfSeparation(deg: Int32, cs: List[String]): Unit \ IO = {
let alreadyFound = inject cs into AlreadyFound;
let nextDegree = query rules, alreadyFound, related select (x) from NextDegree(x);
match List.length(nextDegree) {
case count if count > 0 =>
println("Separated by degree ${deg}: ${count}");
degreesOfSeparation(deg + 1, cs ::: nextDegree)
case _ =>
println("Nobody separated by degree ${deg}")
}
};
degreesOfSeparation(1, "Tyrion Lannister" :: Nil)
```

Flix allows us to seamlessly move data back and forth between Flix and Datalog by `inject`

ing facts from Flix to Datalog and `query`

ing data from Datalog to Flix. This allows us to use the strengths of a general purpose language like Flix for things it’s good at (e.g. reading JSON files) and Datalog for things that it’s good at (e.g. inferring new facts from existing ones).

In the next part of this series, we’ll look at one of the most powerful aspects of Flix’s implementation of Datalog: lattice semantics.

]]>The code to accompany this series is available here.

Flix is an exciting new programming language which provides the power of strongly typed functional programming with the accessibility of “programmer friendly” languages like Go, Rust, or Ruby.

This article isn’t about Flix, though, it’s about Datalog, and the reason why Flix plus Datalog creates something greater than the sum of its parts.

Logic programming is one of the “big three” programming language paradigms:

**Imperative programming**: Languages like C, C++, Java, JavaScript, Go, Kotlin, …

**Functional programming**: Languages like Haskell, Clojure, Elixir, F#, …

**Logic programming**: Prolog, Datalog, Answer-set Programming, Mercury, …

But … if it’s one of the three major programming paradigms, how come nobody uses it? Perhaps you created a couple of toy Prolog programs at college, but you’ve almost certainly not used it since then.

There are good reasons why logic programming hasn’t caught on outside of a very few specialised areas, but that may be about to change. By including logic programming as a first-class element of the language, Flix allows us to seamlessly move between functional and logic programming, leveraging the strengths of both.

This is analogous to the way that C# embeds database queries directly in the language through LINQ (and of course, you don’t use LINQ to implement every single method; instead you use C# for most of your code and drop into LINQ when it makes sense).

In this first article, we’ll explore the basics of Datalog. And then in the rest of the series we’ll see how the combination of Datalog and Flix brings huge expressive power.

Datalog programs are built from two components: *facts* and *rules*. Here are some facts about Game of Thrones:

```
GreatHouse("Stark").
GreatHouse("Targaryen").
ParentOf("Rhaegar Targaryen", "Jon Snow").
ParentOf("Aerys II Targaryen", "Daenerys Targaryen").
Coordinates("Kings Landing", 6.697, 12.759).
Coordinates("Winterfell", 5.666, 6.218).
```

You can read these facts as:

- Stark is one of the Great Houses.
- Rhaegar Targaryen is a parent of Jon Snow.
- Kings Landing is located at coordinates 6.697, 12.759.

A fact can have any number of terms of (almost) any type.

Rules create new facts by making logical inferences from existing facts. Imagine, for example, that we know the following facts:

```
ParentOf("Tywin Lannister", "Cersei Lannister").
ParentOf("Tywin Lannister", "Jaime Lannister").
ParentOf("Cersei Lannister", "Joffrey Baratheon").
ParentOf("Cersei Lannister", "Myrcella Baratheon").
```

Then we can create additional facts about who is who’s grandparent with the following rule:

```
GrandparentOf(x, y) :- ParentOf(x, c), ParentOf(c, y).
```

Which you can read as “`x`

is a grandparent of `y`

**if** we can find at least one person `c`

where `x`

is a parent of `c`

, **and** `c`

is a parent of `y`

”.

Here’s a complete Flix program which does this:

```
def main(): Unit \ IO =
let got = #{
ParentOf("Tywin Lannister", "Cersei Lannister").
ParentOf("Tywin Lannister", "Jaime Lannister").
ParentOf("Cersei Lannister", "Joffrey Baratheon").
ParentOf("Cersei Lannister", "Myrcella Baratheon").
GrandparentOf(x, y) :- ParentOf(x, c), ParentOf(c, y).
};
let grandparents = query got select (x, y) from GrandparentOf(x, y);
println(grandparents)
```

The code within the `#{ ... }`

block is our datalog, the rest of the code is the Flix program that it’s embedded within. Our Datalog program is executed when we call `query`

to extract the new `GrandparentOf`

facts.

Here’s what it outputs:

```
(Tywin Lannister, Joffrey Baratheon) :: (Tywin Lannister, Myrcella Baratheon) :: Nil
```

This is a list of 2-element tuples, with `::`

indicating the list “cons” operator and `Nil`

the end of the list. We can interpret these results as saying that Tywin Lannister is the grandparent of both Joffrey Baratheon and Myrcella Baratheon.

Here’s a slightly different program which uses the same facts to generate additional facts about siblings:

```
def main(): Unit \ IO =
let got = #{
ParentOf("Tywin Lannister", "Cersei Lannister").
ParentOf("Tywin Lannister", "Jaime Lannister").
ParentOf("Cersei Lannister", "Joffrey Baratheon").
ParentOf("Cersei Lannister", "Myrcella Baratheon").
SiblingOf(x, y) :- ParentOf(p, x), ParentOf(p, y).
};
let siblings = query got select (x, y) from SiblingOf(x, y);
println(siblings)
```

The key line is:

```
SiblingOf(x, y) :- ParentOf(p, x), ParentOf(p, y).
```

Which you can read as “`x`

is a sibling of `y`

**if** we can find at least one person `p`

where `p`

is a parent of `x`

**and** `p`

is a parent of `y`

”.

And here’s what it outputs:

```
(Cersei Lannister, Cersei Lannister) :: (Cersei Lannister, Jaime Lannister) ::
(Jaime Lannister, Cersei Lannister) :: (Jaime Lannister, Jaime Lannister) ::
(Joffrey Baratheon, Joffrey Baratheon) :: (Joffrey Baratheon, Myrcella Baratheon) ::
(Myrcella Baratheon, Joffrey Baratheon) :: (Myrcella Baratheon, Myrcella Baratheon) ::
Nil
```

Were you surprised that there were so many answers?

The list is as long as it is because:

- If
`SiblingOf(x, y)`

is true, then so is`Sibling(y, x)`

(i.e. if you are my sibling, then I am also your sibling). - As we’ve written it,
`Sibling`

considers anyone with a parent to be their own sibling.

The first point above is probably exactly what we want (i.e. we really do want the sibling relationship to be reflexive). But perhaps we don’t want people being their own siblings.

Here’s a slightly modified version of our program which adds a *guard* to our rule, which eliminates cases where `x`

and `y`

represent the same person:

```
def main(): Unit \ IO =
let got = #{
ParentOf("Tywin Lannister", "Cersei Lannister").
ParentOf("Tywin Lannister", "Jaime Lannister").
ParentOf("Cersei Lannister", "Joffrey Baratheon").
ParentOf("Cersei Lannister", "Myrcella Baratheon").
SiblingOf(x, y) :- ParentOf(p, x), ParentOf(p, y), if x != y.
};
let siblings = query got select (x, y) from SiblingOf(x, y);
println(siblings)
```

And here’s what it outputs:

```
(Cersei Lannister, Jaime Lannister) :: (Jaime Lannister, Cersei Lannister) ::
(Joffrey Baratheon, Myrcella Baratheon) :: (Myrcella Baratheon, Joffrey Baratheon) ::
Nil
```

A Datalog program consists of *facts* and *rules*. Rules generate new facts from existing ones.

In the next part of this series, we’ll dig deeper into the integration between Flix and Datalog.

]]>