Tom’s blog

Giving presentations with LaTeX Beamer and pdfpc

2021-10-17T00:00:00+02:00

If you, like me, give scientific presentations from time to time, chances are that you are using LaTeX and its beamer class. The output is usually a PDF file that can be shown with pretty much every PDF viewer in some full screen presentation mode. A significantly more capable tool for displaying PDF presentations is pdfpc. It has a lot of nice features that are familiar from dedicated presentation tools like Microsoft PowerPoint or LibreOffice Impress, most importantly support for a dual-screen setup with a presenter view in addition to the full screen slides visible to the audience.

One of the most useful features in my opinion is the ability to add presenter notes to every slide. And the really cool thing is that you can embed these notes invisibly into the PDF by using the LaTeX package pdfpc. It may look like this:

\documentclass{beamer}

\usepackage{pdfpc}
\newcommand<>{\talknote}[1]{\only#2{\pdfpcnote{- #1}\relax}}

\begin{document}

\begin{frame}{Slide One}

	\talknote<1>{This note is only shown in the beginning}

	\talknote<3-4>{This note is shown for **some points**}

	\begin{itemize}
		\item<2-> A first point
		\item<3-> Another point
		\item<4-> Point number three
	\end{itemize}

	\talknote{This note is shown all the time}

\end{frame}

\begin{frame}{Slide Two}

\end{frame}

\end{document}

I defined a newcommand called talknote to be able to add overlay specifications to notes. This way, I can easily add notes that are only displayed for some time while flicking through the overlays of a slide. The notes are also formatted as list items (pdfpc understands Markdown). The command pdfpcnote is provided by the pdfpc package.

When opening the produced PDF file with pdfpc and going through the slides, the presenter view looks like this:

Colored and word-based diff for svn

2021-03-21T00:00:00+01:00

At work, we use Subversion (yes, I know) for version control of the LaTeX sources of the papers we write. One of the nice things about version control is the possibility of viewing diffs, which is really useful when collaborating on text. However, LaTeX sources often have very long lines, which makes it hard to spot differences in the usual line-based diff. For viewing word-based differences, there is wdiff instead. To use colors for marking added and removed words, Prof. Iain Murray has published a cwdiff shell script in 2011.

I have adapted that shell script a bit. First, tput setf didn’t work on my system, so I replaced it with tput setaf. Next, the script now marks additions and removals only with colors. Third, I played around with the color choices a bit until they looked good in my solarized color scheme. Finally, the script compares the files given in $6 and $7, which allows using it directly as svns diff-cmd.

The complete shell script is below:

#!/bin/sh
diffnew=$(tput setaf 7)$(tput bold)
diffold=$(tput setaf 1)$(tput bold)
reset=$(tput op)$(tput sgr0)

wdiff \
    --start-delete "${diffold}" \
    --end-delete "${reset}" \
    --start-insert "${diffnew}" \
    --end-insert "${reset}" \
    -n "$6" "$7" \
    | grep -C2 '\[' \
    | less

I recommend that you read the well-documented original script for more background.

I finished my PhD

2021-02-14T00:00:00+01:00

It is now official! After handing in my thesis in September and defending it in January, I can now call myself Dr.-Ing. (Doctor of Engineering).

Using streams for purely functional random number generation in Scala

2020-09-16T00:00:00+02:00

The generation of (pseudo-)random numbers is a perfect example for the differences between imperative (object-oriented) and purely functional programming. Scala’s support for both of these programming paradigms means that we can translate between them. In this post we start with the imperative random number generation from the Scala standard library and wrap it up in a purely functional random number generator. This exemplifies how mutable objects can be used even in functional code if the mutability is not observable from the outside. The key is a lazy data structure that queries a private mutable object on demand.

Imperative random number generation

The scala standard library provides a random number generator in a class Random, offering methods like nextInt:

val rng = new scala.util.Random(seed = 42)
println(rng.nextInt); // -1170105035 
println(rng.nextInt); // 234785527
println(rng.nextInt); // -1360544799

As you see, subsequent calls to nextInt on a Random object produce different results. Therefore, the expression rng.nextInt is neither deterministic nor referentially transparent - we could not replace all occurrences of that expression with a memoized value. This is because calling nextInt has side effects - it mutates the internal state of the object rng.

Whereas this is the idiomatic way to express random number generation in imperative programming, purely functional programming avoids side effects and mutation in favor of determinism and referential transparency. You may now wonder whether it is possible to use an imperative random number generator like scala.util.Random in a purely functional program at all. The answer is yes.

Purely functional random number generation

A purely functional random number generator can be expressed in Scala as follows:

trait RNG {
    def nextInt: (RNG, Int)
}
val rng: RNG = mkRNG(seed = 42) // initialize with seed = 42
println(rng.nextInt); // (RNG(seed = 13), -1170105035) 
println(rng.nextInt); // (RNG(seed = 13), -1170105035) 
println(rng.nextInt); // (RNG(seed = 13), -1170105035) 

Now RNG objects are immutable and generating a random number returns a changed RNG object instead of mutating r. The expression rng.nextInt is deterministic and referentially transparent. Next we look into an implementation of the RNG trait that wraps an imperative random number generator.

Wrapping an imperative random number generator in a purely functional random number generator

The fundamental idea of the RNG trait that it contains some immutable state that allows generating a deterministic random number stream. One way to do this is to use the last generated number as the state and some mathemagical formula to produce the next state. However, we can also make the stream of random numbers explicit:

def mkRNG(seed: Long): RNG = StreamRNG(stream(seed))

case class StreamRNG(stream: LazyList[Int]) extends RNG {
    def nextInt: (RNG, Int) = (StreamRNG(stream.tail), stream.head)
}

def stream(seed: Long): LazyList[Int] = {
    val random = new Random(seed)
    def mkStream: LazyList[Int] = random.nextInt #:: mkStream
    mkStream
}

The important part is the function stream with its recursive inner function mkStream. It produces a lazily evaluated, deterministic stream of random numbers with a private, mutable scala.util.Random object. Elements of this stream are memoized, therefore the expression random.nextInt is only evaluated once per stream element.

A StreamRNG object points to a specific element in the stream, and to get the next random number we just return the head of the stream and use the tail as the new state. Different StreamRNG objects can point to different elements in the stream, but the overall sequence of elements is shared between them. In addition, the head of the stream gets garbage-collected if it is no longer accessible by any StreamRNG object.

Bonus: Wrapping a purely functional random number generator in an imperative random number generator

The reverse direction is easier.

class MyRandom(seed: Int) {
    var rng: RNG = mkRNG(seed)
    def nextInt: Int = {
        val (rng2, r) = rng.nextInt
        rng = rng2
        r
    }
}

Saving the current state in a var allows overriding it, restoring mutability.

Lazy sequences in the Scala standard library

2020-05-16T00:00:00+02:00

One way to learn the idioms and idiosyncrasies of a programming language is to follow the development of its standard library. The Scala standard libary, for example, deprecated its data structure for lazy sequences Stream and replaced it with a new data structure called LazyList in the 2.13 release. However, LazyList has different laziness characteristics than Stream, which recently sparked some discussion on GitHub. I found the discussion quite interesting and will try to illustrate the difference between both data structures in this post.

Stream

This is a simplified version of the definition of Stream:

class Stream[A](hd: A, tl: => Stream[A]) {
    val head: A = hd
    lazy val tail: Stream[A] = tl
    def apply(i: Int): A = if (i == 0) head else tail(i - 1)
}

A Stream consists of cons cells with an eagerly evaluated head and a lazily evaluated tail. When a Stream object is instantiated with expressions for the head and tail, only the head expression is immediately evaluated, whereas the tail is only evaluated when it is first accessed. Therefore, a Stream always looks like the following. All cons cells have an evaluated head, and all cons cells except the last one have an evaluated tail pointing at the next cons cell.

[  head, tail--]-->[  head, tail--]-->[  head, UNEVALUATED  ]

One way to construct such a stream is to use a recursive function. By injecting some printlns we can see in what order the function calls and the expressions for the head and the tail are evaluated. #:: is syntactical sugar for creating a Stream cons cell:

def mkStream(i: Int): Stream[Int] = {
    println(s"Evaluating cell $i")

    { println(s"Evaluating head for cell $i"); i } #:: { println(s"Evaluating tail for cell $i"); mkStream(i + 1) }
}

We can now force the evaluation of a few cons cells and observe the output.

val stream = mkStream(0)
println()
stream(1)
println()
stream(2)

The output is:

Evaluating cell 0
Evaluating head for cell 0

Evaluating tail for cell 0
Evaluating cell 1
Evaluating head for cell 1

Evaluating tail for cell 1
Evaluating cell 2
Evaluating head for cell 2

As you can see, the head of each cell is evaluated when the cell is constructed, but the tail is only evaluated to access the next cell.

LazyList

This is a simplified version of the definition of LazyList, the replacement for Stream:

class LazyList[A](hd: => A, tl: => LazyList[A]) {
    lazy val (head, tail): (A, LazyList[A]) = (hd, tl)
    def apply(i: Int): A = if (i == 0) head else tail(i - 1)
}

Now both head and tail are lazily evaluated. Therefore, the actual cons cell is only constructed when either head or tail are accessed. Interestingly (and maybe confusingly), both head and tail are always evaluated together. For example, accessing the head of an unevaluated LazyList cons cell will construct the cons cell and also evaluate its tail. Thus, LazyList structures look like this:

[  head, tail--]-->[  head, tail--]-->UNEVALUATED

With a recursive function to construct a LazyList similar to the one shown above for Stream and equivalent accesses to the individual cons cells, we get the following output:

Evaluating cell 0

Evaluating head for cell 0
Evaluating tail for cell 0
Evaluating cell 1
Evaluating head for cell 1
Evaluating tail for cell 1
Evaluating cell 2

Evaluating head for cell 2
Evaluating tail for cell 2
Evaluating cell 3

The first call to construct the LazyList evaluates neither the head nor the tail. In that sense, LazyList is lazier than Stream. However, evaluating the third cons cell (containing the 2) also invokes the function to construct the next cons cell. Here, LazyList is less lazy than Stream.

Conclusion

One immediate conclusion refers to porting uses of Stream to LazyList. To exploit laziness in LazyList in its current (Scala 2.13.2) version, it has to be used differently than Stream. If you are using a recursive function to construct the LazyList, make sure that you do the work in the expressions for head and tail, as these are lazily evaluated. Everything you do in the function outside of these expressions will the evaluated as soon as the previous cons cell is evaluated, even if only the head of that previous cons cell is accessed. So, for example, instead of

def mkLazyList(i: Int): LazyList[Int] = {
    val content = doExpensiveCalculation(i)
    content #:: mkLazyList(i + 1)
}

def mkLazyList(i: Int): LazyList[Int] = {
    doExpensiveCalculation(i) #:: mkLazyList(i + 1)
}

def mkLazyList(i: Int): LazyList[Int] = {
    lazy val content = doExpensiveCalculation(i)
    content #:: mkLazyList(i + 1)
}

At the time of this writing, a pull request for making the tail of a LazyList cons cell lazy again has been opened.

Producing pictures of formulas with the LaTeX standalone package

2020-05-02T00:00:00+02:00

Just a quick post with some LaTeX snippet that I found very useful to produce nice pictures of formulas for online teaching. It can be compiled with pdflatex -shell-escape filename and you need to have poppler installed. Then you get a nice, high-resolution png of a LaTeX formula.

\documentclass[
	convert={command=\unexpanded{pdftoppm\space -png \space -r \space 600 \space \infile \space > \outfile}},
	border=1
]{standalone}

\begin{document}
$e = m c^2$
\end{document}

Maybe that’s useful for someone.

Using the Sony WH-1000XM3 with PulseAudio as a Headphone only

2020-04-25T00:00:00+02:00

Due to the Corona-caused lock-down, I have been in many video calls via Skype, Zoom, etc. At first, I used the integrated microphone of my laptop as well as some wired speakers plugged into the headphone jack. But it turns out I also have to do some teaching via video conferencing software. Therefore, I ordered a dedicated USB microphone (a FiFine K670) for better audio. My plan was to use that microphone and my Sony WH-1000XM3 bluetooth headphones.

Now the problem I encountered was that each time I had the headphones connected to the laptop and I started a Skype/Zoom/Whatever call, they automatically went into HSP/HFP mode. This is the bluetooth mode for calls, meaning that the microphone can be used, but the audio quality on the headphones is reduced. I want the headphones to be in A2DP mode all the time, which has better audio quality, but turns off the integrated microphone in the headset.

The reason for this automatic switch is a “feature” that was introduced in PulseAudio 10. It automatically switches the bluetooth profile as soon as a “phone” application is started. To turn off this behavior, I did the following (see the release notes of PulseAudio 11 for background):

tom@klotz ~> man default.pa
tom@klotz ~> mkdir -p .config/pulse
tom@klotz ~> cp /etc/pulse/default.pa .config/pulse
tom@klotz ~> vi .config/pulse/default.pa

First, check the man pages for the path of the user-specific PulseAudio config file. Then, create the corresponding path and copy the system-wide config file to the user-specific config file location. Finally, find the following section in the config file:

.ifexists module-bluetooth-policy.so
load-module module-bluetooth-policy
.endif

and add the option auto_switch=0:

.ifexists module-bluetooth-policy.so
load-module module-bluetooth-policy auto_switch=0
.endif

This way, the bluetooth mode will never change automatically.

Side note: it might also be a good idea to update Bluez to at least 5.52, as that version fixes a bug with selecting the bluetooth mode.

The Hollywood Principle: functors are frameworks

2019-10-01T00:00:00+02:00

Recently I’ve been reading and thinking about type constructors in functional programming. Often these type constructors correspond to a basic construct from category theory: functors (at least). In programming, a functor is often described as something that “can be mapped over”, meaning that a type constructor F[_] is a functor if you can define a function map(a: F[A])(f: A => B): F[B].

Another way to think about a functor F[_] is the following. A value of type F[A] is different from a value of type A because you cannot apply a function of type A => B to it. Instead, you lift the function to another function with the type F[A] => F[B]. This lifting is essentially the same thing as mapping, but focuses on the function rather than on the value the function is applied to. To emphasize this, we can swap the parameters of the function: lift(f: A => B)(a: F[A]): F[B]. When only applying the first parameter of lift, you obtain the lifted function of type F[A] => F[B].

Now, the interesting aspect to think about is who controls the application of the function. As long as you have a plain function of type A => B, you can just apply it to a value of type A and get a B. However, if you lift the function into a functor and apply it to a F[A], you transfer the responsibility for calling the original function to the functor. For example, if you lift a function into the Option[_] functor, it will be called once or not all; if you lift it into the List[_] functor, it might be called multiple times; and if you lift it into a functor like Future[_] or Task[_], it might be called at a much later time.

This inversion of control is pretty much what the so-called “Hollywood Principle” is about: “don’t call us, we’ll call you”. In object-oriented programming, the Hollywood Principle is associated with using a framework (where you give up control) rather than a library (where your retain control). Thus, in some sense, functional type constructors are comparable to OO frameworks. I find this an interesting way to approach functors and their relatives.

Edit: Discussion on lobste.rs

Running Fail2Ban and PF on FreeBsd 12.0

2019-07-14T00:00:00+02:00

I just spent more time than I’m willing to admit by looking into the combination of pf, fail2ban, and jails on FreeBSD 12.0. In particular, I am running a mail server in a jail and use pf to redirect all SMTP traffic to it. The setup for the redirection follows a post on the FreeBSD forum that I stumbled upon when I initially set up the server. Shortly after that I installed fail2ban and, as described in the setup instructions, inserted the line anchor f2b/* into /etc/pf.conf. I configured fail2ban to block access to the SSH port for anyone who tried to log in three times without success. That was easy to test and worked perfectly. I also wanted fail2ban to block SMTP access to the mail server for anyone who tried funny things with it (which happens shockingly fast after getting a public DNS entry for the domain). However, I never really tested the SMTP port blocking. Only now, some 6 months later, I noticed that fail2ban never blocked the SMTP port (i.e., the redirection to the mail jail). I could login to the server and (unsuccessfully, of course) try to send spam to other domains as often as I wanted. After trying out all kinds of configurations, I found out that the reason was in the /etc/pf.conf. I had some config like the following:

# router-facing interface 
ext_if="re0" 

# LAN IP address
ext_ip=192.168.xxx.yyy

# jail subnet
jails="{10.1.1.0/24}"

# NAT for the jails
nat on $ext_if proto {tcp udp icmp} from $jails to any -> $ext_ip

# Redirection to the jails based on port
# The keyword pass in the lines below caused my problems.
# - http
rdr pass on $ext_if proto tcp from any to $ext_ip port http -> 10.1.1.xxx
# - smtp
rdr pass on $ext_if proto tcp from any to $ext_ip port smtp -> 10.1.1.yyy
# etc.

# block all traffic by default
block all

# anchor for fail2ban rules
anchor "f2b/*"

# pass traffic to jails
pass in on $ext_if proto tcp from any to any port {http, smtp} keep state

# pass outbound traffic
pass out quick on $ext_if keep state

The problem was that the rdr directives contain the keyword pass. I copied that from the FreeBSD forum post without really thinking very much about it (pure cargo cult). But what this did was make all traffic to the jails pass before the fail2ban rules in the f2b anchor hierarchy were even checked. The SSH traffic was not redirected to any jail, so it was correctly blocked by the anchor. The SMTP traffic, however, was not checked against the rules created by fail2ban. Just removing pass from the rdr directives fixed this. I even had the directive to pass the traffic to the jails in place already.

Configuring OpenLDAP server to allow Nextcloud users to change their password

2019-02-17T00:00:00+01:00

The Nextcloud documentation contains a section about user authentication with LDAP. For simple setups, users can authenticate via anonymous binds. In that case, you don’t need a dedicated LDAP account with permissions to access the user accounts.

If you want to allow users to change their own password, however, Nextcloud needs to access the LDAP server with an account that has write permissions for the users passwords. I could not find any information on how to achieve this with an OpenLDAP server, so here is a short guide.

Create the account

First, you need to create an LDAP object for managing users. Create an object with a password, for example a person. Use an LDAP client of your choice for that, or ldapadd, or something else. In my LDAP server, the user accounts that Nextcloud uses live under ou=users,dc=example,dc=com, so I named the manager account peopleManager. It’s probably a good idea to create this manager account in a tree separate from your regular users, but I didn’t.

Configure permissions

Next, we give the manager account the permission to access all user’s passwords without disrupting the existing permissions. I created the following configuration in the slapd.conf:

access to dn.subtree="ou=people,dc=example,dc=com" attrs=userPassword
        by self write
        by dn="cn=peopleManager,ou=people,dc=example,dc=com" write
        by anonymous auth
        by * none

This lets users write their own password (e.g., when using ldappasswd), but also gives the peopleManager write access to the password field. Anonymous authentication is possible as well.

The config file is order-dependent, so insert the above before other access control configurations.

Password policies

Finally, you will want to make sure that the passwords are stored with strong encryption. To achieve that, you OpenLDAP server has to be compiled with the Password Policy overlay (PPOLICY=ON on FreeBSD). Then, add the following to your slapd.conf:

# Password hashing
password-hash {CRYPT}
password-crypt-salt-format "$6$%.16s"

and, in your backend configuration (usually at the botton of the slapd.conf file:

# force password hashing for clients (looking at you, nextcloud)
overlay ppolicy
ppolicy_hash_cleartext

That should give you some pointers and terms to google for.