This post explains how to use one-dimensional causal and dilated convolutions in autoregressive neural networks such as WaveNet. For implementation details, I will use the notation of the tensorflow.keras.layers package, although the concepts themselves are framework-independent.

Say we have some temporal data, for example recordings of human speech. At a sample rate of 16,000 Hz, one second of recorded speech is a one-dimensional array of 16,000 values, as visualized here. Based on the recordings we have, we can compute a probabilistic model of the value at the next time step given the values at the previous time steps. Having a good model for this would be really helpful as it would allow us to generate speech ourselves.

A simple approach would be to model the next value using an affine transformation (linear combination + bias) of the four previous values. Implemented in Keras, this would be a single Dense layer with units=1:

Read more

The Gini Coefficient is a popular metric on Kaggle, especially for imbalanced class values. But googling "Gini Coefficient" gives you mostly economic explanations. Here is a descriptive explanation with regard to using it as an evaluation metric in classification. The Jupyter Notebook for this post is here.

Spoiler Alert: The Gini coefficients are the orange areas. The normalized Gini coefficient is the left one divided by the right one.

Read more
The iCloud key-value storage is like the UserDefaults but synced across devices. It also survives uninstalls of the app. I used it today to add iCloud backups to Emoji Diary. It required 4 lines of code.

Activate the iCloud capability

Select your project in Xcode and then select the target under "Project and Targets". Activate iCloud and check "Key-Value storage".

Read more
  • Jekyll is the most popular static site generator.
  • GitHub Pages hosts your jekyll sites for free under *
  • Gulp lets you automate your build process (minifying .css files, concatenating all .js files etc.).
  • Babel lets you write ES6 (cool new JavaScript) even though not all browsers support it by transpiling it into ES5 (lame old JavaScript).

I switched to using Jekyll for this blog yesterday and I love it. There are blog posts describing how to combine a subset of the above, but I could not find one for combining all four. However, each of these four components brings its own peculiarities to consider, so no blog post worked out of the box. Here is the solution that I ended up with:

Read more

I let a neural network read long texts one letter at a time. Its task was to predict the next letter based on those it had seen so far. Over time, it recognized patterns between letters. Find out what it learned by feeding it some letters below. When you click the send button on the right, it will read your text and auto-complete it.

You can choose between networks that read a lot of Wikipedia articles, US Congress transcripts etc.

Generate text from
Read more
Which map is correct?

In a geography exam, the correct answer would be the left / upper one. It displays the actual positions of four cities in the US. But that does not make the other map incorrect. It just displays other data. Specifically, it approximates the travel times between the four cities. This means that the closer two cities are on the right map the faster you can travel between them with public transport. We can calculate such maps using Multidimensional Scaling. What is Multidimensional Scaling? How can it help us to approximate travel times? And what is the relationship between the left map with the actual positions and the right map? We are about to find out.

Read more

I found out some helpful and some surprising properties of Java 6 today. Did you know that to this day, Java has Go To statements? Here are some facts about the language I was not aware of.

Read more