This summer, I took part in a workshop called “machine learning for designers” organised at CIID and taught by Gene Kogan and **Andreas Refsgaard**.
Here is a short list of methods used during the workshop
The first method we used is called TSNE, a clustering algorithm. The idea with TSNE is to let the computer sort and organise a set of data according to the similarity of the different elements. It then renders this organisation in a 2D picture where the distance between two elements represents their perceived difference.
Open framework apps and Wekinator
The second tool that was introduced to us is Wekinator, developed by Rebbeca Fiebrink. Wekinator allows for a fast and easy classification of elements or regression.The user trains the algorithm themselves by showing it a few samples of a certain element (video, sound…) and classifying it. After training, the algorithm proceeds to classify the element itself by estimating which of its trained data is the closest.
The interface is intuitive and easy to use. Andreas and Gene developped a lot of open framework applets to make it easy to diversify the inputs and outputs. During the workshop, we tried it with a Kinect, a leap, real-time video, and sound.
Sonic Telekinesis with leap
Using already-trained algorithms
YOLO. YOLO is an already-trained algorithm used to classify objects in video and pictures.
It is powerful, and you may have seen it in a few projects recently, such as: zero likes
When it came to choosing which project to work on, we decided with Jen Sykes to focus on the representation of countries and how computers understand it.
Flags are an interesting starting point as they are supposed to synthesise the full identity of a country in a rectangular 2D representation. As images, they are perfect for TSNE, and we were curious to see what relationship the computer would draw between them. Could it see a relationship between countries that were invisible to the human eye? Can it draw causality in former colonial empires? Can flags tell us something about the similarity of two nations or two people?
Running TSNE on the sample gave us this result:
To pursue our quest to understand what meaning a computer could make of flags, we decided to use YOLO9000. An interesting feature of YOLO is that you can lower the certainty threshold. By doing that, you force the computer to reveal things that it has a low confidence in seeing. The machine starts hallucinating objects and persons on its own. YOLO9000 was not trained on a flag dataset, so we knew it was not going to be able to distinguish normal vexillology features such as colors, symbols, and structures. However, we were curious to see what it could hallucinate from these flags. What would these flags mean if they were interpreted as everyday objects? It resulted in an almost mystical description of the flags. Most of them were disappointing, but a few of them were good.