Machine Learning

Machine learning (ML) is the scientific study of algorithms and statistical models that computer systems use to perform a specific task without using explicit instructions, relying on patterns and inference instead. It is seen as a subset of artificial intelligence. Wikipedia

Commercial applications

There are many commercial applications for machine learning. Usage increases over time alongside with the advancement of the techniques and the available computing power.

Predictions

Machine learning is used to predict possible outcomes based on trained models. Ex: Financial markets, consumer behavior, healthcare, capacity planning, etc.

Classification, recommendations, curation

Large datasets (images, videos, texts, behaviors, etc.) can be categorized and curated automatically. Many recommendation engines are also powered by machine learning. Ex: Retail, content platforms, social media, etc.

Generating content

ML can generate content based on trained models. The repetitive textual content that requires to be adapted with some modifications is a good example. Ex: Contracts, chatbots, customer service agents, etc.

Artistic applications

Style transfer

Style transfer is a technique where machine learning algorithm is attempting to re-construct one image using the style of another image.

...artificial system based on a Deep Neural Network that creates artistic images of high perceptual quality. The system uses neural representations to separate and recombine content and style of arbitrary images, providing a neural algorithm for the creation of artistic images. Gatys, Leon A., et al. “A Neural Algorithm of Artistic Style.” ArXiv:1508.06576 [Cs, q-Bio], Sept. 2015. arXiv.org, http://arxiv.org/abs/1508.06576.

Gatys, Leon A., et al. published their paper in 2015. Since then, there have been many derivative works, including a commercial photo editor that makes the technology available for a broader audience that doesn't necessarily code.

Colorizing old images

Below is an example of the usage of DeOldify library for the colorization of black and white films.

Exploring latent space

Mario Klingemann's Memories of Passersby creates a never-ending stream of portraits in latent space using portraits painted by Western European artists as training models.

Refik Anadol uses LA Phil's visual and audio archive to create a generative system based on machine learning. Imaginary content in latent space produced by the system is projected to the facade of the building.

First machine learning painting sold in auction

In 2018, a Paris-based collective, Obvious, created a painting using GAN. Painting was sold for $432,500 at Christie's, and it created a lot of controversy with it. Mainly related to questions about authorship because painting resembles Mario Klingemann's work and Mario shares openly his methods and open sources his tools.

Using TensorFlow

TensorFlow is an open-source platform for machine learning. Tensorflow.js is a javascript library for training and deploying models in the browser and on Node.js.

Examples below are using ml5.js. ml5.js provides access to machine learning algorithms and models building on top of TensorFlow.js.

Style Transfer using ml5.js

This is based on examples provided by ml5.js.

In this example, we will use models that are already trained for ml5.js. If you want to train your models, you can have a look at the guides.

<script src="https://unpkg.com/ml5@latest/dist/ml5.min.js"
  type="text/javascript">
</script>

The first step is to define styles, load, and switch them with mouse clicks.

let availableStyles = [
  "la_muse", "matta", "rain_princess",
  "scream", "udnie", "wave",
  "wreck", "matilde_perez", "mathura"];
let currentStyle = 0;

function getCurrentStyle() {
  return "models/"+availableStyles[currentStyle];
}

function mousePressed() {
  nextStyle();
}

function nextStyle() {
  modelReady = false;
  currentStyle += 1;
  currentStyle %= availableStyles.length;

  // Re-create a new Style Transfer method with a defined style.
  // Video is the second argument, callback function is the third.
  style = ml5.styleTransfer(getCurrentStyle(), video, modelLoaded);
}

Then, we initiate the webcam, capture it, and create a Style Transfer with it.

let video;
let result;

function setup() {
  video = createCapture(VIDEO);
  video.size(320, 240);
  video.hide();

  result = createImg('');
  result.hide();

  // Create a new Style Transfer method with a defined style.
  // Video is the second argument, callback function is the third.
  style = ml5.styleTransfer(getCurrentStyle(), video, modelLoaded);
}

Callback functions are triggered when the model is loaded (it takes a little bit of time) and when each frame is processed, respectively.

function modelLoaded() {
  console.log("Model loaded");
  style.transfer(gotResult);
  modelReady = true;
}

function gotResult(err, img) {
  result.attribute('src', img.src);
  
  if (modelReady) {
    style.transfer(gotResult);
  }
}

Finally, we draw two images to the canvas: The original webcam image and the style transfer result next to it.

function draw(){
  background(255);
  image(video, 0, 0, 320, 240);

  // Display the result only if model is loaded and ready
  if (modelReady) {
    image(result, 320, 0, 320, 240);
  } else {
    text("Loading model...", 320, 0);
  }
}

You can find the full code at GitHub and the demo in Glitch.

Read More

Training style transfer models with Paperspace

GAN (Generative Adversarial Network)

AI Art Just Got Awesome

Bias and ethics of machine learning

Disclaimer: This is by no means a thorough study of machine learning, its history, or its applications. It is merely an introduction to machine learning concepts from the course's perspective. If you believe I missed an important technique, or an artwork feel free to reach out on Twitter.