WebCam Fun

This is day 19 in my #javascript30 journey. This is the free course from Wes Bos that lets you brush up on your JavaScript skills via 30 projects.

Yesterday we worked on a tallying string times with reduce. You can keep track of all the projects we’re building here.

Today we’re having some fun playing around with the webcam.

Day 19 - Webcam Fun

Today we are taking our webcam input and porting it onto a canvas element. We will be adding a whole bunch of fun overlays and even creating our own green screen filter. COOL!

First up, we have to get our project running on a local server. This is to do with the security restrictions on getting access to the webcam.

Wes has already written the requirements for us using browser-sync so all we need to do is npm i & npm start.

Access the Webcam

First up we have to access the video camera feed.

To do this we use the MediaDevices.getUserMedia() method. This will return a promise depending on whether the user allows or denies the webcam access.

We can await a valid response from this and then pipe the video blob into a localURL and then onto the video player.

We also need to put an error catch in place in case the user denies our access:

const video = document.querySelector('.player');

function getVideo() {  
  navigator.mediaDevices.getUserMedia({ video: true, audio: false })
    .then(localMediaStream => {
      console.log(localMediaStream);
      video.src = window.URL.createObjectURL(localMediaStream);
      video.play();
    })
    .catch(err => {
      console.error(`OH NO!!!`, err);
    });
}

getVideo()  

This is awesome:

Webcam to Canvas

Now that we have the video feed we want to paint this onto the canvas.

We first have to set the width & height of the canvas equal to the width & height of the webcam feed.

Then we need to set the canvas to draw an image from our video every 16 milliseconds.

Finally, we need to add an event listener that tells us that the video is being piped correctly and calls the paintToCanvas method:

function paintToCanvas() {  
  const width = video.videoWidth
  const height = video.videoHeight
  canvas.width = width
  canvas.height = height

  return setInterval(() => {
    ctx.drawImage(video, 0, 0, width, height)
  }, 16)
}

video.addEventListener('canplay', paintToCanvas)  

Now we have a replication of the video on the canvas that we can play with!

Take Photo

Now we want to implement the ability to capture a screenshot from the video.

Our HTML button is linked up to call our function:

Take Photo  

Now we want the user to have some audio feedback whenever they take a screenshot so we will put this in first. We just need to ensure the audio file is always reset to the beginning before playing it.

We then will need to access the data from the canvas and output it as a jpeg. Then we want to create a link and set the attributes of it to download the image. We also want to create some JSX that will print out our image. Then we want to insert this into our strip HTML element.

function takePhoto() {  
  snap.currentTime = 0;
  snap.play()

  const data = canvas.toDataURL('image/jpeg')
  const link = document.createElement('a')
  link.href = data
  link.setAttribute('download', 'awesome')
  link.textContent = 'Download Image'
  link.innerHTML = `Awesome Screenshot`
  strip.insertBefore(link, strip.firsChild)
}

LOOK IT WORKS:

Add Filters

The image data on the canvas can be broken down into the rgba values of each individual pixel. This results in a MASSIVE array that has a pattern like this:

[r1,g1,b1,a1,r2,g2,b2,a2,r3,g3,b3,a3,r4,g4,b4,a4]

Where r1 = the red value for the first pixel, g1 = the green value for the first pixel etc.

With this knowledge we can take this massive array and alter the colours of the video.

First, we need to access the pixels array from the canvas. We do this by altering the paintToCanvas function:

function paintToCanvas() {  
  const width = video.videoWidth
  const height = video.videoHeight
  canvas.width = width
  canvas.height = height

  return setInterval(() => {
    ctx.drawImage(video, 0, 0, width, height)
    //take the pixels out of the image
    let pixels = ctx.getImageData(0, 0, width, height)
    //edit the pixels
    pixels = redEffect(pixels)
    //insert the pixels back into the canvas
    ctx.putImageData(pixels, 0, 0 )
  }, 16)
}

Then we need to iterate through the pattern that we identified earlier and edit the RGBA values. Note, the pixel array doesn’t have the standard array methods available on it so you need to use a for loop.

function redEffect(pixels){  
  for(let i = 0; i < pixels.data.length; i += 4) {
    pixels.data[i] = pixels.data[i] + 100 //red value
    pixels.data[i + 1] = pixels.data[i+ 1] - 50 //green value
    pixels.data[i + 2] = pixels.data[i + 2] * 0.5 //red value
  }
  return pixels
}

This edits the entire picture with a red filter:

Ghosting 3D filter

For the next filter we do a for loop over the pixels array but this time we move a specific colour range to the left or right. This gives an awesome split/3D effect.

function rgbSplit(pixels){  
  for(let i = 0; i < pixels.data.length; i += 4) {
    pixels.data[i - 150] = pixels.data[i] //red value
    pixels.data[i + 100] = pixels.data[i+ 1] //green value
    pixels.data[i - 550] = pixels.data[i + 2] //blue value
  }
  return pixels
}

We also added a globalAplpha to the canvas that shows a transparency of the previous image over the current one so it gives a ghost like effect.

function paintToCanvas() {  
  const width = video.videoWidth
  const height = video.videoHeight
  canvas.width = width
  canvas.height = height

  return setInterval(() => {
    ctx.drawImage(video, 0, 0, width, height)
    //take the pixels out of the image
    let pixels = ctx.getImageData(0, 0, width, height)
    //edit the pixels
    pixels = rgbSplit(pixels)
    //add ghosting illusion
    ctx.globalAlpha = 0.4
    //insert the pixels back into the canvas
    ctx.putImageData(pixels, 0, 0 )
  }, 16)
}

This looks pretty cool:

Green Screen Filter

The final filter that we’re going to put in is a green screen filter. This is AWESOME.

It allows you to select the specific colour that you want to omit from the video feed. So we could put a video or image behind the video and the background would be transparent so this would show through. GREAT.

To make this happen we need to create an object to hold the ‘green’ colour range that we are looking for, access to the sliders in the UI, and a massive for loop that will check if each pixel is within the ‘green’ range (and if it is set the alpha to 0).

function greenScreen(pixels) {  
  //object to hold min & max 'green'
  const levels = {};

  //grab the sliders from the HTML
  document.querySelectorAll('.rgb input').forEach((input) => {
    levels[input.name] = input.value;
  });

  //massive for loop. If it's between the min/max values then set the alpha to 0
  for (i = 0; i < pixels.data.length; i = i + 4) {
    red = pixels.data[i + 0];
    green = pixels.data[i + 1];
    blue = pixels.data[i + 2];
    alpha = pixels.data[i + 3];

    if (red >= levels.rmin
      && green >= levels.gmin
      && blue >= levels.bmin
      && red <= levels.rmax
      && green <= levels.gmax
      && blue <= levels.bmax) {
      // take it out!
      pixels.data[i + 3] = 0;
    }
  }

  return pixels;
}

Now we have lift off!

You can play around with the webcam and filters here.

You can keep track of all the projects in this JavaScript30 challenge here.