Every website that deals with video streaming in any way has a way of showing a short preview of a video without actually playing it. YouTube, for instance, plays a 3-4 seconds long excerpt from the video whenever the user hovers over the thumbnail. Other popular way of creating a preview is to take a few frames from a video and make a slideshow.
We are going to take a closer look at how to implement both of these approaches.
How to manipulate a video with Node.js
Manipulating a video with Node.js itself would be extremely hard so instead we are going to use the most popular video manipulation tool called ffmpeg. In the documentation we read:
FFmpeg is the leading multimedia framework, able to decode, encode, transcode, mux, demux, stream, filter and play pretty much anything that humans and machines have created. It supports the most obscure ancient formats up to the cutting edge. No matter if they were designed by some standards committee, the community or a corporation. It is also highly portable: FFmpeg compiles, runs, and passes our testing infrastructure FATE across Linux, Mac OS X, Microsoft Windows, the BSDs, Solaris, etc. under a wide variety of build environments, machine architectures, and configurations.
Boasting such an impressive resume ffmpeg is the perfect choice for video manipulation done from the inside of the program able to run in many different environments.
FFmpeg is accessible through CLI (command line interface), but the framework can be easily controlled through the node-fluent-ffmpeg library. The library, available on NPM, generates the ffmpeg commands for us and executes them. It also implements many useful features such as tracking the progress of a command and error handling.
Although the commands can get pretty complicated quickly, there’s a very good documentation available for the tool. Also, in our examples there’s not going to be anything too fancy going on.
The installation process is pretty straight-forward if you are on mac/linux. For windows, please refer here. The fluent-ffmpeg library depends on ffmpeg executable being either on our $PATH (so it is callable from the CLI like: ffmpeg ...) or our providing the paths to the executables through the environmental variables.
The exemplary .env file:
FFMPEG_PATH="D:/ffmpeg/bin/ffmpeg.exe"
FFPROBE_PATH="D:/ffmpeg/bin/ffprobe.exe"
Both paths have to be set if they are not already available in our $PATH.
Creating a preview
Now that we know what tools to use for video manipulation from within Node.js runtime, let’s create the previews in the formats mentioned above. I will be using the Childish Gambino - This is America video for testing purposes.
Video fragment
The video fragment preview is pretty straightforward to create - all we have to do is slice the video at the right moment. In order for the fragment to be a meaningful and representative sample of the video content, it is best if we get it from a point somewhere around the 25% - 75% of the total length of the video. For this we of course must first get the video duration.
In order to get the duration of the video we can use ffprobe which comes with ffmpeg. ffprobe is a tool that lets us, among other things, get the metadata of a video.
Let’s create a helper function that gets the duration for us:
export const getVideoInfo = (inputPath) => {
return new Promise((resolve, reject) => {
return ffmpeg.ffprobe(inputPath, (error, videoInfo) => {
if (error) {
return reject(error);
}
const { duration, size } = videoInfo.format;
return resolve({
size,
durationInSeconds: Math.floor(duration),
});
});
});
};
The ffmpeg.ffprobe method calls the provided callback with the video metadata. The videoInfo is an object containing many useful properties, but we are interested only in theformat object, in which there is the duration property. The duration is provided in seconds.
Now we can create a function for creating the preview.
Before we do that, let’s take a look at the ffmpeg command used to create the fragment:
ffmpeg -ss 146 -i video.mp4 -y -an -t 4 fragment-preview.mp4
-ss 146 - start video processing at the 146 seconds mark of the video (146 is just a placeholder here, our code will randomly generate the number of seconds)
-i video.mp4 - the input file path.
-y - overwrite any existing files while generating the output.
-an - remove audio from the generated fragment.
-t 4 - the duration of the fragment (in seconds).
fragment-preview.mp4 - the path of the output file.
Now that we know what the command is going to look like, let’s take a look at the node code that will generate it for us.
const createFragmentPreview = async (inputPath, outputPath, fragmentDurationInSeconds = 4) => {
return new Promise(async (resolve, reject) => {
const { durationInSeconds: videoDurationInSeconds } = await getVideoInfo(inputPath);
const startTimeInSeconds = getStartTimeInSeconds(videoDurationInSeconds, fragmentDurationInSeconds);
return ffmpeg()
.input(inputPath)
.inputOptions([`-ss ${startTimeInSeconds}`])
.outputOptions([`-t ${fragmentDurationInSeconds}`])
.noAudio()
.output(outputPath)
.on('end', resolve)
.on('error', reject)
.run();
});
};
First, we use the previously created getVideoInfo function to get the duration of the video. Then we get the start time using the getStartTimeInSeconds helper function.
Let’s think about the start time (the -ss parameter) because it may be tricky to get it right. The start time has to be somewhere between 25% and 75% of the video length since that is where the most representative fragment will be.
But we also have to make sure that the randomly generated start time plus the duration of the fragment is not bigger than the duration of the video (startTime + fragmentDuration <= videoDuration), because if that were the case the fragment would be cut short due to there not being enough video left.
With these requirements in mind let’s create the function:
const getStartTimeInSeconds = (videoDurationInSeconds, fragmentDurationInSeconds) => {
// by subtracting the fragment duration we can be sure that the resulting
// start time + fragment duration will be less than the video duration
const safeVideoDurationInSeconds = videoDurationInSeconds - fragmentDurationInSeconds;
// if the fragment duration is longer than the video duration
if (safeVideoDurationInSeconds <= 0) {
return 0;
}
return getRandomIntegerInRange(0.25 * safeVideoDurationInSeconds, 0.75 * safeVideoDurationInSeconds);
};
First we subtract the fragment duration from the video duration. By doing so we can be sure that the resulting start time + the fragment duration will be smaller than the video duration.
If the result of the subtraction is less than 0 then the start time has to be 0, because the fragment duration is longer than the actual video. For example, if the video were 4 seconds long and the expected fragment were to be 6 seconds long, the fragment would be entire video.
The function returns a random number of seconds from the range between 25% and 75% of the video length using the helper function: getRandomIntegerInRange.
export const getRandomIntegerInRange = (min, max) => {
const minInt = Math.ceil(min);
const maxInt = Math.floor(max);
return Math.floor(Math.random() * (maxInt - minInt + 1) + minInt);
};
It makes use of, among other things, Math.random() to get a pseudo-random integer in the range. The helper is brilliantly explained here.
Now, coming back to the command, all that’s left to do is to set the command’s parameters with the generated values and run it.
return ffmpeg()
.input(inputPath)
.inputOptions([`-ss ${startTimeInSeconds}`])
.outputOptions([`-t ${fragmentDurationInSeconds}`])
.noAudio()
.output(outputPath)
.on('end', resolve)
.on('error', reject)
.run();
The code is self-explanatory. We make use of .noAudio() method to generate the -an parameter. Also we attach the resolve and reject listeners on the end and error events respectively. As a result we have a function that is easy to deal with due to it being wrapped in a promise.
In a real world setting we would probably take in a stream and output a stream from the function, but here I decided to use promises to make the code easier to understand.
Here are a few sample results from running the function on the This is America video. The videos were converted to gifs to embed them more easily.
Since the users are probably going to view the previews in small viewports, we could do without an unnecessarily high resolution and thus save on the file size.
Frames interval
The second option is to get x frames evenly spread throughout the video. For example if we had a video that was 100 seconds long and we wanted 5 frames out of it for the preview, we would take a frame every 20 seconds. Then we could either merge them together in a video (using ffmpeg) or load them to the website and manipulate them with JavaScript.
Let’s take a look at the command:
ffmpeg -i video.mp4 -y -vf fps=1/24 thumb%04d.jpg
-i video.mp4 - the input video file.
-y - output overwrites any existing files.
-vf fps=1/24 - the filter that takes a frame every 24 (in this case) seconds.
thumb%04d.jpg - the output pattern that generates files in the following fashion: thumb0001.jpg, thumb0002.jpg, etc. The %04d part specifies that there should be four decimal numbers.
With the command also being pretty straightforward, let’s implement it in node.
export const createXFramesPreview = (inputPath, outputPattern, numberOfFrames) => {
return new Promise(async (resolve, reject) => {
const { durationInSeconds } = await getVideoInfo(inputPath);
// 1/frameIntervalInSeconds = 1 frame each x seconds
const frameIntervalInSeconds = Math.floor(durationInSeconds / numberOfFrames);
return ffmpeg()
.input(inputPath)
.outputOptions([`-vf fps=1/${frameIntervalInSeconds}`])
.output(outputPattern)
.on('end', resolve)
.on('error', reject)
.run();
});
};
As was the case with the previous function, we must know the length of the video first in order to calculate when to extract each frame. We get it with the previously defined helper getVideoInfo.
Next we divide the duration of the video by the number of frames (passed as an argument numberOfFrames). We use the Math.floor() function to make sure that the number is an integer and multiplied again by the number of frames is lower or equal to the duration of the video.
Then we generate the command with the values and execute it. Once again we attach the resolve and reject functions to the end and error events respectively to wrap the output in the promise.
Here are some of the generated images (frames):
As stated above, we could now load the images in a browser and use JavaScript to make them into a slideshow or generate a slideshow with ffmpeg. Let’s create a command for the latter approach as an exercise:
ffmpeg -framerate 1/0.6 -i thumb%04d.jpg slideshow.mp4
-framerate 1/0.6 - each frame should be seen for 0.6 seconds.
-i thumb%04d.jpg - the pattern for the images to be included in the slideshow
slideshow.mp4 - the output video file name.
Here’s the slideshow video generated from 10 extracted frames. Frame was extracted every 24 seconds.
This preview shows us a very good overview of the content of the video.
Fun fact
In order to prepare the result videos for embedding in the article I had to convert them to the .gif format. There are many online converters available as well as apps that could do this for me. But writing a post about using ffmpeg it felt weird to not even try and use it in this situation. Sure enough, converting a video to the gif format could be done with one command:
ffmpeg -i video.mp4 -filter_complex "[0:v] split [a][b];[a] palettegen [p];[b][p] paletteuse" converted-video.gif
Here’s the blog post explaining the logic behind it.
Now sure, this command is not that easy to understand because of the complex filter, but it goes a long way to show how many use cases ffmpeg has and how useful it is to be familiar with this tool.
Instead of using the online converters, where the conversion could take some time due to the tools being free and doing it on the server side, I executed the command and had the gif ready after only a few seconds.
Summary
It is not very likely that you will find yourself in need to create previews of videos, but hopefully by now you know how to use ffmpeg and its basic command syntax well enough to use it in any potential projects. Regarding the previews formats, I would probably go with the video fragment option as more people will be familiar with it because of YouTube. We should probably generate the previews of the video with low quality to keep the previews file sizes small since they have to be loaded on users browsers. The previews are usually shown in very small viewport so the low resolution should not be a problem.
Also available on LogRocket.
