Deep Understanding of Seek
With ExoPlayer, seek operation can be done by player.seekTo(positionMs). In this post, I will look into how seeking can be archived inside ExoPlayer. To be specific, I will address these two.
- How accurate ExoPlayer can seek, and why.
- How fast ExoPlayer can seek and why.
*) This post is based on the version 2.11.0 of ExoPlayer.
*) I am not familiar with other players, so they could have different seeking logic inside.
Seek Accuracy
To understand seek accuracy, you have to know PTS. PTS is the abbreviation for presentation time stamp, and it means the time when a specific frame should be shown on screen. A video consists of a group of frames and each frame has its own PTS and these frames should be decoded beforehand to be present at their PTS.
There are roughly two types of seeking in terms of seek accuracy. One of them is frame accurate seeking and how it works is that it finds the closest frame among all the frames a video contains and starts playing from the frame. So PTS is the minimum time unit that a player can seek in this case. It basically means that even if you want to seek to 10s, you are not able to seek to exact 10s unless there is a frame that has a presentation time stamp representing exact 10s.
On the other hand, there is another type of seeking called key-frame accurate seeking. Instead of finding a frame among all frames like frame accurate seeking, key-frame accurate seeking only deals with key-frames so it’s faster than frame accurate seeking but less accurate because it only seeks to one of the key-frames.
ExoPlayer uses frame accurate seeking by default.
The Time Seeking takes
As I mentioned above, ExoPlayer does frame accurate seeking, but how does ExoPlayer find a frame to seek to? ExoPlayer uses the concept of SyncPoints to find a frame. SyncPoints are frames that can be divided into some reasonable groups of frames in which ExoPlayer can find a frame to seek to.
What are the frames ExoPlayer considers as sync points? They depend on what type of streaming protocol or container format is used. For DASH or HLS sync points are chunk boundaries, for progressive mp4 sync points are key-frames because a player can identify key-frames from the stbl box.
Once a player finds a seeking frame, it starts decoding from the beginning of the section in which the player finds the frame. Most notable thing is, instead of decoding from the seeking frame, a player puts frames from the previous sync point and decode and skip them until the seeking frame is come out. This is also the part for a player to take time to seek. (I do not know why ExoPlayer does this way instead of decoding from the previous key-frame of the seeking frame.)
Seek Parameters
Seeking takes some amount of time because of the time finding a frame and decoding and skipping frames from the previous sync point. However, ExoPlayer provides a way to change the behavior via ExoPlayer.setSeekParameters(). SeekParameters has four types of parameters such as EXACT, CLOSEST_SYNC, PREVIOUS_SYNC and NEXT_SYNC. As the name implies, EXAXT is the default behavior and does frame accurate seeking. Other parameters treat a sync point as a seeking frame, so they are faster because they do not waste time decoding and skipping frames, but less accurate because they only seek to any of the sync points.
HLS is slow to seek?
If you seek in HLS stream, you will find that it takes much time to seek compared with DASH or progressive mp4. The reason why HLS seeking is slow is that the sections between sync points are considerably longer due to the length of chunk which is normally 6~10s. Because of this, a player has to download 6~10s chunk after finding a seeking frame and if the seeking frame is located at the end of the chunk, a player has to decode and skip almost all the frames in the chunk. In fact, if you perform seeking to the front and the end of a chunk in a good wifi condition (like 60Mbps), you can see that it takes much time for seeking to the end of the chunk. In detail, seeking to the front of the chunk took 85ms, and seeking to the end of the chunk took 757ms.
Then how about using SeekParameters to make seeking faster? In fact, ExoPlayer does not support SeekParameters for HLS. I guess it is because the length between sync points would be really long for HLS like 6~10s and seeking would be extremely inaccurate. In short, ExoPlayer does frame accurate seeking every time even if SeekParameters is set.