In this chapter, you are asked to design YouTube. The solution to this question can be applied to other interview questions like designing a video sharing platform such as…
- 14.01Design YouTubeIn this chapter, you are asked to design YouTube. The solution to this question can be applied to other interview questions like designing a video sharing platform such as…
- 14.02Step 1 - Understand the problem and establish design scopeAs revealed in Figure 14-1, besides watching a video, you can do a lot more on YouTube. For example, comment, share, or like a video, save a video to playlists, subscribe to a…
- 14.03Back of the envelope estimationThe following estimations are based on many assumptions, so it is important to communicate with the interviewer to make sure she is on the same page. Assume the product has 5…
- 14.04Step 2 - Propose high-level design and get buy-inAs discussed previously, the interviewer recommended leveraging existing cloud services instead of building everything from scratch. CDN and blob storage are the cloud services…
- 14.05Video uploading flowFigure 14-4 shows the high-level design for the video uploading. It consists of the following components: User: A user watches YouTube on devices such as a computer, mobile…
- 14.06Video streaming flowWhenever you watch a video on YouTube, it usually starts streaming immediately and you do not wait until the whole video is downloaded. Downloading means the whole video is…
- 14.07Step 3 - Design deep diveIn the high-level design, the entire system is broken down in two parts: video uploading flow and video streaming flow. In this section, we will refine both flows with…
- 14.08Video transcodingWhen you record a video, the device (usually a phone or camera) gives the video file a certain format. If you want the video to be played smoothly on other devices, the video…
- 14.09Directed acyclic graph (DAG) modelTranscoding a video is computationally expensive and time-consuming. Besides, different content creators may have different video processing requirements. For instance, some…
- 14.10Video transcoding architectureThe proposed video transcoding architecture that leverages the cloud services, is shown in Figure 14-10. The architecture has six main components: preprocessor, DAG scheduler,…
- 14.11System optimizationsAt this point, you ought to have good understanding about the video uploading flow, video streaming flow and video transcoding. Next, we will refine the system with…
- 14.12Error handlingFor a large-scale system, system errors are unavoidable. To build a highly fault-tolerant system, we must handle errors gracefully and recover from them fast. Two types of…
- 14.13Step 4 - Wrap upIn this chapter, we presented the architecture design for video streaming services like YouTube. If there is extra time at the end of the interview, here are a few additional…
References
Step 4 - Wrap up
- YouTube by the numbers: https://www.omnicoreagency.com/youtube-statistics/
- 2019 YouTube Demographics: https://blog.hubspot.com/marketing/youtube-demographics
- Cloudfront Pricing: https://aws.amazon.com/cloudfront/pricing/
- Netflix on AWS: https://aws.amazon.com/solutions/case-studies/netflix/
- Akamai homepage: https://www.akamai.com/
- Binary large object: https://en.wikipedia.org/wiki/Binary_large_object
- Here’s What You Need to Know About Streaming Protocols: https://www.dacast.com/blog/streaming-protocols/
- SVE: Distributed Video Processing at Facebook Scale: https://www.cs.princeton.edu/~wlloyd/papers/sve-sosp17.pdf
- Weibo video processing architecture (in Chinese): https://www.upyun.com/opentalk/399.html
- Delegate access with a shared access signature: https://docs.microsoft.com/en-us/rest/api/storageservices/delegate-access-with-shared-accesssignature
- YouTube scalability talk by early YouTube employee: https://www.youtube.com/watch? v=w5WVu624fY8
- Understanding the characteristics of internet short video sharing: A youtube-based measurement study. https://arxiv.org/pdf/0707.3670.pdf
- Content Popularity for Open Connect: https://netflixtechblog.com/content-popularity-for-open-connect-b86d56f613b