Uploaded byC4Media
How Netflix Directs 1/3rd of Internet Traffic
The document discusses Netflix's content delivery strategy presented at QCon San Francisco, detailing how Netflix manages its streaming infrastructure using a combination of a data plane and control plane to optimize user experience. It covers technical aspects such as content positioning, load distribution, and dynamic business rules that ensure efficient streaming across various devices and networks. Additionally, it highlights the evolution of Netflix’s architecture and the challenges faced in providing a seamless playback experience.
More Related Content
![[Infographic] How will Internet of Things (IoT) change the world as we know it?](https://cdn.slidesharecdn.com/ss_thumbnails/iotinfographicv3-160309101328-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to How Netflix Directs 1/3rd of Internet Traffic
![[BDD 2025 - Mobile Development] Mobile Engineer and Software Engineer: Are we...](https://cdn.slidesharecdn.com/ss_thumbnails/md-mobileengineerandsoftwareengineerarewestillrelevantsidiqpermana-251127010650-55224ef1-thumbnail.jpg?width=640&height=640&fit=bounds)
![[BDD 2025 - Full-Stack Development] Digital Accessibility: Why Developers nee...](https://cdn.slidesharecdn.com/ss_thumbnails/fs-digitalaccessibilitywhydevelopersneedtoknowandcarein2025-251127011019-0674441d-thumbnail.jpg?width=640&height=640&fit=bounds)
![[BDD 2025 - Artificial Intelligence] Building AI Systems That Users (and Comp...](https://cdn.slidesharecdn.com/ss_thumbnails/ai-buildingaisystemsthatusersandcompanieslove-251124030845-038f7732-thumbnail.jpg?width=640&height=640&fit=bounds)
![[BDD 2025 - Artificial Intelligence] AI for the Underdogs: Innovation for Sma...](https://cdn.slidesharecdn.com/ss_thumbnails/ai-aifortheunderdogsinnovationforsmallbusinesses-251124030839-72a599a4-thumbnail.jpg?width=640&height=640&fit=bounds)
How Netflix Directs 1/3rd of Internet Traffic
- 1. How Netflix directs1/3rd of Haley Tucker Mohit Vora QCon San Francisco Nov 16, 2015
- 2. InfoQ.com: News &Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations /netflix-streaming-arch
- 3. Purpose of QCon -to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide Presented at QCon San Francisco www.qconsf.com
- 6.
- 7.
- 9. Project 366 #59;280212 Days Gone By..., CC BY-SA, Pete 2012, Flickr
- 10.
- 11. How do webuild a streaming “tape”?
- 12. Determine the preferredexperience DEVICETITLE CONNECTIONS COUNTRY NETWORK Broadband - wired or wifi Cellular - Edge, 3G, LTE, ... CUSTOMER
- 13. That’s exactly whatI want ...now where can I get it?
- 14. Point the deviceto appropriate locations Steering
- 15.
- 16. Uh-oh, the content is encrypted! Keymaster,CC BY-SA, Sean McGrath 2007, Flickr
- 17.
- 18.
- 19. SESSION (START, STOP, PAUSE, RESUME,KEEPALIVE) SESSION EVENTS
- 20. LICENSE PLAYBACK MANIFEST GENERATE PLAYBACK MANIFEST SESSION (START, STOP, PAUSE, RESUME,KEEPALIVE) PLAYBACK LIFECYCLE
- 22.
- 23. What is aContent Delivery Network?
- 24.
- 25.
- 26. FILLING WHEN YOUSLEEP Dreaming…,CCBY-SA,EleniBoulsaiki2009,Flickr
- 27. FILLING WHEN YOUSLEEP Open Connect A NETFLIX ORIGINAL READ XOR WRITE ONEWAY,CCBY-SA,KennyLouie2010,Flickr
- 33.
- 34.
- 35.
- 36.
- 37.
- 38.
- 39.
- 40. STREAM ISP DATA CENTER NETFLIX DEVICE IXP DATA CENTER NFLX ROUTER ISP ROUTER ISP ROUTER IXPINTERCONNECTION NETFLIX
- 41.
- 42. OPEN CONNECTSTREAM NETFLIX DEVICE CDN CONTROL PLANE DEVICE CONTROL PLANE DON’T KEEPSECRETS Network Proximity Content Positioning Load Distribution
- 43. Network Proximity Social Networkin a Course, CC BY-SA, Hans Põldoja 2010, Flickr
- 44.
- 45.
- 46. Border Gateway Protocol TAKEAWAY BGPROUTE 175.231.128.0/24 (+ proximity attributes) Use BGP
- 47. ISP2 DATA CENTER ISP2 BGP ROUTES CONTROL PLANE IXPDATA CENTER ISP1 BGP ROUTES ISP1 DATA CENTER ISP1 NFLX BGP ROUTE 175.231.128.0/24 (+ proximity attributes)
- 48.
- 49. LOCALIZE TRAFFIC ISP DATA CENTER SERVECACHE MISS
- 50. HOW DO WEDETERMINE WHAT CONTENT WILL BE POPULAR TOMORROW?
- 51.
- 52.
- 53. MINIMIZE FILL CHURN ISP DATACENTER OFF PEAK FILL
- 54. USE HISTORICAL DATA CONTENTRANKBYTES STREAMED bytesStreamed/bytesStored
- 55. IS ONE DAYOF HISTORY ENOUGH?
- 56. EXPONENTIALLY WEIGHTED MOVING AVERAGE WEIGHT DAYSAGO 0 10 20 30 40 … = 0.9 TAKEAWAY Weigh Recent Data Higher
- 57. HOW SHOULD CONTENTBE ALLOCATED?
- 58. MILLIONS OF FILES THOUSANDS OF SERVERS HOWSHOULD CONTENT BE ALLOCATED?
- 59. SVR4 SVR2 SVR1 SVR3 FILE1 FILE3 FILE1 TAKEAWAY ALLOCATE MULTIPLE REPLICAS RESILIENTTO CLUSTER CHANGES REPEATABLE Consistent Hashing
- 60. ISP2 DATA CENTER WHAT TO FILL? CONTROL PLANE IXPDATA CENTER WHERE TO FILL FROM? ISP1 DATA CENTER S3 FILL OVER HTTP
- 61.
- 62. CONTENT RANKBYTES STREAMED LOTS OF THROUGHPUT LOTSOF STORAGE CONTENT WITH CONFLICTING CONSTRAINTS
- 63. SSD BASED SPINNING DISK BASED WITHINCLUSTERS ON EACH SERVER MEMORY CONTENT RANK BYTES STREAMED SSD SPINNING DISK TAKEAWAY Tier Infrastructure
- 64. ACROSS SERVERS WITHIN CLUSTERS BALANCE BALANCE ACROSSEQUIDISTANT CLUSTERS HOW DO WE BALANCE LOAD?
- 65.
- 66. USING CONTENT DISTRIBUTION HOWDO WE BALANCE LOAD?
- 67. FLIP A COIN ANDWHEN WE HAVE EQUALLY ATTRACTIVE LOCATIONS TO SERVE FROM –
- 68. INCIDENT LOAD SYSTEM METRICS MAX INSANESANE HOW DOWE LOAD SERVERS OPTIMALLY?
- 69. … AMIDST EVERCHANGING INTERNET WEATHER
- 70. TRAFFIC t … AND DAILYTRAFFIC EBBS AND FLOWS
- 71. + SERVE STREAMS FEEDBACK - TRAFFIC EFFECTON SYSTEM METRICS CONTROL WE INTRODUCE A FEEDBACK LOOP
- 72.
- 73. TAKEAWAY PID CONTROLLER Process Variable SetPoint Control Variable Current RPM Desired RPM Input Voltage System Metrics System Metrics Max Controlled Traffic DC MOTOR
- 74. TAKEAWAY PID CONTROLLER Process Variable SetPoint Control Variable System Metrics System Metrics Max Controlled Traffic Current RPM Desired RPM Input Voltage LOADING SERVERS
- 75. ISP2 DATA CENTER CONTROL TO 80% CONTROL PLANE IXPDATA CENTER NO CONTROL ISP1 DATA CENTER 0.0 < CONTROL VAR < 1.0
- 76. TRAFFIC t NEXT HOP TRAFFIC SHIFTSTO NEXT HOP LOCATION
- 77.
- 78. STREAM NETFLIX DEVICE CDN CONTROL PLANE PLAYBACK SERVICES STEERING Got URLs for f1,f2, …, fn? Yes, here’s the URLs PROXIMITY HEALTH CONTENT CASS KAFKA OPEN CONNECT
- 79.
- 80. API STEERING SESSION MANIFEST DRM LICENSE How did weevolve from here...
- 81. API STEERING SESSION MANIFEST DRM LICENSE CLIENT SCRIPTS SERVICE LAYER RULES INSIGHTS ...tohere. 5 SOLUTIONS CACHE
- 82. DEVICE CUSTOMER TITLE NETWORK Broadband - wiredor wifi Cellular - Edge, 3G, LTE, ... CONNECTIONS COUNTRY High dimensionalityCHALLENGE
- 84. How can wequickly alter the playback experience in a targeted manner?
- 85.
- 86.
- 87.
- 90.
- 91. Pinpoint what isbrokenCHALLENGE Haystacks,CCBY-SA,JohnPavelka2008,Flickr
- 92. 3:00 AM :Pager goes off
- 93.
- 94. OK...error code 105is elevated. But why?
- 95.
- 96.
- 97. Large amount ofstateCHALLENGE
- 98. How can weenable faster UIs and low-end devices?
- 99. We introduced aserver-side caching tier MANIFESTSCUSTOMERA CUSTOMERA CUSTOMERB
- 100. Watch out forresiliency issues!! Ping Pong project, CC BY-SA, Michael Knowles 2008, Flickr
- 101.
- 102. Managing device protocolsCHALLENGE Squarepeg, round hole, CC BY-SA, Simon Law 2006, Flickr
- 103. Can we allowdevices to define their own protocols?
- 104.
- 105.
- 106. Enabling high-velocity innovationCHALLENGE CCBY-SA, Nathan E Photography 2008, Flickr
- 107. How can weexpose new data with the least amount of churn?
- 108. API MANIFEST Stream ● Bitrate ●Framerate ● Dynamic Data Stream’ ● Bitrate ● Dynamic Data This works from API: ● stream.getBitrate() ● stream.getDynamicData().get(“FRAME_RATE”) Works both ways!
- 109. This works fromCLIENT SCRIPT! ● stream.getDynamicData().get(“BIT_RATE”) ● stream.getDynamicData().get(“FRAME_RATE”) CLIENT SCRIPT Stream’’ ● Dynamic Data Works both ways! API MANIFEST Stream ● Bitrate ● Framerate ● Dynamic Data Stream’ ● Bitrate ● Dynamic Data Works both ways!
- 110.
- 111. TAKEAWAYS ● BGP basedproximity ● Tiered Infrastructure ● PID Controller ● EWMA for historical data ● Consistent Hashing ● Dynamic business rules ● Detailed domain insights ● Reduce client state ● Client-driven protocols ● Data pass-thru
- 112. TAKEAWAYS ● BGP basedproximity ● Tiered Infrastructure ● PID Controller ● EWMA for historical data ● Consistent Hashing ● Dynamic business rules ● Detailed domain insights ● Reduce client state ● Client-driven protocols ● Data pass-thru Questions? Haley Tucker @hwilson1204 Mohit Vora @mohitvora
- 113. STREAM NETFLIX DEVICE NETFLIX DEVICE STREAM SPINNING DISK SERVERS SSD SERVERS WHATTO FILL? WHERE TO FILL FROM? API CLIENT SCRIPTS SERVICE LAYER CACHE CONTROL DON’T KEEP SECRETS STEERING SESSION MANIFEST DRMLICENSE RULES CACHE INSIGHTS IXP DATA CENTER ISP1 ISP2 ISP2 BGP ROUTES ISP1 BGP ROUTES CONTROL TO 80%
- 114. ● Background imagefrom https://www.flickr.com/photos/centralasian/4099515384, Image was cropped and red lines and dots were drawn on top, https://creativecommons.org/licenses/by/2.0/. ● Image from https://www.flickr.com/photos/28705377@N04/4142872268, No modifications made, https://creativecommons.org/licenses/by/2.0/. ● Image of cassette is from https://www.flickr.com/photos/comedynose/6939206771, Image was cropped, https://creativecommons.org/licenses/by/2.0/. ● Image of speaker is from https://www.flickr.com/photos/av_hire_london/5578975575, No changes made, https://creativecommons.org/licenses/by/2.0/. ● Image of television is from https://www.flickr.com/photos/jvcamerica/3660897684/, No changes made, https://creativecommons.org/licenses/by/2.0/. ● Image of text is from https://www.flickr.com/photos/dno1967b/5754743006, No changes made, https://creativecommons.org/licenses/by/2.0/. ● Background image from https://www.flickr.com/photos/mcgraths/866572532, Image was cropped, https://creativecommons.org/licenses/by/2.0/. ● Image from https://www.flickr.com/photos/thatguyfromcchs08/2300190277, Image is dimmed, https://creativecommons.org/licenses/by/2.0/. ● Image from https://www.flickr.com/photos/mknowles/3134373590, Image was cropped, https: //creativecommons.org/licenses/by-sa/2.0/. Image Attributions
- 115. Watch the videowith slide synchronization on InfoQ.com! http://www.infoq.com/presentations/netflix- streaming-arch