SamKnows Analysis of Broadband Performance in Canada - March & April 2016

Download this report in PDF.

Dated: September 1st, 2016

Modified: December 1st 2016

Revisions to the Broadband Measurement Report – December 2016

Contents

About The Project

The Canadian Radio-Television and Telecommunications Commission (CRTC) has commissioned SamKnows to conduct a study of the performance of broadband services sold to Canadian consumers. SamKnows is a global leader in broadband measurement and has been working with governments, Internet Service Providers (ISPs), content service providers, application developers, consumer groups, and academics to accurately measure Internet performance since 2009. In doing so, SamKnows has built a global Internet measurement platform, which now spans five continents and conducts many millions of measurements each day.

Data presented in this report was collected between 15th March 2016 and 14th April 2016.

4,808 Whiteboxes were deployed to Canadian volunteers as a part of this study. Data from 3,056 of these Whiteboxes was used in this report.Footnote 1 For purposes of reporting, graphical representations of speed tier groups are split into four “buckets”: 5-9Mbps, 10-15Mbps, 16-39Mbps, and 40Mbps+. The ISPs participating in this project included all the major wireline service providers in Canada other than Sasktel. Specifically Bell Canada, Bell Aliant, Cogeco, Eastlink, MTS, Northwestel, Rogers, Shaw, TELUS and Videotron all participated on a voluntary basis and measurements covered all geographic regions of Canada in a mix of urban and rural settings. These ISPs use the following technologies to provide Internet access service: digital subscriber line (DSL)Footnote 2, hybrid-fibre co-axial cable (Cable / HFC)Footnote 3 and fibre to the home (FTTH)Footnote 4. Testing has not included any ISPs using satellite or fixed wireless technologies, nor did it include resellers of these ISPs’ networks.

The test methodology employed is the same as the one SamKnows uses around the globe with other regulators and ISPs. A full description of the test methodology can be found at https://goo.gl/piIKqA.

SamKnows recommends that a minimum sample of 40 measurement probes should be reporting data per stratum in order to provide sufficient statistical accuracy in the results. This is the approach taken in this report. Additional information on sample size methodology can be found at https://goo.gl/xvwa9T.

Any comments on the analysis in this document should be directed to Roxanne Robinson (roxanne@samknows.com).

Executive Summary

This report presents the findings of the measurement study that SamKnows is conducting in Canada on behalf of the CRTC. This report looks at data collected between the 15th March 2016 and the 14th April 2016 and investigates performance on an ISP and plan level. A preliminary report that focused on performance by region and access technology was released in March 2016 (based upon results collected in October and November 2015). The overall results in this report remain consistent with those in the Preliminary report.

4,808 SamKnows Whiteboxes have been deployed in Canadian homes across a range of ISPs and products. Each Whitebox conducts end-to-end performance measurements 24 hours a day, 7 days a week to destinations representative of Canadian consumers’ Internet usage.

In this report, results are reported by technology, ISP and speed bucket. Technologies have been identified as DSL, Cable/HFC (including all DOCSIS variants) and FTTH.

During data collection for the previous reportFootnote 5, which showed preliminary findings of broadband performance in Canada, we experienced issues with a third party that carried some of the measurement traffic, affecting the reliability of some measurement results. This was taken into account when writing the report and only unaffected data was used to create the resulting charts and tables. These issues have since been resolved and did not resurface at any point in this study's measurement period.

Unless otherwise stated, all results presented in this report are taken from the peak period, which is defined as 7-11pm local time on weekdays. Moreover, all reported figures are subject to a minimum sample size of 40 Whiteboxes. These provisions are consistent with those used in SamKnows’ other projects around the globe. The key findings contained in this report are as follows:

Figure 1 below shows a comparison of download and upload speed by technology, expressed as a percentage of advertised speed. As stated previously, all technologies exceeded their advertised rates on average.

Figure 1: Download Speed vs Upload Speed as a percentage of Advertised Speed by technology

Bar Chart of Figure 1: Download Speed vs Upload Speed as a Percentage of Advertised Speed by technology

Long Description

This bar chart illustrates a comparison of download and upload speed by technology, expressed as a percentage of advertised speed. The percentages for each speed range by technology are as follows:

Technology Download Speed Throughput Upload Speed Throughput
DSL 103% 111%
Cable/HFC 105% 104%
FTTH 121% 120%

Figure 2 below shows a comparison of download and upload speed by ISP as a percentage of advertised speed. As stated previously, the majority of ISPs met or exceeded their advertised download and upload speeds. Bell Aliant’s 7x0.64Mbps and TELUS’ 6x1Mbps plans in the 5-9Mbps bucket were shown to perform below advertised speeds for both download and upload.

TELUS’ 15x1Mbps plans significantly exceeded its advertised upload speed. Bell Canada, Bell Aliant and and TELUS also overprovisioned some of their FTTH services.

Figure 2: Download Speed vs Upload Speed as a percentage of Advertised Speed by ISP and technology

Bar Chart of Figure 2: Download Speed vs Upload Speed as a percentage of Advertised Speed by ISP and technology
Long Description

This bar chart illustrates a comparison of download and upload speed by ISP as a percentage of advertised speed as follows:

Technology-ISP Download Speed Throughput Upload Speed Throughput
DSL-Bell Canada 112% 93%
DSL-Bell Aliant 77% 81%
DSL-MTS 103% 95%
DSL-TELUS 99% 151%
Cogeco-Cable/HFC 101% 101%
Cable/HFC-Eastlink 101% 105%
Cable/HFC-Northwestel 98% 94%
Cable/HFC-Rogers 108% 108%
Cable/HFC-Shaw 109% 104%
Cable/HFC-Videotron 106% 103%
FTTH-Bell Canada 124% 113%
FTTH-Bell Aliant 125% 111%
FTTH-TELUS 105% 174%

Methodology

Whiteboxes

SamKnows deployed 4,808 Whiteboxes to Canadian consumers for the purposes of this project. The Whitebox is a consumer-grade device that is installed in a user’s home network between their home modem/router and their devices. The Whitebox’s core function is to measure the quality of the user’s Internet connection.

The measurements are conducted autonomously by the Whitebox to a variety of destinations on the Internet. No user interaction is required to conduct measurements; they are executed automatically according to a test schedule. The Whitebox does not base any of its measurement results on end user activity.

End-user cross-trafficFootnote 7 is monitored continuously by the Whitebox. If cross-traffic exceeds a certain threshold then measurements are not conducted until the cross-traffic subsides. This ensures that the Whitebox’s measurements are not distorted by end-user activity, and that the Whitebox’s measurement traffic does not interfere with a user’s experience of the Internet.

A full description of the Whitebox and its features can be found at https://goo.gl/JVMnBn.

Measurements

The Whiteboxes run a suite of active performance measurements according to a pre-defined test schedule. These include the following network measurements: download speed, upload speed, latency and packet loss, as well as application measurements such as web browsing performance.

A full description of the methodology underpinning each test can be found at https://goo.gl/fruqy6.

An overview of the test schedule used for the Measuring Broadband Canada project can be found in Appendix A.

Test destinations

The measurements carried out by the Whiteboxes are conducted against two different types of destination servers.

Firstly, there are dedicated measurement servers. These are installed at major peering and Internet exchanges at the following locations:

Each server met minimum specifications set out by SamKnows, with the exception of the one in Winnipeg. The server provisioned in Winnipeg has a 100Mbps connection rather than the standard 1Gbps as this was the highest connection available for a dedicated server in Winnipeg. This did not affect the results as only Whiteboxes based in Winnipeg were able to test to this server and these were configured in a manner that ensured the 100Mbps link would never be saturated. Full details of the hardware, software, network connectivity and monitoring requirements are available at: https:// goo.gl/RUu8QK.

The dedicated measurement servers are utilized for the download speed, upload speed, latency and packet loss measurements.

Secondly, there are real applications / content providers that measurements are carried out against. For the purposes of the Measuring Broadband Canada project, this is limited to the web browsing measurements only. The websites tested against were as follows:

Sample plan

SamKnows constructed a sample plan to govern the distribution of Whiteboxes amongst Canadian Internet consumers. This sample plan was built using subscriber data provided by the ISPs. This considered which plans and technologies met the minimum sample size requirement of 5% of the ISP’s overall wireline subscriber base. It also considered which provinces the product was available in, and what market share each province represented for the product.

Full details of the sample plan are available in Appendix A.

Recruitment and validation

The ISPs assisted in the recruitment of panelists for the project by sending emails soliciting volunteers to their customer base. These emails directed users to sign up at a SamKnows-controlled signup website. Users were then given the opportunity to sign up to the project to receive a free Whitebox, which would also provide them access to their own measurement results.

Once a user completed the signup process, their name, address and telephone number were shared with their ISP (with the user’s consent) in order for the ISP to confirm the plan that they were subscribed to. The validated ISP, product and geographic location was then checked against the sample plan to ensure that the volunteer was eligible.

Assuming so, the volunteer was sent a Whitebox and data collection began. The validation process continued to run at weekly intervals with the ISPs, thus ensuring that if the user changed ISP, product or geographic location then SamKnows would be aware and it could be accounted for in the report.

All collaborative ISPs involved in the program and the CRTC signed a ‘Code of Conduct’ which protects against ISPs intentionally changing or affecting test results as well as governing what ISPs may do with the data they have access to. While the identity of each panelist was made known to the applicable ISP as part of the speed tier validation process, the actual Unit ID for the associated Whitebox was not released to the ISP and specific test results were not directly assignable against a specific panelist.

The Code of Conduct is included in Appendix A.

A note on reported regions

This report contains multiple charts showcasing performance verticals by region. For the purposes of this report, regions are comprised of the following provinces:

Key Performance Indicators

Download Throughput

Download throughput is the measure of the capacity of the user’s broadband connection. Higher speeds are more desirable, as it can allow the user to retrieve data (be it a web page, music file, or anything else) more quickly.

To characterize the user’s maximum access link capacity, measurements were conducted between panelists’ homes and the nearest test server. Test servers were deployed in multiple metropolitan areas throughout Canada.

It is common for broadband providers around the world to differentiate their product offerings by an advertised access speed, and this is a key part of their advertising. Speeds are typically expressed in megabits per second (commonly abbreviated to ‘Mbps’ or ‘Mbit/s’). To enable comparability between different products and technologies which may feature vastly different speeds, most regulators conducting measurement studies around the world report on the percentage of advertised speed that products and technologies achieve. This report follows the same convention. For purposes of reporting, the data was split into four speed tier “buckets”: 5-9Mbps, 10-15Mbps, 16-39Mbps, and 40Mbps+.

Figure 3 below shows download speed as a percentage of advertised speed, broken down by region and access technology. Download speed in almost all regions met or exceeded the advertised rates, regardless of access technology. The one exception was DSL in the East, which achieved 75% on average.

Figure 3: Peak vs Off-Peak Download Speed as a percentage of Advertised Speed by technology and region

Bar Chart of Figure 3: Peak vs Off-Peak Download Speed as a percentage of Advertised Speed by technology and region
Long Description

This bar chart illustrates download speed as a percentage of advertised speed, broken down by region and access technology as follow:.

Technology-Region Download Speed Throughput –Off-Peak Download Speed Throughput-Peak
DSL-Central 111% 108%
DSL-East 79% 75%
DSL-West & North 105% 101%
Cable/HFC-Central 108% 105%
Cable/HFC-East 102% 101%
Cable/HFC-West & North 107% 105%
FTTH-Central 124% 123%
FTTH-East 123% 123%
FTTH-West & North 117% 116%

Figure 4 shows download speeds as a percentage of advertised speeds, broken down by ISP and access technology during the peak and off-peak time periods. The vast majority of ISPs met or exceeded advertised rates. Only Northwestel, TELUS (DSL only) and Bell Aliant (DSL only) products fell below 100% of their advertised rates during peak hours. FTTH services delivered an average of 121% of advertised speed, and all ISPs providing FTTH services delivered in excess of the advertised speed. Cable/HFC services delivered an average of 105%, and DSL services achieved an average of 103%.

DSL services showed more variance between ISPs. Bell Aliant fell short of the advertised rate, achieving 77% of advertised speed. This increased variability for DSL is not surprising given the effect copper loop lengths (i.e. how far a customer is from the nearest central office or node) have on line performance.

Figure 4: Peak vs Off-Peak Download Speed as a percentage of Advertised Speed by technology and ISP

Bar Chart of Figure 4: Peak vs Off-Peak Download Speed as a percentage of Advertised Speed by technology and ISP
Long Description

This bar chart illustrates download speeds as a percentage of advertised speeds, broken down by ISP and access technology during the peak and off-peak time periods as follows:

Technology-ISP Download Speed Throughput –Off-Peak Download Speed Throughput-Peak
DSL-Bell Canada 114% 112%
DSL-Bell Aliant 79% 77%
DSL-MTS 109% 103%
DSL-TELUS 102% 99%
Cable/HFC-Cogeco 105% 101%
Cable/HFC-Eastlink 104% 101%
Cable/HFC-Northwestel 100% 98%
Cable/HFC-Rogers 110% 108%
Cable/HFC-Shaw 111% 109%
Cable/HFC-Videotron 107% 106%
FTTH-Bell Canada 125% 124%
FTTH-Bell Aliant 125% 125%
FTTH-TELUS 107% 105%

Figure 5 depicts the average download speed by time of day as a percentage of advertised speed achieved by ISPs during peak and off-peak hours. Speed is typically lowest between 7pm and 11pm. MTS experienced the largest variance in speeds with a 6% change between peak and off peak hours. All ISPs delivered on average between 97% and 120% of advertised speeds, although some individual plans fell short of this.

Figure 5: Download Speed as a percentage of Advertised Speed by ISP and time of day

Line Chart of Figure 5: Download Speed as a percentage of Advertised Speed by ISP and time of day
Long Description

This line chart illustrates the average download speed as a percentage of advertised achieved by ISPs as follows:

Time of Day Bell Canda Shaw Bell Aliant Rogers Videotron MTS Cogeco Eastlink TELUS Northwestel
00:00 – 02:00 118% 111% 109% 110% 107% 108% 105% 103% 103% 100%
02:00 – 04:00 120% 112% 110% 110% 107% 109% 105% 105% 103% 101%
04:00 – 06:00 119% 112% 110% 110% 107% 110% 106% 105% 103% 99%
06:00 – 08:00 119% 112% 109% 110% 107% 109% 105% 105% 103% 101%
08:00 – 10:00 119% 111% 109% 110% 107% 108% 105% 104% 102% 100%
10:00 – 12:00 119% 111% 109% 109% 107% 108% 105% 104% 102% 98%
12:00 – 14:00 119% 111% 109% 109% 107% 108% 105% 103% 102% 100%
14:00 – 16:00 118% 111% 109% 109% 107% 108% 104% 103% 102% 100%
16:00 – 18:00 118% 110% 109% 109% 106% 107% 104% 102% 102% 98%
18:00 – 20:00 118% 110% 109% 109% 106% 105% 103% 102% 101% 99%
20:00 – 22:00 117% 108% 108% 107% 106% 103% 100% 101% 100% 97%
22:00 – 00:00 118% 109% 108% 108% 106% 105% 102% 102% 101% 98%

Figures 6 through 8 depict download speed as a percentage of advertised speed for ISPs’ individual products. Two products were below 90% of the advertised rate. Bell Aliant achieved an average of 77% of advertised speed on its 7x0.64Mbps DSL plan and TELUS’ 6x1Mbps DSL plan delivered 85% of the advertised speed. Six products exceeded 120% of the advertised rate: Bell Canada’s 5x1Mbps, 15x15Mbps, 25x25Mbps and 50x50Mbps products, as well as Bell Aliant’s 100x50Mbps and 150x50Mbps products.

Some of the MTS and TELUS service tiers are provisioned using either DSL or FTTH technologies, depending on where the customer resides. For purposes of reporting on service tier performance in the following Figures, these results have been grouped together.

Figure 6: Download Speed as a percentage of Advertised Speed by Product, 5-9Mbps speed bucket

Bar Chart of Figure 6: Download Speed as a percentage of Advertised Speed by Product, 5-9Mbps speed bucket
Long Description

This bar chart illustrates download speed as a percentage of advertised speed for ISPs’ individual products in the 5-9Mbps speed bucket as follows:.

ISP Product Download Speed Throughput
MTS 5X0.512 100%
Bell Canada 5X1 135%
Videotron 5X1 102%
TELUS 6X1 85%
Cogeco 6X2 103%
Bell Aliant 7X0.64 77%
Shaw 7.5X0.5 105%

Figure 7: Download Speed as a percentage of Advertised Speed by Product, 10-15Mbps speed bucket

Bar Chart of Figure 7: Download Speed as a percentage of Advertised Speed by Product, 10-15Mbps speed bucket
Long Description

This bar chart illustrates download speed as a percentage of advertised speed for ISPs’ individual products in the 10-15 Mbps speed bucket as follows:

ISP Product Download Speed Throughput
Shaw 10X0.5 102%
Videotron 10X1.5 102%
MTS 10X2 111%
TELUS 15X1 104%
Cogeco 15X2 101%
Bell Canada 15X10 104%
Bell Canada 15X15 124%

Figure 8: Download Speed as a percentage of Advertised Speed by Product, 16-39Mbps speed bucket

Bar Chart of Figure 8: Download Speed as a percentage of Advertised Speed by Product, 16-39Mbps speed bucket
Long Description

This bar chart illustrates download speed as a percentage of advertised speed for ISPs’ individual products in the 16-39 Mbps speed bucket as follows:

ISP Product Download Speed Throughput
Northwestel 16X0.768 101%
Eastlink 20X2 105%
MTS 25X2 100%
Shaw 25X2.5 110%
TELUS 25X5 104%
Bell Canada 25X10 106%
Bell Canada 25X25 125%
Rogers 30X5 107%
Shaw 30X5 110%
Videotron 30X10 109%

Figure 9: Download Speed as a percentage of Advertised Speed by Product, 40Mbps+ speed bucket

Bar Chart of Figure 9: Download Speed as a percentage of Advertised Speed by Product, 40Mbps+ speed bucket
Long Description

This bar chart illustrates download speed as a percentage of advertised speed for ISPs’ individual products in the 40Mbps+ speed bucket as follows:

ISP Product Download Speed Throughput
Cogeco 40X10 101%
Northwestel 50X2 94%
Eastlink 50X5 99%
Shaw 50X5 114%
Bell Canada 50X10 107%
TELUS 50X10 105%
Bell Canada 50X50 124%
Cogeco 60X10 100%
Rogers 60X10 107%
Videotron 60X10 108%
Rogers 70X10 100%
Eastlink 100X10 106%
Rogers 100X10 115%
Bell Aliant 100X50 125%
Eastlink 150X10 95%
Bell Aliant 150X50 126%
Rogers 250X20 102%

Table 1 shows the advertised download speed for each ISP and corresponding products included in this study, expressed both in absolute terms and as a percentage of the advertised rates. Some products are classified as both DSL and FTTH as they are provided in either type of access technology depending on the user’s area. As such, the results shown for said products e.g. TELUS’ 6x1Mbps plan or MTS’s 10x2Mbps product are an average across both technologies.

Table 1: Download Speed by ISP and product
ISP Technology Package Actual Download Speed (Mbps) Actual/Advertised Download Speed (%)
MTS DSL 5x0.512 5.02 100%
Bell Canada DSL 5x1 6.77 135%
Videotron Cable/HFC 5x1 5.09 102%
TELUS DSL, FTTH 6x1 5.10 85%
Cogeco Cable/HFC 6x2 6.16 103%
Bell Aliant DSL 7x0.64 5.36 77%
Shaw Cable/HFC 7.5x0.5 7.88 105%
Shaw Cable/HFC 10x0.5 10.22 102%
Videotron Cable/HFC 10x1.5 10.18 102%
MTS DSL, FTTH 10x2 11.14 111%
TELUS DSL, FTTH 15x1 15.66 104%
Cogeco Cable/HFC 15x2 15.17 101%
Bell Canada DSL 15x10 15.58 104%
Bell Canada FTTH 15x15 18.61 124%
Northwestel Cable/HFC 16x0.768 16.22 101%
Eastlink Cable/HFC 20x2 21.03 105%
MTS DSL, FTTH 25x2 24.90 100%
Shaw Cable/HFC 25x2.5 27.42 110%
TELUS DSL, FTTH 25x5 26.00 104%
Bell Canada DSL 25x10 26.60 106%
Bell Canada FTTH 25x25 31.18 125%
Rogers Cable/HFC 30x5 32.16 107%
Shaw Cable/HFC 30x5 33.09 110%
Videotron Cable/HFC 30x10 32.73 109%
Cogeco Cable/HFC 40x10 40.34 101%
Northwestel Cable/HFC 50x2 47.24 94%
Eastlink Cable/HFC 50x5 49.59 99%
Shaw Cable/HFC 50x5 56.86 114%
Bell Canada DSL 50x10 53.26 107%
TELUS DSL, FTTH 50x10 52.59 105%
Bell Canada FTTH 50x50 62.14 124%
Cogeco Cable/HFC 60x10 60.28 100%
Rogers Cable/HFC 60x10 64.35 107%
Videotron Cable/HFC 60x10 64.79 108%
Rogers Cable/HFC 70x10 70.04 100%
Eastlink Cable/HFC 100x10 106.19 106%
Rogers Cable/HFC 100x10 115.19 115%
Bell Aliant FTTH 100x50 124.72 125%
Eastlink Cable/HFC 150x10 143.12 95%
Bell Aliant FTTH 150x50 189.34 126%
Rogers Cable/HFC 250x20 254.18 102%

TELUS upload tiers were tested only in the provinces of Alberta and British Columbia. Upload speed tests were not performed in Quebec for TELUS’ 15x1.5Mbps, 25x3Mbps, and 50x12Mbps products, as the upload tiers did not meet the minimum number of whiteboxes threshold required for inclusion.

Upload Throughput

Upload throughput is the measure of how fast data can be transmitted from the home to the Internet. Higher speeds can allow for pictures, music and documents to be uploaded and shared more quickly.

To characterize the user’s maximum access link capacity, measurements were conducted between a nearby test server and panelists’ homes.

Historically, the amount of data that users download has vastly outweighed the amount of data that users upload. This has led technologies to be engineered to be asymmetric; i.e. they offer faster download rates than upload rates. However, as can be seen by comparing the download speeds to upload speeds, this ratio is falling for new services, such as those delivered using FTTH, which was designed with support for symmetric speeds in mind.

As with the download throughput, results in this section are presented as a percentage of the ISP’s advertised product in addition to results for the absolute level of upload speed. This enables comparability between products of vastly different speeds.

Figure 10 depicts upload speed as a percentage of advertised speed, broken down by region and access technology during the peak and off-peak periods. The majority of technologies and regions exceeded their respective advertised speeds, with the exception of DSL in the Central and East regions, where they achieved a respective 86% and 77% of advertised during peak hours. The East region was mostly influenced by Bell Aliant’s 7x0.64Mbps product, whereas Bell Canada’s 15x10 product had the greatest impact in the Central region.

In contrast, DSL technology in the West & North regions exceeded the advertised rates considerably, driven primarily by the overprovisioning of TELUS’ 15x1Mbps product. FTTH services in the West & North regions also showed an overprovisioned level of upload speed. This was also due to the influence of TELUS’ 15x1Mbps FTTH product.

Figure 10: Peak vs Off-Peak Upload Speed as a percentage of Advertised Speed by technology and region

Bar Chart of Figure 10: Peak vs Off-Peak Upload Speed as a percentage of Advertised Speed by technology and region
Long Description

This bar chart illustrates upload speed as a percentage of advertised speed, broken down by region and access technology during the peak and off-peak period as follows:

Technology-Region Upload Speed Throughput –Off-Peak Upload Speed Throughput-Peak
DSL-Central 86% 86%
DSL-East 77% 77%
DSL-West & North 132% 131%
Cable/HFC-Central 105% 105%
Cable/HFC-East 106% 106%
Cable/HFC-West & North 103% 102%
FTTH-Central 113% 113%
FTTH-East 110% 110%
FTTH-West & North 139% 138%

Figure 11 depicts upload speeds as a percentage of advertised speeds, broken down by ISP and access technology. Like with download speed, the majority of ISPs met or exceeded advertised rates. With the exception of Northwestel’s Cable/HFC services, all Cable/HFC and FTTH services exceeded the advertised upload rates. FTTH services delivered an average of 120% of advertised speed during the peak period. DSL services delivered an average of 108%, and Cable/HFC services achieved an average of 103%.

The majority of DSL services did not meet their advertised upload rates. Bell Canada DSL delivered an average of 85% of advertised upload rate - impacted particularly by the 15x10Mbps product, which delivered 60% of advertised speed. Bell Aliant also did not meet its advertised upload rates, delivering 81% of advertised speed. The average performance for the technology was lifted by TELUS, which achieved 151%.

TELUS overprovisioned its FTTH services, contributing to the technology's very high overall average, delivering 174% of advertised speed.

Figure 11: Peak vs Off-Peak Upload Speed as a percentage of Advertised Speed by technology and ISP

Bar Chart of Figure 11: Peak vs Off-Peak Upload Speed as a percentage of Advertised Speed by technology and ISP.
Long Description

This bar chart illustrates upload speeds as a percentage of advertised speeds, broken down by ISP and access technology as follows:

Technology-ISP Upload Speed Throughput –Off-Peak Upload Speed Throughput-Peak
DSL-Bell Canada 85% 85%
DSL-Bell Aliant 81% 81%
DSL-MTS 96% 95%
DSL-TELUS 152% 151%
Cable/HFC-Cogeco 102% 101%
Cable/HFC-Eastlink 105% 105%
Cable/HFC-Northwestel 94% 94%
Cable/HFC-Rogers 109% 108%
Cable/HFC-Shaw 104% 104%
Cable/HFC-Videotron 104% 103%
FTTH-Bell Canada 113% 113%
FTTH-Bell Aliant 111% 111%
FTTH-TELUS 175% 174%

Figure 12 shows the average upload speed for each ISP as a percentage of advertised speed by hour of day, across all access technologies. Upload speeds are very stable across all hours of the day. Most ISPs showed less than 1% variation at any given time of the day, with the greatest amount of variation by any ISP being only 2%.

Figure 12: Upload Speed as a percentage of Advertised Speed by ISP and time of day

Line Chart of Figure 12 Upload Speed as a percentage of Advertised Speed by ISP and time of day
Long Description

This line chart illustrates the average upload speed for each ISP as a percentage of advertised by hour of day, across all access technologies as follows:

Time of Day TELUS Bell Canada Rogers Eastlink Shaw Videotron Cogeco Bell Aliant MTS Northwestel
00:00 – 02:00 156% 97% 106% 105% 104% 104% 102% 101% 97% 94%
02:00 – 04:00 156% 97% 106% 105% 104% 104% 102% 101% 96% 94%
04:00 – 06:00 156% 97% 106% 105% 105% 104% 102% 101% 97% 95%
06:00 – 08:00 156% 97% 106% 105% 104% 104% 102% 101% 97% 94%
08:00 – 10:00 156% 97% 106% 105% 104% 104% 102% 101% 97% 94%
10:00 – 12:00 156% 97% 106% 105% 104% 104% 102% 101% 97% 94%
12:00 – 14:00 156% 97% 106% 105% 104% 103% 102% 101% 98% 94%
14:00 – 16:00 156% 97% 106% 105% 104% 103% 102% 101% 97% 94%
16:00 – 18:00 155% 97% 106% 105% 104% 103% 102% 101% 97% 95%
18:00 – 20:00 155% 97% 106% 105% 104% 103% 101% 101% 96% 94%
20:00 – 22:00 155% 97% 106% 105% 104% 103% 101% 101% 96% 94%
22:00 – 00:00 155% 97% 105% 105% 104% 103% 102% 101% 96% 94%

Figures 13 through 16 depict upload speed as a percentage of advertised speed for each product. As with download speed, upload speed across most products met or exceeded the advertised rate. Bell Aliant's 7x0.64Mbps DSL plan, TELUS’ 6x1Mbps DSL plan and Bell Canada’s 5x1Mbps, 15x10Mbps and 25x10 DSL plans did not achieve 90% of advertised upload speeds. Both the TELUS and Bell Aliant plans delivered 81% whilst Bell Canada’s plan delivered 86%, 60% and 84% of advertised speeds respectively. Bell Aliant’s plan and TELUS’ plan were the same DSL products that delivered download rates below 90% of their advertised download speed. All other plans delivered 92% of advertised speed or higher. TELUS’ 15x1Mbps FTTH plan delivered vastly above the advertised rates - delivering 311% of advertised speed.

Figure 13: Upload Speed as a percentage of Advertised Speed by Product, 5-9Mbps speed bucket

Bar Chart of Figure 13: Upload Speed as a percentage of Advertised Speed by Product, 5-9Mbps speed bucket
Long Description

This bar chart illustrates upload speed as a percentage of advertised speed for each product in the 5-9Mbps speed bucket as follows:

ISP Product Upload Speed Throughput
MTS 5X0.512 96%
Bell Canada 5X1 86%
Videotron 5X1 99%
TELUS 6X1 81%
Cogeco 6X2 102%
Bell Aliant 7X0.64 81%
Shaw 7.5X0.5 105%

Figure 14: Upload Speed as a percentage of Advertised Speed by Product, 10-15Mbps speed bucket

Bar Chart of Figure 14: Upload Speed as a percentage of Advertised Speed by Product, 10-15Mbps speed bucket
Long Description

This bar chart illustrates upload speed as a percentage of advertised speed for each product in the 10-15Mbps speed bucket as follows:

ISP Product Upload Speed Throughput
Shaw 10X0.5 102%
Videotron 10X1.5 99%
MTS 10X2 95%
TELUS 15X1 311%
Cogeco 15X2 100%
Bell Canada 15X15 118%
Bell Canada 15X10 60%

Figure 15: Upload Speed as a percentage of Advertised Speed by Product, 16-39Mbps speed bucket

Bar Chart of Figure 15: Upload Speed as a percentage of Advertised Speed by Product, 16-39Mbps speed bucket
Long Description

This bar chart illustrates upload speed as a percentage of advertised speed for each product in the 16-39Mbps speed bucket as follows:

ISP Product Upload Speed Throughput
Northwestel 16X0.768 92%
Eastlink 20X2 107%
MTS 25X2 95%
Shaw 25X2.5 105%
TELUS 25X5 120%
Bell Canada 25X10 84%
Bell Canada 25X25 109%
Rogers 30X5 106%
Shaw 30X5 105%
Videotron 30X10 105%

Figure 16: Upload Speed as a percentage of Advertised Speed by Product, 40Mbps+ speed bucket

Bar Chart of Figure 16: Upload Speed as a percentage of Advertised Speed by Product, 40Mbps+ speed bucket
Long Description

This line bar chart illustrates upload speed as a percentage of advertised speed for each product in the 40Mbps+ speed bucket as follows:

ISP Product Upload Speed Throughput
Cogeco 40X10 101%
Northwestel 50X2 94%
Eastlink 50X5 101%
Shaw 50X5 105%
Bell Canada 50X10 98%
TELUS 50X10 122%
Bell Canada 50X50 113%
Cogeco 60X10 101%
Rogers 60X10 108%
Videotron 60X10 108%
Rogers 70X10 105%
Eastlink 100X10 108%
Rogers 100X10 107%
Bell Aliant 100X50 111%
Eastlink 150X10 107%
Bell Aliant 150X50 111%
Rogers 250X20 106%

Table 2 shows the advertised upload speed for each ISP and corresponding products included in this study, expressed both in absolute terms and as a percentage of the advertised rates.

Table 2: Upload Speed by ISP and product.
ISP Technology Package Actual Upload Speed (Mbps) Actual/Advertised Upload Speed (%)
MTS DSL 5x0.512 0.49 96%
Bell Canada DSL 5x1 0.86 86%
Videotron Cable/HFC 5x1 0.99 99%
TELUS DSL, FTTH 6x1 0.81 81%
Cogeco Cable/HFC 6x2 2.03 102%
Bell Aliant DSL 7x0.64 0.52 81%
Shaw Cable/HFC 7.5x0.5 0.52 105%
Shaw Cable/HFC 10x0.5 0.51 102%
Videotron Cable/HFC 10x1.5 1.49 99%
MTS DSL, FTTH 10x2 1.90 95%
TELUS DSL, FTTH 15x1 3.11 311%
Cogeco Cable/HFC 15x2 1.99 100%
Bell Canada DSL 15x10 5.98 60%
Bell Canada FTTH 15x15 17.77 118%
Northwestel Cable/HFC 16x0.768 0.71 92%
Eastlink Cable/HFC 20x2 2.14 107%
MTS DSL, FTTH 25x2 1.89 95%
Shaw Cable/HFC 25x2.5 2.63 105%
TELUS DSL, FTTH 25x5 5.99 120%
Bell Canada DSL 25x10 8.42 84%
Bell Canada FTTH 25x25 27.35 109%
Rogers Cable/HFC 30x5 5.29 106%
Shaw Cable/HFC 30x5 5.24 105%
Videotron Cable/HFC 30x10 10.52 105%
Cogeco Cable/HFC 40x10 10.10 101%
Northwestel Cable/HFC 50x2 1.88 94%
Eastlink Cable/HFC 50x5 5.06 101%
Shaw Cable/HFC 50x5 5.23 105%
Bell Canada DSL 50x10 9.78 98%
TELUS DSL, FTTH 50x10 12.24 122%
Bell Canada FTTH 50x50 56.71 113%
Cogeco Cable/HFC 60x10 10.06 101%
Rogers Cable/HFC 60x10 10.82 108%
Videotron Cable/HFC 60x10 10.75 108%
Rogers Cable/HFC 70x10 10.52 105%
Eastlink Cable/HFC 100x10 10.79 108%
Rogers Cable/HFC 100x10 10.75 107%
Bell Aliant FTTH 100x50 55.50 111%
Eastlink Cable/HFC 150x10 10.70 107%
Bell Aliant FTTH 150x50 55.43 111%
Rogers Cable/HFC 250x20 21.14 106%

TELUS upload tiers were tested only in the provinces of Alberta and British Columbia. Upload speed tests were not performed in Quebec for TELUS’ 15x1.5Mbps, 25x3Mbps, and 50x12Mbps products, as the upload tiers did not meet the minimum number of whiteboxes threshold required for inclusion.

Latency

Latency is a measure of how long it takes a packet to travel between point A and point B. It is a significant factor in Internet performance, as latency is a fundamental property of the infrastructure upon which everything else must build. If you have a high-latency link, then it does not matter how fast your broadband connection is; you will be limited by latency.

The results presented in this section show ‘round-trip’ latency (i.e. how long it takes for a packet to travel from point A to point B and then back to point A). While round-trip latency is the most common latency measurement taken (for example, the ‘ping’ utility captures round-trip latency), the ‘round-trip’ qualifier is very often omitted. For the remainder of this document ‘latency’ should be taken to mean ‘round-trip latency’. Please note that the geographical proximity of the servers to the end user will affect latency results, as latency is proportional to distance.

Latency is almost always expressed in milliseconds. Lower results are better. Latency itself has a lower bound governed by the speed of light, and often there are technological limitations which raise that lower bound. For example, DSL services typically have higher latencies than FTTH services.

Whilst latency is unrelated to line speed, an increase in latency can have a detrimental effect on how long it takes to transfer files and other objects. Moreover, an increase in latency during peak hours is an early indicator of congestion somewhere on the network path, as routers are taking longer to receive data packets and pass them on. It is worth noting that even the highest latencies exhibited here would more than be adequate for any common Internet application at present. For the majority of use cases, the approximately 10ms latency difference between the best and worst service would be indistinguishable.

Figure 17 below shows peak period latency by technology. DSL services yielded the highest latencies, at 21.8ms on average. Cable/HFC services were near behind at 20.3ms. Latency of FTTH services was lowest at 9.2ms. This matches the behaviour of web page loading time, emphasizing the existence of a positive correlation between them. All access technologies also show very little difference between the peak and off-peak latencies, with FTTH displaying the smallest difference.

Figure 17: Peak vs Off-Peak Latency by technology

Bar Chart of Figure 17: Peak vs Off-Peak Latency by technology
Long Description

This bar chart illustrates peak period latency by technology as follows:

Technology Latency –Off-Peak Latency-Peak
DSL 21.2 milliseconds 21.8 milliseconds
Cable/HFC 19.6 milliseconds 20.3 milliseconds
FTTH 9.2 milliseconds 9.2 milliseconds

Figure 18 shows latency by access technology and speed bucket. Latency for FTTH services are consistently very low, at 10.2ms or less, regardless of speed bucket and time of day. DSL and Cable/HFC services exhibited higher latencies that varied by speed bucket too. For DSL, latency improved with the faster services, which is most likely caused by the use of newer DSL variants (such as VDSL) coupled with shorter copper loop lengths. However, average latency for the 5-9Mbps DSL bucket was pulled upwards by one ISP who deliver services to remote areas. Latency for cable services increased with higher speed services. This counterintuitive behavior is a function of the sample for Cable/HFC, which included an ISP delivering high speed cable services to some remote areas of the East.

Figure 18: Latency by advertised download speed and technology

Bar Chart of Figure 18: Latency by advertised download speed and technology
Long Description

This bar chart illustrates latency by access technology and speed bucket as follows:

Technology/Speed Bucket Latency –Off-Peak Latency-Peak
DSL/5-9Mbps 25.5 milliseconds 26.9 milliseconds
DSL/10-15Mbps 21.1 milliseconds 21.3 milliseconds
DSL/16-39 Mbps 17.9 milliseconds 18.1 milliseconds
DSL/40+ Mbps 17.7 milliseconds 17.8 milliseconds
Cable/HFC/5-9 Mbps 15.7 milliseconds 16.3 milliseconds
Cable/HFC/10-15 Mbps 16.4 milliseconds 17.2 milliseconds
Cable/HFC/16-39 Mbps 19.8 milliseconds 20.5 milliseconds
Cable/HFC/40+ Mbps 21.2 milliseconds 21.2 milliseconds
FTTH/10-15 Mbps 10.2 milliseconds 10.3 milliseconds
FTTH/16-39 Mbps 8.8 milliseconds 8.8 milliseconds
FTTH/40+ Mbps 9.0 milliseconds 9.1 milliseconds

Packet Loss

The rate of packet loss describes how likely it is that a packet sent from point A will not reach point B. Packet loss is closely related to latency and is a fundamental metric in determining how applications perform on a broadband connection. A high rate of packet loss will prevent many applications from working to a satisfactory level. A small increase in packet loss during peak hours is to be expected, as networks are busier and congestion at even one point in a network path may lead to a packet being dropped. Packet loss is generally measured and expressed as a percentage of the overall data packets sent.

Figure 19 below shows that packet loss was low across all access technologies, with the highest level of packet loss seen being 0.17%. This was observed on lower speed DSL during peak hours. FTTH technology delivered significantly lower packet loss than DSL and Cable/HFC, at just 0.04%. It is worth noting that ISPs may not be responsible for packets being lost; packet loss can occur anywhere along a network path, including after the time traffic leaves the ISP’s network.

Figure 19: Peak vs Off-Peak Packet Loss by technology

Bar Chart of Figure 19: Peak vs Off-Peak Packet Loss by technology
Long Description

This bar chart illustrates peak period packet loss by technology as follows:

Technology Latency –Off-Peak Latency-Peak
DSL 0.13 % 0.17 %
Cable/HFC 0.1% 0.13 %
FTTH 0.04 % 0.04 %

Figure 20 depicts packet loss by access technology and speed bucket. FTTH services delivered the lowest levels of packet loss across all speed buckets, at 0.07% or lower on average. The fastest FTTH services in the 40Mbps+ bucket delivered 0.02% packet loss. Cable/HFC services exhibited very similar levels of packet loss regardless of speed bucket, with all speed buckets delivering between 0.13% and 0.14% of packet loss during peak hours. There is a consistent 0.02-0.04% increase in packet loss during peak hours. DSL services also delivered very low levels of packet loss, with the exception of services in the 5-9Mbps bucket where an average of 0.39% packet loss was found during peak hours. With the exception of the 5-9Mbps DSL service, the packet loss exhibited here across all technologies and buckets would be imperceptible to modern Internet applications. Users of some 5-9Mbps DSL services may notice some small intermittent delays with this level of packet loss, such as lag when loading web pages.

Figure 20: Packet Loss by advertised download speed and technology

Bar Chart of Figure 20: Packet Loss by advertised download speed and technology
Long Description

This bar chart illustrates packet loss by access technology and speed bucket as follows:

Technology/Speed Bucket Packet Loss –Off-Peak Packet Loss-Peak
DSL/5-9Mbps 2.62% 3.86%
DSL/10-15Mbps 0.70% 0.73%
DSL/16-39 Mbps 0.75% 0.73%
DSL/40+ Mbps 0.44% 0.49%
Cable/HFC/5-9 Mbps 1.1% 1.43%
Cable/HFC/10-15 Mbps 0.82% 1.26%
Cable/HFC/16-39 Mbps 1.14% 1.24%
Cable/HFC/40+ Mbps 0.97% 1.25%
FTTH/10-15 Mbps 0.69% 0.68%
FTTH/16-39 Mbps 0.51% 0.51%
FTTH/40+ Mbps 0.32% 0.23%

Web Page Loading Time

The web page loading time test captures how long it takes for all of the elements of a web page to be received by an end user. Unlike other measurements, this test is conducted against real websites on the Internet, rather than dedicated test servers.

Web page loading time is heavily influenced by many factors, including a user’s download speed, latency, the speed of the web server and congestion in other networks. The proximity of the user to the web server is one of the factors in determining web page load times. Content providers (such as Google and Facebook) will typically host their web servers in major metropolitan areas. Customers in remote locations will often have high latency to the web servers, which will impact page load times.

Figure 21 shows web page loading time by access technology during the peak and off-peak time periods. Loading times proved generally low across all access technologies, with DSL exhibiting the highest loading times. This is to be expected as products with lower advertised rates and higher average latencies typically use DSL. All access technologies are also very consistent between the peak and off-peak periods, with loading times proving only slightly higher during peak hours. FTTH in particular shows almost no change between the peak and off-peak periods.

Figure 21: Peak vs Off-Peak Web page Loading Time by technology

Bar Chart of Figure 21: Peak vs Off-Peak Web page Loading Time by technology
Long Description

This bar chart illustrates web page loading time by access technology during the peak and off-peak time periods as follows:

Technology Web page loading time –Off-Peak Web page loading time-Peak
DSL 1.3 seconds 1.3 seconds
Cable/HFC 0.9 seconds 0.9 seconds
FTTH 0.7 seconds 0.7 seconds

Figure 22 below shows average web page loading times by ISP and access technology. ISPs using DSL services generally deliver higher loading times than Cable/HFC or FTTH services. Bell Aliant and MTS show the highest web page loading times. This is due to the fact that Bell Aliant DSL users are concentrated in rural areas such as New Brunswick, Newfoundland and Prince Edward Island. Similarly, MTS’s users are also concentrated in rural areas, where the remoteness leads to higher latencies and lower speed products being available. Bell Canada’s web browsing performance also compares favourably to Bell Aliant’s due to its significantly better throughput and latency results. Differences between the peak and off-peak measurement periods are also larger for ISPs providing DSL services compared to other access technologies.

Web page loading times for ISPs providing Cable/HFC and FTTH services never exceed 1 second with the exception of Northwestel’s Cable/HFC services, which show a loading time of 1.4 seconds during the peak period. This reflects its lower download speed compared to other Cable/HFC and FTTH services.

Figure 22: Peak vs Off-Peak Web page Loading Time by technology and ISP

Bar Chart of Figure 22: Peak vs Off-Peak Web page Loading Time by technology and ISP
Long Description

This bar chart illustrates average web page loading times by ISP and access technology …as follows:

Technology-ISP Web page loading time –Off-Peak Web page loading time-Peak
DSL-Bell Canada 0.9 seconds 0.9 seconds
DSL-Bell Aliant 1.9 seconds 2.1 seconds
DSL-MTS 1.7 seconds 1.8 seconds
DSL-TELUS 1.2 seconds 1.2 seconds
Cable/HFC-Cogeco 0.9 seconds 0.9 seconds
Cable/HFC-Eastlink 0.8 seconds 0.8 seconds
Cable/HFC-Northwestel 1.3 seconds 1.4 seconds
Cable/HFC-Rogers 0.8 seconds 0.8 seconds
Cable/HFC-Shaw 0.9 seconds 1.0 seconds
Cable/HFC-Videotron 0.9 seconds 1.0 seconds
FTTH-Bell Canada 0.6 seconds 0.6 seconds
FTTH-Bell Aliant 0.7 seconds 0.7 seconds
FTTH-TELUS 0.8 seconds 0.8 seconds

Studies in other markets have shown that download speed determines web browsing performance up to approximately 10Mbps, after which latency becomes the dominant factor. This is indicated in figure 23 below, which depicts average web page loading time by the advertised download speed for each type of access technology.

As expected, web page loading time improved as download speed increased, although this improvement tails off rapidly as advertised rates go above 10Mbps. 5-9Mbps DSL services web pages loaded in the longest time of 2.2 seconds. This improved to 1.1 seconds for 10-15Mbps DSL services. The rate of improvement declined after this point, with 40Mbps+ DSL services loading pages in 0.7 seconds. Similar behavior is observed on Cable/HFC services, with a significant improvement between 5-9Mbps services and 10-15M services (1.4 seconds versus 1.0 seconds). FTTH services yielded the lowest page load times, even with their slowest tiers. This is driven by the lower latencies that FTTH services exhibited, as seen earlier in the latency analysis.

Figure 23: Web page loading time by advertised download speed and technology

Bar Chart of Figure 23: Web page loading time by advertised download speed and technology
Long Description

This bar chart illustrates average web page loading time by the advertised download speed for each type of access technology as follows:

Technology/Speed Bucket Web page loading time –Off-Peak Web page loading time -Peak
DSL/5-9Mbps 2.01 seconds 2.14 seconds
DSL/10-15Mbps 1.12 seconds 1.13 seconds
DSL/16-39 Mbps 0.87 seconds 0.89seconds
DSL/40+ Mbps 0.70seconds 0.71 seconds
Cable/HFC/5-9 Mbps 1.38 seconds 1.41 seconds
Cable/HFC/10-15 Mbps 0.99seconds 1.02seconds
Cable/HFC/16-39 Mbps 0.88 seconds 0.92 seconds
Cable/HFC/40+ Mbps 0.79 seconds 0.83 seconds
FTTH/10-15 Mbps 0.79 seconds 0.80 seconds
FTTH/16-39 Mbps 0.68 seconds 0.69 seconds
FTTH/40+ Mbps 0.63 seconds 0.64 seconds

DSL services displayed higher web page loading times across all regions, as shown in figure 24 below. In particular, DSL services in the East displayed the highest average web page loading time of 2.1 seconds during the peak period compared to other regions as well as compared to all other technologies in all regions. This reflects the low download speed of DSL in this region. Latency in the East region is also higher than in the Central region. Given that DSL users in the East region are concentrated in rural areas, it is to be expected that latency is higher as packets of information have to travel longer distances to an exchange. This also impacts the performance of web browsing in the region. In contrast, web page loading times of FTTH technology were the lowest across all regions, displaying a maximum loading time of 0.77 seconds during peak hours in the West & North region.

Figure 24: Peak vs Off-Peak Web page Loading Time by technology and region

Bar Chart of Figure 24: Peak vs Off-Peak Web page Loading Time by technology and region
Long Description

This bar chart illustrates average web page loading time by each type of access technology and region as follows:

Technology-Region Web page loading time –Off-Peak Web page loading time -Peak
DSL-Central 1.1 seconds 1.1 seconds
DSL-East 2.0 seconds 2.1 seconds
DSL-West & North 1.4 seconds 1.5 seconds
Cable/HFC-Central 0.8 seconds 0.9 seconds
Cable/HFC-East 1.1 seconds 1.1 seconds
Cable/HFC-West & North 1.0 seconds 1.0 seconds
FTTH-Central 0.6 seconds 0.6 seconds
FTTH-East 0.7 seconds 0.8 seconds
FTTH-West & North 0.8 seconds 0.8 seconds

Conclusion

This report represents the first step in providing public access to quality data about the performance of Canadian Internet services.

The majority of broadband services met or exceeded their advertised speeds, with some notable exceptions. FTTH services achieved 121% of advertised download speeds on average, with Cable/HFC and DSL services delivering 105% and 103% respectively. However, two DSL products, one each from Bell Aliant and TELUS, delivered below their advertised download speeds, with download speeds during peak hours falling as low as 77% for Bell Aliant.

Upload speeds generally also met or exceeded advertised rates although – as with download speeds – several DSL products were found to provide speeds below those advertised. Both Bell Aliant and TELUS had DSL products achieving 81% of their respective advertised rates, whilst three of Bell Canada’s products delivered between 60% and 86%. Some ISPs overprovisioned the upload speeds of their products. In the most extreme example, one DSL product from TELUS achieved 311% of the advertised upload rate.

Differences between peak and off-peak performance were minimal. On average, download speeds varied by at most 6%, and upload speeds by at most 2%, regardless of technology.

In the latency, packet loss and web browsing metrics, FTTH services delivered the strongest and most consistent results. FTTH services showed almost no difference in results between the peak and off-peak measurement periods. More variability was found in DSL and Cable/HFC services. However, even the highest latencies exhibited during testing would be more than adequate for any common Internet application. The latency and web browsing results measured in Canada compare favorably to those measured in other jurisdictions, including the United States.

Whilst this report has presented numerous findings about the current state of Canadian Internet performance, it is also important to recognize its limitations. This report presents a snapshot of current performance; ongoing monitoring should be conducted to ensure that the market continues to operate as effectively as it does currently, and that new products being deployed are meeting their advertised service levels.

Moreover, this report focuses heavily on network layer measurements, such as throughput, latency and packet loss. Increasingly, the Internet is being used for delivering services such as on-demand video, gaming and real-time communications and consumers have an interest in learning how these services perform on their Internet connections.

In support of the CRTC 2016 three-year plan, the Measuring Broadband Canada platform will continue to collect measurements on the performance of Canadian Internet connections by including more ISP participants, and may in the future expand beyond network layer measurements to new measurements that enhance the utility of the project.


Appendix A

August 2016

Important Notice

Copyright

The material in this document is protected by Copyright. No part of the materials in this document may be reproduced for any purpose whatsoever without the written permission of SamKnows.

Sample Plan

The below is the sample plan created for the Measuring Broadband Canada report. The parties acknowledge that a limited exception to the 25,000 minimum subscriber rule for testing a specific speed package has been made to address competitive inequity between ISPs serving the same area.

Sample Plan
ISP Download Upload
Bell 5 1
Bell 15 10
Bell 15 15
Bell 25 10
Bell 25 25
Bell 50 10
Bell 50 50
Bell Aliant 7 0.64
Bell Aliant 100 30
Bell Aliant 150 30
COGECO 6 2
COGECO 15 2
COGECO 40 10
COGECO 60 10
EASTLINK 20 2
EASTLINK 100 10
EASTLINK 50 5
EASTLINK 150 10
MTS 5 0.512
MTS 10 2
MTS 25 2
NORTHWESTEL 16 0.768
NORTHWESTEL 50 2
ROGERS 30 5
ROGERS 60 10
ROGERS 70 10
ROGERS 100 10
ROGERS 250 20
SHAW 7.5 0.5
SHAW 10 0.5
SHAW 25 2.5
SHAW 30 5
SHAW 50 5
TELUS 6 1
TELUS 15 1
TELUS 25 5
TELUS 50 10
VIDEOTRON G.P. 5 1
VIDEOTRON G.P. 10 1.5
VIDEOTRON G.P. 30 10
VIDEOTRON G.P. 60 10

Code of Conduct

2015 CRTC MEASURING BROADBAND CANADA MEASUREMENT PLATFORM

May 21st, 2015

The Canadian Radio-television and Telecommunications Commission (the “CRTC”) has established a Canadian Broadband Measurement Project (the “Project”) in collaboration with participating Canadian ISPs (“ISPs”) and SamKnows Limited (“SamKnows”).

To ensure the effective and unbiased use of the data, and the integrity and validity of study results, this Code of Conduct has been drawn up which each of the ISPs, SamKnows and the CRTC (each a “Participant”) agrees to sign.

The undersigned, as a Participant in the Project, agree with the following principles:

  1. At all times during the Project, the Participants agree to act in good faith.
  2. The Participants agree not to tamper with the testing infrastructure or methodology, or take any other actions which influence the results of any test, through positive action or omission, for any individual panelist or participating ISP.
  3. The Participants acknowledge that it will not be a violation of the principle set out in paragraph 2 above for ISPs to:
  4. Operate and manage their business, which includes modifying or improving services delivered to any class of subscribers that may or may not include panelists among them, provided that such actions are consistent with normal business practices;
  5. Address service issues for individual panelists at the request of the panelists or based on information not derived from the trial; and
  6. Advise their customer service representatives of the identity of subscribers who are panelists in the trial so that those representatives may address service and billing questions.
  7. The Participants acknowledge that it will not be a violation of the principle set out in paragraph 2 above to monitor the tests and components of the testing architecture provided that no impact to the CRTC data occurs. In particular, the Participants acknowledge that the ISPs may advise SamKnows wherever a technical concern is observed in respect of an individual panelist, so that SamKnows may contact said panelist to investigate and remedy the problem.
  8. Without limiting the terms of the Non-Disclosure Agreement entered into on or around January 26, 2015, by the ISPs and SamKnows, the ISPs and SamKnows agree not to publish any data generated by the tests, nor make any public statements about or based on such data until such time as the CRTC releases data or makes a public statement regarding any of the results of the tests, except where expressly permitted by the CRTC, all provided that any data published is data previously released by the CRTC and any public statements made are based solely on such previously released data.
  9. For greater certainty, this Code of Conduct does not apply to any SamKnows measurement panel, or any data generated from such panel, that any ISP operates independently of the CRTC Broadband Canada Measurement Platform.
  10. The Participants shall ensure that their employees, agents and representatives, as appropriate, act in accordance with this Code of Conduct.
Date modified: